Automatic Computer Analysis of Digital Images of Triple-Antibody-Stained Prostate Biopsies ()
1. Introduction
Prostatic adenocarcinoma is one of the most common tumour types throughout the world, and consequently accurate histological diagnosis is an important issue worldwide [1]. However, the visual diagnosis of prostatic adenocarcinoma by light microscopy is associated with several challenges. Ordinary microscopy is to some extent subjective, and this is reflected in high intra-pathologist and inter-pathologist variability, resulting in both over- and under-diagnosis of prostate cancer [2] [3].
Prostatic adenocarcinoma is the most prevalent type of cancer in men in Sweden, with over 10,000 new cases diagnosed every year [4]. It makes up around 30% of all male cancer cases and occurs mainly in older men; accordingly, 70% of the tumours are diagnosed in men aged 70 or older. Over 2000 men die from prostatic cancer every year, and prostatic adenocarcinoma is the most common cause of death due to a malignant tumour among men in Sweden.
There is no organised screening for prostatic adenocarcinoma in Sweden, but middle-aged and older men are recommended to screen themselves for prostatic adenocarcinoma by having a blood test for analysis of PSA (prostatic-specific antigen). High levels of PSA indicate an increased risk of prostatic adenocarcinoma. Men with elevated levels of PSA are recommended to obtain a referral to a urological surgeon so that biopsy samples can be taken from the prostate gland. Such biopsies form the ultimate basis of a diagnosis of prostatic adenocarcinoma, although other visual methods such as ultrasonography and data tomography have a role as adjuncts [5] [6].
In Sweden, around 20,000 men are examined by means of biopsies from the prostate gland every year; since in most cases 12 biopsies are collected from every man, this means that around 250,000 biopsies from prostate glands are examined every year by light microscopy, by doctors trained in surgical pathology.
Currently, rapid progress is being made in the use of digital techniques such as scanning microscopy and automatic analysis of digital images in the field of laboratory medicine. It is likely that these techniques will play a much more dominating role as an adjunct to ordinary visual microscopy in the near future [7] [8] [9].
The time interval from presentation at a hospital out-patient department to treatment for men with prostate cancer in Sweden is around 6 months, due to lack of available resources, and among the latter, the lack of surgical pathologists is quite an important component. The situation is similar internationally [10].
The aim of the present article is to describe a method for rapid screening of prostate biopsies by automatic computer analysis of digital images obtained by scanning microscopy. The analysis is performed after triple antibody immuno-staining of the biopsies. The study focused on developing a method for identifying and separating out all normal biopsies, and indicating different pathological changes, such as low-grade prostatic intraepithelial neoplasia (LGPIN), high-grade prostatic intraepithelial neoplasia (HGPIN) and adenocarcinoma, using different colour frames on the images, thus making it possible to markedly reduce the number of biopsies that have to be sent for careful visual microscopic examination. This will allow more rapid diagnosis of pathological changes by a surgical pathologist. Notably, as a rule, the majority of the prostate biopsies show normal tissue, not malignant or pre-malignant morphological changes [5].
2. Materials and Methods
The study was carried out at the Department of Pathology and Cytology, County Hospital, Gävle, Sweden, a department equipped with facilities for digital pathology and a scanning microscope (Hammamatsu, Nano Zoomer S360) allowing a magnification of ×800. Accordingly, histological sections are examined on a data screen and not by ordinary visual light microscopy.
Prostate biopsies are carried out on men being investigated on the basis of a blood test showing elevated PSA levels (as a rule, 12 ultrasound-assisted biopsies are obtained in each patient). The needle biopsies (0.9 mm in diameter, 18 ga) are fixed in 10% neutral buffered formalin, embedded in paraffin and sectioned in about 4-μm-thin sections. The sections are routinely stained with haematoxylin-eosin and examined on a data screen. In selected cases the examination is completed with triple immuno-staining of the biopsies (see below).
During 2016, all prostate biopsies were immuno-stained in addition to ordinary staining with haematoxylin-eosin. For the immune-stain (Ventana instrument), three different antibodies were used: AMACR (alphamethylacyl-CoA racemase) antibody (clone name P504S), p63 and CK5 (cytokeratin 5) according to a certified protocol (Roche). Each glass slide was labelled with a serial number and personal identification code. The glass slides also contained antibody control sections from normal kidney (AMACR) and normal skin (p63 and CK5). Digital images of the triple-immune-stained biopsies were obtained using a scanning microscope (Hammamatsu, Nano Zoomer S 360) allowing a magnification of ×800.
The prostate gland is formed of glandular epithelium surrounded by connective tissue. Myoepithelial cells are located at the periphery of the gland. For pathological analysis, myoepithelial cells are immuno-stained with p63 antibodies in the nucleus (diaminobenzidine) and CK5 antibodies in the cytoplasm (red alkaline phosphatase). The gland’s epithelial cells do not stain with AMACR. However, the AMACR antibody does stain pre-malignant epithelial cells, such as LGPIN and HGPIN, and those that have undergone transformation to adenocarcinoma. LGPIN is, relatively, the mostly weakly stained, whereas HGPIN and cancer stain more strongly. The myoepithelial cells do not constitute a component of malignant prostate tissues and thus cannot be identified. Consequently, in prostatic adenocarcinoma the glandular cells are as a rule strongly immuno-stained with AMACR (brown staining) whereas the peripheral myoepithelial cells have disappeared (Figure 1(C)). In normal prostate gland the glandular cells are unstained with AMACR and the peripheral myoepithelial cells are immune-stained with antibodies to p63 and CK5 (brown nucleus and red-stained cytoplasm) (Figure 2(B)). Thus, in the staining pattern with triple immuno-staining (antibodies to AMACR, p63 and CK5), the two colours used, brown and red, show fundamentally different pictures in normal prostate gland and in prostatic adenocarcinoma. This discrepant staining pattern can be put to use by constructing an automatic computer program that can be used to analyse digital images. Haematoxylin was used for background staining (light blue) of the sections.
In total, biopsies from 564 men were collected during 2016 and digital images were obtained by a scanning microscope. From the digital archive of prostate images, consecutive, non-selected biopsies from 191 men (corresponding to 2094 biopsies) were collected and the digital images were used for automatic computer analysis (CelldaTM, MM18 medical AB, Uppsala, Sweden). The computer program is based on a classic analysis system for measuring colour saturation, colour type and colour distribution. Images from biopsies recorded as normal by the computer program were indicated by a green frame around the edge of the image, while LGPIN changes were indicated by a blue frame and HGPIN changes by a yellow frame. Areas in biopsies identified as prostate adenocarcinoma were indicated by one or more red frames. Tissue artifacts observed by the computer were also indicated by a red frame.
3. Image Analysis
In order to assign the patient to the “normal” or “abnormal” category, the program must determine whether any image belonging to that patient contains signs of cancer. Thus, the program runs image analysis on the input images.
The analysis is based on defining cancer colours (by means of a list of possible value combinations for hue, saturation and brightness) and then searching the images for sufficient quantities of pixels within the cancer colour range. The detection result is further refined by looking for red colour in the image (indicating healthy cells) and reducing the weighting of cancer detection near it. In addition, reduction of false positives is needed, and is achieved by:
1) Detecting and removing intestine (artifacts) by shape analysis (a high concentration of tiny white vacuoles in one location indicates intestine).
2) Removing thin outer edges of prostate biopsies from analysis, because they contain disproportionate amounts of cancer colour false positives (usually connective tissue) and are therefore ignored even if there is no cancer anywhere else.
In the Cellda program, the cancer colour definition is input from a cancerColor.png input file. Cellda does not itself change or determine the cancer colour definition. We use a separate program for creating and refining the cancer colour definition, and then saving that as the cancerColor.png file.
The results of the automatic computer analysis performed by Cellda were compared with the original visual-microscopy anatomic pathology diagnosis given at the time the biopsies were collected, which were used as a reference. The original diagnosis was mainly based on haematoxylin-eosin-stained sections, but triple-antibody-stained sections had occasionally been used as an adjunct in cases where changes of uncertain significance occurred, such as atypical changes or those suggestive of malignancy.
All prostate adenocarcinomas were originally classified according to the Gleason grading system. Gleason 3 + 3 occurred in 18 cases (24%), Gleason 3 + 4 in 24 cases (31%), Gleason 4 + 3 in 10 cases (13%), Gleason 4 + 4 + in 0 cases (0%), Gleason 3 + 5 in 8 cases (10%), Gleason 5 + 3 in 1 case (1%), Gleason 4 + 5 in 8 cases (10%), Gleason 5 + 4 in 8 cases (10%) and Gleason 5 + 5 in 1 case (1%).
4. Results
One aim of the study was to investigate to what extent automatic computer analysis of digital images of immuno-stained histological sections could be used to identify normal (benign) and non-malignant prostate tissue and distinguish it from prostate tissue with pre-malignant and malignant changes. Another aim was to examine to what extent the different pre-malignant alterations such as LGPIN and HGPIN could be identified and distinguished from each other and from invasive adenocarcinoma. Various kinds of benign changes such as inflammation, fibro-myo-glandular hyperplasia and metaplasia in the prostate gland were not the focus of the analysis.
Of the 191 men included in the study, 121/191 (63.4%) were diagnosed as having prostate adenocarcinoma and 70/191 (36.6%) as having no malignancy on the basis of visual microscopy (Table 1). A total of 2174 biopsy samples were visually examined, of which 660/2174 (30.4%) were malignant and 1514/2174 (69.6%) non-malignant (Table 2). In comparison, the Cellda computer program identified 134/191 (70.2%) men as having cancer (Figure 1(A), Figure 1(B) and Figure 1(C)) and 57/191(29.8%) as having no malignancy (Figure 2(A) and Figure 2(B)). On the biopsy level, 761/2094 biopsies (36.3%) were regarded as malignant and 1333/2094 (63.7%) as non-malignant after exclusion of red frames showing tissue artifacts; the artifacts were mainly caused by folding of the tissue section, which occurred in 262/2524 (10.4%) areas with red frames.
![]()
Table 1. Correlation* in 191 men between a Cellda computer analysis of digital images of prostate biopsies, after antibody staining (AMACRA, p63 and CK5), and ordinary visual microscopy.
*Agreement 92.1% and kappa value 0.823 (almost perfect agreement) [26]; **Includes LGPIN (low-grade prostatic intraepithelial neoplasia) and HGPIN (high-grade prostatic intraepithelial neoplasia) alterations.
![]()
Table 2. Correlation between Cellda computer analysis of digital images of prostate biopsies after antibody staining (AMACR/p63/CK5) and ordinary visual microscopy of biopsies from 191 men.
*The Cellda program and the visual microscopy did not always examine identical number of biopsies; **Benign cases by visual microscopy included cases with LGPIN (low-grade prostatic intraepithelial neoplasia) and HGPIN (high-grade prostatic intraepithelial neoplasia).
![]()
Figure 1. A glass slide analysed by Cellda surrounded by a red frame indicating the presence of abnormal biopsies (A). A number of smaller red frames (B) indicate prostate cancer (red arrow) and one yellow frame indicates HGPIN (yellow arrow). An area within one red frame, (C) is shown at higher magnification and demonstrates the presence of prostate cancer (Gleason 3 + 3).
The computer program identified 1092/2094 (52.1%) biopsies as benign,i.e. it surrounded the glass slide image of these biopsies with a green frame (Figure 2(A) and Figure 2(B)). All these biopsies were also benign according to the result of the visual microscopic analysis (reference), indicating 100% agreement with visual microscopy among these cases.
Computer analysis of biopsies classified by visual microscopy as benign also identified LGPIN in 87/2094 (4.2%) (Figure 3(A) and Figure 3(B)) and HGPIN in 154/2094 (7.4%) (Figure 4(A) and Figure 4(B)). When these biopsies were included among the benign biopsies, the proportion of benign biopsies increased to 63.7%. The discrepancy between the Cellda analysis and the visual microscopic analysis was only 5.9% and could mainly be explained by the finding that the computer analysis identified HGPIN lesions with borderline changes, throwing a suspicion on invasive cancer, leading to a computer diagnosis of adenocarcinoma. The discrepancy was partly due to the observation that the computer program grouped HPGIN lesions with areas showing partly undefined borders and a lack of red stained myoepithelial cells as malignant (Figure 5(A) and Figure 5(B)). Accordingly, the Cellda analysis identified more biopsies with malignant changes or changes suggestive of malignancy.
![]()
Figure 2. A glass slide analysed by Cellda with prostate biopsies surrounded by a green frame (A) indicating that all biopsies on the slide are normal, as is shown at a higher magnification (B). The two tissue sections at the top of A represent control sections of the antibody-staining.
![]()
Figure 3. Prostate biopsy analysed by Cellda with two blue frames (A) indicating the presence of LGPIN (arrow) in these two areas, as demonstrated at higher magnification (B).
![]()
Figure 4. Picture of a prostate biopsy (A), analysed by Cellda containing a yellow frame (arrow) indicating the presence of HGPIN as demonstrated at higher magnification (B).
![]()
Figure 5. Two prostate biopsies (A) analysed by Cellda with a red frame in the right biopsy, indicating prostate cancer. At higher magnification it is morphologically considered as a borderline case (red arrows) between HGPIN and early invasive cancer (B).
Of the 14 men with a visual microscopic diagnosis of non-malignant disease and a diagnosis of cancer on the basis of computer analysis, 11 showed HGPIN, in some cases extensive, with from 1 to 18 areas with yellow (HGPIN) frames in addition to the red frames indicating cancer. Three of these men had a previous or later diagnosis of adenocarcinoma. Biopsies from two men contained areas with blue frames indicating LGPIN, and one man with a malignant diagnosis was without pre-malignant changes in conjunction with a malignant diagnosis. This further, underlines that most of these men were on the borderline between pre-malignant and malignant disease.
The results on the individual level shown in Table 1 indicate a 92.1% rate of agreement between the two methods and a kappa value of 0.823. The sensitivity was 99.2 and the specificity was 0.80.
The discrepancy was mainly due to the higher number of cancers recognised by the Cellda analysis (Figure 5(A) and Figure 5(B)), as also indicated in Table 1. One case was recorded after visual microscopy as cancer Gleason 3 + 3 occurring in one biopsy in a small focus of 0.7 mm in diameter. By Cellda analysis, this was considered as a case with LGPIN, but the small cancer focus was not identified.
5. Discussions
Prostate biopsies still constitute the ultimate basis for the diagnosis of prostatic adenocarcinoma, although other visual methods such as ultrasonography and data tomography are used as adjuncts. Non-microscopy methods have too low a specificity for a secure diagnosis of prostate cancer [5].
During the past decade, scanning microscopy and visual analysis of digital images on a data screen have become more commonly used as a diagnostic method in pathology departments, and the method is beginning to replace ordinary visual microscopy. This trend facilitates the application of computer techniques for analysis of digitised microscopic tissue sections. In the long run, the computer method will probably gradually relieve the pressure on pathologists and reduce their workload [9].
In line with this trend, a number of recent scientific publications have investigated computer methods mainly based on deep learning and artificial intelligence (AI), including studies of prostate biopsies [11] [12] [13]. The investigations are usually performed on haematoxylin-eosin-stained prostate sections. The focus is often on the goal of grading prostate cancer according to Gleason, in order to obtain results that are less time consuming and more reproducible than those obtained with visual microscopy [14] - [19]. It is well known that agreement between pathologists in the assessment of biopsies and Gleason grading is less than optimal. It has also been suggested that an AI system could improve sensitivity by detecting adenocarcinoma foci that would otherwise be accidentally overlooked [3] [4].
This study used a computer method based on classical image analysis, and the tissue sections were not haematoxylin-eosin-stained but stained with a triple antibody stain (AMACRA, p62 and CK5). This is because antibody staining gives sharper and stronger colour identification of the different tissue components. It is well known that AMACRA antibody staining is negative in normal prostate glands and positive in the presence of HGPIN and adenocarcinoma. Meanwhile, the myoepithelial cells in the periphery of the glandular structures stain with p63 in the nucleus and CK5 in the cytoplasm—features that are of importance in the evaluation of prostate tissue structures [20] [21] [22] [23].
The most prominent problem with the present automatic computer analysis as performed with Cellda was the labelling of tissue artifacts, which were mostly due to folding of the tissue sections or by overstaining caused by variations in tissue thickness. This labelling of artifacts occurred in 10.4% of the “indications” (red frames) produced by the computer program.
On the biopsy level, the computer program identified 63.7% of biopsies as non-malignant, and the corresponding figure for visual microscopy was 69.6%. The discrepancy was only 5.9%, indicating good correlation. Cancer was identified by the computer program in 36.3% of biopsies and by visual microscopy in 30.4%. The discrepancy was minor and was caused by the occurrence of artifacts in red frame areas. The computer program identified 5.9% more biopsies with cancer in comparison with visual microscopy. This discrepancy can be explained by the computer program identifying early adenocarcinoma or borderline cases with HGPIN and focal loss of myoepithelial cells as suggestive of infiltrating adenocarcinoma but with insufficient evidence of indisputable invasive adenocarcinoma.
In antibody-stained sections the HGPIN lesions were easily recognised, showing dark brown staining of glandular cells and red staining of the surrounding myoepithelial cells. This observation is of some significance, since men with HGPIN, preferably of multiple origin, are at increased risk of developing adenocarcinoma compared with men with only normal biopsies [24] [25]. In accordance, 25% of biopsies regarded as non-malignant after visual analysis showed HGPIN alterations after computer analysis and after exclusion of the 14 males with borderline alterations.
This investigation is to our knowledge the first to describe automatic computer analysis of prostate biopsies stained with a triple antibody stain. It is possible that automatic scanning of immune-stained prostate tissue, followed by digital computer analysis of the images, could be used as a screening method in the future. This method would allow normal prostate images to be sorted out from those showing premalignant and malignant alterations [18]. The normal biopsies could thus be set aside from those passed on for visual microscopic examination by a specialist in surgical pathology, considerably reducing the workload for pathologists.
It might also be possible to introduce computer analysis as a tool in the diagnosis and Gleason grading of prostate adenocarcinoma [14] - [19]. The advantage would be that the well-known problem with variation between diagnoses obtained by different pathologists would be reduced [2] [3]. The method would also be expected to be considerably faster and more cost effective than the present visual procedure, especially given the lack of specialists in surgical pathology.
The computer program is undergoing a process of refinement. One of the main objects of concern is the identification of artifacts by the computer program and refining the handling of the tissue biopsies, by more careful sectioning, to avoid the occurrence of tissue artifacts. In addition, by adding new information to the computer program it may even prove possible to Gleason-grade cancerous biopsies.
Acknowledgements
The authors (EW and SN) are shareholders of MM18 Medical AB. Financial support has been received from EIT Health, EU, Uppsala University Innovation, ALMI and the National Board of Health and Welfare in Sweden. The support with the development of the computer program by Tadas Marcinkevicius is appreciated. The computer program, Cellda is patent pending. The study was approved by the local ethics committee, Dnr. 2020-03586.
All authors made equal contributions to the study.