Neural network-based segmentation and classification system for automated grading of histologic sections of bladder carcinoma

OBJECTIVE: To develop an image analysis system for automated nuclear segmentation and classification of histologic bladder sections employing quantitative nuclear features. STUDY DESIGN: Ninety-two cases were classified into three classes by experienced pathologists according to the WHO grading system: 18 cases as grade 1, 45 as grade 2, and 29 as grade 3. Nuclear segmentation was performed by means of an artificial neural network (ANN)-based pixel classification algorithm, and each case was represented by 36 nuclei features. Automated grading of bladder tumor histologic sections was performed by an ANN classifier implemented in a two-stage hierarchic tree. RESULTS: On average, 95% of the nuclei were correctly detected. At the first stage of the hierarchic tree, classifier performance in discriminating between cases of grade 1 and 2 and cases of grade 3 was 89%. At the second stage, 79% of grade 1 cases were correctly distinguished from grade 2 cases. CONCLUSION: The proposed image analysis system provides the means to reduce subjectivity in grading bladder tumors and may contribute to more accurate diagnosis and prognosis since it relies on nuclear features, the value of which has been confirmed.     

Directional features for automatic tumor classification of mammogram images

One way for breast cancer diagnosis is provided by taking radiographic (X-ray) images (termed mammograms) for suspect patients, images further used by physicians to identify potential abnormal areas thorough visual inspection. When digital mammograms are available, computer-aided based diagnostic may help the physician in having a more accurate decision. This implies automatic abnormal areas detection using segmentation, followed by tumor classification. This work aims at describing an approach to deal with the classification of digital mammograms. Patches around tumors are manually extracted to segment the abnormal areas from the remaining of the image, considered as background. The mammogram images are filtered using Gabor wavelets and directional features are extracted at different orientation and frequencies. Principal Component Analysis is employed to reduce the dimension of filtered and unfiltered high-dimensional data. Proximal Support Vector Machines are used to final classify the data. Superior mammogram image classification performance is attained when Gabor features are extracted instead of using original mammogram images. The robustness of Gabor features for digital mammogram images distorted by Poisson noise with different intensity levels is also addressed.     

A multi-step directional generalized gradient vector flow snake for target tumor segmentation in US-guided high-intensity focused ultrasound ablation

Getting precise locations of target tumors can help to ensure ablation of cancerous tissues and avoid unwanted destruction of healthy tissues in high-intensity focused ultrasound (HIFU) treatment system. Because of speckle noise and spurious boundaries in ultrasound images, traditional image segmentation methods are not suitable for achieving the precise locations of target tumors in HIFU ablation. In this paper, a multi-step directional generalized gradient vector flow snake model is introduced for target tumor segmentation. In the first step, the traditional generalized gradient vector flow (GGVF) snake is used to obtain an approximate contour of the tumor. According to the approximate contour, a new distance map is generated. Subsequently, a new directional edge map is created by calculating a scalar product of the gradients of the distance map and the initial image. In this process, the gradient directional information and the magnitude information of the distance map are used to attenuate unwanted edges and highlight the real edges in the new directional edge map. Finally, a refined GGVF field is derived from a diffusion operation of the gradient vectors of the directional edge map. The GGVF field is used to refine the tumor's contour, by directing the approximate contour to edges with the desired gradient directionality. Based on the newly developed snake model, the influences of the spurious boundaries and the speckle noise are significantly reduced in the ultrasound image segmentation. Experimental results indicate that this technique is greatly useful for target tumor segmentation in HIFU treatment system     

Use of nuclear image cytometry, histopathological grading and DNA cytometry to make breast cancer prognosis more objective.

Feulgen-stained tissue sections of 187 invasive ductal carcinomas (94 with lymph node metastases; mean follow-up: 44 months) were studied using computer assisted image cytometry. Based on survival time, the prognostic significance of nuclear image analysis was compared with the results using conventional histopathological grading according to Bloom and Richardson, as well as with image cytometric DNA measurements. The histopathological grading has the disadvantage of poor interobserver reproducibility (71.1%). Despite statistically significant differences between the actuarial survival curves of grade 1 and grade 3 patients, the prognostic significance of the conventional grading method for individual patients seems to be low and the number of grade 2 cases (42.8%) is large. The quantitative morphological method for analyzing nuclear images gives more reproducible results. Compared to histopathological grading, the predictive values for good or poor prognosis are clearly higher and the number of cases with uncertain prognosis is significantly smaller (20.9%). DNA ploidy measurements also make it possible to distinguish statistically significant differences between favorable and unfavorable prognoses with respect to over-all survival time. However, the classification accuracy based on the best single parameter (DNA-histogram type according to Auer) is 70.2% compared with 78.9% for nuclear image analysis.     

Synergistic Tissue Counterstaining and Image Segmentation Techniques for Accurate, Quantitative Immunohistochemistry

Quantitative analysis of digitized IHC-stained tissue sections is increasingly used in research studies and clinical practice. Accurate quantification of IHC staining, however, is often complicated by conventional tissue counterstains caused by the color convolution of the IHC chromogen and the counterstain. To overcome this issue, we implemented a new counterstain, Acid Blue 129, which provides homogeneous tissue background staining. Furthermore, we combined this counterstaining technique with a simple, robust, fully automated image segmentation algorithm, which takes advantage of the high degree of color separation between the 3-amino-9-ethyl-carbazole (AEC) chromogen and the Acid Blue 129 counterstain. Rigorous validation of the automated technique against manual segmentation data, using Ki-67 IHC sections from rat C6 glioma and β-amyloid IHC sections from transgenic mice with amyloid precursor protein (APP) mutations, has shown the automated method to produce highly accurate results compared with ground truth estimates based on the manually segmented images. The synergistic combination of the novel tissue counterstaining and image segmentation techniques described in this study will allow for accurate, reproducible, and efficient quantitative IHC studies for a wide range of antibodies and tissues. (J Histochem Cytochem 56:873–880, 2008)     

Quantitative Segmentation of Fluorescence Microscopy Images of Heterogeneous Tissue: Application to the Detection of Residual Disease in Tumor Margins

Purpose

To develop a robust tool for quantitative in situ pathology that allows visualization of heterogeneous tissue morphology and segmentation and quantification of image features.

Materials and Methods

Tissue excised from a genetically engineered mouse model of sarcoma was imaged using a subcellular resolution microendoscope after topical application of a fluorescent anatomical contrast agent: acriflavine. An algorithm based on sparse component analysis (SCA) and the circle transform (CT) was developed for image segmentation and quantification of distinct tissue types. The accuracy of our approach was quantified through simulations of tumor and muscle images. Specifically, tumor, muscle, and tumor+muscle tissue images were simulated because these tissue types were most commonly observed in sarcoma margins. Simulations were based on tissue characteristics observed in pathology slides. The potential clinical utility of our approach was evaluated by imaging excised margins and the tumor bed in a cohort of mice after surgical resection of sarcoma.

Results

Simulation experiments revealed that SCA+CT achieved the lowest errors for larger nuclear sizes and for higher contrast ratios (nuclei intensity/background intensity). For imaging of tumor margins, SCA+CT effectively isolated nuclei from tumor, muscle, adipose, and tumor+muscle tissue types. Differences in density were correctly identified with SCA+CT in a cohort of ex vivo and in vivo images, thus illustrating the diagnostic potential of our approach.

Conclusion

The combination of a subcellular-resolution microendoscope, acriflavine staining, and SCA+CT can be used to accurately isolate nuclei and quantify their density in anatomical images of heterogeneous tissue.     

Reproducibility and Prognosis of Quantitative Features Extracted from CT Images

We study the reproducibility of quantitative imaging features that are used to describe tumor shape, size, texture from computed tomography (CT) scans of non-small cell lung cancer (NSCLC). CT images are dependent on various scanning factors. We focus on characterizing image features that are reproducible in the presence of variations due to patient factors and segmentation methods. Thirty-two NSCLC nonenhanced lung CT scans were obtained from the Reference Image Database to Evaluate Response data set. The tumors were segmented using both manual (radiologist expert) and ensemble (software-automated) methods. A set of features (219 threedimensional and 110 two-dimensional) was computed, quantitative image features were statistically filtered to identify a subset of reproducible and nonredundant features. The variability in the repeated experiment was measured by the test-retest concordance correlation coefficient (CCC_TreT). The natural range in the features, normalized to variance, was measured by the dynamic range (DR). In this study, there were 29 features across segmentation methods found with CCC_TreT and DR ≥ 0.9 and R²_Bet ≥ 0.95. These reproducible features were tested for predicting radiologist prognostic score; some texture features (run-length and Laws kernels) had an area under the curve of 0.9. The representative features were tested for their prognostic capabilities using an independent NSCLC data set (59 lung adenocarcinomas), where one of the texture features, run-length gray-level nonuniformity, was statistically significant in separating the samples into survival groups (P ≤ .046).     

Requirements for the valid quantification of immunostains on tissue microarray materials using image analysis

Antibody‐based proteomics applied to tissue microarray (TMA) technology provides a very efficient means of visualizing and locating antigen expression in large collections of normal and pathological tissue samples. To characterize antigen expression on TMAs, the use of image analysis methods avoids the effects of human subjectivity evidenced in manual microscopical analysis. Thus, these methods have the potential to significantly enhance both precision and reproducibility. Although some commercial systems include tools for the quantitative evaluation of immunohistochemistry‐stained images, there exists no clear agreement on best practices to allow for correct and reproducible quantification results. Our study focuses on practical aspects regarding (i) image acquisition (ii) segmentation of staining and counterstaining areas and (iii) extraction of quantitative features. We illustrate our findings using a commercial system to quantify different immunohistochemistry markers targeting proteins with different expression patterns (cytoplasmic, nuclear or membranous) in colon cancer or brain tumor TMAs. Our investigations led us to identify several steps that we consider essential for standardizing computer‐assisted immunostaining quantification experiments. In addition, we propose a data normalization process based on reference materials to be able to compare measurements between studies involving different TMAs. In conclusion, we recommend certain critical prerequisites that commercial or in‐house systems should satisfy in order to permit valid immunostaining quantification.     

SegNet and Salp Water Optimization-driven Deep Belief Network for Segmentation and Classification of Brain Tumor

Classification of brain tumor in Magnetic Resonance Imaging (MRI) images is highly popular in treatment planning, early diagnosis, and outcome evaluation. It is very difficult for classifying and diagnosing tumors from several images. Thus, an automatic prediction strategy is essential in classifying brain tumors as malignant, core, edema, or benign. In this research, a novel approach using Salp Water Optimization-based Deep Belief network (SWO-based DBN) is introduced to classify brain tumor. At the initial stage, the input image is pre-processed to eradicate the artifacts present in input image. Following pre-processing, the segmentation is executed by SegNet, where the SegNet is trained using the proposed SWO. Moreover, the Convolutional Neural Network (CNN) features are employed to mine the features for future processing. At last, the introduced SWO-based DBN technique efficiently categorizes the brain tumor with respect to the extracted features. Thereafter, the produced output of the introduced SegNet + SWO-based DBN is made use of in brain tumor segmentation and classification. The developed technique produced better results with highest values of accuracy at 0.933, specificity at 0.880, and sensitivity at 0.938 using BRATS, 2018 datasets and accuracy at 0.921, specificity at 0.853, and sensitivity at 0.928 for BRATS, 2020 dataset.     

Automated measurements on cell nuclei in tissue sections. Validation procedures

A methodology was developed for fully automated measurements of nuclear features in Feulgen-stained tissue sections by means of videomicroscopy and image analysis. Segmentation is performed within one minute on 512 X 512 optical density (OD) images covering about 75 nuclei, resulting in a graphic contour overlay. The corresponding image subset is scanned by an object data extraction program, producing the raw figures for statistical interpretation. The segmentation software was evaluated by three tests, involving comparison with manual delineation and assessment of the influence of OD. Two case studies (ACTH-stimulated adrenal cortex and pancreatic carcinoma) illustrate the biologic accuracy and medical significance of the described methodology.     

Computerized image analysis as a tool to quantify infiltrating leukocytes: a comparison between high- and low-magnification images.

The purpose of the present study was to establish a rapid and reproducible method for quantification of tissue-infiltrating leukocytes using computerized image analysis. To achieve this, the staining procedure, the image acquisition, and the image analysis method were optimized. Because of the adaptive features of the human eye, computerized image analysis is more sensitive to variations in staining compared with manual image analysis. To minimize variations in staining, an automated immunostainer was used. With a digital scanner camera, low-magnification images could be sampled at high resolution, thus making it possible to analyze larger tissue sections. Image analysis was performed by color thresholding of the digital images based on values of hue, saturation, and intensity color mode, which we consider superior to the red, green, and blue color mode for analysis of most histological stains. To evaluate the method, we compared computerized analysis of images with a x100 or a x12.5 magnification to assess leukocytes infiltrating rat brain tumors after peripheral immunizations with tumor cells genetically modified to express rat interferon-gamma (IFN-gamma) or medium controls. The results generated by both methods correlated well and did not show any significant differences. The method allows efficient and reproducible processing of large tissue sections that is less time-consuming than conventional methods and can be performed with standard equipment and software.(J Histochem Cytochem 49:1073-1079, 2001)     

Colorization and automated segmentation of human T2 MR brain images for characterization of soft tissues

Characterization of tissues like brain by using magnetic resonance (MR) images and colorization of the gray scale image has been reported in the literature, along with the advantages and drawbacks. Here, we present two independent methods; (i) a novel colorization method to underscore the variability in brain MR images, indicative of the underlying physical density of bio tissue, (ii) a segmentation method (both hard and soft segmentation) to characterize gray brain MR images. The segmented images are then transformed into color using the above-mentioned colorization method, yielding promising results for manual tracing. Our color transformation incorporates the voxel classification by matching the luminance of voxels of the source MR image and provided color image by measuring the distance between them. The segmentation method is based on single-phase clustering for 2D and 3D image segmentation with a new auto centroid selection method, which divides the image into three distinct regions (gray matter (GM), white matter (WM), and cerebrospinal fluid (CSF) using prior anatomical knowledge). Results have been successfully validated on human T2-weighted (T2) brain MR images. The proposed method can be potentially applied to gray-scale images from other imaging modalities, in bringing out additional diagnostic tissue information contained in the colorized image processing approach as described.     

Automated cell nuclear segmentation in color images of hematoxylin and eosin-stained breast biopsy

OBJECTIVE: To develop an automated, reproducible epithelial cell nuclear segmentation method to quantify cytologic features quickly and accurately from breast biopsy. STUDY DESIGN: The method, based on fuzzy c-mean clustering of the hue-band of color images and the watershed transform, was applied to 39 images from 3 histologic types (typical hyperplasia, atypical hyperplasia, and ductal carcinoma in situ [cribriform and solid]). RESULTS: The performance of the segmentation algorithm was evaluated by visually determining the percentage of badly segmented nuclei (approximately 25% for all types), the percentage of nuclei that remained in clumps (4.5-16.7%) and the percentage of missed nuclei (0.4-1.5%) for each image. CONCLUSION: The segmentation algorithm was sensitive in that a small percentage of nuclei were missed. However, the percentage of badly segmented nuclei was on the order of 25%, and the percentage of nuclei that remained in clumps was on the order of 10% of the total number of nuclei in the duct. Even so, > 600 nuclei per duct, on average, were segmented correctly; that was a sufficient number by which to calculate accurate quantitative, cytologic, morphometric measurements of epithelial cell nuclei in stained tissue sections of breast biopsy.     

Robust Volume Assessment of Brain Tissues for 3-Dimensional Fourier Transformation MRI via a Novel Multispectral Technique

A new TRIO algorithm method integrating three different algorithms is proposed to perform brain MRI segmentation in the native coordinate space, with no need of transformation to a standard coordinate space or the probability maps for segmentation. The method is a simple voxel-based algorithm, derived from multispectral remote sensing techniques, and only requires minimal operator input to depict GM, WM, and CSF tissue clusters to complete classification of a 3D high-resolution multislice-multispectral MRI data. Results showed very high accuracy and reproducibility in classification of GM, WM, and CSF in multislice-multispectral synthetic MRI data. The similarity indexes, expressing overlap between classification results and the ground truth, were 0.951, 0.962, and 0.956 for GM, WM, and CSF classifications in the image data with 3% noise level and 0% non-uniformity intensity. The method particularly allows for classification of CSF with 0.994, 0.961 and 0.996 of accuracy, sensitivity and specificity in images data with 3% noise level and 0% non-uniformity intensity, which had seldom performed well in previous studies. As for clinical MRI data, the quantitative data of brain tissue volumes aligned closely with the brain morphometrics in three different study groups of young adults, elderly volunteers, and dementia patients. The results also showed very low rates of the intra- and extra-operator variability in measurements of the absolute volumes and volume fractions of cerebral GM, WM, and CSF in three different study groups. The mean coefficients of variation of GM, WM, and CSF volume measurements were in the range of 0.03% to 0.30% of intra-operator measurements and 0.06% to 0.45% of inter-operator measurements. In conclusion, the TRIO algorithm exhibits a remarkable ability in robust classification of multislice-multispectral brain MR images, which would be potentially applicable for clinical brain volumetric analysis and explicitly promising in cross-sectional and longitudinal studies of different subject groups.     

Urothelial neoplasms of the upper urinary tract. A correlation between cytologic and histologic findings in 43 patients with urothelial neoplasms of the renal pelvis or ureter.

The present study elucidates the possibilities to diagnose and classify urothelial tumors of the upper urinary tract by means of exfoliative cytology on voided urine using a membrane filter method. In a series of 30 patients with renal pelvic tumors and 13 patients with ureteral tumors an overall agreement between cytology and histopathology was obtained in 25 cases (58%). None of the Grade 1 tumors or of the non-invasive Grade 2 tumors were regarded as positive by cytology whilst two out of five invasive Grade 2 tumors had positive cytologic reports. The series included 24 patients with poorly differentiated or anaplastic tumors, 17 of whom had positive cytology (71%). By excluding from the series 13 patients with obstructed urinary passages or radiologically non-functioning kidneys on the tumor side an agreement between cytology and pathology was reached in 83 per cent of the cases, regardless of tumor grade, and in 17 out of 18 patients with Grade 3-4 tumors (94%).     

The Correlation Between Texture Features and Fibrous Cap Thickness of Lipid-Rich Atheroma Based on Optical Coherence Tomography Imaging

Fibrous cap thickness (FCT) is seen as critical to plaque vulnerability. Therefore, the development of automatic algorithms for the quantification of FCT is for estimating cardiovascular risk of patients. Intravascular optical coherence tomography (IVOCT) is currently the only in vivo imaging modality with which FCT, the critical component of plaque vulnerability, can be assessed accurately. This study was aimed to discussion the correlation between the texture features of OCT images and the FCT in lipid-rich atheroma. Methods: Firstly, a full automatic segmentation algorithm based on unsupervised fuzzy c means (FCM) clustering with geometric constrains was developed to segment the ROIs of IVOCT images. Then, 32 features, which are associated with the structural and biochemical changes of tissue, were carried out to describe the properties of ROIs. The FCT in grayscale IVOCT images were manually measured by two independent observers. In order to analysis the correlation between IVOCT image features and manual FCT measurements, linear regression approach was performed. Results: Inter-observer agreement of the twice manual FCT measurements was excellent with an intraclass correlation coefficient (ICC) of 0.99. The correlation coefficient between each individual feature set and mean FCT of OCT images were 0.68 for FOS, 0.80 for GLCM, 0.74 for NGTDM, 0.72 for FD, 0.62 for IM and 0.58 for SP. The fusion image features of automatic segmented ROIs and FCT measurements improved the results significantly with a high correlation coefficient (r= 0.91, p<0.001). Conclusion The OCT images features demonstrated the perfect performances and could be used for automatic qualitative analysis and the identification of high-risk plaques instead manual FCT measurements.     

Pixel classification method in optical coherence tomography for tumor segmentation and its complementary usage with OCT microangiography

A novel machine‐learning method to distinguish between tumor and normal tissue in optical coherence tomography (OCT) has been developed. Pre‐clinical murine ear model implanted with mouse colon carcinoma CT‐26 was used. Structural‐image‐based feature sets were defined for each pixel and machine learning classifiers were trained using “ground truth” OCT images manually segmented by comparison with histology. The accuracy of the OCT tumor segmentation method was then quantified by comparing with fluorescence imaging of tumors expressing genetically encoded fluorescent protein KillerRed that clearly delineates tumor borders. Because the resultant 3D tumor/normal structural maps are inherently co‐registered with OCT derived maps of tissue microvasculature, the latter can be color coded as belonging to either tumor or normal tissue. Applications to radiomics‐based multimodal OCT analysis are envisioned.      

Evaluation of radiomic texture feature error due to MRI acquisition and reconstruction: A simulation study utilizing ground truth

The purpose of this study was to examine the dependence of image texture features on MR acquisition parameters and reconstruction using a digital MR imaging phantom. MR signal was simulated in a parallel imaging radiofrequency coil setting as well as a single element volume coil setting, with varying levels of acquisition noise, three acceleration factors, and four image reconstruction algorithms. Twenty-six texture features were measured on the simulated images, ground truth images, and clinical brain images. Subtle algorithm-dependent errors were observed on reconstructed phantom images, even in the absence of added noise. Sources of image error include Gibbs ringing at image edge gradients (tissue interfaces) and well-known artifacts due to high acceleration; two of the iterative reconstruction algorithms studied were able to mitigate these image errors. The difference of the texture features from ground truth, and their variance over reconstruction algorithm and parallel imaging acceleration factor, were compared to the clinical “effect size”, i.e., the feature difference between high- and low-grade tumors on T1- and T2-weighted brain MR images of twenty glioma patients. The measured feature error (difference from ground truth) was small for some features, but substantial for others. The feature variance due to reconstruction algorithm and acceleration factor were generally smaller than the clinical effect size. Certain texture features may be preserved by MR imaging, but adequate precautions need to be taken regarding their validity and reliability. We present a general simulation framework for assessing the robustness and accuracy of radiomic textural features under various MR acquisition/reconstruction scenarios.     

Computerized quantification of tissue vascularization using high-resolution slide scanning of whole tumor sections.

Assessment of tissue vascularization using immunohistochemical techniques for microvessel detection has been limited by difficulties in generating reproducible quantitative data. The distinction of individual blood vessels and the selection of microscopic fields to be analyzed remain two factors of subjectivity. In this study, we used imaging analysis software and a high-resolution slide scanner for measurement of CD31-immunostained endothelial area (EA) in whole sections of human neuroblastoma xenograft and murine mammary adenocarcinoma tumors. Imaging analysis software provided objective criteria for analysis of sections of different tumors. The use of the criteria on images of entire tumor section acquired with the slide scanner constituted a rapid method to quantify tumor vascularization. Compared with previously described methods, the "hot spot" and the "random fields" methods, EA measurements obtained with our "whole section scanning" method were more reproducible with 8.6% interobserver disagreement for the "whole section scanning" method vs 42.2% and 39.0% interobserver disagreement for the "hot spot" method and the "random fields," respectively. Microvessel density was also measured with the whole section scanning method and provided additional data on the distribution and the size of the blood vessels. Therefore, this method constitutes a time efficient and reproducible method for quantification of tumor vascularization.