From Voxels to Knowledge: A Practical Guide to the Segmentation of Complex Electron Microscopy 3D-Data

Modern 3D electron microscopy approaches have recently allowed unprecedented insight into the 3D ultrastructural organization of cells and tissues, enabling the visualization of large macromolecular machines, such as adhesion complexes, as well as higher-order structures, such as the cytoskeleton and cellular organelles in their respective cell and tissue context. Given the inherent complexity of cellular volumes, it is essential to first extract the features of interest in order to allow visualization, quantification, and therefore comprehension of their 3D organization. Each data set is defined by distinct characteristics, e.g., signal-to-noise ratio, crispness (sharpness) of the data, heterogeneity of its features, crowdedness of features, presence or absence of characteristic shapes that allow for easy identification, and the percentage of the entire volume that a specific region of interest occupies. All these characteristics need to be considered when deciding on which approach to take for segmentation.The six different 3D ultrastructural data sets presented were obtained by three different imaging approaches: resin embedded stained electron tomography, focused ion beam- and serial block face- scanning electron microscopy (FIB-SEM, SBF-SEM) of mildly stained and heavily stained samples, respectively. For these data sets, four different segmentation approaches have been applied: (1) fully manual model building followed solely by visualization of the model, (2) manual tracing segmentation of the data followed by surface rendering, (3) semi-automated approaches followed by surface rendering, or (4) automated custom-designed segmentation algorithms followed by surface rendering and quantitative analysis. Depending on the combination of data set characteristics, it was found that typically one of these four categorical approaches outperforms the others, but depending on the exact sequence of criteria, more than one approach may be successful. Based on these data, we propose a triage scheme that categorizes both objective data set characteristics and subjective personal criteria for the analysis of the different data sets.     

Integrated functional visualization of eukaryotic genomes

Background  

Increasing amounts of data from large scale whole genome analysis efforts demands convenient tools for manipulation, visualization and investigation. Whole genome plots offer an intuitive window to the analysis. We describe two applications that enable users to easily plot and explore whole genome data from their own or other researchers' experiments.     

Identification of coherent patterns in gene expression data using an efficient biclustering algorithm and parallel coordinate visualization

Background  

The DNA microarray technology allows the measurement of expression levels of thousands of genes under tens/hundreds of different conditions. In microarray data, genes with similar functions usually co-express under certain conditions only [1]. Thus, biclustering which clusters genes and conditions simultaneously is preferred over the traditional clustering technique in discovering these coherent genes. Various biclustering algorithms have been developed using different bicluster formulations. Unfortunately, many useful formulations result in NP-complete problems. In this article, we investigate an efficient method for identifying a popular type of biclusters called additive model. Furthermore, parallel coordinate (PC) plots are used for bicluster visualization and analysis.     

Real-time three-dimensional ultrasound methods for shape analysis and visualization.

Real-time three-dimensional (RT3D) ultrasound is a relatively new imaging modality that uses a special ultrasound transducer consisting of a matrix array of elements. The array electronically steers an ultrasound beam to interrogate a 3D volume of tissue. The real-time nature of RT3D ultrasound differentiates it from reconstructed 3D ultrasound, in which a conventional ultrasound transducer is moved mechanically through the third dimension. RT3D ultrasound is considerably faster than reconstructed 3D ultrasound, making it suitable for capturing continuous rapid motion such as that of the beating heart. Although RT3D ultrasound has not yet found widespread clinical use, these scanners are presently employed in more than 20 locations worldwide, primarily for cardiac research. The author helped develop the RT3D ultrasound technology as well as specialized analysis and visualization methods for the resulting data. In developing such methods, it has been necessary to consider the physical and mathematical processes by which the ultrasound data are collected. Difficulties arise because of high noise, variation in contrast and intensity between scans, ultrasound's nonrectilinear coordinate system, and the anisotropic nature of the echoes themselves. This article reviews these specific difficulties and provides solutions that are applicable to generalized analysis and visualization of RT3D ultrasound data. Some of the methods presented can also be applied to other imaging modalities with nonrectilinear coordinates.     

Microscopy-based multicolor tissue cytometry at the single-cell level.

Cytomics is a novel perspective from which to look at life. As with genomics and proteomics before, this discipline requires novel and innovative techniques and technologies to focus on its substrate of research--the cytome. With cytomics being the discipline that analyzes cellular systems and their interdependencies, advanced microscopy represents a key technology in cytomics research. Yet, conventional microscopy-based investigations, i.e., "look and conclude" analyses, do not meet the major cytomics criteria of 1) relating multiple parameters to each other, 2) within large populations of cells, 3) on a single-cell basis, and 4) in a quantitative and observer-independent manner. However, emerging improvements in the fields of fluorophore technology, sensitive fluorescence detection devices, and sophisticated image analysis procedures, are important and necessary steps into the cytomics era. Tissue represents an important class of cytomes, hence tissue cytometry--on the single cell level--can be expected to become an important cytomics technology. In this report, the techniques and technologies of microscopy-based multicolor tissue cytometry (MMTC) are outlined and applications are discussed, including the phenotypic characterization of tissue infiltrating leukocytes, in situ quantification of proliferation markers and tumor suppressors, and in situ quantification of apoptosis.     

Cytomics - importance of multimodal analysis of cell function and proliferation in oncology

Cancer is a highly complex and heterogeneous disease involving a succession of genetic changes (frequently caused or accompanied by exogenous trauma), and resulting in a molecular phenotype that in turn results in a malignant specification. The development of malignancy has been described as a multistep process involving self-sufficiency in growth signals, insensitivity to antigrowth signals, evasion of apoptosis, limitless replicative potential, sustained angiogenesis, and finally tissue invasion and metastasis. The quantitative analysis of networking molecules within the cells might be applied to understand native-state tissue signalling biology, complex drug actions and dysfunctional signalling in transformed cells, that is, in cancer cells. High-content and high-throughput single-cell analysis can lead to systems biology and cytomics. The application of cytomics in cancer research and diagnostics is very broad, ranging from the better understanding of the tumour cell biology to the identification of residual tumour cells after treatment, to drug discovery. The ultimate goal is to pinpoint in detail these processes on the molecular, cellular and tissue level. A comprehensive knowledge of these will require tissue analysis, which is multiplex and functional; thus, vast amounts of data are being collected from current genomic and proteomic platforms for integration and interpretation as well as for new varieties of updated cytomics technology. This overview will briefly highlight the most important aspects of this continuously developing field.     

Quantification of vascular density using a semiautomated technique for immunostained specimens

OBJECTIVE: To develop a semiautomated, quantitative techniquefor the assessment of vascular density in immunohistochemically stained tissue sections using diaminobenzidine tetrahydrochloride (DAB) and hematoxylin as chromagens. STUDY DESIGN: A semiautomated thresholding technique was developed to quantitate vascular density in tissue sections stained with anti-CD31 (1 degrees antibody). The immunohistochemically stained specimens were digitally imaged using a 24-bit color camera. The blue component of the RGB image was segmented using a variable high-pass filter. After thresholding, the segmented areas (CD31 positive) were quantified and vascular density determined. The validity of the method was verified by calculating the precision of the technique using the coefficient of repeatability and by quantifying its agreement with manual analysis according to the Bland-Altman approach. RESULTS: Vascular endothelial cells were specifically selected using anti-CD31 as the primary antibody and the appropriate horseradish peroxidase-conjugated secondary antibody. Utilizing the semiautomated thresholding technique, the separation of DAB-stained tissuefrom non-DAB-stained tissue was achieved. The method developed possesses a low coefficient of repeatability (0.49%), agrees well with manual assessment (mean difference = -0.29 +/- 0.92%), is highly automated and is user friendly. CONCLUSION: A novel semiautomated technique for the quantification of vascular density was developed. This technique provides a method for reproducible measurement of immunostaining procedures (immunohistochemistry, immunocytochemistry and in situ hybridization) utilizing immunoperoxidase techniques with DAB as a chromagen.     

Quantification and its Applications in Fluorescent Microscopy Imaging

Fluorescent microscope imaging technologies have developed at a rapid pace in recent years. High-throughput 2D fluorescent imaging platforms are now in wide use and are being applied on a proteome wide scale. Multiple fluorophore 3D imaging of live cells is being used to give detailed localization and subcellular structure information. Further, 2D and 3D video microscopy are giving important insights into the dynamics of protein localization and transport. In parallel with these developments, significant research has gone into developing new methodologies for quantifying and extracting meaning from the imaging data. Here we outline and give entry points to the literature on approaches to quantification such as segmentation, tracking, automated classification and data visualization. Particular attention is paid to the distinction between and application of concrete quantification measures such as number of objects in a cell, and abstract measures such as texture.     

Interactive Data Visualization for HIV Cohorts: Leveraging Data Exchange Standards to Share and Reuse Research Tools

Objective

To develop and disseminate tools for interactive visualization of HIV cohort data.

Design and Methods

If a picture is worth a thousand words, then an interactive video, composed of a long string of pictures, can produce an even richer presentation of HIV population dynamics. We developed an HIV cohort data visualization tool using open-source software (R statistical language). The tool requires that the data structure conform to the HIV Cohort Data Exchange Protocol (HICDEP), and our implementation utilized Caribbean, Central and South America network (CCASAnet) data.

Results

This tool currently presents patient-level data in three classes of plots: (1) Longitudinal plots showing changes in measurements viewed alongside event probability curves allowing for simultaneous inspection of outcomes by relevant patient classes. (2) Bubble plots showing changes in indicators over time allowing for observation of group level dynamics. (3) Heat maps of levels of indicators changing over time allowing for observation of spatial-temporal dynamics. Examples of each class of plot are given using CCASAnet data investigating trends in CD4 count and AIDS at antiretroviral therapy (ART) initiation, CD4 trajectories after ART initiation, and mortality.

Conclusions

We invite researchers interested in this data visualization effort to use these tools and to suggest new classes of data visualization. We aim to contribute additional shareable tools in the spirit of open scientific collaboration and hope that these tools further the participation in open data standards like HICDEP by the HIV research community.     

Fast quantification of immunohistochemistry tissue microarrays in lung carcinoma

Tissue microarrays (TMAs) are an effective tool for high-throughput molecular analysis of tissues to help identify new diagnostic and prognostic markers and targets in human cancers. We have developed a fully automated method for rapid, continuous and quantitative analysis of TMAs based on immunohistochemistry. The method deals with complex and varying tissue architectures, segments tumour cells from normal cells, conducts cell compartmentalisation, identifies nuclei and cytoplasm and produces three different continuous measurements of marker expression levels within tumour cell nuclei, tumour cell cytoplasm and total tumour cell protein expression. We have demonstrated this method using three independent protein markers (BAK, BAX and a novel biomarker, named KS) over 7 TMAs, involving 2 BAK stained TMAs with 229 tumour tissue cores, 2 BAX stained TMAs with 229 tumour tissue cores and 3 KS stained TMAs with 373 tumour cores of lung carcinomas. We validated the automated method, showing that the automated scoring is significantly correlated with the pathologist-based scoring.     

Interpreting plots of a multidimensional dose-response surface in a parallel coordinate system

The dose-response surface for a combination of drugs is a multidimensional figure. Consequently, it is not possible to view such a surface using orthogonal axes when the number of dimensions exceeds 3. Parallel axes have been used to represent hyperdimensional figures. This paper reports on the use of parallel coordinate axes to plot the dose-response surface and its contours of constant response (isobols) for a combination of drugs. It is shown that patterns formed by intersecting line segments in the parallel system can aid in the interpretation of the fitted dose-response surface. More generally, analytic results are developed that permit the ready visualization and characterization of interaction effects of a polynomial model.     

DNA analysis servers: plot.it,bend.it,model.it and IS

The WWW servers at http://www.icgeb.trieste.it/dna/ are dedicated to the analysis of user-submitted DNA sequences; plot.it creates parametric plots of 45 physicochemical, as well as statistical, parameters; bend.it calculates DNA curvature according to various methods. Both programs provide 1D as well as 2D plots that allow localisation of peculiar segments within the query. The server model.it creates 3D models of canonical or bent DNA starting from sequence data and presents the results in the form of a standard PDB file, directly viewable on the user's PC using any molecule manipulation program. The recently established introns server allows statistical evaluation of introns in various taxonomic groups and the comparison of taxonomic groups in terms of length, base composition, intron type etc. The options include the analysis of splice sites and a probability test for exon-shuffling.     

Point pattern analysis under conditions of replicated sampling

Aims The analysis of point patterns, which deals with data sets consisting of mapped locations of organisms in a study region, is especially important to plant ecological studies because the locations of plants can often be approximated as points. However, few studies used point pattern analysis with data collected by replicated sampling a principle procedure of acquiring data in ecological research. Therefore, we explore the applicability of point pattern analysis under conditions of replicated sampling in this studyMethodsThree replicated 5 m × 5 m plots of homogenous communities were established on a site with eight years of restoration in Nei Mongol steppe. In each plot, the locations of individuals in Leymus chinensis and Stipa grandis populations were mapped. O-Ring function was used to describe the population patterns and species association between L. chinensis and S. grandis for each plot as well as the integrative data of the three replicates.Important findings Population patterns and species associations differed among the three replicated plots. This illustrates that if point pattern analysis was applied to describe the population patterns and species associations only by using data from a single plot sampling, the results could be misleading. Whereas it would be more reliable to integrate the data of replicated plots in the point pattern analysis because in this way the resulting O-Ring function is a weighted average, where the weight is the number of points in the replicate i divided by the total number of points in all replicated plots.     

Fusion of radiostereometric analysis data into computed tomography space: application to the elbow joint

Improvement of joint prostheses is dependent upon information concerning the biomechanical properties of the joint. Radiostereometric analysis (RSA) and electromagnetic techniques have been applied in previous cadaver and in vivo studies on the elbow joint to provide valuable information concerning joint motion axes. However, such information is limited to mathematically calculated positions of the axes according to an orthogonal coordinate system and is difficult to relate to individual skeletal anatomy. The aim of this study was to evaluate the in vivo application of a new fusion method to provide three-dimensional (3D) visualization of flexion axes according to bony landmarks. In vivo RSA data of the elbow joint's flexion axes was combined with data obtained by 3D computed tomography (CT). Results were obtained from five healthy subjects after one was excluded due to an instable RSA marker. The median error between imported and transformed RSA marker coordinates and those obtained in the CT volume was 0.22 mm. Median maximal rotation error after transformation of the rigid RSA body to the CT volume was 0.003 degrees . Points of interception with a plane calculated in the RSA orthogonal coordinate system were imported into the CT volume, facilitating the 3D visualization of the flexion axes. This study demonstrates a successful fusion of RSA and CT data, without significant loss of RSA accuracy. The method could be used for relating individual motion axes to a 3D representation of relevant joint anatomy, thus providing important information for clinical applications such as the development of joint prostheses.     

Quantification of tumour vascularity in squamous cell carcinoma of the tongue using CARD amplification, a systematic sampling technique, and true colour image analysis.

The aims of this study of head and neck tissue samples were to develop an immunohistochemical protocol based on the catalysed reporter deposition (CARD) technique to enhance staining results for use in automated true colour image analysis, to assess the reproducibility of systematic tissue sampling in the angiogenic hot spot selection, and quantification of microvessel density (MVD) and other vessel characteristics. The latter data were compared between six metastasised tongue squamous cell carcinomas, vs. four non-metastasised. In comparison to the standard immunohistochemical protocol with anti-CD34 antibodies, CARD amplification resulted in both more intensely stained and larger numbers of vessels. Averaging the 10 most vascularised fields of the 40 to 60 systematically sampled fields in a tissue section resulted in an overall acceptable interobserver reproducibility for most assessed vessel parameters (r> or =0.76 and p< or =0.01). The percentage vessels with diameter <5 microm was significantly higher in the non-metastasised tongue carcinomas (p=0.02). However, for a number of tumours the effect of tissue sampling was significant.We conclude that CARD amplification is needed for reliable segmentation of vessels by image analysis systems, and that tumour heterogeneity is a limiting factor for all procedures in which tumour vascularity is assessed in a single tissue section.Figures on http://www.esacp.org/acp/2001/22-4/hannen.htm.     

Severe interference between retinal angiography and automated four-color flow cytometry analysis of blood mononuclear cells.

BACKGROUND: Retinal angiography has become a widely used diagnostic tool. It requires the intravenous administration of the fluorescent dyes fluorescein and indocyanin green. We recently received blood taken 8 h after retinal angiography, without our knowing it. We describe the failure of an automated flow cytometry system in the enumeration of lymphocyte subpopulations in this sample. METHODS: Cell enumeration was achieved by the use of the lyse-no wash MultiTEST procedure (Becton-Dickinson) together with the FACSCalibur cytometer. Absolute cell counts were obtained using TruCount beads. Data were analyzed automatically by the MultiSET and manually with the CellQuest softwares. RESULTS: The dot plots obtained with this sample looked quite abnormal. All monuclear cells stained brightly in the FITC channel irrespective of anti-CD3-FITC conjugate binding. This resulted in a major undercompensation for the increased spillover of the fluorescein emission into the PE-channel. PE-labeled cell and TruCount bead events coalesced. The MultiSET software failed to draw proper gatings and proved useless. Alternative manual gatings could partially rescue the analysis. CONCLUSIONS: Clinicians and cytometrists should be aware that, because of dye entry or binding, blood mononuclear cells collected shortly after retinal angiography are not suitable even for common cytometry applications.     

Rapid 3D phenotypic analysis of neurons and organoids using data-driven cell segmentation-free machine learning

Phenotypic profiling of large three-dimensional microscopy data sets has not been widely adopted due to the challenges posed by cell segmentation and feature selection. The computational demands of automated processing further limit analysis of hard-to-segment images such as of neurons and organoids. Here we describe a comprehensive shallow-learning framework for automated quantitative phenotyping of three-dimensional (3D) image data using unsupervised data-driven voxel-based feature learning, which enables computationally facile classification, clustering and advanced data visualization. We demonstrate the analysis potential on complex 3D images by investigating the phenotypic alterations of: neurons in response to apoptosis-inducing treatments and morphogenesis for oncogene-expressing human mammary gland acinar organoids. Our novel implementation of image analysis algorithms called Phindr3D allowed rapid implementation of data-driven voxel-based feature learning into 3D high content analysis (HCA) operations and constitutes a major practical advance as the computed assignments represent the biology while preserving the heterogeneity of the underlying data. Phindr3D is provided as Matlab code and as a stand-alone program (https://github.com/DWALab/Phindr3D).     

Digital immunohistochemistry platform for the staining variation monitoring based on integration of image and statistical analyses with laboratory information system

Background

Digital immunohistochemistry (IHC) is one of the most promising applications brought by new generation image analysis (IA). While conventional IHC staining quality is monitored by semi-quantitative visual evaluation of tissue controls, IA may require more sensitive measurement. We designed an automated system to digitally monitor IHC multi-tissue controls, based on SQL-level integration of laboratory information system with image and statistical analysis tools.

Methods

Consecutive sections of TMA containing 10 cores of breast cancer tissue were used as tissue controls in routine Ki67 IHC testing. Ventana slide label barcode ID was sent to the LIS to register the serial section sequence. The slides were stained and scanned (Aperio ScanScope XT), IA was performed by the Aperio/Leica Colocalization and Genie Classifier/Nuclear algorithms. SQL-based integration ensured automated statistical analysis of the IA data by the SAS Enterprise Guide project. Factor analysis and plot visualizations were performed to explore slide-to-slide variation of the Ki67 IHC staining results in the control tissue.

Results

Slide-to-slide intra-core IHC staining analysis revealed rather significant variation of the variables reflecting the sample size, while Brown and Blue Intensity were relatively stable. To further investigate this variation, the IA results from the 10 cores were aggregated to minimize tissue-related variance. Factor analysis revealed association between the variables reflecting the sample size detected by IA and Blue Intensity. Since the main feature to be extracted from the tissue controls was staining intensity, we further explored the variation of the intensity variables in the individual cores. MeanBrownBlue Intensity ((Brown+Blue)/2) and DiffBrownBlue Intensity (Brown-Blue) were introduced to better contrast the absolute intensity and the colour balance variation in each core; relevant factor scores were extracted. Finally, tissue-related factors of IHC staining variance were explored in the individual tissue cores.

Conclusions

Our solution enabled to monitor staining of IHC multi-tissue controls by the means of IA, followed by automated statistical analysis, integrated into the laboratory workflow. We found that, even in consecutive serial tissue sections, tissue-related factors affected the IHC IA results; meanwhile, less intense blue counterstain was associated with less amount of tissue, detected by the IA tools.