Automated Processing of Imaging Data through Multi-tiered Classification of Biological Structures Illustrated Using Caenorhabditis elegans

Quantitative imaging has become a vital technique in biological discovery and clinical diagnostics; a plethora of tools have recently been developed to enable new and accelerated forms of biological investigation. Increasingly, the capacity for high-throughput experimentation provided by new imaging modalities, contrast techniques, microscopy tools, microfluidics and computer controlled systems shifts the experimental bottleneck from the level of physical manipulation and raw data collection to automated recognition and data processing. Yet, despite their broad importance, image analysis solutions to address these needs have been narrowly tailored. Here, we present a generalizable formulation for autonomous identification of specific biological structures that is applicable for many problems. The process flow architecture we present here utilizes standard image processing techniques and the multi-tiered application of classification models such as support vector machines (SVM). These low-level functions are readily available in a large array of image processing software packages and programming languages. Our framework is thus both easy to implement at the modular level and provides specific high-level architecture to guide the solution of more complicated image-processing problems. We demonstrate the utility of the classification routine by developing two specific classifiers as a toolset for automation and cell identification in the model organism Caenorhabditis elegans. To serve a common need for automated high-resolution imaging and behavior applications in the C. elegans research community, we contribute a ready-to-use classifier for the identification of the head of the animal under bright field imaging. Furthermore, we extend our framework to address the pervasive problem of cell-specific identification under fluorescent imaging, which is critical for biological investigation in multicellular organisms or tissues. Using these examples as a guide, we envision the broad utility of the framework for diverse problems across different length scales and imaging methods.     

Recovering the dynamics of root growth and development using novel image acquisition and analysis methods

Roots are highly responsive to environmental signals encountered in the rhizosphere, such as nutrients, mechanical resistance and gravity. As a result, root growth and development is very plastic. If this complex and vital process is to be understood, methods and tools are required to capture the dynamics of root responses. Tools are needed which are high-throughput, supporting large-scale experimental work, and provide accurate, high-resolution, quantitative data. We describe and demonstrate the efficacy of the high-throughput and high-resolution root imaging systems recently developed within the Centre for Plant Integrative Biology (CPIB). This toolset includes (i) robotic imaging hardware to generate time-lapse datasets from standard cameras under infrared illumination and (ii) automated image analysis methods and software to extract quantitative information about root growth and development both from these images and via high-resolution light microscopy. These methods are demonstrated using data gathered during an experimental study of the gravitropic response of Arabidopsis thaliana.     

Semiautomatic microscopy as a tool for cytologists

Despite impressive advances in the application of computer image analysis to cytology, many of the identification tasks that cytologists are called on to perform remain refractory to automated image analysis. The major reason is that a large fraction of these images, though simple for a human to deal with, are too complex to yield to current image analysis methodologies. It may be years before automated computer image analysis is reduced to clinical practicality. Even then, it is not clear that all cytologic image analyses will prove amenable to automation. In the meantime, semiautomatic image analysis (computer-aided microscopy) can provide a viable alternative, especially to persistently difficult image analysis problems. In semiautomatic image analysis, the onerous tasks of data acquisition--e.g., stage movement, data entry and storage--are left to the computer, while the decision-making tasks-e.g., identifying a cell's morphologic class--are left to the observer. Such a system proves to be easy and flexible to use as well as economical to build. It can also provide a reliable data base for the later evaluation of fully automated systems as they are developed. One such semiautomatic system, the Image Combining Computer Microscope (ICCM), is described, and the range of its application is illustrated. Some of the examples of ICCM applications discussed are: neuronal cell plots, three-dimensional dendrite tracking, serial section reconstruction of axons and mapping of plaques and tangles in Alzheimer's disease. They illustrate how powerful a semiautomated system can be in handling complex image analysis problems. It is suggested that semiautomated image analysis provides a viable long-range alternative to many cytologic image analysis problems.     

Mapping the gene expression universe

Methods to globally survey gene expression provide valuable insights into gene function during development. In particular, comprehensive in situ hybridization studies have demonstrated that gene expression patterns are extraordinarily diverse and new imaging techniques have been introduced to capture these patterns with higher resolution at the tissue, cellular, and subcellular levels. The analysis of massive image databases can be greatly facilitated by computer vision techniques once annotated image sets reach the crucial mass sufficient to train the computer in pattern recognition. Ultimately, genome-wide atlases of gene expression during development will record gene activity in living animals with at least cellular resolution and in the context of morphogenetic events. These emerging datasets will lead to great advances in the field of comparative genomics and revolutionize our ability to decipher and model developmental processes for a variety of organisms.     

Microscopy Image Browser: A Platform for Segmentation and Analysis of Multidimensional Datasets

Understanding the structure–function relationship of cells and organelles in their natural context requires multidimensional imaging. As techniques for multimodal 3-D imaging have become more accessible, effective processing, visualization, and analysis of large datasets are posing a bottleneck for the workflow. Here, we present a new software package for high-performance segmentation and image processing of multidimensional datasets that improves and facilitates the full utilization and quantitative analysis of acquired data, which is freely available from a dedicated website. The open-source environment enables modification and insertion of new plug-ins to customize the program for specific needs. We provide practical examples of program features used for processing, segmentation and analysis of light and electron microscopy datasets, and detailed tutorials to enable users to rapidly and thoroughly learn how to use the program.     

Automated feature detection and imaging for high-resolution screening of zebrafish embryos

The development of automated microscopy platforms has enabled large-scale observation of biological processes, thereby complementing genome scale biochemical techniques. However, commercially available systems are restricted either by fixed-field-of-views, leading to potential omission of features of interest, or by low-resolution data of whole objects lacking cellular detail. This limits the efficiency of high-content screening assays, especially when large complex objects are used as in whole-organism screening. Here we demonstrate a toolset for automated intelligent high-content screening of whole zebrafish embryos at cellular resolution on a standard wide-field screening microscope. Using custom-developed algorithms, predefined regions of interest-such as the brain-are automatically detected. The regions of interest are subsequently imaged automatically at high magnification, enabling rapid capture of cellular resolution data. We utilize this approach for acquiring 3-D datasets of embryonic brains of transgenic zebrafish. Moreover, we report the development of a mold design for accurate orientation of zebrafish embryos for dorsal imaging, thereby facilitating standardized imaging of internal organs and cellular structures. The toolset is flexible and can be readily applied for the imaging of different specimens in various applications.     

Combined confocal and wide-field high-resolution cytometry of fluorescent in situ hybridization-stained cells

BACKGROUND: The recently developed technique of high-resolution cytometry (HRCM) enables automated acquisition and analysis of fluorescent in situ hybridization (FISH)-stained cell nuclei using conventional wide-field fluorescence microscopy. The method has now been extended to confocal imaging and offers the opportunity to combine the advantages of confocal and wide-field modes. METHODS: We have automated image acquisition and analysis from a standard inverted fluorescence microscope equipped with a confocal module with Nipkow disk and a cooled digital CCD camera. The system is fully controlled by a high-performance computer that performs both acquisition and related on-line image analysis. The system can be used either for an automatic two (2D) and three-dimensional (3D) analysis of FISH- stained interphase nuclei or for a semiautomatic 3D analysis of FISH-stained cells in tissues. The user can select which fluorochromes are acquired using wide-field mode and which using confocal mode. The wide-field and confocal images are overlaid automatically in computer memory. The developed software compensates automatically for both chromatic color shifts and spatial shifts caused by switching to a different imaging mode. RESULTS: Using the combined confocal and wide-field HRCM technique, it is possible to take advantage of both imaging modes. Images of some dyes (such as small hybridization dots or counterstain images of individual interphase nuclei) do not require confocal quality and can be acquired quickly in wide-field mode. On the contrary, images of other dyes (such as chromosome territories or counterstain images of cells in tissues) do require improved quality and are acquired in confocal mode. The dual-mode approach is two to three times faster compared with the single-mode confocal approach and the spectrum of its applications is much broader compared with both single-mode confocal and single-mode wide-field systems. CONCLUSIONS: The combination of high speed specific to the wide-field mode and high quality specific to the confocal mode gives optimal system performance.     

Using Light Sheet Fluorescence Microscopy to Image Zebrafish Eye Development

Light sheet fluorescence microscopy (LSFM) is gaining more and more popularity as a method to image embryonic development. The main advantages of LSFM compared to confocal systems are its low phototoxicity, gentle mounting strategies, fast acquisition with high signal to noise ratio and the possibility of imaging samples from various angles (views) for long periods of time. Imaging from multiple views unleashes the full potential of LSFM, but at the same time it can create terabyte-sized datasets. Processing such datasets is the biggest challenge of using LSFM. In this protocol we outline some solutions to this problem. Until recently, LSFM was mostly performed in laboratories that had the expertise to build and operate their own light sheet microscopes. However, in the last three years several commercial implementations of LSFM became available, which are multipurpose and easy to use for any developmental biologist. This article is primarily directed to those researchers, who are not LSFM technology developers, but want to employ LSFM as a tool to answer specific developmental biology questions. Here, we use imaging of zebrafish eye development as an example to introduce the reader to LSFM technology and we demonstrate applications of LSFM across multiple spatial and temporal scales. This article describes a complete experimental protocol starting with the mounting of zebrafish embryos for LSFM. We then outline the options for imaging using the commercially available light sheet microscope. Importantly, we also explain a pipeline for subsequent registration and fusion of multiview datasets using an open source solution implemented as a Fiji plugin. While this protocol focuses on imaging the developing zebrafish eye and processing data from a particular imaging setup, most of the insights and troubleshooting suggestions presented here are of general use and the protocol can be adapted to a variety of light sheet microscopy experiments.     

Volume scanning electron microscopy for imaging biological ultrastructure

Electron microscopy (EM) has been a key imaging method to investigate biological ultrastructure for over six decades. In recent years, novel volume EM techniques have significantly advanced nanometre‐scale imaging of cells and tissues in three dimensions. Previously, this had depended on the slow and error‐prone manual tasks of cutting and handling large numbers of sections, and imaging them one‐by‐one with transmission EM. Now, automated volume imaging methods mostly based on scanning EM (SEM) allow faster and more reliable acquisition of serial images through tissue volumes and achieve higher z‐resolution. Various software tools have been developed to manipulate the acquired image stacks and facilitate quantitative analysis. Here, we introduce three volume SEM methods: serial block‐face electron microscopy (SBEM), focused ion beam SEM (FIB‐SEM) and automated tape‐collecting ultramicrotome SEM (ATUM‐SEM). We discuss and compare their capabilities, provide an overview of the full volume SEM workflow for obtaining 3D datasets and showcase different applications for biological research.     

Image-based screening: a technology in transition

Image-based screening (IBS) has proven itself with whole-well assays in which throughput and assay miniaturization are priorities. Recent interest, however, has centered upon the use of automated imaging technology to conduct assays at subcellular resolution. These in vitro assays have the potential to increase lead quality at early stages in drug discovery. Subcellular IBS is not yet mature and, although some assays provide reliable data at reasonable throughput, many others have yet to demonstrate robust application. Developments in image acquisition, analysis and informatics technologies are ongoing and are expected to broaden the usefulness of subcellular IBS.     

Computer‐automated bird detection and counts in high‐resolution aerial images: a review

Bird surveys conducted using aerial images can be more accurate than those using airborne observers, but can also be more time‐consuming if images must be analyzed manually. Recent advances in digital cameras and image‐analysis software offer unprecedented potential for computer‐automated bird detection and counts in high‐resolution aerial images. We review the literature on this subject and provide an overview of the main image‐analysis techniques. Birds that contrast sharply with image backgrounds (e.g., bright birds on dark ground) are generally the most amenable to automated detection, in some cases requiring only basic image‐analysis software. However, the sophisticated analysis capabilities of modern object‐based image analysis software provide ways to detect birds in more challenging situations based on a variety of attributes including color, size, shape, texture, and spatial context. Some techniques developed to detect mammals may also be applicable to birds, although the prevalent use of aerial thermal‐infrared images for detecting large mammals is of limited applicability to birds because of the low pixel resolution of thermal cameras and the smaller size of birds. However, the increasingly high resolution of true‐color cameras and availability of small unmanned aircraft systems (drones) that can fly at very low altitude now make it feasible to detect even small shorebirds in aerial images. Continued advances in camera and drone technology, in combination with increasingly sophisticated image analysis software, now make it possible for investigators involved in monitoring bird populations to save time and resources by increasing their use of automated bird detection and counts in aerial images. We recommend close collaboration between wildlife‐monitoring practitioners and experts in the fields of remote sensing and computer science to help generate relevant, accessible, and readily applicable computer‐automated aerial photographic census techniques.     

Toward high-content/high-throughput imaging and analysis of embryonic morphogenesis

In vivo study of embryonic morphogenesis tremendously benefits from recent advances in live microscopy and computational analyses. Quantitative and automated investigation of morphogenetic processes opens the field to high-content and high-throughput strategies. Following experimental workflow currently developed in cell biology, we identify the key challenges for applying such strategies in developmental biology. We review the recent progress in embryo preparation and manipulation, live imaging, data registration, image segmentation, feature computation, and data mining dedicated to the study of embryonic morphogenesis. We discuss a selection of pioneering studies that tackled the current methodological bottlenecks and illustrated the investigation of morphogenetic processes in vivo using quantitative and automated imaging and analysis of hundreds or thousands of cells simultaneously, paving the way for high-content/high-throughput strategies and systems analysis of embryonic morphogenesis.     

A general purpose computerized display and analysis system for image processing in nuclear medicine.

In this work a general purpose image processor is described, which takes into account the special features and the various requirements for analysing images in nuclear medicine. A new approach for system structure involving encoding and representing medical images is given. This encoder, which is part of the image processor, is efficient especially for long-term storage and for certain pattern recognition in medical imaging. In this processor a statistical package is incorporated for collecting medical statistics using the acquired patient data. This information is used together with the encoder for an automated diagnostic system.     

Pharmacological profiling of chemokine receptor-directed compounds using high-content screening

High-content screening, typically defined as automated fluorescence microscopy combined with image analysis, is now well established as a means to study test compound effects in cellular disease-modeling systems. In this work, the authors establish several high-content screening assays in the 384-well format to measure the activation of the CC-type chemokine receptors 2B and 3 (CCR2B, CCR3). As a cellular model system, the authors use Chinese hamster ovary cells, stably transfected with 1 of the respective receptors. They characterize receptor stimulation by human monocyte chemoattractant protein-1 for CCR2B and by human eotaxin-1 for CCR3: Receptor internalization and receptor-induced phosphorylation of ERK1/2 (pERK) were quantified using fluorescence imaging and image analysis. The 4 assay formats were robust, displayed little day-to-day variability, and delivered good Z' statistics for both CCRs. For each of the 2 receptors, the authors evaluated the potency of inhibitory compounds in the internalization format and the pERK assay and compared the results with those from other assays (ligand displacement binding, Ca(2+) mobilization, guanosine triphosphate exchange, chemotaxis). Both physiological agonists and test compounds differed significantly with respect to potencies and efficacies in the various profiling assays. The diverse assay formats delivered partially overlapping and partially complementary information, enabling the authors to reduce the probability of test compound-related technology artifacts and to specify the mode of action for individual test compounds. Transfer of the high-content screening format to a fully automated medium-throughput screening platform for CCR3 enabled the profiling of large compound numbers with respect to G protein signaling and possible tolerance-inducing liabilities.     

Image analysis, neural networks, and the taxonomic impediment to biodiversity studies

The taxonomic impediment to biodiversity studies may be influenced radically by the application of new technology, in particular, desktop image analysers and neural networks. The former offer an opportunity to automate objective feature measurement processes, and the latter provide powerful pattern recognition and data analysis tools which are able to 'learn' patterns in multivariate data. The coupling of these technologies may provide a realistic opportunity for the automation of routine species identifications. The potential benefits and limitations of these technologies, along with the development of automated identification systems are reviewed.     

Evaluation of <Emphasis Type="Italic">Isochrysis galbana</Emphasis> (clone T-ISO) cell numbers by digital image analysis of color intensity

In microalgal cultivation, measuring cell numbers as a means to monitor growth rates is a long-standing problem. Many automated counting systems and schemes have been developed; among these are image analysis systems. However, such imaging systems have presented difficulties in dealing with the complexities of computer recognition of individual microscopic cells. It is known that the coloration of microalgae suspension is species specific and that color intensity increases are typically associated with increasing numbers. Using this qualitative insight, the present work describes the design, construction, and comparative performance of an inexpensive digital imaging system optimized for counting microalgal cells. The system circumvents the need to count individual cells and extracts cell numbers directly from the macroscopic color intensity of a microalgal suspension. The results suggest, using Isochrysis galbana (T-ISO) as an illustrative example, that this scheme is potentially useful for inexpensive and automated biomonitoring of microalgal cell numbers. Percentage difference comparisons with a standard Coulter Counter indicated that the three algorithms tested provided better than 10% accuracy over density thresholds of 1.52 × 10⁶ to 8.10 × 10⁶ cells mL⁻¹ with precision of 4% attainable at high density concentrations.     

Automated live cell imaging systems reveal dynamic cell behavior

Automated time‐lapsed microscopy provides unique research opportunities to visualize cells and subcellular components in experiments with time‐dependent parameters. As accessibility to these systems is increasing, we review here their use in cell science with a focus on stem cell research. Although the use of time‐lapsed imaging to answer biological questions dates back nearly 150 years, only recently have the use of an environmentally controlled chamber and robotic stage controllers allowed for high‐throughput continuous imaging over long periods at the cell and subcellular levels. Numerous automated imaging systems are now available from both companies that specialize in live cell imaging and from major microscope manufacturers. We discuss the key components of robots used for time‐lapsed live microscopic imaging, and the unique data that can be obtained from image analysis. We show how automated features enhance experimentation by providing examples of uniquely quantified proliferation and migration live cell imaging data. In addition to providing an efficient system that drastically reduces man‐hours and consumes fewer laboratory resources, this technology greatly enhances cell science by providing a unique dataset of temporal changes in cell activity. © 2011 American Institute of Chemical Engineers Biotechnol. Prog., 2011     

Quantitative phase imaging of cells in a flow cytometry arrangement utilizing Michelson interferometer‐based off‐axis digital holographic microscopy

We combined Michelson‐interferometer‐based off‐axis digital holographic microscopy (DHM) with a common flow cytometry (FCM) arrangement. Utilizing object recognition procedures and holographic autofocusing during the numerical reconstruction of the acquired off‐axis holograms, sharply focused quantitative phase images of suspended cells in flow were retrieved without labeling, from which biophysical cellular features of distinct cells, such as cell radius, refractive index and dry mass, can be subsequently retrieved in an automated manner. The performance of the proposed concept was first characterized by investigations on microspheres that were utilized as test standards. Then, we analyzed two types of pancreatic tumor cells with different morphology to further verify the applicability of the proposed method for quantitative live cell imaging. The retrieved biophysical datasets from cells in flow are found in good agreement with results from comparative investigations with previously developed DHM methods under static conditions, which demonstrates the effectiveness and reliability of our approach. Our results contribute to the establishment of DHM in imaging FCM and prospect to broaden the application spectrum of FCM by providing complementary quantitative imaging as well as additional biophysical cell parameters which are not accessible in current high‐throughput FCM measurements.     

Artificial neural network-aided image analysis system for cell counting.

BACKGROUND: In histological preparations containing debris and synthetic materials, it is difficult to automate cell counting using standard image analysis tools, i.e., systems that rely on boundary contours, histogram thresholding, etc. In an attempt to mimic manual cell recognition, an automated cell counter was constructed using a combination of artificial intelligence and standard image analysis methods. METHODS: Artificial neural network (ANN) methods were applied on digitized microscopy fields without pre-ANN feature extraction. A three-layer feed-forward network with extensive weight sharing in the first hidden layer was employed and trained on 1,830 examples using the error back-propagation algorithm on a Power Macintosh 7300/180 desktop computer. The optimal number of hidden neurons was determined and the trained system was validated by comparison with blinded human counts. System performance at 50x and lO0x magnification was evaluated. RESULTS: The correlation index at 100x magnification neared person-to-person variability, while 50x magnification was not useful. The system was approximately six times faster than an experienced human. CONCLUSIONS: ANN-based automated cell counting in noisy histological preparations is feasible. Consistent histology and computer power are crucial for system performance. The system provides several benefits, such as speed of analysis and consistency, and frees up personnel for other tasks.