Automated screening of 2D crystallization trials using transmission electron microscopy: A high-throughput tool-chain for sample preparation and microscopic analysis

We have built and extensively tested a tool-chain to prepare and screen two-dimensional crystals of membrane proteins by transmission electron microscopy (TEM) at room temperature. This automated process is an extension of a new procedure described recently that allows membrane protein 2D crystallization in parallel (Iacovache et al., 2010). The system includes a gantry robot that transfers and prepares the crystalline solutions on grids suitable for TEM analysis and an entirely automated microscope that can analyze 96 grids at once without human interference. The operation of the system at the user level is solely controlled within the MATLAB environment: the commands to perform sample handling (loading/unloading in the microscope), microscope steering (magnification, focus, image acquisition, etc.) as well as automatic crystal detection have been implemented. Different types of thin samples can efficiently be screened provided that the particular detection algorithm is adapted to the specific task. Hence, operating time can be shared between multiple users. This is a major step towards the integration of transmission electron microscopy into a high throughput work-flow.     

How to measure diagnosis-associated information in virtual slides

The distribution of diagnosis-associated information in histological slides is often spatial dependent. A reliable selection of the slide areas containing the most significant information to deriving the associated diagnosis is a major task in virtual microscopy. Three different algorithms can be used to select the appropriate fields of view: 1) Object dependent segmentation combined with graph theory; 2) time series associated texture analysis; and 3) geometrical statistics based upon geometrical primitives. These methods can be applied by sliding technique (i.e., field of view selection with fixed frames), and by cluster analysis. The implementation of these methods requires a standardization of images in terms of vignette correction and gray value distribution as well as determination of appropriate magnification (method 1 only). A principle component analysis of the color space can significantly reduce the necessary computation time. Method 3 is based upon gray value dependent segmentation followed by graph theory application using the construction of (associated) minimum spanning tree and Voronoi's neighbourhood condition. The three methods have been applied on large sets of histological images comprising different organs (colon, lung, pleura, stomach, thyroid) and different magnifications, The trials resulted in a reproducible and correct selection of fields of view in all three methods. The different algorithms can be combined to a basic technique of field of view selection, and a general theory of "image information" can be derived. The advantages and constraints of the applied methods will be discussed.     

Automated 100-position specimen loader and image acquisition system for transmission electron microscopy

We report the development of a novel, multi-specimen imaging system for high-throughput transmission electron microscopy. Our cartridge-based loading system, called the "Gatling", permits the sequential examination of as many as 100 specimens in the microscope for room temperature electron microscopy using mechanisms for rapid and automated specimen exchange. The software for the operation of the Gatling and automated data acquisition has been implemented in an updated version of our in-house program AutoEM. In the current implementation of the system, the time required to deliver 95 specimens into the microscope and collect overview images from each is about 13 h. Regions of interest are identified from a low magnification atlas generation from each specimen and an unlimited number of higher magnifications images can be subsequently acquired from these regions using fully automated data acquisition procedures that can be controlled from a remote interface. We anticipate that the availability of the Gatling will greatly accelerate the speed of data acquisition for a variety of applications in biology, materials science, and nanotechnology that require rapid screening and image analysis of multiple specimens.     

Software development of the EMI head scanner.

A method has been developed to run the general purpose operating system RDOS on the same disc of the head scanner computer as is used for scanner software and data. This made it possible to develop additional software in high level programming language for image processing, based on original image data on the disc. All new images produced by the program are stored on the disc in the same format as the original images. This makes it possible to handle processed images exactly as the original ones and to do multiple operations. The following processing has been included in the program so far: subtraction, smoothing, density profiles, vertical reconstructions, magnification and labelling. A set of operator commands has been developed which are very similar to the ordinary commands for the scanner, which makes the program to appear being a direct extension of the standard scanner software.     

Automated acquisition of electron microscopic random conical tilt sets

Single particle reconstruction using the random conical tilt data collection geometry is a robust method for the initial determination of macromolecular structures by electron microscopy. Unfortunately, the broad adoption of this powerful approach has been limited by the practical challenges inherent in manual data collection of the required pairs of matching high and low tilt images (typically 60 degrees and 0 degrees). The microscopist is obliged to keep the imaging area centered during tilting as well as to maintain accurate focus in the tilted image while minimizing the overall electron dose, a challenging and time consuming process. To help solve these problems, we have developed an automated system for the rapid acquisition of accurately aligned and focused tilt pairs. The system has been designed to minimize the dose incurred during alignment and focusing, making it useful in both negative stain and cryo-electron microscopy. The system includes a feature for montaging untilted images to ensure that all of the particles in the tilted image may be used in the reconstruction.     

An Innovative Method for Obtaining Consistent Images and Quantification of Histochemically Stained Specimens

Obtaining digital images of color brightfield microscopy is an important aspect of biomedical research and the clinical practice of diagnostic pathology. Although the field of digital pathology has had tremendous advances in whole-slide imaging systems, little effort has been directed toward standardizing color brightfield digital imaging to maintain image-to-image consistency and tonal linearity. Using a single camera and microscope to obtain digital images of three stains, we show that microscope and camera systems inherently produce image-to-image variation. Moreover, we demonstrate that post-processing with a widely used raster graphics editor software program does not completely correct for session-to-session inconsistency. We introduce a reliable method for creating consistent images with a hardware/software solution (ChromaCal™; Datacolor Inc., NJ) along with its features for creating color standardization, preserving linear tonal levels, providing automated white balancing and setting automated brightness to consistent levels. The resulting image consistency using this method will also streamline mean density and morphometry measurements, as images are easily segmented and single thresholds can be used. We suggest that this is a superior method for color brightfield imaging, which can be used for quantification and can be readily incorporated into workflows.     

Artificial neural network-aided image analysis system for cell counting.

BACKGROUND: In histological preparations containing debris and synthetic materials, it is difficult to automate cell counting using standard image analysis tools, i.e., systems that rely on boundary contours, histogram thresholding, etc. In an attempt to mimic manual cell recognition, an automated cell counter was constructed using a combination of artificial intelligence and standard image analysis methods. METHODS: Artificial neural network (ANN) methods were applied on digitized microscopy fields without pre-ANN feature extraction. A three-layer feed-forward network with extensive weight sharing in the first hidden layer was employed and trained on 1,830 examples using the error back-propagation algorithm on a Power Macintosh 7300/180 desktop computer. The optimal number of hidden neurons was determined and the trained system was validated by comparison with blinded human counts. System performance at 50x and lO0x magnification was evaluated. RESULTS: The correlation index at 100x magnification neared person-to-person variability, while 50x magnification was not useful. The system was approximately six times faster than an experienced human. CONCLUSIONS: ANN-based automated cell counting in noisy histological preparations is feasible. Consistent histology and computer power are crucial for system performance. The system provides several benefits, such as speed of analysis and consistency, and frees up personnel for other tasks.     

Quantification of fluorescence in situ hybridization signals by image cytometry.

In this study we aimed at the development of a cytometric system for quantification of specific DNA sequences using fluorescence in situ hybridization (ISH) and digital imaging microscopy. The cytochemical and cytometric aspects of a quantitative ISH procedure were investigated, using human peripheral blood lymphocyte interphase nuclei and probes detecting high copy number target sequences as a model system. These chromosome-specific probes were labeled with biotin, digoxigenin, or fluorescein. The instrumentation requirements are evaluated. Quantification of the fluorescence ISH signals was performed using an epi-fluorescence microscope with a multi-wavelength illuminator, equipped with a cooled charge couple device (CCD) camera. The performance of the system was evaluated using fluorescing beads and a homogeneously fluorescing specimen. Specific image analysis programs were developed for the automated segmentation and analysis of the images provided by ISH. Non-uniform background fluorescence of the nuclei introduces problems in the image analysis segmentation procedures. Different procedures were tested. Up to 95% of the hybridization signals could be correctly segmented using digital filtering techniques (min-max filter) to estimate local background intensities. The choice of the objective lens used for the collection of images was found to be extremely important. High magnification objectives with high numerical aperture, which are frequently used for visualization of fluorescence, are not optimal, since they do not have a sufficient depth of field. The system described was used for quantification of ISH signals and allowed accurate measurement of fluorescence spot intensities, as well as of fluorescence ratios obtained with double-labeled probes.     

A program system for interactive measurements on digitized cell images.

Using a high precision image scanner and a PDP-8/F minicomputer, we have developed a program system for interactive measurements on microscopic images. By giving simple keyboard commands, the operator can run the image scanner and manipulate the digitized images. The interface between the operator and the microscope-computer system is a Tektronix 4010 graphic terminal. The system allows objects to be isolated and parameters to be calculated from each object, e.g., parameters characterizing shape of the object, irregularity in light transmission over the object, area, integrated light transmission, etc. Objects are isolated and parameters are calculated under complete operator control using interactive computer graphics technique. Calculated parameters may be stored in dedicated data records, which are stored in files for later statistical analysis. The system also includes a statistical evaluation part. Technically, the system consists of a command scanner, which translates commands into internal representation, a parser, which checks the syntax of the commands, and an interpreter, which executes the commands. The system is designed so that new commands can be added easily.     

Grid computing in image analysis

Diagnostic surgical pathology or tissue–based diagnosis still remains the most reliable and specific diagnostic medical procedure. The development of whole slide scanners permits the creation of virtual slides and to work on so-called virtual microscopes. In addition to interactive work on virtual slides approaches have been reported that introduce automated virtual microscopy, which is composed of several tools focusing on quite different tasks. These include evaluation of image quality and image standardization, analysis of potential useful thresholds for object detection and identification (segmentation), dynamic segmentation procedures, adjustable magnification to optimize feature extraction, and texture analysis including image transformation and evaluation of elementary primitives. Grid technology seems to possess all features to efficiently target and control the specific tasks of image information and detection in order to obtain a detailed and accurate diagnosis. Grid technology is based upon so-called nodes that are linked together and share certain communication rules in using open standards. Their number and functionality can vary according to the needs of a specific user at a given point in time. When implementing automated virtual microscopy with Grid technology, all of the five different Grid functions have to be taken into account, namely 1) computation services, 2) data services, 3) application services, 4) information services, and 5) knowledge services. Although all mandatory tools of automated virtual microscopy can be implemented in a closed or standardized open system, Grid technology offers a new dimension to acquire, detect, classify, and distribute medical image information, and to assure quality in tissue–based diagnosis.     

An Image Skeletonization-Based Tool for Pollen Tube Morphology Analysis and Phenotyping

The mechanism underlying pollen tube growth involves diverse genes and molecular pathways. Alterations in the regulatory genes or pathways cause phenotypic changes reflected by cellular morphology, which can be captured using fluorescence microscopy. Determining and classifying pollen tube morphological phenotypes in such microscopic images is key to our understanding the involvement of genes and pathways. In this context, we propose a computational method to extract quantitative morphological features, and demonstrate that these features reflect morphological differences relevant to distinguish different defects of pollen tube growth. The corresponding software tool furthermore includes a novel semi-automated image segmentation approach, allowing to highly accurately identify the boundary of a pollen tube in a microscopic image.     

The rejection of noncellular artifacts in Papanicolaou-stained slide specimens by an automated high-resolution system. Identification of important cytometric features

The important cytodiagnostic features that permit discrimination of typical cell types by high-resolution image analysis and pattern recognition techniques have been previously studied in detail. An automated system for the diagnosis of Papanicolaou-stained specimens must also deal, however, with the "real world" of extraneous noncellular artifacts and debris found on every slide. Features that are ideal for the separation of typical normal and abnormal cells may not be adequate by themselves to reject these objects. A new set of discriminatory features must be found. In order to identify those features, a large set of images acquired using the TICAS high-resolution television rapid-scanning system was analyzed and studied. These images, from a variety of slide types, included normal cells, abnormal cells and noncellular artifacts identified by low-resolution preprocessing logic as suspicious enough to warrant high-resolution study. The results indicate that the more important features for such discrimination are not those traditionally important in distinguishing abnormal from normal cells but include color relations, shape measures, boundary properties and texture features.     

Basic aspects of and recent developments in telepathology in Europe, with specific emphasis on quality assurance

Telepathology is the diagnostic work of a pathologist from a distance and includes all specific fields of diagnostic pathology, such as frozen section services, expert consultation, cytometric and histometric measurement, and continuous education. For about 15 years experience has been collected at several universities in the United States and Europe based upon analog telephone lines (9.2 kbaud), digitized lines (ISDN, 64 kbaud), broad band connections (1.5 Mbaud) and the World Wide Web (28 kbaud). Potential use can be expected in the application of telepresentation, remote slide preparation, remote central diagnostics and telediscussion. The transfer of still images is well developed; that of live images is used in only a few institutions for frozen section services. The image quality and spatial resolution as well as the transfer speed are sufficient for expert consultations, morphometric measurements, quality assurance and education. All applications focus on discontinuous work flow. Although the European Community focuses on user needs and standardization aspects of telepathology by sponsoring a widespread telepathology project (Europath), implementation of telepathology into routine application in the continuous work flow has still to be developed. The technical equipment has still to be adjusted to the labor flow charts in routine pathologic diagnostic procedures. Telepathology seems to be the appropriate technique to offer both improvement in diagnostic quality and inclusion of the "control institution" into diagnostic responsibility.     

Image analysis of low magnification images of fine needle aspirates of the breast produces useful discrimination between benign and malignant cases

Image analysis of low magnification images of fine needle aspirates of the breast produces useful discrimination between benign and malignant cases
Fine needle aspirates of the breast (FNAB) ( n =362; 204 malignant, 158 benign), prepared by cytocentrifuge methods and stained by the Papanicolaou technique, were analysed using a semi‐automated image analysis system at a low magnification which precluded resolution of nuclear detail. The measured parameters were integrated optical density, fractal textural dimension, number of cellular objects (single cells and contiguous groups of cells), distance between cellular objects (mean, s.d., skewness and kurtosis), area of cellular objects (mean, s.d., skewness, kurtosis) and the nearest neighbour statistic. The cases were divided into a 200‐case training set and a 162‐case test set. Analysis was performed by logistic regression and the multi‐layer Perceptron type of artificial neural network. Logistic regression and the neural network produced similar performances with a sensitivity of 82–83%, specificity 85% and a positive predictive value for a malignant result of 85%. A non‐parametric analysis of all the predictor variables showed that all except the mean area of cellular objects and the s.d. of this measurement were significant discriminants ( P <0.05), but most were highly interrelated and this was reflected in the selection of only three predictor variables by forward and backward conditional logistic regression. This study shows that much diagnostic information is present in low power views of FNAB, and that image analysis could form the basis of a semi‐automated decision‐support aid.     

Analysis of gene expression levels in individual bacterial cells without image segmentation

Studies of stochasticity in gene expression typically make use of fluorescent protein reporters, which permit the measurement of expression levels within individual cells by fluorescence microscopy. Analysis of such microscopy images is almost invariably based on a segmentation algorithm, where the image of a cell or cluster is analyzed mathematically to delineate individual cell boundaries. However segmentation can be ineffective for studying bacterial cells or clusters, especially at lower magnification, where outlines of individual cells are poorly resolved. Here we demonstrate an alternative method for analyzing such images without segmentation. The method employs a comparison between the pixel brightness in phase contrast vs fluorescence microscopy images. By fitting the correlation between phase contrast and fluorescence intensity to a physical model, we obtain well-defined estimates for the different levels of gene expression that are present in the cell or cluster. The method reveals the boundaries of the individual cells, even if the source images lack the resolution to show these boundaries clearly.     

Adaptive color basis transformation. An aid in image segmentation

Multispectral images of stained cells enable the use of color differences to segment and/or to discriminate between image components, such as cell types and cellular subcomponents. When the spectral characteristics of the image components do not change over the area of a slide or from slide to slide, one can create a constant weighted linear combination of spectral images to generate one-dimensional or two-dimensional images that have the desired contrast between the image components that must be discriminated. However, when the spectral characteristics are not constant, i.e., when they vary from image to image, a constant weighted linear combination cannot be employed; instead, an appropriate solution must be found for each selected image. This is usually a time-consuming, manual procedure that cannot be employed in a fully automated process of discriminating and segmenting stained cells. This paper describes an algorithm that uses principal components decomposition basis vectors to generate a nonstatic weighted linear combination of color images that can be used by an automated system. This algorithm relies on a semiconstant relationship between the areas (sizes) of the image components that are to be discriminated and/or segmented. The technique has been successfully applied as an aid in the segmentation of images of stained cervical smears; the images were acquired with a three-chip CCD camera that generates three broad-band color images.     

Measuring beyond the microscope field of view using digital images in squamous cell carcinoma of the lip

OBJECTIVE: To study the impact of using digital images for measuring the size of the tumors, assisting with the prognostic evaluation of carcinomas of the oral mucosa. STUDY DESIGN: The depth of invasion of 12 cases of squamous cell carcinoma of the lower lip was assessed through a microscope and through digital images. All measurements of depth of invasion were assessed in a direction orthogonal to the lip surface. First, assessment of depth of invasion was done at the microscope, using an eyepiece reticule with an engraved scale. Second, depth of invasion was assessed by digital images, using a program module developed to assist pathologists with linear measuring. When the depth of the tumor was larger than the field of view at the proper magnification, several images were taken to include the whole area of invasion. The images were finally mounted in a single image and the depth of invasion measured. RESULTS: The results show positive and negative differences between assessments when assessing depths of < 2 mm. At greater depths (> or = 2 mm), the difference was always negative, showing that for deep invasion, measurements of longer distances were always performed on digital images. CONCLUSION: Measurements with digital images beyond the field of view at proper magnification could sig nificantly alter the diagnostic and prognostic assessment made using the microscope.