Mobile Phone Based Clinical Microscopy for Global Health Applications

Light microscopy provides a simple, cost-effective, and vital method for the diagnosis and screening of hematologic and infectious diseases. In many regions of the world, however, the required equipment is either unavailable or insufficiently portable, and operators may not possess adequate training to make full use of the images obtained. Counterintuitively, these same regions are often well served by mobile phone networks, suggesting the possibility of leveraging portable, camera-enabled mobile phones for diagnostic imaging and telemedicine. Toward this end we have built a mobile phone-mounted light microscope and demonstrated its potential for clinical use by imaging P. falciparum-infected and sickle red blood cells in brightfield and M. tuberculosis-infected sputum samples in fluorescence with LED excitation. In all cases resolution exceeded that necessary to detect blood cell and microorganism morphology, and with the tuberculosis samples we took further advantage of the digitized images to demonstrate automated bacillus counting via image analysis software. We expect such a telemedicine system for global healthcare via mobile phone – offering inexpensive brightfield and fluorescence microscopy integrated with automated image analysis – to provide an important tool for disease diagnosis and screening, particularly in the developing world and rural areas where laboratory facilities are scarce but mobile phone infrastructure is extensive.     

Multi-Contrast Imaging and Digital Refocusing on a Mobile Microscope with a Domed LED Array

We demonstrate the design and application of an add-on device for improving the diagnostic and research capabilities of CellScope—a low-cost, smartphone-based point-of-care microscope. We replace the single LED illumination of the original CellScope with a programmable domed LED array. By leveraging recent advances in computational illumination, this new device enables simultaneous multi-contrast imaging with brightfield, darkfield, and phase imaging modes. Further, we scan through illumination angles to capture lightfield datasets, which can be used to recover 3D intensity and phase images without any hardware changes. This digital refocusing procedure can be used for either 3D imaging or software-only focus correction, reducing the need for precise mechanical focusing during field experiments. All acquisition and processing is performed on the mobile phone and controlled through a smartphone application, making the computational microscope compact and portable. Using multiple samples and different objective magnifications, we demonstrate that the performance of our device is comparable to that of a commercial microscope. This unique device platform extends the field imaging capabilities of CellScope, opening up new clinical and research possibilities.     

Automated selection of DAB-labeled tissue for immunohistochemical quantification.

The increased use of immunohistochemistry (IHC) in both clinical and basic research settings has led to the development of techniques for acquiring quantitative information from immunostains. Staining correlates with absolute protein levels and has been investigated as a clinical tool for patient diagnosis and prognosis. For these reasons, automated imaging methods have been developed in an attempt to standardize IHC analysis. We propose a novel imaging technique in which brightfield images of diaminobenzidene (DAB)-labeled antigens are converted to normalized blue images, allowing automated identification of positively stained tissue. A statistical analysis compared our method with seven previously published imaging techniques by measuring each one's agreement with manual analysis by two observers. Eighteen DAB-stained images showing a range of protein levels were used. Accuracy was assessed by calculating the percentage of pixels misclassified using each technique compared with a manual standard. Bland-Altman analysis was then used to show the extent to which misclassification affected staining quantification. Many of the techniques were inconsistent in classifying DAB staining due to background interference, but our method was statistically the most accurate and consistent across all staining levels.     

Analysis of stained objects in histological sections by spectral imaging and differential absorption.

We describe a new light microscopic imaging system and method to perform high through put color image analysis on histological tissue sections. The system features a computer-controlled, random-access liquid crystal tunable filter and high-resolution digital camera on a conventional brightfield microscope. For any combination of stains, the method determines the spectral transmittance of each stain on the slide and selects two or more wavelengths at which the differential absorption between stain and counterstain is greatest and the exposure time is reasonably short. Flatfield corrected digital images at these wavelengths are acquired and divided to produce a gray scale ratio image. The ratio image is calculated such that the stained features of interest are highlighted above a uniform background and the counterstained features are highlighted below background. Image threshold procedures using either visual inspection or a threshold value determined by the image mean intensity and standard deviation are used to segment the stained features of interest for subsequent morphometry. Results are presented for peroxidase-AEC-labeled tumor tissue and trichrome-stained biomaterial implant tissues. In principle, the method should work for any combination of colored stains. (J Histochem Cytochem 47:1307-1313, 1999)     

Combined confocal and wide-field high-resolution cytometry of fluorescent in situ hybridization-stained cells

BACKGROUND: The recently developed technique of high-resolution cytometry (HRCM) enables automated acquisition and analysis of fluorescent in situ hybridization (FISH)-stained cell nuclei using conventional wide-field fluorescence microscopy. The method has now been extended to confocal imaging and offers the opportunity to combine the advantages of confocal and wide-field modes. METHODS: We have automated image acquisition and analysis from a standard inverted fluorescence microscope equipped with a confocal module with Nipkow disk and a cooled digital CCD camera. The system is fully controlled by a high-performance computer that performs both acquisition and related on-line image analysis. The system can be used either for an automatic two (2D) and three-dimensional (3D) analysis of FISH- stained interphase nuclei or for a semiautomatic 3D analysis of FISH-stained cells in tissues. The user can select which fluorochromes are acquired using wide-field mode and which using confocal mode. The wide-field and confocal images are overlaid automatically in computer memory. The developed software compensates automatically for both chromatic color shifts and spatial shifts caused by switching to a different imaging mode. RESULTS: Using the combined confocal and wide-field HRCM technique, it is possible to take advantage of both imaging modes. Images of some dyes (such as small hybridization dots or counterstain images of individual interphase nuclei) do not require confocal quality and can be acquired quickly in wide-field mode. On the contrary, images of other dyes (such as chromosome territories or counterstain images of cells in tissues) do require improved quality and are acquired in confocal mode. The dual-mode approach is two to three times faster compared with the single-mode confocal approach and the spectrum of its applications is much broader compared with both single-mode confocal and single-mode wide-field systems. CONCLUSIONS: The combination of high speed specific to the wide-field mode and high quality specific to the confocal mode gives optimal system performance.     

Color standardization and optimization in Whole Slide Imaging

Introduction

Standardization and validation of the color displayed by digital slides is an important aspect of digital pathology implementation. While the most common reason for color variation is the variance in the protocols and practices in the histology lab, the color displayed can also be affected by variation in capture parameters (for example, illumination and filters), image processing and display factors in the digital systems themselves.

Method

We have been developing techniques for color validation and optimization along two paths. The first was based on two standard slides that are scanned and displayed by the imaging system in question. In this approach, one slide is embedded with nine filters with colors selected especially for H&;E stained slides (looking like tiny Macbeth color chart); the specific color of the nine filters were determined in our previous study and modified for whole slide imaging (WSI). The other slide is an H&;E stained mouse embryo. Both of these slides were scanned and the displayed images were compared to a standard. The second approach was based on our previous multispectral imaging research.

Discussion

As a first step, the two slide method (above) was used to identify inaccurate display of color and its cause, and to understand the importance of accurate color in digital pathology. We have also improved the multispectral-based algorithm for more consistent results in stain standardization. In near future, the results of the two slide and multispectral techniques can be combined and will be widely available.We have been conducting a series of researches and developing projects to improve image quality to establish Image Quality Standardization. This paper discusses one of most important aspects of image quality – color.

     

Unsupervised organization of cervical cells using bright‐field and single‐shot digital holographic microscopy

We report results on unsupervised organization of cervical cells using microscopy of Pap‐smear samples in brightfield (3‐channel color) as well as high‐resolution quantitative phase imaging modalities. A number of morphological parameters are measured for each of the 1450 cell nuclei (from 10 woman subjects) imaged in this study. The principal component analysis (PCA) methodology applied to this data shows that the cell image clustering performance improves significantly when brightfield as well as phase information is utilized for PCA as compared to when brightfield‐only information is used. The results point to the feasibility of an image‐based tool that will be able to mark suspicious cells for further examination by the pathologist. More importantly, our results suggest that the information in quantitative phase images of cells that is typically not used in clinical practice is valuable for automated cell classification applications in general.      

An update on the validation of whole slide imaging systems following FDA approval of a system for a routine pathology diagnostic service in the United States

Pathologists have used light microscopes and glass slides to interpret the histologic appearance of normal and diseased tissues for more than 150 years. The quality of both microtomes used to cut tissue sections and microscopes has improved significantly during the past few decades, but the process of rendering diagnoses has changed little. By contrast, major advances in digital technology have occurred since the introduction of hand held electronic devices, including the development of whole slide imaging (WSI) systems with software packages that can convert microscope images into virtual (digital) slides that can be viewed on computer monitors and via the internet. To date, however, these technological developments have had minimal impact on the way pathologists perform their daily work, with the exception of using computers to access electronic medical records and scholarly web sites for pertinent information to assist interpretation of cases. Traditional practice is likely to change significantly during the next decade, especially since the Federal Drug Administration in the USA has approved the first WSI system for routine diagnostic practice. I review here the development and slow acceptance of WSI by pathology departments. I focus on recent advances in validation of WSI systems that is required for routine diagnostic reporting of pathology cases using this technology.     

A High-Throughput Automated Technique for Counting Females of Heterodera glycines using a Fluorescence-Based Imaging System

The soybean cyst nematode (SCN), Heterodera glycines, is the most damaging pathogen of soybean. Methods to phenotype soybean varieties for resistance to SCN are currently very laborious and time consuming. Streamlining a portion of this phenotyping process could increase productivity and accuracy. Here we report an automated method to count SCN females using a fluorescence-based imaging system that is well suited to high-throughput SCN phenotyping methods used in greenhouse screening. For optimal automated imaging, females were washed from roots at 30 days post-inoculation into small Petri dishes. Using a Kodak Image Station 4000MM Pro, the Petri dishes were scanned using excitation and emission wavelengths of 470 nm and 535 nm, respectively. Fluorescent images were captured and analyzed with Carestream Molecular Imaging Software for automated counting. We demonstrate that the automated fluorescent-based imaging system is just as accurate (r(2) ≥ 0.95) and more efficient (>50% faster) than manual counting under a microscope. This method can greatly improve the consistency and turnaround of data while reducing the time and labor commitment associated with SCN female counting.     

Computer‐automated bird detection and counts in high‐resolution aerial images: a review

Bird surveys conducted using aerial images can be more accurate than those using airborne observers, but can also be more time‐consuming if images must be analyzed manually. Recent advances in digital cameras and image‐analysis software offer unprecedented potential for computer‐automated bird detection and counts in high‐resolution aerial images. We review the literature on this subject and provide an overview of the main image‐analysis techniques. Birds that contrast sharply with image backgrounds (e.g., bright birds on dark ground) are generally the most amenable to automated detection, in some cases requiring only basic image‐analysis software. However, the sophisticated analysis capabilities of modern object‐based image analysis software provide ways to detect birds in more challenging situations based on a variety of attributes including color, size, shape, texture, and spatial context. Some techniques developed to detect mammals may also be applicable to birds, although the prevalent use of aerial thermal‐infrared images for detecting large mammals is of limited applicability to birds because of the low pixel resolution of thermal cameras and the smaller size of birds. However, the increasingly high resolution of true‐color cameras and availability of small unmanned aircraft systems (drones) that can fly at very low altitude now make it feasible to detect even small shorebirds in aerial images. Continued advances in camera and drone technology, in combination with increasingly sophisticated image analysis software, now make it possible for investigators involved in monitoring bird populations to save time and resources by increasing their use of automated bird detection and counts in aerial images. We recommend close collaboration between wildlife‐monitoring practitioners and experts in the fields of remote sensing and computer science to help generate relevant, accessible, and readily applicable computer‐automated aerial photographic census techniques.     

Image-based plant phenotyping with incremental learning and active contours

Plant phenotyping investigates how a plant's genome, interacting with the environment, affects the observable traits of a plant (phenome). It is becoming increasingly important in our quest towards efficient and sustainable agriculture. While sequencing the genome is becoming increasingly efficient, acquiring phenotype information has remained largely of low throughput. Current solutions for automated image-based plant phenotyping, rely either on semi-automated or manual analysis of the imaging data, or on expensive and proprietary software which accompanies costly hardware infrastructure. While some attempts have been made to create software applications that enable the analysis of such images in an automated fashion, most solutions are tailored to particular acquisition scenarios and restrictions on experimental design. In this paper we propose and test, a method for the segmentation and the automated analysis of time-lapse plant images from phenotyping experiments in a general laboratory setting, that can adapt to scene variability. The method involves minimal user interaction, necessary to establish the statistical experiments that may follow. At every time instance (i.e., a digital photograph), it segments the plants in images that contain many specimens of the same species. For accurate plant segmentation we propose a vector valued level set formulation that incorporates features of color intensity, local texture, and prior knowledge. Prior knowledge is incorporated using a plant appearance model implemented with Gaussian mixture models, which utilizes incrementally information from previously segmented instances. The proposed approach is tested on Arabidopsis plant images acquired with a static camera capturing many subjects at the same time. Our validation with ground truth segmentations and comparisons with state-of-the-art methods in the literature shows that the proposed method is able to handle images with complicated and changing background in an automated fashion. An accuracy of 96.7% (dice similarity coefficient) was observed, which was higher than other methods used for comparison. While here it was tested on a single plant species, the fact that we do not employ shape driven models and we do not rely on fully supervised classification (trained on a large dataset) increases the ease of deployment of the proposed solution for the study of different plant species in a variety of laboratory settings. Our solution will be accompanied by an easy to use graphical user interface and, to facilitate adoption, we will make the software available to the scientific community.     

Sequential Superresolution Imaging of Multiple Targets Using a Single Fluorophore

Fluorescence superresolution (SR) microscopy, or fluorescence nanoscopy, provides nanometer scale detail of cellular structures and allows for imaging of biological processes at the molecular level. Specific SR imaging methods, such as localization-based imaging, rely on stochastic transitions between on (fluorescent) and off (dark) states of fluorophores. Imaging multiple cellular structures using multi-color imaging is complicated and limited by the differing properties of various organic dyes including their fluorescent state duty cycle, photons per switching event, number of fluorescent cycles before irreversible photobleaching, and overall sensitivity to buffer conditions. In addition, multiple color imaging requires consideration of multiple optical paths or chromatic aberration that can lead to differential aberrations that are important at the nanometer scale. Here, we report a method for sequential labeling and imaging that allows for SR imaging of multiple targets using a single fluorophore with negligible cross-talk between images. Using brightfield image correlation to register and overlay multiple image acquisitions with ~10 nm overlay precision in the x-y imaging plane, we have exploited the optimal properties of AlexaFluor647 for dSTORM to image four distinct cellular proteins. We also visualize the changes in co-localization of the epidermal growth factor (EGF) receptor and clathrin upon EGF addition that are consistent with clathrin-mediated endocytosis. These results are the first to demonstrate sequential SR (s-SR) imaging using direct stochastic reconstruction microscopy (dSTORM), and this method for sequential imaging can be applied to any superresolution technique.     

Accurate color imaging of pathology slides using holography and absorbance spectrum estimation of histochemical stains

Holographic microscopy presents challenges for color reproduction due to the usage of narrow‐band illumination sources, which especially impacts the imaging of stained pathology slides for clinical diagnoses. Here, an accurate color holographic microscopy framework using absorbance spectrum estimation is presented. This method uses multispectral holographic images acquired and reconstructed at a small number (e.g., three to six) of wavelengths, estimates the absorbance spectrum of the sample, and projects it onto a color tristimulus. Using this method, the wavelength selection is optimized to holographically image 25 pathology slide samples with different tissue and stain combinations to significantly reduce color errors in the final reconstructed images. The results can be used as a practical guide for various imaging applications and, in particular, to correct color distortions in holographic imaging of pathology samples spanning different dyes and tissue types.      

COST Action “EuroTelepath”: digital pathology integration in electronic health record,including primary care centres

INTRODUCTION: Digital pathology includes the information technology that allows for the management of information, including data and images, generated in an anatomic pathology department. COST ACTION IC0604: The integration of digital slides in the electronic health record is one of the main objectives of COST Action IC0604 "Telepathology Network in Europe" (EURO-TELEPATH). Fostering use of medical informatics standards and adapting them to current needs is needed to manage efficiently extremely large medical images, like digital slide files. DIGITAL SLIDES IN PATHOLOGY: Digital slides can play a role in disease prevention, primary diagnosis, and second opinion. In all these tasks, automated image analysis can also be a most valuable tool. INTEROPERABILITY IN PATHOLOGY INFORMATION SYSTEMS: In order to achieve an efficient interoperability between pathology information systems with other clinical information systems, obtaining a seamless integration of pathology images (gross pictures and digital slides) with LIS-Pathology Information system in a web environment is an important task. Primary care information systems should also be included in the integration, since primary care centres play an essential role in the generation of clinical information and specimen collection. A common terminology, based in SNOMED CT is also needed. CONCLUSIONS: Main barrier in the integration of digital slides in pathology workflow and eHealth record is the cost of current digital slide scanners. Pathology information system vendors should participate in standardization bodies.     

A graphical model approach to automated classification of protein subcellular location patterns in multi-cell images

Background  

Knowledge of the subcellular location of a protein is critical to understanding how that protein works in a cell. This location is frequently determined by the interpretation of fluorescence microscope images. In recent years, automated systems have been developed for consistent and objective interpretation of such images so that the protein pattern in a single cell can be assigned to a known location category. While these systems perform with nearly perfect accuracy for single cell images of all major subcellular structures, their ability to distinguish subpatterns of an organelle (such as two Golgi proteins) is not perfect. Our goal in the work described here was to improve the ability of an automated system to decide which of two similar patterns is present in a field of cells by considering more than one cell at a time. Since cells displaying the same location pattern are often clustered together, considering multiple cells may be expected to improve discrimination between similar patterns.     

Quantitative Imaging with a Mobile Phone Microscope

Use of optical imaging for medical and scientific applications requires accurate quantification of features such as object size, color, and brightness. High pixel density cameras available on modern mobile phones have made photography simple and convenient for consumer applications; however, the camera hardware and software that enables this simplicity can present a barrier to accurate quantification of image data. This issue is exacerbated by automated settings, proprietary image processing algorithms, rapid phone evolution, and the diversity of manufacturers. If mobile phone cameras are to live up to their potential to increase access to healthcare in low-resource settings, limitations of mobile phone–based imaging must be fully understood and addressed with procedures that minimize their effects on image quantification. Here we focus on microscopic optical imaging using a custom mobile phone microscope that is compatible with phones from multiple manufacturers. We demonstrate that quantitative microscopy with micron-scale spatial resolution can be carried out with multiple phones and that image linearity, distortion, and color can be corrected as needed. Using all versions of the iPhone and a selection of Android phones released between 2007 and 2012, we show that phones with greater than 5 MP are capable of nearly diffraction-limited resolution over a broad range of magnifications, including those relevant for single cell imaging. We find that automatic focus, exposure, and color gain standard on mobile phones can degrade image resolution and reduce accuracy of color capture if uncorrected, and we devise procedures to avoid these barriers to quantitative imaging. By accommodating the differences between mobile phone cameras and the scientific cameras, mobile phone microscopes can be reliably used to increase access to quantitative imaging for a variety of medical and scientific applications.     

Computerized image analysis as a tool to quantify infiltrating leukocytes: a comparison between high- and low-magnification images.

The purpose of the present study was to establish a rapid and reproducible method for quantification of tissue-infiltrating leukocytes using computerized image analysis. To achieve this, the staining procedure, the image acquisition, and the image analysis method were optimized. Because of the adaptive features of the human eye, computerized image analysis is more sensitive to variations in staining compared with manual image analysis. To minimize variations in staining, an automated immunostainer was used. With a digital scanner camera, low-magnification images could be sampled at high resolution, thus making it possible to analyze larger tissue sections. Image analysis was performed by color thresholding of the digital images based on values of hue, saturation, and intensity color mode, which we consider superior to the red, green, and blue color mode for analysis of most histological stains. To evaluate the method, we compared computerized analysis of images with a x100 or a x12.5 magnification to assess leukocytes infiltrating rat brain tumors after peripheral immunizations with tumor cells genetically modified to express rat interferon-gamma (IFN-gamma) or medium controls. The results generated by both methods correlated well and did not show any significant differences. The method allows efficient and reproducible processing of large tissue sections that is less time-consuming than conventional methods and can be performed with standard equipment and software.(J Histochem Cytochem 49:1073-1079, 2001)     

High-speed multicolor microscopy of repeating dynamic processes

Images of multiply labeled fluorescent samples provide unique insights into the localization of molecules, cells, and tissues. The ability to image multiple channels simultaneously at high speed without cross talk is limited to a few colors and requires dedicated multichannel or multispectral detection procedures. Simpler microscopes, in which each color is imaged sequentially, produce a much lower frame rate. Here, we describe a technique to image, at high frame rate, multiply labeled samples that have a repeating motion. We capture images in a single channel at a time over one full occurrence of the motion then repeat acquisition for other channels over subsequent occurrences. We finally build a high-speed multichannel image sequence by combining the images after applying a normalized mutual information-based time registration procedure. We show that this technique is amenable to image the beating heart of a double-labeled embryonic quail in three channels (brightfield, yellow, and mCherry fluorescent proteins) using a fluorescence wide-field microscope equipped with a single monochrome camera and without fast channel switching optics. We experimentally evaluate the accuracy of our method on image series from a two-channel confocal microscope.     

Automated molecular microscopy: the new Leginon system

We report here on the current state of our efforts in automated molecular microscopy. Our primary automated data acquisition software system, Leginon, has been completely redesigned over the past two years. The new distributed system has been developed using the Python programming language and is compatible with both Linux and Windows operating systems. The new flexible architecture was designed to allow for the development of customized data collection protocols, several of which are described here. The system has been used to acquire data for approximately 150 experiments and we have demonstrated the capacity for high throughput data acquisition by acquiring images of more than 100,000 particles in a single session at the microscope.     

Accuracy of a remote quantitative image analysis in the whole slide images

The rationale for choosing a remote quantitative method supporting a diagnostic decision requires some empirical studies and knowledge on scenarios including valid telepathology standards. The tumours of the central nervous system [CNS] are graded on the base of the morphological features and the Ki-67 labelling Index [Ki-67 LI]. Various methods have been applied for Ki-67 LI estimation. Recently we have introduced the Computerized Analysis of Medical Images [CAMI] software for an automated Ki-67 LI counting in the digital images. Aims of our study was to explore the accuracy and reliability of a remote assessment of Ki-67 LI with CAMI software applied to the whole slide images [WSI]. The WSI representing CNS tumours: 18 meningiomas and 10 oligodendrogliomas were stored on the server of the Warsaw University of Technology. The digital copies of entire glass slides were created automatically by the Aperio ScanScope CS with objective 20x or 40x. Aperio's Image Scope software provided functionality for a remote viewing of WSI. The Ki-67 LI assessment was carried on within 2 out of 20 selected fields of view (objective 40x) representing the highest labelling areas in each WSI. The Ki-67 LI counting was performed by 3 various methods: 1) the manual reading in the light microscope - LM, 2) the automated counting with CAMI software on the digital images - DI , and 3) the remote quantitation on the WSIs - as WSI method. The quality of WSIs and technical efficiency of the on-line system were analysed. The comparative statistical analysis was performed for the results obtained by 3 methods of Ki-67 LI counting. The preliminary analysis showed that in 18% of WSI the results of Ki-67 LI differed from those obtained in other 2 methods of counting when the quality of the glass slides was below the standard range. The results of our investigations indicate that the remote automated Ki-67 LI analysis performed with the CAMI algorithm on the whole slide images of meningiomas and oligodendrogliomas could be successfully used as an alternative method to the manual reading as well as to the digital images quantitation with CAMI software. According to our observation a need of a remote supervision/consultation and training for the effective use of remote quantitative analysis of WSI is necessary.