Closing the translation gap: AI applications in digital pathology

Recent advances in artificial intelligence show tremendous promise to improve the accuracy, reproducibility, and availability of medical diagnostics across a number of medical subspecialities. This is especially true in the field of digital pathology, which has recently witnessed a surge in publications describing state-of-the-art performance for machine learning models across a wide range of diagnostic applications. Nonetheless, despite this promise, there remain significant gaps in translating applications for any of these technologies into actual clinical practice. In this review, we will first give a brief overview of the recent progress in applying AI to digitized pathology images, focusing on how these tools might be applied in clinical workflows in the near term to improve the accuracy and efficiency of pathologists. Then we define and describe in detail the various factors that need to be addressed in order to successfully close the “translation gap” for AI applications in digital pathology.     

An Innovative Method for Obtaining Consistent Images and Quantification of Histochemically Stained Specimens

Obtaining digital images of color brightfield microscopy is an important aspect of biomedical research and the clinical practice of diagnostic pathology. Although the field of digital pathology has had tremendous advances in whole-slide imaging systems, little effort has been directed toward standardizing color brightfield digital imaging to maintain image-to-image consistency and tonal linearity. Using a single camera and microscope to obtain digital images of three stains, we show that microscope and camera systems inherently produce image-to-image variation. Moreover, we demonstrate that post-processing with a widely used raster graphics editor software program does not completely correct for session-to-session inconsistency. We introduce a reliable method for creating consistent images with a hardware/software solution (ChromaCal™; Datacolor Inc., NJ) along with its features for creating color standardization, preserving linear tonal levels, providing automated white balancing and setting automated brightness to consistent levels. The resulting image consistency using this method will also streamline mean density and morphometry measurements, as images are easily segmented and single thresholds can be used. We suggest that this is a superior method for color brightfield imaging, which can be used for quantification and can be readily incorporated into workflows.     

Expanding application of digital pathology in Japan - from education,telepathology to autodiagnosis

Background

Digital pathology, i.e., applications of digital information technologies to pathology practice, has been expanding in the recent decades and the mode of pathology diagnostic practice is changing with enhanced precision. In the present study the changing processes of digital pathology in Japan were investigated and trends to future were discussed.

Methods

The changing status of digital pathology was investigated through reviewing the records of annual meetings of the Japanese Research Society of Telepathology and Pathology Informatics (JRST-PI) and of the Japanese pathology related medical and informatics journals. The results of the Japanese questionnaire survey conducted in 2008-2009 on telepathology and virtual slide were also reviewed. In addition effectiveness of an experimental automatic pathology diagnostic aid system using computer artificial intelligence was investigated by checking its rate of correct diagnosis for given prostate carcinoma digital images.

Results

Telepathology played a central role in the development of digital pathology in Japan. Both macroscopic and microscopic pathology digital images were routinely generated and used for diagnostic purposes in major hospitals. Virtual slide (VS) digital images were used first for education then for conference, consultation and also gradually for routine diagnosis and telepathology. The experimental automatic diagnostic aid system achieved the rate of correct diagnosis around 95% for prostate carcinoma and its use for automatic mapping of cancerous areas in a given tissue image was successful.

Conclusions

Advance in the digital information technologies gave revolutionary impacts on pathology education, conference, consultation, diagnosis, telepathology and also on pathology diagnostic procedures in Japan. The future will be bright for pathologists by the advanced digital pathology but we should pay attention to make the technologies and their effects under our control.

     

Portable telepathology: methods and tools

Telepathology is becoming easier to implement in most pathology departments. In fact e-mail image transmit can be done from almost any pathologist as a simplistic telepathology system. We tried to develop a way to improve capabilities of communication among pathologists with the idea that the system should be affordable for everybody. We took the premise that any pathology department would have microscopes and computers with Internet connection, and selected a few elements to convert them into a telepathology station. Needs were reduced to a camera to collect images, a universal microscope adapter for the camera, a device to connect the camera to the computer, and a software for the remote image transmit. We found out a microscope adapter (MaxView Plus) that allowed us connect almost any domestic digital camera to any microscope. The video out signal from the camera was sent to the computer through an Aver Media USB connector. At last, we selected a group of portable applications that were assembled into a USB memory device. Portable applications are computer programs that can be carried generally on USB flash drives, but also in any other portable device, and used on any (Windows) computer without installation. Besides, when unplugging the device, none of personal data is left behind. We selected open-source applications, and based the pathology image transmission to VLC Media Player due to its functionality as streaming server, portability and ease of use and configuration. Audio transmission was usually done through normal phone lines. We also employed alternative videoconferencing software, SightSpeed for bi-directional image transmission from microscopes, and conventional cameras allowing visual communication and also image transmit from gross pathology specimens. All these elements allowed us to install and use a telepathology system in a few minutes, fully prepared for real time image broadcast.     

SurfaceSlide: a multitouch digital pathology platform

Background

Digital pathology provides a digital environment for the management and interpretation of pathological images and associated data. It is becoming increasing popular to use modern computer based tools and applications in pathological education, tissue based research and clinical diagnosis. Uptake of this new technology is stymied by its single user orientation and its prerequisite and cumbersome combination of mouse and keyboard for navigation and annotation.

Methodology

In this study we developed SurfaceSlide, a dedicated viewing platform which enables the navigation and annotation of gigapixel digitised pathological images using fingertip touch. SurfaceSlide was developed using the Microsoft Surface, a 30 inch multitouch tabletop computing platform. SurfaceSlide users can perform direct panning and zooming operations on digitised slide images. These images are downloaded onto the Microsoft Surface platform from a remote server on-demand. Users can also draw annotations and key in texts using an on-screen virtual keyboard. We also developed a smart caching protocol which caches the surrounding regions of a field of view in multi-resolutions thus providing a smooth and vivid user experience and reducing the delay for image downloading from the internet. We compared the usability of SurfaceSlide against Aperio ImageScope and PathXL online viewer.

Conclusion

SurfaceSlide is intuitive, fast and easy to use. SurfaceSlide represents the most direct, effective and intimate human–digital slide interaction experience. It is expected that SurfaceSlide will significantly enhance digital pathology tools and applications in education and clinical practice.     

Digital microscopy in phycological research, with special reference to microalgae

Digital imaging technology is gradually being incorporated into all areas of biological research, but there is a distinct lack of information resources targeted at scientists in their specialist areas. There is a wealth of potential applications for digital images in phycology, including morphometric or visual analysis of specimens, taxonomic databases and publication of digital micrographs in lieu of photomicrographs. Here, we provide a review of digital imaging in general and its potential for the field of microalgal research in particular. We also present a number of imaging techniques that are critical for image acquisition and optimization, which can enable beginners to build their own libraries of high quality digital images. Resolution requirements of digital cameras are explained and related to microscope resolution. The benefits of digital imaging technology are discussed and contrasted with those of traditional silver halide technology.     

COST Action “EuroTelepath”: digital pathology integration in electronic health record,including primary care centres

INTRODUCTION: Digital pathology includes the information technology that allows for the management of information, including data and images, generated in an anatomic pathology department. COST ACTION IC0604: The integration of digital slides in the electronic health record is one of the main objectives of COST Action IC0604 "Telepathology Network in Europe" (EURO-TELEPATH). Fostering use of medical informatics standards and adapting them to current needs is needed to manage efficiently extremely large medical images, like digital slide files. DIGITAL SLIDES IN PATHOLOGY: Digital slides can play a role in disease prevention, primary diagnosis, and second opinion. In all these tasks, automated image analysis can also be a most valuable tool. INTEROPERABILITY IN PATHOLOGY INFORMATION SYSTEMS: In order to achieve an efficient interoperability between pathology information systems with other clinical information systems, obtaining a seamless integration of pathology images (gross pictures and digital slides) with LIS-Pathology Information system in a web environment is an important task. Primary care information systems should also be included in the integration, since primary care centres play an essential role in the generation of clinical information and specimen collection. A common terminology, based in SNOMED CT is also needed. CONCLUSIONS: Main barrier in the integration of digital slides in pathology workflow and eHealth record is the cost of current digital slide scanners. Pathology information system vendors should participate in standardization bodies.     

Applications of computer vision and machine learning techniques for digitized herbarium specimens: A systematic literature review

Herbaria contain the treasure of millions of specimens that have been preserved for several years for scientific studies. To increase the rate of scientific discoveries, digitization of these specimens is currently ongoing to facilitate the easy access and sharing of data to a wider scientific community. Online digital repositories such as Integrated Digitized Biocollection and the Global Biodiversity Information Facility have already accumulated millions of specimen images yet to be explored. This presents the perfect time to take advantage of the opportunity to automate the identification process and increase the rate of novel discoveries using computer vision (CV) and machine learning (ML) techniques. In this study, a systematic literature review of more than 70 peer-reviewed publications was conducted focusing on the application of computer vision and machine learning techniques to digitized herbarium specimens. The study categorizes the different techniques and applications that are commonly used for digitized herbarium specimens and highlights existing challenges together with their potential solutions. We hope this study will serve as a firm foundation for new researchers in the relevant disciplines and will also be enlightening to both computer science and ecology experts.     

Application of radiomics and machine learning in head and neck cancers

With the continuous development of medical image informatics technology, more and more high-throughput quantitative data could be extracted from digital medical images, which has resulted in a new kind of omics-Radiomics. In recent years, in addition to genomics, proteomics and metabolomics, radiomic has attracted the interest of more and more researchers. Compared to other omics, radiomics can be perfectly integrated with clinical data, even with the pathology and molecular biomarker, so that the study can be closer to the clinical reality and more revealing of the tumor development. Mass data will also be generated in this process. Machine learning, due to its own characteristics, has a unique advantage in processing massive radiomic data. By analyzing mass amounts of data with strong clinical relevance, people can construct models that more accurately reflect tumor development and progression, thereby providing the possibility of personalized and sequential treatment of patients. As one of the cancer types whose treatment and diagnosis rely on imaging examination, radiomics has a very broad application prospect in head and neck cancers (HNC). Until now, there have been some notable results in HNC. In this review, we will introduce the concepts and workflow of radiomics and machine learning and their current applications in head and neck cancers, as well as the directions and applications of artificial intelligence in the treatment and diagnosis of HNC.     

Lensfree computational microscopy tools for cell and tissue imaging at the point-of-care and in low-resource settings

The recent revolution in digital technologies and information processing methods present important opportunities to transform the way optical imaging is performed, particularly toward improving the throughput of microscopes while at the same time reducing their relative cost and complexity. Lensfree computational microscopy is rapidly emerging toward this end, and by discarding lenses and other bulky optical components of conventional imaging systems, and relying on digital computation instead, it can achieve both reflection and transmission mode microscopy over a large field-of-view within compact, cost-effective and mechanically robust architectures. Such high throughput and miniaturized imaging devices can provide a complementary toolset for telemedicine applications and point-of-care diagnostics by facilitating complex and critical tasks such as cytometry and microscopic analysis of e.g., blood smears, Pap tests and tissue samples. In this article, the basics of these lensfree microscopy modalities will be reviewed, and their clinically relevant applications will be discussed.     

Image Analysis Algorithms for Immunohistochemical Assessment of Cell Death Events and Fibrosis in Tissue Sections

Cell death is of broad physiological and pathological importance, making quantification of biochemical events associated with cell demise a high priority for experimental pathology. Fibrosis is a common consequence of tissue injury involving necrotic cell death. Using tissue specimens from experimental mouse models of traumatic brain injury, cardiac fibrosis, and cancer, as well as human tumor specimens assembled in tissue microarray (TMA) format, we undertook computer-assisted quantification of specific immunohistochemical and histological parameters that characterize processes associated with cell death. In this study, we demonstrated the utility of image analysis algorithms for color deconvolution, colocalization, and nuclear morphometry to characterize cell death events in tissue specimens: (a) subjected to immunostaining for detecting cleaved caspase-3, cleaved poly(ADP-ribose)-polymerase, cleaved lamin-A, phosphorylated histone H2AX, and Bcl-2; (b) analyzed by terminal deoxyribonucleotidyl transferase–mediated dUTP nick end labeling assay to detect DNA fragmentation; and (c) evaluated with Masson''s trichrome staining. We developed novel algorithm-based scoring methods and validated them using TMAs as a high-throughput format. The proposed computer-assisted scoring methods for digital images by brightfield microscopy permit linear quantification of immunohistochemical and histochemical stainings. Examples are provided of digital image analysis performed in automated or semiautomated fashion for successful quantification of molecular events associated with cell death in tissue sections. (J Histochem Cytochem 57:649–663, 2009)     

Medical images and automated interpretation

The application of automated interpretation to medical images is discussed and the main methods of medical imaging are briefly described. The factors behind the process of human clinical interpretation are also considered. Because human interpretation can be aided by processing of the raw image, the standard methods of image processing are mentioned. Automated interpretation of images in other fields is relevant, and segmentation of images is an important initial part of the process. There are already some applications of automated interpretation of medical images. Some are more complete than others. This will undoubtedly be an important developing field of work which will draw on experience from other areas, and be spurred on by the increasing complexity of medical imaging methods and shortage of expertise for human interpretation.     

A Rapid Stain-Clearing Method for Video Based Cytological Analysis of Cotton Megagametophytes

Optical “clearing” is a cost saving method for preparing large numbers of whole, dissected or thickly sectioned cytological specimens such as plant ovules and ovaries. Minimal labor is required and specimens retain three-dimensional integrity. Previous development of high contrast stain-clearing methods using hemalum to impart contrast has facilitated analysis and photography under brightfield illumination for small ovules. The deep stain intensity of hemalum, however, often precludes adequate light transmission and contrast within internal focal planes, limiting the applicability of hemalum-based stain-clearing to small specimens. Having encountered this problem for nucelli of cotton (Gossypium barbadense L.), which are roughly 300 μm thick at fertilization, we have developed a modified stain-clearing system. The two key features of these new methods are the use of azure, C, which allows the intensity of staining to be readily regulated, and contrast manipulation via video signal and image processing. Intensity of azure C stain was readily controlled by modifying the staining and/or dehydration media to produce relatively low contrast specimens. Analysis was facilitated by indirect viewing on a video monitor using adjustments of sensitivity, exposure, and contrast of the charge-coupled device (CCD) camera. Digital processing provided further enhancement. Acceptable images were obtained from virtually all specimens. These methods, which combine low contrast (high transmittance) specimens with high contrast imaging, should facilitate data acquisition on reproduction, thus the developmental and genetic characterization of reproductive mutants. Other applications, e.g., in pathology and embryology, are readily envisioned.     

emScope: A Tool Kit for Control and Automation of a Remote Electron Microscope

We have developed a portable and extensible set of tools and applications for control of a remote transmission electron microscope (TEM). These tools will be used to develop a system for automatically acquiring large numbers of high quality images from a TEM in a routine, robust, and efficient manner. In particular we plan to develop applications that allow for data collection from specimens preserved in vitreous ice.     

A Rapid Stain-Clearing Method for Video Based Cytological Analysis of Cotton Megagametophytes

Optical “clearing” is a cost saving method for preparing large numbers of whole, dissected or thickly sectioned cytological specimens such as plant ovules and ovaries. Minimal labor is required and specimens retain three-dimensional integrity. Previous development of high contrast stain-clearing methods using hemalum to impart contrast has facilitated analysis and photography under brightfield illumination for small ovules. The deep stain intensity of hemalum, however, often precludes adequate light transmission and contrast within internal focal planes, limiting the applicability of hemalum-based stain-clearing to small specimens. Having encountered this problem for nucelli of cotton (Gossypium barbadense L.), which are roughly 300 μm thick at fertilization, we have developed a modified stain-clearing system. The two key features of these new methods are the use of azure, C, which allows the intensity of staining to be readily regulated, and contrast manipulation via video signal and image processing. Intensity of azure C stain was readily controlled by modifying the staining and/or dehydration media to produce relatively low contrast specimens. Analysis was facilitated by indirect viewing on a video monitor using adjustments of sensitivity, exposure, and contrast of the charge-coupled device (CCD) camera. Digital processing provided further enhancement. Acceptable images were obtained from virtually all specimens. These methods, which combine low contrast (high transmittance) specimens with high contrast imaging, should facilitate data acquisition on reproduction, thus the developmental and genetic characterization of reproductive mutants. Other applications, e.g., in pathology and embryology, are readily envisioned.     

Virtual telepathology in Egypt,applications of WSI in Cairo University

Telepathology, the practice of pathology at a long distance, has advanced continuously since 1986. Today, fourth-generation telepathology systems, so-called virtual slide telepathology systems, are being used for education applications. Both conventional and innovative surgical pathology diagnostic services are being designed and implemented as well. We have a successful experience in Egypt in applying the static & dynamic techniques in a pilot project between the Italian Hospital in Cairo (NPO) and the Civico Hospital in Palermo This project began in 2003 and continued till now. In 2004, centers in Venice, London and Pittsburgh participated actively in our project. During the past seven years we consulted on many problematic pathological cases with these different specialized pathological centers in Italy, UK & USA. In addition to the highly specialized scientific value of consulting on the cases and exchanging knowledge, we saved a lot of time and money and succeeded in providing our patients with a better medical service. In view of this success we have already established a new Digital Telepathology unit (DTU) in the pathology department, Cairo University, using the latest technique of telepathology which is Whole Slide Imaging (WSI) since one year. This unit is considered the first Digital pathology unit in all the universities of the whole Middle East. During the passed year we created a digital pathology library for the under graduate students using the WSI technique and changed the teaching method of the histopathology slides to be completely digital. We are building another digital pathology library (for post graduate candidates) which will be available to all pathology candidates in Egyptian universities & universities in the surrounding Arabic countries. We are also creating a digital pathology network between pathology centers in the Middle East for exchanging knowledge & telepathology.     

Content-based histopathology image retrieval using CometCloud

Background

The development of digital imaging technology is creating extraordinary levels of accuracy that provide support for improved reliability in different aspects of the image analysis, such as content-based image retrieval, image segmentation, and classification. This has dramatically increased the volume and rate at which data are generated. Together these facts make querying and sharing non-trivial and render centralized solutions unfeasible. Moreover, in many cases this data is often distributed and must be shared across multiple institutions requiring decentralized solutions. In this context, a new generation of data/information driven applications must be developed to take advantage of the national advanced cyber-infrastructure (ACI) which enable investigators to seamlessly and securely interact with information/data which is distributed across geographically disparate resources. This paper presents the development and evaluation of a novel content-based image retrieval (CBIR) framework. The methods were tested extensively using both peripheral blood smears and renal glomeruli specimens. The datasets and performance were evaluated by two pathologists to determine the concordance.

Results

The CBIR algorithms that were developed can reliably retrieve the candidate image patches exhibiting intensity and morphological characteristics that are most similar to a given query image. The methods described in this paper are able to reliably discriminate among subtle staining differences and spatial pattern distributions. By integrating a newly developed dual-similarity relevance feedback module into the CBIR framework, the CBIR results were improved substantially. By aggregating the computational power of high performance computing (HPC) and cloud resources, we demonstrated that the method can be successfully executed in minutes on the Cloud compared to weeks using standard computers.

Conclusions

In this paper, we present a set of newly developed CBIR algorithms and validate them using two different pathology applications, which are regularly evaluated in the practice of pathology. Comparative experimental results demonstrate excellent performance throughout the course of a set of systematic studies. Additionally, we present and evaluate a framework to enable the execution of these algorithms across distributed resources. We show how parallel searching of content-wise similar images in the dataset significantly reduces the overall computational time to ensure the practical utility of the proposed CBIR algorithms.     

Image-based plant phenotyping with incremental learning and active contours

Plant phenotyping investigates how a plant's genome, interacting with the environment, affects the observable traits of a plant (phenome). It is becoming increasingly important in our quest towards efficient and sustainable agriculture. While sequencing the genome is becoming increasingly efficient, acquiring phenotype information has remained largely of low throughput. Current solutions for automated image-based plant phenotyping, rely either on semi-automated or manual analysis of the imaging data, or on expensive and proprietary software which accompanies costly hardware infrastructure. While some attempts have been made to create software applications that enable the analysis of such images in an automated fashion, most solutions are tailored to particular acquisition scenarios and restrictions on experimental design. In this paper we propose and test, a method for the segmentation and the automated analysis of time-lapse plant images from phenotyping experiments in a general laboratory setting, that can adapt to scene variability. The method involves minimal user interaction, necessary to establish the statistical experiments that may follow. At every time instance (i.e., a digital photograph), it segments the plants in images that contain many specimens of the same species. For accurate plant segmentation we propose a vector valued level set formulation that incorporates features of color intensity, local texture, and prior knowledge. Prior knowledge is incorporated using a plant appearance model implemented with Gaussian mixture models, which utilizes incrementally information from previously segmented instances. The proposed approach is tested on Arabidopsis plant images acquired with a static camera capturing many subjects at the same time. Our validation with ground truth segmentations and comparisons with state-of-the-art methods in the literature shows that the proposed method is able to handle images with complicated and changing background in an automated fashion. An accuracy of 96.7% (dice similarity coefficient) was observed, which was higher than other methods used for comparison. While here it was tested on a single plant species, the fact that we do not employ shape driven models and we do not rely on fully supervised classification (trained on a large dataset) increases the ease of deployment of the proposed solution for the study of different plant species in a variety of laboratory settings. Our solution will be accompanied by an easy to use graphical user interface and, to facilitate adoption, we will make the software available to the scientific community.