Epithelium segmentation and automated Gleason grading of prostate cancer via deep learning in label‐free multiphoton microscopic images

In the current clinical care practice, Gleason grading system is one of the most powerful prognostic predictors for prostate cancer (PCa). The grading system is based on the architectural pattern of cancerous epithelium in histological images. However, the standard procedure of histological examination often involves complicated tissue fixation and staining, which are time‐consuming and may delay the diagnosis and surgery. In this study, label‐free multiphoton microscopy (MPM) was used to acquire subcellular‐resolution images of unstained prostate tissues. Then, a deep learning architecture (U‐net) was introduced for epithelium segmentation of prostate tissues in MPM images. The obtained segmentation results were then merged with the original MPM images to train a classification network (AlexNet) for automated Gleason grading. The developed method achieved an overall pixel accuracy of 92.3% with a mean F1 score of 0.839 for epithelium segmentation. By merging the segmentation results with the MPM images, the accuracy of Gleason grading was improved from 72.42% to 81.13% in hold‐out test set. Our results suggest that MPM in combination with deep learning holds the potential to be used as a fast and powerful clinical tool for PCa diagnosis.     

In vivo label‐free functional photoacoustic monitoring of ischemic reperfusion

Pressure ulcer formation is a common problem among patients confined to bed or restricted to wheelchairs. The ulcer forms when the affected skin and underlying tissues go through repeated cycles of ischemia and reperfusion, leading to inflammation. This theory is evident by intravital imaging studies performed in immune cell–specific, fluorescent reporter mouse skin with induced ischemia‐reperfusion (I‐R) injuries. However, traditional confocal or multiphoton microscopy cannot accurately monitor the progression of vascular reperfusion by contrast agents, which leaks into the interstitium under inflammatory conditions. Here, we develop a dual‐wavelength micro electro mechanical system (MEMS) scanning–based optical resolution photoacoustic microscopy (OR‐PAM) system for continuous label‐free functional imaging of vascular reperfusion in an IR mouse model. This MEMS‐OR‐PAM system provides fast scanning speed for concurrent dual‐wavelength imaging, which enables continuous monitoring of the reperfusion process. During reperfusion, the revascularization of blood vessels and the oxygen saturation (sO₂) changes in both arteries and veins are recorded, from which the local oxygen extraction ratios of the ischemic tissue and the unaffected tissue can be quantified. Our MEMS‐OR‐PAM system provides novel perspectives to understand the I‐R injuries. It solves the problem of dynamic label‐free functional monitoring of the vascular reperfusion at high spatial resolution.      

Automated label‐free detection of injured neuron with deep learning by two‐photon microscopy

Stroke is a significant cause of morbidity and long‐term disability globally. Detection of injured neuron is a prerequisite for defining the degree of focal ischemic brain injury, which can be used to guide further therapy. Here, we demonstrate the capability of two‐photon microscopy (TPM) to label‐freely identify injured neurons on unstained thin section and fresh tissue of rat cerebral ischemia‐reperfusion model, revealing definite diagnostic features compared with conventional staining images. Moreover, a deep learning model based on convolutional neural network is developed to automatically detect the location of injured neurons on TPM images. We then apply deep learning‐assisted TPM to evaluate the ischemic regions based on tissue edema, two‐photon excited fluorescence signal intensity, as well as neuronal injury, presenting a novel manner for identifying the infarct core, peri‐infarct area, and remote area. These results propose an automated and label‐free method that could provide supplementary information to augment the diagnostic accuracy, as well as hold the potential to be used as an intravital diagnostic tool for evaluating the effectiveness of drug interventions and predicting potential therapeutics.     

Back Cover: Automated classification of hepatocellular carcinoma differentiation using multiphoton microscopy and deep learning (J. Biophotonics 7/2019)

This schematic depicts the classification of multiphoton images with different degrees of HCC differentiation using the VGG‐16 neural network. The convolution layer is further trained based on the original weights. The weights of the fully connected layers are initialized as a random number and the training is restarted to improve its classification accuracy. Further details can be found in the article by Hongxin Lin, Chao Wei, Guangxing Wang, et al. ( e201800435 ).

     

Integration of diffraction phase microscopy and Raman imaging for label‐free morpho‐molecular assessment of live cells

Label‐free quantitative imaging is highly desirable for studying live cells by extracting pathophysiological information without perturbing cell functions. Here, we demonstrate a novel label‐free multimodal optical imaging system with the capability of providing comprehensive morphological and molecular attributes of live cells. Our morpho‐molecular microscopy (3M) system draws on the combined strength of quantitative phase microscopy (QPM) and Raman microscopy to probe the morphological features and molecular fingerprinting characteristics of each cell under observation. While the commonr‐path geometry of our QPM system allows for highly sensitive phase measurement, the Raman microscopy is equipped with dual excitation wavelengths and utilizes the same detection and dispersion system, making it a distinctive multi‐wavelength system with a small footprint. We demonstrate the applicability of the 3M system by investigating nucleated and nonnucleated cells. This integrated label‐free platform has a promising potential in preclinical research, as well as in clinical diagnosis in the near future.      

Multi‐ATOM: Ultrahigh‐throughput single‐cell quantitative phase imaging with subcellular resolution

A growing body of evidence has substantiated the significance of quantitative phase imaging (QPI) in enabling cost‐effective and label‐free cellular assays, which provides useful insights into understanding the biophysical properties of cells and their roles in cellular functions. However, available QPI modalities are limited by the loss of imaging resolution at high throughput and thus run short of sufficient statistical power at the single‐cell precision to define cell identities in a large and heterogeneous population of cells—hindering their utility in mainstream biomedicine and biology. Here we present a new QPI modality, coined multiplexed asymmetric‐detection time‐stretch optical microscopy (multi‐ATOM) that captures and processes quantitative label‐free single‐cell images at ultrahigh throughput without compromising subcellular resolution. We show that multi‐ATOM, based upon ultrafast phase‐gradient encoding, outperforms state‐of‐the‐art QPI in permitting robust phase retrieval at a QPI throughput of >10 000 cell/sec, bypassing the need for interferometry which inevitably compromises QPI quality under ultrafast operation. We employ multi‐ATOM for large‐scale, label‐free, multivariate, cell‐type classification (e.g. breast cancer subtypes, and leukemic cells vs peripheral blood mononuclear cells) at high accuracy (>94%). Our results suggest that multi‐ATOM could empower new strategies in large‐scale biophysical single‐cell analysis with applications in biology and enriching disease diagnostics.      

Label‐free quantitative chemical imaging and classification analysis of adipogenesis using mouse embryonic stem cells

Stem cells have received much attention recently for their potential utility in regenerative medicine. The identification of their differentiated progeny often requires complex staining procedures, and is challenging for intermediary stages which are a priori unknown. In this work, the ability of label‐free quantitative coherent anti‐Stokes Raman scattering (CARS) micro‐spectroscopy to identify populations of intermediate cell states during the differentiation of murine embryonic stem cells into adipocytes is assessed. Cells were imaged at different days of differentiation by hyperspectral CARS, and images were analysed with an unsupervised factorization algorithm providing Raman‐like spectra and spatially resolved maps of chemical components. Chemical decomposition combined with a statistical analysis of their spatial distributions provided a set of parameters that were used for classification analysis. The first 2 principal components of these parameters indicated 3 main groups, attributed to undifferentiated cells, cells differentiated into committed white pre‐adipocytes, and differentiating cells exhibiting a distinct protein globular structure with adjacent lipid droplets. An unsupervised classification methodology was developed, separating undifferentiated cell from cells in other stages, using a novel method to estimate the optimal number of clusters. The proposed unsupervised classification pipeline of hyperspectral CARS data offers a promising new tool for automated cell sorting in lineage analysis.      

Inside Cover: Rapid discrimination of colon cancer cells with single base mutation in KRAS gene segment using laser tweezers Raman spectroscopy (J. Biophotonics 3/2019)

Laser tweezers Raman spectroscopy as a label‐free and non‐invasive technology was employed to examine the colon cancer cells with single base mutation in KRAS gene segment for the first time. As a result of the comparison, a high correct classification was achieved. Our preliminary results showed that the LTRS system has a great potential for further applications in the rapid and label‐free detection of circulating tumor cells in liquid biopsy. Further details can be found in the article by Mengmeng Liu, Xiujie Liu, Zufang Huang, et al. ( e201800332 ).

     

Front Cover: Dynamic observation and quantification of type I/II collagen in chondrogenesis of mesenchymal stem cells by second‐order susceptibility microscopy (J. Biophotonics 2/2019)

Top row is time‐lapse, multiphoton imaging of induced chondrogenesis from stem cells. The second row shows the time‐lapse second harmonic generation imaging of generated collagen. However, only the third row of second order susceptibility imaging allows the differentiation of the content of the two types of collagen (I and II) that were produced over time. The method allows the noninvasive and label‐free discrimination of different collagen species in real time and can be used for quality control in tissue engineering. Further details can be found in the article by Chiu‐Mei Hsueh, Hung‐Ming Lin, Te‐Yu Tseng, et al. ( e201800097 ).

     

Label free imaging of cell‐substrate contacts by holographic total internal reflection microscopy

The study of cell adhesion contacts is pivotal to understand cell mechanics and interaction at substrates or chemical and physical stimuli. We designed and built a HoloTIR microscope for label‐free quantitative phase imaging of total internal reflection. Here we show for the first time that HoloTIR is a good choice for label‐free study of focal contacts and of cell/substrate interaction as its sensitivity is enhanced in comparison with standard TIR microscopy. Finally, the simplicity of implementation and relative low cost, due to the requirement of less optical components, make HoloTIR a reasonable alternative, or even an addition, to TIRF microscopy for mapping cell/substratum topography. As a proof of concept, we studied the formation of focal contacts of fibroblasts on three substrates with different levels of affinity for cell adhesion.

     

Vibrational spectroscopic imaging and live cell video microscopy for studying differentiation of primary human alveolar epithelial cells

Alveolar type II (ATII) cells in the peripheral human lung spontaneously differentiate toward ATI cells, thus enabling air‐blood barrier formation. Here, linear Raman and coherent anti‐Stokes Raman scattering (CARS) microscopy are applied to study cell differentiation of freshly isolated ATII cells. The Raman spectra can successfully be correlated with gradual morphological and molecular changes during cell differentiation. Alveolar surfactant rich vesicles in ATII cells are identified based on phospholipid vibrations, while ATI‐like cells are characterized by the absence of vesicular structures. Complementary, CARS microscopy allows for three‐dimensional visualization of lipid vesicles within ATII cells and their secretion, while hyperspectral CARS enables the distinction between cellular proteins and lipids according to their vibrational signatures. This study paves the path for further label‐free investigations of lung cells and the role of the pulmonary surfactant, thus also providing a basis for rational development of future lung therapeutics.      

Objective quantification of nerves in immunohistochemistry specimens of thyroid cancer utilising deep learning

Accurate quantification of nerves in cancer specimens is important to understand cancer behaviour. Typically, nerves are manually detected and counted in digitised images of thin tissue sections from excised tumours using immunohistochemistry. However the images are of a large size with nerves having substantial variation in morphology that renders accurate and objective quantification difficult using existing manual and automated counting techniques. Manual counting is precise, but time-consuming, susceptible to inconsistency and has a high rate of false negatives. Existing automated techniques using digitised tissue sections and colour filters are sensitive, however, have a high rate of false positives. In this paper we develop a new automated nerve detection approach, based on a deep learning model with an augmented classification structure. This approach involves pre-processing to extract the image patches for the deep learning model, followed by pixel-level nerve detection utilising the proposed deep learning model. Outcomes assessed were a) sensitivity of the model in detecting manually identified nerves (expert annotations), and b) the precision of additional model-detected nerves. The proposed deep learning model based approach results in a sensitivity of 89% and a precision of 75%. The code and pre-trained model are publicly available at https://github.com/IA92/Automated_Nerves_Quantification.     

Label‐free and non‐invasive discrimination of HaCaT and melanoma cells in a co‐culture model by hyperspectral confocal reflectance microscopy

A novel hyperspectral confocal microscopy method to separate different cell populations in a co‐culture model is presented here. The described methodological and instrumental approach allows discrimination of different cell types using a non‐invasive, label free method with good accuracy with a single cell resolution. In particular, melanoma cells are discriminated from HaCaT cells by hyperspectral confocal imaging, principal component analysis and optical frequencies signing, as confirmed by fluorescence labelling cross check. The identification seems to be quite robust to be insensitive to the cellular shape within the studied samples, enabling to separate cells according to their cytotype down to a single cell sensitivity.

Set of hyperspectral images of melanoma‐keratinocytes co‐culture model (left), score plot of principal component analysis and spectral analysis of principal components coefficients (center), label‐free spectral identification of cell populations (right).     

Inside Cover: A low‐cost,automated parasite diagnostic system via a portable,robotic microscope and deep learning (J. Biophotonics 9/2019)

A low‐cost, automated microscope is combined with machine learning to bring veterinary parasite diagnosis to the point of need. The authors present an inexpensive robotic microscope that automatically focuses, scans, and images a large area McMaster chamber. A deep learning image segmentation pipeline identifies and counts eggs of parasitic worms and single‐celled parasites in goats, dogs, and monkeys, yielding >96% diagnostic accuracy without the need for a trained user. Further details can be found in the article by Yaning Li, Rui Zheng, Yizhen Wu, et al. ( e201800410 ).

     

Label‐free and non‐invasive monitoring of porcine trophoblast derived cells: differentiation in serum and serum‐free media

Traditional approaches to characterize stem cell differentiation are time‐consuming, lengthy and invasive. Here, Raman microspectroscopy (RM) and atomic force microscopy (AFM) – both considered as non‐invasive techniques – are applied to detect the biochemical and biophysical properties of trophoblast derived stem‐like cells incubated up to 10 days under conditions designed to induce differentiation. Significant biochemical and biophysical differences between control cells and differentiated cells were observed. Quantitative real time PCR was also applied to analyze gene expression. The relationship between cell differentiation and associated cellular biochemical and biomechanical changes were discussed.

Monitoring trophoblast cells differentiation     

Perspectives,potentials and trends of ex vivo and in vivo optical molecular pathology

It is pivotal for medical applications, such as noninvasive histopathologic characterization of tissue, to realize label‐free and molecule‐specific representation of morphologic and biochemical composition in real‐time with subcellular spatial resolution. This unmet clinical need requires new approaches for rapid and reliable real‐time assessment of pathologies to complement established diagnostic tools. Photonic imaging combined with digitalization offers the potential to provide the clinician the requested information both under in vivo and ex vivo conditions. This report summarizes photonic approaches and their use in combination with image processing, machine learning and augmented virtual reality that might solve current challenges in modern medicine. Details are given for pathology, intraoperative diagnosis in head and neck cancer and endoscopic diagnosis in gastroenterology.      

Multicolor multiphoton microscopy based on a nanosecond supercontinuum laser source

Multicolor multiphoton microscopy is experimentally demonstrated for the first time on a spectral bandwidth of excitation of 300 nm (full width half maximum) thanks to the implementation a nanosecond supercontinuum (SC) source compact and simple with a low repetition rate. The interest of such a wide spectral bandwidth, never demonstrated until now, is highlighted in vivo: images of glioma tumor cells stably expressing eGFP grafted on the brain of a mouse and its blood vessels network labelled with Texas Red^® are obtained. These two fluorophores have a spectral bandwidth covering the whole 300 nm available. In parallel, a similar image quality is obtained on a sample of mouse muscle in vitro when excited with this nanosecond SC source or with a classical high rate, femtosecond and quasi monochromatic laser. This opens the way for (i) a simple and very complete biological characterization never performed to date with multiphoton processes, (ii) multiple means of contrast in nonlinear imaging allowed by the use of numerous fluorophores and (iii) other multiphoton processes like three‐photon ones.

     

Automated quantitative cytological analysis using portable microfluidic microscopy

In this article, a portable microfluidic microscopy based approach for automated cytological investigations is presented. Inexpensive optical and electronic components have been used to construct a simple microfluidic microscopy system. In contrast to the conventional slide‐based methods, the presented method employs microfluidics to enable automated sample handling and image acquisition. The approach involves the use of simple in‐suspension staining and automated image acquisition to enable quantitative cytological analysis of samples. The applicability of the presented approach to research in cellular biology is shown by performing an automated cell viability assessment on a given population of yeast cells. Further, the relevance of the presented approach to clinical diagnosis and prognosis has been demonstrated by performing detection and differential assessment of malaria infection in a given sample.

     

Label‐free,zeptomole cancer biomarker detection by surface‐enhanced fluorescence on nanoporous gold disk plasmonic nanoparticles

We experimentally demonstrate a label‐free biosensor for the ERBB2 cancer gene DNA target based on the distance‐dependent detection of surface‐enhanced fluorescence (SEF) on nanoporous gold disk (NPGD) plasmonic nanoparticles. We achieve detection of 2.4 zeptomole of DNA target on the NPGD substrate with an upper concentration detection limit of 1 nM. Without the use of molecular spacers, the NPGD substrate as an SEF platform was shown to provide higher net fluorescence for visible and NIR fluorophores compared to glass and non‐porous gold substrates. The enhanced fluorescence signals in patterned nanoporous gold nanoparticles make NPGD a viable material for further reducing detection limits for biomolecular targets used in clinical assays.

With patterned nanoporous gold disk (NPGD) plasmonic nanoparticles, a label‐free biosensor that makes use of distance‐dependent detection of surface‐enhanced fluorescence (SEF) is constructed and tested for zeptomole detection of ERBB2 cancer gene DNA targets.     

Dynamic multiphoton imaging of acellular dermal matrix scaffolds seeded with mesenchymal stem cells in diabetic wound healing

Significantly effective therapies need to be developed for chronic nonhealing diabetic wounds. In this work, the topical transplantation of mesenchymal stem cell (MSC) seeded on an acellular dermal matrix (ADM) scaffold is proposed as a novel therapeutic strategy for diabetic cutaneous wound healing. GFP‐labeled MSCs were cocultured with an ADM scaffold that was decellularized from normal mouse skin. These cultures were subsequently transplanted as a whole into the full‐thickness cutaneous wound site in streptozotocin‐induced diabetic mice. Wounds treated with MSC‐ADM demonstrated an increased percentage of wound closure. The treatment of MSC‐ADM also greatly increased angiogenesis and rapidly completed the reepithelialization of newly formed skin on diabetic mice. More importantly, multiphoton microscopy was used for the intravital and dynamic monitoring of collagen type I (Col‐I) fibers synthesis via second harmonic generation imaging. The synthesis of Col‐I fibers during diabetic wound healing is of great significance for revealing wound repair mechanisms. In addition, the activity of GFP‐labeled MSCs during wound healing was simultaneously traced via two‐photon excitation fluorescence imaging. Our research offers a novel advanced nonlinear optical imaging method for monitoring the diabetic wound healing process while the ADM and MSCs interact in situ. Schematic of dynamic imaging of ADM scaffolds seeded with mesenchymal stem cells in diabetic wound healing using multiphoton microscopy. PMT, photo‐multiplier tube.