Automated classification of hepatocellular carcinoma differentiation using multiphoton microscopy and deep learning

In the case of hepatocellular carcinoma (HCC) samples, classification of differentiation is crucial for determining prognosis and treatment strategy decisions. However, a label‐free and automated classification system for HCC grading has not been yet developed. Hence, in this study, we demonstrate the fusion of multiphoton microscopy and a deep‐learning algorithm for classifying HCC differentiation to produce an innovative computer‐aided diagnostic method. Convolutional neural networks based on the VGG‐16 framework were trained using 217 combined two‐photon excitation fluorescence and second‐harmonic generation images; the resulting classification accuracy of the HCC differentiation grade was over 90%. Our results suggest that a combination of multiphoton microscopy and deep learning can realize label‐free, automated methods for various tissues, diseases and other related classification problems.      

Label‐free,zeptomole cancer biomarker detection by surface‐enhanced fluorescence on nanoporous gold disk plasmonic nanoparticles

We experimentally demonstrate a label‐free biosensor for the ERBB2 cancer gene DNA target based on the distance‐dependent detection of surface‐enhanced fluorescence (SEF) on nanoporous gold disk (NPGD) plasmonic nanoparticles. We achieve detection of 2.4 zeptomole of DNA target on the NPGD substrate with an upper concentration detection limit of 1 nM. Without the use of molecular spacers, the NPGD substrate as an SEF platform was shown to provide higher net fluorescence for visible and NIR fluorophores compared to glass and non‐porous gold substrates. The enhanced fluorescence signals in patterned nanoporous gold nanoparticles make NPGD a viable material for further reducing detection limits for biomolecular targets used in clinical assays.

With patterned nanoporous gold disk (NPGD) plasmonic nanoparticles, a label‐free biosensor that makes use of distance‐dependent detection of surface‐enhanced fluorescence (SEF) is constructed and tested for zeptomole detection of ERBB2 cancer gene DNA targets.     

Multimodal imaging to explore endogenous fluorescence of fresh and fixed human healthy and tumor brain tissues

To complement a project toward label‐free optical biopsy and enhanced resection which the overall goal is to develop a multimodal nonlinear endomicroscope, this multimodal approach aims to enhance the accuracy in classifying brain tissue into solid tumor, infiltration and normal tissue intraoperatively. Multiple optical measurements based on one‐ and two‐photon spectral and lifetime autofluorescence, including second harmonic generation imaging, were acquired. As a prerequisite, studying the effect of the time of measurement postexcision on tissue's spectral/lifetime fluorescence properties was warranted, so spectral and lifetime fluorescences of fresh brain tissues were measured using a point‐based linear endoscope. Additionally, a comparative study on tissue's optical properties obtained by multimodal nonlinear optical imaging microscope from fresh and fixed tissue was necessary to test whether clinical validation of the nonlinear endomicroscope is feasible by extracting optical signatures from fixed tissue rather than from freshly excised samples. The former is generally chosen for convenience. Results of this study suggest that an hour is necessary postexcision to have consistent fluorescence intensities\lifetimes. The fresh (a,b,c) vs fixed (d,e,f) tissue study indicates that while all optical signals differ after fixation. The characteristic features extracted from one‐ and two‐photon excitation still discriminate normal brain (a,d) cortical tissue, glioblastoma (GBM) (b,e) and metastases (c,f).      

Autofluorescence imaging identifies tumor cell‐cycle status on a single‐cell level

The goal of this study is to validate fluorescence intensity and lifetime imaging of metabolic co‐enzymes NAD(P)H and FAD (optical metabolic imaging, or OMI) as a method to quantify cell‐cycle status of tumor cells. Heterogeneity in tumor cell‐cycle status (e. g. proliferation, quiescence, apoptosis) increases drug resistance and tumor recurrence. Cell‐cycle status is closely linked to cellular metabolism. Thus, this study applies cell‐level metabolic imaging to distinguish proliferating, quiescent, and apoptotic populations. Two‐photon microscopy and time‐correlated single photon counting are used to measure optical redox ratio (NAD(P)H fluorescence intensity divided by FAD intensity), NAD(P)H and FAD fluorescence lifetime parameters. Redox ratio, NAD(P)H and FAD lifetime parameters alone exhibit significant differences (p<0.05) between population means. To improve separation between populations, linear combination models derived from partial least squares ‐ discriminant analysis (PLS‐DA) are used to exploit all measurements together. Leave‐one‐out cross validation of the model yielded high classification accuracies (92.4 and 90.1 % for two and three populations, respectively). OMI and PLS‐DA also identifies each sub‐population within heterogeneous samples. These results establish single‐cell analysis with OMI and PLS‐DA as a label‐free method to distinguish cell‐cycle status within intact samples. This approach could be used to incorporate cell‐level tumor heterogeneity in cancer drug development.

     

Label‐free optical projection tomography for quantitative three‐dimensional anatomy of mouse embryo

Recent progress in three‐dimensional optical imaging techniques allows visualization of many comprehensive biological specimens. Optical clearing methods provide volumetric and quantitative information by overcoming the limited depth of light due to scattering. However, current imaging technologies mostly rely on the synthetic or genetic fluorescent labels, thus limits its application to whole‐body visualization of generic mouse models. Here, we report a label‐free optical projection tomography (LF‐OPT) technique for quantitative whole mouse embryo imaging. LF‐OPT is based on the attenuation contrast of light rather than fluorescence, and it utilizes projection imaging technique similar to computed tomography for visualizing the volumetric structure. We demonstrate this with a collection of mouse embryo morphologies in different stages using LF‐OPT. Additionally, we extract quantitative organ information applicable toward high‐throughput phenotype screening. Our results indicate that LF‐OPT can provide multi‐scale morphological information in various tissues including bone, which can be difficult in conventional optical imaging technique.     

Diagnosis of early gastric cancer based on fluorescence hyperspectral imaging technology combined with partial‐least‐square discriminant analysis and support vector machine

This study investigated the feasibility of using fluorescence hyperspectral imaging technology to diagnose of early‐stage gastric cancer. Fluorescence spectral images of 76 patients who were pathologically diagnosed as non‐atrophic gastritis, premalignant lesions and gastric cancer were collected. Fluorescence spectra at 100‐pixel points were randomly extracted after binarization. Diagnostic models of non‐atrophic gastritis, premalignant lesions and gastric cancer were constructed through partial‐least‐square discriminant analysis (PLS‐DA) and support vector machine (SVM) algorithms. The prediction effects of PLS‐DA and SVM models were compared. Results showed that the average spectra of normal, precancerous and gastric cancer tissues significantly differed at 496, 546, 640 and 670 nm, and regular changes in fluorescence intensity at 546 nm were in the following order: normal > precancerous lesions > gastric cancer. Additionally, the effect of the diagnostic model established by SVM is significantly better than PLS‐DA which accuracy, specificity and sensitivity are above 94%. Experimental results revealed that the fast diagnostic model of early gastric cancer by combining fluorescence hyperspectral imaging technology and improved SVM was effective and feasible, thereby providing an accurate and rapid method for diagnosing early‐stage gastric cancer.      

Widefield fluorescence lifetime imaging of protoporphyrin IX for fluorescence‐guided neurosurgery: An ex vivo feasibility study

Achieving a maximal safe extent of resection during brain tumor surgery is the goal for improved patient prognosis. Fluorescence‐guided neurosurgery using 5‐aminolevulinic acid (5‐ALA) induced protoporphyrin IX has thereby become a valuable tool enabling a high frequency of complete resections and a prolonged progression‐free survival in glioblastoma patients. We present a widefield fluorescence lifetime imaging device with 250 mm working distance, working under similar conditions such as surgical microscopes based on a time‐of‐flight dual tap CMOS camera. In contrast to intensity‐based fluorescence imaging, our method is invariant to light scattering and absorption while being sensitive to the molecular composition of the tissue. We evaluate the feasibility of lifetime imaging of protoporphyrin IX using our system to analyze brain tumor phantoms and fresh 5‐ALA‐labeled human tissue samples. The results demonstrate the potential of our lifetime sensing device to go beyond the limitation of current intensity‐based fluorescence‐guided neurosurgery.      

Integration of diffraction phase microscopy and Raman imaging for label‐free morpho‐molecular assessment of live cells

Label‐free quantitative imaging is highly desirable for studying live cells by extracting pathophysiological information without perturbing cell functions. Here, we demonstrate a novel label‐free multimodal optical imaging system with the capability of providing comprehensive morphological and molecular attributes of live cells. Our morpho‐molecular microscopy (3M) system draws on the combined strength of quantitative phase microscopy (QPM) and Raman microscopy to probe the morphological features and molecular fingerprinting characteristics of each cell under observation. While the commonr‐path geometry of our QPM system allows for highly sensitive phase measurement, the Raman microscopy is equipped with dual excitation wavelengths and utilizes the same detection and dispersion system, making it a distinctive multi‐wavelength system with a small footprint. We demonstrate the applicability of the 3M system by investigating nucleated and nonnucleated cells. This integrated label‐free platform has a promising potential in preclinical research, as well as in clinical diagnosis in the near future.      

Label‐free and non‐invasive discrimination of HaCaT and melanoma cells in a co‐culture model by hyperspectral confocal reflectance microscopy

A novel hyperspectral confocal microscopy method to separate different cell populations in a co‐culture model is presented here. The described methodological and instrumental approach allows discrimination of different cell types using a non‐invasive, label free method with good accuracy with a single cell resolution. In particular, melanoma cells are discriminated from HaCaT cells by hyperspectral confocal imaging, principal component analysis and optical frequencies signing, as confirmed by fluorescence labelling cross check. The identification seems to be quite robust to be insensitive to the cellular shape within the studied samples, enabling to separate cells according to their cytotype down to a single cell sensitivity.

Set of hyperspectral images of melanoma‐keratinocytes co‐culture model (left), score plot of principal component analysis and spectral analysis of principal components coefficients (center), label‐free spectral identification of cell populations (right).     

In vivo label‐free functional photoacoustic monitoring of ischemic reperfusion

Pressure ulcer formation is a common problem among patients confined to bed or restricted to wheelchairs. The ulcer forms when the affected skin and underlying tissues go through repeated cycles of ischemia and reperfusion, leading to inflammation. This theory is evident by intravital imaging studies performed in immune cell–specific, fluorescent reporter mouse skin with induced ischemia‐reperfusion (I‐R) injuries. However, traditional confocal or multiphoton microscopy cannot accurately monitor the progression of vascular reperfusion by contrast agents, which leaks into the interstitium under inflammatory conditions. Here, we develop a dual‐wavelength micro electro mechanical system (MEMS) scanning–based optical resolution photoacoustic microscopy (OR‐PAM) system for continuous label‐free functional imaging of vascular reperfusion in an IR mouse model. This MEMS‐OR‐PAM system provides fast scanning speed for concurrent dual‐wavelength imaging, which enables continuous monitoring of the reperfusion process. During reperfusion, the revascularization of blood vessels and the oxygen saturation (sO₂) changes in both arteries and veins are recorded, from which the local oxygen extraction ratios of the ischemic tissue and the unaffected tissue can be quantified. Our MEMS‐OR‐PAM system provides novel perspectives to understand the I‐R injuries. It solves the problem of dynamic label‐free functional monitoring of the vascular reperfusion at high spatial resolution.      

Towards automated cancer screening: Label‐free classification of fixed cell samples using wavelength modulated Raman spectroscopy

The ability to provide quantitative, objective and automated pathological analysis would provide enormous benefits for national cancer screening programmes, in terms of both resource reduction and improved patient wellbeing. The move towards molecular pathology through spectroscopic methods shows great promise, but has been restricted by spectral quality, acquisition times and lack of direct clinical application. In this paper, we present the application of wavelength modulated Raman spectroscopy for the automated label‐ and fluorescence‐free classification of fixed squamous epithelial cells in suspension, such as those produced during a cervical smear test. Direct comparison with standard Raman spectroscopy shows marked improvement of sensitivity and specificity when considering both human papillomavirus (sensitivity +12.0%, specificity +5.3%) and transformation status (sensitivity +10.3%, specificity +11.1%). Studies on the impact of intracellular sampling location and storage effects suggest that wavelength modulated Raman spectroscopy is sufficiently robust to be used in fixed cell classification, but requires further investigations of potential sources of molecular variation in order to improve current clinical tools.      

Multi‐ATOM: Ultrahigh‐throughput single‐cell quantitative phase imaging with subcellular resolution

A growing body of evidence has substantiated the significance of quantitative phase imaging (QPI) in enabling cost‐effective and label‐free cellular assays, which provides useful insights into understanding the biophysical properties of cells and their roles in cellular functions. However, available QPI modalities are limited by the loss of imaging resolution at high throughput and thus run short of sufficient statistical power at the single‐cell precision to define cell identities in a large and heterogeneous population of cells—hindering their utility in mainstream biomedicine and biology. Here we present a new QPI modality, coined multiplexed asymmetric‐detection time‐stretch optical microscopy (multi‐ATOM) that captures and processes quantitative label‐free single‐cell images at ultrahigh throughput without compromising subcellular resolution. We show that multi‐ATOM, based upon ultrafast phase‐gradient encoding, outperforms state‐of‐the‐art QPI in permitting robust phase retrieval at a QPI throughput of >10 000 cell/sec, bypassing the need for interferometry which inevitably compromises QPI quality under ultrafast operation. We employ multi‐ATOM for large‐scale, label‐free, multivariate, cell‐type classification (e.g. breast cancer subtypes, and leukemic cells vs peripheral blood mononuclear cells) at high accuracy (>94%). Our results suggest that multi‐ATOM could empower new strategies in large‐scale biophysical single‐cell analysis with applications in biology and enriching disease diagnostics.      

Label‐free grading and staging of urothelial carcinoma through multimodal fibre‐probe spectroscopy

Urothelial carcinoma (UC) is the most common bladder tumour. Proper treatment requires tumour resection for diagnosing its grade (aggressiveness) and stage (invasiveness). White‐light cystoscopy and histopathological examination are the gold standard procedures for clinical and histopathological diagnostics, respectively. However, cystoscopy is limited in terms of specificity, histology requires long tissue processing, both procedures rely on operator's experience. Multimodal optical spectroscopy can provide a powerful tool for detecting, staging and grading bladder tumours in a fast, reliable and label‐free modality. In this study, we collected fluorescence, Raman and reflectance spectra from 50 biopsies obtained from 32 patients undergoing transurethral resection of bladder tumour using a multimodal fibre‐probe. Principal component analysis allowed distinguishing normal from pathological tissues, as well as discriminating tumour stages and grades. Each individual spectroscopic technique provided high specificity and sensitivity in classifying all tissues; however, a multimodal approach resulted in a considerable increase in diagnostic accuracy (≥95%), which is of paramount importance for tumour grading and staging. The presented method offers the potential for being applied in cystoscopy and for providing an automated diagnosis of UC at the clinical level, with an improvement with respect to current state‐of‐the‐art procedures.      

Mapping of retardance,diattenuation and polarizance vector on Poincare sphere for diagnosis and classification of cervical precancer

The mapping of diattenuation, polarizance and retardance vector (normalized Stokes vector) on Poincare sphere, evaluated from Mueller matrix of optically anisotropic stromal region of cervical tissues, is presented for cervical precancer detection and its staging. This reveals that the changes in the polarization states shown by these normalized Stokes vectors corresponds to the degradation of linearly arranged collagen fibers, breakage of the collagen cross links in the stromal region and change in the density of scattering sites when cervical cancer evolves. The distinct nature of real and imaginary parts of the refractive index for linear, linear‐45 and circular polarization from the optically anisotropic stromal region underscore the various polarization structures of the connective tissue region which get modified during the pathological changes. It has been found that versatility of these vectors for normal and precancerous cervical tissue of various grades may be utilized as a key distinction for qualitative staging of cervical precancer tissue. Quantitative classification of precancerous stages of cervical precancer has been determined with 95%–100% sensitivity and 93%–100% specificity through the evaluation of linear and circular diattenuation, linear polarizance and linear birefringence from the components of the respective vectors.     

Insight into plant cell wall chemistry and structure by combination of multiphoton microscopy with Raman imaging

Spontaneous Raman scattering microspectroscopy, second harmonic generation (SHG) and 2‐photon excited fluorescence (2PF) were used in combination to characterize the morphology together with the chemical composition of the cell wall in native plant tissues. As the data obtained with unstained sections of Sorghum bicolor root and leaf tissues illustrate, nonresonant as well as pre‐resonant Raman microscopy in combination with hyperspectral analysis reveals details about the distribution and composition of the major cell wall constituents. Multivariate analysis of the Raman data allows separation of different tissue regions, specifically the endodermis, xylem and lumen. The orientation of cellulose microfibrils is obtained from polarization‐resolved SHG signals. Furthermore, 2‐photon autofluorescence images can be used to image lignification. The combined compositional, morphological and orientational information in the proposed coupling of SHG, Raman imaging and 2PF presents an extension of existing vibrational microspectroscopic imaging and multiphoton microscopic approaches not only for plant tissues.      

Fluorescence life‐time imaging and steady state polarization for examining binding of fluorophores to gold nanoparticles

Nanocomposites as multifunctional agents are capable of combing imaging and cell biology technologies. The conventional methods used for validation of the conjugation process of nanoparticles (NPs) to fluorescent molecules such as spectroscopy analysis and surface potential measurements, are not sufficient. In this paper we present a new and highly sensitive procedure that uses the combination of (1) fluorescence spectrum, (2) fluorescence lifetime, and (3) steady state fluorescence polarization measurements. We characterize and analyze gold NPs with Lucifer yellow (LY) surface coating as a model. We demonstrate the ability to differentiate between LY‐GNP (the conjugated complex) and a mixture of coated NP and free dyes. We suggest the approach for neuroscience applications where LY is used for detecting and labeling cells, studying morphology and intracellular communications.

Histograms of Fluorescence lifetime imaging (FLIM) of free LY dye (Left) in comparison to the conjugated dye to gold nanoparticles, LY‐GNP (Middle) enable the differentiation between LY‐GNP (the conjugated complex) and a mixture of coated NP and free dyes (Right).     

Distinction between breast cancer cell subtypes using third harmonic generation microscopy

Third Harmonic Generation (THG) microscopy as a non‐invasive, label free imaging methodology, allows linkage of lipid profiles with various breast cancer cells. The collected THG signal arise mostly from the lipid droplets and the membrane lipid bilayer. Quantification of THG signal can accurately distinguish HER2‐positive cells. Further analysis using Fourier transform infrared (FTIR) spectra reveals cancer‐specific profiles, correlating lipid raft‐corresponding spectra to THG signal, associating thus THG to chemical information.

THG imaging of a cancer cell.     

Identification of distinctive features in human intracranial tumors by label‐free nonlinear multimodal microscopy

Nonlinear multimodal microscopy offers a series of label‐free techniques with potential for intraoperative identification of tumor borders in situ using novel endoscopic devices. Here, we combined coherent anti‐Stokes Raman scattering, two‐photon excited fluorescence (TPEF) and second harmonic generation imaging to analyze biopsies of different human brain tumors, with the aim to understand whether the morphological information carried by single field of view images, similar to what delivered by present endoscopic systems, is sufficient for tumor recognition. We imaged 40 human biopsies of high and low grade glioma, meningioma, as well as brain metastases of melanoma, breast, lung and renal carcinoma, in comparison with normal brain parenchyma. Furthermore, five biopsies of schwannoma were analyzed and compared with nonpathological nerve tissue. Besides the high cellularity, the typical features of tumor, which were identified and quantified, are intracellular and extracellular lipid droplets, aberrant vessels, extracellular matrix collagen and diffuse TPEF. Each tumor type displayed a particular morphochemistry characterized by specific patterns of the above‐mentioned features. Nonlinear multimodal microscopy performed on fresh unprocessed biopsies confirmed that the technique has the ability to visualize tumor structures and discern normal from neoplastic tissue likewise in conditions close to in situ.      

Label‐free multi‐parametric imaging of single cells: dual picosecond optoacoustic microscopy

Advances in microscopy with new visualization possibilities often bring dramatic progress to our understanding of the intriguing cellular machinery. Picosecond optoacoustic micro‐spectroscopy is an optical technique based on ultrafast pump‐probe generation and detection of hypersound on time durations of picoseconds and length scales of nanometers. It is experiencing a renaissance as a versatile imaging tool for cell biology research after a plethora of applications in solid‐state physics. In this emerging context, this work reports on a dual‐probe architecture to carry out real‐time parallel detection of the hypersound propagation inside a cell that is cultured on a metallic substrate, and of the hypersound reflection at the metal/cell adhesion interface. Using this optoacoustic modality, several biophysical properties of the cell can be measured in a noncontact and label‐free manner. Its abilities are demonstrated with the multiple imaging of a mitotic macrophage‐like cell in a single run experiment.      

High and ultra‐high definition of infrared spectral histopathology gives an insight into chemical environment of lung metastases in breast cancer

Using high definition (HD) and ultra‐high definition (UHD) of Fourier‐transform infrared (FTIR) spectroscopic imaging, we characterized spectrally pulmonary metastases in a murine model of breast cancer comparing them with histopathological results (Hematoxylin and eosin [H&E] staining). This comparison showed excellent agreement between the methods in case of localization of metastases with size below 1 mm and revealed that label‐free HD and UHD IR spectral histopathology distinguish the type of neoplastic cells. We primary focused on differentiation between metastatic foci in the pleural cavity from cancer cells present in lung parenchyma and inflamed cells present in extracellular matrix of lungs due to growing of advanced metastases. In addition, a combination of unsupervised clustering and IR imaging indicated the high sensitivity of FTIR spectroscopy to identify chemical features of small macrometastases located under the pleural cavity and during epithelial‐mesenchymal transition. FTIR‐based spectral histopathology was proved to detect not only phases of breast cancer metastasis to lungs but also to differentiate various origins of metastases seeded from breast cancer.