Intraoperative multispectral and hyperspectral label‐free imaging: A systematic review of in vivo clinical studies

Multispectral and hyperspectral imaging (HSI) are emerging optical imaging techniques with the potential to transform the way surgery is performed but it is not clear whether current systems are capable of delivering real‐time tissue characterization and surgical guidance. We conducted a systematic review of surgical in vivo label‐free multispectral and HSI systems that have been assessed intraoperatively in adult patients, published over a 10‐year period to May 2018. We analysed 14 studies including 8 different HSI systems. Current in‐vivo HSI systems generate an intraoperative tissue oxygenation map or enable tumour detection. Intraoperative tissue oxygenation measurements may help to predict those patients at risk of postoperative complications and in‐vivo intraoperative tissue characterization may be performed with high specificity and sensitivity. All systems utilized a line‐scanning or wavelength‐scanning method but the spectral range and number of spectral bands employed varied significantly between studies and according to the system's clinical aim. The time to acquire a hyperspectral cube dataset ranged between 5 and 30 seconds. No safety concerns were reported in any studies. A small number of studies have demonstrated the capabilities of intraoperative in‐vivo label‐free HSI but further work is needed to fully integrate it into the current surgical workflow.      

Optical coherence hyperspectral microscopy with a single supercontinuum light source

In the paper, we have developed an optical coherence hyperspectral microscopy with a single supercontinuum light source. The microscopy consists of optical coherence tomography (OCT) and hyperspectral imaging (HSI), which can visualize the structural and functional characteristics of biological tissues. The 500 to 700 nm band is selected for HSI and OCT imaging, where HSI enables imaging of oxygen saturation and hemoglobin (Hb) content, while OCT acquires structural characteristics to assess the morphology of biological tissues. The system performance of the optical coherence hyperspectral microscopy is verified by normal mice ears, and the practical applications of the microscopy is further performed in 4T1 and inflammation Balb/c mice ears in vivo. The experimental results demonstrate that the microscopy has potential to provide complementary information for clinical applications.     

Integrated photoacoustic and hyperspectral dual‐modality microscopy for co‐imaging of melanoma and cutaneous squamous cell carcinoma in vivo

Skin carcinoma such as melanoma (MM) and cutaneous squamous cell carcinoma (cSCC) are considered as the highest mortality and the most aggressive skin cancers in dermatology. In view that early diagnosis and treatment can greatly improve the survival rate and life quality of the patients, developing noninvasive and effective evaluation methods is of great significance for the detection and identification of early stage cutaneous cancers. In this article, we propose a hybrid photoacoustic and hyperspectral dual‐modality microscopy to evaluate and differentiate skin carcinoma by structural and multiphysiological parameters. The proposed system's imaging abilities are verified by mimic phantoms and normal mice experiments. Furthermore, in vivo characterization and evaluation results of MM and cSCC mice are obtained successfully, which prove this novel method could be used as a reliable and useful method for skin cancer detection in early stages.     

Characterization of blood coagulation dynamics and oxygenation in ex‐vivo retinal vessels by fluorescence hyperspectral imaging

Blood coagulation mechanisms forming a blood clot and preventing hemorrhage have been extensively studied in the last decades. Knowing the mechanisms behind becomes very important particularly in the case of blood vessel diseases. Real‐time and accurate diagnostics accompanied by the therapy are particularly needed, for example, in diseases related to retinal vasculature. In our study, we employ for the first time fluorescence hyperspectral imaging (fHSI) combined with the spectral analysis algorithm concept to assess physical as well as functional information of blood coagulation in real‐time. By laser‐induced local disruption of retinal vessels to mimic blood leaking and subsequent coagulation and a proper fitting algorithm, we were able to reveal and quantify the extent of local blood coagulation through direct identification of the change of oxyhemoglobin concentration within few minutes. We confirmed and illuminated the spatio‐temporal evolution of the essential role of erythrocytes in the coagulation cascade as the suppliers of oxygenated hemoglobin. By additional optical tweezers force manipulation, we showed immediate aggregation of erythrocytes at the coagulation site. The presented fluorescence‐based imaging concept could become a valuable tool in various blood coagulation diagnostics as well as theranostic systems if coupled with the laser therapy.     

Differentiating aquatic plant communities in a eutrophic river using hyperspectral and multispectral remote sensing

1. This study evaluates the efficacy of remote sensing technology to monitor species composition, areal extent and density of aquatic plants (macrophytes and filamentous algae) in impoundments where their presence may violate water‐quality standards. 2. Multispectral satellite (IKONOS) images and more than 500 in situ hyperspectral samples were acquired to map aquatic plant distributions. By analyzing field measurements, we created a library of hyperspectral signatures for a variety of aquatic plant species, associations and densities. We also used three vegetation indices. Normalized Difference Vegetation Index (NDVI), near‐infrared (NIR)‐Green Angle Index (NGAI) and normalized water absorption depth (D_H)_, at wavelengths 554, 680, 820 and 977 nm to differentiate among aquatic plant species composition, areal density and thickness in cases where hyperspectral analysis yielded potentially ambiguous interpretations. 3. We compared the NDVI derived from IKONOS imagery with the in situ, hyperspectral‐derived NDVI. The IKONOS‐based images were also compared to data obtained through routine visual observations. Our results confirmed that aquatic species composition alters spectral signatures and affects the accuracy of remote sensing of aquatic plant density. The results also demonstrated that the NGAI has apparent advantages in estimating density over the NDVI and the D_H. 4. In the feature space of the three indices, 3D scatter plot analysis revealed that hyperspectral data can differentiate several aquatic plant associations. High‐resolution multispectral imagery provided useful information to distinguish among biophysical aquatic plant characteristics. Classification analysis indicated that using satellite imagery to assess Lemna coverage yielded an overall agreement of 79% with visual observations and >90% agreement for the densest aquatic plant coverages. 5. Interpretation of biophysical parameters derived from high‐resolution satellite or airborne imagery should prove to be a valuable approach for assessing the effectiveness of management practices for controlling aquatic plant growth in inland waters, as well as for routine monitoring of aquatic plants in lakes and suitable lentic environments.     

Autofluorescence hyperspectral imaging of radiofrequency ablation lesions in porcine cardiac tissue

Radiofrequency ablation (RFA) is a widely used treatment for atrial fibrillation, the most common cardiac arrhythmia. Here, we explore autofluorescence hyperspectral imaging (aHSI) as a method to visualize RFA lesions and interlesional gaps in the highly collagenous left atrium. RFA lesions made on the endocardial surface of freshly excised porcine left atrial tissue were illuminated by UV light (365 nm), and hyperspectral datacubes were acquired over the visible range (420–720 nm). Linear unmixing was used to delineate RFA lesions from surrounding tissue, and lesion diameters derived from unmixed component images were quantitatively compared to gross pathology. RFA caused two consistent changes in the autofluorescence emission profile: a decrease at wavelengths below 490 nm (ascribed to a loss of endogenous NADH) and an increase at wavelengths above 490 nm (ascribed to increased scattering). These spectral changes enabled high resolution, in situ delineation of RFA lesion boundaries without the need for additional staining or exogenous markers. Our results confirm the feasibility of using aHSI to visualize RFA lesions at clinically relevant locations. If integrated into a percutaneous visualization catheter, aHSI would enable widefield optical surgical guidance during RFA procedures and could improve patient outcome by reducing atrial fibrillation recurrence.

     

Dual light‐emitting diode‐based multichannel microscopy for whole‐slide multiplane,multispectral and phase imaging

We report the development of a multichannel microscopy for whole‐slide multiplane, multispectral and phase imaging. We use trinocular heads to split the beam path into 6 independent channels and employ a camera array for parallel data acquisition, achieving a maximum data throughput of approximately 1 gigapixel per second. To perform single‐frame rapid autofocusing, we place 2 near‐infrared light‐emitting diodes (LEDs) at the back focal plane of the condenser lens to illuminate the sample from 2 different incident angles. A hot mirror is used to direct the near‐infrared light to an autofocusing camera. For multiplane whole‐slide imaging (WSI), we acquire 6 different focal planes of a thick specimen simultaneously. For multispectral WSI, we relay the 6 independent image planes to the same focal position and simultaneously acquire information at 6 spectral bands. For whole‐slide phase imaging, we acquire images at 3 focal positions simultaneously and use the transport‐of‐intensity equation to recover the phase information. We also provide an open‐source design to further increase the number of channels from 6 to 15. The reported platform provides a simple solution for multiplexed fluorescence imaging and multimodal WSI. Acquiring an instant focal stack without z‐scanning may also enable fast 3‐dimensional dynamic tracking of various biological samples.      

Automatic identification and quantitative morphometry of unstained spinal nerve using molecular hyperspectral imaging technology

Quantitative observation of nerve fiber sections is often complemented by morphological analysis in both research and clinical condition. However, existing manual or semi-automated methods are tedious and labour intensive, fully automated morphometry methods are complicated as the information of color or gray images captured by traditional microscopy is limited. Moreover, most of the methods are time-consuming as the nerve sections need to be stained with some reagents before observation. To overcome these shortcomings, a molecular hyperspectral imaging system is developed and used to observe the spinal nerve sections. The molecular hyperspectral images contain both the structural and biochemical information of spinal nerve sections which is very useful for automatic identification and quantitative morphological analysis of nerve fibers. This characteristic makes it possible for researchers to observe the unstained spinal nerve and live cells in their native environment. To evaluate the performance of the new method, the molecular hyperspectral images were captured and the improved spectral angle mapper algorithm was proposed and used to segment the myelin contours. Then the morphological parameters such as myelin thickness and myelin area were calculated and evaluated. With these morphological parameters, the three dimension surface view images were drawn to help the investigators observe spinal nerve at different angles. The experiment results show that the hyperspectral based method has the potential to identify the spinal nerve more accurate than the traditional method as the new method contains both the spectral and spatial information of nerve sections.     

Automatic detection of basal cell carcinoma by hyperspectral imaging

The purpose of this study was to test the ability of hyperspectral imaging (HSI) combined with unsupervised anomaly detectors to automatically differentiate basal cell carcinoma (BCC) from normal skin. Hyperspectral images of the face of a female patient with a BCC of the lower lip were acquired using a visible/near-infrared HSI system and two anomaly detection algorithms (Reed-Xiaoli and Reed-Xiaoli/Uniform Target hybrid anomaly detectors) were used to detect pathological tissue from normal skin. The results revealed that the receiver operating characteristic curve of the Reed-Xiaoli/Uniform Target hybrid detector was higher than that of the Reed-Xiaoli detector in the range of false positive rates between 0 and 0.8. The area under curve values were good (0.7074 and 0.8607, respectively) with Reed-Xiaoli/Uniform Target hybrid detector performing better. In conclusion, HSI combined with either of two anomaly detectors can play a promising role in the automated screening of BCC.     

Scattering based hyperspectral imaging of plasmonic nanoplate clusters towards biomedical applications

A new optical scattering contrast‐agent based on polymer‐nanoparticle encapsulated silver nanoplates (PESNs) is presented. Silver nanoplates were chosen due to the flexibility of tuning their plasmon frequencies. The polymer coating preserves their physical and optical properties and confers other advantages such as controlled contrast agent delivery. Finite difference time domain (FDTD) simulations model the interaction of light with the nanoplates in different orientations in the cluster. Hyperspectral dark field microscopy (HYDFM) observes the scattering spectra of the PESNs. An unsupervised sequential maximum angle convex cone (SMACC) image analysis resolves spectral endmembers corresponding to different stacking orientations of the nanoplates. The orientation‐dependent endmembers qualitatively agree with the FDTD results. For contrast enhancement, the uptake and spatial distribution of PESNs are demonstrated by an HYDFM study of single melanoma cells to result in an enhanced contrast of up to 400%. A supervised spatial mapping of the endmembers obtained by the unsupervised SMACC algorithm reveals spatial distributions of PESNs with various clustering orientations of encapsulated nanoplates. Our study demonstrates tunability in plasmonics properties in clustered metal nanoparticles and its utility for the development of scatter‐based imaging contrast agents for a broad range of applications, including studies of single cells and other biomedical systems.

     

Shedding light on fibered confocal fluorescence microscopy: Applications in biomedical imaging and therapies

Discoveries of major importance in life sciences and preclinical research are linked to the invention of microscopes that enable imaging of cells and their microstructures. Imaging technologies involving in vivo procedures using fluorescent dyes that permit labelling of cells have been developed over the last two decades. Fibered confocal fluorescence microscopy (FCFM) is an imaging technology equipped with fiber‐optic probes to deliver light to organs and tissues of live animals. This enables not only in vivo detection of fluorescent signals and visualization of cells, but also the study of dynamic processes, such cell proliferation, apoptosis and angiogenesis, under physiological and pathological conditions. This will allow the diagnosis of diseased organs and tissues and the evaluation of the efficacy of new therapies in animal models of human diseases. The aim of this report is to shed light on FCFM and its potential medical applications and discusses some factors that compromise the reliability and reproducibility of monitoring biological processes by FCFM. This report also highlights the issues concerning animal experimentation and welfare, and the contributions of FCFM to the 3Rs principals, replacement, reduction and refinement.      

Understanding tissue specific compositions of bioenergy feedstocks through hyperspectral Raman imaging

Hyperspectral Raman imaging was used to study the tissue/cell type specific distribution of lignin and cellulose polymers within the plant cell walls. Distinct differences in cell wall compositions were identified between two potential bioenergy feedstocks: corn stover and Eucalyptus globulus. Characteristic bands of 627, 1,175, 1,206, and 1,428 cm⁻¹ were only observed for corn stover and 1,381 cm⁻¹ was only present in E. globulus. One‐dimensional and two‐dimensional chemical maps of lignin and cellulose were generated for the stem of corn stover, ranging from the epidermis to the pith area and revealed that lignin and cellulose abundance varies significantly among different cell types in the following order: sclerenchyma cells and tracheids (∼5 times) > epidermal cells (∼3 times) > bundle sheath cells > parenchyma cells. The Raman mapping methods developed on corn stover were also validated on E. globulus and clearly highlighted their difference in lignin composition. Biotechnol. Bioeng. 2011;108: 286–295. © 2010 Wiley Periodicals, Inc.     

First experience in clinical application of hyperspectral endoscopy for evaluation of colonic polyps

Early detection and resection of adenomatous polyps prevents their progression to colorectal cancer (CRC), significantly improving patient outcomes. Polyps are typically identified and removed during white-light colonoscopy. Unfortunately, the rate of interval cancers that arise between CRC screening events remains high, linked to poor visualization of polyps during screening and incomplete polyp removal. Here, we sought to evaluate the potential of a hyperspectral endoscope (HySE) to enhance polyp discrimination for detection and resection. We designed, built and tested a new compact HySE in a proof-of-concept clinical study. We successfully collected spectra from three tissue types in seven patients undergoing routine colonoscopy screening. The acquired spectral data from normal tissue and polyps, both pre- and post- resection, were subjected to quantitative analysis using spectral angle mapping and machine learning, which discriminated the data by tissue type, meriting further investigation of HySE as a clinical tool.     

Reflection‐mode switchable subwavelength Bessel‐beam and Gaussian‐beam photoacoustic microscopy in vivo

We have developed a reflection‐mode switchable subwavelength Bessel‐beam (BB) and Gaussian‐beam (GB) photoacoustic microscopy (PAM) system. To achieve both reflection‐mode and high resolution, we tightly attached a very small ultrasound transducer to an optical objective lens with numerical aperture of 1.0 and working distance of 2.5 mm. We used axicon and an achromatic doublet in our system to obtain the extended depth of field (DOF) of the BB. To compare the DOF performance achieved with our BB‐PAM system against GB‐PAM system, we designed our system so that the GB can be easily generated by simply removing the lenses. Using a 532 nm pulse laser, we achieved the lateral resolutions of 300 and 270 nm for BB‐PAM and GB‐PAM, respectively. The measured DOF of BB‐PAM was approximately 229 μm, which was about 7× better than that of GB‐PAM. We imaged the vasculature of a mouse ear using BB‐PAM and GB‐PAM and confirmed that the DOF of BB‐PAM is much better than the DOF of GB‐PAM. Thus, we believe that the high resolution achieved at the extended DOF by our system is very practical for wide range of biomedical research including red blood cell (RBC) migration in blood vessels at various depths and observation of cell migration or cell culture.      

Enhanced spatial resolution for snapshot hyperspectral imaging of blood perfusion and melanin information within human tissue

We report a reconstruction method to achieve high spatial resolution for hyperspectral imaging of chromophore features in skin in vivo. The method utilizes an established structure‐adaptive normalized convolution algorithm to reconstruct high spatial resolution of hyperspectral images from snapshot low‐resolution hyperspectral image sequences captured by a snapshot spectral camera. The reconstructed images at chromophore‐sensitive wavebands are used to map the skin features of interest. We demonstrate the method experimentally by mapping the blood perfusion and melanin features (moles) on the facial skin. The method relaxes the constrains of the relatively low spatial resolution in the snapshot hyperspectral camera, making it more usable in imaging applications.     

STED microscopy of living cells – new frontiers in membrane and neurobiology

Recent developments in fluorescence far‐field microscopy such as STED microscopy have accomplished observation of the living cell with a spatial resolution far below the diffraction limit. Here, we briefly review the current approaches to super‐resolution optical microscopy and present the implementation of STED microscopy for novel insights into live cell mechanisms, with a focus on neurobiology and plasma membrane dynamics.     

Dynamic imaging of zebrafish heart with multi-planar light sheet microscopy

Light sheet fluorescence microscopy has become a research hotspot in biomedicine because of low phototoxicity, high speed, and high resolution. However, the conventional methods to acquire three-dimensional spatial information are mainly based on scanning, which inevitably increases photodamage and is not real-time. Here, we propose a method to generate controllable multi-planar illumination with a dielectric isosceles triangular array and a design of multi-planar light sheet fluorescence microscopy system. We carry out experiments of three-dimensional illumination beam measurement, volumetric imaging of fluorescent microspheres, and dynamic in vivo imaging of zebrafish heart to evaluate the performance of this system. In addition, we apply this system to study the effects of bisphenol fluorene on the heart shape and heart-beating rate of zebrafish. Our experiment results indicate that the multi-planar light sheet microscopy system provides a novel and feasible method for three-dimensional selected plane imaging and low-phototoxicity in vivo imaging.     

Diagnosis of early gastric cancer based on fluorescence hyperspectral imaging technology combined with partial‐least‐square discriminant analysis and support vector machine

This study investigated the feasibility of using fluorescence hyperspectral imaging technology to diagnose of early‐stage gastric cancer. Fluorescence spectral images of 76 patients who were pathologically diagnosed as non‐atrophic gastritis, premalignant lesions and gastric cancer were collected. Fluorescence spectra at 100‐pixel points were randomly extracted after binarization. Diagnostic models of non‐atrophic gastritis, premalignant lesions and gastric cancer were constructed through partial‐least‐square discriminant analysis (PLS‐DA) and support vector machine (SVM) algorithms. The prediction effects of PLS‐DA and SVM models were compared. Results showed that the average spectra of normal, precancerous and gastric cancer tissues significantly differed at 496, 546, 640 and 670 nm, and regular changes in fluorescence intensity at 546 nm were in the following order: normal > precancerous lesions > gastric cancer. Additionally, the effect of the diagnostic model established by SVM is significantly better than PLS‐DA which accuracy, specificity and sensitivity are above 94%. Experimental results revealed that the fast diagnostic model of early gastric cancer by combining fluorescence hyperspectral imaging technology and improved SVM was effective and feasible, thereby providing an accurate and rapid method for diagnosing early‐stage gastric cancer.      

Multivariate analysis of Brillouin imaging data by supervised and unsupervised learning

Brillouin imaging relies on the reliable extraction of subtle spectral information from hyperspectral datasets. To date, the mainstream practice has been to use line fitting of spectral features to retrieve the average peak shift and linewidth parameters. Good results, however, depend heavily on sufficient signal-to-noise ratio and may not be applicable in complex samples that consist of spectral mixtures. In this work, we thus propose the use of various multivariate algorithms that can be used to perform supervised or unsupervised analysis of the hyperspectral data, with which we explore advanced image analysis applications, namely unmixing, classification and segmentation in a phantom and live cells. The resulting images are shown to provide more contrast and detail, and obtained on a timescale ∼10² faster than fitting. The estimated spectral parameters are consistent with those calculated from pure fitting.     

Label‐free and non‐invasive discrimination of HaCaT and melanoma cells in a co‐culture model by hyperspectral confocal reflectance microscopy

A novel hyperspectral confocal microscopy method to separate different cell populations in a co‐culture model is presented here. The described methodological and instrumental approach allows discrimination of different cell types using a non‐invasive, label free method with good accuracy with a single cell resolution. In particular, melanoma cells are discriminated from HaCaT cells by hyperspectral confocal imaging, principal component analysis and optical frequencies signing, as confirmed by fluorescence labelling cross check. The identification seems to be quite robust to be insensitive to the cellular shape within the studied samples, enabling to separate cells according to their cytotype down to a single cell sensitivity.

Set of hyperspectral images of melanoma‐keratinocytes co‐culture model (left), score plot of principal component analysis and spectral analysis of principal components coefficients (center), label‐free spectral identification of cell populations (right).