Real‐time augmented reality for delineation of surgical margins during neurosurgery using autofluorescence lifetime contrast

Current clinical brain imaging techniques used for surgical planning of tumor resection lack intraoperative and real‐time feedback; hence surgeons ultimately rely on subjective evaluation to identify tumor areas and margins. We report a fluorescence lifetime imaging (FLIm) instrument (excitation: 355 nm; emission spectral bands: 390/40 nm, 470/28 nm, 542/50 nm and 629/53 nm) that integrates with surgical microscopes to provide real‐time intraoperative augmentation of the surgical field of view with fluorescent derived parameters encoding diagnostic information. We show the functionality and safety features of this instrument during neurosurgical procedures in patients undergoing craniotomy for the resection of brain tumors and/or tissue with radiation damage. We demonstrate in three case studies the ability of this instrument to resolve distinct tissue types and pathology including cortex, white matter, tumor and radiation‐induced necrosis. In particular, two patients with effects of radiation‐induced necrosis exhibited longer fluorescence lifetimes and increased optical redox ratio on the necrotic tissue with respect to non‐affected cortex, and an oligodendroglioma resected from a third patient reported shorter fluorescence lifetime and a decrease in optical redox ratio than the surrounding white matter. These results encourage the use of FLIm as a label‐free and non‐invasive intraoperative tool for neurosurgical guidance.     

Lightsheet fluorescence lifetime imaging microscopy with wide‐field time‐correlated single photon counting

We report on wide‐field time‐correlated single photon counting (TCSPC)‐based fluorescence lifetime imaging microscopy (FLIM) with lightsheet illumination. A pulsed diode laser is used for excitation, and a crossed delay line anode image intensifier, effectively a single‐photon sensitive camera, is used to record the position and arrival time of the photons with picosecond time resolution, combining low illumination intensity of microwatts with wide‐field data collection. We pair this detector with the lightsheet illumination technique, and apply it to 3D FLIM imaging of dye gradients in human cancer cell spheroids, and C. elegans.     

Implantable self‐aligning fiber‐optic optomechanical devices for in vivo intraocular pressure‐sensing in artificial cornea

Artificial cornea is an effective treatment of corneal blindness. Yet, intraocular pressure (IOP) measurements for glaucoma monitoring remain an urgent unmet need. Here, we present the integration of a fiber‐optic Fabry‐Perot pressure sensor with an FDA‐approved keratoprosthesis for real‐time IOP measurements using a novel strategy based on optical‐path self‐alignment with micromagnets. Additionally, an alternative noncontact sensor‐interrogation approach is demonstrated using a bench‐top optical coherence tomography system. We show stable pressure readings with low baseline drift (<2.8 mm Hg) for >4.5 years in vitro and efficacy in IOP interrogation in vivo using fiber‐optic self‐alignment, with good initial agreement with the actual IOP. Subsequently, IOP drift in vivo was due to retroprosthetic membrane (RPM) formation on the sensor secondary to surgical inflammation (more severe in the current pro‐fibrotic rabbit model). This study paves the way for clinical adaptation of optical pressure sensors with ocular implants, highlighting the importance of controlling RPM in clinical adaptation.     

Two‐ and three‐photon absorption cross‐section characterization for high‐brightness,cell‐specific multiphoton fluorescence brain imaging

We demonstrate an accurate quantitative characterization of absolute two‐ and three‐photon absorption (2PA and 3PA) action cross sections of a genetically encodable fluorescent marker Sypher3s. Both 2PA and 3PA action cross sections of this marker are found to be remarkably high, enabling high‐brightness, cell‐specific two‐ and three‐photon fluorescence brain imaging. Brain imaging experiments on sliced samples of rat's cortical areas are presented to demonstrate these imaging modalities. The 2PA action cross section of Sypher3s is shown to be highly sensitive to the level of pH, enabling pH measurements via a ratiometric readout of the two‐photon fluorescence with two laser excitation wavelengths, thus paving the way toward fast optical pH sensing in deep‐tissue experiments.     

Pseudo‐real‐time retinal layer segmentation for high‐resolution adaptive optics optical coherence tomography

We present a pseudo‐real‐time retinal layer segmentation for high‐resolution Sensorless Adaptive Optics‐Optical Coherence Tomography (SAO‐OCT). Our pseudo‐real‐time segmentation method is based on Dijkstra's algorithm that uses the intensity of pixels and the vertical gradient of the image to find the minimum cost in a geometric graph formulation within a limited search region. It segments six retinal layer boundaries in an iterative process according to their order of prominence. The segmentation time is strongly correlated to the number of retinal layers to be segmented. Our program permits en face images to be extracted during data acquisition to guide the depth specific focus control and depth dependent aberration correction for high‐resolution SAO‐OCT systems. The average processing times for our entire pipeline for segmenting six layers in a retinal B‐scan of 496 × 400 and 240 × 400 pixels are around 25.60 and 13.76 ms, respectively. When reducing the number of layers segmented to only two layers, the time required for a 240 × 400 pixel image is 8.26 ms.     

Optimal imaging windows of indocyanine green‐assisted near‐infrared dental imaging with rat model and its comparison to X‐ray imaging

In this study, we used rat animal model to compare the efficiency of indocyanine green (ICG)‐assisted dental near‐infrared fluorescence imaging with X‐ray imaging, and we optimized the imaging window for both unerupted and erupted molars. The results show that the morphology of the dental structures was observed clearly from ICG‐assisted dental images (especially through the endoscope). A better image contrast was easily acquired at the short imaging windows (<10 minutes) for unerupted and erupted molars. For unerupted molars, there is another optimized imaging window (48‐96 hours) with a prominent glow‐in‐the‐dark effect: only the molars remain bright. This study also revealed that the laser ablation of dental follicles can disrupt the molar development, and our method is able to efficiently detect laser‐treated molars and acquire the precise morphology. Thus, ICG‐assisted dental imaging has the potential to be a safer and more efficient imaging modality for the real‐time diagnosis of dental diseases.     

Real‐time video mosaicking to guide handheld in vivo microscopy

Handheld and endoscopic optical‐sectioning microscopes are being developed for noninvasive screening and intraoperative consultation. Imaging a large extent of tissue is often desired, but miniature in vivo microscopes tend to suffer from limited fields of view. To extend the imaging field during clinical use, we have developed a real‐time video mosaicking method, which allows users to efficiently survey larger areas of tissue. Here, we modified a previous post‐processing mosaicking method so that real‐time mosaicking is possible at >30 frames/second when using a device that outputs images that are 400 × 400 pixels in size. Unlike other real‐time mosaicking methods, our strategy can accommodate image rotations and deformations that often occur during clinical use of a handheld microscope. We perform a feasibility study to demonstrate that the use of real‐time mosaicking is necessary to enable efficient sampling of a desired imaging field when using a handheld dual‐axis confocal microscope.     

In vivo study of metabolic dynamics and heterogeneity in brown and beige fat by label‐free multiphoton redox and fluorescence lifetime microscopy

In this work, the metabolic characteristics of adipose tissues in live mouse model were investigated using a multiphoton redox ratio and fluorescence lifetime imaging technology. By analyzing the intrinsic fluorescence of metabolic coenzymes, we measured the optical redox ratios of adipocytes in vivo and studied their responses to thermogenesis. The fluorescence lifetime imaging further revealed changes in protein bindings of metabolic coenzymes in the adipocytes during thermogenesis. Our study uncovered significant heterogeneity in the cellular structures and metabolic characteristics of thermogenic adipocytes in brown and beige fat. Subgroups of brown and beige adipocytes were identified based on the distinct lipid size distributions, redox ratios, fluorescence lifetimes and thermogenic capacities. The results of our study show that this label‐free imaging technique can shed new light on in vivo study of metabolic dynamics and heterogeneity of adipose tissues in live organisms.     

Cancer cell‐specific protein delivery by optoporation with laser‐irradiated gold nanorods

The delivery of macromolecules into living cells is challenging since in most cases molecules are endocytosed and remain in the endo‐lysosomal pathway where they are degraded before reaching their target. Here, a method is presented to selectively improve cell membrane permeability by nanosecond laser irradiation of gold nanorods (GNRs) with visible or near‐infrared irradiation in order to deliver proteins across the plasma membrane, avoiding the endo lysosomal pathway. GNRs were labeled with the anti‐EGFR (epidermal growth factor receptor) antibody Erbitux to target human ovarian carcinoma cells OVCAR‐3. Irradiation with nanosecond laser pulses at wavelengths of 532 nm or 730 nm is used for transient permeabilization of the cell membranes. As a result of the irradiation, the uptake of an anti‐Ki‐67 antibody was observed in about 50 % of the cells. The results of fluorescence lifetime imaging show that the GNR detached from the membrane after irradiation.     

Wide‐field monitoring and real‐time local recording of microvascular networks on small animals with a dual‐raster‐scanned photoacoustic microscope

Photoacoustic microscopy (PAM) provides a new method for the imaging of small‐animals with high‐contrast and deep‐penetration. However, the established PAM systems have suffered from a limited field‐of‐view or imaging speed, which are difficult to both monitor wide‐field activity of organ and record real‐time change of local tissue. Here, we reported a dual‐raster‐scanned photoacoustic microscope (DRS‐PAM) that integrates a two‐dimensional motorized translation stage for large field‐of‐view imaging and a two‐axis fast galvanometer scanner for real‐time imaging. The DRS‐PAM provides a flexible transition from wide‐field monitoring the vasculature of organs to real‐time imaging of local dynamics. To test the performance of DRS‐PAM, clear characterization of angiogenesis and functional detail was illustrated, hemodynamic activities of vasculature in cerebral cortex of a mouse were investigated. Furthermore, response of tumor to treatment were successfully monitored during treatment. The experimental results demonstrate the DRS‐PAM holds the great potential for biomedical research of basic biology.     

Characterization of blood coagulation dynamics and oxygenation in ex‐vivo retinal vessels by fluorescence hyperspectral imaging

Blood coagulation mechanisms forming a blood clot and preventing hemorrhage have been extensively studied in the last decades. Knowing the mechanisms behind becomes very important particularly in the case of blood vessel diseases. Real‐time and accurate diagnostics accompanied by the therapy are particularly needed, for example, in diseases related to retinal vasculature. In our study, we employ for the first time fluorescence hyperspectral imaging (fHSI) combined with the spectral analysis algorithm concept to assess physical as well as functional information of blood coagulation in real‐time. By laser‐induced local disruption of retinal vessels to mimic blood leaking and subsequent coagulation and a proper fitting algorithm, we were able to reveal and quantify the extent of local blood coagulation through direct identification of the change of oxyhemoglobin concentration within few minutes. We confirmed and illuminated the spatio‐temporal evolution of the essential role of erythrocytes in the coagulation cascade as the suppliers of oxygenated hemoglobin. By additional optical tweezers force manipulation, we showed immediate aggregation of erythrocytes at the coagulation site. The presented fluorescence‐based imaging concept could become a valuable tool in various blood coagulation diagnostics as well as theranostic systems if coupled with the laser therapy.     

Autofluorescence imaging identifies tumor cell‐cycle status on a single‐cell level

The goal of this study is to validate fluorescence intensity and lifetime imaging of metabolic co‐enzymes NAD(P)H and FAD (optical metabolic imaging, or OMI) as a method to quantify cell‐cycle status of tumor cells. Heterogeneity in tumor cell‐cycle status (e. g. proliferation, quiescence, apoptosis) increases drug resistance and tumor recurrence. Cell‐cycle status is closely linked to cellular metabolism. Thus, this study applies cell‐level metabolic imaging to distinguish proliferating, quiescent, and apoptotic populations. Two‐photon microscopy and time‐correlated single photon counting are used to measure optical redox ratio (NAD(P)H fluorescence intensity divided by FAD intensity), NAD(P)H and FAD fluorescence lifetime parameters. Redox ratio, NAD(P)H and FAD lifetime parameters alone exhibit significant differences (p<0.05) between population means. To improve separation between populations, linear combination models derived from partial least squares ‐ discriminant analysis (PLS‐DA) are used to exploit all measurements together. Leave‐one‐out cross validation of the model yielded high classification accuracies (92.4 and 90.1 % for two and three populations, respectively). OMI and PLS‐DA also identifies each sub‐population within heterogeneous samples. These results establish single‐cell analysis with OMI and PLS‐DA as a label‐free method to distinguish cell‐cycle status within intact samples. This approach could be used to incorporate cell‐level tumor heterogeneity in cancer drug development.

     

Toward the measurement of multiple fluorescence lifetimes in flow cytometry: maximizing multi‐harmonic content from cells and microspheres

Flow cytometry is a powerful means for in vitro cellular analyses where multi‐fluorescence and multi‐angle light scattering can indicate unique biochemical or morphological features of single cells. Yet, to date, flow cytometry systems have lacked the ability to capture complex fluorescence dynamics due to the transient nature of flowing cells. In this contribution we introduce a simple approach for measuring multiple fluorescence lifetimes from a single cytometric event. We leverage square wave modulation, Fourier analysis, and high frequency digitization and show the ability to resolve more than one fluorescence lifetime from fluorescently‐labelled cells and microspheres.

Illustration of a flow cytometer capable of capturing multiple fluorescence lifetime measurements; creating potential for multi‐parametric, time‐resolved signals to be captured for every color channel.     

Reducing data acquisition for light‐sheet microscopy by extrapolation between imaged planes

Light‐sheet fluorescence microscopy (LSFM) is a powerful technique that can provide high‐resolution images of biological samples. Therefore, this technique offers significant improvement for three‐dimensional (3D) imaging of living cells. However, producing high‐resolution 3D images of a single cell or biological tissues, normally requires high acquisition rate of focal planes, which means a large amount of sample sections. Consequently, it consumes a vast amount of processing time and memory, especially when studying real‐time processes inside living cells. We describe an approach to minimize data acquisition by interpolation between planes using a phase retrieval algorithm. We demonstrate this approach on LSFM data sets and show reconstruction of intermediate sections of the sparse samples. Since this method diminishes the required amount of acquisition focal planes, it also reduces acquisition time of samples as well. Our suggested method has proven to reconstruct unacquired intermediate planes from diluted data sets up to 10× fold. The reconstructed planes were found correlated to the original preacquired samples (control group) with correlation coefficient of up to 90%. Given the findings, this procedure appears to be a powerful method for inquiring and analyzing biological samples.     

Tissue clearing‐based method for unobstructed three‐dimensional imaging of mouse penis with subcellular resolution

Although mice are widely used to elucidate factors contributing to penile disorders and develop treatment options, quantification of tissue changes upon intervention is either limited to minuscule tissue volume (histology) or acquired with limited spatial resolution (MRI/CT). Thus, imaging method suitable for expeditious acquisition of the entire mouse penis with subcellular resolution is described that relies on both aqueous‐ (clear, unobstructed brain imaging cocktails and computational analysis) and solvent‐based (fluorescence‐preserving capability imaging of solvent‐cleared organs) tissue optical clearing (TOC). The combined TOC approach allows to image mouse penis innervation and vasculature with unprecedented detail and, for the first time, reveals the three‐dimensional structure of murine penis fibrocartilage.     

Dual‐ and single‐shot susceptibility ratio measurements with circular polarizations in second‐harmonic generation microscopy

Polarization‐resolved second‐harmonic generation (P‐SHG) microscopy is a technique capable of characterizing nonlinear optical properties of noncentrosymmetric biomaterials by extracting the nonlinear susceptibility tensor components ratio , with z‐axis parallel and x‐axis perpendicular to the C₆ symmetry axis of molecular fiber, such as a myofibril or a collagen fiber. In this paper, we present two P‐SHG techniques based on incoming and outgoing circular polarization states for a fast extraction of : A dual‐shot configuration where the SHG circular anisotropy generated using incident right‐ and left‐handed circularly‐polarized light is measured; and a single‐shot configuration for which the SHG circular anisotropy is measured using only one incident circular polarization state. These techniques are used to extract the of myosin fibrils in the body wall muscles of Drosophila melanogaster larva. The results are in good agreement with values obtained from the double Stokes‐Mueller polarimetry. The dual‐ and single‐shot circular anisotropy measurements can be used for fast imaging that is independent of the in‐plane orientation of the sample. They can be used for imaging of contracting muscles, or for high throughput imaging of large sample areas.     

Mesoscopic fluorescence lifetime imaging: Fundamental principles,clinical applications and future directions

Fluorescence lifetime imaging (FLIm) is an optical spectroscopic imaging technique capable of real-time assessments of tissue properties in clinical settings. Label-free FLIm is sensitive to changes in tissue structure and biochemistry resulting from pathological conditions, thus providing optical contrast to identify and monitor the progression of disease. Technical and methodological advances over the last two decades have enabled the development of FLIm instrumentation for real-time, in situ, mesoscopic imaging compatible with standard clinical workflows. Herein, we review the fundamental working principles of mesoscopic FLIm, discuss the technical characteristics of current clinical FLIm instrumentation, highlight the most commonly used analytical methods to interpret fluorescence lifetime data and discuss the recent applications of FLIm in surgical oncology and cardiovascular diagnostics. Finally, we conclude with an outlook on the future directions of clinical FLIm.     

Front Cover: Wide‐field monitoring and real‐time local recording of microvascular networks on small animals with a dual‐raster‐scanned photoacoustic microscope

A dual‐raster‐scanned photoacoustic microscope (DRS‐PAM) was reported, which integrates a two‐dimensional motorized translation stage for large field‐of‐view imaging and a two‐axis fast galvanometer scanner for real‐time imaging. The DRS‐PAM provides a flexible transition from wide‐field monitoring the vasculature of organs to real‐time imaging of local dynamics. Further details can be found in the article by Fei Yang, Zhiyang Wang, Wuyu Zhang, et al. ( e202000022 ).

     

Fiber‐based fluorescence lifetime imaging of recellularization processes on vascular tissue constructs

New techniques able to monitor the maturation of tissue engineered constructs over time are needed for a more efficient control of developmental parameters. Here, a label‐free fluorescence lifetime imaging (FLIm) approach implemented through a single fiber‐optic interface is reported for nondestructive in situ assessment of vascular biomaterials. Recellularization processes of antigen removed bovine pericardium scaffolds with endothelial cells and mesenchymal stem cells were evaluated on the serous and the fibrous sides of the scaffolds, 2 distinct extracellular matrix niches, over the course of a 7 day culture period. Results indicated that fluorescence lifetime successfully report cell presence resolved from extracellular matrix fluorescence. The recellularization process was more rapid on the serous side than on the fibrous side for both cell types, and endothelial cells expanded faster than mesenchymal stem cells on antigen‐removed bovine pericardium. Fiber‐based FLIm has the potential to become a nondestructive tool for the assessment of tissue maturation by allowing in situ imaging of intraluminal vascular biomaterials.      

Quadrature demodulation in line‐scan focal modulation microscopy for imaging three‐dimensional zebrafish neural structure

Visualizing biological processes in neuroscience requires in vivo functional imaging at single‐neuron resolution, high image acquisition speed and strong optical sectioning ability. However, due to light scattering of in tissue, very often conventional wide‐field fluorescence microscopes are unable to resolve cells in the presence of a strong out‐of‐focus background. Line‐scan focal modulation microscopy enables high temporal resolution and good optical sectioning ability at the same time. Here we demonstrate a quadrature demodulation method to extract the focal information with an extended frequency bandwidth and therefore higher spatial resolution. The performance of the demodulation scheme in line‐scan focal modulation microscope has been evaluated by performing imaging experiments with fluorescence beads and zebrafish neural structure. Reduced background, reduced artifacts and more detailed morphological information are evident in the obtained images.