High-speed volumetric imaging in vivo based on structured illumination microscopy with interleaved reconstruction

Wide-field fluorescence microscopy (WFFM) is widely adopted in biomedical studies, due to its high imaging speed over large field-of-views. However, WFFM is susceptible to out-of-focus background. To overcome this problem, structured illumination microscopy (SIM) was proposed as a wide-field, optical-sectioning technique, which needs multiple raw images for image reconstruction and thus has a lower imaging speed. Here we propose SIM with interleaved reconstruction, to make SIM of lossless speed. We apply this method in volumetric imaging of neural network dynamics in brains of zebrafish larva in vivo.     

Non-local means improves total-variation constrained photoacoustic image reconstruction

Photoacoustic/Optoacoustic tomography aims to reconstruct maps of the initial pressure rise induced by the absorption of light pulses in tissue. This reconstruction is an ill-conditioned and under-determined problem, when the data acquisition protocol involves limited detection positions. The aim of the work is to develop an inversion method which integrates denoising procedure within the iterative model-based reconstruction to improve quantitative performance of optoacoustic imaging. Among the model-based schemes, total-variation (TV) constrained reconstruction scheme is a popular approach. In this work, a two-step approach was proposed for improving the TV constrained optoacoustic inversion by adding a non-local means based filtering step within each TV iteration. Compared to TV-based reconstruction, inclusion of this non-local means step resulted in signal-to-noise ratio improvement of 2.5 dB in the reconstructed optoacoustic images.     

Double-flow convolutional neural network for rapid large field of view Fourier ptychographic reconstruction

Fourier ptychographic microscopy is a promising imaging technique which can circumvent the space-bandwidth product of the system and achieve a reconstruction result with wide field-of-view (FOV), high-resolution and quantitative phase information. However, traditional iterative-based methods typically require multiple times to get convergence, and due to the wave vector deviation in different areas, the millimeter-level full-FOV cannot be well reconstructed once and typically required to be separated into several portions with sufficient overlaps and reconstructed separately, which makes traditional methods suffer from long reconstruction time for a large-FOV (of the order of minutes) and limits the application in real-time large-FOV monitoring of live sample in vitro. Here we propose a novel deep-learning based method called DFNN which can be used in place of traditional iterative-based methods to increase the quality of single large-FOV reconstruction and reducing the processing time from 167.5 to 0.1125 second. In addition, we demonstrate that by training based on the simulation dataset with high-entropy property (Opt. Express 28, 24 152 [2020]), DFNN could has fine generalizability and little dependence on the morphological features of samples. The superior robustness of DFNN against noise is also demonstrated in both simulation and experiment. Furthermore, our model shows more robustness against the wave vector deviation. Therefore, we could achieve better results at the edge areas of a single large-FOV reconstruction. Our method demonstrates a promising way to perform real-time single large-FOV reconstructions and provides further possibilities for real-time large-FOV monitoring of live samples with sub-cellular resolution.     

Smartphone‐based spectral imaging otoscope: System development and preliminary study for evaluation of its potential as a mobile diagnostic tool

We develop a novel smartphone‐based spectral imaging otoscope for telemedicine and examine its capability for the mobile diagnosis of middle ear diseases. The device was applied to perform spectral imaging and analysis of an ear‐mimicking phantom and a normal and abnormal tympanic membrane for evaluation of its potential for the mobile diagnosis. Spectral classified images were obtained via online spectral analysis in a remote server. The phantom experimental results showed that it allowed us to distinguish four different fluids located behind a semitransparent membrane. Also, in the spectral classified images of normal ears (n = 3) and an ear with chronic otitis media (n = 1), the normal and abnormal regions in each ear could be quantitatively distinguished with high contrast. These preliminary results thus suggested that it might have the potentials for providing quantitative information for the mobile diagnosis of various middle ear diseases.     

Enhanced spatial resolution for snapshot hyperspectral imaging of blood perfusion and melanin information within human tissue

We report a reconstruction method to achieve high spatial resolution for hyperspectral imaging of chromophore features in skin in vivo. The method utilizes an established structure‐adaptive normalized convolution algorithm to reconstruct high spatial resolution of hyperspectral images from snapshot low‐resolution hyperspectral image sequences captured by a snapshot spectral camera. The reconstructed images at chromophore‐sensitive wavebands are used to map the skin features of interest. We demonstrate the method experimentally by mapping the blood perfusion and melanin features (moles) on the facial skin. The method relaxes the constrains of the relatively low spatial resolution in the snapshot hyperspectral camera, making it more usable in imaging applications.     

Development of an imaging device for label-free parathyroid gland identification and vascularity assessment

During thyroid surgeries, it is important for surgeons to accurately identify healthy parathyroid glands and assess their vascularity to preserve their function postoperatively, thus preventing hypoparathyroidism and hypocalcemia. Near infrared autofluorescence detection enables parathyroid identification, while laser speckle contrast imaging allows assessment of parathyroid vascularity. Here, we present an imaging system combining the two techniques to perform both functions, simultaneously and label-free. An algorithm to automate the segmentation of a parathyroid gland in the fluorescence image to determine its average speckle contrast is also presented, reducing a barrier to clinical translation. Results from imaging ex vivo tissue samples show that the algorithm is equivalent to manual segmentation. Intraoperative images from representative procedures are presented showing successful implementation of the device to identify and assess vascularity of healthy and diseased parathyroid glands.     

Real‐time video mosaicking to guide handheld in vivo microscopy

Handheld and endoscopic optical‐sectioning microscopes are being developed for noninvasive screening and intraoperative consultation. Imaging a large extent of tissue is often desired, but miniature in vivo microscopes tend to suffer from limited fields of view. To extend the imaging field during clinical use, we have developed a real‐time video mosaicking method, which allows users to efficiently survey larger areas of tissue. Here, we modified a previous post‐processing mosaicking method so that real‐time mosaicking is possible at >30 frames/second when using a device that outputs images that are 400 × 400 pixels in size. Unlike other real‐time mosaicking methods, our strategy can accommodate image rotations and deformations that often occur during clinical use of a handheld microscope. We perform a feasibility study to demonstrate that the use of real‐time mosaicking is necessary to enable efficient sampling of a desired imaging field when using a handheld dual‐axis confocal microscope.     

N2NSR-OCT: Simultaneous denoising and super-resolution in optical coherence tomography images using semisupervised deep learning

Optical coherence tomography (OCT) imaging shows a significant potential in clinical routines due to its noninvasive property. However, the quality of OCT images is generally limited by inherent speckle noise of OCT imaging and low sampling rate. To obtain high signal-to-noise ratio (SNR) and high-resolution (HR) OCT images within a short scanning time, we presented a learning-based method to recover high-quality OCT images from noisy and low-resolution OCT images. We proposed a semisupervised learning approach named N2NSR-OCT, to generate denoised and super-resolved OCT images simultaneously using up- and down-sampling networks (U-Net (Semi) and DBPN (Semi)). Additionally, two different super-resolution and denoising models with different upscale factors (2× and 4× ) were trained to recover the high-quality OCT image of the corresponding down-sampling rates. The new semisupervised learning approach is able to achieve results comparable with those of supervised learning using up- and down-sampling networks, and can produce better performance than other related state-of-the-art methods in the aspects of maintaining subtle fine retinal structures.     

High‐speed X‐ray‐induced luminescence computed tomography

X‐ray‐induced luminescence computed tomography (XLCT) is an emerging molecular imaging. Challenges in improving spatial resolution and reducing the scan time in a whole‐body field of view (FOV) still remain for practical in vivo applications. In this study, we present a novel XLCT technique capable of obtaining three‐dimensional (3D) images from a single snapshot. Specifically, a customed two‐planar‐mirror component is integrated into a cone beam XLCT imaging system to obtain multiple optical views of an object simultaneously. Furthermore, a compressive sensing based algorithm is adopted to improve the efficiency of 3D XLCT image reconstruction. Numerical simulations and experiments were conducted to validate the single snapshot X‐ray‐induced luminescence computed tomography (SS‐XLCT). The results show that the 3D distribution of the nanophosphor targets can be visualized much faster than conventional cone beam XLCT imaging method that was used in our comparisons while maintaining comparable spatial resolution as in conventional XLCT imaging. SS‐XLCT has the potential to harness the power of XLCT for rapid whole‐body in vivo molecular imaging of small animals.     

Ex vivo three-dimensional elemental imaging of mouse brain tissue block by laser-induced breakdown spectroscopy

Measurement and reconstruction of an elemental image of large brain tissue will be beneficial to the diagnosis of neurological brain diseases. Herein, laser-induced breakdown spectroscopy (LIBS) is introduced for three dimensional (3D) elemental analysis of paraffin-embedded mouse brain tissue blocks. It is used for the first time towards the mapping of mouse brain block samples. A micro-LIBS prototype is developed for brain elemental imaging and a layer-by-layer approach is used to reconstruct the 3D distribution of Ca, Mg, Na, Cu, and P in the brain tissue. Images are captured with 50 μm lateral resolution and 300 μm depth resolution. The images show that the reclamation area of the cortex surface is enriched with Ca and Mg. In contrast, the Cu distribution is circular and is found primarily in the entirety of the cerebral cortex for the paraffin-embedded brain samples. Elemental imaging results suggest that the highest P intensity is found in the cerebellum nearby the middle sagittal plane in the left-brain paraffin block. These preliminary results indicate that LIBS is a potentially powerful tool for elemental bioimaging of the whole brain and may further improve the understanding of complex brain mechanisms.     

DeconvTest: Simulation framework for quantifying errors and selecting optimal parameters of image deconvolution

Deconvolution is an essential step of image processing that aims to compensate for the image blur caused by the microscope's point spread function. With many existing deconvolution methods, it is challenging to choose the method and its parameters most appropriate for particular image data at hand. To facilitate this task, we developed DeconvTest: an open‐source Python‐based framework for generating synthetic microscopy images, deconvolving them with different algorithms, and quantifying reconstruction errors. In contrast to existing software, DeconvTest combines all components required to analyze deconvolution performance in a systematic, high‐throughput and quantitative manner. We demonstrate the power of the framework by using it to identify the optimal deconvolution settings for synthetic and real image data. Based on this, we provide a guideline for (a) choosing optimal values of deconvolution parameters for image data at hand and (b) optimizing imaging conditions for best results in combination with subsequent image deconvolution.     

Reducing data acquisition for light‐sheet microscopy by extrapolation between imaged planes

Light‐sheet fluorescence microscopy (LSFM) is a powerful technique that can provide high‐resolution images of biological samples. Therefore, this technique offers significant improvement for three‐dimensional (3D) imaging of living cells. However, producing high‐resolution 3D images of a single cell or biological tissues, normally requires high acquisition rate of focal planes, which means a large amount of sample sections. Consequently, it consumes a vast amount of processing time and memory, especially when studying real‐time processes inside living cells. We describe an approach to minimize data acquisition by interpolation between planes using a phase retrieval algorithm. We demonstrate this approach on LSFM data sets and show reconstruction of intermediate sections of the sparse samples. Since this method diminishes the required amount of acquisition focal planes, it also reduces acquisition time of samples as well. Our suggested method has proven to reconstruct unacquired intermediate planes from diluted data sets up to 10× fold. The reconstructed planes were found correlated to the original preacquired samples (control group) with correlation coefficient of up to 90%. Given the findings, this procedure appears to be a powerful method for inquiring and analyzing biological samples.     

Single-particle reconstruction using L(2)-gradient flow

In this paper, we present an iterative algorithm for reconstructing a three-dimensional density function from a set of two dimensional electron microscopy images. By minimizing an energy functional consisting of a fidelity term and a regularization term, an L²-gradient flow is derived. The flow is integrated by a finite element method in the spatial direction and an explicit Euler scheme in the temporal direction. Our method compares favorably with those of the weighted back projection, Fourier method, algebraic reconstruction technique and simultaneous iterative reconstruction technique.     

Application of mid-infrared microscopic imaging for the diagnosis and classification of human lymphomas

Mid-infrared (MIR) microscopic imaging of indolent and aggressive lymphomas was performed including formalin-fixed and paraffin-embedded samples of six follicular lymphomas and 12 diffuse large B-cell-lymphomas as well as reactive lymph nodes to investigate benefits and challenges for lymphoma diagnosis. MIR images were compared to defined pathological characteristics such as indolent versus aggressive versus reactive, germinal centre versus activated cell-of-origin (COO) subtypes, or a low versus a high proliferative index and level of PD-L1 expression. We demonstrated that MIR microscopic imaging can differentiate between reactive lymph nodes, indolent and aggressive lymphoma samples. Also, it has potential to be used in the subtyping of lymphomas, as shown with the differentiation between COO subtypes, the level of proliferation and PD-L1 expression. MIR microscopic imaging is a promising tool for diagnosis and subtyping of lymphoma and further evaluation is needed to fully explore the advantages and disadvantages of this method for pathological diagnosis.     

Histological classification of atherosclerotic arteries using high-speed confocal Raman microscopy with machine learning

Confocal Raman microscopy is a useful tool to observe composition and constitution of label-free samples at high spatial resolution. However, accurate characterization of microstructure of tissue and its application in diagnostic imaging are challenging due to weak Raman scattering signal and complex chemical composition of tissue. We have developed a method to improve imaging speed, diffraction efficiency, and spectral resolution of confocal Raman microscopy. In addition to the novel imaging technique, the machine learning method enables confocal Raman microscopy to visualize accurate histology of tissue sections. Here, we have demonstrated the performance of the proposed method by measuring histological classification of atherosclerotic arteries and compared the histological confocal Raman images with the conventional staining method. Our new confocal Raman microscopy enables us to comprehend the structure and biochemical composition of tissue and diagnose the buildup of atherosclerotic plaques in the arterial wall without labeling.     

Water Selective Imaging and bSSFP Banding Artifact Correction in Humans and Small Animals at 3T and 7T,Respectively

Introduction

The purpose of this paper is to develop an easy method to generate both fat signal and banding artifact free 3D balanced Steady State Free Precession (bSSFP) images at high magnetic field.

Methods

In order to suppress fat signal and bSSFP banding artifacts, two or four images were acquired with the excitation frequency of the water-selective binomial radiofrequency pulse set On Resonance or shifted by a maximum of 3/4TR. Mice and human volunteers were imaged at 7T and 3T, respectively to perform whole-body and musculoskeletal imaging. “Sum-Of-Square” reconstruction was performed and combined or not with parallel imaging.

Results

The frequency selectivity of 1-2-3-2-1 or 1-3-3-1 binomial pulses was preserved after (3/4TR) frequency shifting. Consequently, whole body small animal 3D imaging was performed at 7T and enabled visualization of small structures within adipose tissue like lymph nodes. In parallel, this method allowed 3D musculoskeletal imaging in humans with high spatial resolution at 3T. The combination with parallel imaging allowed the acquisition of knee images with ~500μm resolution images in less than 2min. In addition, ankles, full head coverage and legs of volunteers were imaged, demonstrating the possible application of the method also for large FOV.

Conclusion

In conclusion, this robust method can be applied in small animals and humans at high magnetic fields. The high SNR and tissue contrast obtained in short acquisition times allows to prescribe bSSFP sequence for several preclinical and clinical applications.     

Comparative study of deep neural networks with unsupervised Noise2Noise strategy for noise reduction of optical coherence tomography images

As a powerful diagnostic tool, optical coherence tomography (OCT) has been widely used in various clinical setting. However, OCT images are susceptible to inherent speckle noise that may contaminate subtle structure information, due to low-coherence interferometric imaging procedure. Many supervised learning-based models have achieved impressive performance in reducing speckle noise of OCT images trained with a large number of noisy-clean paired OCT images, which are not commonly feasible in clinical practice. In this article, we conducted a comparative study to investigate the denoising performance of OCT images over different deep neural networks through an unsupervised Noise2Noise (N2N) strategy, which only trained with noisy OCT samples. Four representative network architectures including U-shaped model, multi-information stream model, straight-information stream model and GAN-based model were investigated on an OCT image dataset acquired from healthy human eyes. The results demonstrated all four unsupervised N2N models offered denoised OCT images with a performance comparable with that of supervised learning models, illustrating the effectiveness of unsupervised N2N models in denoising OCT images. Furthermore, U-shaped models and GAN-based models using UNet network as generator are two preferred and suitable architectures for reducing speckle noise of OCT images and preserving fine structure information of retinal layers under unsupervised N2N circumstances.     

3D-Printed high-NA catadioptric thin lens for suppression of XPM background in Stimulated Raman Scattering microscopy

Stimulated Raman Scattering (SRS) is a fast chemical imaging technique with remarkable bioscience applications. Cross Phase Modulation (XPM) is a ubiquitous nonlinear phenomenon that can create spurious background signals that render difficult a high-contrast imaging in SRS measurements. The XPM-induced signal is usually suppressed using high numerical aperture (NA) microscope objectives or condensers to collect the transmitted excitation beam. However, these high NA optics feature short working distances, hence they are not compatible with stage-top incubators, that are necessary to perform live-cell time-lapse experiments in controlled environments. Here, we show a 3D printed high NA compact catadioptric lens that fits inside stage-top incubators and allows the collection of XPM-free SRS signals. The lens delivers SRS images and spectra with a quality comparable to a signal collection with a high-NA microscope objective. We also demonstrate the compatibility of the 3D printed lens with other nonlinear microscopies usually associated with SRS in multimodal microscopes.     

Multicomposite super‐resolution microscopy: Enhanced Airyscan resolution with radial fluctuation and sample expansions

Either modulated illumination or temporal fluctuation analysis can assist super‐resolution techniques in overcoming the diffraction limit of conventional optical microscopy. As they are not contradictory to each other, an effective combination of spatial and temporal super‐resolution mechanisms would further improve the resolution of fluorescent images. Here, a super‐resolution imaging method called fluctuation‐enhanced Airyscan technology (FEAST) is proposed, which achieves ~40 nm lateral imaging resolution and is useful for a range of fluorescent proteins and organic dyes. It was demonstrated not only to obtain different subcellular super‐resolution images, but also to improve the accuracy of counting the average human epidermal growth factor receptor 2 (HER2) copy number for diagnosis in breast cancer. Furthermore, the combination of FEAST and sample expansion microscopy (Ex‐FEAST) improves the lateral resolution to ~26 nm.     

Three-dimensional spatial reconstruction of coronary arteries based on fusion of intravascular optical coherence tomography and coronary angiography

We present a three-dimensional (3D) spatial reconstruction of coronary arteries based on fusion of intravascular optical coherence tomography (IVOCT) and digital subtraction angiography (DSA). Centerline of vessel in DSA images is exacted by multi-scale filtering, adaptive segmentation, morphology thinning and Dijkstra's shortest path algorithm. We apply the cross-correction between lumen shapes of IVOCT and DSA images and match their stenosis positions to realize co-registration. By matching the location and tangent direction of the vessel centerline of DSA images and segmented lumen coordinates of IVOCT along pullback path, 3D spatial models of vessel lumen are reconstructed. Using 1121 distinct positions selected from eight vessels, the correlation coefficient between 3D IVOCT model and DSA image in measuring lumen radius is 0.94% and 97.7% of the positions fall within the limit of agreement by Bland–Altman analysis, which means that the 3D spatial reconstruction IVOCT models and DSA images have high matching level.