High contrast imaging and flexible photomanipulation for quantitative in vivo multiphoton imaging with polygon scanning microscope

In this study, we introduce two key improvements that overcome limitations of existing polygon scanning microscopes while maintaining high spatial and temporal imaging resolution over large field of view (FOV). First, we proposed a simple and straightforward means to control the scanning angle of the polygon mirror to carry out photomanipulation without resorting to high speed optical modulators. Second, we devised a flexible data sampling method directly leading to higher image contrast by over 2‐fold and digital images with 100 megapixels (10 240 × 10 240) per frame at 0.25 Hz. This generates sub‐diffraction limited pixels (60 nm per pixels over the FOV of 512 μm) which increases the degrees of freedom to extract signals computationally. The unique combined optical and digital control recorded fine fluorescence recovery after localized photobleaching (r ~10 μm) within fluorescent giant unilamellar vesicles and micro‐vascular dynamics after laser‐induced injury during thrombus formation in vivo. These new improvements expand the quantitative biological‐imaging capacity of any polygon scanning microscope system.

     

A compact high‐speed full‐field optical coherence microscope for high‐resolution in vivo skin imaging

A compact high‐speed full‐field optical coherence microscope has been developed for high‐resolution in vivo imaging of biological tissues. The interferometer, in the Linnik configuration, has a size of 11 × 11 × 5 cm³ and a weight of 210 g. Full‐field illumination with low‐coherence light is achieved with a high‐brightness broadband light‐emitting diode. High‐speed full‐field detection is achieved by using part of the image sensor of a high‐dynamic range CMOS camera. En face tomographic images are acquired at a rate of 50 Hz, with an integration time of 0.9 ms. The image spatial resolution is 0.9 μm × 1.2 μm (axial × transverse), over a field of view of 245 × 245 μm². Images of human skin, revealing in‐depth cellular‐level structures, were obtained in vivo and in real‐time without the need for stabilization of the subject. The system can image larger fields, up to 1 × 1 mm², but at a reduced depth.      

Imaging sebaceous gland using optical coherence tomography with deep learning assisted automatic identification

Imaging sebaceous glands and evaluating morphometric parameters are important for diagnosis and treatment of serum problems. In this article, we investigate the feasibility of high-resolution optical coherence tomography (OCT) in combination with deep learning assisted automatic identification for these purposes. Specifically, with a spatial resolution of 2.3 μm × 6.2 μm (axial × lateral, in air), OCT is capable of clearly differentiating sebaceous gland from other skin structures and resolving the sebocyte layer. In order to achieve efficient and timely imaging analysis, a deep learning approach built upon ResNet18 is developed to automatically classify OCT images (with/without sebaceous gland), with a classification accuracy of 97.9%. Based on the result of automatic identification, we further demonstrate the possibility to measure gland size, sebocyte layer thickness and gland density.     

Portable optical‐resolution photoacoustic microscopy for volumetric imaging of multiscale organisms

Photoacoustic microscopy (PAM) provides a fundamentally new tool for a broad range of studies of biological structures and functions. However, the use of PAM has been largely limited to small vertebrates due to the large size/weight and the inconvenience of the equipment. Here, we describe a portable optical‐resolution photoacoustic microscopy (pORPAM) system for 3‐dimensional (3D) imaging of small‐to‐large rodents and humans with a high spatiotemporal resolution and a large field of view. We show extensive applications of pORPAM to multiscale animals including mice and rabbits. In addition, we image the 3D vascular networks of human lips, and demonstrate the feasibility of pORPAM to observe the recovery process of oral ulcer and cancer‐associated capillary loops in human oral cavities. This technology is promising for broad biomedical studies from fundamental biology to clinical diseases.      

Chloroplast-located flavonoids can scavenge singlet oxygen

* The hypothesis was tested that flavonoids may scavenge singlet oxygen ((1)O(2)) in mesophyll cells of Phillyrea latifolia exposed to excess-light stress. * In cross-sections taken from leaves developed at 10% (shade) or 100% (sun) solar irradiance, we evaluated the excess photosynthetically active radiation (PAR)-induced accumulation of (1)O(2) in mesophyll cells by imaging the fluorescence quenching of the specific (1)O(2) probe N-[2-(diethylamino)ethyl]-N-[(2,5-dihydro-2,2,5,5-tetramethyl-1H-pyrrol-3-yl)methyl]-5-(dimethylamino)-1-naphthalenesulfonamide (DanePy). The intracellular location of flavonoids was also analyzed using three-dimensional deconvolution microscopy. * Photo-induced quenching of DanePy fluorescence was markedly greater in the mesophyll of shade leaves than in that of sun leaves, the former showing a negligible accumulation of mesophyll flavonoids. The photo-induced generation of (1)O(2) was inversely related to the content of flavonoids in the mesophyll cells of sun leaves. Flavonoids were located in the chloroplasts, and were likely associated with the chloroplast envelope. * Here we provide relevant evidence for the potential scavenger activity of chloroplast-located flavonoids against (1)O(2) and new insights into the photo-protective role of flavonoids in higher plants.     

From Animaculum to single molecules: 300 years of the light microscope

Although not laying claim to being the inventor of the light microscope, Antonj van Leeuwenhoek (1632–1723) was arguably the first person to bring this new technological wonder of the age properly to the attention of natural scientists interested in the study of living things (people we might now term ‘biologists’). He was a Dutch draper with no formal scientific training. From using magnifying glasses to observe threads in cloth, he went on to develop over 500 simple single lens microscopes (Baker & Leeuwenhoek 1739 Phil. Trans. 41, 503–519. (doi:10.1098/rstl.1739.0085)) which he used to observe many different biological samples. He communicated his finding to the Royal Society in a series of letters (Leeuwenhoek 1800 The select works of Antony Van Leeuwenhoek, containing his microscopical discoveries in many of the works of nature, vol. 1) including the one republished in this edition of Open Biology. Our review here begins with the work of van Leeuwenhoek before summarizing the key developments over the last ca 300 years, which has seen the light microscope evolve from a simple single lens device of van Leeuwenhoek''s day into an instrument capable of observing the dynamics of single biological molecules inside living cells, and to tracking every cell nucleus in the development of whole embryos and plants.     

A low‐cost,automated parasite diagnostic system via a portable,robotic microscope and deep learning

Manual hand counting of parasites in fecal samples requires costly components and substantial expertise, limiting its use in resource‐constrained settings and encouraging overuse of prophylactic medication. To address this issue, a cost‐effective, automated parasite diagnostic system that does not require special sample preparation or a trained user was developed. It is composed of an inexpensive (~US$350), portable, robotic microscope that can scan over the size of an entire McMaster chamber (100 mm²) and capture high‐resolution (~1 μm lateral resolution) bright field images without need for user intervention. Fecal samples prepared using the McMaster flotation method were imaged, with the imaging region comprising the entire McMaster chamber. These images are then automatically segmented and analyzed using a trained convolution neural network (CNN) to robustly separate eggs from background debris. Simple postprocessing of the CNN output yields both egg species and egg counts. The system was validated by comparing accuracy with hand‐counts by a trained operator, with excellent performance. As a further demonstration of utility, the system was used to conveniently quantify drug response over time in a single animal, showing residual disease due to Anthelmintic resistance after 2 weeks.     

A new deep learning method for image deblurring in optical microscopic systems

Deconvolution is the most commonly used image processing method in optical imaging systems to remove the blur caused by the point‐spread function (PSF). While this method has been successful in deblurring, it suffers from several disadvantages, such as slow processing time due to multiple iterations required to deblur and suboptimal in cases where the experimental operator chosen to represent PSF is not optimal. In this paper, we present a deep‐learning‐based deblurring method that is fast and applicable to optical microscopic imaging systems. We tested the robustness of proposed deblurring method on the publicly available data, simulated data and experimental data (including 2D optical microscopic data and 3D photoacoustic microscopic data), which all showed much improved deblurred results compared to deconvolution. We compared our results against several existing deconvolution methods. Our results are better than conventional techniques and do not require multiple iterations or pre‐determined experimental operator. Our method has several advantages including simple operation, short time to compute, good deblur results and wide application in all types of optical microscopic imaging systems. The deep learning approach opens up a new path for deblurring and can be applied in various biomedical imaging fields.     

Structure-based protein design with deep learning

Since the first revelation of proteins functioning as macromolecular machines through their three dimensional structures, researchers have been intrigued by the marvelous ways the biochemical processes are carried out by proteins. The aspiration to understand protein structures has fueled extensive efforts across different scientific disciplines. In recent years, it has been demonstrated that proteins with new functionality or shapes can be designed via structure-based modeling methods, and the design strategies have combined all available information — but largely piece-by-piece — from sequence derived statistics to the detailed atomic-level modeling of chemical interactions. Despite the significant progress, incorporating data-derived approaches through the use of deep learning methods can be a game changer. In this review, we summarize current progress, compare the arc of developing the deep learning approaches with the conventional methods, and describe the motivation and concepts behind current strategies that may lead to potential future opportunities.     

Confocal scanning optical microscopy and three-dimensional imaging

Summary— Confocal scanning optical microscopy has significant advantages over conventional fluorescence microscopy: it rejects the out-of-locus light and provides a greater resolution than the wide-field microscope. In laser scanning optical microscopy, the specimen is scanned by a diffraction-limited spot of laser light and the fluorescence emission (or the reflected light) is focused onto a photodetector. The imaged point is then digitized, stored into the memory of a computer and displayed at the appropriate spatial position on a graphic device as a part of a two-dimensional image. Thus, confocal scanning optical microscopy allows accurate non-invasive optical sectioning and further three-dimensional reconstruction of biological specimens. Here we review the recent technological aspects of the principles and uses of the confocal microscope, and we introduce the different methods of three-dimensional imaging.     

Two‐photon focal modulation microscopy for high‐resolution imaging in deep tissue

Two‐photon microscopy (2PM) is one of the most widely used tools for in vivo deep tissue imaging. However, the spatial resolution and penetration depth are still limited due to the strong scattering background. Here we demonstrate a two‐photon focal modulation microscopy. By utilizing the modulation and demodulation techniques, background rejection capability is enhanced, thus spatial resolution and imaging penetration depth are improved. Compared with 2PM, the transverse resolution is increased by 70%, while the axial resolution is increased to 2‐fold. Furthermore, when applied in conventional 2PM mode, it can achieve inertial‐free scanning in either transverse or axial direction with in principle unlimited scanning speed. Finally, we applied 2PFMM in thick scattering samples to further examine the imaging performance. The results show that the signal‐to‐background ratio of 2PFMM can be improved up to five times of 2PM at the depth of 500 μm. Fluorescent imaging in the mouse brain tissue. 3D Thy1‐GFP hippocampal neurons imaged by (A) 2PM compared with (B) 2PFMM; (C‐H) xy maximum‐intensity projection imaged by 2PM compared with 2PFMM. Scale bar 50 μm.      

AI applications to medical images: From machine learning to deep learning

PurposeArtificial intelligence (AI) models are playing an increasing role in biomedical research and healthcare services. This review focuses on challenges points to be clarified about how to develop AI applications as clinical decision support systems in the real-world context.MethodsA narrative review has been performed including a critical assessment of articles published between 1989 and 2021 that guided challenging sections.ResultsWe first illustrate the architectural characteristics of machine learning (ML)/radiomics and deep learning (DL) approaches. For ML/radiomics, the phases of feature selection and of training, validation, and testing are described. DL models are presented as multi-layered artificial/convolutional neural networks, allowing us to directly process images. The data curation section includes technical steps such as image labelling, image annotation (with segmentation as a crucial step in radiomics), data harmonization (enabling compensation for differences in imaging protocols that typically generate noise in non-AI imaging studies) and federated learning. Thereafter, we dedicate specific sections to: sample size calculation, considering multiple testing in AI approaches; procedures for data augmentation to work with limited and unbalanced datasets; and the interpretability of AI models (the so-called black box issue). Pros and cons for choosing ML versus DL to implement AI applications to medical imaging are finally presented in a synoptic way.ConclusionsBiomedicine and healthcare systems are one of the most important fields for AI applications and medical imaging is probably the most suitable and promising domain. Clarification of specific challenging points facilitates the development of such systems and their translation to clinical practice.     

Deep learning for computational biology

Technological advances in genomics and imaging have led to an explosion of molecular and cellular profiling data from large numbers of samples. This rapid increase in biological data dimension and acquisition rate is challenging conventional analysis strategies. Modern machine learning methods, such as deep learning, promise to leverage very large data sets for finding hidden structure within them, and for making accurate predictions. In this review, we discuss applications of this new breed of analysis approaches in regulatory genomics and cellular imaging. We provide background of what deep learning is, and the settings in which it can be successfully applied to derive biological insights. In addition to presenting specific applications and providing tips for practical use, we also highlight possible pitfalls and limitations to guide computational biologists when and how to make the most use of this new technology.     

Photodamage in deep tissue two‐photon optical biopsy of human skin

Photodamage, induced by femtosecond laser radiation, was studied in thick samples of human skin tissue (healthy skin and neoplastic lesions). Photobleaching, photoionization, and thermomechanical damage effects were characterized comparatively. The laser power dependence of the damage rates allowed to connect macroscopic effects to underlying molecular processes. Optical effects were correlated to histopathological changes. Tissue alterations were found only from thermomechanical cavitation and limited to superficial layers of the epidermis. From the depth‐dependencies of all damage thresholds a depth‐dependent power‐compensation scheme was defined allowing for damage‐free deep tissue optical biopsy.

Damage‐induced luminescence pattern for different excitation powers and a corresponding threshold analysis.     

Optical clearing methods: An overview of the techniques used for the imaging of 3D spheroids

Spheroids have emerged as in vitro models that reproduce in a great extent the architectural microenvironment found in human tissues. However, the imaging of 3D cell cultures is highly challenging due to its high thickness, which results in a light-scattering phenomenon that limits light penetration. Therefore, several optical clearing methods, widely used in the imaging of animal tissues, have been recently explored to render spheroids with enhanced transparency. These methods are aimed to homogenize the microtissue refractive index (RI) and can be grouped into four different categories, namely (a) simple immersion in an aqueous solution with high RI; (b) delipidation and dehydration followed by RI matching; (c) delipidation and hyperhydration followed by RI matching; and (d) hydrogel embedding followed by delipidation and RI matching. In this review, the main optical clearing methods, their mechanism of action, advantages, and disadvantages are described. Furthermore, the practical examples of the optical clearing methods application for the imaging of 3D spheroids are highlighted.     

Simulated four-dimensional CT for markerless tumor tracking using a deep learning network with multi-task learning

IntroductionOur markerless tumor tracking algorithm requires 4DCT data to train models. 4DCT cannot be used for markerless tracking for respiratory-gated treatment due to inaccuracies and a high radiation dose. We developed a deep neural network (DNN) to generate 4DCT from 3DCT data.MethodsWe used 2420 thoracic 4DCT datasets from 436 patients to train a DNN, designed to export 9 deformation vector fields (each field representing one-ninth of the respiratory cycle) from each CT dataset based on a 3D convolutional autoencoder with shortcut connections using deformable image registration. Then 3DCT data at exhale were transformed using the predicted deformation vector fields to obtain simulated 4DCT data. We compared markerless tracking accuracy between original and simulated 4DCT datasets for 20 patients. Our tracking algorithm used a machine learning approach with patient-specific model parameters. For the training stage, a pair of digitally reconstructed radiography images was generated using 4DCT for each patient. For the prediction stage, the tracking algorithm calculated tumor position using incoming fluoroscopic image data.ResultsDiaphragmatic displacement averaged over 40 cases for the original 4DCT were slightly higher (<1.3 mm) than those for the simulated 4DCT. Tracking positional errors (95th percentile of the absolute value of displacement, “simulated 4DCT” minus “original 4DCT”) averaged over the 20 cases were 0.56 mm, 0.65 mm, and 0.96 mm in the X, Y and Z directions, respectively.ConclusionsWe developed a DNN to generate simulated 4DCT data that are useful for markerless tumor tracking when original 4DCT is not available. Using this DNN would accelerate markerless tumor tracking and increase treatment accuracy in thoracoabdominal treatment.     

Dual‐color STED super‐resolution microscope using a single laser source

STED (stimulated emission depletion) microscopy is one of the most promising super‐resolution fluorescence microscopies，due to its fast imaging and ultra‐high resolution. In this paper, we present a dual‐color STED microscope with a single laser source. Polarization beam splitters are used to separate the output from a supercontinuum laser source into four laser beams, including two excitation beams (488, 635 nm) and two depletion beams (592, 775 nm). These four laser beams are then used to build a low cost dual‐color STED system to achieve a spatial resolution of 75 nm in cell samples.     

Single‐shot speckle correlation fluorescence microscopy in thick scattering tissue with image reconstruction priors

Deep tissue imaging in the multiple scattering regime remains at the frontier of fluorescence microscopy. Speckle correlation imaging (SCI) can computationally uncover objects hidden behind a scattering layer, but has only been demonstrated with scattered laser illumination and in geometries where the scatterer is in the far field of the target object. Here, SCI is extended to imaging a planar fluorescent signal at the back surface of a 500‐μm‐thick slice of mouse brain. The object is reconstructed from a single snapshot through phase retrieval using a proximal algorithm that easily incorporates image priors. Simulations and experiments demonstrate improved image recovery with this approach compared to the conventional SCI algorithm.      

超分辨显微成像技术在活细胞成像中的应用与发展

超分辨显微成像技术(super-resolution microscopy,SRM)可以绕过光学衍射极限对成像分辨率的限制,让以前观察不到的纳米级结构实现可视化,这一重大研究进展推动了现代生命科学和生物医学研究的进步与发展.细胞是生物体的基本组成单位,对活细胞内部的细微结构和动力学过程进行研究是掌握生命本质必不可少的途...     

Integrating photoacoustic microscopy with other imaging technologies for multimodal imaging

As a hybrid optical microscopic imaging technology, photoacoustic microscopy images the optical absorption contrasts and takes advantage of low acoustic scattering of biological tissues to achieve high-resolution anatomical and functional imaging. When combined with other imaging modalities, photoacoustic microscopy-based multimodal technologies can provide complementary contrast mechanisms to reveal complementary information of biological tissues. To achieve intrinsically and precisely registered images in a multimodal photoacoustic microscopy imaging system, either the ultrasonic transducer or the light source can be shared among the different imaging modalities. These technologies are the major focus of this minireview. It also covered the progress of the recently developed penta-modal photoacoustic microscopy imaging system featuring a novel dynamic focusing technique enabled by OCT contour scan.