Diseased thyroid tissue classification in OCT images using deep learning: Towards surgical decision support

Intraoperative guidance tools for thyroid surgery based on optical coherence tomography (OCT) could aid distinguish between normal and diseased tissue. However, OCT images are difficult to interpret, thus, real-time automatic analysis could support the clinical decision-making. In this study, several deep learning models were investigated for thyroid disease classification on 2D and 3D OCT data obtained from ex vivo specimens of 22 patients undergoing surgery and diagnosed with several thyroid pathologies. Additionally, two open-access datasets were used to evaluate the custom models. On the thyroid dataset, the best performance was achieved by the 3D vision transformer model with a Matthew's correlation coefficient (MCC) of 0.79 (accuracy = 0.90) for the normal-versus-abnormal classification. On the open-access datasets, the custom models achieved the best performance (MCC > 0.88, accuracy > 0.96). Results obtained for the normal-versus-abnormal classification suggest OCT, complemented with deep learning-based analysis, as a tool for real-time automatic diseased tissue identification in thyroid surgery.     

N2NSR-OCT: Simultaneous denoising and super-resolution in optical coherence tomography images using semisupervised deep learning

Optical coherence tomography (OCT) imaging shows a significant potential in clinical routines due to its noninvasive property. However, the quality of OCT images is generally limited by inherent speckle noise of OCT imaging and low sampling rate. To obtain high signal-to-noise ratio (SNR) and high-resolution (HR) OCT images within a short scanning time, we presented a learning-based method to recover high-quality OCT images from noisy and low-resolution OCT images. We proposed a semisupervised learning approach named N2NSR-OCT, to generate denoised and super-resolved OCT images simultaneously using up- and down-sampling networks (U-Net (Semi) and DBPN (Semi)). Additionally, two different super-resolution and denoising models with different upscale factors (2× and 4× ) were trained to recover the high-quality OCT image of the corresponding down-sampling rates. The new semisupervised learning approach is able to achieve results comparable with those of supervised learning using up- and down-sampling networks, and can produce better performance than other related state-of-the-art methods in the aspects of maintaining subtle fine retinal structures.     

Detecting mouse squamous cell carcinoma from submicron full-field optical coherence tomography images by deep learning

The standard medical practice for cancer diagnosis requires histopathology, which is an invasive and time-consuming procedure. Optical coherence tomography (OCT) is an alternative that is relatively fast, noninvasive, and able to capture three-dimensional structures of epithelial tissue. Unlike most previous OCT systems, which cannot capture crucial cellular-level information for squamous cell carcinoma (SCC) diagnosis, the full-field OCT (FF-OCT) technology used in this paper is able to produce images at sub-micron resolution and thereby facilitates the development of a deep learning algorithm for SCC detection. Experimental results show that the SCC detection algorithm can achieve a classification accuracy of 80% for mouse skin. Using the sub-micron FF-OCT imaging system, the proposed SCC detection algorithm has the potential for in-vivo applications.     

Comparative study of deep neural networks with unsupervised Noise2Noise strategy for noise reduction of optical coherence tomography images

As a powerful diagnostic tool, optical coherence tomography (OCT) has been widely used in various clinical setting. However, OCT images are susceptible to inherent speckle noise that may contaminate subtle structure information, due to low-coherence interferometric imaging procedure. Many supervised learning-based models have achieved impressive performance in reducing speckle noise of OCT images trained with a large number of noisy-clean paired OCT images, which are not commonly feasible in clinical practice. In this article, we conducted a comparative study to investigate the denoising performance of OCT images over different deep neural networks through an unsupervised Noise2Noise (N2N) strategy, which only trained with noisy OCT samples. Four representative network architectures including U-shaped model, multi-information stream model, straight-information stream model and GAN-based model were investigated on an OCT image dataset acquired from healthy human eyes. The results demonstrated all four unsupervised N2N models offered denoised OCT images with a performance comparable with that of supervised learning models, illustrating the effectiveness of unsupervised N2N models in denoising OCT images. Furthermore, U-shaped models and GAN-based models using UNet network as generator are two preferred and suitable architectures for reducing speckle noise of OCT images and preserving fine structure information of retinal layers under unsupervised N2N circumstances.     

Cross-domain retinopathy classification with optical coherence tomography images via a novel deep domain adaptation method

Deep learning based retinopathy classification with optical coherence tomography (OCT) images has recently attracted great attention. However, existing deep learning methods fail to work well when training and testing datasets are different due to the general issue of domain shift between datasets caused by different collection devices, subjects, imaging parameters, etc. To address this practical and challenging issue, we propose a novel deep domain adaptation (DDA) method to train a model on a labeled dataset and adapt it to an unlabelled dataset (collected under different conditions). It consists of two modules for domain alignment, that is, adversarial learning and entropy minimization. We conduct extensive experiments on three public datasets to evaluate the performance of the proposed method. The results indicate that there are large domain shifts between datasets, resulting a poor performance for conventional deep learning methods. The proposed DDA method can significantly outperform existing methods for retinopathy classification with OCT images. It achieves retinopathy classification accuracies of 0.915, 0.959 and 0.990 under three cross-domain (cross-dataset) scenarios. Moreover, it obtains a comparable performance with human experts on a dataset where no labeled data in this dataset have been used to train the proposed DDA method. We have also visualized the learnt features by using the t-distributed stochastic neighbor embedding (t-SNE) technique. The results demonstrate that the proposed method can learn discriminative features for retinopathy classification.     

Three‐dimensional mapping of the attenuation coefficient in optical coherence tomography to enhance breast tissue microarchitecture contrast

Effective intraoperative tumor margin assessment is needed to reduce re‐excision rates in breast‐conserving surgery (BCS). Mapping the attenuation coefficient in optical coherence tomography (OCT) throughout a sample to create an image (attenuation imaging) is one promising approach. For the first time, three‐dimensional OCT attenuation imaging of human breast tissue microarchitecture using a wide‐field (up to ~45 × 45 × 3.5 mm) imaging system is demonstrated. Representative results from three mastectomy and one BCS specimen (from 31 specimens) are presented with co‐registered postoperative histology. Attenuation imaging is shown to provide substantially improved contrast over OCT, delineating nuanced features within tumors (including necrosis and variations in tumor cell density and growth patterns) and benign features (such as sclerosing adenosis). Additionally, quantitative micro‐elastography (QME) images presented alongside OCT and attenuation images show that these techniques provide complementary contrast, suggesting that multimodal imaging could increase tissue identification accuracy and potentially improve tumor margin assessment.     

An automatic diagnostic system of coronary artery lesions in Kawasaki disease using intravascular optical coherence tomography imaging

Intravascular optical coherence tomography (IV‐OCT) is a light‐based imaging modality with high resolution, which employs near‐infrared light to provide tomographic intracoronary images. Morbidity caused by coronary heart disease is a substantial cause of acute coronary syndrome and sudden cardiac death. The most common intracoronay complications caused by coronary artery disease are intimal hyperplasia, calcification, fibrosis, neovascularization and macrophage accumulation, which require efficient prevention strategies. OCT can provide discriminative information of the intracoronary tissues, which can be used to train a robust fully automatic tissue characterization model based on deep learning. In this study, we aimed to design a diagnostic model of coronary artery lesions. Particularly, we trained a random forest using convolutional neural network features to distinguish between normal and diseased arterial wall structure. Then, based on the arterial wall structure, fully convolutional network is designed to extract the tissue layers in normal cases, and pathological tissues regardless of lesion type in pathological cases. Then, the type of the lesions can be characterized with high precision using our previous model. The results demonstrate the robustness of the model with the approximate overall accuracy up to 90%.      

A novel automatic 3D stitching algorithm for optical coherence tomography angiography and its application in dermatology

Optical coherence tomography (OCT) and OCT angiography (OCTA) techniques offer numerous advantages in clinical skin applications but the field of view (FOV) of current commercial systems are relatively limited to cover the entire skin lesion. The typical method to expand the FOV is to apply wide field objective lens. However, lateral resolution is often sacrificed when scanning with these lenses. To overcome this drawback, we developed an automated 3D stitching method for creating high-resolution skin structure and vascular volumes with large field of view, which was realized by montaging multiple adjacent OCT and OCTA volumes. The proposed stitching method is demonstrated by montaging 3 × 3 OCT and OCTA volumes (nine OCT/OCTA volumes as one data set with each volume covers 2.5 cm × 2.5 cm area) of healthy thin and thick skin from six volunteers. The proposed stitching protocol achieves high flexibility and repeatable for all the participants. Moreover, according to evaluation of structural similarity index and feature similarity index, our proposed stitched result has a superior similarity to single scanning protocol in large-scaled. We had also verified its improved performance through assessing metrics of vessel contrast-noise-ratio (CNR) from 2.07 ± 0.44 (single large-scaled scanning protocol) to 3.05 ± 0.51 (proposed 3 × 3 sub-volume stitching method).     

p53 induces skin aging by depleting Blimp1+ sebaceous gland cells

p53 is an important inducer of organismal aging. However, its roles in the aging of skin remain unclear. Here we show that mice with chronic activation of p53 develop an aging phenotype in the skin associated with a reduction of subcutaneous fat and loss of sebaceous gland (SG). The reduction in the fat layer may result from the decrease of mammalian TOR complex 1 (mTORC1) activity accompanied by elevated expression of energy expenditure genes, and possibly as compensatory effects, leading to the elevation of peroxisome proliferator-activated receptor (PPAR)γ, an inducer of sebocyte differentiation. In addition, Blimp1⁺ sebocytes become depleted concomitantly with an increase in cellular senescence, which can be reversed by PPARγ antagonist (BADGE) treatment. Therefore, our results indicate that p53-mediated aging of the skin involves not only thinning through the loss of subdermal fat, but also xerosis or drying of the skin through declining sebaceous gland activity.     

Retinal cross-section motion correction in three-dimensional retinal optical coherence tomography

Motion correction is an important issue in ophthalmic optical coherence tomography (OCT), and can improve the ability of data sets to reflect the physiological structures of tissues and make visualization and subsequent analysis easier. In this study, we present a novel method to correct the cross-sectional motion artifacts in retinal OCT volumes. Motion along the x-direction (fast-scan direction) is corrected through the normalized cross-correlation algorithm, while axial motion compensation is performed using the polynomial fitting method on the inner segment/outer segment (IS/OS) layer segmented by the shortest path faster algorithm (SPFA). The results of volunteers with central serous chorioretinopathy demonstrate that the proposed method effectively corrects motion artifacts in OCT volumes and may have potential application value in the evaluation of ophthalmic diseases such as diabetic retinopathy, glaucoma and age-related macular degeneration.     

Optical coherence tomography image denoising using a generative adversarial network with speckle modulation

Optical coherence tomography (OCT) is widely used for biomedical imaging and clinical diagnosis. However, speckle noise is a key factor affecting OCT image quality. Here, we developed a custom generative adversarial network (GAN) to denoise OCT images. A speckle‐modulating OCT (SM‐OCT) was built to generate low speckle images to be used as the ground truth. In total, 210 000 SM‐OCT images were used for training and validating the neural network model, which we call SM‐GAN. The performance of the SM‐GAN method was further demonstrated using online benchmark retinal images, 3D OCT images acquired from human fingers and OCT videos of a beating fruit fly heart. The denoise performance of the SM‐GAN model was compared to traditional OCT denoising methods and other state‐of‐the‐art deep learning based denoise networks. We conclude that the SM‐GAN model presented here can effectively reduce speckle noise in OCT images and videos while maintaining spatial and temporal resolutions.     

Gingivitis resolution followed by optical coherence tomography and fluorescence imaging: A case study

Gingivitis is highly prevalent in adults, and if left untreated, can progress to periodontitis. In this article, we present an interesting case study where the resolution of gingivitis was followed over a period of 10 days using optical coherence tomography (OCT) and light-induced autofluorescence (LIAF). We demonstrate that OCT and its functional angiography can distinctively capture the changes during the resolution of gingivitis; while LIAF can detect red-fluorescent signals associated with mature plaque present at the inflamed site. The acute inflammatory region showed evidence of angiogenesis based on the quantification of vessel density and number; while no angiogenesis was detected within the less inflamed region. Gingival thickness showed a reduction of 140 ± 26 μm on average, measured between the peak gingivitis event and the period wherein the inflammation was resolved. Vessels in the angiogenesis site was found to reduce exponentially. The mildly inflamed site showed a decreasing trend in the vessel size, which however was within the error of the measurement.     

Quantitative assessment of optical coherence tomography angiography algorithms for neuroimaging

Optical coherence tomography (OCT) angiography can noninvasively map microvascular networks and quantify blood flow in a cerebral cortex with a resolution of 1 to 10 μm and a penetration depth of 2 to 3 mm incorporating OCT signals and angiography algorithms. Different angiography algorithms have been developed in recent years; however, the performance of the algorithms has not been assessed quantitatively for neuroimaging applications. In this paper, we developed four metrics including vascular connectivity, contrast‐to‐noise ratio, signal‐to‐noise ratio and processing time to quantitatively assess the performance of OCT angiography algorithms in image quality and computation speed. After the imaging of a rat cortex using an OCT system, the cerebral microvascular networks were visualized by seven algorithms, and the performance of the algorithms was quantified and compared. Quantitative performance assessment of the algorithms can provide suggestions for the selection of appropriate OCT angiography algorithms in neuroimaging.     

Preclinical mouse model of optical coherence tomography for subcortical brain imaging without dissection

The purpose of this study was to investigate the feasibility of using optical coherence tomography (OCT) to identify internal brain lesions, specifically intracerebral hemorrhage, without dissection. Mice with artificially injected brain hematomas were used to test the OCT system, and the recorded images were compared with microscopic images of the same mouse brains after hematoxylin and eosin staining. The intracranial structures surrounding the hematomas were clearly visualized by the OCT system without dissection. These images reflect the ability of OCT to determine the extent of a lesion in several planes. OCT is a useful technology, and these findings could be used as a starting point for future research in intraoperative imaging.     

Attenuation corrected-optical coherence tomography for quantitative assessment of skin wound healing and scar morphology

Imaging the structural modifications of underlying tissues is vital to monitor wound healing. Optical coherence tomography (OCT) images high-resolution sub-surface information, but suffers a loss of intensity with depth, limiting quantification. Hence correcting the attenuation loss is important. We performed swept source-OCT of full-thickness excision wounds for 300 days in mice skin. We used single-scatter attenuation models to determine and correct the attenuation loss in the images. The phantom studies established the correspondence of corrected-OCT intensity (reflectivity) with matrix density and hydration. We histologically validated the corrected-OCT and measured the wound healing rate. We noted two distinct phases of healing—rapid and steady-state. We also detected two compartments in normal scars using corrected OCT that otherwise were not visible in the OCT scans. The OCT reflectivity in the scar compartments corresponded to distinct cell populations, mechanical properties and composition. OCT reflectivity has potential applications in evaluating the therapeutic efficacy of healing and characterizing scars.     

Pseudo‐real‐time retinal layer segmentation for high‐resolution adaptive optics optical coherence tomography

We present a pseudo‐real‐time retinal layer segmentation for high‐resolution Sensorless Adaptive Optics‐Optical Coherence Tomography (SAO‐OCT). Our pseudo‐real‐time segmentation method is based on Dijkstra's algorithm that uses the intensity of pixels and the vertical gradient of the image to find the minimum cost in a geometric graph formulation within a limited search region. It segments six retinal layer boundaries in an iterative process according to their order of prominence. The segmentation time is strongly correlated to the number of retinal layers to be segmented. Our program permits en face images to be extracted during data acquisition to guide the depth specific focus control and depth dependent aberration correction for high‐resolution SAO‐OCT systems. The average processing times for our entire pipeline for segmenting six layers in a retinal B‐scan of 496 × 400 and 240 × 400 pixels are around 25.60 and 13.76 ms, respectively. When reducing the number of layers segmented to only two layers, the time required for a 240 × 400 pixel image is 8.26 ms.     

Three‐dimensional multiphoton/optical coherence tomography for diagnostic applications in dermatology

A preliminary clinical trial using state‐of‐the‐art multiphoton tomography (MPT) and optical coherence tomography (OCT) for three‐dimensional (3D) multimodal in vivo imaging of normal skin, nevi, scars and pathologic skin lesions has been conducted. MPT enabled visualization of sub‐cellular details with axial and transverse resolutions of <2 μm and <0.5 μm, respectively, from a volume of 0.35 × 0.35 × 0.2 mm³ at a frame rate of 0.14 Hz (512 × 512 pixels). State‐of‐the‐art OCT, operating at a center wavelength of 1300 nm, was capable of acquiring 3D images depicting the layered architecture of skin with axial and transverse resolutions ～8 μm and ～20 μm, respectively, from a volume of 7 × 3.5 × 1.5 mm³ at a frame rate of 46 Hz (1024 × 1024 pixels). This study demonstrates the clinical diagnostic potential of MPT/OCT for pre‐screening relatively large areas of skin using 3D OCT to identify suspicious regions at microscopic level and subsequently using high resolution MPT to obtain zoomed in, sub‐cellular level information of the respective regions (© 2013 WILEY‐VCH Verlag GmbH & Co. KGaA, Weinheim)     

Noninvasive multimodal imaging by integrating optical coherence tomography with autofluorescence imaging for dental applications

We report the development of an integrated multifunctional imaging system capable of providing anatomical (optical coherence tomography, OCT), functional (OCT angiography, OCTA) and molecular imaging (light‐induced autofluorescence, LIAF) for in vivo dental applications. Blue excitation light (405 nm) was used for LIAF imaging, while the OCT was powered by a 1310 nm swept laser source. A red‐green‐blue digital camera, with a 450 nm cut‐on broadband optical filter, was used for LIAF detection. The exciting light source and camera were integrated directly with the OCT scanning probe. The integrated system used two noninvasive imaging modalities to improve the speed of in vivo OCT data collection and to better target the regions of interest. The newly designed system maintained the ability to detect differences between healthy and hypomineralized teeth, identify dental biofilm and visualize the microvasculature of gingival tissue. The development of the integrated OCT‐LIAF system provides an opportunity to conduct clinical studies more efficiently, examining changes in oral conditions over time.     

Label-free bacteria identification for clinical applications

We have developed a system for bacteria identification based on absorption spectroscopy in the mid-infrared spectral range. The data collected are analyzed with a deep learning algorithm. It is based on a neural-network model which takes one-dimensional signal vectors and outputs a probability score of identification of a bacterium type by extracting micro and macro scale features, using convolutions and nonlinear operations. The results are achieved in real time and do not require any offline postprocessing. The study was done on 12 of the most common bacteria usually seen in clinical microbiology laboratories. The system sensitivity is 0.94 ± 0.04, with a specificity of 0.95 ± 0.02. The system can be extended to additional bacterium types and variants with no change to its hardware or software, but only updating the model's parameters. The system's accuracy, size, ease of operation and low cost make it suitable for use in any type of clinical setting.