Exploration research on the fusion of multimodal spectrum technology to improve performance of rapid diagnosis scheme for thyroid dysfunction

The spectral fusion by Raman spectroscopy and Fourier infrared spectroscopy combined with pattern recognition algorithms is utilized to diagnose thyroid dysfunction serum, and finds the spectral segment with the highest sensitivity to further advance diagnosis speed. Compared with the single infrared spectroscopy or Raman spectroscopy, the proposal can improve the detection accuracy, and can obtain more spectral features, indicating greater differences between thyroid dysfunction and normal serum samples. For discriminating different samples, principal component analysis (PCA) was first used for feature extraction to reduce the dimension of high‐dimension spectral data and spectral fusion. Then, support vector machine (SVM), back propagation neural network, extreme learning machine and learning vector quantization algorithms were employed to establish the discriminant diagnostic models. The accuracy of spectral fusion of the best analytical model PCA‐SVM, single Raman spectral accuracy and single infrared spectral accuracy is 83.48%, 78.26% and 80%, respectively. The accuracy of spectral fusion is higher than the accuracy of single spectrum in five classifiers. And the diagnostic accuracy of spectral fusion in the range of 2000 to 2500 cm^?1 is 81.74%, which greatly improves the sample measure speed and data analysis speed than analysis of full spectra. The results from our study demonstrate that the serum spectral fusion technique combined with multivariate statistical methods have great potential for the screening of thyroid dysfunction.     

Deep learning a boon for biophotonics?

This review covers original articles using deep learning in the biophotonic field published in the last years. In these years deep learning, which is a subset of machine learning mostly based on artificial neural network geometries, was applied to a number of biophotonic tasks and has achieved state‐of‐the‐art performances. Therefore, deep learning in the biophotonic field is rapidly growing and it will be utilized in the next years to obtain real‐time biophotonic decision‐making systems and to analyze biophotonic data in general. In this contribution, we discuss the possibilities of deep learning in the biophotonic field including image classification, segmentation, registration, pseudostaining and resolution enhancement. Additionally, we discuss the potential use of deep learning for spectroscopic data including spectral data preprocessing and spectral classification. We conclude this review by addressing the potential applications and challenges of using deep learning for biophotonic data.     

A representation learning approach for recovering scatter-corrected spectra from Fourier-transform infrared spectra of tissue samples

Infrared spectra obtained from cell or tissue specimen have commonly been observed to involve a significant degree of scattering effects, often Mie scattering, which probably overshadows biochemically relevant spectral information by a nonlinear, nonadditive spectral component in Fourier transform infrared (FTIR) spectroscopic measurements. Correspondingly, many successful machine learning approaches for FTIR spectra have relied on preprocessing procedures that computationally remove the scattering components from an infrared spectrum. We propose an approach to approximate this complex preprocessing function using deep neural networks. As we demonstrate, the resulting model is not just several orders of magnitudes faster, which is important for real-time clinical applications, but also generalizes strongly across different tissue types. Using Bayesian machine learning approaches, our approach unveils model uncertainty that coincides with a band shift in the amide I region that occurs when scattering is removed computationally based on an established physical model. Furthermore, our proposed method overcomes the trade-off between computation time and the corrected spectrum being biased towards an artificial reference spectrum.     

Automated label‐free detection of injured neuron with deep learning by two‐photon microscopy

Stroke is a significant cause of morbidity and long‐term disability globally. Detection of injured neuron is a prerequisite for defining the degree of focal ischemic brain injury, which can be used to guide further therapy. Here, we demonstrate the capability of two‐photon microscopy (TPM) to label‐freely identify injured neurons on unstained thin section and fresh tissue of rat cerebral ischemia‐reperfusion model, revealing definite diagnostic features compared with conventional staining images. Moreover, a deep learning model based on convolutional neural network is developed to automatically detect the location of injured neurons on TPM images. We then apply deep learning‐assisted TPM to evaluate the ischemic regions based on tissue edema, two‐photon excited fluorescence signal intensity, as well as neuronal injury, presenting a novel manner for identifying the infarct core, peri‐infarct area, and remote area. These results propose an automated and label‐free method that could provide supplementary information to augment the diagnostic accuracy, as well as hold the potential to be used as an intravital diagnostic tool for evaluating the effectiveness of drug interventions and predicting potential therapeutics.     

Rapid diagnosis of infection etiology in febrile pediatric oncology patients using infrared spectroscopy of leukocytes

Rapid diagnosis of the etiology of infection is highly important for an effective treatment of the infected patients. Bacterial and viral infections are serious diseases that can cause death in many cases. The human immune system deals with many viral and bacterial infections that cause no symptoms and pass quietly without treatment. However, oncology patients undergoing chemotherapy have a very weak immune system caused by leukopenia, and even minor pathogen infection threatens their lives. For this reason, physicians tend to prescribe immediately several types of antibiotics for febrile pediatric oncology patients (FPOPs). Uncontrolled use of antibiotics is one of the major contributors to the development of resistant bacteria. Therefore, for oncology patients, a rapid and objective diagnosis of the etiology of the infection is extremely critical. Current identification methods are time‐consuming (>24 h). In this study, the potential of midinfrared spectroscopy in tandem with machine learning algorithms is evaluated for rapid and objective diagnosis of the etiology of infections in FPOPs using simple peripheral blood samples. Our results show that infrared spectroscopy enables the diagnosis of the etiology of infection as bacterial or viral within 70 minutes after the collection of the blood sample with 93% sensitivity and 88% specificity.     

An automatic diagnostic system of coronary artery lesions in Kawasaki disease using intravascular optical coherence tomography imaging

Intravascular optical coherence tomography (IV‐OCT) is a light‐based imaging modality with high resolution, which employs near‐infrared light to provide tomographic intracoronary images. Morbidity caused by coronary heart disease is a substantial cause of acute coronary syndrome and sudden cardiac death. The most common intracoronay complications caused by coronary artery disease are intimal hyperplasia, calcification, fibrosis, neovascularization and macrophage accumulation, which require efficient prevention strategies. OCT can provide discriminative information of the intracoronary tissues, which can be used to train a robust fully automatic tissue characterization model based on deep learning. In this study, we aimed to design a diagnostic model of coronary artery lesions. Particularly, we trained a random forest using convolutional neural network features to distinguish between normal and diseased arterial wall structure. Then, based on the arterial wall structure, fully convolutional network is designed to extract the tissue layers in normal cases, and pathological tissues regardless of lesion type in pathological cases. Then, the type of the lesions can be characterized with high precision using our previous model. The results demonstrate the robustness of the model with the approximate overall accuracy up to 90%.      

A new strategy for canine visceral leishmaniasis diagnosis based on FTIR spectroscopy and machine learning

Visceral leishmaniasis is a neglected disease caused by protozoan parasites of the genus Leishmania. The successful control of the disease depends on its accurate and early diagnosis, which is usually made by combining clinical symptoms with laboratory tests such as serological, parasitological, and molecular tests. However, early diagnosis based on serological tests may exhibit low accuracy due to lack of specificity caused by cross-reactivities with other pathogens, and sensitivity issues related, among other reasons, to disease stage, leading to misdiagnosis. In this study was investigated the use of mid-infrared spectroscopy and multivariate analysis to perform a fast, accurate, and easy canine visceral leishmaniasis diagnosis. Canine blood sera of 20 noninfected, 20 Leishmania infantum, and eight Trypanosoma evansi infected dogs were studied. The data demonstrate that principal component analysis with machine learning algorithms achieved an overall accuracy above 85% in the diagnosis.     

Grayscale representation of infrared microscopy images by extended multiplicative signal correction for registration with histological images

Fourier‐transform infrared (FTIR) microspectroscopy is rounding the corner to become a label‐free routine method for cancer diagnosis. In order to build infrared‐spectral based classifiers, infrared images need to be registered with Hematoxylin and Eosin (H&E) stained histological images. While FTIR images have a deep spectral domain with thousands of channels carrying chemical and scatter information, the H&E images have only three color channels for each pixel and carry mainly morphological information. Therefore, image representations of infrared images are needed that match the morphological information in H&E images. In this paper, we propose a novel approach for representation of FTIR images based on extended multiplicative signal correction highlighting morphological features that showed to correlate well with morphological information in H&E images. Based on the obtained representations, we developed a strategy for global‐to‐local image registration for FTIR images and H&E stained histological images of parallel tissue sections.     

Performances of machine learning algorithms for mapping fractional cover of an invasive plant species in a dryland ecosystem

In recent years, an increasing number of distribution maps of invasive alien plant species (IAPS) have been published using different machine learning algorithms (MLAs). However, for designing spatially explicit management strategies, distribution maps should include information on the local cover/abundance of the IAPS. This study compares the performances of five MLAs: gradient boosting machine in two different implementations, random forest, support vector machine and deep learning neural network, one ensemble model and a generalized linear model; thereby identifying the best‐performing ones in mapping the fractional cover/abundance and distribution of IPAS, in this case called Prosopis juliflora (SW. DC.). Field level Prosopis cover and spatial datasets of seventeen biophysical and anthropogenic variables were collected, processed, and used to train and validate the algorithms so as to generate fractional cover maps of Prosopis in the dryland ecosystem of the Afar Region, Ethiopia. Out of the seven tested algorithms, random forest performed the best with an accuracy of 92% and sensitivity and specificity >0.89. The next best‐performing algorithms were the ensemble model and gradient boosting machine with an accuracy of 89% and 88%, respectively. The other tested algorithms achieved comparably low performances. The strong explanatory variables for Prosopis distributions in all models were NDVI, elevation, distance to villages and distance to rivers; rainfall, temperature, near‐infrared and red reflectance, whereas topographic variables, except for elevation, did not contribute much to the current distribution of Prosopis. According to the random forest model, a total of 1.173 million ha (12.33% of the study region) was found to be invaded by Prosopis to varying degrees of cover. Our findings demonstrate that MLAs can be successfully used to develop fractional cover maps of plant species, particularly IAPS so as to design targeted and spatially explicit management strategies.     

Epithelium segmentation and automated Gleason grading of prostate cancer via deep learning in label‐free multiphoton microscopic images

In the current clinical care practice, Gleason grading system is one of the most powerful prognostic predictors for prostate cancer (PCa). The grading system is based on the architectural pattern of cancerous epithelium in histological images. However, the standard procedure of histological examination often involves complicated tissue fixation and staining, which are time‐consuming and may delay the diagnosis and surgery. In this study, label‐free multiphoton microscopy (MPM) was used to acquire subcellular‐resolution images of unstained prostate tissues. Then, a deep learning architecture (U‐net) was introduced for epithelium segmentation of prostate tissues in MPM images. The obtained segmentation results were then merged with the original MPM images to train a classification network (AlexNet) for automated Gleason grading. The developed method achieved an overall pixel accuracy of 92.3% with a mean F1 score of 0.839 for epithelium segmentation. By merging the segmentation results with the MPM images, the accuracy of Gleason grading was improved from 72.42% to 81.13% in hold‐out test set. Our results suggest that MPM in combination with deep learning holds the potential to be used as a fast and powerful clinical tool for PCa diagnosis.     

Classifying T cell activity in autofluorescence intensity images with convolutional neural networks

The importance of T cells in immunotherapy has motivated developing technologies to improve therapeutic efficacy. One objective is assessing antigen‐induced T cell activation because only functionally active T cells are capable of killing the desired targets. Autofluorescence imaging can distinguish T cell activity states in a non‐destructive manner by detecting endogenous changes in metabolic co‐enzymes such as NAD(P)H. However, recognizing robust activity patterns is computationally challenging in the absence of exogenous labels. We demonstrate machine learning methods that can accurately classify T cell activity across human donors from NAD(P)H intensity images. Using 8260 cropped single‐cell images from six donors, we evaluate classifiers ranging from traditional models that use previously‐extracted image features to convolutional neural networks (CNNs) pre‐trained on general non‐biological images. Adapting pre‐trained CNNs for the T cell activity classification task provides substantially better performance than traditional models or a simple CNN trained with the autofluorescence images alone. Visualizing the images with dimension reduction provides intuition into why the CNNs achieve higher accuracy than other approaches. Our image processing and classifier training software is available at https://github.com/gitter‐lab/t‐cell‐classification .     

Deep Convolution Neural Network Based System for Early Diagnosis of Alzheimer's Disease

ObjectivesAlzheimer's Disease (AD) is the most general type of dementia. In all leading countries, it is one of the primary reasons of death in senior citizens. Currently, it is diagnosed by calculating the MSME score and by the manual study of MRI Scan. Also, different machine learning methods are utilized for automatic diagnosis but existing has some limitations in terms of accuracy. So, main objective of this paper to include a preprocessing method before CNN model to increase the accuracy of classification.Materials and methodIn this paper, we present a deep learning-based approach for detection of Alzheimer's Disease from ADNI database of Alzheimer's disease patients, the dataset contains fMRI and PET images of Alzheimer's patients along with normal person's image. We have applied 3D to 2D conversion and resizing of images before applying VGG-16 architecture of Convolution neural network for feature extraction. Finally, for classification SVM, Linear Discriminate, K means clustering, and Decision tree classifiers are used.ResultsThe experimental result shows that the average accuracy of 99.95% is achieved for the classification of the fMRI dataset, while the average accuracy of 73.46% is achieved with the PET dataset. On comparing results on the basis of accuracy, specificity, sensitivity and on some other parameters we found that these results are better than existing methods.Conclusionsthis paper, suggested a unique way to increase the performance of CNN models by applying some preprocessing on image dataset before sending to CNN architecture for feature extraction. We applied this method on ADNI database and on comparing the accuracies with other similar approaches it shows better results.     

Machine learning monitoring for laser osteotomy

This work proposes a new online monitoring method for an assistance during laser osteotomy. The method allows differentiating the type of ablated tissue and the applied dose of laser energy. The setup analyzes the laser-induced acoustic emission, detected by an airborne microphone sensor. The analysis of the acoustic signals is carried out using a machine learning algorithm that is pre-trained in a supervised manner. The efficiency of the method is experimentally evaluated with several types of tissues, which are: skin, fat, muscle, and bone. Several cutting-edge machine learning frameworks are tested for the comparison with the resulting classification accuracy in the range of 84–99%. It is shown that the datasets for the training of the machine learning algorithms are easy to collect in real-life conditions. In the future, this method could assist the doctors during laser osteotomy, minimizing the damage of the nearby healthy tissues and provide cleaner pathologic tissue removal.     

Label‐free diagnosis for colorectal cancer through coffee ring‐assisted surface‐enhanced Raman spectroscopy on blood serum

Surface‐enhanced Raman spectroscopy (SERS) is garnering considerable attention for the swift diagnosis of pathogens and abnormal biological status, that is, cancers. In this work, a simple, fast and inexpensive optical sensing platform is developed by the design of SERS sampling and data analysis. The pretreatment of spectral measurement employed gold nanoparticle colloid mixing with the serum from patients with colorectal cancer (CRC). The droplet of particle‐serum mixture formed coffee‐ring‐like region at the rim, providing strong and stable SERS profiles. The obtained spectra from cancer patients and healthy volunteers were analyzed by unsupervised principal component analysis (PCA) and supervised machine learning model, such as support‐vector machine (SVM), respectively. The results demonstrate that the SVM model provides the superior performance in the classification of CRC diagnosis compared with PCA. In addition, the values of carcinoembryonic antigen from the blood samples were compiled with the corresponding SERS spectra for SVM calculation, yielding improved prediction results.     

Automated classification of hepatocellular carcinoma differentiation using multiphoton microscopy and deep learning

In the case of hepatocellular carcinoma (HCC) samples, classification of differentiation is crucial for determining prognosis and treatment strategy decisions. However, a label‐free and automated classification system for HCC grading has not been yet developed. Hence, in this study, we demonstrate the fusion of multiphoton microscopy and a deep‐learning algorithm for classifying HCC differentiation to produce an innovative computer‐aided diagnostic method. Convolutional neural networks based on the VGG‐16 framework were trained using 217 combined two‐photon excitation fluorescence and second‐harmonic generation images; the resulting classification accuracy of the HCC differentiation grade was over 90%. Our results suggest that a combination of multiphoton microscopy and deep learning can realize label‐free, automated methods for various tissues, diseases and other related classification problems.      

Effect of hemolysis on Fourier transform infrared and Raman spectra of blood plasma

Hemolysis is a very common phenomenon and is referred as the release of intracellular components from red blood cells to the extracellular fluid. Hemolyzed samples are often rejected in clinics due to the interference of hemoglobin and intracellular components in laboratory measurements. Plasma and serum based vibrational spectroscopy studies are extensively applied to generate spectral biomarkers for various diseases. However, no studies have reported the effect of hemolysis in blood based vibrational spectroscopy studies. This study was undertaken to evaluate the effect of hemolysis on infrared and Raman spectra of blood plasma. In this study, prostate cancer plasma samples (n = 30) were divided into three groups (nonhemolyzed, mildly hemolyzed, and moderately hemolyzed) based on the degree of hemolysis and FTIR and Raman spectra were recorded using high throughput (HT)‐FTIR and HT‐Raman spectroscopy. Discrimination was observed between the infrared and Raman spectra of nonhemolyzed and hemolyzed plasma samples using principal component analysis. A classical least square fitting analysis showed differences in the weighting of pure components in nonhemolyzed and hemolyzed plasma samples. Therefore, it is worth to consider the changes in spectral features due to hemolysis when comparing the results within and between experiments.     

3DUNetGSFormer: A deep learning pipeline for complex wetland mapping using generative adversarial networks and Swin transformer

Many ecosystems, particularly wetlands, are significantly degraded or lost as a result of climate change and anthropogenic activities. Simultaneously, developments in machine learning, particularly deep learning methods, have greatly improved wetland mapping, which is a critical step in ecosystem monitoring. Yet, present deep and very deep models necessitate a greater number of training data, which are costly, logistically challenging, and time-consuming to acquire. Thus, we explore and address the potential and possible limitations caused by the availability of limited ground-truth data for large-scale wetland mapping. To overcome this persistent problem for remote sensing data classification using deep learning models, we propose 3D UNet Generative Adversarial Network Swin Transformer (3DUNetGSFormer) to adaptively synthesize wetland training data based on each class's data availability. Both real and synthesized training data are then imported to a novel deep learning architecture consisting of cutting-edge Convolutional Neural Networks and vision transformers for wetland mapping. Results demonstrated that the developed wetland classifier obtained a high level of kappa coefficient, average accuracy, and overall accuracy of 96.99%, 97.13%, and 97.39%, respectively, for the data in three pilot sites in and around Grand Falls-Windsor, Avalon, and Gros Morne National Park located in Canada. The results show that the proposed methodology opens a new window for future high-quality wetland data generation and classification. The developed codes are available at https://github.com/aj1365/3DUNetGSFormer.     

Intraspecific differentiation of sandflies specimens by optical spectroscopy and multivariate analysis

Lutzomyia longipalpis and Lutzomyia cruzi are the main sandflies species involved in the transmission of Leishmania infantum protozoan in Brazil. The morphological characteristics can be used for species identification of males specimens, while females are indistinguishable. Although, sandflies identification is essential to understand vectorial capacity, and susceptibility to infectious agents or insecticides, there is a lack of new strategies for specimen identification. In this study, Fourier transform infrared photoacoustic spectroscopy combined with multivariate analysis identified intraspecific differences between Lutzomyia populations. Successfully group clustering was achieved by principal component analysis. The main differences observed can be related to the protein content of the specimens. A classification with 100% accuracy was obtained using machine learning approach, allowing the identification of sandflies specimens.     

Hyperspectral imaging and characterization of allergic contact dermatitis in the short‐wave infrared

Short‐wave infrared hyperspectral imaging is applied to diagnose and monitor a case of allergic contact dermatitis (ACD) due to poison ivy exposure in one subject. This approach directly demonstrates increased tissue fluid content in ACD lesional skin with a spectral signature that matches the spectral signature of intradermally injected normal saline. The best contrast between the affected and unaffected skin is achieved through a selection of specific wavelengths at 1070, 1340 and 1605 nm and combining them in a pseudo‐red‐green‐blue color space. An image derived from these wavelengths normalized to unaffected skin defines a “tissue fluid index” that may aid in the quantitative diagnosis and monitoring of ACD. Further clinical testing of this promising approach towards disease detection and monitoring with tissue fluid content quantification is warranted.     

Automated analysis of high‐content microscopy data with deep learning

Existing computational pipelines for quantitative analysis of high‐content microscopy data rely on traditional machine learning approaches that fail to accurately classify more than a single dataset without substantial tuning and training, requiring extensive analysis. Here, we demonstrate that the application of deep learning to biological image data can overcome the pitfalls associated with conventional machine learning classifiers. Using a deep convolutional neural network (DeepLoc) to analyze yeast cell images, we show improved performance over traditional approaches in the automated classification of protein subcellular localization. We also demonstrate the ability of DeepLoc to classify highly divergent image sets, including images of pheromone‐arrested cells with abnormal cellular morphology, as well as images generated in different genetic backgrounds and in different laboratories. We offer an open‐source implementation that enables updating DeepLoc on new microscopy datasets. This study highlights deep learning as an important tool for the expedited analysis of high‐content microscopy data.