Image-based plant phenotyping with incremental learning and active contours

Plant phenotyping investigates how a plant's genome, interacting with the environment, affects the observable traits of a plant (phenome). It is becoming increasingly important in our quest towards efficient and sustainable agriculture. While sequencing the genome is becoming increasingly efficient, acquiring phenotype information has remained largely of low throughput. Current solutions for automated image-based plant phenotyping, rely either on semi-automated or manual analysis of the imaging data, or on expensive and proprietary software which accompanies costly hardware infrastructure. While some attempts have been made to create software applications that enable the analysis of such images in an automated fashion, most solutions are tailored to particular acquisition scenarios and restrictions on experimental design. In this paper we propose and test, a method for the segmentation and the automated analysis of time-lapse plant images from phenotyping experiments in a general laboratory setting, that can adapt to scene variability. The method involves minimal user interaction, necessary to establish the statistical experiments that may follow. At every time instance (i.e., a digital photograph), it segments the plants in images that contain many specimens of the same species. For accurate plant segmentation we propose a vector valued level set formulation that incorporates features of color intensity, local texture, and prior knowledge. Prior knowledge is incorporated using a plant appearance model implemented with Gaussian mixture models, which utilizes incrementally information from previously segmented instances. The proposed approach is tested on Arabidopsis plant images acquired with a static camera capturing many subjects at the same time. Our validation with ground truth segmentations and comparisons with state-of-the-art methods in the literature shows that the proposed method is able to handle images with complicated and changing background in an automated fashion. An accuracy of 96.7% (dice similarity coefficient) was observed, which was higher than other methods used for comparison. While here it was tested on a single plant species, the fact that we do not employ shape driven models and we do not rely on fully supervised classification (trained on a large dataset) increases the ease of deployment of the proposed solution for the study of different plant species in a variety of laboratory settings. Our solution will be accompanied by an easy to use graphical user interface and, to facilitate adoption, we will make the software available to the scientific community.     

Multi-well plate cell contraction assay detects negatively correlated cellular responses to pharmacological inhibitors in contractility and migration

To enable large-scale screening of signaling molecules and drugs that regulate cellular contractility-associated mechanotransduction, we previously modified, particularly in terms of the capability of efficiently collecting big data, conventional methodologies using wrinkled substrates to determine the cellular contractility. Here, we present a new system to perform the wrinkle-based cell force assay in a multi-well plate format conformed to standardized geometric configurations and compatible with available technologies such as automated plate readers. With this highly improved throughput in terms of hardware as well as software using a deep learning-based technology, we evaluated the effect of treating cells with various types of pharmacological inhibitors on the cellular contractility. We found opposite responses of cells to the inhibitors between the contractility and collective migration activities. While similar inverse relationships between the contractility and migration have been reported in separate studies, our results here with the high-throughput screening system more broadly generalized these observations.     

Markerless 2D kinematic analysis of underwater running: A deep learning approach

Kinematic analysis is often performed with a camera system combined with reflective markers placed over bony landmarks. This method is restrictive (and often expensive), and limits the ability to perform analyses outside of the lab. In the present study, we used a markerless deep learning-based method to perform 2D kinematic analysis of deep water running, a task that poses several challenges to image processing methods. A single GoPro camera recorded sagittal plane lower limb motion. A deep neural network was trained using data from 17 individuals, and then used to predict the locations of markers that approximated joint centres. We found that 300–400 labelled images were sufficient to train the network to be able to position joint markers with an accuracy similar to that of a human labeler (mean difference < 3 pixels, around 1 cm). This level of accuracy is sufficient for many 2D applications, such as sports biomechanics, coaching/training, and rehabilitation. The method was sensitive enough to differentiate between closely-spaced running cadences (45–85 strides per minute in increments of 5). We also found high test–retest reliability of mean stride data, with between-session correlation coefficients of 0.90–0.97. Our approach represents a low-cost, adaptable solution for kinematic analysis, and could easily be modified for use in other movements and settings. Using additional cameras, this approach could also be used to perform 3D analyses. The method presented here may have broad applications in different fields, for example by enabling markerless motion analysis to be performed during rehabilitation, training or even competition environments.     

Hyperspectral imaging combined with machine learning as a tool to obtain high‐throughput plant salt‐stress phenotyping

The rapid selection of salinity‐tolerant crops to increase food production in salinized lands is important for sustainable agriculture. Recently, high‐throughput plant phenotyping technologies have been adopted that use plant morphological and physiological measurements in a non‐destructive manner to accelerate plant breeding processes. Here, a hyperspectral imaging (HSI) technique was implemented to monitor the plant phenotypes of 13 okra (Abelmoschus esculentus L.) genotypes after 2 and 7 days of salt treatment. Physiological and biochemical traits, such as fresh weight, SPAD, elemental contents and photosynthesis‐related parameters, which require laborious, time‐consuming measurements, were also investigated. Traditional laboratory‐based methods indicated the diverse performance levels of different okra genotypes in response to salinity stress. We introduced improved plant and leaf segmentation approaches to RGB images extracted from HSI imaging based on deep learning. The state‐of‐the‐art performance of the deep‐learning approach for segmentation resulted in an intersection over union score of 0.94 for plant segmentation and a symmetric best dice score of 85.4 for leaf segmentation. Moreover, deleterious effects of salinity affected the physiological and biochemical processes of okra, which resulted in substantial changes in the spectral information. Four sample predictions were constructed based on the spectral data, with correlation coefficients of 0.835, 0.704, 0.609 and 0.588 for SPAD, sodium concentration, photosynthetic rate and transpiration rate, respectively. The results confirmed the usefulness of high‐throughput phenotyping for studying plant salinity stress using a combination of HSI and deep‐learning approaches.     

Structure-based protein design with deep learning

Since the first revelation of proteins functioning as macromolecular machines through their three dimensional structures, researchers have been intrigued by the marvelous ways the biochemical processes are carried out by proteins. The aspiration to understand protein structures has fueled extensive efforts across different scientific disciplines. In recent years, it has been demonstrated that proteins with new functionality or shapes can be designed via structure-based modeling methods, and the design strategies have combined all available information — but largely piece-by-piece — from sequence derived statistics to the detailed atomic-level modeling of chemical interactions. Despite the significant progress, incorporating data-derived approaches through the use of deep learning methods can be a game changer. In this review, we summarize current progress, compare the arc of developing the deep learning approaches with the conventional methods, and describe the motivation and concepts behind current strategies that may lead to potential future opportunities.     

AI applications to medical images: From machine learning to deep learning

PurposeArtificial intelligence (AI) models are playing an increasing role in biomedical research and healthcare services. This review focuses on challenges points to be clarified about how to develop AI applications as clinical decision support systems in the real-world context.MethodsA narrative review has been performed including a critical assessment of articles published between 1989 and 2021 that guided challenging sections.ResultsWe first illustrate the architectural characteristics of machine learning (ML)/radiomics and deep learning (DL) approaches. For ML/radiomics, the phases of feature selection and of training, validation, and testing are described. DL models are presented as multi-layered artificial/convolutional neural networks, allowing us to directly process images. The data curation section includes technical steps such as image labelling, image annotation (with segmentation as a crucial step in radiomics), data harmonization (enabling compensation for differences in imaging protocols that typically generate noise in non-AI imaging studies) and federated learning. Thereafter, we dedicate specific sections to: sample size calculation, considering multiple testing in AI approaches; procedures for data augmentation to work with limited and unbalanced datasets; and the interpretability of AI models (the so-called black box issue). Pros and cons for choosing ML versus DL to implement AI applications to medical imaging are finally presented in a synoptic way.ConclusionsBiomedicine and healthcare systems are one of the most important fields for AI applications and medical imaging is probably the most suitable and promising domain. Clarification of specific challenging points facilitates the development of such systems and their translation to clinical practice.     

Modeling of dynamical systems through deep learning

This review presents a modern perspective on dynamical systems in the context of current goals and open challenges. In particular, our review focuses on the key challenges of discovering dynamics from data and finding data-driven representations that make nonlinear systems amenable to linear analysis. We explore various challenges in modern dynamical systems, along with emerging techniques in data science and machine learning to tackle them. The two chief challenges are (1) nonlinear dynamics and (2) unknown or partially known dynamics. Machine learning is providing new and powerful techniques for both challenges. Dimensionality reduction methods are used for projecting dynamical methods in reduced form, and these methods perform computational efficiency on real-world data. Data-driven models drive to discover the governing equations and give laws of physics. The identification of dynamical systems through deep learning techniques succeeds in inferring physical systems. Machine learning provides advanced new and powerful algorithms for nonlinear dynamics. Advanced deep learning methods like autoencoders, recurrent neural networks, convolutional neural networks, and reinforcement learning are used in modeling of dynamical systems.     

Complex networks and deep learning for EEG signal analysis

Electroencephalogram (EEG) signals acquired from brain can provide an effective representation of the human’s physiological and pathological states. Up to now, much work has been conducted to study and analyze the EEG signals, aiming at spying the current states or the evolution characteristics of the complex brain system. Considering the complex interactions between different structural and functional brain regions, brain network has received a lot of attention and has made great progress in brain mechanism research. In addition, characterized by autonomous, multi-layer and diversified feature extraction, deep learning has provided an effective and feasible solution for solving complex classification problems in many fields, including brain state research. Both of them show strong ability in EEG signal analysis, but the combination of these two theories to solve the difficult classification problems based on EEG signals is still in its infancy. We here review the application of these two theories in EEG signal research, mainly involving brain–computer interface, neurological disorders and cognitive analysis. Furthermore, we also develop a framework combining recurrence plots and convolutional neural network to achieve fatigue driving recognition. The results demonstrate that complex networks and deep learning can effectively implement functional complementarity for better feature extraction and classification, especially in EEG signal analysis.     

Automated analysis of high‐content microscopy data with deep learning

Existing computational pipelines for quantitative analysis of high‐content microscopy data rely on traditional machine learning approaches that fail to accurately classify more than a single dataset without substantial tuning and training, requiring extensive analysis. Here, we demonstrate that the application of deep learning to biological image data can overcome the pitfalls associated with conventional machine learning classifiers. Using a deep convolutional neural network (DeepLoc) to analyze yeast cell images, we show improved performance over traditional approaches in the automated classification of protein subcellular localization. We also demonstrate the ability of DeepLoc to classify highly divergent image sets, including images of pheromone‐arrested cells with abnormal cellular morphology, as well as images generated in different genetic backgrounds and in different laboratories. We offer an open‐source implementation that enables updating DeepLoc on new microscopy datasets. This study highlights deep learning as an important tool for the expedited analysis of high‐content microscopy data.     

A deep learning approach to gold nanoparticle quantification in computed tomography

IntroductionDeep learning (DL) is used to classify, detect, and quantify gold nanoparticles (AuNPs) in a human-sized phantom with a clinical MDCT scanner.MethodsAuNPs were imaged at concentrations between 0.0274 and 200 mgAu/mL in a 33 cm phantom. 1 mm-thick CT image slices were acquired at 120 kVp with a CTDI_vol of 23.6 mGy. A convolutional neural network (CNN) was trained on 544 images to classify 17 different tissue types and AuNP concentrations. A second set of 544 images was then used for testing.ResultsAuNPs were classified with 95% accuracy at 0.1095 mgAu/mL and 97% accuracy at 0.2189 mgAu/mL. Both these concentrations are lower than what humans can visually perceive (0.3–1.4 mgAu/mL). AuNP concentrations were also classified with 95% accuracy at 150 and 200 mgAu/mL. These high concentrations result in CT numbers that are at or above the 12-bit limit for CT’s dynamic range where extended Hounsfield scales are otherwise required for measuring differences in contrast.ConclusionsWe have shown that DL can be used to detect AuNPs at concentrations lower than what humans can visually perceive and can also quantify very high AuNP concentrations that exceed the typical 12-bit dynamic range of clinical MDCT scanners. This second finding is possible due to inhomogeneous AuNP distributions and characteristic streak artifacts. It may even be possible to extend this approach beyond AuNP imaging in CT for quantifying high density objects without extended Hounsfield scales.     

Task-based characterization of a deep learning image reconstruction and comparison with filtered back-projection and a partial model-based iterative reconstruction in abdominal CT: A phantom study

PurposeWe aimed to thoroughly characterize image quality of a novel deep learning image reconstruction (DLIR), and investigate its potential for dose reduction in abdominal CT in comparison with filtered back-projection (FBP) and a partial model-based iterative reconstruction (ASiR-V).MethodsWe scanned a phantom at three dose levels: regular (7 mGy), low (3 mGy) and ultra-low (1 mGy). Images were reconstructed using DLIR (low, medium and high levels) and ASiR-V (0% = FBP, 50% and 100%). Noise and contrast-dependent spatial resolution were characterized by computing noise power spectra and target transfer functions, respectively. Detectability indexes of simulated acute appendicitis or colonic diverticulitis (low contrast), and calcium-containing urinary stones (high contrast) (|ΔHU| = 50 and 500, respectively) were calculated using the nonprewhitening with eye filter model observer.ResultsAt all dose levels, increasing DLIR and ASiR-V levels both markedly decreased noise magnitude compared with FBP, with DLIR low and medium maintaining noise texture overall. For both low- and high-contrast spatial resolution, DLIR not only maintained, but even slightly enhanced spatial resolution in comparison with FBP across all dose levels. Conversely, increasing ASiR-V impaired low-contrast spatial resolution compared with FBP. Overall, DLIR outperformed ASiR-V in all simulated clinical scenarios. For both low- and high-contrast diagnostic tasks, increasing DLIR substantially enhanced detectability at any dose and contrast levels for any simulated lesion size.ConclusionsUnlike ASiR-V, DLIR substantially reduces noise while maintaining noise texture and slightly enhancing spatial resolution overall. DLIR outperforms ASiR-V by enabling higher detectability of both low- and high-contrast simulated abdominal lesions across all investigated dose levels.     

Preserving image texture while reducing radiation dose with a deep learning image reconstruction algorithm in chest CT: A phantom study

PurposeTo assess whether a deep learning image reconstruction algorithm (TrueFidelity) can preserve the image texture of conventional filtered back projection (FBP) at reduced dose levels attained by ASIR-V in chest CT.MethodsPhantom images were acquired using a clinical chest protocol (7.6 mGy) and two levels of dose reduction (60% and 80%). Images were reconstructed with FBP, ASIR-V (50% and 100% blending) and TrueFidelity (low (DL-L), medium (DL-M) and high (DL-H) strength). Noise (SD), noise power spectrum (NPS) and task-based transfer function (TTF) were calculated. Noise texture was quantitatively compared by computing root-mean-square deviations (RMSD) of NPS with respect to FBP. Four experienced readers performed a contrast-detail evaluation. The dose reducing potential of TrueFidelity compared to ASIR-V was assessed by fitting SD and contrast-detail as a function of dose.ResultsDL-M and DL-H reduced noise and NPS area compared to FBP and 50% ASIR-V, at all dose levels. At 7.6 mGy, NPS of ASIR-V 50/100% was shifted towards lower frequencies (f_peak = 0.22/0.13 mm⁻¹, RMSD = 0.14/0.38), with respect to FBP (f_peak = 0.30 mm⁻¹). Marginal difference was observed for TrueFidelity: f_peak = 0.33/0.30/0.30 mm⁻¹ and RMSD = 0.03/0.04/0.07 for L/M/H strength. Values of TTF_50% were independent of DL strength and higher compared to FBP and ASIR-V, at all dose and contrast levels. Contrast-detail was highest for DL-H at all doses. Compared to 50% ASIR-V, DL-H had an estimated dose reducing potential of 50% on average, without impairing noise, texture and detectability.ConclusionsTrueFidelity preserves the image texture of FBP, while outperforming ASIR-V in terms of noise, spatial resolution and detectability at lower doses.     

深度学习在蜜蜂研究中的应用

实现对蜜蜂蜂群的实时动态监测,有助于养蜂业的数字化与智能化发展,对大幅提升养蜂管理水平具有重要意义。深度学习作为人工智能的一种新的研究方向,目前已被广泛应用于昆虫分类学、行为学、害虫生物防治等领域。随着深度学习检测算法的迅速发展,基于深度学习的蜜蜂蜂群监测技术不断涌现,为智能化养蜂提供了可能。为促进深度学习在蜜蜂领域的进一步应用,本文梳理了深度学习在蜜蜂的物种识别、行为跟踪监测、蜂群健康监测和蜂巢监测等方面的研究进展,分析了深度学习技术在蜜蜂蜂群监测研究及应用中存在的一些问题和未来发展方向,为深度学习在蜜蜂领域的应用提出了建议。     

Comparison of different deep learning architectures for synthetic CT generation from MR images

PurposeAmong the different available methods for synthetic CT generation from MR images for the task of MR-guided radiation planning, the deep learning algorithms have and do outperform their conventional counterparts. In this study, we investigated the performance of some most popular deep learning architectures including eCNN, U-Net, GAN, V-Net, and Res-Net for the task of sCT generation. As a baseline, an atlas-based method is implemented to which the results of the deep learning-based model are compared.MethodsA dataset consisting of 20 co-registered MR-CT pairs of the male pelvis is applied to assess the different sCT production methods' performance. The mean error (ME), mean absolute error (MAE), Pearson correlation coefficient (PCC), structural similarity index (SSIM), and peak signal-to-noise ratio (PSNR) metrics were computed between the estimated sCT and the ground truth (reference) CT images.ResultsThe visual inspection revealed that the sCTs produced by eCNN, V-Net, and ResNet, unlike the other methods, were less noisy and greatly resemble the ground truth CT image. In the whole pelvis region, the eCNN yielded the lowest MAE (26.03 ± 8.85 HU) and ME (0.82 ± 7.06 HU), and the highest PCC metrics were yielded by the eCNN (0.93 ± 0.05) and ResNet (0.91 ± 0.02) methods. The ResNet model had the highest PSNR of 29.38 ± 1.75 among all models. In terms of the Dice similarity coefficient, the eCNN method revealed superior performance in major tissue identification (air, bone, and soft tissue).ConclusionsAll in all, the eCNN and ResNet deep learning methods revealed acceptable performance with clinically tolerable quantification errors.     

Videomicroscopic extraction of specific information on cell proliferation and migration in vitro

In vitro cell imaging is a useful exploratory tool for cell behavior monitoring with a wide range of applications in cell biology and pharmacology. Combined with appropriate image analysis techniques, this approach has been shown to provide useful information on the detection and dynamic analysis of cell events. In this context, numerous efforts have been focused on cell migration analysis. In contrast, the cell division process has been the subject of fewer investigations. The present work focuses on this latter aspect and shows that, in complement to cell migration data, interesting information related to cell division can be extracted from phase-contrast time-lapse image series, in particular cell division duration, which is not provided by standard cell assays using endpoint analyses. We illustrate our approach by analyzing the effects induced by two sigma-1 receptor ligands (haloperidol and 4-IBP) on the behavior of two glioma cell lines using two in vitro cell models, i.e., the low-density individual cell model and the high-density scratch wound model. This illustration also shows that the data provided by our approach are suggestive as to the mechanism of action of compounds, and are thus capable of informing the appropriate selection of further time-consuming and more expensive biological evaluations required to elucidate a mechanism.     

Cell phenotyping in saliva of individuals under psychological stress

Total leukocytes, NK cells, B and T lymphocytes present in the saliva of medical students with or without stress were quantified by flow cytometry in 10,000 events. The symptoms of psychological stress were monitored with Lipp’s Inventory of Stress Symptoms for Adults (ISSL). No significant differences were observed in the number of cells phenotyped in students with and those without psychological stress. However, a negative correlation was observed between the number of NK cells and T lymphocytes in students with stress (r = −0.8173; p = 0.0058), suggesting that innate immunity is predominant in the adaptation phase.