A 3D Network Based Shape Prior for Automatic Myocardial Disease Segmentation in Delayed-Enhancement MRI

Objectives: In this work, a new deep learning model for relevant myocardial infarction segmentation from Late Gadolinium Enhancement (LGE)-MRI is proposed. Moreover, our novel segmentation method aims to detect microvascular-obstructed regions accurately. Material and methods: We first segment the anatomical structures, i.e., the left ventricular cavity and the myocardium, to achieve a preliminary segmentation. Then, a shape prior based framework that fuses the 3D U-Net architecture with 3D Autoencoder segmentation framework to constrain the segmentation process of pathological tissues is applied. Results: The proposed network reached outstanding myocardial segmentation compared with the human-level performance with the average Dice score of ‘0.9507’ for myocardium, ‘0.7656’ for scar, and ‘0.8377’ for MVO on the validation set consisting of 16 DE-MRI volumes selected from the training EMIDEC dataset. Conclusion: It is concluded that our approach's extensive validation and comprehensive comparison against existing state-of-the-art deep learning models on three annotated datasets, including healthy and diseased exams, make this proposal a reliable tool to enhance MI diagnosis.     

A Review of the Deep Learning Methods for Medical Images Super Resolution Problems

Super resolution problems are widely discussed in medical imaging. Spatial resolution of medical images are not sufficient due to the constraints such as image acquisition time, low irradiation dose or hardware limits. To address these problems, different super resolution methods have been proposed, such as optimization or learning-based approaches. Recently, deep learning methods become a thriving technology and are developing at an exponential speed. We think it is necessary to write a review to present the current situation of deep learning in medical imaging super resolution. In this paper, we first briefly introduce deep learning methods, then present a number of important deep learning approaches to solve super resolution problems, different architectures as well as up-sampling operations will be introduced. Afterwards, we focus on the applications of deep learning methods in medical imaging super resolution problems, the challenges to overcome will be presented as well.     

AI for COVID-19 Detection from Radiographs: Incisive Analysis of State of the Art Techniques,Key Challenges and Future Directions

Background and objectiveIn recent years, Artificial Intelligence has had an evident impact on the way research addresses challenges in different domains. It has proven to be a huge asset, especially in the medical field, allowing for time-efficient and reliable solutions. This research aims to spotlight the impact of deep learning and machine learning models in the detection of COVID-19 from medical images. This is achieved by conducting a review of the state-of-the-art approaches proposed by the recent works in this field.MethodsThe main focus of this study is the recent developments of classification and segmentation approaches to image-based COVID-19 detection. The study reviews 140 research papers published in different academic research databases. These papers have been screened and filtered based on specified criteria, to acquire insights prudent to image-based COVID-19 detection.ResultsThe methods discussed in this review include different types of imaging modality, predominantly X-rays and CT scans. These modalities are used for classification and segmentation tasks as well. This review seeks to categorize and discuss the different deep learning and machine learning architectures employed for these tasks, based on the imaging modality utilized. It also hints at other possible deep learning and machine learning architectures that can be proposed for better results towards COVID-19 detection. Along with that, a detailed overview of the emerging trends and breakthroughs in Artificial Intelligence-based COVID-19 detection has been discussed as well.ConclusionThis work concludes by stipulating the technical and non-technical challenges faced by researchers and illustrates the advantages of image-based COVID-19 detection with Artificial Intelligence techniques.     

A Comparison of Supervised Machine Learning Algorithms and Feature Vectors for MS Lesion Segmentation Using Multimodal Structural MRI

Machine learning is a popular method for mining and analyzing large collections of medical data. We focus on a particular problem from medical research, supervised multiple sclerosis (MS) lesion segmentation in structural magnetic resonance imaging (MRI). We examine the extent to which the choice of machine learning or classification algorithm and feature extraction function impacts the performance of lesion segmentation methods. As quantitative measures derived from structural MRI are important clinical tools for research into the pathophysiology and natural history of MS, the development of automated lesion segmentation methods is an active research field. Yet, little is known about what drives performance of these methods. We evaluate the performance of automated MS lesion segmentation methods, which consist of a supervised classification algorithm composed with a feature extraction function. These feature extraction functions act on the observed T1-weighted (T1-w), T2-weighted (T2-w) and fluid-attenuated inversion recovery (FLAIR) MRI voxel intensities. Each MRI study has a manual lesion segmentation that we use to train and validate the supervised classification algorithms. Our main finding is that the differences in predictive performance are due more to differences in the feature vectors, rather than the machine learning or classification algorithms. Features that incorporate information from neighboring voxels in the brain were found to increase performance substantially. For lesion segmentation, we conclude that it is better to use simple, interpretable, and fast algorithms, such as logistic regression, linear discriminant analysis, and quadratic discriminant analysis, and to develop the features to improve performance.     

Benchmarking of deep learning algorithms for 3D instance segmentation of confocal image datasets

Segmenting three-dimensional (3D) microscopy images is essential for understanding phenomena like morphogenesis, cell division, cellular growth, and genetic expression patterns. Recently, deep learning (DL) pipelines have been developed, which claim to provide high accuracy segmentation of cellular images and are increasingly considered as the state of the art for image segmentation problems. However, it remains difficult to define their relative performances as the concurrent diversity and lack of uniform evaluation strategies makes it difficult to know how their results compare. In this paper, we first made an inventory of the available DL methods for 3D cell segmentation. We next implemented and quantitatively compared a number of representative DL pipelines, alongside a highly efficient non-DL method named MARS. The DL methods were trained on a common dataset of 3D cellular confocal microscopy images. Their segmentation accuracies were also tested in the presence of different image artifacts. A specific method for segmentation quality evaluation was adopted, which isolates segmentation errors due to under- or oversegmentation. This is complemented with a 3D visualization strategy for interactive exploration of segmentation quality. Our analysis shows that the DL pipelines have different levels of accuracy. Two of them, which are end-to-end 3D and were originally designed for cell boundary detection, show high performance and offer clear advantages in terms of adaptability to new data.     

Machine learning in quantitative PET: A review of attenuation correction and low-count image reconstruction methods

The rapid expansion of machine learning is offering a new wave of opportunities for nuclear medicine. This paper reviews applications of machine learning for the study of attenuation correction (AC) and low-count image reconstruction in quantitative positron emission tomography (PET). Specifically, we present the developments of machine learning methodology, ranging from random forest and dictionary learning to the latest convolutional neural network-based architectures. For application in PET attenuation correction, two general strategies are reviewed: 1) generating synthetic CT from MR or non-AC PET for the purposes of PET AC, and 2) direct conversion from non-AC PET to AC PET. For low-count PET reconstruction, recent deep learning-based studies and the potential advantages over conventional machine learning-based methods are presented and discussed. In each application, the proposed methods, study designs and performance of published studies are listed and compared with a brief discussion. Finally, the overall contributions and remaining challenges are summarized.     

State-of-the-art computational methods to predict protein–protein interactions with high accuracy and coverage

Prediction of protein–protein interactions (PPIs) commonly involves a significant computational component. Rapid recent advances in the power of computational methods for protein interaction prediction motivate a review of the state-of-the-art. We review the major approaches, organized according to the primary source of data utilized: protein sequence, protein structure, and protein co-abundance. The advent of deep learning (DL) has brought with it significant advances in interaction prediction, and we show how DL is used for each source data type. We review the literature taxonomically, present example case studies in each category, and conclude with observations about the strengths and weaknesses of machine learning methods in the context of the principal sources of data for protein interaction prediction.     

CoordConv-Unet: Investigating CoordConv for Organ Segmentation

ObjectivesConvolutional neural networks (CNNs) have established state-of-the-art performance in computer vision tasks such as object detection and segmentation. One of the major remaining challenges concerns their ability to capture consistent spatial and anatomically plausible attributes in medical image segmentation. To address this issue, many works advocate to integrate prior information at the level of the loss function. However, prior-based losses often suffer from local solutions and training instability. The CoordConv layers are extensions of convolutional neural network wherein convolution is conditioned on spatial coordinates. The objective of this paper is to investigate CoordConv as a proficient substitute to convolutional layers for medical image segmentation tasks when trained under prior-based losses.MethodsThis work introduces CoordConv-Unet which is a novel structure that can be used to accommodate training under anatomical prior losses. The proposed architecture demonstrates a dual role relative to prior constrained CNN learning: it either demonstrates a regularizing role that stabilizes learning while maintaining system performance, or improves system performance by allowing the learning to be more stable and to evade local minima.ResultsTo validate the performance of the proposed model, experiments are conducted on two well-known public datasets from the Decathlon challenge: a mono-modal MRI dataset dedicated to segmentation of the left atrium, and a CT image dataset whose objective is to segment the spleen, an organ characterized with varying size and mild convexity issues.ConclusionResults show that, despite the inadequacy of CoordConv when trained with the regular dice baseline loss, the proposed CoordConv-Unet structure can improve significantly model performance when trained under anatomically constrained prior losses.     

Deep learning-based image processing in optical microscopy

Optical microscopy has emerged as a key driver of fundamental research since it provides the ability to probe into imperceptible structures in the biomedical world. For the detailed investigation of samples, a high-resolution image with enhanced contrast and minimal damage is preferred. To achieve this, an automated image analysis method is preferable over manual analysis in terms of both speed of acquisition and reduced error accumulation. In this regard, deep learning (DL)-based image processing can be highly beneficial. The review summarises and critiques the use of DL in image processing for the data collected using various optical microscopic techniques. In tandem with optical microscopy, DL has already found applications in various problems related to image classification and segmentation. It has also performed well in enhancing image resolution in smartphone-based microscopy, which in turn enablse crucial medical assistance in remote places.Graphical abstract     

Deep learning a boon for biophotonics?

This review covers original articles using deep learning in the biophotonic field published in the last years. In these years deep learning, which is a subset of machine learning mostly based on artificial neural network geometries, was applied to a number of biophotonic tasks and has achieved state‐of‐the‐art performances. Therefore, deep learning in the biophotonic field is rapidly growing and it will be utilized in the next years to obtain real‐time biophotonic decision‐making systems and to analyze biophotonic data in general. In this contribution, we discuss the possibilities of deep learning in the biophotonic field including image classification, segmentation, registration, pseudostaining and resolution enhancement. Additionally, we discuss the potential use of deep learning for spectroscopic data including spectral data preprocessing and spectral classification. We conclude this review by addressing the potential applications and challenges of using deep learning for biophotonic data.     

AMB-Wnet: Embedding attention model in multi-bridge Wnet for exploring the mechanics of disease

In recent years, progressive application of convolutional neural networks in image processing has successfully filtered into medical diagnosis. As a prerequisite for images detection and classification, object segmentation in medical images has attracted a great deal of attention. This study is based on the fact that most of the analysis of pathological diagnoses requires nuclei detection as the starting phase for obtaining an insight into the underlying biological process and further diagnosis. In this paper, we introduce an embedded attention model in multi-bridge Wnet (AMB-Wnet) to achieve suppression of irrelevant background areas and obtain good features for learning image semantics and modality to automatically segment nuclei, inspired by the 2018 Data Science Bowl. The proposed architecture, consisting of the redesigned down sample group, up-sample group, and middle block (a new multiple-scale convolutional layers block), is designed to extract different level features. In addition, a connection group is proposed instead of skip-connection to transfer semantic information among different levels. In addition, the attention model is well embedded in the connection group, and the performance of the model is improved without increasing the amount of calculation. To validate the model's performance, we evaluated it using the BBBC038V1 data sets for nuclei segmentation. Our proposed model achieves 85.83% F1-score, 97.81% accuracy, 86.12% recall, and 83.52% intersection over union. The proposed AMB-Wnet exhibits superior results compared to the original U-Net, MultiResUNet, and recent Attention U-Net architecture.     

Artificial intelligence and machine learning for medical imaging: A technology review

Artificial intelligence (AI) has recently become a very popular buzzword, as a consequence of disruptive technical advances and impressive experimental results, notably in the field of image analysis and processing. In medicine, specialties where images are central, like radiology, pathology or oncology, have seized the opportunity and considerable efforts in research and development have been deployed to transfer the potential of AI to clinical applications. With AI becoming a more mainstream tool for typical medical imaging analysis tasks, such as diagnosis, segmentation, or classification, the key for a safe and efficient use of clinical AI applications relies, in part, on informed practitioners. The aim of this review is to present the basic technological pillars of AI, together with the state-of-the-art machine learning methods and their application to medical imaging. In addition, we discuss the new trends and future research directions. This will help the reader to understand how AI methods are now becoming an ubiquitous tool in any medical image analysis workflow and pave the way for the clinical implementation of AI-based solutions.     

DeepWL: Robust EPID based Winston-Lutz analysis using deep learning,synthetic image generation and optical path-tracing

Radiation therapy requires clinical linear accelerators to be mechanically and dosimetrically calibrated to a high standard. One important quality assurance test is the Winston-Lutz test which localises the radiation isocentre of the linac.In the current work we demonstrate a novel method of analysing EPID based Winston-Lutz QA images using a deep learning model trained only on synthetic image data. In addition, we propose a novel method of generating the synthetic WL images and associated ‘ground-truth’ masks using an optical path-tracing engine to ‘fake’ mega-voltage EPID images.The model called DeepWL was trained on 1500 synthetic WL images using data augmentation techniques for 180 epochs. The model was built using Keras with a TensorFlow backend on an Intel Core i5-6500T CPU and trained in approximately 15 h. DeepWL was shown to produce ball bearing and multi-leaf collimator field segmentations with a mean dice coefficient of 0.964 and 0.994 respectively on previously unseen synthetic testing data. When DeepWL was applied to WL data measured on an EPID, the predicted mean displacements were shown to be statistically similar to the Canny Edge detection method. However, the DeepWL predictions for the ball bearing locations were shown to correlate better with manual annotations compared with the Canny edge detection algorithm.DeepWL was demonstrated to analyse Winston-Lutz images with an accuracy suitable for routine linac quality assurance with some statistical evidence that it may outperform Canny Edge detection methods in terms of segmentation robustness and the resultant displacement predictions.     

Automatic Region-Based Brain Classification of MRI-T1 Data

Image segmentation of medical images is a challenging problem with several still not totally solved issues, such as noise interference and image artifacts. Region-based and histogram-based segmentation methods have been widely used in image segmentation. Problems arise when we use these methods, such as the selection of a suitable threshold value for the histogram-based method and the over-segmentation followed by the time-consuming merge processing in the region-based algorithm. To provide an efficient approach that not only produce better results, but also maintain low computational complexity, a new region dividing based technique is developed for image segmentation, which combines the advantages of both regions-based and histogram-based methods. The proposed method is applied to the challenging applications: Gray matter (GM), White matter (WM) and cerebro-spinal fluid (CSF) segmentation in brain MR Images. The method is evaluated on both simulated and real data, and compared with other segmentation techniques. The obtained results have demonstrated its improved performance and robustness.     

Enhanced radial basis function neural network for tomato plant disease leaf image segmentation

Primary crop losses in agriculture are due to leaf diseases, which farmers cannot identify early. If the diseases are not detected early and correctly, then the farmer will have to undergo huge losses. Therefore, in the field of agriculture, the detection of leaf diseases in tomato crops plays a vital role. Recent advances in computer vision and deep learning techniques have made disease prediction easy in agriculture. Tomato crop front side leaf images are considered for research due to their high exposure to diseases. The image segmentation process assumes a significant role in identifying disease affected areas on tomato leaf images. Therefore, this paper develops an efficient tomato crop leaf disease segmentation model using an enhanced radial basis function neural network (ERBFNN). The proposed ERBFNN is enhanced using the modified sunflower optimization (MSFO) algorithm. Initially, the noise present in the images is removed by a Gaussian filter followed by CLAHE (contrast-limited adaptive histogram equalization) based on contrast enhancement and un-sharp masking. Then, color features are extracted from each leaf image and given to the segmentation stage to segment the disease portion of the input image. The performance of the proposed ERBFNN approach is estimated using different metrics such as accuracy, Jaccard coefficient (JC), Dice's coefficient (DC), precision, recall, F-Measure, sensitivity, specificity, and mean intersection over union (MIoU) and are compared with existing state-of-the-art methods of radial basis function (RBF), fuzzy c-means (FCM), and region growing (RG). The experimental results show that the proposed ERBFNN segmentation model outperformed with an accuracy of 98.92% compared to existing state-of-the-art methods like RBFNN, FCM, and RG, as well as previous research work.     

Multiview Locally Linear Embedding for Effective Medical Image Retrieval

Content-based medical image retrieval continues to gain attention for its potential to assist radiological image interpretation and decision making. Many approaches have been proposed to improve the performance of medical image retrieval system, among which visual features such as SIFT, LBP, and intensity histogram play a critical role. Typically, these features are concatenated into a long vector to represent medical images, and thus traditional dimension reduction techniques such as locally linear embedding (LLE), principal component analysis (PCA), or laplacian eigenmaps (LE) can be employed to reduce the “curse of dimensionality”. Though these approaches show promising performance for medical image retrieval, the feature-concatenating method ignores the fact that different features have distinct physical meanings. In this paper, we propose a new method called multiview locally linear embedding (MLLE) for medical image retrieval. Following the patch alignment framework, MLLE preserves the geometric structure of the local patch in each feature space according to the LLE criterion. To explore complementary properties among a range of features, MLLE assigns different weights to local patches from different feature spaces. Finally, MLLE employs global coordinate alignment and alternating optimization techniques to learn a smooth low-dimensional embedding from different features. To justify the effectiveness of MLLE for medical image retrieval, we compare it with conventional spectral embedding methods. We conduct experiments on a subset of the IRMA medical image data set. Evaluation results show that MLLE outperforms state-of-the-art dimension reduction methods.     

Statistical region-based active contours for segmentation: An overview

In this paper we propose a brief survey on geometric variational approaches and more precisely on statistical region-based active contours for medical image segmentation. In these approaches, image features are considered as random variables whose distribution may be either parametric, and belongs to the exponential family, or non-parametric estimated with a kernel density method. Statistical region-based terms are listed and reviewed showing that these terms can depict a wide spectrum of segmentation problems. A shape prior can also be incorporated to the previous statistical terms. A discussion of some optimization schemes available to solve the variational problem is also provided. Examples on real medical images are given to illustrate some of the given criteria.     

On-line integration of computer controlled diagnostic devices and medical information systems in undergraduate medical physics education for physicians

We designed and evaluated an innovative computer-aided-learning environment based on the on-line integration of computer controlled medical diagnostic devices and a medical information system for use in the preclinical medical physics education of medical students. Our learning system simulates the actual clinical environment in a hospital or primary care unit. It uses a commercial medical information system for on-line storage and processing of clinical type data acquired during physics laboratory classes. Every student adopts two roles, the role of ‘patient’ and the role of ‘physician’. As a ‘physician’ the student operates the medical devices to clinically assess ‘patient’ colleagues and records all results in an electronic ‘patient’ record. We also introduced an innovative approach to the use of supportive education materials, based on the methods of adaptive e-learning. A survey of student feedback is included and statistically evaluated.The results from the student feedback confirm the positive response of the latter to this novel implementation of medical physics and informatics in preclinical education. This approach not only significantly improves learning of medical physics and informatics skills but has the added advantage that it facilitates students’ transition from preclinical to clinical subjects.     

Methods for acquisition of quantitative from confocal images of gene expression in situ

In this review we summarize original methods for the extraction quantitative information from the confocal images of gene expression patterns. These methods include image segmentation, extraction of quantitative numerical data on gene expression, removal of background signal and spatial registration. Finally it is possible to construct a spatiotemporal atlas of gene expression form individual images obtained at each developmental stage. Initially all methods were developed to extract quantitative numerical information form confocal images of segmentation gene expression in Drosophila melanogaster. Application of these methods to Drosophila images makes it possible to reveal new mechanisms of formation of segmentation gene expression domains, as well as to construct the quantitative atlas of segmentation gene expression. Most image processing procedures can be easily adapted to process a wide range of biological images.