Artificial intelligence and machine learning for medical imaging: A technology review

Artificial intelligence (AI) has recently become a very popular buzzword, as a consequence of disruptive technical advances and impressive experimental results, notably in the field of image analysis and processing. In medicine, specialties where images are central, like radiology, pathology or oncology, have seized the opportunity and considerable efforts in research and development have been deployed to transfer the potential of AI to clinical applications. With AI becoming a more mainstream tool for typical medical imaging analysis tasks, such as diagnosis, segmentation, or classification, the key for a safe and efficient use of clinical AI applications relies, in part, on informed practitioners. The aim of this review is to present the basic technological pillars of AI, together with the state-of-the-art machine learning methods and their application to medical imaging. In addition, we discuss the new trends and future research directions. This will help the reader to understand how AI methods are now becoming an ubiquitous tool in any medical image analysis workflow and pave the way for the clinical implementation of AI-based solutions.     

AI applications to medical images: From machine learning to deep learning

PurposeArtificial intelligence (AI) models are playing an increasing role in biomedical research and healthcare services. This review focuses on challenges points to be clarified about how to develop AI applications as clinical decision support systems in the real-world context.MethodsA narrative review has been performed including a critical assessment of articles published between 1989 and 2021 that guided challenging sections.ResultsWe first illustrate the architectural characteristics of machine learning (ML)/radiomics and deep learning (DL) approaches. For ML/radiomics, the phases of feature selection and of training, validation, and testing are described. DL models are presented as multi-layered artificial/convolutional neural networks, allowing us to directly process images. The data curation section includes technical steps such as image labelling, image annotation (with segmentation as a crucial step in radiomics), data harmonization (enabling compensation for differences in imaging protocols that typically generate noise in non-AI imaging studies) and federated learning. Thereafter, we dedicate specific sections to: sample size calculation, considering multiple testing in AI approaches; procedures for data augmentation to work with limited and unbalanced datasets; and the interpretability of AI models (the so-called black box issue). Pros and cons for choosing ML versus DL to implement AI applications to medical imaging are finally presented in a synoptic way.ConclusionsBiomedicine and healthcare systems are one of the most important fields for AI applications and medical imaging is probably the most suitable and promising domain. Clarification of specific challenging points facilitates the development of such systems and their translation to clinical practice.     

Artificial intelligence in radiotherapy

Artificial intelligence (AI) has already been implemented widely in the medical field in the recent years. This paper first reviews the background of AI and radiotherapy. Then it explores the basic concepts of different AI algorithms and machine learning methods, such as neural networks, that are available to us today and how they are being implemented in radiotherapy and diagnostic processes, such as medical imaging, treatment planning, patient simulation, quality assurance and radiation dose delivery. It also explores the ongoing research on AI methods that are to be implemented in radiotherapy in the future. The review shows very promising progress and future for AI to be widely used in various areas of radiotherapy. However, basing on various concerns such as availability and security of using big data, and further work on polishing and testing AI algorithms, it is found that we may not ready to use AI primarily in radiotherapy at the moment.     

Artificial intelligence: Deep learning in oncological radiomics and challenges of interpretability and data harmonization

Over the last decade there has been an extensive evolution in the Artificial Intelligence (AI) field. Modern radiation oncology is based on the exploitation of advanced computational methods aiming to personalization and high diagnostic and therapeutic precision. The quantity of the available imaging data and the increased developments of Machine Learning (ML), particularly Deep Learning (DL), triggered the research on uncovering “hidden” biomarkers and quantitative features from anatomical and functional medical images. Deep Neural Networks (DNN) have achieved outstanding performance and broad implementation in image processing tasks. Lately, DNNs have been considered for radiomics and their potentials for explainable AI (XAI) may help classification and prediction in clinical practice. However, most of them are using limited datasets and lack generalized applicability. In this study we review the basics of radiomics feature extraction, DNNs in image analysis, and major interpretability methods that help enable explainable AI. Furthermore, we discuss the crucial requirement of multicenter recruitment of large datasets, increasing the biomarkers variability, so as to establish the potential clinical value of radiomics and the development of robust explainable AI models.     

Artificial intelligence for compound pharmacokinetics prediction

Optimisation of compound pharmacokinetics (PK) is an integral part of drug discovery and development. Animal in vivo PK data as well as human and animal in vitro systems are routinely utilised to evaluate PK in humans. In recent years machine learning and artificial intelligence (AI) emerged as a major tool for modelling of in vivo animal and human PK, enabling prediction from chemical structure early in drug discovery, and therefore offering opportunities to guide the design and prioritisation of molecules based on relevant in vivo properties and, ultimately, predicting human PK at the point of design. This review presents recent advances in machine learning and AI models for in vivo animal and human PK for small-molecule compounds as well as some examples for antibody therapeutics.     

Artificial intelligence and the medical physics profession - A Swedish perspective

BackgroundThere is a continuous and dynamic discussion on artificial intelligence (AI) in present-day society. AI is expected to impact on healthcare processes and could contribute to a more sustainable use of resources allocated to healthcare in the future. The aim for this work was to establish a foundation for a Swedish perspective on the potential effect of AI on the medical physics profession.Materials and methodsWe designed a survey to gauge viewpoints regarding AI in the Swedish medical physics community. Based on the survey results and present-day situation in Sweden, a SWOT analysis was performed on the implications of AI for the medical physics profession.ResultsOut of 411 survey recipients, 163 responded (40%). The Swedish medical physicists with a professional license believed (90%) that AI would change the practice of medical physics but did not foresee (81%) that AI would pose a risk to their practice and career. The respondents were largely positive to the inclusion of AI in educational programmes. According to self-assessment, the respondents’ knowledge of and workplace preparedness for AI was generally low.ConclusionsFrom the survey and SWOT analysis we conclude that AI will change the medical physics profession and that there are opportunities for the profession associated with the adoption of AI in healthcare. To overcome the weakness of limited AI knowledge, potentially threatening the role of medical physicists, and build upon the strong position in Swedish healthcare, medical physics education and training should include learning objectives on AI.     

Artificial intelligence in the diagnosis of COVID-19: challenges and perspectives

Artificial intelligence (AI) is being used to aid in various aspects of the COVID-19 crisis, including epidemiology, molecular research and drug development, medical diagnosis and treatment, and socioeconomics. The association of AI and COVID-19 can accelerate to rapidly diagnose positive patients. To learn the dynamics of a pandemic with relevance to AI, we search the literature using the different academic databases (PubMed, PubMed Central, Scopus, Google Scholar) and preprint servers (bioRxiv, medRxiv, arXiv). In the present review, we address the clinical applications of machine learning and deep learning, including clinical characteristics, electronic medical records, medical images (CT, X-ray, ultrasound images, etc.) in the COVID-19 diagnosis. The current challenges and future perspectives provided in this review can be used to direct an ideal deployment of AI technology in a pandemic.     

Artificial intelligence in image reconstruction: The change is here

Innovations in CT have been impressive among imaging and medical technologies in both the hardware and software domain. The range and speed of CT scanning improved from the introduction of multidetector-row CT scanners with wide-array detectors and faster gantry rotation speeds. To tackle concerns over rising radiation doses from its increasing use and to improve image quality, CT reconstruction techniques evolved from filtered back projection to commercial release of iterative reconstruction techniques, and recently, of deep learning (DL)-based image reconstruction. These newer reconstruction techniques enable improved or retained image quality versus filtered back projection at lower radiation doses. DL can aid in image reconstruction with training data without total reliance on the physical model of the imaging process, unique artifacts of PCD-CT due to charge sharing, K-escape, fluorescence x-ray emission, and pulse pileups can be handled in the data-driven fashion. With sufficiently reconstructed images, a well-designed network can be trained to upgrade image quality over a practical/clinical threshold or define new/killer applications. Besides, the much smaller detector pixel for PCD-CT can lead to huge computational costs with traditional model-based iterative reconstruction methods whereas deep networks can be much faster with training and validation. In this review, we present techniques, applications, uses, and limitations of deep learning-based image reconstruction methods in CT.     

Novel artificial intelligence machine learning approaches to precisely predict survival and site-specific recurrence in cervical cancer: A multi-institutional study

BackgroundMachine learning (ML) has been gradually integrated into oncologic research but seldom applied to predict cervical cancer (CC), and no model has been reported to predict survival and site-specific recurrence simultaneously. Thus, we aimed to develop ML models to predict survival and site-specific recurrence in CC and to guide individual surveillance.MethodsWe retrospectively collected data on CC patients from 2006 to 2017 in four hospitals. The survival or recurrence predictive value of the variables was analyzed using multivariate Cox, principal component, and K-means clustering analyses. The predictive performances of eight ML models were compared with logistic or Cox models. A novel web-based predictive calculator was developed based on the ML algorithms.ResultsThis study included 5112 women for analysis (268 deaths, 343 recurrences): (1) For site-specific recurrence, larger tumor size was associated with local recurrence, while positive lymph nodes were associated with distant recurrence. (2) The ML models exhibited better prognostic predictive performance than traditional models. (3) The ML models were superior to traditional models when multiple variables were used. (4) A novel predictive web-based calculator was developed and externally validated to predict survival and site-specific recurrence.ConclusionML models might be a better analytic approach in CC prognostic prediction than traditional models as they can predict survival and site-specific recurrence simultaneously, especially when using multiple variables. Moreover, our novel web-based calculator may provide clinicians with useful information and help them make individual postoperative follow-up plans and further treatment strategies.     

Recent trends in stem cell-based therapies and applications of artificial intelligence in regenerative medicine

Stem cells are undifferentiated cells that can self-renew and differentiate into diverse types of mature and functional cells while maintaining their original identity. This profound potential of stem cells has been thoroughly investigated for its significance in regenerative medicine and has laid the foundation for cell-based therapies. Regenerative medicine is rapidly progressing in healthcare with the prospect of repair and restoration of specific organs or tissue injuries or chronic disease conditions where the body’s regenerative process is not sufficient to heal. In this review, the recent advances in stem cell-based therapies in regenerative medicine are discussed, emphasizing mesenchymal stem cell-based therapies as these cells have been extensively studied for clinical use. Recent applications of artificial intelligence algorithms in stem cell-based therapies, their limitation, and future prospects are highlighted.     

Machine learning in Magnetic Resonance Imaging: Image reconstruction

Magnetic Resonance Imaging (MRI) plays a vital role in diagnosis, management and monitoring of many diseases. However, it is an inherently slow imaging technique. Over the last 20 years, parallel imaging, temporal encoding and compressed sensing have enabled substantial speed-ups in the acquisition of MRI data, by accurately recovering missing lines of k-space data. However, clinical uptake of vastly accelerated acquisitions has been limited, in particular in compressed sensing, due to the time-consuming nature of the reconstructions and unnatural looking images. Following the success of machine learning in a wide range of imaging tasks, there has been a recent explosion in the use of machine learning in the field of MRI image reconstruction.A wide range of approaches have been proposed, which can be applied in k-space and/or image-space.Promising results have been demonstrated from a range of methods, enabling natural looking images and rapid computation.In this review article we summarize the current machine learning approaches used in MRI reconstruction, discuss their drawbacks, clinical applications, and current trends.     

Artificial Intelligence in magnetic Resonance guided Radiotherapy: Medical and physical considerations on state of art and future perspectives

Over the last years, technological innovation in Radiotherapy (RT) led to the introduction of Magnetic Resonance-guided RT (MRgRT) systems.Due to the higher soft tissue contrast compared to on-board CT-based systems, MRgRT is expected to significantly improve the treatment in many situations. MRgRT systems may extend the management of inter- and intra-fraction anatomical changes, offering the possibility of online adaptation of the dose distribution according to daily patient anatomy and to directly monitor tumor motion during treatment delivery by means of a continuous cine MR acquisition.Online adaptive treatments require a multidisciplinary and well-trained team, able to perform a series of operations in a safe, precise and fast manner while the patient is waiting on the treatment couch.Artificial Intelligence (AI) is expected to rapidly contribute to MRgRT, primarily by safely and efficiently automatising the various manual operations characterizing online adaptive treatments. Furthermore, AI is finding relevant applications in MRgRT in the fields of image segmentation, synthetic CT reconstruction, automatic (on-line) planning and the development of predictive models based on daily MRI.This review provides a comprehensive overview of the current AI integration in MRgRT from a medical physicist’s perspective. Medical physicists are expected to be major actors in solving new tasks and in taking new responsibilities: their traditional role of guardians of the new technology implementation will change with increasing emphasis on the managing of AI tools, processes and advanced systems for imaging and data analysis, gradually replacing many repetitive manual tasks.     

Requirements and reliability of AI in the medical context

The digital information age has been a catalyst in creating a renewed interest in Artificial Intelligence (AI) approaches, especially the subclass of computer algorithms that are popularly grouped into Machine Learning (ML). These methods have allowed one to go beyond limited human cognitive ability into understanding the complexity in the high dimensional data. Medical sciences have seen a steady use of these methods but have been slow in adoption to improve patient care. There are some significant impediments that have diluted this effort, which include availability of curated diverse data sets for model building, reliable human-level interpretation of these models, and reliable reproducibility of these methods for routine clinical use. Each of these aspects has several limiting conditions that need to be balanced out, considering the data/model building efforts, clinical implementation, integration cost to translational effort with minimal patient level harm, which may directly impact future clinical adoption. In this review paper, we will assess each aspect of the problem in the context of reliable use of the ML methods in oncology, as a representative study case, with the goal to safeguard utility and improve patient care in medicine in general.     

Application of Artificial Intelligence/Machine Vision & Learning for the Development of a Live Single-cell Phenotypic Biomarker Test to Predict Prostate Cancer Tumor Aggressiveness

To assess the usefulness and applications of machine vision (MV) and machine learning (ML) techniques that have been used to develop a single cell-based phenotypic (live and fixed biomarkers) platform that correlates with tumor biological aggressiveness and risk stratification, 100 fresh prostate samples were acquired, and areas of prostate cancer were determined by post-surgery pathology reports logged by an independent pathologist. The prostate samples were dissociated into single-cell suspensions in the presence of an extracellular matrix formulation. These samples were analyzed via live-cell microscopy. Dynamic and fixed phenotypic biomarkers per cell were quantified using objective MV software and ML algorithms. The predictive nature of the ML algorithms was developed in two stages. First, random forest (RF) algorithms were developed using 70% of the samples. The developed algorithms were then tested for their predictive performance using the blinded test dataset that contained 30% of the samples in the second stage. Based on the ROC (receiver operating characteristic) curve analysis, thresholds were set to maximize both sensitivity and specificity. We determined the sensitivity and specificity of the assay by comparing the algorithm-generated predictions with adverse pathologic features in the radical prostatectomy (RP) specimens. Using MV and ML algorithms, the biomarkers predictive of adverse pathology at RP were ranked and a prostate cancer patient risk stratification test was developed that distinguishes patients based on surgical adverse pathology features. The ability to identify and track large numbers of individual cells over the length of the microscopy experimental monitoring cycles, in an automated way, created a large biomarker dataset of primary biomarkers. This biomarker dataset was then interrogated with ML algorithms used to correlate with post-surgical adverse pathology findings. Algorithms were generated that predicted adverse pathology with >0.85 sensitivity and specificity and an AUC (area under the curve) of >0.85. Phenotypic biomarkers provide cellular and molecular details that are informative for predicting post-surgical adverse pathologies when considering tumor biopsy samples. Artificial intelligence ML-based approaches for cancer risk stratification are emerging as important and powerful tools to compliment current measures of risk stratification. These techniques have capabilities to address tumor heterogeneity and the molecular complexity of prostate cancer. Specifically, the phenotypic test is a novel example of leveraging biomarkers and advances in MV and ML for developing a powerful prognostic and risk-stratification tool for prostate cancer patients.     

Markerless 2D kinematic analysis of underwater running: A deep learning approach

Kinematic analysis is often performed with a camera system combined with reflective markers placed over bony landmarks. This method is restrictive (and often expensive), and limits the ability to perform analyses outside of the lab. In the present study, we used a markerless deep learning-based method to perform 2D kinematic analysis of deep water running, a task that poses several challenges to image processing methods. A single GoPro camera recorded sagittal plane lower limb motion. A deep neural network was trained using data from 17 individuals, and then used to predict the locations of markers that approximated joint centres. We found that 300–400 labelled images were sufficient to train the network to be able to position joint markers with an accuracy similar to that of a human labeler (mean difference < 3 pixels, around 1 cm). This level of accuracy is sufficient for many 2D applications, such as sports biomechanics, coaching/training, and rehabilitation. The method was sensitive enough to differentiate between closely-spaced running cadences (45–85 strides per minute in increments of 5). We also found high test–retest reliability of mean stride data, with between-session correlation coefficients of 0.90–0.97. Our approach represents a low-cost, adaptable solution for kinematic analysis, and could easily be modified for use in other movements and settings. Using additional cameras, this approach could also be used to perform 3D analyses. The method presented here may have broad applications in different fields, for example by enabling markerless motion analysis to be performed during rehabilitation, training or even competition environments.     

Heart rate variability as a marker of cardiovascular dysautonomia in post-COVID-19 syndrome using artificial intelligence

IntroductionCardiovascular dysautonomia comprising postural orthostatic tachycardia syndrome (POTS) and orthostatic hypotension (OH) is one of the presentations in COVID-19 recovered subjects. We aim to determine the prevalence of cardiovascular dysautonomia in post COVID-19 patients and to evaluate an Artificial Intelligence (AI) model to identify time domain heart rate variability (HRV) measures most suitable for short term ECG in these subjects.MethodsThis observational study enrolled 92 recently COVID-19 recovered subjects who underwent measurement of heart rate and blood pressure response to standing up from supine position and a 12-lead ECG recording for 60 s period during supine paced breathing. Using feature extraction, ECG features including those of HRV (RMSSD and SDNN) were obtained. An AI model was constructed with ShAP AI interpretability to determine time domain HRV features representing post COVID-19 recovered state. In addition, 120 healthy volunteers were enrolled as controls.ResultsCardiovascular dysautonomia was present in 15.21% (OH:13.04%; POTS:2.17%). Patients with OH had significantly lower HRV and higher inflammatory markers. HRV (RMSSD) was significantly lower in post COVID-19 patients compared to healthy controls (13.9 ± 11.8 ms vs 19.9 ± 19.5 ms; P = 0.01) with inverse correlation between HRV and inflammatory markers. Multiple perceptron was best performing AI model with HRV(RMSSD) being the top time domain HRV feature distinguishing between COVID-19 recovered patients and healthy controls.ConclusionPresent study showed that cardiovascular dysautonomia is common in COVID-19 recovered subjects with a significantly lower HRV compared to healthy controls. The AI model was able to distinguish between COVID-19 recovered patients and healthy controls.