AI applications to medical images: From machine learning to deep learning

PurposeArtificial intelligence (AI) models are playing an increasing role in biomedical research and healthcare services. This review focuses on challenges points to be clarified about how to develop AI applications as clinical decision support systems in the real-world context.MethodsA narrative review has been performed including a critical assessment of articles published between 1989 and 2021 that guided challenging sections.ResultsWe first illustrate the architectural characteristics of machine learning (ML)/radiomics and deep learning (DL) approaches. For ML/radiomics, the phases of feature selection and of training, validation, and testing are described. DL models are presented as multi-layered artificial/convolutional neural networks, allowing us to directly process images. The data curation section includes technical steps such as image labelling, image annotation (with segmentation as a crucial step in radiomics), data harmonization (enabling compensation for differences in imaging protocols that typically generate noise in non-AI imaging studies) and federated learning. Thereafter, we dedicate specific sections to: sample size calculation, considering multiple testing in AI approaches; procedures for data augmentation to work with limited and unbalanced datasets; and the interpretability of AI models (the so-called black box issue). Pros and cons for choosing ML versus DL to implement AI applications to medical imaging are finally presented in a synoptic way.ConclusionsBiomedicine and healthcare systems are one of the most important fields for AI applications and medical imaging is probably the most suitable and promising domain. Clarification of specific challenging points facilitates the development of such systems and their translation to clinical practice.     

Artificial intelligence: Deep learning in oncological radiomics and challenges of interpretability and data harmonization

Over the last decade there has been an extensive evolution in the Artificial Intelligence (AI) field. Modern radiation oncology is based on the exploitation of advanced computational methods aiming to personalization and high diagnostic and therapeutic precision. The quantity of the available imaging data and the increased developments of Machine Learning (ML), particularly Deep Learning (DL), triggered the research on uncovering “hidden” biomarkers and quantitative features from anatomical and functional medical images. Deep Neural Networks (DNN) have achieved outstanding performance and broad implementation in image processing tasks. Lately, DNNs have been considered for radiomics and their potentials for explainable AI (XAI) may help classification and prediction in clinical practice. However, most of them are using limited datasets and lack generalized applicability. In this study we review the basics of radiomics feature extraction, DNNs in image analysis, and major interpretability methods that help enable explainable AI. Furthermore, we discuss the crucial requirement of multicenter recruitment of large datasets, increasing the biomarkers variability, so as to establish the potential clinical value of radiomics and the development of robust explainable AI models.     

基于机器学习的医学影像分析在药物研发和精准医疗方面的应用

MRI,PET,和CT等医学影像在新药研发和精准医疗中起着越来越重要的作用。影像技术可以被用来诊断疾病,评估药效,选择适应患者,或者确定用药剂量。 随着人工智能技术的发展,特别是机器学习以及深度学习技术在医学影像中的应用,使得我们可以用更短的时间,更少的放射剂量获取更高质量的影像。这些技术还可以帮助放射科医生缩短读片时间,提高诊断准确率。除此之外,机器学习技术还可以提高量化分析的可行性和精度,帮助建立影像与基因以及疾病的临床表现之间的关系。首先根据不同形态的医学影像,简单介绍他们在药物研发和精准医疗中的应用。并对机器学习在医学影像中的功能作一概括总结。最后讨论这个领域的挑战和机遇。     

A Review of the Deep Learning Methods for Medical Images Super Resolution Problems

Super resolution problems are widely discussed in medical imaging. Spatial resolution of medical images are not sufficient due to the constraints such as image acquisition time, low irradiation dose or hardware limits. To address these problems, different super resolution methods have been proposed, such as optimization or learning-based approaches. Recently, deep learning methods become a thriving technology and are developing at an exponential speed. We think it is necessary to write a review to present the current situation of deep learning in medical imaging super resolution. In this paper, we first briefly introduce deep learning methods, then present a number of important deep learning approaches to solve super resolution problems, different architectures as well as up-sampling operations will be introduced. Afterwards, we focus on the applications of deep learning methods in medical imaging super resolution problems, the challenges to overcome will be presented as well.     

Data preparation for artificial intelligence in medical imaging: A comprehensive guide to open-access platforms and tools

The vast amount of data produced by today’s medical imaging systems has led medical professionals to turn to novel technologies in order to efficiently handle their data and exploit the rich information present in them. In this context, artificial intelligence (AI) is emerging as one of the most prominent solutions, promising to revolutionise every day clinical practice and medical research. The pillar supporting the development of reliable and robust AI algorithms is the appropriate preparation of the medical images to be used by the AI-driven solutions. Here, we provide a comprehensive guide for the necessary steps to prepare medical images prior to developing or applying AI algorithms. The main steps involved in a typical medical image preparation pipeline include: (i) image acquisition at clinical sites, (ii) image de-identification to remove personal information and protect patient privacy, (iii) data curation to control for image and associated information quality, (iv) image storage, and (v) image annotation. There exists a plethora of open access tools to perform each of the aforementioned tasks and are hereby reviewed. Furthermore, we detail medical image repositories covering different organs and diseases. Such repositories are constantly increasing and enriched with the advent of big data. Lastly, we offer directions for future work in this rapidly evolving field.     

Artificial intelligence in radiotherapy

Artificial intelligence (AI) has already been implemented widely in the medical field in the recent years. This paper first reviews the background of AI and radiotherapy. Then it explores the basic concepts of different AI algorithms and machine learning methods, such as neural networks, that are available to us today and how they are being implemented in radiotherapy and diagnostic processes, such as medical imaging, treatment planning, patient simulation, quality assurance and radiation dose delivery. It also explores the ongoing research on AI methods that are to be implemented in radiotherapy in the future. The review shows very promising progress and future for AI to be widely used in various areas of radiotherapy. However, basing on various concerns such as availability and security of using big data, and further work on polishing and testing AI algorithms, it is found that we may not ready to use AI primarily in radiotherapy at the moment.     

Radiomics in radiooncology – Challenging the medical physicist

PurposeNoticing the fast growing translation of artificial intelligence (AI) technologies to medical image analysis this paper emphasizes the future role of the medical physicist in this evolving field. Specific challenges are addressed when implementing big data concepts with high-throughput image data processing like radiomics and machine learning in a radiooncology environment to support clinical decisions.MethodsBased on the experience of our interdisciplinary radiomics working group, techniques for processing minable data, extracting radiomics features and associating this information with clinical, physical and biological data for the development of prediction models are described. A special emphasis was placed on the potential clinical significance of such an approach.ResultsClinical studies demonstrate the role of radiomics analysis as an additional independent source of information with the potential to influence the radiooncology practice, i.e. to predict patient prognosis, treatment response and underlying genetic changes. Extending the radiomics approach to integrate imaging, clinical, genetic and dosimetric data (‘panomics’) challenges the medical physicist as member of the radiooncology team.ConclusionsThe new field of big data processing in radiooncology offers opportunities to support clinical decisions, to improve predicting treatment outcome and to stimulate fundamental research on radiation response both of tumor and normal tissue. The integration of physical data (e.g. treatment planning, dosimetric, image guidance data) demands an involvement of the medical physicist in the radiomics approach of radiooncology. To cope with this challenge national and international organizations for medical physics should organize more training opportunities in artificial intelligence technologies in radiooncology.     

Artificial intelligence in the diagnosis of COVID-19: challenges and perspectives

Artificial intelligence (AI) is being used to aid in various aspects of the COVID-19 crisis, including epidemiology, molecular research and drug development, medical diagnosis and treatment, and socioeconomics. The association of AI and COVID-19 can accelerate to rapidly diagnose positive patients. To learn the dynamics of a pandemic with relevance to AI, we search the literature using the different academic databases (PubMed, PubMed Central, Scopus, Google Scholar) and preprint servers (bioRxiv, medRxiv, arXiv). In the present review, we address the clinical applications of machine learning and deep learning, including clinical characteristics, electronic medical records, medical images (CT, X-ray, ultrasound images, etc.) in the COVID-19 diagnosis. The current challenges and future perspectives provided in this review can be used to direct an ideal deployment of AI technology in a pandemic.     

Artificial intelligence in image reconstruction: The change is here

Innovations in CT have been impressive among imaging and medical technologies in both the hardware and software domain. The range and speed of CT scanning improved from the introduction of multidetector-row CT scanners with wide-array detectors and faster gantry rotation speeds. To tackle concerns over rising radiation doses from its increasing use and to improve image quality, CT reconstruction techniques evolved from filtered back projection to commercial release of iterative reconstruction techniques, and recently, of deep learning (DL)-based image reconstruction. These newer reconstruction techniques enable improved or retained image quality versus filtered back projection at lower radiation doses. DL can aid in image reconstruction with training data without total reliance on the physical model of the imaging process, unique artifacts of PCD-CT due to charge sharing, K-escape, fluorescence x-ray emission, and pulse pileups can be handled in the data-driven fashion. With sufficiently reconstructed images, a well-designed network can be trained to upgrade image quality over a practical/clinical threshold or define new/killer applications. Besides, the much smaller detector pixel for PCD-CT can lead to huge computational costs with traditional model-based iterative reconstruction methods whereas deep networks can be much faster with training and validation. In this review, we present techniques, applications, uses, and limitations of deep learning-based image reconstruction methods in CT.     

Artificial intelligence applications in medical imaging: A review of the medical physics research in Italy

PurposeTo perform a systematic review on the research on the application of artificial intelligence (AI) to imaging published in Italy and identify its fields of application, methods and results.Materials and MethodsA Pubmed search was conducted using terms Artificial Intelligence, Machine Learning, Deep learning, imaging, and Italy as affiliation, excluding reviews and papers outside time interval 2015–2020. In a second phase, participants of the working group AI4MP on Artificial Intelligence of the Italian Association of Physics in Medicine (AIFM) searched for papers on AI in imaging.ResultsThe Pubmed search produced 794 results. 168 studies were selected, of which 122 were from Pubmed search and 46 from the working group. The most used imaging modality was MRI (44%) followed by CT(12%) ad radiography/mammography (11%). The most common clinical indication were neurological diseases (29%) and diagnosis of cancer (25%). Classification was the most common task for AI (57%) followed by segmentation (16%). 65% of studies used machine learning and 35% used deep learning. We observed a rapid increase of research in Italy on artificial intelligence in the last 5 years, peaking at 155% from 2018 to 2019.ConclusionsWe are witnessing an unprecedented interest in AI applied to imaging in Italy, in a diversity of fields and imaging techniques. Further initiatives are needed to build common frameworks and databases, collaborations among different types of institutions, and guidelines for research on AI.     

From cancer big data to treatment: Artificial intelligence in cancer research

In recent years, developing the idea of “cancer big data” has emerged as a result of the significant expansion of various fields such as clinical research, genomics, proteomics and public health records. Advances in omics technologies are making a significant contribution to cancer big data in biomedicine and disease diagnosis. The increasingly availability of extensive cancer big data has set the stage for the development of multimodal artificial intelligence (AI) frameworks. These frameworks aim to analyze high-dimensional multi-omics data, extracting meaningful information that is challenging to obtain manually. Although interpretability and data quality remain critical challenges, these methods hold great promise for advancing our understanding of cancer biology and improving patient care and clinical outcomes. Here, we provide an overview of cancer big data and explore the applications of both traditional machine learning and deep learning approaches in cancer genomic and proteomic studies. We briefly discuss the challenges and potential of AI techniques in the integrated analysis of omics data, as well as the future direction of personalized treatment options in cancer.     

Extendable and explainable deep learning for pan-cancer radiogenomics research

Radiogenomics is a field where medical images and genomic profiles are jointly analyzed to answer critical clinical questions. Specifically, people want to identify non-invasive imaging biomarkers that are associated with both genomic features and clinical outcomes. Deep learning is an advanced computer science technique that has been applied in many fields, including medical image and genomic data analysis. This review summarizes the current state of deep learning in pan-cancer radiogenomic research, discusses its limitations, and indicates the potential future directions. Traditional machine learning in radiomics, genomics, and radiogenomics have also been briefly discussed. We also summarize the main pan-cancer radiogenomic research resources. Two characteristics of deep learning are emphasized when discussing its application to pan-cancer radiogenomics, which are extendibility and explainability.     

Radiomics software for breast imaging optimization and simulation studies

Background and ObjectiveThe development, control and optimisation of new x-ray breast imaging modalities could benefit from a quantitative assessment of the resulting image textures. The aim of this work was to develop a software tool for routine radiomics applications in breast imaging, which will also be available upon request.MethodsThe tool (developed in MATLAB) allows image reading, selection of Regions of Interest (ROI), analysis and comparison. Requirements towards the tool also included convenient handling of common medical and simulated images, building and providing a library of commonly applied algorithms and a friendly graphical user interface. Initial set of features and analyses have been selected after a literature search. Being open, the tool can be extended, if necessary.ResultsThe tool allows semi-automatic extracting of ROIs, calculating and processing a total of 23 different metrics or features in 2D images and/or in 3D image volumes. Computations of the features were verified against computations with other software packages performed with test images. Two case studies illustrate the applicability of the tool – (i) features on a series of 2D ‘left’ and ‘right’ CC mammograms acquired on a Siemens Inspiration system were computed and compared, and (ii) evaluation of the suitability of newly proposed and developed breast phantoms for x-ray-based imaging based on reference values from clinical mammography images. Obtained results could steer the further development of the physical breast phantoms.ConclusionsA new image analysis toolbox was realized and can now be used in a multitude of radiomics applications, on both clinical and test images.     

Expanding the medical physicist curricular and professional programme to include Artificial Intelligence

PurposeTo provide a guideline curriculum related to Artificial Intelligence (AI), for the education and training of European Medical Physicists (MPs).Materials and methodsThe proposed curriculum consists of two levels: Basic (introducing MPs to the pillars of knowledge, development and applications of AI, in the context of medical imaging and radiation therapy) and Advanced. Both are common to the subspecialties (diagnostic and interventional radiology, nuclear medicine, and radiation oncology). The learning outcomes of the training are presented as knowledge, skills and competences (KSC approach).ResultsFor the Basic section, KSCs were stratified in four subsections: (1) Medical imaging analysis and AI Basics; (2) Implementation of AI applications in clinical practice; (3) Big data and enterprise imaging, and (4) Quality, Regulatory and Ethical Issues of AI processes. For the Advanced section instead, a common block was proposed to be further elaborated by each subspecialty core curriculum. The learning outcomes were also translated into a syllabus of a more traditional format, including practical applications.ConclusionsThis AI curriculum is the first attempt to create a guideline expanding the current educational framework for Medical Physicists in Europe. It should be considered as a document to top the sub-specialties’ curriculums and adapted by national training and regulatory bodies. The proposed educational program can be implemented via the European School of Medical Physics Expert (ESMPE) course modules and – to some extent – also by the national competent EFOMP organizations, to reach widely the medical physicist community in Europe.     

Enhancing the impact of Artificial Intelligence in Medicine: A joint AIFM-INFN Italian initiative for a dedicated cloud-based computing infrastructure

Artificial Intelligence (AI) techniques have been implemented in the field of Medical Imaging for more than forty years. Medical Physicists, Clinicians and Computer Scientists have been collaborating since the beginning to realize software solutions to enhance the informative content of medical images, including AI-based support systems for image interpretation. Despite the recent massive progress in this field due to the current emphasis on Radiomics, Machine Learning and Deep Learning, there are still some barriers to overcome before these tools are fully integrated into the clinical workflows to finally enable a precision medicine approach to patients’ care. Nowadays, as Medical Imaging has entered the Big Data era, innovative solutions to efficiently deal with huge amounts of data and to exploit large and distributed computing resources are urgently needed. In the framework of a collaboration agreement between the Italian Association of Medical Physicists (AIFM) and the National Institute for Nuclear Physics (INFN), we propose a model of an intensive computing infrastructure, especially suited for training AI models, equipped with secure storage systems, compliant with data protection regulation, which will accelerate the development and extensive validation of AI-based solutions in the Medical Imaging field of research. This solution can be developed and made operational by Physicists and Computer Scientists working on complementary fields of research in Physics, such as High Energy Physics and Medical Physics, who have all the necessary skills to tailor the AI-technology to the needs of the Medical Imaging community and to shorten the pathway towards the clinical applicability of AI-based decision support systems.     

Optical and digital microscopic imaging techniques and applications in pathology

The conventional optical microscope has been the primary tool in assisting pathological examinations. The modern digital pathology combines the power of microscopy, electronic detection, and computerized analysis. It enables cellular-, molecular-, and genetic-imaging at high efficiency and accuracy to facilitate clinical screening and diagnosis. This paper first reviews the fundamental concepts of microscopic imaging and introduces the technical features and associated clinical applications of optical microscopes, electron microscopes, scanning tunnel microscopes, and fluorescence microscopes. The interface of microscopy with digital image acquisition methods is discussed. The recent developments and future perspectives of contemporary microscopic imaging techniques such as three-dimensional and in vivo imaging are analyzed for their clinical potentials.     

LV-GAN: A deep learning approach for limited-view optoacoustic imaging based on hybrid datasets

The optoacoustic imaging (OAI) methods are rapidly evolving for resolving optical contrast in medical imaging applications. In practice, measurement strategies are commonly implemented under limited-view conditions due to oversized image objectives or system design limitations. Data acquired by limited-view detection may impart artifacts and distortions in reconstructed optoacoustic (OA) images. We propose a hybrid data-driven deep learning approach based on generative adversarial network (GAN), termed as LV-GAN, to efficiently recover high quality images from limited-view OA images. Trained on both simulation and experiment data, LV-GAN is found capable of achieving high recovery accuracy even under limited detection angles less than 60^°. The feasibility of LV-GAN for artifact removal in biological applications was validated by ex vivo experiments based on two different OAI systems, suggesting high potential of a ubiquitous use of LV-GAN to optimize image quality or system design for different scanners and application scenarios.