Artificial intelligence and machine learning for medical imaging: A technology review

Artificial intelligence (AI) has recently become a very popular buzzword, as a consequence of disruptive technical advances and impressive experimental results, notably in the field of image analysis and processing. In medicine, specialties where images are central, like radiology, pathology or oncology, have seized the opportunity and considerable efforts in research and development have been deployed to transfer the potential of AI to clinical applications. With AI becoming a more mainstream tool for typical medical imaging analysis tasks, such as diagnosis, segmentation, or classification, the key for a safe and efficient use of clinical AI applications relies, in part, on informed practitioners. The aim of this review is to present the basic technological pillars of AI, together with the state-of-the-art machine learning methods and their application to medical imaging. In addition, we discuss the new trends and future research directions. This will help the reader to understand how AI methods are now becoming an ubiquitous tool in any medical image analysis workflow and pave the way for the clinical implementation of AI-based solutions.     

AI applications to medical images: From machine learning to deep learning

PurposeArtificial intelligence (AI) models are playing an increasing role in biomedical research and healthcare services. This review focuses on challenges points to be clarified about how to develop AI applications as clinical decision support systems in the real-world context.MethodsA narrative review has been performed including a critical assessment of articles published between 1989 and 2021 that guided challenging sections.ResultsWe first illustrate the architectural characteristics of machine learning (ML)/radiomics and deep learning (DL) approaches. For ML/radiomics, the phases of feature selection and of training, validation, and testing are described. DL models are presented as multi-layered artificial/convolutional neural networks, allowing us to directly process images. The data curation section includes technical steps such as image labelling, image annotation (with segmentation as a crucial step in radiomics), data harmonization (enabling compensation for differences in imaging protocols that typically generate noise in non-AI imaging studies) and federated learning. Thereafter, we dedicate specific sections to: sample size calculation, considering multiple testing in AI approaches; procedures for data augmentation to work with limited and unbalanced datasets; and the interpretability of AI models (the so-called black box issue). Pros and cons for choosing ML versus DL to implement AI applications to medical imaging are finally presented in a synoptic way.ConclusionsBiomedicine and healthcare systems are one of the most important fields for AI applications and medical imaging is probably the most suitable and promising domain. Clarification of specific challenging points facilitates the development of such systems and their translation to clinical practice.     

A red line not to cross: Evaluating the limitation and properness of gel image tuning procedures

Currently, results of gel electrophoresis are commonly documented in digital formats by image acquisition instruments. In this study, gel images tuned by a common image processing software package, Photoshop, were assessed to understand the transforming algorithms and their impacts on quantitative analysis. TotalLab 100, an electrophoresis gel image analysis software package, was applied for image quantitation and evaluation. The three most frequently used image tuning functions—adjustments of the brightness, contrast, and grayscale span (level) of images—were investigated using both data generated from a standard grayscale tablet and an actual electrophoresis gel image. The influences of these procedures were analyzed for the grayscale transformation between the input and output images. Although all three procedures differentially improved the visualization of the input image, adjusting the contrast of images disrupted the quantitative information because of its nonlinear transforming algorithm. Under certain conditions, adjusting the brightness or the level of images could preserve the quantitative information because of the linear transforming algorithms. It was found that when the minimum and maximum grayscales of a gel image were recognized, using a commercial software package to maximally stretch the level may significantly improve the quality of a gel image without jeopardizing quantitative analysis.     

Human vs. machine: Detecting wildlife in camera trap images

As the capacity to collect and store large amounts of data expands, identifying and evaluating strategies to efficiently convert raw data into meaningful information is increasingly necessary. Across disciplines, this data processing task has become a significant challenge, delaying progress and actionable insights. In ecology, the growing use of camera traps (i.e., remotely triggered cameras) to collect information on wildlife has led to an enormous volume of raw data (i.e., images) in need of review and annotation. To expedite camera trap image processing, many have turned to the field of artificial intelligence (AI) and use machine learning models to automate tasks such as detecting and classifying wildlife in images. To contribute understanding of the utility of AI tools for processing wildlife camera trap images, we evaluated the performance of a state-of-the-art computer vision model developed by Microsoft AI for Earth named MegaDetector using data from an ongoing camera trap study in Arctic Alaska, USA. Compared to image labels determined by manual human review, we found MegaDetector reliably determined the presence or absence of wildlife in images generated by motion detection camera settings (≥94.6% accuracy), however, performance was substantially poorer for images collected with time-lapse camera settings (≤61.6% accuracy). By examining time-lapse images where MegaDetector failed to detect wildlife, we gained practical insights into animal size and distance detection limits and discuss how those may impact the performance of MegaDetector in other systems. We anticipate our findings will stimulate critical thinking about the tradeoffs of using automated AI tools or manual human review to process camera trap images and help to inform effective implementation of study designs.     

Registration methods in radiotherapy

PurposeThe aim of this study is to present a short and comprehensive review of the methods of medical image registration, their conditions and applications in radiotherapy. A particular focus was placed on the methods of deformable image registration.MethodsTo structure and deepen the knowledge on medical image registration in radiotherapy, a medical literature analysis was made using the Google Scholar browser and the medical database of the PubMed library.ResultsChronological review of image registration methods in radiotherapy based on 34 selected articles. A particular attention was given to show: (i) potential regions of the application of different methods of registration, (ii) mathematical basis of the deformable methods and (iii) the methods of quality control for the registration process.ConclusionsThe primary aim of the medical image registration process is to connect the contents of images. What we want to achieve is a complementary or extended knowledge that can be used for more precise localisation of pathogenic lesions and continuous improvement of patient treatment. Therefore, the choice of imaging mode is dependent on the type of clinical study. It is impossible to visualise all anatomical details or functional changes using a single modality machine. Therefore, fusion of various modality images is of great clinical relevance. A natural problem in analysing the fusion of medical images is geographical errors related to displacement. The registered images are performed not at the same time and, very often, at different respiratory phases.     

Secure Similar Image Search and Copyright Protection over Encrypted Medical Image Databases

The recent advances in digital medical imaging and storage in cloud are bringing about more demands for efficient and secure image retrieval and management. Typically, medical images are very sensitive to changes where any modifications in its content may bring about an erroneous medical diagnosis. Therefore, securing medical images is a very essential process and the major task is, the medical image must maintain their sensitive contents at the time of reconstruction. The proposed methodology executes a secure image encryption and search of medical images proficiently over encrypted image database without leaking any sensitive data. It also ensures medical data integrity by introducing an efficient recovery mechanism on ROI of the image. The proposed scheme obtains recovery information about the image from the ROI of the medical data and embeds it in the RONI region using IWT transform which act as a reversible watermarking. If any alterations or tampers are caused to ROI at the third-party end, then it can be identified and recovered from the obtained recovery data. Besides, the model also executes a Copyright protection scheme to locate the authorized users, who illegally duplicate and distribute the retrieved image to unauthorized entities.     

Enhancing the impact of Artificial Intelligence in Medicine: A joint AIFM-INFN Italian initiative for a dedicated cloud-based computing infrastructure

Artificial Intelligence (AI) techniques have been implemented in the field of Medical Imaging for more than forty years. Medical Physicists, Clinicians and Computer Scientists have been collaborating since the beginning to realize software solutions to enhance the informative content of medical images, including AI-based support systems for image interpretation. Despite the recent massive progress in this field due to the current emphasis on Radiomics, Machine Learning and Deep Learning, there are still some barriers to overcome before these tools are fully integrated into the clinical workflows to finally enable a precision medicine approach to patients’ care. Nowadays, as Medical Imaging has entered the Big Data era, innovative solutions to efficiently deal with huge amounts of data and to exploit large and distributed computing resources are urgently needed. In the framework of a collaboration agreement between the Italian Association of Medical Physicists (AIFM) and the National Institute for Nuclear Physics (INFN), we propose a model of an intensive computing infrastructure, especially suited for training AI models, equipped with secure storage systems, compliant with data protection regulation, which will accelerate the development and extensive validation of AI-based solutions in the Medical Imaging field of research. This solution can be developed and made operational by Physicists and Computer Scientists working on complementary fields of research in Physics, such as High Energy Physics and Medical Physics, who have all the necessary skills to tailor the AI-technology to the needs of the Medical Imaging community and to shorten the pathway towards the clinical applicability of AI-based decision support systems.     

Procurement,commissioning and QA of AI based solutions: An MPE’s perspective on introducing AI in clinical practice

PurposeIn this study, we propose a framework to help the MPE take up a unique and important role at the introduction of AI solutions in clinical practice, and more in particular at procurement, acceptance, commissioning and QA.Material and methodsThe steps for the introduction of Medical Radiological Equipment in a hospital setting were extrapolated to AI tools. Literature review and in-house experience was added to prepare similar, yet dedicated test methods.ResultsProcurement starts from the clinical cases to be solved and is usually a complex process with many stakeholders and possibly many candidate AI solutions. Specific KPIs and metrics need to be defined. Acceptance testing follows, to verify the installation and test for critical exams. Commissioning should test the suitability of the AI tool for the intended use in the local institution. Results may be predicted from peer reviewed papers that treat representative populations. If not available, local data sets can be prepared to assess the KPIs, or ‘virtual clinical trials’ could be used to create large, simulated test data sets. Quality assurance must be performed periodically to verify if KPIs are stable, especially if the software is upscaled or upgraded, and as soon as self-learning AI tools would enter the medical practice.DiscussionMPEs are well placed to bridge between manufacturer and medical team and help from procurement up to reporting to the management board. More work is needed to establish consolidated test protocols.     

Artificial intelligence in the diagnosis of COVID-19: challenges and perspectives

Artificial intelligence (AI) is being used to aid in various aspects of the COVID-19 crisis, including epidemiology, molecular research and drug development, medical diagnosis and treatment, and socioeconomics. The association of AI and COVID-19 can accelerate to rapidly diagnose positive patients. To learn the dynamics of a pandemic with relevance to AI, we search the literature using the different academic databases (PubMed, PubMed Central, Scopus, Google Scholar) and preprint servers (bioRxiv, medRxiv, arXiv). In the present review, we address the clinical applications of machine learning and deep learning, including clinical characteristics, electronic medical records, medical images (CT, X-ray, ultrasound images, etc.) in the COVID-19 diagnosis. The current challenges and future perspectives provided in this review can be used to direct an ideal deployment of AI technology in a pandemic.     

Reconstruction of 3-Dimensional Histology Volume and its Application to Study Mouse Mammary Glands

Histology volume reconstruction facilitates the study of 3D shape and volume change of an organ at the level of macrostructures made up of cells. It can also be used to investigate and validate novel techniques and algorithms in volumetric medical imaging and therapies. Creating 3D high-resolution atlases of different organs^1,2,3 is another application of histology volume reconstruction. This provides a resource for investigating tissue structures and the spatial relationship between various cellular features. We present an image registration approach for histology volume reconstruction, which uses a set of optical blockface images. The reconstructed histology volume represents a reliable shape of the processed specimen with no propagated post-processing registration error. The Hematoxylin and Eosin (H&E) stained sections of two mouse mammary glands were registered to their corresponding blockface images using boundary points extracted from the edges of the specimen in histology and blockface images. The accuracy of the registration was visually evaluated. The alignment of the macrostructures of the mammary glands was also visually assessed at high resolution.This study delineates the different steps of this image registration pipeline, ranging from excision of the mammary gland through to 3D histology volume reconstruction. While 2D histology images reveal the structural differences between pairs of sections, 3D histology volume provides the ability to visualize the differences in shape and volume of the mammary glands.     

Rapid pedobarographic image registration based on contour curvature and optimization

Image registration, the process of optimally aligning homologous structures in multiple images, has recently been demonstrated to support automated pixel-level analysis of pedobarographic images and, subsequently, to extract unique and biomechanically relevant information from plantar pressure data. Recent registration methods have focused on robustness, with slow but globally powerful algorithms. In this paper, we present an alternative registration approach that affords both speed and accuracy, with the goal of making pedobarographic image registration more practical for near-real-time laboratory and clinical applications. The current algorithm first extracts centroid-based curvature trajectories from pressure image contours, and then optimally matches these curvature profiles using optimization based on dynamic programming. Special cases of disconnected images (that occur in high-arched subjects, for example) are dealt with by introducing an artificial spatially linear bridge between adjacent image clusters. Two registration algorithms were developed: a ‘geometric’ algorithm, which exclusively matched geometry, and a ‘hybrid’ algorithm, which performed subsequent pseudo-optimization. After testing the two algorithms on 30 control image pairs considered in a previous study, we found that, when compared with previously published results, the hybrid algorithm improved overlap ratio (p=0.010), but both current algorithms had slightly higher mean-squared error, assumedly because they did not consider pixel intensity. Nonetheless, both algorithms greatly improved the computational efficiency (25±8 and 53±9 ms per image pair for geometric and hybrid registrations, respectively). These results imply that registration-based pixel-level pressure image analyses can, eventually, be implemented for practical clinical purposes.     

Radiomics in radiooncology – Challenging the medical physicist

PurposeNoticing the fast growing translation of artificial intelligence (AI) technologies to medical image analysis this paper emphasizes the future role of the medical physicist in this evolving field. Specific challenges are addressed when implementing big data concepts with high-throughput image data processing like radiomics and machine learning in a radiooncology environment to support clinical decisions.MethodsBased on the experience of our interdisciplinary radiomics working group, techniques for processing minable data, extracting radiomics features and associating this information with clinical, physical and biological data for the development of prediction models are described. A special emphasis was placed on the potential clinical significance of such an approach.ResultsClinical studies demonstrate the role of radiomics analysis as an additional independent source of information with the potential to influence the radiooncology practice, i.e. to predict patient prognosis, treatment response and underlying genetic changes. Extending the radiomics approach to integrate imaging, clinical, genetic and dosimetric data (‘panomics’) challenges the medical physicist as member of the radiooncology team.ConclusionsThe new field of big data processing in radiooncology offers opportunities to support clinical decisions, to improve predicting treatment outcome and to stimulate fundamental research on radiation response both of tumor and normal tissue. The integration of physical data (e.g. treatment planning, dosimetric, image guidance data) demands an involvement of the medical physicist in the radiomics approach of radiooncology. To cope with this challenge national and international organizations for medical physics should organize more training opportunities in artificial intelligence technologies in radiooncology.     

Artificial intelligence: Deep learning in oncological radiomics and challenges of interpretability and data harmonization

Over the last decade there has been an extensive evolution in the Artificial Intelligence (AI) field. Modern radiation oncology is based on the exploitation of advanced computational methods aiming to personalization and high diagnostic and therapeutic precision. The quantity of the available imaging data and the increased developments of Machine Learning (ML), particularly Deep Learning (DL), triggered the research on uncovering “hidden” biomarkers and quantitative features from anatomical and functional medical images. Deep Neural Networks (DNN) have achieved outstanding performance and broad implementation in image processing tasks. Lately, DNNs have been considered for radiomics and their potentials for explainable AI (XAI) may help classification and prediction in clinical practice. However, most of them are using limited datasets and lack generalized applicability. In this study we review the basics of radiomics feature extraction, DNNs in image analysis, and major interpretability methods that help enable explainable AI. Furthermore, we discuss the crucial requirement of multicenter recruitment of large datasets, increasing the biomarkers variability, so as to establish the potential clinical value of radiomics and the development of robust explainable AI models.     

中国的野生动物红外相机监测需要统一的标准

采用及时、可靠的方法对物种开展有效监测是生物多样性保护的基础。红外相机技术可以获得兽类物种的影像、元数据和分布信息, 是监测生物多样性的有效途径。这项技术在野外便于部署, 规程易于标准化, 可提供野生动物凭证标本(影像)以及物种拍摄位置、拍摄日期与时间、拍摄细节(相机型号等)等附属信息。这些特性使得我们可以积累数以百万计的影像资料和野生动物监测数据。在中国, 红外相机技术已得到广泛应用, 众多机构正在使用红外相机采集并存储野生动物影像以及相关联的元数据。目前, 亟需对红外相机元数据结构进行标准化, 以促进不同机构之间以及与外部保护团体之间的数据共享。迄今全球已建立有数个国际数据共享平台, 例如Wildlife Insights, 但他们离不开与中国的合作, 以有效追踪全球可持续发展的进程。达成这样的合作需要3个基础: 共同的数据标准、数据共享协议和数据禁用政策。我们倡议, 中国保护领域的政府主管部门、机构团体一起合作, 共同制定在国内单位之间以及与国际机构之间共享监测数据的政策、机制与途径。     

医学图像融合技术的研究

利用图像融合技术,将不同模态的医学图像有机地结合在一起,可以充分利用各种医学图像的优点,为临床诊断和治疗提供帮助。本文主要介绍了医学图像融合技术的基本概念、发展情况、常用方法及面临的困难等,并对医学图像的研究前景作了预测。     

AMB-Wnet: Embedding attention model in multi-bridge Wnet for exploring the mechanics of disease

In recent years, progressive application of convolutional neural networks in image processing has successfully filtered into medical diagnosis. As a prerequisite for images detection and classification, object segmentation in medical images has attracted a great deal of attention. This study is based on the fact that most of the analysis of pathological diagnoses requires nuclei detection as the starting phase for obtaining an insight into the underlying biological process and further diagnosis. In this paper, we introduce an embedded attention model in multi-bridge Wnet (AMB-Wnet) to achieve suppression of irrelevant background areas and obtain good features for learning image semantics and modality to automatically segment nuclei, inspired by the 2018 Data Science Bowl. The proposed architecture, consisting of the redesigned down sample group, up-sample group, and middle block (a new multiple-scale convolutional layers block), is designed to extract different level features. In addition, a connection group is proposed instead of skip-connection to transfer semantic information among different levels. In addition, the attention model is well embedded in the connection group, and the performance of the model is improved without increasing the amount of calculation. To validate the model's performance, we evaluated it using the BBBC038V1 data sets for nuclei segmentation. Our proposed model achieves 85.83% F1-score, 97.81% accuracy, 86.12% recall, and 83.52% intersection over union. The proposed AMB-Wnet exhibits superior results compared to the original U-Net, MultiResUNet, and recent Attention U-Net architecture.     

Automated quantification of nuclear immunohistochemical markers with different complexity

Manual quantification of immunohistochemically stained nuclear markers is still laborious and subjective and the use of computerized systems for digital image analysis have not yet resolved the problems of nuclear clustering. In this study, we designed a new automatic procedure for quantifying various immunohistochemical nuclear markers with variable clustering complexity. This procedure consisted of two combined macros. The first, developed with a commercial software, enabled the analysis of the digital images using color and morphological segmentation including a masking process. All information extracted with this first macro was automatically exported to an Excel datasheet, where a second macro composed of four different algorithms analyzed all the information and calculated the definitive number of positive nuclei for each image. One hundred and eighteen images with different levels of clustering complexity was analyzed and compared with the manual quantification obtained by a trained observer. Statistical analysis indicated a great reliability (intra-class correlation coefficient > 0.950) and no significant differences between the two methods. Bland–Altman plot and Kaplan–Meier curves indicated that the results of both methods were concordant around 90% of analyzed images. In conclusion, this new automated procedure is an objective, faster and reproducible method that has an excellent level of accuracy, even with digital images with a high complexity.     

Artificial intelligence in radiotherapy

Artificial intelligence (AI) has already been implemented widely in the medical field in the recent years. This paper first reviews the background of AI and radiotherapy. Then it explores the basic concepts of different AI algorithms and machine learning methods, such as neural networks, that are available to us today and how they are being implemented in radiotherapy and diagnostic processes, such as medical imaging, treatment planning, patient simulation, quality assurance and radiation dose delivery. It also explores the ongoing research on AI methods that are to be implemented in radiotherapy in the future. The review shows very promising progress and future for AI to be widely used in various areas of radiotherapy. However, basing on various concerns such as availability and security of using big data, and further work on polishing and testing AI algorithms, it is found that we may not ready to use AI primarily in radiotherapy at the moment.     

An overview of image-processing methods for Affymetrix GeneChips

We present an overview of image-processing methods for Affymetrix GeneChips. All GeneChips are affected to some extent by spatially coherent defects and image processing has a number of potential impacts on the downstream analysis of GeneChip data. Fortunately, there are now a number of robust and accurate algorithms, which identify the most disabling defects. One group of algorithms concentrate on the transformation from the original hybridisation DAT image to the representative CEL file. Another set uses dedicated pattern recognition routines to detect different types of hybridisation defect in replicates. A third type exploits the information provided by public repositories of GeneChips (such as GEO). The use of these algorithms improves the sensitivity of GeneChips, and should be a prerequisite for studies in which there are only few probes per relevant biological signal, such as exon arrays and SNP chips.