Medical image registration: a review

This paper presents a review of automated image registration methodologies that have been used in the medical field. The aim of this paper is to be an introduction to the field, provide knowledge on the work that has been developed and to be a suitable reference for those who are looking for registration methods for a specific application. The registration methodologies under review are classified into intensity or feature based. The main steps of these methodologies, the common geometric transformations, the similarity measures and accuracy assessment techniques are introduced and described.     

Application of Dual Modality Contrast Agent Combined with Multi-Scale Representation in Ultrasound-Magnetic Resonance Imaging Registration Scheme

To achieve the image registration/fusion and perfect the quality of the integration, with dual modality contrast agent (DMCA), a novel multi-scale representation registration method between ultrasound imaging (US) and magnetic resonance imaging (MRI) is presented in the paper, and how DMCA influence on registration accuracy is chiefly discussed. Owing to US’s intense speckle noise, it is a tremendous challenge to register US with any other modality images. How to improve the algorithms for US processing has become the bottleneck, and in the short term it is difficult to have a breakthrough. In that case, DMCA is employed in both US and MRI to enhance the region of interest. Then, because multi-scale representation is a strategy that attempts to diminish or eliminate several possible local minima and lead to convex optimization problems to be solved quickly and more efficiently, a multi-scale representation Gaussian pyramid based affine registration (MRGP-AR) scheme is constructed to complete the US-MRI registration process. In view of the above-mentioned method, the comparison tests indicate that US-MRI registration/fusion may be a remarkable method for gaining high-quality registration image. The experiments also show that it is feasible that novel nano-materials combined with excellent algorithm are used to solve some hard tasks in medical image processing field.     

Noninvasive high resolution mechanical strain maps of the spine intervertebral disc using nonrigid registration of magnetic resonance images

High resolution strain measurements are of particular interest in load bearing tissues such as the intervertebral disc (IVD), permitting characterization of biomechanical conditions which could lead to injury and degenerative outcomes. Magnetic resonance (MR) imaging produces excellent image contrast in cartilaginous tissues, allowing for image-based strain determination. Nonrigid registration (NRR) of MR images has previously demonstrated sub-voxel registration accuracy although its accuracy and precision in determining strain has not been evaluated. Accuracy and precision of NRR-derived strain measurements were evaluated using computer generated deformations applied to both computer generated images and MR images. Two different measures of registration similarity--the cost function which drives the registration algorithm--were compared: Mutual Information (MI) and Least Squares (LS). Strain error was evaluated with respect to signal-to-noise ratio (SNR), contrast-to-noise ratio (CNR), and strain heterogeneity. Additionally, the creep strain response from an in vitro loaded porcine IVD is shown and comparisons between similarity measures are presented. MI showed a decrease in strain precision with increasing CNR and decreasing SNR while LS was insensitive to both. Both similarity measures showed a decrease in strain precision with increasing strain heterogeneity. When computer generated heterogeneous strains were applied to MR images of the IVD, LS showed substantially lower strain error in comparison to MI. Results suggest that LS-driven NRR provides a more accurate image-based method for mapping large and heterogeneous strain fields and this method can be applied to studies of the IVD and, potentially, other soft tissues which present sufficient image texture.     

Anatomical atlas in the context of head and neck radiotherapy and its use to automatic segmentation

In this paper we present a methodology to form an anatomical atlas based on the analysis of dense deformation fields recovered by the Morphons non-rigid registration algorithm. The methodology is based on measuring the bending energy required to register the whole database to a reference, and the atlas is the one image in the database which yields the smallest bending energy when taken as reference. The suitability of our atlas is demonstrated in the context of head and neck radiotherapy through its application to a database with thirty-one computed tomography (CT) images of the head and neck region. In head and neck radiotherapy, CT is the most frequently used modality for the segmentation of organs at risk and clinical target volumes. One challenge brought by the use of CT images is the presence of important artifacts caused by dental implants. The presence of such artifacts hinders the use of intensity averages, thus severely limiting the application of most atlas building techniques described in the literature in this context. The results presented in the paper show that our bending energy model faithfully represents the shape variability of patients in the head and neck region; they also show its good performance in segmentation of volumes of interest in radiotherapy. Moreover, when compared to other atlases of similar performance in automatic segmentation, our atlas presents the desirable feature of not being blurred after intensity averaging.     

A novel framework for spatial normalization of dose distributions in voxel-based analyses of brain irradiation outcomes

PurposeTo devise a novel Spatial Normalization framework for Voxel-based analysis (VBA) in brain radiotherapy. VBAs rely on accurate spatial normalization of different patients’ planning CTs on a common coordinate system (CCS). The cerebral anatomy, well characterized by MRI, shows instead poor contrast in CT, resulting in potential inaccuracies in VBAs based on CT alone.MethodsWe analyzed 50 meningioma patients treated with proton-therapy, undergoing planning CT and T1-weighted (T1w) MRI. The spatial normalization pipeline based on MR and CT images consisted in: intra-patient registration of CT to T1w, inter-patient registration of T1w to MNI space chosen as CCS, doses propagation to MNI.The registration quality was compared with that obtained by Statistical Parametric Mapping software (SPM), used as benchmark. To evaluate the accuracy of dose normalization, the dose organ overlap (DOO) score was computed on gray matter, white matter and cerebrospinal fluid before and after normalization. In addition, the trends in the DOOs distribution were investigated by means of cluster analysis.ResultsThe registration quality was higher for the proposed method compared to SPM (p < 0.001). The DOO scores showed a significant improvement after normalization (p < 0.001). The cluster analysis highlighted 2 clusters, with one of them including the majority of data and exhibiting acceptable DOOs.ConclusionsOur study presents a robust tool for spatial normalization, specifically tailored for brain dose VBAs. Furthermore, the cluster analysis provides a formal criterion for patient exclusion in case of non-acceptable normalization results. The implemented framework lays the groundwork for future reliable VBAs in brain irradiation studies.     

Structural similarity regularized and sparse coding based super-resolution for medical images

Recently the single image super-resolution reconstruction (SISR) via sparse coding has attracted increasing interests. Considering that there are obviously repetitive image structures in medical images, in this study we propose a regularized SISR method via sparse coding and structural similarity. The pixel based recovery is incorporated as a regularization term to exploit the non-local structural similarities of medical images, which is very helpful in further improving the quality of recovered medical images. An alternative variables optimization algorithm is proposed and some medical images including CT, MRI and ultrasound images are used to investigate the performance of our proposed method. The results show the superiority of our method to its counterparts.     

Optical-CT Dual-Modality Mapping Base on DRR Registration

Optical-CT dual-modality imaging requires the mapping between 2D fluorescence images and 3D body surface light flux. In this paper, we proposed an optical-CT dual-modality image mapping algorithm based on the Digitally Reconstructed Radiography (DRR) registration. In the process of registration, a series of DRR images were computed from CT data using the ray casting algorithm. Then, the improved HMNI similarity strategy based on Hausdorff distance was used to complete the registration of the white-light optical images and DRR virtual images. According to the corresponding relationship obtained by the image registration and the Lambert’s cosine law based on the pin-hole imaging model, the 3D light intensity distribution on the surface of the object could be solved. The feasibility and effectiveness of the mapping algorithm are verified by the irregular phantom and mouse experiments.     

A method for measuring joint kinematics designed for accurate registration of kinematic data to models constructed from CT data

A method for measuring three-dimensional kinematics that incorporates the direct cross-registration of experimental kinematics with anatomic geometry from Computed Tomography (CT) data has been developed. Plexiglas registration blocks were attached to the bones of interest and the specimen was CT scanned. Computer models of the bone surface were developed from the CT image data. Determination of discrete kinematics was accomplished by digitizing three pre-selected contiguous surfaces of each registration block using a three-dimensional point digitization system. Cross-registration of bone surface models from the CT data was accomplished by identifying the registration block surfaces within the CT images. Kinematics measured during a biomechanical experiment were applied to the computer models of the bone surface. The overall accuracy of the method was shown to be at or below the accuracy of the digitization system used. For this experimental application, the accuracy was better than +/-0.1mm for position and 0.1 degrees for orientation for linkage digitization and better than +/-0.2mm and +/-0.2 degrees for CT digitization. Surface models of the radius and ulna were constructed from CT data, as an example application. Kinematics of the bones were measured for simulated forearm rotation. Screw-displacement axis analysis showed 0.1mm (proximal) translation of the radius (with respect to the ulna) from supination to neutral (85.2 degrees rotation) and 1.4mm (proximal) translation from neutral to pronation (65.3 degrees rotation). The motion of the radius with respect to the ulna was displayed using the surface models. This methodology is a useful tool for the measurement and application of rigid-body kinematics to computer models.     

Medical image registration using fuzzy theory

Mutual information (MI)-based registration, which uses MI as the similarity measure, is a representative method in medical image registration. It has an excellent robustness and accuracy, but with the disadvantages of a large amount of calculation and a long processing time. In this paper, by computing the medical image moments, the centroid is acquired. By applying fuzzy c-means clustering, the coordinates of the medical image are divided into two clusters to fit a straight line, and the rotation angles of the reference and floating images are computed, respectively. Thereby, the initial values for registering the images are determined. When searching the optimal geometric transformation parameters, we put forward the two new concepts of fuzzy distance and fuzzy signal-to-noise ratio (FSNR), and we select FSNR as the similarity measure between the reference and floating images. In the experiments, the Simplex method is chosen as multi-parameter optimisation. The experimental results show that this proposed method has a simple implementation, a low computational cost, a fast registration and good registration accuracy. Moreover, it can effectively avoid trapping into the local optima. It is adapted to both mono-modality and multi-modality image registrations.     

Non-rigid registration of serial dedicated breast CT,longitudinal dedicated breast CT and PET/CT images using the diffeomorphic demons method

Rationale and objectivesDedicated breast CT and PET/CT scanners provide detailed 3D anatomical and functional imaging data sets and are currently being investigated for applications in breast cancer management such as diagnosis, monitoring response to therapy and radiation therapy planning. Our objective was to evaluate the performance of the diffeomorphic demons (DD) non-rigid image registration method to spatially align 3D serial (pre- and post-contrast) dedicated breast computed tomography (CT), and longitudinally-acquired dedicated 3D breast CT and positron emission tomography (PET)/CT images.MethodsThe algorithmic parameters of the DD method were optimized for the alignment of dedicated breast CT images using training data and fixed. The performance of the method for image alignment was quantitatively evaluated using three separate data sets; (1) serial breast CT pre- and post-contrast images of 20 women, (2) breast CT images of 20 women acquired before and after repositioning the subject on the scanner, and (3) dedicated breast PET/CT images of 7 women undergoing neo-adjuvant chemotherapy acquired pre-treatment and after 1 cycle of therapy.ResultsThe DD registration method outperformed no registration (p < 0.001) and conventional affine registration (p ≤ 0.002) for serial and longitudinal breast CT and PET/CT image alignment. In spite of the large size of the imaging data, the computational cost of the DD method was found to be reasonable (3–5 min).ConclusionsCo-registration of dedicated breast CT and PET/CT images can be performed rapidly and reliably using the DD method. This is the first study evaluating the DD registration method for the alignment of dedicated breast CT and PET/CT images.     

Evaluation of a Learning-based Deformable Registration Method on Abdominal CT Images

BackgroundReliable image comparisons, based on fast and accurate deformable registration methods, are recognized as key steps in the diagnosis and follow-up of cancer as well as for radiation therapy planning or surgery. In the particular case of abdominal images, the images to compare often differ widely from each other due to organ deformation, patient motion, movements of gastrointestinal tract or breathing. As a consequence, there is a need for registration methods that can cope with both local and global large and highly non-linear deformations.MethodDeformable registration of medical images traditionally relies on the iterative minimization of a cost function involving a large number of parameters. For complex deformations and large datasets, this process is computationally very demanding, leading to processing times that are incompatible with the clinical routine workflow. Moreover, the highly non-convex nature of these optimization problems leads to a high risk of convergence toward local minima. Recently, deep learning approaches using Convolutional Neural Networks (CNN) have led to major breakthroughs by providing computationally fast unsupervised methods for the registration of 2D and 3D images within seconds. Among all the proposed approaches, the VoxelMorph learning-based framework pioneered to learn in an unsupervised way the complex mapping, parameterized using a CNN, between every couple of 2D or 3D pairs of images and the corresponding deformation field by minimizing a standard intensity-based similarity metrics over the whole learning database. Voxelmorph has so far only been evaluated on brain images. The present study proposes to evaluate this method in the context of inter-subject registration of abdominal CT images, which present a greater challenge in terms of registration than brain images, due to greater anatomical variability and significant organ deformations.ResultsThe performances of VoxelMorph were compared with the current top-performing non-learning-based deformable registration method “Symmetric Normalization” (SyN), implemented in ANTs, on two representative databases: LiTS and 3D-IRCADb-01. Three different experiments were carried out on 2D or 3D data, the atlas-based or pairwise registration, using two different similarity metrics, namely (MSE and CC). Accuracy of the registration was measured by the Dice score, which quantifies the volume overlap for the selected anatomical region.All the three experiments exhibit that the two deformable registration methods significantly outperform the affine registration and that VoxelMorph accuracy is comparable or even better than the reference non-learning based registration method ANTs (SyN), with a drastically reduced computation time.ConclusionBy substituting a time consuming optimization problem, VoxelMorph has made an outstanding achievement in learning-based registration algorithm, where a registration function is trained and thus, able to perform deformable registration almost accurately on abdominal images, while reducing the computation time from minutes to seconds and from seconds to milliseconds in comparison to ANTs (SyN) on a CPU.     

Registration of fundus images for generating wide field composite images of the retina ]

The composition of retinal images presents high demands to the applied methods. Substantially different lighting conditions between the images, glarings and fade-outs within one image, large textureless regions and non-linear distortions are the main challenges. We present a fully automatic algorithm for the registration of images of the human retina and their overlay to wide field montage images combining area-based and point-based approaches. The algorithm combines an area-based as well as a point-based approach for determining similarities between images. Various measures of similarity were investigated, where the normalized correlation coefficient was superior compared to the usual definitions of transinformation. The transformation of the images was based on a quadratic model that can be derived from the spherical surface of the retina. This model was compared to four other parameterized transformations and performed best both visually and quantitatively in terms of measured misregistration. Problems may occur if the images are extremely defocused or contain very little relevant structural information.     

Medical image registration using fuzzy theory

Mutual information (MI)-based registration, which uses MI as the similarity measure, is a representative method in medical image registration. It has an excellent robustness and accuracy, but with the disadvantages of a large amount of calculation and a long processing time. In this paper, by computing the medical image moments, the centroid is acquired. By applying fuzzy c-means clustering, the coordinates of the medical image are divided into two clusters to fit a straight line, and the rotation angles of the reference and floating images are computed, respectively. Thereby, the initial values for registering the images are determined. When searching the optimal geometric transformation parameters, we put forward the two new concepts of fuzzy distance and fuzzy signal-to-noise ratio (FSNR), and we select FSNR as the similarity measure between the reference and floating images. In the experiments, the Simplex method is chosen as multi-parameter optimisation. The experimental results show that this proposed method has a simple implementation, a low computational cost, a fast registration and good registration accuracy. Moreover, it can effectively avoid trapping into the local optima. It is adapted to both mono-modality and multi-modality image registrations.     

Validation of dose distribution computation on sCT images generated from MRI scans by Philips MRCAT

AimTo evaluate calculation of treatment plans based on synthetic-CT (sCT) images generated from MRI.BackgroundBecause of better soft tissue contrast, MR images are used in addition to CT images for radiotherapy planning. However, registration of CT and MR images or repositioning between scanning sessions introduce systematic errors, hence suggestions for MRI-only therapy. The lack of information on electron density necessary for dose calculation leads to sCT (synthetic CT) generation. This work presents a comparison of dose distribution calculated on standard CT and sCT.Materials and methods10 prostate patients were included in this study. CT and MR images were collected for each patient and then water equivalent (WE) and MRCAT images were generated. The radiation plans were optimized on CT and then recalculated on MRCAT and WE data. 2D gamma analysis was also performed.ResultsThe mean differences in the majority of investigated DVH points were in order of 1% up to 10%, including both MRCAT and WE dose distributions. Mean gamma pass for acceptance criteria 1%/1 mm were greater than 82.5%. Prescribed doses for target volumes and acceptable doses for organs at risk were met in almost all cases.ConclusionsThe dose calculation accuracy on MRCAT was not significantly compromised in the majority of clinical relevant DVH points. The introduction of MRCAT into practise would eliminate systematic errors, increase patients’ comfort and reduce treatment expenses. Institutions interested in MRCAT commissioning must, however, consider changes to established workflow.     

Evaluation of the Block Matching deformable registration algorithm in the field of head-and-neck adaptive radiotherapy

Background and purposeTo compare the accuracy of the Block Matching deformable registration (DIR) against rigid image registration (RIR) for head-and-neck multi-modal images CT to cone-beam CT (CBCT) registration.Material and methodsPlanning-CT and weekly CBCT of 10 patients were used for this study. Several volumes, including medullary canal (MC), thyroid cartilage (TC), hyoid bone (HB) and submandibular gland (SMG) were transposed from CT to CBCT images using either DIR or RIR. Transposed volumes were compared with the manual delineation of these volumes on every CBCT. The parameters of similarity used for analysis were: Dice Similarity Index (DSI), 95%-Hausdorff Distance (95%-HD) and difference of volumes (cc).ResultsWith DIR, the major mean difference of volumes was −1.4 cc for MC, revealing limited under-segmentation. DIR limited variability of DSI and 95%-HD. It significantly improved DSI for TC and HB and 95%-HD for all structures but SMG. With DIR, mean 95%-HD (mm) was 3.01 ± 0.80, 5.33 ± 2.51, 4.99 ± 1.69, 3.07 ± 1.31 for MC, TC, HB and SMG, respectively. With RIR, it was 3.92 ± 1.86, 6.94 ± 3.98, 6.44 ± 3.37 and 3.41 ± 2.25, respectively.ConclusionBlock Matching is a valid algorithm for deformable multi-modal CT to CBCT registration. Values of 95%-HD are useful for ongoing development of its application to the cumulative dose calculation.     

Normalization of multicolor fluorescence in situ hybridization (M-FISH) images for improving color karyotyping.

BACKGROUND: Multiplex or multicolor fluorescence in situ hybridization (M-FISH) is a recently developed cytogenetic technique for cancer diagnosis and research on genetic disorders. By simultaneously viewing the multiply labeled specimens in different color channels, M-FISH facilitates the detection of subtle chromosomal aberrations. The success of this technique largely depends on the accuracy of pixel classification (color karyotyping). Improvements in classifier performance would allow the elucidation of more complex and more subtle chromosomal rearrangements. Normalization of M-FISH images has a significant effect on the accuracy of classification. In particular, misalignment or misregistration across multiple channels seriously affects classification accuracy. Image normalization, including automated registration, must be done before pixel classification. METHODS AND RESULTS: We studied several image normalization approaches that affect image classification. In particular, we developed an automated registration technique to correct misalignment across the different fluor images (caused by chromatic aberration and other factors). This new registration algorithm is based on wavelets and spline approximations that have computational advantages and improved accuracy. To evaluate the performance improvement brought about by these data normalization approaches, we used the downstream pixel classification accuracy as a measurement. A Bayesian classifier assumed that each of 24 chromosome classes had a normal probability distribution. The effects that this registration and other normalization steps have on subsequent classification accuracy were evaluated on a comprehensive M-FISH database established by Advanced Digital Imaging Research (http://www.adires.com/05/Project/MFISH_DB/MFISH_DB.shtml). CONCLUSIONS: Pixel misclassification errors result from different factors. These include uneven hybridization, spectral overlap among fluors, and image misregistration. Effective preprocessing of M-FISH images can decrease the effects of those factors and thereby increase pixel classification accuracy. The data normalization steps described in this report, such as image registration and background flattening, can significantly improve subsequent classification accuracy. An improved classifier in turn would allow subtle DNA rearrangements to be identified in genetic diagnosis and cancer research.     

Dynamic Lung Modeling and Tumor Tracking Using Deformable Image Registration and Geometric Smoothing

A greyscale-based fully automatic deformable image registration algorithm, based on an optical flow method together with geometric smoothing, is developed for dynamic lung modeling and tumor tracking. In our computational processing pipeline, the input data is a set of 4D CT images with 10 phases. The triangle mesh of the lung model is directly extracted from the more stable exhale phase (Phase 5). In addition, we represent the lung surface model in 3D volumetric format by applying a signed distance function and then generate tetrahedral meshes. Our registration algorithm works for both triangle and tetrahedral meshes. In CT images, the intensity value reflects the local tissue density. For each grid point, we calculate the displacement from the static image (Phase 5) to match with the moving image (other phases) by using merely intensity values of the CT images. The optical flow computation is followed by a regularization of the deformation field using geometric smoothing. Lung volume change and the maximum lung tissue movement are used to evaluate the accuracy of the application. Our testing results suggest that the application of deformable registration algorithm is an effective way for delineating and tracking tumor motion in image-guided radiotherapy.     

Evolutionarily distinctive species often capture more phylogenetic diversity than expected

Evolutionary distinctiveness measures of how evolutionarily isolated a species is relative to other members of its clade. Recently, distinctiveness metrics that explicitly incorporate time have been proposed for conservation prioritization. However, we found that such measures differ qualitatively in how well they capture the total amount of evolution (termed phylogenetic diversity, or PD) represented by a set of species. We used simulation and simple graph theory to explore this relationship with reference to phylogenetic tree shape. Overall, the distinctiveness measures capture more PD on more unbalanced trees and on trees with many splits near the present. The rank order of performance was robust across tree shapes, with apportioning measures performing best and node-based measures performing worst. A sample of 50 ultrametric trees from the literature showed the same patterns. Taken together, this suggests that distinctiveness metrics may be a useful addition to other measures of value for conservation prioritization of species. The simplest measure, the age of a species, performed surprisingly well, suggesting that new measures that focus on tree shape near the tips may provide a transparent alternative to more complicated full-tree approaches.     

Registration of pedobarographic image data in the frequency domain

Image registration has been used to support pixel-level data analysis on pedobarographic image data sets. Some registration methods have focused on robustness and sacrificed speed, but a recent approach based on external contours offered both high computational processing speed and high accuracy. However, since contours can be influenced by local perturbations, we sought more global methods. Thus, we propose two new registration methods based on the Fourier transform, cross-correlation and phase correlation which offer high computational speed. We found out that both proposed methods revealed high accuracy for the similarity measures considered, using control geometric transformations. Additionally, both methods revealed high computational processing speed which, combined with their accuracy and robustness, allows their implementation in near-real-time applications. Furthermore, we found that the current methods were robust to moderate levels of noise, and consequently, do not require noise removal procedure like the contours method does.     

Detection of Time-Varying Structures by Large Deformation Diffeomorphic Metric Mapping to Aid Reading of High-Resolution CT Images of the Lung

Objectives

To evaluate the accuracy of advanced non-linear registration of serial lung Computed Tomography (CT) images using Large Deformation Diffeomorphic Metric Mapping (LDDMM).

Methods

Fifteen cases of lung cancer with serial lung CT images (interval: 62.2±26.9 days) were used. After affine transformation, three dimensional, non-linear volume registration was conducted using LDDMM with or without cascading elasticity control. Registration accuracy was evaluated by measuring the displacement of landmarks placed on vessel bifurcations for each lung segment. Subtraction images and Jacobian color maps, calculated from the transformation matrix derived from image warping, were generated, which were used to evaluate time-course changes of the tumors.

Results

The average displacement of landmarks was 0.02±0.16 mm and 0.12±0.60 mm for proximal and distal landmarks after LDDMM transformation with cascading elasticity control, which was significantly smaller than 3.11±2.47 mm and 3.99±3.05 mm, respectively, after affine transformation. Emerged or vanished nodules were visualized on subtraction images, and enlarging or shrinking nodules were displayed on Jacobian maps enabled by highly accurate registration of the nodules using LDDMM. However, some residual misalignments were observed, even with non-linear transformation when substantial changes existed between the image pairs.

Conclusions

LDDMM provides accurate registration of serial lung CT images, and temporal subtraction images with Jacobian maps help radiologists to find changes in pulmonary nodules.