Global Patch Matching (GPM) for freehand 3D ultrasound reconstruction

Background

3D ultrasound volume reconstruction from B-model ultrasound slices can provide more clearly and intuitive structure of tissue and lesion for the clinician.

Methods

This paper proposes a novel Global Path Matching method for the 3D reconstruction of freehand ultrasound images. The proposed method composes of two main steps: bin-filling scheme and hole-filling strategy. For the bin-filling scheme, this study introduces two operators, including the median absolute deviation and the inter-quartile range absolute deviation, to calculate the invariant features of each voxel in the 3D ultrasound volume. And the best contribution range for each voxel is obtained by calculating the Euclidian distance between current voxel and the voxel with the minimum invariant features. Hence, the intensity of the filling vacant voxel can be obtained by weighted combination of the intensity distribution of pixels in the best contribution range. For the hole-filling strategy, three conditions, including the confidence term, the data term and the gradient term, are designed to calculate the weighting coefficient of the matching patch of the vacant voxel. While the matching patch is obtained by finding patches with the best similarity measure that defined by the three conditions in the whole 3D volume data.

Results

Compared with VNN, PNN, DW, FMM, BI and KR methods, the proposed Global Path Matching method can restore the 3D ultrasound volume with minimum difference.

Conclusions

Experimental results on phantom and clinical data sets demonstrate the effectiveness and robustness of the proposed method for the reconstruction of ultrasound volume.

     

Efficient implementation of a filtered back-projection algorithm using a voxel-by-voxel approach

The large amount of image data necessary for high-resolution 3D reconstruction of macromolecular assemblies leads to significant increases in the computational time. One of the most time consuming operations is 3D density map reconstruction, and software optimization can greatly reduce the time required for any given structural study. The majority of algorithms proposed for improving the computational effectiveness of a 3D reconstruction are based on a ray-by-ray projection of each image into the reconstructed volume. In this paper, we propose a novel fast implementation of the "filtered back-projection" algorithm based on a voxel-by-voxel principle. Our version of this implementation has been exhaustively tested using both model and real data. We compared 3D reconstructions obtained by the new approach with results obtained by the filtered Back-Projections algorithm and the Fourier-Bessel algorithm commonly used for reconstructing icosahedral viruses. These computational experiments demonstrate the robustness, reliability, and efficiency of this approach.     

Comparative study on the performance of textural image features for active contour segmentation

We present a computerized method for the semi-automatic detection of contours in ultrasound images.The novelty of our study is the introduction of a fast and efficient image function relating to parametric active contour models.This new function is a combination of the gray-level information and first-order statistical features,called standard deviation parameters.In a comprehensive study,the developed algorithm and the efficiency of segmentation were first tested for synthetic images.Tests were also performed on breast and liver ultrasound images.The proposed method was compared with the watershed approach to show its efficiency.The performance of the segmentation was estimated using the area error rate.Using the standard deviation textural feature and a 5×5 kernel,our curve evolution was able to produce results close to the minimal area error rate(namely 8.88% for breast images and 10.82% for liver images).The image resolution was evaluated using the contrast-to-gradient method.The experiments showed promising segmentation results.     

A fast reconstruction algorithm for electron microscope tomography

We have implemented a Fast Fourier Summation algorithm for tomographic reconstruction of three-dimensional biological data sets obtained via transmission electron microscopy. We designed the fast algorithm to reproduce results obtained by the direct summation algorithm (also known as filtered or R-weighted backprojection). For two-dimensional images, the new algorithm scales as O(N(theta)M log M)+O(MN log N) operations, where N(theta) is the number of projection angles and M x N is the size of the reconstructed image. Three-dimensional reconstructions are constructed from sequences of two-dimensional reconstructions. We demonstrate the algorithm on real data sets. For typical sizes of data sets, the new algorithm is 1.5-2.5 times faster than using direct summation in the space domain. The speed advantage is even greater as the size of the data sets grows. The new algorithm allows us to use higher order spline interpolation of the data without additional computational cost. The algorithm has been incorporated into a commonly used package for tomographic reconstruction.     

Terrestrial laser scanner based 3D reconstruction of trees and retrieval of leaf area index in a forest environment

Three-dimensional reconstruction of trees and the estimation of biophysical parameters is significant for the management of forest resources, ecological studies carbon cycle and biodiversity. Terrestrial LiDAR data provides detailed, objective and three-dimensional measurement of forest structure and exact metrics of the tree canopies. Several methods for tree detection including canopy height models and raster interpolation models are based on commercial software and huge data processing. The objective of the given study is the three-dimensional reconstruction of trees by implementing segmentation algorithms and thereby estimating the Leaf Area Index of individual tree segments by terrestrial laser scanned data in the Mudumalai forests of Western Ghats, India. The hierarchical minimum cut segmentation method is used for the three-dimensional reconstruction of the individual trees by tracking cylinders along individual branches and trees in a hierarchical order. Super voxel clustering method is also implemented in the study for tree reconstruction and estimating the tree parameters. Leaf area index is calculated by applying a multivariate regression technique for the heights and the diameter obtained from both the segmentation methods. Results obtained indicated a strong correlation with the in-situ measurements which are obtained from the instruments. The approach addresses the applicability of segmentation algorithms which can be run fully automatically. The approach successfully reconstructed a high precision and realistic model of trees in the Western Ghats region which failed in the case of traditional tree modeling methods which requires multiple instruments operating simultaneously for extracting each parameter. The method proved that using TLS; multiple forest parameters can be estimated simultaneously.     

IVUS Validation of Patient Coronary Artery Lumen Area Obtained from CT Images

Aims

Accurate computed tomography (CT)-based reconstruction of coronary morphometry (diameters, length, bifurcation angles) is important for construction of patient-specific models to aid diagnosis and therapy. The objective of this study is to validate the accuracy of patient coronary artery lumen area obtained from CT images based on intravascular ultrasound (IVUS).

Methods and Results

Morphometric data of 5 patient CT scans with 11 arteries from IVUS were reconstructed including the lumen cross sectional area (CSA), diameter and length. The volumetric data from CT images were analyzed at sub-pixel accuracy to obtain accurate vessel center lines and CSA. A new center line extraction approach was used where an initial estimated skeleton in discrete value was obtained using a traditional thinning algorithm. The CSA was determined directly without any circular shape assumptions to provide accurate reconstruction of stenosis. The root-mean-square error (RMSE) for CSA and diameter were 16.2% and 9.5% respectively.

Conclusions

The image segmentation and CSA extraction algorithm for reconstruction of coronary arteries proved to be accurate for determination of vessel lumen area. This approach provides fundamental morphometric data for patient-specific models to diagnose and treat coronary artery disease.     

一种图像分层表述的快速算法

提出一种新的灰度图像分层表述算法.该算法的核心是基于一系列具有高度规律性的灰度函数gn(x,y)来逼近不规则的原图像灰度函数f(x,y).本算法具有快速收敛性,是一个良好的逼近器.由于gn具有高度规律性,致使用较少的存储空间就可实现对它的存储.这一算法为生物图像的数据处理和重构提供一条新的途径.     

Bayesian Inference of Initial Models in Cryo-Electron Microscopy Using Pseudo-atoms

Single-particle cryo-electron microscopy is widely used to study the structure of macromolecular assemblies. Tens of thousands of noisy two-dimensional images of the macromolecular assembly viewed from different directions are used to infer its three-dimensional structure. The first step is to estimate a low-resolution initial model and initial image orientations. This is a challenging global optimization problem with many unknowns, including an unknown orientation for each two-dimensional image. Obtaining a good initial model is crucial for the success of the subsequent refinement step. We introduce a probabilistic algorithm for estimating an initial model. The algorithm is fast, has very few algorithmic parameters, and yields information about the precision of estimated model parameters in addition to the parameters themselves. Our algorithm uses a pseudo-atomic model to represent the low-resolution three-dimensional structure, with isotropic Gaussian components as moveable pseudo-atoms. This leads to a significant reduction in the number of parameters needed to represent the three-dimensional structure, and a simplified way of computing two-dimensional projections. It also contributes to the speed of the algorithm. We combine the estimation of the unknown three-dimensional structure and image orientations in a Bayesian framework. This ensures that there are very few parameters to set, and specifies how to combine different types of prior information about the structure with the given data in a systematic way. To estimate the model parameters we use Markov chain Monte Carlo sampling. The advantage is that instead of just obtaining point estimates of model parameters, we obtain an ensemble of models revealing the precision of the estimated parameters. We demonstrate the algorithm on both simulated and real data.     

Three-dimensional reconstruction of the actin cytoskeleton from stereo images

In eucaryotic cells, actin filaments are abundant components in the cytoskeleton where they form a complex three dimensional (3D) structural network that provides the cell with its shape and mechanical properties. However, understanding the structural and mechanical properties of actin filaments composing the cell cytoskeleton is often hampered by the inability to faithfully reconstruct the three-dimensional geometric relationships. This paper presents a vision-based reconstruction approach that automatically reconstitutes the three-dimensional structures of cytoskeletal polymers from stereo image pairs taken at the different tilt angles. The approach finds corresponding points between two images and recovers the depth information about the structures. The computational process consists of three major procedures: feature representation, stereo matching, and disparity refinement, implemented in a multi-resolution manner based on a coarse-to-fine strategy. The reconstruction depicts the three-dimensional structure of cytoskeletal polymers and their geometric relationships. New and useful information becomes available and allows quantitative analysis of the structure. Measurement of the cytoskeleton geometrical properties and the filament concentration in a defined volume are obtained by direct calculation.     

Finite element 3D reconstruction of the pulmonary acinus imaged by synchrotron X-ray tomography

The alveolated structure of the pulmonary acinus plays a vital role in gas exchange function. Three-dimensional (3D) analysis of the parenchymal region is fundamental to understanding this structure-function relationship, but only a limited number of attempts have been conducted in the past because of technical limitations. In this study, we developed a new image processing methodology based on finite element (FE) analysis for accurate 3D structural reconstruction of the gas exchange regions of the lung. Stereologically well characterized rat lung samples (Pediatr Res 53: 72-80, 2003) were imaged using high-resolution synchrotron radiation-based X-ray tomographic microscopy. A stack of 1,024 images (each slice: 1024 x 1024 pixels) with resolution of 1.4 mum(3) per voxel were generated. For the development of FE algorithm, regions of interest (ROI), containing approximately 7.5 million voxels, were further extracted as a working subunit. 3D FEs were created overlaying the voxel map using a grid-based hexahedral algorithm. A proper threshold value for appropriate segmentation was iteratively determined to match the calculated volume density of tissue to the stereologically determined value (Pediatr Res 53: 72-80, 2003). The resulting 3D FEs are ready to be used for 3D structural analysis as well as for subsequent FE computational analyses like fluid dynamics and skeletonization.     

The "third" dimension in craniofacial surgery

A new method for reconstruction of a three-dimensional surface from a sequence of high-resolution axial CT scans has been developed. This algorithm is realized as a set of computer programs that can operate on commercially available CT scanners or evaluation consoles. The program is both efficient and easy to implement. No operator intervention is required. The images produced simulate photographs of the skull. Frontal, lateral, oblique, bird's eye, worm's eye, and rear views are generated. As with photographs and conventional radiographs, each of these projections uniquely displays specific anatomic details. This method of osseous surface reconstruction is now routinely applied to all patients evaluated for major craniofacial reconstruction at our institution. The images are useful in defining aberrant anatomy, planning surgical procedures, and evaluating the results of such operations. This method replaces an inexact concept in the surgeon's imagination with a three-dimensional image of the craniofacial skeleton.     

Efficient and cost‐effective 3D cellular imaging by sub‐voxel‐resolving light‐sheet add‐on microscopy

Light‐sheet fluorescence microscopy (LSFM) allows volumetric live imaging at high‐speed and with low photo‐toxicity. Various LSFM modalities are commercially available, but their size and cost limit their access by the research community. A new method, termed sub‐voxel‐resolving (SVR) light‐sheet add‐on microscopy (SLAM), is presented to enable fast, resolution‐enhanced light‐sheet fluorescence imaging from a conventional wide‐field microscope. This method contains two components: a miniature add‐on device to regular wide‐field microscopes, which contains a horizontal laser light‐sheet illumination path to confine fluorophore excitation at the vicinity of the focal plane for optical sectioning; an off‐axis scanning strategy and a SVR algorithm that utilizes sub‐voxel spatial shifts to reconstruct the image volume that results in a twofold increase in resolution. SLAM method has been applied to observe the muscle activity change of crawling C. elegans, the heartbeat of developing zebrafish embryo, and the neural anatomy of cleared mouse brains, at high spatiotemporal resolution. It provides an efficient and cost‐effective solution to convert the vast number of in‐service microscopes for fast 3D live imaging with voxel‐super‐resolved capability.     

Transverse-axial tubular system in guinea pig ventricular cardiomyocyte: 3D reconstruction,quantification and its possible role in K+ accumulation-depletion phenomenon in single cells

Summary— K⁺ accumulation-depletion (AD) phenomena were found in single guinea pig ventricular myocytes using the patch-clamp method in whole cell configuration. We suggest that the cardiomyocyte transverse-axial tubular system (TATS) lumen is the restricted extracellular space where the K⁺ AD could take place. A three-dimensional (3D) reconstruction of the TATS in a cardiomyocyte segment from serial ultrafine sections was made by three-dimensional isosurface rendering and quantitative data were obtained from the image processing. This original approach of the TATS intricated network gave a new vision of this membrane system; moreover, quantitative data about the tubular membrane importance (52.6% of the total plasma membrane) and its surface area versus the tubular volume fraction (S_TATS/V_TATS = 13.5 μm²/μm³ would fit in the electrophysiological results. The hypothesis whereby this ‘extracellular’ compartment could play, in single cells, a role as important as that of narrow clefts in the whole heart is discussed.     

Morphometric and three-dimensional study of platelets during activation in the rat

In vivo activation of platelets, produced by damaging an artery in the rat with ultrasound, was studied with the electron microscope. We performed both a three-dimensional reconstruction by thin serial sections and a morphometric study of the activation process. This is characterized by exocytosis of the content of granules, widening of tubules of the open canalicular system, and emission of pseudopodia. The three-dimensional reconstruction suggested that some extended pseudopodia adhere to the arterial intima, and confirmed former observations that a locomotor apparatus differentiates in platelets adhering to the arterial intima. We speculate that the contraction of some pseudopodia may pull the platelet body toward the arterial wall. The morphometric study revealed that during activation the volume of the open canalicular system volume increases and that of the dense granules decreases; all other compartments did not change.     

Exploiting desktop supercomputing for three-dimensional electron microscopy reconstructions using ART with blobs

Three-dimensional electron microscopy allows direct visualization of biological macromolecules close to their native state. The high impact of this technique in the structural biology field is highly correlated with the development of new image processing algorithms. In order to achieve subnanometer resolution, the size and number of images involved in a three-dimensional reconstruction increase and so do computer requirements. New chips integrating multiple processors are hitting the market at a reduced cost. This high-integration, low-cost trend has just begun and is expected to bring real supercomputers to our laboratory desktops in the coming years. This paper proposes a parallel implementation of a computation-intensive algorithm for three-dimensional reconstruction, ART, that takes advantage of the computational power in modern multicore platforms. ART is a sophisticated iterative reconstruction algorithm that has turned out to be well suited for the conditions found in three-dimensional electron microscopy. In view of the performance obtained in this work, these modern platforms are expected to play an important role to face the future challenges in three-dimensional electron microscopy.     

Smooth surface meshing for automated finite element model generation from 3D image data

Finite element (FE) modelling based on data from three-dimensional high-resolution computed tomography (CT) imaging systems provides a non-invasive method to assess structural mechanics. Automated mesh generation from these voxel based image data can be achieved by direct conversion to hexahedron elements, however these model representations have jagged edges. This paper proposes an automated method to generate smoothed FE meshes from voxel-based image data. Mesh fairing processes are utilized that allow constraints that control the smoothing process, and are computationally efficient. Surfaces of the mesh on the exterior, as well as interfaces between two tissues, can be smoothed by varying fairing parameters and constraint criteria. The method was tested on a variety of real and simulated three-dimensional data sets, resulting in both hexahedron and tetrahedron meshes. It was shown that the fairing process is linearly related to the number of smoothing iterations, and that peak stresses are reduced in FE simulations of the smoothed models. Although developed for micro-CT data sets, this fast and reliable mesh smoothing method could be applied to any three-dimensional image data where node and element connectivity have been defined.     

Vessel network extraction and analysis of mouse pulmonary vasculature via X-ray micro-computed tomographic imaging

In this work, non-invasive high-spatial resolution three-dimensional (3D) X-ray micro-computed tomography (μCT) of healthy mouse lung vasculature is performed. Methodologies are presented for filtering, segmenting, and skeletonizing the collected 3D images. Novel methods for the removal of spurious branch artefacts from the skeletonized 3D image are introduced, and these novel methods involve a combination of distance transform gradients, diameter-length ratios, and the fast marching method (FMM). These new techniques of spurious branch removal result in the consistent removal of spurious branches without compromising the connectivity of the pulmonary circuit. Analysis of the filtered, skeletonized, and segmented 3D images is performed using a newly developed Vessel Network Extraction algorithm to fully characterize the morphology of the mouse pulmonary circuit. The removal of spurious branches from the skeletonized image results in an accurate representation of the pulmonary circuit with significantly less variability in vessel diameter and vessel length in each generation. The branching morphology of a full pulmonary circuit is characterized by the mean diameter per generation and number of vessels per generation. The methods presented in this paper lead to a significant improvement in the characterization of 3D vasculature imaging, allow for automatic separation of arteries and veins, and for the characterization of generations containing capillaries and intrapulmonary arteriovenous anastomoses (IPAVA).     

IPET and FETR: experimental approach for studying molecular structure dynamics by cryo-electron tomography of a single-molecule structure

The dynamic personalities and structural heterogeneity of proteins are essential for proper functioning. Structural determination of dynamic/heterogeneous proteins is limited by conventional approaches of X-ray and electron microscopy (EM) of single-particle reconstruction that require an average from thousands to millions different molecules. Cryo-electron tomography (cryoET) is an approach to determine three-dimensional (3D) reconstruction of a single and unique biological object such as bacteria and cells, by imaging the object from a series of tilting angles. However, cconventional reconstruction methods use large-size whole-micrographs that are limited by reconstruction resolution (lower than 20 Å), especially for small and low-symmetric molecule (<400 kDa). In this study, we demonstrated the adverse effects from image distortion and the measuring tilt-errors (including tilt-axis and tilt-angle errors) both play a major role in limiting the reconstruction resolution. Therefore, we developed a “focused electron tomography reconstruction” (FETR) algorithm to improve the resolution by decreasing the reconstructing image size so that it contains only a single-instance protein. FETR can tolerate certain levels of image-distortion and measuring tilt-errors, and can also precisely determine the translational parameters via an iterative refinement process that contains a series of automatically generated dynamic filters and masks. To describe this method, a set of simulated cryoET images was employed; to validate this approach, the real experimental images from negative-staining and cryoET were used. Since this approach can obtain the structure of a single-instance molecule/particle, we named it individual-particle electron tomography (IPET) as a new robust strategy/approach that does not require a pre-given initial model, class averaging of multiple molecules or an extended ordered lattice, but can tolerate small tilt-errors for high-resolution single “snapshot” molecule structure determination. Thus, FETR/IPET provides a completely new opportunity for a single-molecule structure determination, and could be used to study the dynamic character and equilibrium fluctuation of macromolecules.     

3D TOF-PET image reconstruction using total variation regularization

In this paper we introduce a semi-analytic algorithm for 3-dimensional image reconstruction for positron emission tomography (PET). The method consists of the back-projection of the acquired data into the most likely image voxel according to time-of-flight (TOF) information, followed by the filtering step in the image space using an iterative optimization algorithm with a total variation (TV) regularization. TV regularization in image space is more computationally efficient than usual iterative optimization methods for PET reconstruction with full system matrix that use TV regularization. The efficiency comes from the one-time TOF back-projection step that might also be described as a reformatting of the acquired data. An important aspect of our work concerns the evaluation of the filter operator of the linear transform mapping an original radioactive tracer distribution into the TOF back-projected image. We obtain concise, closed-form analytical formula for the filter operator. The proposed method is validated with the Monte Carlo simulations of the NEMA IEC phantom using a one-layer, 50 cm-long cylindrical device called Jagiellonian PET scanner. The results show a better image quality compared with the reference TOF maximum likelihood expectation maximization algorithm.     

3-D reconstruction of biological objects using underwater video technique and image processing

This paper describes a 3-D reconstruction method which allows accurate measurements of volume, surface area and other morphometric measurements of three-dimensional biological objects, without removing them from the sea. It represents a novel approach based on multiple views (eight resulted to be sufficient) from underwater video images and a new image processing procedure (MOD3D), whose application has met the basic requirements (i.e. to work on images recorded in turbid waters, with nonuniform lighting, to investigate large areas and in reasonable time, etc.) imposed when operating in the marine environment with simple, easy-to-use and nonprofessional equipment. It is a noninvasive, nondestructive and in the field fast method, thus suitable for sampling also at relevant depth, whose applicability has specifically been set up for a range of growth forms from massive to submassive and irregularly shaped. The accuracy of the method was assessed using models with three levels of 3-D complexity: simple, moderate and complex morphology. A high accuracy of volume measurements made through MOD3D image analysis software was achieved when compared with the laboratory water displacement method, which represents the most accurate method for volume measurement, with an overall mean percent error of about 1.7% (S.D. 2.2%). For all three levels of morphologic complexity, no significant differences (p>0.05) were found. Volume measurements obtained in field based on geometric approximation resulted rough, with significant differences from the MOD3D values (p<0.05). The geometric approximation was lower than MOD3D for simple and moderate morphology, and variable for complex morphology. For all three models, MOD3D values for surface area computation were consistently lower (mean error 13%) than the foil-wrapping values (p<0.05), due to overlap error when foil wrapping. Two applications were made with the bryozoan Pentapora fascialis and the coral Cladocora caespitosa to quantify carbonate standing stock and biomass of these two carbonate framework builders, whose importance has been recently recognised among the temperate sublittoral benthic species. Time required for the 3-D reconstruction method (about 3 h) makes it suitable for routine application particularly for relatively large area investigations, with irregularly shaped objects on rough substrate and several biological objects within the area.