Constructing complex 3D biological environments from medical imaging using high performance computing

Extracting information about the structure of biological tissue from static image data is a complex task requiring computationally intensive operations. Here, we present how multicore CPUs and GPUs have been utilized to extract information about the shape, size, and path followed by the mammalian oviduct, called the fallopian tube in humans, from histology images, to create a unique but realistic 3D virtual organ. Histology images were processed to identify the individual cross sections and determine the 3D path that the tube follows through the tissue. This information was then related back to the histology images, linking the 2D cross sections with their corresponding 3D position along the oviduct. A series of linear 2D spline cross sections, which were computationally generated for the length of the oviduct, were bound to the 3D path of the tube using a novel particle system technique that provides smooth resolution of self-intersections. This results in a unique 3D model of the oviduct, which is grounded in reality. The GPU is used for the processor intensive operations of image processing and particle physics based simulations, significantly reducing the time required to generate a complete model.     

Film tomography as a tool for three-dimensional image construction and gene expression studies

In order to observe three-dimensional (3D) expression patterns of genes in whole animals, whole organs, or whole tissues, in situ hybridization (ISH) of many sections must be carried out and then used to construct a 3D image. For this purpose, we have developed an automatic microtome to prepare tissue sections with an adhesive film. We used commercially available film suitable for sectioning and ISH. We constructed a microtome and, after adherence of the film to a paraffin-embedded tissue block, cut the block with a blade to prepare sections on film. Then, the sections-on-film were automatically set in a plastic frame that was the same size as a conventional glass slide. With this automatic microtome, tissue sections can be made for ISH or immunohistochemistry in addition to conventional hematoxylin and eosin staining without specific training. We demonstrate that we can construct 3D images of gene expression patterns obtained by ISH on sections prepared with this automatic microtome. We have designated this method as 'Film Tomography (FITO)'.     

Combined method of whole mount and block-face imaging: Acquisition of 3D data of gene expression pattern from conventional in situ hybridization

Visualization of spatiotemporal expression of a gene of interest is a fundamental technique for analyzing the involvements of genes in organ development. In situ hybridization (ISH) is one of the most popular methods for visualizing gene expression. When conventional ISH is performed on sections or whole-mount specimens, the gene expression pattern is represented in 2-dimensional (2D) microscopic images or in the surface view of the specimen. To obtain 3-dimensional (3D) data of gene expression from conventional ISH, the “serial section method” has traditionally been employed. However, this method requires an extensive amount of time and labor because it requires researchers to collect a tremendous number of sections, label all sections by ISH, and image them before 3D reconstruction. Here, we proposed a rapid and low-cost 3D imaging method that can create 3D gene expression patterns from conventional ISH-labeled specimens. Our method consists of a combination of whole-mount ISH and Correlative Microscopy and Blockface imaging (CoMBI). The whole-mount ISH-labeled specimens were sliced using a microtome or cryostat, and all block-faces were imaged and used to reconstruct 3D images by CoMBI. The 3D data acquired using our method showed sufficient quality to analyze the morphology and gene expression patterns in the developing mouse heart. In addition, 2D microscopic images of the sections can be obtained when needed. Correlating 2D microscopic images and 3D data can help annotate gene expression patterns and understand the anatomy of developing organs. These results indicated that our method can be useful in the field of developmental biology.     

A new numerical approach to mechanically analyse biological structures

In this work, an advanced discretization meshless technique is used to study the structural response of a human brain due to an impact load. The 2D and 3D brain geometrical models, and surrounding structures, were obtained through the processing of medical images, allowing to achieve a realistic geometry for the virtual model and to define the distribution of the mechanical properties accordingly with the medical images colour scale. Additionally, a set of essential and natural boundary conditions were assumed in order to reproduce a sudden impact force applied to the cranium. Then, a structural numerical analysis was performed using the Natural Neighbour Radial Point Interpolation Method (NNRPIM). The obtained results were compared with the finite element method (FEM) and a solution available in the literature. This work shows that the NNRPIM is a robust and accurate numerical technique, capable to produce results very close to other numerical approaches. In addition, the variable fields obtained with the meshless method are much smoother than the FEM corresponding solution.     

Actin filaments, stereocilia, and hair cells of the bird cochlea. II. Packing of actin filaments in the stereocilia and in the cuticular plate and what happens to the organization when the stereocilia are bent

A comparison of hair cells from different parts of the cochlea reveals the same organization of actin filaments; the elements that vary are the length and number of the filaments. Thin sections of stereocilia reveal that the actin filaments are hexagonally packed and from diffraction patterns of these sections we found that the actin filaments are aligned such that the crossover points of adjacent actin filaments are in register. As a result, the cross-bridges that connect adjacent actin filaments are easily seen in longitudinal sections. The cross-bridges appear as regularly spaced bands that are perpendicular to the axis of the stereocilium. Particularly interesting is that, unlike what one might predict, when a stereocilium is bent or displaced, as might occur during stimulation by sound, the actin filaments are not compressed or stretched but slide past one another so that the bridges become tilted relative to the long axis of the actin filament bundle. In the images of bent bundles, the bands of cross- bridges are then tilted off perpendicular to the stereocilium axis. When the stereocilium is bent at its base, all cross-bridges in the stereocilium are affected. Thus, resistance to bending or displacement must be property of the number of bridges present, which in turn is a function of the number of actin filaments present and their respective lengths. Since hair cells in different parts of the cochlea have stereocilia of different, yet predictable lengths and widths, this means that the force needed to displace the stereocilia of hair cells located at different regions of the cochlea will not be the same. This suggests that fine tuning of the hair cells must be a built-in property of the stereocilia. Perhaps its physiological vulnerability may result from changes of stereociliary structure.     

Confocal DNA cytometry: a contour-based segmentation algorithm for automated three-dimensional image segmentation

BACKGROUND: Confocal laser scanning microscopy (CLSM) presents the opportunity to perform three-dimensional (3D) DNA content measurements on intact cells in thick histological sections. So far, these measurements have been performed manually, which is quite time-consuming. METHODS: In this study, an intuitive contour-based segmentation algorithm for automatic 3D CLSM image cytometry of nuclei in thick histological sections is presented. To evaluate the segmentation algorithm, we measured the DNA content and volume of human liver and breast cancer nuclei in 3D CLSM images. RESULTS: A high percentage of nuclei could be segmented fully automatically (e.g., human liver, 92%). Comparison with (time-consuming) interactive measurements on the same CLSM images showed that the results were well correlated (liver, r = 1.00; breast, r = 0.92). CONCLUSIONS: Automatic 3D CLSM image cytometry enables measurement of volume and DNA content of large numbers of nuclei in thick histological sections within an acceptable time. This makes large-scale studies feasible, whereby the advantages of CLSM can be exploited fully. The intuitive modular segmentation algorithm presented in this study detects and separates overlapping objects, also in two-dimensional (2D) space. Therefore, this algorithm may also be suitable for other applications.     

A multiresolution approach to orientation assignment in 3D electron microscopy of single particles

Three-dimensional (3D) electron microscopy (3DEM) aims at the determination of the spatial distribution of the Coulomb potential of macromolecular complexes. The 3D reconstruction of a macromolecule using single-particle techniques involves thousands of 2D projections. One of the key parameters required to perform such a 3D reconstruction is the orientation of each projection image as well as its in-plane orientation. This information is unknown experimentally and must be determined using image-processing techniques. We propose the use of wavelets to match the experimental projections with those obtained from a reference 3D model. The wavelet decomposition of the projection images provides a framework for a multiscale matching algorithm in which speed and robustness to noise are gained. Furthermore, this multiresolution approach is combined with a novel orientation selection strategy. Results obtained from computer simulations as well as experimental data encourage the use of this approach.     

Two- and three-dimensional image reconstructions from stained and autoradiographed histological sections

A complete system has been developed to utilize histologicalserial sections for two- and three-dimensional image reconstructions.Eighty to 120 sections are digitized using a personal computingsystem augmented with a imaging board and CCD camera. The imagefiles are transmitted to a VAX computer for processing and imagereconstruction, and the processed images are transmitted backto the personal computer for display and recording using a filmrecorder or PostScript printer. The software developed for thesystem allows serial sections to be placed into proper registrationin a 256³ array, 256 grey levels. Autoradiographs of the sectionsare obtained in the presence of appropriate standards whichare used to recalibrate grey levels to represent linearly theradioactivity of each pixel in the sections and scale the valuesto allow maximum use of the grey scale. Starting from coronallysectioned material the system has been used to analyse and reconstructrat nasal turbinates. In two dimensions horizontal and sagittalsections have been obtained while in three dimensions back-to-frontand surface-rendered images have been constructed. Useful renderingof differential metabolic activity within an organ of complexgeometry has been obtained, and there appears to be no reasonwhy the system cannot be used for any material for which serialsectioning is appropriate. Received on November 29, 1989; accepted on February 28, 1990     

Application of an image-based weighted measure of skeletal bending stiffness to great ape mandibles

Traditional measures of structural stiffness in the primate skeleton do not consider the heterogeneous material stiffness distribution of bone. This assumption of homogeneity introduces an unknown degree of error in estimating stiffness in skeletal elements. Measures of weighted stiffness can be developed by including heterogeneous grayscale variations evident in computed tomographic (CT) images. Since gray scale correlates with material stiffness, the distribution of bone quality and quantity can be simultaneously considered. We developed weighted measures of bending resistance and applied these to CT images at three locations along the mandibular corpus in the hominoids Gorilla, Pongo, and Pan. We calculated the traditional (unweighted) moment of inertia for comparison to our weighted measure, which weighs each pixel by its gray-scale value. This weighing results in assignment of reduced moment of inertia values to sections of reduced density. Our weighted and unweighted moments differ by up to 22%. These differences are not consistent among sections, however, such that they cannot be calculated by simple correction of unweighted moments. The effect of this result is that the rank ordering of individual sections within species changes if weighted moments are considered. These results suggest that the use of weighted moments may spur different interpretations of comparative data sets that rely on stiffness measures as estimates of biomechanical competence.     

Three-dimensional computer reconstructions from serial sections of cell nuclei

The analysis of ultrathin serial sections as 3-dimensional (3D) information requires interpretation and display of a large amount of data. We suggest a simple way to solve this problem; it permits presentation of a series of sections as a 3D color image of good quality. It involves a picture system with specialized hardware and software written for this purpose. 3D images of cellular organelles have been drawn either by manually defining the contour of the objects or by thresholding of the volumes in the structures. These 2 methods allow rapid drawing of the image on the screen. It is possible to determine the position, shape and size of 3D structures. This interactive system allows the user to choose between several options: colors, removal of parts of the object, and cutout.     

The interactive presentation of 3D information obtained from reconstructed datasets and 3D placement of single histological sections with the 3D portable document format

Interpretation of the results of anatomical and embryological studies relies heavily on proper visualization of complex morphogenetic processes and patterns of gene expression in a three-dimensional (3D) context. However, reconstruction of complete 3D datasets is time consuming and often researchers study only a few sections. To help in understanding the resulting 2D data we developed a program (TRACTS) that places such arbitrary histological sections into a high-resolution 3D model of the developing heart. The program places sections correctly, robustly and as precisely as the best of the fits achieved by five morphology experts. Dissemination of 3D data is severely hampered by the 2D medium of print publication. Many insights gained from studying the 3D object are very hard to convey using 2D images and are consequently lost or cannot be verified independently. It is possible to embed 3D objects into a pdf document, which is a format widely used for the distribution of scientific papers. Using the freeware program Adobe Reader to interact with these 3D objects is reasonably straightforward; creating such objects is not. We have developed a protocol that describes, step by step, how 3D objects can be embedded into a pdf document. Both the use of TRACTS and the inclusion of 3D objects in pdf documents can help in the interpretation of 2D and 3D data, and will thus optimize communication on morphological issues in developmental biology.     

Variation in human hair ultrastructure among three biogeographic populations

Human scalp hairs are often examined microscopically to study the variation and diversity among a range of visible morphological traits. In this study, we focused on the ultrastructure of human scalp hair within its keratinized matrix, emphasizing, the density and distribution of melanosomes, variation in cuticle thickness within populations, and the relationship of hair fiber ultrastructure with biogeographic ancestry. We used transmission electron microscopy (TEM) to visualize hair cross-sections and generate micron-scale resolution images for analysis of particle morphology and the layered hair matrix. Our results revealed considerable variation in all parameters examined, including the relationship of ultrastructure to biogeographic ancestry. Among the three metapopulations studied (European, African, and East Asian), we identified hair cross-sectional shape, cuticle dimensions, and melanosome distribution as traits that reveal statistically significant ancestry-related patterns. This study establishes trait patterns in hair morphology and ultrastructure among three biogeographically defined metapopulations to improve the current understanding of human variation in hair form and establish a foundation for future studies on the genetic and developmental bases of phenotypic variation in hair ultrastructure related to genotype.     

Nitric oxide spatial distribution in single cultured hippocampus neurons: investigation by projection of reconstructed 3-D image and visualization technique

Recent studies have revealed a non-homogeneous distribution of nitric oxide synthase (NOS) in neurons. However, it is not yet clear whether the intracellular distribution of NOS represents the intracellular nitric oxide (NO) distribution. In the present study, software developed in our laboratory was applied to the reconstructed image obtained from confocal slice images in order to project the 3-D reconstructed images in any direction and to cut the neuron in different sections. This enabled the spatial distribution of NO to be visualized in any direction and section. In single neurons, NO distribution was seen to be heterogeneous. After stimulation with glutamate, the spatial changes in different areas of the neuron were different. These findings are consistent with immunocytochemical data on the intracellular localization of nNOS in hippocampus neurons, and will help to elucidate the specificity of nitric oxide signaling. Finally, the administration of SNAP and L-NAME was used to examine DAF-2 distribution in the neurons. The results showed this distribution to be homogenous; therefore, it did not account for the NO distribution results.     

An ImageJ-based tool for three-dimensional registration between different types of microscopic images

Three-dimensional (3D) registration (i.e., alignment) between two microscopic images is very helpful to study tissues that do not adhere to substrates, such as mouse embryos and organoids, which are often 3D rotated during imaging. However, there is no 3D registration tool easily accessible for experimental biologists. Here we developed an ImageJ-based tool which allows for 3D registration accompanied with both quantitative evaluation of the accuracy and reconstruction of 3D rotated images. In this tool, several landmarks are manually provided in two images to be aligned, and 3D rotation is computed so that the distances between the paired landmarks from the two images are minimized. By simultaneously providing multiple points (e.g., all nuclei in the regions of interest) other than the landmarks in the two images, the correspondence of each point between the two images, i.e., to which nucleus in one image a certain nucleus in another image corresponds, is quantitatively explored. Furthermore, 3D rotation is applied to one of the two images, resulting in reconstruction of 3D rotated images. We demonstrated that this tool successfully achieved 3D registration and reconstruction of images in mouse pre- and post-implantation embryos, where one image was obtained during live imaging and another image was obtained from fixed embryos after live imaging. This approach provides a versatile tool applicable for various tissues and species.     

The effect of overabundant projection directions on 3D reconstruction algorithms

The experimental process of collecting images from macromolecules in an electron microscope is such that it does not allow for prior specification of the angular distribution of the projection images. As a consequence, an uneven distribution of projection directions may occur. Concerns have been raised recently about the behavior of 3D reconstruction algorithms for the case of unevenly distributed projections. It has been illustrated on experimental data that in the case of a heavily uneven distribution of projection directions some algorithms tend to elongate the reconstructed volumes along the overloaded direction so much as to make a quantitative biological analysis impossible. In answer to these concerns we have developed a strategy for quantitative comparison and optimization of 3D reconstruction algorithms. We apply this strategy to quantitatively analyze algebraic reconstruction techniques (ART) with blobs, simultaneous iterative reconstruction techniques (SIRT) with voxels, and weighted backprojection (WBP). We show that the elongation artifacts that had been previously reported can be strongly reduced. With our specific choices for the free parameters of the three algorithms, WBP reconstructions tend to be inferior to those obtained with either SIRT or ART and the results obtained with ART are comparable to those with SIRT, but at a very small fraction of the computational cost of SIRT.     

Spatial distribution of oil in groundnut and sunflower seeds by nuclear magnetic resonance imaging

Abstract. The nuclear magnetic resonance imaging technique has been used to obtain images of different transverse and vertical sections in groundnut and sunflower seeds. Separate images have been obtained for oil and water components in the seeds. The spatial distribution of oil and water inside the seed has been obtained from the detailed analysis of the images. In the immature groundnut seeds obtained commercially, complementary oil and water distributions have been observed. Attempts have been made to explain these results.     

Visualization of the left extraperitoneal space and spatial relationships to its related spaces by the visible human project

Background

The major hindrance to multidetector CT imaging of the left extraperitoneal space (LES), and the detailed spatial relationships to its related spaces, is that there is no obvious density difference between them. Traditional gross anatomy and thick-slice sectional anatomy imagery are also insufficient to show the anatomic features of this narrow space in three-dimensions (3D). To overcome these obstacles, we used a new method to visualize the anatomic features of the LES and its spatial associations with related spaces, in random sections and in 3D.

Methods

In conjunction with Mimics® and Amira® software, we used thin-slice cross-sectional images of the upper abdomen, retrieved from the Chinese and American Visible Human dataset and the Chinese Virtual Human dataset, to display anatomic features of the LES and spatial relationships of the LES to its related spaces, especially the gastric bare area. The anatomic location of the LES was presented on 3D sections reconstructed from CVH2 images and CT images.

Principal Findings

What calls for special attention of our results is the LES consists of the left sub-diaphragmatic fat space and gastric bare area. The appearance of the fat pad at the cardiac notch contributes to converting the shape of the anteroexternal surface of the LES from triangular to trapezoidal. Moreover, the LES is adjacent to the lesser omentum and the hepatic bare area in the anterointernal and right rear direction, respectively.

Conclusion

The LES and its related spaces were imaged in 3D using visualization technique for the first time. This technique is a promising new method for exploring detailed communication relationships among other abdominal spaces, and will promote research on the dynamic extension of abdominal diseases, such as acute pancreatitis and intra-abdominal carcinomatosis.     

Extended-Depth 3D Super-Resolution Imaging Using Probe-Refresh STORM

Single-molecule localization microscopy methods for super-resolution fluorescence microscopy such as STORM (stochastic optical reconstruction microscopy) are generally limited to thin three-dimensional (3D) sections (≤600 nm) because of photobleaching of molecules outside the focal plane. Although multiple focal planes may be imaged before photobleaching by focusing progressively deeper within the sample, image quality is compromised in this approach because the total number of measurable localizations is divided between detection planes. Here, we solve this problem on fixed samples by developing an imaging method that we call probe-refresh STORM (prSTORM), which allows bleached fluorophores to be straightforwardly replaced with nonbleached fluorophores. We accomplish this by immunostaining the sample with DNA-conjugated antibodies and then reading out their distribution using fluorescently-labeled DNA-reporter oligonucleotides that can be fully replaced in successive rounds of imaging. We demonstrate that prSTORM can acquire 3D images over extended depths without sacrificing the density of localizations at any given plane. We also show that prSTORM can be adapted to obtain high-quality, 3D multichannel images with extended depth that would be challenging or impossible to achieve using established probe methods.     

Reconstruction of 3-Dimensional Histology Volume and its Application to Study Mouse Mammary Glands

Histology volume reconstruction facilitates the study of 3D shape and volume change of an organ at the level of macrostructures made up of cells. It can also be used to investigate and validate novel techniques and algorithms in volumetric medical imaging and therapies. Creating 3D high-resolution atlases of different organs^1,2,3 is another application of histology volume reconstruction. This provides a resource for investigating tissue structures and the spatial relationship between various cellular features. We present an image registration approach for histology volume reconstruction, which uses a set of optical blockface images. The reconstructed histology volume represents a reliable shape of the processed specimen with no propagated post-processing registration error. The Hematoxylin and Eosin (H&E) stained sections of two mouse mammary glands were registered to their corresponding blockface images using boundary points extracted from the edges of the specimen in histology and blockface images. The accuracy of the registration was visually evaluated. The alignment of the macrostructures of the mammary glands was also visually assessed at high resolution.This study delineates the different steps of this image registration pipeline, ranging from excision of the mammary gland through to 3D histology volume reconstruction. While 2D histology images reveal the structural differences between pairs of sections, 3D histology volume provides the ability to visualize the differences in shape and volume of the mammary glands.