A Method for 3D Histopathology Reconstruction Supporting Mouse Microvasculature Analysis

Structural abnormalities of the microvasculature can impair perfusion and function. Conventional histology provides good spatial resolution with which to evaluate the microvascular structure but affords no 3-dimensional information; this limitation could lead to misinterpretations of the complex microvessel network in health and disease. The objective of this study was to develop and evaluate an accurate, fully automated 3D histology reconstruction method to visualize the arterioles and venules within the mouse hind-limb. Sections of the tibialis anterior muscle from C57BL/J6 mice (both normal and subjected to femoral artery excision) were reconstructed using pairwise rigid and affine registrations of 5 µm-thick, paraffin-embedded serial sections digitized at 0.25 µm/pixel. Low-resolution intensity-based rigid registration was used to initialize the nucleus landmark-based registration, and conventional high-resolution intensity-based registration method. The affine nucleus landmark-based registration was developed in this work and was compared to the conventional affine high-resolution intensity-based registration method. Target registration errors were measured between adjacent tissue sections (pairwise error), as well as with respect to a 3D reference reconstruction (accumulated error, to capture propagation of error through the stack of sections). Accumulated error measures were lower (p<0.01) for the nucleus landmark technique and superior vasculature continuity was observed. These findings indicate that registration based on automatic extraction and correspondence of small, homologous landmarks may support accurate 3D histology reconstruction. This technique avoids the otherwise problematic “banana-into-cylinder” effect observed using conventional methods that optimize the pairwise alignment of salient structures, forcing them to be section-orthogonal. This approach will provide a valuable tool for high-accuracy 3D histology tissue reconstructions for analysis of diseased microvasculature.     

A Reference-Free Method for Brightness Compensation and Contrast Enhancement of Micrographs of Serial Sections

Three-dimensional (3D) reconstruction of an organ or tissue from a stack of histologic serial sections provides valuable morphological information. The procedure includes section preparation of the organ or tissue, micrographs acquisition, image registration, 3D reconstruction, and visualization. However, the brightness and contrast through the image stack may not be consistent due to imperfections in the staining procedure, which may cause difficulties in micro-structure identification using virtual sections, region segmentation, automatic target tracing, etc. In the present study, a reference-free method, Sequential Histogram Fitting Algorithm (SHFA), is therefore developed for adjusting the severe and irregular variance of brightness and contrast within the image stack. To apply the SHFA, the gray value histograms of individual images are first calculated over the entire image stack and a set of landmark gray values are chosen. Then the histograms are transformed so that there are no abrupt changes in progressing through the stack. Finally, the pixel gray values of the original images are transformed into the desired ones based on the relationship between the original and the transformed histograms. The SHFA is tested on an image stacks from mouse kidney sections stained with toluidine blue, and captured by a slide scanner. As results, the images through the entire stack reveal homogenous brightness and consistent contrast. In addition, subtle color differences in the tissue are well preserved so that the morphological details can be recognized, even in virtual sections. In conclusion, compared with the existing histogram-based methods, the present study provides a practical method suitable for compensating brightness, and improving contrast of images derived from a large number of serial sections of biological organ.     

Facilitating tumor functional assessment by spatially relating 3D tumor histology and in vivo MRI: image registration approach

Background

Magnetic resonance imaging (MRI), together with histology, is widely used to diagnose and to monitor treatment in oncology. Spatial correspondence between these modalities provides information about the ability of MRI to characterize cancerous tissue. However, registration is complicated by deformations during pathological processing, and differences in scale and information content.

Methodology/Principal Findings

This study proposes a methodology for establishing an accurate 3D relation between histological sections and high resolution in vivo MRI tumor data. The key features of the methodology are: 1) standardized acquisition and processing, 2) use of an intermediate ex vivo MRI, 3) use of a reference cutting plane, 4) dense histological sampling, 5) elastic registration, and 6) use of complete 3D data sets. Five rat pancreatic tumors imaged by T2*-w MRI were used to evaluate the proposed methodology. The registration accuracy was assessed by root mean squared (RMS) distances between manually annotated landmark points in both modalities. After elastic registration the average RMS distance decreased from 1.4 to 0.7 mm. The intermediate ex vivo MRI and the reference cutting plane shared by all three 3D images (in vivo MRI, ex vivo MRI, and 3D histology data) were found to be crucial for the accurate co-registration between the 3D histological data set and in vivo MRI. The MR intensity in necrotic regions, as manually annotated in 3D histology, was significantly different from other histologically confirmed regions (i.e., viable and hemorrhagic). However, the viable and the hemorrhagic regions showed a large overlap in T2^*-w MRI signal intensity.

Conclusions

The established 3D correspondence between tumor histology and in vivo MRI enables extraction of MRI characteristics for histologically confirmed regions. The proposed methodology allows the creation of a tumor database of spatially registered multi-spectral MR images and multi-stained 3D histology.     

Constructing complex 3D biological environments from medical imaging using high performance computing

Extracting information about the structure of biological tissue from static image data is a complex task requiring computationally intensive operations. Here, we present how multicore CPUs and GPUs have been utilized to extract information about the shape, size, and path followed by the mammalian oviduct, called the fallopian tube in humans, from histology images, to create a unique but realistic 3D virtual organ. Histology images were processed to identify the individual cross sections and determine the 3D path that the tube follows through the tissue. This information was then related back to the histology images, linking the 2D cross sections with their corresponding 3D position along the oviduct. A series of linear 2D spline cross sections, which were computationally generated for the length of the oviduct, were bound to the 3D path of the tube using a novel particle system technique that provides smooth resolution of self-intersections. This results in a unique 3D model of the oviduct, which is grounded in reality. The GPU is used for the processor intensive operations of image processing and particle physics based simulations, significantly reducing the time required to generate a complete model.     

An ImageJ-based tool for three-dimensional registration between different types of microscopic images

Three-dimensional (3D) registration (i.e., alignment) between two microscopic images is very helpful to study tissues that do not adhere to substrates, such as mouse embryos and organoids, which are often 3D rotated during imaging. However, there is no 3D registration tool easily accessible for experimental biologists. Here we developed an ImageJ-based tool which allows for 3D registration accompanied with both quantitative evaluation of the accuracy and reconstruction of 3D rotated images. In this tool, several landmarks are manually provided in two images to be aligned, and 3D rotation is computed so that the distances between the paired landmarks from the two images are minimized. By simultaneously providing multiple points (e.g., all nuclei in the regions of interest) other than the landmarks in the two images, the correspondence of each point between the two images, i.e., to which nucleus in one image a certain nucleus in another image corresponds, is quantitatively explored. Furthermore, 3D rotation is applied to one of the two images, resulting in reconstruction of 3D rotated images. We demonstrated that this tool successfully achieved 3D registration and reconstruction of images in mouse pre- and post-implantation embryos, where one image was obtained during live imaging and another image was obtained from fixed embryos after live imaging. This approach provides a versatile tool applicable for various tissues and species.     

JULIDE: a software tool for 3D reconstruction and statistical analysis of autoradiographic mouse brain sections

In this article we introduce JULIDE, a software toolkit developed to perform the 3D reconstruction, intensity normalization, volume standardization by 3D image registration and voxel-wise statistical analysis of autoradiographs of mouse brain sections. This software tool has been developed in the open-source ITK software framework and is freely available under a GPL license. The article presents the complete image processing chain from raw data acquisition to 3D statistical group analysis. Results of the group comparison in the context of a study on spatial learning are shown as an illustration of the data that can be obtained with this tool.     

A methodology for registration of a histological slide and in vivo MRI volume based on optimizing mutual information

We present a method for registering histology and in vivo imaging that requires minimal microtoming and is automatic following the user's initialization. In this demonstration, we register a single hematoxylin-and-eosin-stained histological slide of a coronal section of a rat brain harboring a 9L gliosarcoma with an in vivo 7T MR image volume of the same brain. Because the spatial resolution of the in vivo MRI is limited, we add the step of obtaining a high spatial resolution, ex vivo MRI in situ for intermediate registration. The approach taken was to maximize mutual information in order to optimize the registration between all pairings of image data whether the sources are MRI, tissue block photograph, or stained sample photograph. The warping interpolant used was thin plate splines with the appropriate basis function for either 2-D or 3-D applications. All registrations were implemented by user initialization of the approximate pose between the two data sets, followed by automatic optimization based on maximizing mutual information. Only the higher quality anatomical images were used in the registration process; however, the spatial transformation was directly applied to a quantitative diffusion image. Quantitative diffusion maps from the registered location appeared highly correlated with the H&E slide. Overall, this approach provides a robust method for coregistration of in vivo images with histological sections and will have broad applications in the field of functional and molecular imaging.     

Focus gradient correction applied to tilt series image data used in electron tomography

The resolution in 3D reconstructions from tilt series is limited to the information below the first zero of the contrast transfer function unless the signal is corrected computationally. The restoration is usually based on the assumption of a linear space-invariant system and a linear relationship between object mass density and observed image contrast. The space-invariant model is no longer valid when applied to tilted micrographs because the defocus varies in a direction perpendicular to the tilt axis and with it the shape of the associated point spread function. In this paper, a method is presented for determining the defocus gradient in thin specimens such as sections and 2D crystals, and for restoration of the images subsequently used for 3D reconstruction. The alignment procedure for 3D reconstruction includes area matching and tilt geometry refinement. A map with limited resolution computed from uncorrected micrographs is compared to a volume computed from corrected micrographs with extended resolution.     

Animated visualization of a high resolution color three dimensional digital computer model of the whole human head

The interactive visualization of animated images through a computerized three dimensional (3D) full color model of an unstained cadaveric human head is presented. Serial full color images were taken of the blockface of a cryomicrotomed frozen human head every 200 μm. From this series of images a three dimensional digital model with a resultant pixel resolution of 200 μm³ was created on a UNIX workstation. Using this database, resampled images were computed along orthogonal axes and written sequentially to a write-once-read-many times (WORM) videodisc unit. Playback of this customized videodisc dataset provides animations of the digitally reconstructed slices and 3D reconstructed surface models. An interactive interface to the animated sequences is provided through a PC based tutorial package. This tutorial program is able to access videodisc frames to display animations and labeled still images in a software window to illustrate various neuroanatomic topics. The technique of animation as applied to this high resolution 3D model provides insight into complex spatial relationships and has great potential in research and as a teaching tool in the neurosciences.     

Three-dimensional reconstruction by confocal laser scanning microscopy in routine pathologic specimens of benign and malignant lesions of the human breast

 Confocal laser scanning microscopy (CLSM) has become an exciting new instrument because of its increased resolution over conventional wide-field microscopy and its high performance three-dimensional (3D) optical sectioning. Although CLSM has been used extensively in cell biology, few applications have been reported in routine clinical pathology. In this study, 3D reconstruction was performed on routine formalin-fixed, paraffin-embedded tissues of normal mammary duct, simple ductal hyperplasia, intraductal papillary hyperplasia, ductal carcinoma in situ, invasive carcinoma, and lymph node metastatic carcinomas of the human breast by using computer-assisted CLSM in conjunction with a 3D reconstruction software package (microVoxel). The selected specimens were sectioned at 30 μm, mounted on glass slides, and stained with the DNA fluorescent probe, YOYO-1 iodide. The nuclear DNA and chromatin texture were clearly demonstrated after pretreatment with RNAase and hydrolysis with 2 N HCl. High quality 3D images were obtained by processing the optical section stacks with volume render and surface display parameters in microVoxel. 3D morphologic characteristics of different breast lesions were examined in various orientations by angular image rotation. The clearly benign lesions (simple ductal hyperplasia and intraductal papillary hyperplasia) revealed similar 3D morphologic features, including: (1) smooth nuclear surface and homogeneous chromatin fluorescence intensity; (2) hyperplastic cell nuclei showing similar shape and volume; and (3) clear-cut margin of basement membrane defined by spindle-shaped myocytes of the ductal outer layer. In contrast, carcinomas displayed remarkably different features in 3D morphology, including: (1) irregular nuclear surface; (2) marked nuclear pleomorphism (irregular, angulated and indented shape of nuclear volume); (3) irregular and coarse chromatin texture; (4) chaotic arrangement of tumor cell nuclei; and (5) absence of myocytes, indicating no clear margin at the site of infiltration of cancer cells. In conclusion, nuclear structure, specifically demonstrated by CLSM of YOYO-1 iodide fluorescently stained cells, used in tandem with 3D volume morphologic reconstruction, may provide a useful research diagnostic tool in pathology. Accepted: 7 November 1996     

Structure of HIV-1 Capsid Assemblies by Cryo-electron Microscopy and Iterative Helical Real-space Reconstruction

Cryo-electron microscopy (cryo-EM), combined with image processing, is an increasingly powerful tool for structure determination of macromolecular protein complexes and assemblies. In fact, single particle electron microscopy¹ and two-dimensional (2D) electron crystallography² have become relatively routine methodologies and a large number of structures have been solved using these methods. At the same time, image processing and three-dimensional (3D) reconstruction of helical objects has rapidly developed, especially, the iterative helical real-space reconstruction (IHRSR) method³, which uses single particle analysis tools in conjunction with helical symmetry. Many biological entities function in filamentous or helical forms, including actin filaments⁴, microtubules⁵, amyloid fibers⁶, tobacco mosaic viruses⁷, and bacteria flagella⁸, and, because a 3D density map of a helical entity can be attained from a single projection image, compared to the many images required for 3D reconstruction of a non-helical object, with the IHRSR method, structural analysis of such flexible and disordered helical assemblies is now attainable.In this video article, we provide detailed protocols for obtaining a 3D density map of a helical protein assembly (HIV-1 capsid⁹ is our example), including protocols for cryo-EM specimen preparation, low dose data collection by cryo-EM, indexing of helical diffraction patterns, and image processing and 3D reconstruction using IHRSR. Compared to other techniques, cryo-EM offers optimal specimen preservation under near native conditions. Samples are embedded in a thin layer of vitreous ice, by rapid freezing, and imaged in electron microscopes at liquid nitrogen temperature, under low dose conditions to minimize the radiation damage. Sample images are obtained under near native conditions at the expense of low signal and low contrast in the recorded micrographs. Fortunately, the process of helical reconstruction has largely been automated, with the exception of indexing the helical diffraction pattern. Here, we describe an approach to index helical structure and determine helical symmetries (helical parameters) from digitized micrographs, an essential step for 3D helical reconstruction. Briefly, we obtain an initial 3D density map by applying the IHRSR method. This initial map is then iteratively refined by introducing constraints for the alignment parameters of each segment, thus controlling their degrees of freedom. Further improvement is achieved by correcting for the contrast transfer function (CTF) of the electron microscope (amplitude and phase correction) and by optimizing the helical symmetry of the assembly.     

基于图像重建的CT三维重建提升腹部增强扫描图像质量的价值

摘要 目的：探讨基于图像重建的电子计算机断层扫描仪器(Computed Tomography,CT)三维成像提升腹部增强扫描图像质量的价值。方法：2019年11月到2020年10月选择在本院进行腹部CT增强扫描的患者76例作为研究对象,采用电脑随机数字法将研究对象分为对照组和重建组各38例,对照组给予常规扫描成像,重建组给予基于自适应统计迭代重建(adaptive statistical iterative reconstruction,ASIR)的CT三维成像,记录两组成像质量与噪声情况。结果：两名医师对重建组的图像主观质量评分都高于对照组(P<0.05)。重建组的图像相对细腻柔和,能清晰显示图像细小血管断面,末梢血管显示良好,血管壁光滑柔和。重建组的动脉期、门静脉期、平衡期的肝脏CT值高于对照组(P<0.05),动脉期、门静脉期、平衡期的肝脏、胰腺对比噪声比(contrast to noise ratio,CNR)值低于对照组(P<0.05)。重建组的容积剂量指数(volume CT dose index,CTDIvol)和剂量长度乘积(Dose-Length product,DLP)、有效剂量(effective dose,ED)值都低于对照组(P<0.05)。结论：基于图像重建的CT三维成像能提升腹部增强扫描主客观图像质量,降低图像噪声,更利于腹部疾病的显示,从而提高正确诊断率。     

CT-based semi-automatic quantification of vertebral fracture restoration

Minimally invasive surgeries aiming to restore fractured vertebral body are increasing; therefore, our goals were to create a 3D vertebra reconstruction process and design clinical indices to assess the vertebral restoration in terms of heights, angles and volumes. Based on computed tomography (CT)-scan of the vertebral spine, a 3D reconstruction method as well as relevant clinical indices were developed. First, a vertebra initial solution requiring 5 min of manual adjustments is built. Then an image processing algorithm places this solution in the CT-scan images volume to adjust the model's nodes. On the vertebral body's anterior and posterior parts, nine robust heights, volume and endplate angle measurement methods were developed. These parameters were evaluated by reproducibility and accuracy studies. The vertebral body reconstruction accuracy was 1.0 mm; heights and volume accuracy were, respectively, 1.2 and 179 mm³. In conclusion, a 3D vertebra reconstruction process requiring little user time was proposed as well as 3D clinical indices assessing fractured and restored vertebra.     

DSA图像中运动伪影的消除方法

由于病人存在着各种运动(如呼吸、肌肉运动、心脏运动、设备噪声),在成像过程中常会造成图像上出现伪影,干扰医生的正常诊断,为消除这种伪影,本文提出一种基于图像配准思想的全自动消除伪影的方法,该方法能够自动消除DSA图像中的大部分运动伪影,使DSA图像得到较好的增强,并为后面的血管分割和三维重建提供便利,是一种快速有效的方法。     

Segmentation of parotid glands from registered CT and MR images

PurposeTo develop an automatic multimodal method for segmentation of parotid glands (PGs) from pre-registered computed tomography (CT) and magnetic resonance (MR) images and compare its results to the results of an existing state-of-the-art algorithm that segments PGs from CT images only.MethodsMagnetic resonance images of head and neck were registered to the accompanying CT images using two different state-of-the-art registration procedures. The reference domains of registered image pairs were divided on the complementary PG regions and backgrounds according to the manual delineation of PGs on CT images, provided by a physician. Patches of intensity values from both image modalities, centered around randomly sampled voxels from the reference domain, served as positive or negative samples in the training of the convolutional neural network (CNN) classifier. The trained CNN accepted a previously unseen (registered) image pair and classified its voxels according to the resemblance of its patches to the patches used for training. The final segmentation was refined using a graph-cut algorithm, followed by the dilate-erode operations.ResultsUsing the same image dataset, segmentation of PGs was performed using the proposed multimodal algorithm and an existing monomodal algorithm, which segments PGs from CT images only. The mean value of the achieved Dice overlapping coefficient for the proposed algorithm was 78.8%, while the corresponding mean value for the monomodal algorithm was 76.5%.ConclusionsAutomatic PG segmentation on the planning CT image can be augmented with the MR image modality, leading to an improved RT planning of head and neck cancer.     

Combined method of whole mount and block-face imaging: Acquisition of 3D data of gene expression pattern from conventional in situ hybridization

Visualization of spatiotemporal expression of a gene of interest is a fundamental technique for analyzing the involvements of genes in organ development. In situ hybridization (ISH) is one of the most popular methods for visualizing gene expression. When conventional ISH is performed on sections or whole-mount specimens, the gene expression pattern is represented in 2-dimensional (2D) microscopic images or in the surface view of the specimen. To obtain 3-dimensional (3D) data of gene expression from conventional ISH, the “serial section method” has traditionally been employed. However, this method requires an extensive amount of time and labor because it requires researchers to collect a tremendous number of sections, label all sections by ISH, and image them before 3D reconstruction. Here, we proposed a rapid and low-cost 3D imaging method that can create 3D gene expression patterns from conventional ISH-labeled specimens. Our method consists of a combination of whole-mount ISH and Correlative Microscopy and Blockface imaging (CoMBI). The whole-mount ISH-labeled specimens were sliced using a microtome or cryostat, and all block-faces were imaged and used to reconstruct 3D images by CoMBI. The 3D data acquired using our method showed sufficient quality to analyze the morphology and gene expression patterns in the developing mouse heart. In addition, 2D microscopic images of the sections can be obtained when needed. Correlating 2D microscopic images and 3D data can help annotate gene expression patterns and understand the anatomy of developing organs. These results indicated that our method can be useful in the field of developmental biology.     

UCSF tomography: an integrated software suite for real-time electron microscopic tomographic data collection, alignment, and reconstruction

A real-time alignment and reconstruction scheme for electron microscopic tomography (EMT) has been developed and integrated within our UCSF tomography data collection software. This newly integrated software suite provides full automation from data collection to real-time reconstruction by which the three-dimensional (3D) reconstructed volume is immediately made available at the end of each data collection. Real-time reconstruction is achieved by calculating a weighted back-projection on a small Linux cluster (five dual-processor compute nodes) concurrently with the UCSF tomography data collection running on the microscope's computer, and using the fiducial-marker free alignment data generated during the data collection process. The real-time reconstructed 3D volume provides users with immediate feedback to fully asses all aspects of the experiment ranging from sample choice, ice thickness, experimental parameters to the quality of specimen preparation. This information can be used to guide subsequent data collections. Access to the reconstruction is especially useful in low-dose cryo EMT where such information is very difficult to obtain due to extraordinary low signal to noise ratio in each 2D image. In our environment, we generally collect 2048 x 2048 pixel images which are subsequently computationally binned four-fold for the on-line reconstruction. Based upon experiments performed with thick and cryo specimens at various CCD magnifications (50000x-80000x), alignment accuracy is sufficient to support this reduced resolution but should be refined before calculating a full resolution reconstruction. The reduced resolution has proven to be quite adequate to assess sample quality, or to screen for the best data set for full-resolution reconstruction, significantly improving both productivity and efficiency of system resources. The total time from start of data collection to a final reconstructed volume (512 x 512 x 256 pixels) is about 50 min for a +/-70 degrees 2k x 2k pixel tilt series acquired at every 1 degrees.     

3D identification of trabecular bone fracture zone using an automatic image registration scheme: A validation study

Accurate identification of the local fracture zone is an important step towards the failure assessment of trabecular bone. In previous in-vitro studies, local fracture zones were visually identified in micro-CT images by experienced observers. This is a time-consuming and observer-dependent approach and it prevents any large-scale analysis of local trabecular fracture regions. The scope of this study is the application and validation of a new registration scheme for the automatic identification of trabecular bone fracture zones. Six human trabecular specimens were extracted from different anatomical sites. Five specimens were mechanically tested and scanned using micro-CT. For each specimen pre- and post-failure micro-CT datasets were obtained. The sixth specimen was scanned twice without any mechanical compression and was used to test the accuracy of the proposed scheme. The registration scheme was applied to the acquired datasets for the automatic identification of the fracture zone. The proposed scheme comprises of a three-dimensional (3D) automatic registration method to define the differences between the two datasets, and the application of a criterion for defining slices of the pre-failure dataset as "broken" or "unbroken". Identifications of the fracture zones were qualitatively validated against visual identification of observers. Furthermore, "full 3D" fracture zone identification, based on the presented scheme, was proposed. The proposed scheme proved to be more accurate and significantly faster than the currently used visual process.     

One‐to‐one registration of en‐face optical coherence tomography attenuation coefficients with histology of a prostatectomy specimen

Optical coherence tomography (OCT), enables high‐resolution 3D imaging of the morphology of light scattering tissues. From the OCT signal, parameters can be extracted and related to tissue structures. One of the quantitative parameters is the attenuation coefficient; the rate at which the intensity of detected light decays in depth. To couple the quantitative parameters with the histology one‐to‐one registration is needed. The primary aim of this study is to validate a registration method of quantitative OCT parameters to histological tissue outcome through one‐to‐one registration of OCT with histology. We matched OCT images of unstained fixated prostate tissue slices with corresponding histology slides, wherein different histologic types were demarcated. Attenuation coefficients were determined by a supervised automated exponential fit (corrected for point spread function and sensitivity roll‐off related signal losses) over a depth of 0.32 mm starting from 0.10 mm below the automatically detected tissue edge. Finally, the attenuation coefficients corresponding to the different tissue types of the prostate were compared. From the attenuation coefficients, we produced the squared relative residue and goodness‐of‐fit metric R². This article explains the method to perform supervised automated quantitative analysis of OCT data, and the one‐to‐one registration of OCT extracted quantitative data with histopathological outcomes.