Automatic alignment of transmission electron microscope tilt series without fiducial markers

Accurate image alignment is needed for computing three-dimensional reconstructions from transmission electron microscope tilt series. So far, the best results have been obtained by using colloidal gold beads as fiducial markers. If their use has not been possible for some reason, the only option has been the automatic cross-correlation-based registration methods. However, the latter methods are inaccurate and, as we will show, inappropriate for the whole problem. Conversely, we propose a novel method that uses the actual 3D motion model but works without any fiducial markers in the images. The method is based on matching and tracking some interest points of the intensity surface by first solving the underlying geometrical constraint of consecutive images in the tilt series. The results show that our method is near the gold marker alignment in the level of accuracy and hence opens the way for new opportunities in the analysis of electron tomography reconstructions, especially when markers cannot be used.     

Three-dimensional elemental mapping of phosphorus by quantitative electron spectroscopic tomography (QuEST)

We describe the development of quantitative electron spectroscopic tomography (QuEST), which provides 3-D distributions of elements on a nanometer scale. Specifically, it is shown that QuEST can be applied to map the distribution of phosphorus in unstained sections of embedded cells. A series of 2-D elemental maps is derived from images recorded in the energy filtering transmission electron microscope for a range of specimen tilt angles. A quantitative 3-D elemental distribution is then reconstructed from the elemental tilt series. To obtain accurate quantitative elemental distributions it is necessary to correct for plural inelastic scattering at the phosphorus L(2,3) edge, which is achieved by acquiring unfiltered and zero-loss images at each tilt angle. The data are acquired automatically using a cross correlation technique to correct for specimen drift and focus change between successive tilt angles. An algorithm based on the simultaneous iterative reconstruction technique (SIRT) is implemented to obtain quantitative information about the number of phosphorus atoms associated with each voxel in the reconstructed volume. We assess the accuracy of QuEST by determining the phosphorus content of ribosomes in a eukaryotic cell, and then apply it to estimate the density of nucleic acid in chromatin of the cell's nucleus. From our experimental data, we estimate that the sensitivity for detecting phosphorus is 20 atoms in a 2.7 nm-sized voxel.     

Reprint of "Three-dimensional elemental mapping of phosphorus by quantitative electron spectroscopic tomography (QuEST)" [J. Struct. Biol. 160 (2007) 35-48

We describe the development of quantitative electron spectroscopic tomography (QuEST), which provides 3-D distributions of elements on a nanometer scale. Specifically, it is shown that QuEST can be applied to map the distribution of phosphorus in unstained sections of embedded cells. A series of 2-D elemental maps is derived from images recorded in the energy filtering transmission electron microscope for a range of specimen tilt angles. A quantitative 3-D elemental distribution is then reconstructed from the elemental tilt series. To obtain accurate quantitative elemental distributions it is necessary to correct for plural inelastic scattering at the phosphorus L_2,3 edge, which is achieved by acquiring unfiltered and zero-loss images at each tilt angle. The data are acquired automatically using a cross correlation technique to correct for specimen drift and focus change between successive tilt angles. An algorithm based on the simultaneous iterative reconstruction technique (SIRT) is implemented to obtain quantitative information about the number of phosphorus atoms associated with each voxel in the reconstructed volume. We assess the accuracy of QuEST by determining the phosphorus content of ribosomes in a eukaryotic cell, and then apply it to estimate the density of nucleic acid in chromatin of the cell’s nucleus. From our experimental data, we estimate that the sensitivity for detecting phosphorus is 20 atoms in a 2.7 nm-sized voxel.     

A method for determining the crystal orientation for any tilt position when using a transmission electron microscope double-tilt specimen holder

A method is described in this short communiction which provides a rapid and precise determination of the crystal specimen orientation in any tilt position when using a doule-tilt holder in the transmission electron microscope (TEM). It is necessary to establish three pairs of tilt angles (α_i, β_i) (i = 1, 2, 3) where three recognized directions of the crystal specimen are aligned to the electron beam direction respectively, in order to determine the crystal specimen orientation for any tilt position in a double-tilt specimen holder. In addition, the tilt position (α, β), where any orientation [ u v w] is aligned to the beam direction, can be determined with the aid of this method.A software system for computer applications of the above method has beed developed.     

An improved strategy for automated electron microscopic tomography

A prediction-based scheme is proposed and implemented for automated electron microscopic tomography. By assuming that the sample follows a simple geometric rotation and that the optical system can be characterized in terms of an offset between the optical and mechanical axes, it is found that the image movement in the x, y, and z directions due to stage tilt can be dynamically predicted with desired accuracy (15 nm in x-y position and 100 nm in focus). Thus, the microscope optical system (beam/image shift and focus) can be automatically adjusted to compensate for the predicted image movement prior to taking the projected image at each tilt angle. As a consequence, it is not necessary to either record additional images for tracking and focusing during the course of data collections or to spend valuable setup time in a lengthy pre-calibration of stage motions. Furthermore, this scheme is also found to tolerate a significant degree of non-eucentricity and to be quite robust in the collection of regular and cryo low-dose images on thin or thick samples even at magnifications greater than 62000x and angular step as large as 10 degrees. For interested users the software can be freely downloaded for non-profit use at http://www.msg.ucsf.edu/tomography.     

Leginon: an automated system for acquisition of images from vitreous ice specimens

We have developed a system to automatically acquire cryo-electron micrographs. The system is designed to emulate all of the decisions and actions of a highly trained microscopist in collecting data from a vitreous ice specimen. These include identifying suitable areas of vitreous ice at low magnification, determining the presence and location of specimen on the grid, automatically adjusting imaging parameters (focus, astigmatism) under low-dose conditions, and acquiring images at high magnification to either film or a digital camera. This system is responsible for every aspect of image acquisition and can run unattended, other than requiring periodic refilling of the cryogens, for over 24 h. The system has been tested out on a variety of specimens that represent typical challenges in the field of cryo-electron microscopy. The results show that the overall performance of the system is equivalent to that of an experienced microscopist.     

Tilted specimen in the electron microscope: A simple specimen holder and the calculation of tilt angles for crystalline specimens

A simple specimen holder is described for a Siemens electron microscope which will allow the specimen grid to be set at inclinations up to 75° to the electron beam in any azimuthal direction. This device is suitable for measuring tilted images for the three-dimensional reconstruction of crystalline specimens. A method is also described for calculating the tilt angles for such crystalline specimens by comparing the unit cell dimensions in tilted and untilted images.     

Focus gradient correction applied to tilt series image data used in electron tomography

The resolution in 3D reconstructions from tilt series is limited to the information below the first zero of the contrast transfer function unless the signal is corrected computationally. The restoration is usually based on the assumption of a linear space-invariant system and a linear relationship between object mass density and observed image contrast. The space-invariant model is no longer valid when applied to tilted micrographs because the defocus varies in a direction perpendicular to the tilt axis and with it the shape of the associated point spread function. In this paper, a method is presented for determining the defocus gradient in thin specimens such as sections and 2D crystals, and for restoration of the images subsequently used for 3D reconstruction. The alignment procedure for 3D reconstruction includes area matching and tilt geometry refinement. A map with limited resolution computed from uncorrected micrographs is compared to a volume computed from corrected micrographs with extended resolution.     

Multiphase method for automatic alignment of transmission electron microscope images using markers

In order to successfully perform the 3D reconstruction in electron tomography, transmission electron microscope images must be accurately aligned or registered. So far, the problem is solved by either manually showing the corresponding fiducial markers from the set of images or automatically using simple correlation between the images on several rotations and scales. The present solutions, however, share the problem of being inefficient and/or inaccurate. We therefore propose a method in which the registration is automated using conventional colloidal gold particles as reference markers between images. We approach the problem from the computer vision viewpoint; hence, the alignment problem is divided into several subproblems: (1) finding initial matches from successive images, (2) estimating the epipolar geometry between consecutive images, (3) finding and localizing the gold particles with subpixel accuracy in each image, (4) predicting the probable matching gold particles using the epipolar constraint and its uncertainty, (5) matching and tracking the gold beads through the tilt series, and (6) optimizing the transformation parameters for the whole image set. The results show not only the reliability of the suggested method but also a high level of accuracy in alignment, since practically all the visible gold markers can be used.     

Improved specimen preparation for cryo-electron microscopy using a symmetric carbon sandwich technique

Image shift due to beam-induced specimen charging has become the most severe problem in electron microscopy for imaging two-dimensional (2D) crystals of biological macromolecules, especially in the case of highly tilted specimens. Image shift causes diffraction spots perpendicular to the tilt axis to disappear even at medium or low resolution. The yield of good images from tilted specimens prepared on a single layer of continuous carbon support film is therefore very low. In this paper, we have used 2D crystals of aquaporin-4 to investigate the effect of a carbon sandwich preparation method on specimen charging. We find that a larger number of images show sharp diffraction spots perpendicular to the tilt axis if crystals are placed in between two sheets of carbon film as compared to images taken from specimens prepared by the conventional single carbon support film technique. Our results demonstrate that the reproducible carbon sandwich preparation technique overcomes the severe specimen charging problem and thus has the potential to significantly speed up structure analysis by electron crystallography.     

Accurate determination of local defocus and specimen tilt in electron microscopy

Accurate knowledge of defocus and tilt parameters is essential for the determination of three-dimensional protein structures at high resolution using electron microscopy. We present two computer programs, CTFFIND3 and CTFTILT, which determine defocus parameters from images of untilted specimens, as well as defocus and tilt parameters from images of tilted specimens, respectively. Both programs use a simple algorithm that fits the amplitude modulations visible in a power spectrum with a calculated contrast transfer function (CTF). The background present in the power spectrum is calculated using a low-pass filter. The background is then subtracted from the original power spectrum, allowing the fitting of only the oscillatory component of the CTF. CTFTILT determines specimen tilt parameters by measuring the defocus at a series of locations on the image while constraining them to a single plane. We tested the algorithm on images of two-dimensional crystals by comparing the results with those obtained using crystallographic methods. The images also contained contrast from carbon support film that added to the visibility of the CTF oscillations. The tests suggest that the fitting procedure is able to determine the image defocus with an error of about 10nm, whereas tilt axis and tilt angle are determined with an error of about 2 degrees and 1 degrees, respectively. Further tests were performed on images of single protein particles embedded in ice that were recorded from untilted or slightly tilted specimens. The visibility of the CTF oscillations from these images was reduced due to the lack of a carbon support film. Nevertheless, the test results suggest that the fitting procedure is able to determine image defocus and tilt angle with errors of about 100 nm and 6 degrees, respectively.     

Semi-automatic quantitation of dense markers in cytochemistry.

The quantitation of electron dense labelling is very tedious when it is done "by hand". Accordingly we developed software allowing, at electron microscopic level, a semi-automatic counting of dense markers in biological specimens. It includes the digitization of images and extraction of dense particles from the grey level of the background. The definition of the areas of interest was carried out by the observer but all quantitative calculations were done automatically. This method was applied to different biological materials (phospholipid and lysozyme labelling in secretory granules of human submucosal bronchial gland cells). The results obtained by this semi-automatic procedure were in good agreement with those obtained by manual counting of colloidal gold labelling (r = 0.97).     

Automatic liver segmentation in MRI images using an iterative watershed algorithm and artificial neural network

Precise liver segmentation in abdominal MRI images is one of the most important steps for the computer-aided diagnosis of liver pathology. The first and essential step for diagnosis is automatic liver segmentation, and this process remains challenging. Extensive research has examined liver segmentation; however, it is challenging to distinguish which algorithm produces more precise segmentation results that are applicable to various medical imaging techniques. In this paper, we present a new automatic system for liver segmentation in abdominal MRI images. The system includes several successive steps. Preprocessing is applied to enhance the image (edge-preserved noise reduction) by using mathematical morphology. The proposed algorithm for liver region extraction is a combined algorithm that utilizes MLP neural networks and watershed algorithm. The traditional watershed transformation generally results in oversegmentation when directly applied to medical image segmentation. Therefore, we use trained neural networks to extract features of the liver region. The extracted features are used to monitor the quality of the segmentation using the watershed transform and adjust the required parameters automatically. The process of adjusting parameters is performed sequentially in several iterations. The proposed algorithm extracts liver region in one slice of the MRI images and the boundary tracking algorithm is suggested to extract the liver region in other slices, which is left as our future work. This system was applied to a series of test images to extract the liver region. Experimental results showed positive results for the proposed algorithm.     

A cine-EPID based method for jaw detection and quality assurance for tracking jaw in IMRT/VMAT treatments

A new tool with the potential to verify and track jaw position during delivery has been developed. The method should be suitable for independent quality assurance for jaw position during jaw tracking dynamic IMRT and VMAT treatments. The jaw detection and tracking algorithm developed consists of five main steps. Firstly, the image is enhanced by removing a normalised predicted EPID image (that does not include the collimator transmission) from each cine EPID image. Then, using a histogram clustering technique a global intensity threshold level was determined. This threshold level was used to classify each pixel of the image as either under the jaws or under the MLC. Additionally, the collimator angle was automatically detected and used to rotate the image to vertical direction. Finally, this rotation allows the jaw positions to be determined using vertical and horizontal projection profiles. Nine IMRT fields (with static jaws) and a single VMAT clinical field (with dynamic jaws) were tested by determining the root mean square difference between planned and detected jaw positions. The test results give a detection accuracy of ±1 mm RMS error for static jaw IMRT treatments and ±1.5 mm RMS error for the dynamic jaw VMAT treatment. This method is designed for quality assurance and verification in modern radiation therapy; to detect the position of static jaws or verify the position of tracking jaws in more complex treatments. This method uses only information extracted from EPID images and it is therefore independent from the linear accelerator.     

Multiplexed Nanometric 3D Tracking of Microbeads Using an FFT-Phasor Algorithm

Many single-molecule biophysical techniques rely on nanometric tracking of microbeads to obtain quantitative information about the mechanical properties of biomolecules such as chromatin fibers. Their three-dimensional (3D) position can be resolved by holographic analysis of the diffraction pattern in wide-field imaging. Fitting this diffraction pattern to Lorenz-Mie scattering theory yields the bead’s position with nanometer accuracy in three dimensions but is computationally expensive. Real-time multiplexed bead tracking therefore requires a more efficient tracking method, such as comparison with previously measured diffraction patterns, known as look-up tables. Here, we introduce an alternative 3D phasor algorithm that provides robust bead tracking with nanometric localization accuracy in a z range of over 10 μm under nonoptimal imaging conditions. The algorithm is based on a two-dimensional cross correlation using fast Fourier transforms with computer-generated reference images, yielding a processing rate of up to 10,000 regions of interest per second. We implemented the technique in magnetic tweezers and tracked the 3D position of over 100 beads in real time on a generic CPU. The accuracy of 3D phasor tracking was extensively tested and compared to a look-up table approach using Lorenz-Mie simulations, avoiding experimental uncertainties. Its easy implementation, efficiency, and robustness can improve multiplexed biophysical bead-tracking applications, especially when high throughput is required and image artifacts are difficult to avoid.     

Dual-Axis Tomography: An Approach with Alignment Methods That Preserve Resolution

Tomographic reconstructions of biological specimens are now routinely being generated in our high voltage electron microscope by tilting the specimen around two orthogonal axes. Separate tomograms are computed from each tilt series. The two tomograms are aligned to each other with general 3-D linear transformations that can correct for distortions between the two tomograms, thus preserving the inherent resolution of the reconstruction throughout its volume. The 3-D Fourier transforms of the two tomograms are then selectively combined to achieve a single tomogram. Unlike a single-axis tomogram, a dual-axis tomogram shows good resolution for extended features at any orientation in the plane of the specimen; it also has improved resolution in the depth of the specimen. Calculations indicate that the improvements available from double tilting and from tilting to higher angles are largely additive. Actual and model data were used to assess whether varying the increment between tilted views in proportion to the cosine of the tilt angle would allow a reduction in the number of pictures required to achieve a given resolution of reconstruction. Analysis by Fourier sector correlation indicated that the variable tilt increment improved the reconstruction in some respects but degraded it in others. A varying tilt increment thus does not give an unqualified improvement, at least when using back-projection algorithms for the reconstruction.     

Tendon strain measurements with dynamic ultrasound images: evaluation of digital image correlation

Strain is an essential metric in tissue mechanics. Strains and strain distributions during functional loads can help identify damaged and pathologic regions as well as quantify functional compromise. Noninvasive strain measurement in vivo is difficult to perform. The goal of this in vitro study is to determine the efficacy of digital image correlation (DIC) methods to measure strain in B-mode ultrasound images. The Achilles tendons of eight male Wistar rats were removed and mechanically cycled between 0 and 1% strain. Three cine video images were captured for each specimen: (1) optical video for manual tracking of optical markers; (2) optical video for DIC tracking of optical surface markers; and (3) ultrasound video for DIC tracking of image texture within the tissue. All three imaging modalities were similarly able to measure tendon strain during cyclic testing. Manual/ImageJ-based strain values linearly correlated with DIC (optical marker)-based strain values for all eight tendons with a slope of 0.970. DIC (optical marker)-based strain values linearly correlated with DIC (ultrasound texture)-based strain values for all eight tendons with a slope of 1.003. Strain measurement using DIC was as accurate as manual image tracking methods, and DIC tracking was equally accurate when tracking ultrasound texture as when tracking optical markers. This study supports the use of DIC to calculate strains directly from the texture present in standard B-mode ultrasound images and supports the use of DIC for in vivo strain measurement using ultrasound images without additional markers, either artificially placed (for optical tracking) or anatomically in view (i.e., bony landmarks and/or muscle-tendon junctions).     

Vertebrate neural stem cell segmentation, tracking and lineaging with validation and editing

This protocol and the accompanying software program called LEVER (lineage editing and validation) enable quantitative automated analysis of phase-contrast time-lapse images of cultured neural stem cells. Images are captured at 5-min intervals over a period of 5-15 d as the cells proliferate and differentiate. LEVER automatically segments, tracks and generates lineage trees of the stem cells from the image sequence. In addition to generating lineage trees capturing the population dynamics of clonal development, LEVER extracts quantitative phenotypic measurements of cell location, shape, movement and size. When available, the system can include biomolecular markers imaged using fluorescence. It then displays the results to the user for highly efficient inspection and editing to correct any errors in the segmentation, tracking or lineaging. To enable high-throughput inspection, LEVER incorporates features for rapid identification of errors and for learning from user-supplied corrections to automatically identify and correct related errors.     

Nonlinear system modelling via optimal design of neural trees

This paper introduces a flexible neural tree model. The model is computed as a flexible multi-layer feed-forward neural network. A hybrid learning/evolutionary approach to automatically optimize the neural tree model is also proposed. The approach includes a modified probabilistic incremental program evolution algorithm (MPIPE) to evolve and determine a optimal structure of the neural tree and a parameter learning algorithm to optimize the free parameters embedded in the neural tree. The performance and effectiveness of the proposed method are evaluated using function approximation, time series prediction and system identification problems and compared with the related methods.     

High resolution electron tomography and segmentation‐by‐modeling interpretation in Bsoft

Bsoft offers many tools for the processing of tomographic tilt series and the interpretation of tomograms. Since I introduced tomography into Bsoft almost two decades ago, the field has advanced significantly, requiring refinement of old algorithms and development of new ones. The current direct detectors allow us to collect data more efficiently and with better quality, progressing towards automation. The goal is then to also automate alignment of tilt series and reconstruction. I added an estimation of the specimen thickness as well as fiducialless alignment, to augment the existing fiducial‐based alignment. High‐resolution work requires correction for the contrast transfer function, in tomography complicated by the tilted specimen. For this, I developed a method to generate a power spectrum using the whole micrograph, compensating for tilting. This is followed by routine determination of the contrast transfer function, and correction for it during reconstruction. The next steps involve interpretation of the tomogram, either by subtomogram averaging where possible, or by segmentation and modeling otherwise. Such interpretation actually constitutes the main time‐consuming part of tomography and is less amenable to automation compared to the initial reconstruction.