Applications of a bilateral denoising filter in biological electron microscopy

Due to the sensitivity of biological sample to the radiation damage, the low dose imaging conditions used for electron microscopy result in extremely noisy images. The processes of digitization, image alignment, and 3D reconstruction also introduce additional sources of noise in the final 3D structure. In this paper, we investigate the effectiveness of a bilateral denoising filter in various biological electron microscopy applications. In contrast to the conventional low pass filters, which inevitably smooth out both noise and structural features simultaneously, we found that bilateral filter holds a distinct advantage in being capable of effectively suppressing noise without blurring the high resolution details. In as much, we have applied this technique to individual micrographs, entire 3D reconstructions, segmented proteins, and tomographic reconstructions.     

Automated correlation of single particle tilt pairs for Random Conical Tilt and Orthogonal Tilt Reconstructions

One of the major methodological challenges in single particle electron microscopy is obtaining initial reconstructions which represent the structural heterogeneity of the dataset. Random Conical Tilt and Orthogonal Tilt Reconstruction techniques in combination with 3D alignment and classification can be used to obtain initial low-resolution reconstructions which represent the full range of structural heterogeneity of the dataset. In order to achieve statistical significance, however, a large number of 3D reconstructions, and, in turn, a large number of tilted image pairs are required. The extraction of single particle tilted image pairs from micrographs can be tedious and time-consuming, as it requires intensive user input even for semi-automated approaches. To overcome the bottleneck of manual selection of a large number of tilt pairs, we developed an algorithm for the correlation of single particle images from tilted image pairs in a fully automated and user-independent manner. The algorithm reliably correlates correct pairs even from noisy micrographs. We further demonstrate the applicability of the algorithm by using it to obtain initial references both from negative stain and unstained cryo datasets.     

Automation of random conical tilt and orthogonal tilt data collection using feature-based correlation

Visualization by electron microscopy has provided many insights into the composition, quaternary structure, and mechanism of macromolecular assemblies. By preserving samples in stain or vitreous ice it is possible to image them as discrete particles, and from these images generate three-dimensional structures. This ‘single-particle’ approach suffers from two major shortcomings; it requires an initial model to reconstitute 2D data into a 3D volume, and it often fails when faced with conformational variability. Random conical tilt (RCT) and orthogonal tilt (OTR) are methods developed to overcome these problems, but the data collection required, particularly for vitreous ice specimens, is difficult and tedious. In this paper, we present an automated approach to RCT/OTR data collection that removes the burden of manual collection and offers higher quality and throughput than is otherwise possible. We show example datasets collected under stain and cryo conditions and provide statistics related to the efficiency and robustness of the process. Furthermore, we describe the new algorithms that make this method possible, which include new calibrations, improved targeting and feature-based tracking.     

Automated acquisition of electron microscopic random conical tilt sets

Single particle reconstruction using the random conical tilt data collection geometry is a robust method for the initial determination of macromolecular structures by electron microscopy. Unfortunately, the broad adoption of this powerful approach has been limited by the practical challenges inherent in manual data collection of the required pairs of matching high and low tilt images (typically 60 degrees and 0 degrees). The microscopist is obliged to keep the imaging area centered during tilting as well as to maintain accurate focus in the tilted image while minimizing the overall electron dose, a challenging and time consuming process. To help solve these problems, we have developed an automated system for the rapid acquisition of accurately aligned and focused tilt pairs. The system has been designed to minimize the dose incurred during alignment and focusing, making it useful in both negative stain and cryo-electron microscopy. The system includes a feature for montaging untilted images to ensure that all of the particles in the tilted image may be used in the reconstruction.     

Automatic alignment and reconstruction of images for soft X-ray tomography

Soft X-ray tomography (SXT) is a powerful imaging technique that generates quantitative, 3D images of the structural organization of whole cells in a near-native state. SXT is also a high-throughput imaging technique. At the National Center for X-ray Tomography (NCXT), specimen preparation and image collection for tomographic reconstruction of a whole cell require only minutes. Aligning and reconstructing the data, however, take significantly longer. Here we describe a new component of the high throughput computational pipeline used for processing data at the NCXT. We have developed a new method for automatic alignment of projection images that does not require fiducial markers or manual interaction with the software. This method has been optimized for SXT data sets, which routinely involve full rotation of the specimen. This software gives users of the NCXT SXT instrument a new capability - virtually real-time initial 3D results during an imaging experiment, which can later be further refined. The new code, Automatic Reconstruction 3D (AREC3D), is also fast, reliable, and robust. The fundamental architecture of the code is also adaptable to high performance GPU processing, which enables significant improvements in speed and fidelity.     

Improved specimen reconstruction by Hilbert phase contrast tomography

The low signal-to-noise ratio (SNR) in images of unstained specimens recorded with conventional defocus phase contrast makes it difficult to interpret 3D volumes obtained by electron tomography (ET). The high defocus applied for conventional tilt series generates some phase contrast but leads to an incomplete transfer of object information. For tomography of biological weak-phase objects, optimal image contrast and subsequently an optimized SNR are essential for the reconstruction of details such as macromolecular assemblies at molecular resolution. The problem of low contrast can be partially solved by applying a Hilbert phase plate positioned in the back focal plane (BFP) of the objective lens while recording images in Gaussian focus. Images recorded with the Hilbert phase plate provide optimized positive phase contrast at low spatial frequencies, and the contrast transfer in principle extends to the information limit of the microscope. The antisymmetric Hilbert phase contrast (HPC) can be numerically converted into isotropic contrast, which is equivalent to the contrast obtained by a Zernike phase plate. Thus, in-focus HPC provides optimal structure factor information without limiting effects of the transfer function. In this article, we present the first electron tomograms of biological specimens reconstructed from Hilbert phase plate image series. We outline the technical implementation of the phase plate and demonstrate that the technique is routinely applicable for tomography. A comparison between conventional defocus tomograms and in-focus HPC volumes shows an enhanced SNR and an improved specimen visibility for in-focus Hilbert tomography.     

UCSF tomography: an integrated software suite for real-time electron microscopic tomographic data collection, alignment, and reconstruction

A real-time alignment and reconstruction scheme for electron microscopic tomography (EMT) has been developed and integrated within our UCSF tomography data collection software. This newly integrated software suite provides full automation from data collection to real-time reconstruction by which the three-dimensional (3D) reconstructed volume is immediately made available at the end of each data collection. Real-time reconstruction is achieved by calculating a weighted back-projection on a small Linux cluster (five dual-processor compute nodes) concurrently with the UCSF tomography data collection running on the microscope's computer, and using the fiducial-marker free alignment data generated during the data collection process. The real-time reconstructed 3D volume provides users with immediate feedback to fully asses all aspects of the experiment ranging from sample choice, ice thickness, experimental parameters to the quality of specimen preparation. This information can be used to guide subsequent data collections. Access to the reconstruction is especially useful in low-dose cryo EMT where such information is very difficult to obtain due to extraordinary low signal to noise ratio in each 2D image. In our environment, we generally collect 2048 x 2048 pixel images which are subsequently computationally binned four-fold for the on-line reconstruction. Based upon experiments performed with thick and cryo specimens at various CCD magnifications (50000x-80000x), alignment accuracy is sufficient to support this reduced resolution but should be refined before calculating a full resolution reconstruction. The reduced resolution has proven to be quite adequate to assess sample quality, or to screen for the best data set for full-resolution reconstruction, significantly improving both productivity and efficiency of system resources. The total time from start of data collection to a final reconstructed volume (512 x 512 x 256 pixels) is about 50 min for a +/-70 degrees 2k x 2k pixel tilt series acquired at every 1 degrees.     

Automatic alignment of transmission electron microscope tilt series without fiducial markers

Accurate image alignment is needed for computing three-dimensional reconstructions from transmission electron microscope tilt series. So far, the best results have been obtained by using colloidal gold beads as fiducial markers. If their use has not been possible for some reason, the only option has been the automatic cross-correlation-based registration methods. However, the latter methods are inaccurate and, as we will show, inappropriate for the whole problem. Conversely, we propose a novel method that uses the actual 3D motion model but works without any fiducial markers in the images. The method is based on matching and tracking some interest points of the intensity surface by first solving the underlying geometrical constraint of consecutive images in the tilt series. The results show that our method is near the gold marker alignment in the level of accuracy and hence opens the way for new opportunities in the analysis of electron tomography reconstructions, especially when markers cannot be used.     

Fast rotational matching of single-particle images

The presence of noise and absence of contrast in electron micrographs lead to a reduced resolution of the final 3D reconstruction, due to the inherent limitations of single-particle image alignment. The fast rotational matching (FRM) algorithm was introduced recently for an accurate alignment of 2D images under such challenging conditions. Here, we implemented this algorithm for the first time in a standard 3D reconstruction package used in electron microscopy. This allowed us to carry out exhaustive tests of the robustness and reliability in iterative orientation determination, classification, and 3D reconstruction on simulated and experimental image data. A classification test on GroEL chaperonin images demonstrates that FRM assigns up to 13% more images to their correct reference orientation, compared to the classical self-correlation function method. Moreover, at sub-nanometer resolution, GroEL and rice dwarf virus reconstructions exhibit a remarkable resolution gain of 10-20% that is attributed to the novel image alignment kernel.     

The resolution dependence of optimal exposures in liquid nitrogen temperature electron cryomicroscopy of catalase crystals

Electron beam damage is the fundamental limit to resolution in electron cryomicroscopy (cryo-EM) of frozen, hydrated specimens. Radiation damage increases with the number of electrons used to obtain an image and affects information at higher spatial frequencies before low-resolution information. For the experimentalist, a balance exists between electron exposures sufficient to obtain a useful signal-to-noise ratio (SNR) in images and exposures that limit the damage to structural features. In single particle cryo-EM this balance is particularly delicate: low-resolution features must be imaged with a sufficient SNR to allow image alignment so that high-resolution features recorded below the noise level can be recovered by averaging independent images. By measuring the fading of Fourier components from images obtained at 200 kV of thin crystals of catalase embedded in ice, we have determined the electron exposures that will maximize the SNR at resolutions between 86 and 2.9 Å. These data allow for a rational choice of exposure for single particle cryo-EM. For example, for 20 Å resolution, the SNR is maximized at ～20 e^?/Ų, whereas for 3 Å resolution, it is maximized at ～10 e^?/Ų. We illustrate the effects of exposure in single particle cryo-EM with data collected at ～12–15 and ～24–30 e^?/Ų.     

Pattern recognition and classification of images of biological macromolecules using artificial neural networks.

The goal of this work was to analyze an image data set and to detect the structural variability within this set. Two algorithms for pattern recognition based on neural networks are presented, one that performs an unsupervised classification (the self-organizing map) and the other a supervised classification (the learning vector quantization). The approach has a direct impact in current strategies for structural determination from electron microscopic images of biological macromolecules. In this work we performed a classification of both aligned but heterogeneous image data sets as well as basically homogeneous but otherwise rotationally misaligned image populations, in the latter case completely avoiding the typical reference dependency of correlation-based alignment methods. A number of examples on chaperonins are presented. The approach is computationally fast and robust with respect to noise. Programs are available through ftp.     

The Augmented Lagrange Multipliers Method for Matrix Completion from Corrupted Samplings with Application to Mixed Gaussian-Impulse Noise Removal

This paper studies the problem of the restoration of images corrupted by mixed Gaussian-impulse noise. In recent years, low-rank matrix reconstruction has become a research hotspot in many scientific and engineering domains such as machine learning, image processing, computer vision and bioinformatics, which mainly involves the problem of matrix completion and robust principal component analysis, namely recovering a low-rank matrix from an incomplete but accurate sampling subset of its entries and from an observed data matrix with an unknown fraction of its entries being arbitrarily corrupted, respectively. Inspired by these ideas, we consider the problem of recovering a low-rank matrix from an incomplete sampling subset of its entries with an unknown fraction of the samplings contaminated by arbitrary errors, which is defined as the problem of matrix completion from corrupted samplings and modeled as a convex optimization problem that minimizes a combination of the nuclear norm and the -norm in this paper. Meanwhile, we put forward a novel and effective algorithm called augmented Lagrange multipliers to exactly solve the problem. For mixed Gaussian-impulse noise removal, we regard it as the problem of matrix completion from corrupted samplings, and restore the noisy image following an impulse-detecting procedure. Compared with some existing methods for mixed noise removal, the recovery quality performance of our method is dominant if images possess low-rank features such as geometrically regular textures and similar structured contents; especially when the density of impulse noise is relatively high and the variance of Gaussian noise is small, our method can outperform the traditional methods significantly not only in the simultaneous removal of Gaussian noise and impulse noise, and the restoration ability for a low-rank image matrix, but also in the preservation of textures and details in the image.     

Classification and 3D averaging with missing wedge correction in biological electron tomography

Strategies for the determination of 3D structures of biological macromolecules using electron crystallography and single-particle electron microscopy utilize powerful tools for the averaging of information obtained from 2D projection images of structurally homogeneous specimens. In contrast, electron tomographic approaches have often been used to study the 3D structures of heterogeneous, one-of-a-kind objects such as whole cells where image-averaging strategies are not applicable. Complex entities such as cells and viruses, nevertheless, contain multiple copies of numerous macromolecules that can individually be subjected to 3D averaging. Here we present a complete framework for alignment, classification, and averaging of volumes derived by electron tomography that is computationally efficient and effectively accounts for the missing wedge that is inherent to limited-angle electron tomography. Modeling the missing data as a multiplying mask in reciprocal space we show that the effect of the missing wedge can be accounted for seamlessly in all alignment and classification operations. We solve the alignment problem using the convolution theorem in harmonic analysis, thus eliminating the need for approaches that require exhaustive angular search, and adopt an iterative approach to alignment and classification that does not require the use of external references. We demonstrate that our method can be successfully applied for 3D classification and averaging of phantom volumes as well as experimentally obtained tomograms of GroEL where the outcomes of the analysis can be quantitatively compared against the expected results.     

Cryo X-ray nano-tomography of vaccinia virus infected cells

We have performed full-field cryo X-ray microscopy in the water window photon energy range on vaccinia virus (VACV) infected cells to produce tomographic reconstructions. PtK2 cells were infected with a GFP-expressing VACV strain and frozen by plunge fast freezing. The infected cells were selected by light fluorescence microscopy of the GFP marker and subsequently imaged in the X-ray microscope under cryogenic conditions. Tomographic tilt series of X-ray images were used to yield three-dimensional reconstructions showing different cell organelles (nuclei, mitochondria, filaments), together with other structures derived from the virus infection. Among them, it was possible to detect viral factories and two types of viral particles related to different maturation steps of VACV (immature and mature particles), which were compared to images obtained by standard electron microscopy of the same type of cells. In addition, the effect of radiation damage during X-ray tomographic acquisition was analyzed. Thin sections studied by electron microscopy revealed that the morphological features of the cells do not present noticeable changes after irradiation. Our findings show that cryo X-ray nano-tomography is a powerful tool for collecting three-dimensional structural information from frozen, unfixed, unstained whole cells with sufficient resolution to detect different virus particles exhibiting distinct maturation levels.     

Pré-traitement d’un volume IRM par filtrage anisotrope robuste pour la segmentation de l’épaule

A novel pre-treatment process for image segmentation, based on anisotropic diffusion and robust statistics, is presented in this paper. Image smoothing with edge preservation is shown to help upper limb segmentation (shoulder segmentation in particular) in MRI datasets. The anisotropic diffusion process is mainly controlled by an automated stopping function that depends on the values of voxel gradient. Voxel gradients are divided into two classes: one for high values, corresponding to edge voxels or noisy voxels, one for low values. The anisotropic diffusion process is also controlled by a threshold on voxel gradients that separates both classes. A global estimation of this threshold parameter is classically used. In this paper, we propose a new method based on a local robust estimation. It allows a better removing of noise while preserving edges in the images. An entropy criterion is used to quantify the ability of the algorithm to remove noise with different signal to noise ratios in synthetic images. Another quantitative evaluation criterion based on the Pratt Figure of Merit (FOM) is proposed to evaluate the edge preservation and their location accuracy with respect to a manual segmentation. The results on synthetic and MRI data of shoulder show the assets of the local model in terms of areas homogeneity and edges locations.     

Towards an atlas of mammalian cell ultrastructure by cryo soft X-ray tomography

We provide a catalog of 3D cryo soft X-ray tomography (cryo-SXT) images obtained from ～6 to 12μm thick mouse adenocarcinoma cells. Included are multiple representative images of nuclei, nucleoli, nuclear membrane, nuclear membrane channels, mitochondria, lysosomes, endoplasmic reticulum, filaments and plasma membrane, plus three structures not previously described by cryo-SXT, namely Golgi, microvilli and nuclear-membrane blebs. Sections from the 3D cryo-SXT tomograms for all the preceding structures closely resemble those seen by thin-section transmission electron microscopy (TEM). Some structures such as nuclear-membrane channels and nuclear-membrane blebs are more easily detected by cryo-SXT than TEM most likely due to their better contrast and cellular preservation in cryo-SXT combined with the ability to rapidly locate these structures within a full 3D image. We identify and discuss two current limitations in cryo-SXT: variability in image quality and difficulties in detecting weaker contrast structures such as chromatin and various nuclear bodies. Progress on these points is likely to come from the solution of several technical problems in image acquisition, plus the implementation of advanced cryo soft X-ray microscopy approaches such as phase contrast or optical sectioning.     

The discriminative bilateral filter: an enhanced denoising filter for electron microscopy data

Advances in three-dimensional (3D) electron microscopy (EM) and image processing are providing considerable improvements in the resolution of subcellular volumes, macromolecular assemblies and individual proteins. However, the recovery of high-frequency information from biological samples is hindered by specimen sensitivity to beam damage. Low dose electron cryo-microscopy conditions afford reduced beam damage but typically yield images with reduced contrast and low signal-to-noise ratios (SNRs). Here, we describe the properties of a new discriminative bilateral (DBL) filter that is based upon the bilateral filter implementation of Jiang et al. (Jiang, W., Baker, M.L., Wu, Q., Bajaj, C., Chiu, W., 2003. Applications of a bilateral denoising filter in biological electron microscopy. J. Struc. Biol. 128, 82-97.). In contrast to the latter, the DBL filter can distinguish between object edges and high-frequency noise pixels through the use of an additional photometric exclusion function. As a result, high frequency noise pixels are smoothed, yet object edge detail is preserved. In the present study, we show that the DBL filter effectively reduces noise in low SNR single particle data as well as cellular tomograms of stained plastic sections. The properties of the DBL filter are discussed in terms of its usefulness for single particle analysis and for pre-processing cellular tomograms ahead of image segmentation.     

An Adaptive Low-Rank Modeling-Based Active Learning Method for Medical Image Annotation

Active learning is an effective solution to interactively select a limited number of informative examples and use them to train a learning algorithm that can achieve its optimal performance for specific tasks. It is suitable for medical image applications in which unlabeled data are abundant but manual annotation could be very time-consuming and expensive. However, designing an effective active learning strategy for informative example selection is a challenging task, due to the intrinsic presence of noise in medical images, the large number of images, and the variety of imaging modalities. In this study, a novel low-rank modeling-based multi-label active learning (LRMMAL) method is developed to address these challenges and select informative examples for training a classifier to achieve the optimal performance. The proposed method independently quantifies image noise and integrates it with other measures to guide a pool-based sampling process to determine the most informative examples for training a classifier. In addition, an automatic adaptive cross entropy-based parameter determination scheme is proposed for further optimizing the example sampling strategy. Experimental results on varied medical image datasets and comparisons with other state-of-the-art multi-label active learning methods illustrate the superior performance of the proposed method.     

Comparison of two academic software packages for analyzing two-dimensional gel images

One of the key limitations for proteomic studies using two-dimensional (2D) gel is the lack of automatic, fast, robust, and reliable methods for detecting, matching, and quantifying protein spots. Although there are commercial software packages for 2D gel image analysis, extensive human intervention is still needed for spot detection and matching, which is time-consuming and error-prone. Moreover, the commercial software packages are usually expensive and non-open source. Thus, it is very beneficial for researchers to have free software that is fast, fully automatic, and robust. In this paper, we review and compare two recently developed and publicly available software packages, RegStatGel and Pinnacle, for analyzing 2D gel images. These two software packages share some common features and also have some fundamental difference in the aspects of spot detection and quantification. Based on our experience, RegStatGel is much better in terms of spot detection and matching. It also contains more advanced statistical tools and is more user-friendly. In contrast, Pinnacle is quite sensitive to background noise and relies on external statistical software packages for statistical analysis.