Spatiotemporal Rank Filtering Improves Image Quality Compared to Frame Averaging in 2-Photon Laser Scanning Microscopy

Live imaging of biological specimens using optical microscopy is limited by tradeoffs between spatial and temporal resolution, depth into intact samples, and phototoxicity. Two-photon laser scanning microscopy (2P-LSM), the gold standard for imaging turbid samples in vivo, has conventionally constructed images with sufficient signal-to-noise ratio (SNR) generated by sequential raster scans of the focal plane and temporal integration of the collected signals. Here, we describe spatiotemporal rank filtering, a nonlinear alternative to temporal integration, which makes more efficient use of collected photons by selectively reducing noise in 2P-LSM images during acquisition. This results in much higher SNR while preserving image edges and fine details. Practically, this allows for at least a four fold decrease in collection times, a substantial improvement for time-course imaging in biological systems.     

Fuzzy Index to Evaluate Edge Detection in Digital Images

In literature, we can find different metrics to evaluate the detected edges in digital images, like Pratt''s figure of merit (FOM), Jaccard’s index (JI) and Dice’s coefficient (DC). These metrics compare two images, the first one is the reference edges image, and the second one is the detected edges image. It is important to mention that all existing metrics must binarize images before their evaluation. Binarization step causes information to be lost because an incomplete image is being evaluated. In this paper, we propose a fuzzy index (FI) for edge evaluation that does not use a binarization step. In order to process all detected edges, images are represented in their fuzzy form and all calculations are made with fuzzy sets operators and fuzzy Euclidean distance between both images. Our proposed index is compared to the most used metrics using synthetic images, with good results.     

Tensor regularized total variation for denoising of third harmonic generation images of brain tumors

Third harmonic generation (THG) microscopy shows great potential for instant pathology of brain tissue during surgery. However, the rich morphologies contained and the noise associated makes image restoration, necessary for quantification of the THG images, challenging. Anisotropic diffusion filtering (ADF) has been recently applied to restore THG images of normal brain, but ADF is hard‐to‐code, time‐consuming and only reconstructs salient edges. This work overcomes these drawbacks by expressing ADF as a tensor regularized total variation model, which uses the Huber penalty and the L₁ norm for tensor regularization and fidelity measurement, respectively. The diffusion tensor is constructed from the structure tensor of ADF yet the tensor decomposition is performed only in the non‐flat areas. The resulting model is solved by an efficient and easy‐to‐code primal‐dual algorithm. Tests on THG brain tumor images show that the proposed model has comparable denoising performance as ADF while it much better restores weak edges and it is up to 60% more time efficient.      

Preprocessing of two-dimensional gel electrophoresis images

Proteomics produces a huge amount of two-dimensional gel electrophoresis images. Their analysis can yield a lot of information concerning proteins responsible for different diseases or new unidentified proteins. However, an automatic analysis of such images requires an efficient tool for reducing noise in images. This allows proper detection of the spots' borders, which is important in protein quantification (as the spots' areas are used to determine the amounts of protein present in an analyzed mixture). Also in the feature-based matching methods the detected features (spots) can be described by additional attributes, such as area or shape. In our study, a comparison of different methods of noise reduction is performed in order to find out a method best suited for reducing noise in gel images. Among the compared methods there are the classical methods of linear filtering, e.g., the mean and Gaussian filtering, the nonlinear method, i.e., median filtering, and also the methods better suited for processing of nonstationary signals, such as spatially adaptive linear filtering and filtering in the wavelet domain. The best results are obtained by filtering of gel images in the wavelet domain, using the BayesThresh method of threshold value determination.     

The aesthetic experience of 'contour binding'

To find the diagnostic spatial frequency information in different painting styles (cubism, impressionism and realism), we have compared sensitivity (d') in distinguishing signal (subject of the painting) from noise with normal, high-pass and low-pass filtered images at long (150 ms) and short (30 ms) exposure. We found that for cubist-style images, d' increases with high-pass filtering compared with normal and low-pass filtered images, but decreases with low-pass filtering compared with normal images. These results indicate that channels with high spatial resolution provide the diagnostic information to solve the binding problem. Sensitivity for images in impressionist style was instead reduced by both low- and high-pass filtering. This indicates that both high and low spatial frequency channels play a role in solving the binding problem, suggesting the involvement of large collator units that group the response of small channels tuned to the same orientation. The difference between realism, which shows higher sensitivity for low-frequency filtering at short durations and cubism in which the binding problem is solved by high spatial frequency channels, has a corresponding difference in aesthetic judgment: the probability of judging a painting as 'intriguing' is larger with low-pass filtering than with high-pass filtering in realism, while the opposite is true for cubism. This suggests that the aesthetic experience is available during early processing of an image, and could preferentially influence high-level categorization of the subject of a painting.     

Effects of non-local diffusion on structural MRI preprocessing and default network mapping: statistical comparisons with isotropic/anisotropic diffusion

Neuroimaging community usually employs spatial smoothing to denoise magnetic resonance imaging (MRI) data, e.g., Gaussian smoothing kernels. Such an isotropic diffusion (ISD) based smoothing is widely adopted for denoising purpose due to its easy implementation and efficient computation. Beyond these advantages, Gaussian smoothing kernels tend to blur the edges, curvature and texture of images. Researchers have proposed anisotropic diffusion (ASD) and non-local diffusion (NLD) kernels. We recently demonstrated the effect of these new filtering paradigms on preprocessing real degraded MRI images from three individual subjects. Here, to further systematically investigate the effects at a group level, we collected both structural and functional MRI data from 23 participants. We first evaluated the three smoothing strategies' impact on brain extraction, segmentation and registration. Finally, we investigated how they affect subsequent mapping of default network based on resting-state functional MRI (R-fMRI) data. Our findings suggest that NLD-based spatial smoothing maybe more effective and reliable at improving the quality of both MRI data preprocessing and default network mapping. We thus recommend NLD may become a promising method of smoothing structural MRI images of R-fMRI pipeline.     

Curvelet Based Offline Analysis of SEM Images

Manual offline analysis, of a scanning electron microscopy (SEM) image, is a time consuming process and requires continuous human intervention and efforts. This paper presents an image processing based method for automated offline analyses of SEM images. To this end, our strategy relies on a two-stage process, viz. texture analysis and quantification. The method involves a preprocessing step, aimed at the noise removal, in order to avoid false edges. For texture analysis, the proposed method employs a state of the art Curvelet transform followed by segmentation through a combination of entropy filtering, thresholding and mathematical morphology (MM). The quantification is carried out by the application of a box-counting algorithm, for fractal dimension (FD) calculations, with the ultimate goal of measuring the parameters, like surface area and perimeter. The perimeter is estimated indirectly by counting the boundary boxes of the filled shapes. The proposed method, when applied to a representative set of SEM images, not only showed better results in image segmentation but also exhibited a good accuracy in the calculation of surface area and perimeter. The proposed method outperforms the well-known Watershed segmentation algorithm.     

The relative importance of local phase and local amplitude in patchwise image reconstruction

Natural images were subjected to patchwise Fourier analysis, and the local amplitude and phase spectra were swapped between different images. When the patches were large relative to the image size, the appearance of the reconstructed image was similar to that of the image from which the phase information had been derived, in agreement with previous reports of phase-dominance in the global Fourier Transform. However, when the patch size was made sufficiently small, the appearance of reconstructed images was dominated by amplitude rather than phase. This was not simply due to the DC component of the amplitude spectrum. Prior low-pass filtering of the images enhanced the dominance of amplitude information in the patchwise transform. We conclude that patchwise-reconstructed images contain two quite distinct kinds of information for the human observer. The first is the positional information (local sign) of the patches themselves; the second is the textural information within patches, which is dominated by amplitude rather than phase. The reason why the global Fourier Transform is dominated by phase is that in the absence of any other information about local sign, phase is necessary to reconstruct localised features such as edges.     

A Novel Weighted Total Difference Based Image Reconstruction Algorithm for Few-View Computed Tomography

In practical applications of computed tomography (CT) imaging, due to the risk of high radiation dose imposed on the patients, it is desired that high quality CT images can be accurately reconstructed from limited projection data. While with limited projections, the images reconstructed often suffer severe artifacts and the edges of the objects are blurred. In recent years, the compressed sensing based reconstruction algorithm has attracted major attention for CT reconstruction from a limited number of projections. In this paper, to eliminate the streak artifacts and preserve the edge structure information of the object, we present a novel iterative reconstruction algorithm based on weighted total difference (WTD) minimization, and demonstrate the superior performance of this algorithm. The WTD measure enforces both the sparsity and the directional continuity in the gradient domain, while the conventional total difference (TD) measure simply enforces the gradient sparsity horizontally and vertically. To solve our WTD-based few-view CT reconstruction model, we use the soft-threshold filtering approach. Numerical experiments are performed to validate the efficiency and the feasibility of our algorithm. For a typical slice of FORBILD head phantom, using 40 projections in the experiments, our algorithm outperforms the TD-based algorithm with more than 60% gains in terms of the root-mean-square error (RMSE), normalized root mean square distance (NRMSD) and normalized mean absolute distance (NMAD) measures and with more than 10% gains in terms of the peak signal-to-noise ratio (PSNR) measure. While for the experiments of noisy projections, our algorithm outperforms the TD-based algorithm with more than 15% gains in terms of the RMSE, NRMSD and NMAD measures and with more than 4% gains in terms of the PSNR measure. The experimental results indicate that our algorithm achieves better performance in terms of suppressing streak artifacts and preserving the edge structure information of the object.     

Mining-based associative image filtering using harmonic mean

With the development of IT convergence technologies, users can now more easily access useful information. These days, diverse and far-reaching information is being rapidly produced and distributed instantly in digitized format. Studies are continuously seeking to develop more efficient methods of delivering information to a greater number of users. Image filtering, which extracts features of interest from images, was developed to address the weakness of collaborative filtering, which is limited to superficial data analysis. However, image filtering has its own weakness of requiring complicated calculations to obtain the similarity between images. In this study, to resolve these problems, we propose associative image filtering based on the mining method utilizing the harmonic mean. Using data mining’s Apriori algorithm, this study investigated the association among preferred images from an associative image group and obtained a prediction based on user preference mean. In so doing, we observed a positive relationship between the various image preferences and the various distances between images’ color histograms. Preference mean was calculated based on the arithmetic mean, geometric mean, and harmonic mean. We found through performance analysis that the harmonic mean had the highest accuracy. In associative image filtering, we used the harmonic mean in order to anticipate preferences. In testing accuracy with MAE utilizing the proposed method, this study demonstrated an improvement of approximately 12 % on average compared to previous collaborative image filtering.     

Robust filtering and particle picking in micrograph images towards 3D reconstruction of purified proteins with cryo-electron microscopy

In order to make a high resolution model of macromolecular structures from cryo-electron microscope (cryo-EM) raw images one has to be precise at every processing step from particle picking to 3D image reconstruction. In this paper we propose a collection of novel methods for filtering cryo-EM images and for automatic picking of particles. These methods have been developed for two cases: (1) when particles can be identified and (2) when particle are not distinguishable. The advantages of these methods are demonstrated in standard purified protein samples and to generalize them we do not use any ad hoc presumption of the geometry of the particle projections. We have also suggested a filtering method to increase the signal-to-noise (S/N) ratio which has proved to be useful for other levels of reconstruction, i.e., finding orientations and 3D model reconstruction.     

MRI denoising using nonlocal neutrosophic set approach of Wiener filtering

In this paper, a new filtering method is presented to remove the Rician noise from magnetic resonance images (MRI) acquired using single coil MRI acquisition system. This filter is based on nonlocal neutrosophic set (NLNS) approach of Wiener filtering. A neutrosophic set (NS), a part of neutrosophy theory, studies the origin, nature, and scope of neutralities, as well as their interactions with different ideational spectra. Now, we apply the neutrosophic set into image domain and define some concepts and operators for image denoising. First, the nonlocal mean is applied to the noisy MRI. The resultant image is transformed into NS domain, described using three membership sets: true (T), indeterminacy (I) and false (F). The entropy of the neutrosophic set is defined and employed to measure the indeterminacy. The ω-Wiener filtering operation is used on T and F to decrease the set indeterminacy and to remove the noise. The experiments have been conducted on simulated MR images from Brainweb database and clinical MR images. The results show that the NLNS Wiener filter produces better denoising results in terms of qualitative and quantitative measures compared with other denoising methods, such as classical Wiener filter, the anisotropic diffusion filter, the total variation minimization and the nonlocal means filter. The visual and the diagnostic quality of the denoised image are well preserved.     

Biofilm structure differentiation based on multi-resolution analysis

Quantitative parameters for describing the morphology of biofilms are crucial towards establishing the influence of growing conditions on biofilm structure. Parameters used in earlier studies generally fail to differentiate complex three-dimensional structures. This article presents a novel approach of defining a parameter vector based on the energy signature of multi-resolution analysis, which was applied to differentiating biofilm structures from confocal laser scanning microscopy (CLSM) biofilm images. The parameter vector distinguished differences in the spatial arrangements of synthetic images. For real CLSM images, this parameter vector detected subtle differences in biofilm structure for three sample cases: (1) two adjacent images of a CLSM stack; (2) two partial stacks from the same CLSM stack with equal numbers of images but spatially offset by one image; and (3) three complete CLSM stacks from different bacterial strains. It was also observed that filtering the noise in CLSM images enhanced the sensitivity of the differentiation using our parameter vector.     

Biofilm structure differentiation based on multi-resolution analysis

Quantitative parameters for describing the morphology of biofilms are crucial towards establishing the influence of growing conditions on biofilm structure. Parameters used in earlier studies generally fail to differentiate complex three-dimensional structures. This article presents a novel approach of defining a parameter vector based on the energy signature of multi-resolution analysis, which was applied to differentiating biofilm structures from confocal laser scanning microscopy (CLSM) biofilm images. The parameter vector distinguished differences in the spatial arrangements of synthetic images. For real CLSM images, this parameter vector detected subtle differences in biofilm structure for three sample cases: (1) two adjacent images of a CLSM stack; (2) two partial stacks from the same CLSM stack with equal numbers of images but spatially offset by one image; and (3) three complete CLSM stacks from different bacterial strains. It was also observed that filtering the noise in CLSM images enhanced the sensitivity of the differentiation using our parameter vector.     

A computational theory of human stereo vision.

An algorithm is proposed for solving the stereoscopic matching problem. The algorithm consists of five steps: (1) Each image is filtered at different orientations with bar masks of four sizes that increase with eccentricity; the equivalent filters are one or two octaves wide. (2) Zero-crossings in the filtered images, which roughly correspond to edges, are localized. Positions of the ends of lines and edges are also found. (3) For each mask orientation and size, matching takes place between pairs of zero-crossings or terminationss of the same sign in the two images, for a range of disparities up to about the width of the mask's central region. (4) Wide masks can control vergence movements, thus causing small masks to come into correspondence. (5) When a correspondence is achieved, it is stored in a dynamic buffer, called the 2 1/2-D sketch. It is shown that this proposal provides a theoretical framework for most existing psychophysical and neurophysiological data about stereopsis. Several critical experimental predictions are also made, for instance about the size of Panum's area under various conditions. The results of such experiments would tell us whether, for example, cooperativity is necessary for the matching process.     

Low-pass filtering, a new method of fractal analysis: application to PET images of pulmonary blood flow.

The pattern of a spatial structure that repeats itself independently of the scale of magnification or resolution is often characterized by a fractal dimension (D). Two-dimensional low-pass filtering, which may serve as a method to assess D, was applied to functional images of pulmonary perfusion measured by positron emission tomography. The corner frequency of a low-pass filter is inversely proportional to the resolution scale. The method was applied to three types of images: random noise images, synthetic fractal images, and positron emission tomographic images of pulmonary perfusion. Images were processed with two-dimensional low-pass filters of decreasing corner frequencies, and a spatial heterogeneity index, the coefficient of variation, was calculated for each low-pass-filtered image. The natural logarithm of the coefficient of variation scaled linearly with the natural logarithm of the resolution scale for the PET images studied (average R(2) = 0.99). D ranged from 1.25 to 1.36 for the residual distribution of pulmonary perfusion after vertical gradients were removed by linear regression. D of the same data without removal of vertical gradients ranged from 1.11 to 1.14, but the fractal plots had systematic deviations from linearity and a lower linear correlation coefficient (R(2) = 0. 96). The method includes all data in the lung field and is insensitive to the effects of misregistration. We conclude that low-pass filtering offers new insights into the interpretation of D of two-dimensional functional images as a measure of the frequency content of spatial heterogeneity.     

A Zernike-moment-based non-local denoising filter for cryo-EM images

Cryo-electron microscopy (cryo-EM) plays an important role in determining the structure of proteins, viruses, and even the whole cell. It can capture dynamic structural changes of large protein complexes, which other methods such as X-ray crystallography and nuclear magnetic resonance analysis find difficult. The signal-to-noise ratio of cryo-EM images is low and the contrast is very weak, and therefore, the images are very noisy and require filtering. In this paper, a filtering method based on non-local means and Zernike moments is proposed. The method takes into account the rotational symmetry of some biological molecules to enhance the signal-to-noise ratio of cryo-EM images. The method may be useful in cryo-EM image processing such as the automatic selection of particles, orientation determination, and the building of initial models.     

A mathematical model of the primary visual cortex and hypercolumn

A mathematical model of the primary visual cortex is presented. Basically, the model comprises two features. Firstly, in analogy with the principle of the computerized tomography (CT), it assumes that simple cells in each hypercolumn are not merely detecting line segments in images as features, but rather that they are as a whole representing the local image with a certain representation. Secondly, it assumes that each hypercolumn is performing spatial frequency analyses of local images using that representation, and that the resultant spectra are represented by complex cells. The model is analyzed using numerical simulations and its advantages are discussed from the viewpoint of visual information processing. It is shown that 1) the proposed processing is tolerant to shifts in position of input images, and that 2) spatial frequency filtering operations can be easily performed in the model.     

Limulus vision in the marine environment

Horseshoe crabs use vision to find mates. They can reliably detect objects resembling potential mates under a variety of lighting conditions. To understand how they achieve this remarkable performance, we constructed a cell based realistic model of the lateral eye to compute the ensembles of optic nerve activity ("neural images") it transmits to the brain. The neural images reveal a robust encocding of mate-like objects that move underwater during the day. The neural images are much less clear at night, even though the eyes undergo large circadian increases of sensitivity that nearly compensate for the millionfold decreasein underwater lighting after sundown. At night the neurral images are noisy, dominated by bursts of nerve impulses from random photon events that occur at low nighttime levels of illumination. Deciphering the eye's input to the brain begins at the first synaptic level with lowpass temporal and spatial filtering. Both neural filtering mechanisms improve the signal-to-noise properties of the eye's input, yielding clearer neural images of potential mates, especiallyat night. Insights about visual processing by the relatively simple visual system of Limulus may aid in the designof robotic sensors for the marine environment.