DeconvTest: Simulation framework for quantifying errors and selecting optimal parameters of image deconvolution

Deconvolution is an essential step of image processing that aims to compensate for the image blur caused by the microscope's point spread function. With many existing deconvolution methods, it is challenging to choose the method and its parameters most appropriate for particular image data at hand. To facilitate this task, we developed DeconvTest: an open‐source Python‐based framework for generating synthetic microscopy images, deconvolving them with different algorithms, and quantifying reconstruction errors. In contrast to existing software, DeconvTest combines all components required to analyze deconvolution performance in a systematic, high‐throughput and quantitative manner. We demonstrate the power of the framework by using it to identify the optimal deconvolution settings for synthetic and real image data. Based on this, we provide a guideline for (a) choosing optimal values of deconvolution parameters for image data at hand and (b) optimizing imaging conditions for best results in combination with subsequent image deconvolution.     

Comparison of Deconvolution Filters for Photoacoustic Tomography

In this work, we compare the merits of three temporal data deconvolution methods for use in the filtered backprojection algorithm for photoacoustic tomography (PAT). We evaluate the standard Fourier division technique, the Wiener deconvolution filter, and a Tikhonov L-2 norm regularized matrix inversion method. Our experiments were carried out on subjects of various appearances, namely a pencil lead, two man-made phantoms, an in vivo subcutaneous mouse tumor model, and a perfused and excised mouse brain. All subjects were scanned using an imaging system with a rotatable hemispherical bowl, into which 128 ultrasound transducer elements were embedded in a spiral pattern. We characterized the frequency response of each deconvolution method, compared the final image quality achieved by each deconvolution technique, and evaluated each method’s robustness to noise. The frequency response was quantified by measuring the accuracy with which each filter recovered the ideal flat frequency spectrum of an experimentally measured impulse response. Image quality under the various scenarios was quantified by computing noise versus resolution curves for a point source phantom, as well as the full width at half maximum (FWHM) and contrast-to-noise ratio (CNR) of selected image features such as dots and linear structures in additional imaging subjects. It was found that the Tikhonov filter yielded the most accurate balance of lower and higher frequency content (as measured by comparing the spectra of deconvolved impulse response signals to the ideal flat frequency spectrum), achieved a competitive image resolution and contrast-to-noise ratio, and yielded the greatest robustness to noise. While the Wiener filter achieved a similar image resolution, it tended to underrepresent the lower frequency content of the deconvolved signals, and hence of the reconstructed images after backprojection. In addition, its robustness to noise was poorer than that of the Tikhonov filter. The performance of the Fourier filter was found to be the poorest of all three methods, based on the reconstructed images’ lowest resolution (blurriest appearance), generally lowest contrast-to-noise ratio, and lowest robustness to noise. Overall, the Tikhonov filter was deemed to produce the most desirable image reconstructions.     

Optimised generalized polarisation analysis of C-laurdan reveals clear order differences between T cell membrane compartments

Heterogenous packing of plasma membrane lipids is important for cellular processes like signalling, adhesion and sorting of membrane components. Solvatochromic membrane fluorophores that respond to changes from liquid-ordered (l_o) phase to liquid-disordered (l_d) by red shifts in their emission spectra are often used to assess lipid packing. Their response can be quantified using generalized polarisation (GP) using fluorescence microscopy images from two emission ranges, preferably from a region of interest (ROI) limited to a specific membrane compartment. However, image quality is limited by Poisson noise and convolution by the point spread function of the imaging system. Examining GP-analysis of C-laurdan labelled T cells using the image restoration procedure deconvolution, we demonstrate that deconvolution substantially improves the image resolution by making the plasma membrane clearly discernible and facilitating plasma membrane ROI selection. We conclude that automatic ROI selection has advantages over manual ROI selection when it comes to reproducibility and speed, but reliable GP-measurements can also be obtained by manually demarcated ROIs. We find that deconvolution enhances the difference in GP-values between the plasma and intracellular membranes and demonstrate that moving an intensity defined plasma membrane ROI outwards from the cell further improves this differentiation. By systematically changing the key deconvolution regularization parameter signal to noise, we establish a protocol for deconvolution optimisation applicable to any solvatochromic dye and imaging system. The image processing and ROI selection protocol presented improves both the resolution and precision of GP-measurement and will enable detection of smaller changes in membrane order than is currently achievable.     

Deconvolution of confocal images of dihydropyridine and ryanodine receptors in developing cardiomyocytes.

Colocalization of dihydropyridine (DHPR) and ryanodine (RyR) receptors, a key determinant of Ca(2+)-induced Ca2+ release, was previously estimated in 3-, 6-, 10-, and 20-day-old rabbit ventricular myocytes by immunocytochemistry and confocal microscopy. We now report on the effects of deconvolution (using a maximum-likelihood estimation algorithm) on the calculation of colocalization indexes. Clusters of DHPR and RyR can be accurately represented as point sources of fluorescence, which enables a model of their relative distributions to be constructed using images of point spread functions to simulate their fluorescence inside a cell. This model was used to investigate the effects of deconvolution on colocalization as a function of separation distance. Deconvolution resulted in significant improvements in both axial and transverse resolutions, producing significant increases in clarity. Comparisons of intensity profiles (full-width half-maximum) pre- and postdeconvolution showed decreased dispersion of the fluorescent signal and a corresponding decrease in false colocalization as determined by fluorescence modeling. This hypothesis was extended to physiological data previously collected. The number of colocalized voxels was quantified after deconvolution, and the degree of colocalization of DHPR with RyR decreased significantly after deconvolution in all age groups: 3 days (62 +/- 2% before deconvolution, 43 +/- 3 after deconvolution) to 20 days old (79 +/- 1% before deconvolution, 63 +/- 2% after deconvolution). The data demonstrate that confocal images should be deconvolved before any quantitative analysis, such as colocalization index determination, to minimize the detrimental effects of out-of-focus light in coincident voxels.     

A new deep learning method for image deblurring in optical microscopic systems

Deconvolution is the most commonly used image processing method in optical imaging systems to remove the blur caused by the point‐spread function (PSF). While this method has been successful in deblurring, it suffers from several disadvantages, such as slow processing time due to multiple iterations required to deblur and suboptimal in cases where the experimental operator chosen to represent PSF is not optimal. In this paper, we present a deep‐learning‐based deblurring method that is fast and applicable to optical microscopic imaging systems. We tested the robustness of proposed deblurring method on the publicly available data, simulated data and experimental data (including 2D optical microscopic data and 3D photoacoustic microscopic data), which all showed much improved deblurred results compared to deconvolution. We compared our results against several existing deconvolution methods. Our results are better than conventional techniques and do not require multiple iterations or pre‐determined experimental operator. Our method has several advantages including simple operation, short time to compute, good deblur results and wide application in all types of optical microscopic imaging systems. The deep learning approach opens up a new path for deblurring and can be applied in various biomedical imaging fields.     

Compensated Row-Column Ultrasound Imaging System Using Fisher Tippett Multilayered Conditional Random Field Model

3-D ultrasound imaging offers unique opportunities in the field of non destructive testing that cannot be easily found in A-mode and B-mode images. To acquire a 3-D ultrasound image without a mechanically moving transducer, a 2-D array can be used. The row column technique is preferred over a fully addressed 2-D array as it requires a significantly lower number of interconnections. Recent advances in 3-D row-column ultrasound imaging systems were largely focused on sensor design. However, these imaging systems face three intrinsic challenges that cannot be addressed by improving sensor design alone: speckle noise, sparsity of data in the imaged volume, and the spatially dependent point spread function of the imaging system. In this paper, we propose a compensated row-column ultrasound image reconstruction system using Fisher-Tippett multilayered conditional random field model. Tests carried out on both simulated and real row-column ultrasound images show the effectiveness of our proposed system as opposed to other published systems. Visual assessment of the results show our proposed system’s potential at preserving detail and reducing speckle. Quantitative analysis shows that our proposed system outperforms previously published systems when evaluated with metrics such as Peak Signal to Noise Ratio, Coefficient of Correlation, and Effective Number of Looks. These results show the potential of our proposed system as an effective tool for enhancing 3-D row-column imaging.     

High-definition mapping of neural activity using voltage-sensitive dyes

The distribution of patterns of activity in different brain structures has been related to the encoding and processing of sensory information. Consequently, it is important to be able to image the distribution of these patterns to understand basic brain functions. The spatial resolution of voltage-sensitive dye (VSD) methods has recently been enhanced considerably by the use of video imaging techniques. The main factor that now hampers the resolution of VSD patterns is the inherent limitation of the optical systems. Unfortunately, the intrinsic characteristics of VSD images impose important limitations that restrict the use of general deconvolution techniques. To overcomes this problem, in this study an image restoration procedure has been implemented that takes into consideration the limiting characteristics of VSD signals. This technique is based on applying a set of imaging processing steps. First, the signal-to-noise (S/N) ratio of the images was improved to avoid an increase in the noise levels during the deconvolution procedures. For this purpose, a new filter technique was implemented that yielded better results than other methods currently used in optical imaging. Second, focal plane images were deconvolved using a modification of the well-known nearest-neighbor deconvolution algorithm. But to reduce the light exposure of the preparation and simplify image acquisition procedures, adjacent image planes were modeled according to the in-focus image planes and the empirical point spread function (PSF) profiles. Third, resulting focal plane responses were processed to reduce the contribution of optical responses that originate in distant image planes. This method was found to be satisfactory under simulated and real experimental conditions. By comparing the restored and unprocessed images, it was clearly demonstrated that this method can effectively remove the out-of-focus artifacts and produce focal plane images of better quality. Evaluations of the tissue optical properties allowed assessment of the maximum practical optical section thickness using this deconvolution technique in the optical system tested. Determination of the three-dimensional PSF permitted the correct application of deconvolution algorithms and the removal of the contaminating light arising from adjacent as well as distant optical planes. The implementation of this deconvolution approach in salamander olfactory bulb allowed the detailed study of the laminar distribution of voltage-sensitive changes across the bulb layer. It is concluded that (1) this deconvolution procedure is well suited to deconvolved low-contrast images and offers important advantages over other alternatives; (2) this method can be properly used only when the tissue optical properties are first determined; (3) high levels of light scattering in the tissue reduce the optical section capabilities of this technique as well as other deconvolution procedures; and (4) use of the highest numerical aperture in the objectives is advisable because this improves not only the light-collecting efficiency to detect poor-contrast images, but also the spatial frequency differences between adjacent image planes. Under this condition it is possible to overcome some of the limitations imposed by the light scattering/birefringence of the tissue.     

Fast Second Degree Total Variation Method for Image Compressive Sensing

This paper presents a computationally efficient algorithm for image compressive sensing reconstruction using a second degree total variation (HDTV2) regularization. Firstly, a preferably equivalent formulation of the HDTV2 functional is derived, which can be formulated as a weighted L ₁-L ₂ mixed norm of second degree image derivatives under the spectral decomposition framework. Secondly, using the equivalent formulation of HDTV2, we introduce an efficient forward-backward splitting (FBS) scheme to solve the HDTV2-based image reconstruction model. Furthermore, from the averaged non-expansive operator point of view, we make a detailed analysis on the convergence of the proposed FBS algorithm. Experiments on medical images demonstrate that the proposed method outperforms several fast algorithms of the TV and HDTV2 reconstruction models in terms of peak signal to noise ratio (PSNR), structural similarity index (SSIM) and convergence speed.     

Chromatin investigation in the nucleus using a phasor approach to structured illumination microscopy

Chromatin in the nucleus is organized in functional sites at variable level of compaction. Structured illumination microscopy (SIM) can be used to generate three-dimensional super-resolution (SR) imaging of chromatin by changing in phase and in orientation a periodic line illumination pattern. The spatial frequency domain is the natural choice to process SIM raw data and to reconstruct an SR image. Using an alternative approach, we demonstrate that the additional spatial information encoded in the knowledge of the position of the illumination pattern can be efficiently decoded using a generalized version of separation of photon by lifetime tuning (SPLIT) that does not require lifetime measurements. In the resulting SPLIT-SIM, the SR image is obtained by isolating a fraction of the intensity corresponding to the center of the diffraction-limited point spread function. This extends the use of the SPLIT approach from stimulated emission depletion microscopy to SIM. The SPLIT-SIM algorithm is based only on phasor analysis and does not require deconvolution. We show that SPLIT-SIM can be used to generate SR images of chromatin organizational motifs with tunable resolution and can be a valuable tool for the imaging of functional sites in the nucleus.     

Small-animal imaging using clinical positron emission tomography/computed tomography and super-resolution

Considering the high cost of dedicated small-animal positron emission tomography/computed tomography (PET/CT), an acceptable alternative in many situations might be clinical PET/CT. However, spatial resolution and image quality are of concern. The utility of clinical PET/CT for small-animal research and image quality improvements from super-resolution (spatial subsampling) were investigated. National Electrical Manufacturers Association (NEMA) NU 4 phantom and mouse data were acquired with a clinical PET/CT scanner, as both conventional static and stepped scans. Static scans were reconstructed with and without point spread function (PSF) modeling. Stepped images were postprocessed with iterative deconvolution to produce super-resolution images. Image quality was markedly improved using the super-resolution technique, avoiding certain artifacts produced by PSF modeling. The 2 mm rod of the NU 4 phantom was visualized with high contrast, and the major structures of the mouse were well resolved. Although not a perfect substitute for a state-of-the-art small-animal PET/CT scanner, a clinical PET/CT scanner with super-resolution produces acceptable small-animal image quality for many preclinical research studies.     

Enhanced temporal and spatial resolution in super-resolution covariance imaging algorithm with deconvolution optimization

Based on the numerical analysis that covariance exhibits superior statistical precision than cumulant and variance, a new SOFI algorithm by calculating the n orders covariance for each pixel is presented with an almost -fold resolution improvement, which can be enhanced to 2ⁿ via deconvolution. An optimized deconvolution is also proposed by calculating the (n + 1) order SD associated with each n order covariance pixel, and introducing the results into the deconvolution as a damping factor to suppress noise generation. Moreover, a re-deconvolution of the covariance image with the covariance-equivalent point spread function is used to further increase the final resolution by above 2-fold. Simulated and experimental results show that this algorithm can significantly increase the temporal–spatial resolution of SOFI, meanwhile, preserve the sample's structure. Thus, a resolution of 58 nm is achieved for 20 experimental images, and the corresponding acquisition time is 0.8 seconds.     

Noninvasive prediction of vertebral body compressive strength using nonlinear finite element method and an image based technique

Noninvasive prediction of vertebral body strength under compressive loading condition is a valuable tool for the assessment of clinical fractures. This paper presents an effective specimen-specific approach for noninvasive prediction of human vertebral strength using a nonlinear finite element (FE) model and an image based parameter based on the quantitative computed tomography (QCT). Nine thoracolumbar vertebrae excised from three cadavers with an average age of 42 years old were used as the samples. The samples were scanned using the QCT. Then, a segmentation technique was performed on each QCT sectional image. The segmented images were then converted into three-dimensional FE models for linear and nonlinear analyses. A new material model was implemented in our nonlinear model being more compatible with real mechanical behavior of trabecular bone. A new image based MOS (Mechanic of Solids) parameter named minimum sectional strength ((σ_uA)_min) was used for the ultimate compressive strength prediction. Subsequently, the samples were destructively tested under uniaxial compression and their experimental ultimate compressive strengths were obtained. Results indicated that our new implemented FE model can predict ultimate compressive strength of human vertebra with a correlation coefficient (R² = 0.94) better than usual linear and nonlinear FE models (R² = 0.83 and 0.85 respectively). The image based parameter introduced in this study ((σ_uA)_min) was also correlated well with the experimental results (R² = 0.86). Although nonlinear FE method with new implemented material model predicts compressive strength better than the (σ_uA)_min, this parameter is clinically more feasible due to its simplicity and lower computational costs. This can make future applications of the (σ_uA)_min more justified for human vertebral body compressive strength prediction.     

Spread of Q fever within dairy cattle herds: key parameters inferred using a Bayesian approach

Q fever is a worldwide zoonosis caused by Coxiella burnetii. Although ruminants are recognized as the most important source of human infection, no previous studies have focused on assessing the characteristics of the bacterial spread within a cattle herd and no epidemic model has been proposed in this context. We assess the key epidemiological parameters from field data in a Bayesian framework that takes into account the available knowledge, missing data and the uncertainty of the observation process owing to the imperfection of diagnostic tests. We propose an original individual-based Markovian model in discrete time describing the evolution of the infection for each animal. Markov chain Monte Carlo methodology is used to estimate parameters of interest from data consisting of individual health states of 217 cows of five chronically infected dairy herds sampled every week for a four-week period. Outputs are the posterior distributions of the probabilities of transition between health states and of the environmental bacterial load. Our findings show that some herds are characterized by a very low infection risk while others have a mild infection risk and a non-negligible intermittent shedding probability. Moreover, the antibody status seems to be a key point in the bacterial spread (shedders with antibodies shed for a longer period of time than shedders without antibodies). In addition to the biological insights, these estimates also provide information for calibrating simulation models to assess control strategies for C. burnetii infection.     

Semi-Supervised Segmentation of Ultrasound Images Based on Patch Representation and Continuous Min Cut

Ultrasound segmentation is a challenging problem due to the inherent speckle and some artifacts like shadows, attenuation and signal dropout. Existing methods need to include strong priors like shape priors or analytical intensity models to succeed in the segmentation. However, such priors tend to limit these methods to a specific target or imaging settings, and they are not always applicable to pathological cases. This work introduces a semi-supervised segmentation framework for ultrasound imaging that alleviates the limitation of fully automatic segmentation, that is, it is applicable to any kind of target and imaging settings. Our methodology uses a graph of image patches to represent the ultrasound image and user-assisted initialization with labels, which acts as soft priors. The segmentation problem is formulated as a continuous minimum cut problem and solved with an efficient optimization algorithm. We validate our segmentation framework on clinical ultrasound imaging (prostate, fetus, and tumors of the liver and eye). We obtain high similarity agreement with the ground truth provided by medical expert delineations in all applications (94% DICE values in average) and the proposed algorithm performs favorably with the literature.     

用去卷积法提高超声调制光学成像的空间分辨率

超声调制光学成像的空间分辨率取决于光在组织中的散射程度和扫描超声束的聚焦大小。由于组织是强散射介质,实际应用中的超声束都有一定的聚焦宽度(通常是毫米数量级),所以该技术成像空间分辨率一直无法提高。针对这个问题,首次将去卷积图像处理法运用在超声调制光学成像技术中,有效地解决了扫描超声束带来的信号展开,分辨率下降的影响。理论和防真结果表明,处理后的成像分辨率大大提高,图像质量明显改善。该方法无须对系统装置做任何改动,只利用适当的数据处理,就实现了成像超分辨,具有应用价值。     

On impulse response functions computed from dynamic contrast-enhanced image data by algebraic deconvolution and compartmental modeling

Concentration-time courses measured by dynamic contrast-enhanced (DCE) imaging can be described by a convolution of the arterial input with an impulse response function, Q_T(t), characterizing tissue microcirculation. Data analysis is based on two different approaches: computation of Q_T(t) by algebraic deconvolution (AD) and subsequent evaluation according to the indicator dilution theory (IDT) or parameterization of Q_T(t) by analytical expressions derived by compartmental modeling. Pitfalls of both strategies will be addressed in this study.Tissue data acquired by DCE-CT in patients with head-and-neck cancer and simulated by a reference model (MMID4) were analyzed by a two-compartment model (TCM), a permeability-limited two-compartment model (PL-TCM) and AD. Additionally, MMID4 was used to compute the ‘true’ response function that corresponds to the simulated tumor data.TCM and AD yielded accurate fits, whereas PL-TCM performed worse. Nevertheless, the corresponding response functions diverge markedly. The response curves obtained by TCM decrease exponentially in the early perfusion phase and overestimate the tissue perfusion, Q_T(0). AD also resulted in response curves starting with a negative slope and not – as the ‘true’ response function in accordance with the IDT – with a horizontal plateau. They are thus not valid responses in the sense of the IDT that can be used unconditionally for parameter estimation.Response functions differing considerably in shape can result in virtually identical tissue curves. This non-uniqueness makes a strong argument not to use algebraic but rather analytical deconvolution to reduce the class of solutions to representatives that are in accordance with a-priori knowledge. To avoid misinterpretations and systematic errors, users must be aware of the pitfalls inherent to the different concepts.     

Sparse Solution of Fiber Orientation Distribution Function by Diffusion Decomposition

Fiber orientation is the key information in diffusion tractography. Several deconvolution methods have been proposed to obtain fiber orientations by estimating a fiber orientation distribution function (ODF). However, the L ₂ regularization used in deconvolution often leads to false fibers that compromise the specificity of the results. To address this problem, we propose a method called diffusion decomposition, which obtains a sparse solution of fiber ODF by decomposing the diffusion ODF obtained from q-ball imaging (QBI), diffusion spectrum imaging (DSI), or generalized q-sampling imaging (GQI). A simulation study, a phantom study, and an in-vivo study were conducted to examine the performance of diffusion decomposition. The simulation study showed that diffusion decomposition was more accurate than both constrained spherical deconvolution and ball-and-sticks model. The phantom study showed that the angular error of diffusion decomposition was significantly lower than those of constrained spherical deconvolution at 30° crossing and ball-and-sticks model at 60° crossing. The in-vivo study showed that diffusion decomposition can be applied to QBI, DSI, or GQI, and the resolved fiber orientations were consistent regardless of the diffusion sampling schemes and diffusion reconstruction methods. The performance of diffusion decomposition was further demonstrated by resolving crossing fibers on a 30-direction QBI dataset and a 40-direction DSI dataset. In conclusion, diffusion decomposition can improve angular resolution and resolve crossing fibers in datasets with low SNR and substantially reduced number of diffusion encoding directions. These advantages may be valuable for human connectome studies and clinical research.     

Estimation of organ transport function: model-free deconvolution by recursive quadratic programming optimization.

A model-free deconvolution method is proposed for evaluating the frequency distribution function of organ transit times. The deconvolution is treated as a nonlinear constrained optimization problem and it is solved by using a modified constrained variable metric approach. The only constraint implemented in the solution is that all the discrete transport function values are not allowed to become negative. The method is tested on model mathematical systems of known analytical transport functions. The tests are performed on systems that included noise in both the input and output functions. The criteria of successful deconvolution are the reconvolution error and, most importantly, the deviation of the computed transport function from the known analytical one. The proposed method is then applied, as a pilot experiment, to biological data obtained from an isolated, perfused rabbit lung preparation contained within a plethysmograph. The results indicate that this type of deconvolution produces stable estimates which faithfully follow the analytical function while negating the need to assume either any functional form for the behavior of the transport function or any educated initial guess of its values.     

ConfocalCheck - A Software Tool for the Automated Monitoring of Confocal Microscope Performance

Laser scanning confocal microscopy has become an invaluable tool in biomedical research but regular quality testing is vital to maintain the system’s performance for diagnostic and research purposes.Although many methods have been devised over the years to characterise specific aspects of a confocal microscope like measuring the optical point spread function or the field illumination, only very few analysis tools are available. Our aim was to develop a comprehensive quality assurance framework ranging from image acquisition to automated analysis and documentation. We created standardised test data to assess the performance of the lasers, the objective lenses and other key components required for optimum confocal operation.The ConfocalCheck software presented here analyses the data fully automatically. It creates numerous visual outputs indicating potential issues requiring further investigation. By storing results in a web browser compatible file format the software greatly simplifies record keeping allowing the operator to quickly compare old and new data and to spot developing trends.We demonstrate that the systematic monitoring of confocal performance is essential in a core facility environment and how the quantitative measurements obtained can be used for the detailed characterisation of system components as well as for comparisons across multiple instruments.     

三维反卷积显微成像技术浅谈

三维宽场反卷积显微成像技术是应用光学切片方法获取三维标本的二维图像序列,然后通过反卷积图像处理方法进行图像恢复,进而进行三纺重建的一种以光学技术和图像处理技术为核心的业微成橡方法。本讲述了光学切片的基本原理,给出了反卷积处理中点扩展函数的理论模型和实验测试方法,然后对现存的反卷积算法做了对比。对这一领域的发展趋势做了预测。