Robust single-particle tracking in live-cell time-lapse sequences

Single-particle tracking (SPT) is often the rate-limiting step in live-cell imaging studies of subcellular dynamics. Here we present a tracking algorithm that addresses the principal challenges of SPT, namely high particle density, particle motion heterogeneity, temporary particle disappearance, and particle merging and splitting. The algorithm first links particles between consecutive frames and then links the resulting track segments into complete trajectories. Both steps are formulated as global combinatorial optimization problems whose solution identifies the overall most likely set of particle trajectories throughout a movie. Using this approach, we show that the GTPase dynamin differentially affects the kinetics of long- and short-lived endocytic structures and that the motion of CD36 receptors along cytoskeleton-mediated linear tracks increases their aggregation probability. Both applications indicate the requirement for robust and complete tracking of dense particle fields to dissect the mechanisms of receptor organization at the level of the plasma membrane.     

Tracking of Ball and Players in Beach Volleyball Videos

This paper presents methods for the determination of players'' positions and contact time points by tracking the players and the ball in beach volleyball videos. Two player tracking methods are compared, a classical particle filter and a rigid grid integral histogram tracker. Due to mutual occlusion of the players and the camera perspective, results are best for the front players, with 74,6% and 82,6% of correctly tracked frames for the particle method and the integral histogram method, respectively. Results suggest an improved robustness against player confusion between different particle sets when tracking with a rigid grid approach. Faster processing and less player confusions make this method superior to the classical particle filter. Two different ball tracking methods are used that detect ball candidates from movement difference images using a background subtraction algorithm. Ball trajectories are estimated and interpolated from parabolic flight equations. The tracking accuracy of the ball is 54,2% for the trajectory growth method and 42,1% for the Hough line detection method. Tracking results of over 90% from the literature could not be confirmed. Ball contact frames were estimated from parabolic trajectory intersection, resulting in 48,9% of correctly estimated ball contact points.     

Online Multi-Modal Robust Non-Negative Dictionary Learning for Visual Tracking

Dictionary learning is a method of acquiring a collection of atoms for subsequent signal representation. Due to its excellent representation ability, dictionary learning has been widely applied in multimedia and computer vision. However, conventional dictionary learning algorithms fail to deal with multi-modal datasets. In this paper, we propose an online multi-modal robust non-negative dictionary learning (OMRNDL) algorithm to overcome this deficiency. Notably, OMRNDL casts visual tracking as a dictionary learning problem under the particle filter framework and captures the intrinsic knowledge about the target from multiple visual modalities, e.g., pixel intensity and texture information. To this end, OMRNDL adaptively learns an individual dictionary, i.e., template, for each modality from available frames, and then represents new particles over all the learned dictionaries by minimizing the fitting loss of data based on M-estimation. The resultant representation coefficient can be viewed as the common semantic representation of particles across multiple modalities, and can be utilized to track the target. OMRNDL incrementally learns the dictionary and the coefficient of each particle by using multiplicative update rules to respectively guarantee their non-negativity constraints. Experimental results on a popular challenging video benchmark validate the effectiveness of OMRNDL for visual tracking in both quantity and quality.     

Automated detection and tracking of individual and clustered cell surface low density lipoprotein receptor molecules.

We have developed a technique to detect, recognize, and track each individual low density lipoprotein receptor (LDL-R) molecule and small receptor clusters on the surface of human skin fibroblasts. Molecular recognition and high precision (30 nm) simultaneous automatic tracking of all of the individual receptors in the cell surface population utilize quantitative time-lapse low light level digital video fluorescence microscopy analyzed by purpose-designed algorithms executed on an image processing work station. The LDL-Rs are labeled with the biologically active, fluorescent LDL derivative dil-LDL. Individual LDL-Rs and unresolved small clusters are identified by measuring the fluorescence power radiated by the sub-resolution fluorescent spots in the image; identification of single particles is ascertained by four independent techniques. An automated tracking routine was developed to track simultaneously, and without user intervention, a multitude of fluorescent particles through a sequence of hundreds of time-lapse image frames. The limitations on tracking precision were found to depend on the signal-to-noise ratio of the tracked particle image and mechanical drift of the microscope system. We describe the methods involved in (i) time-lapse acquisition of the low-light level images, (ii) simultaneous automated tracking of the fluorescent diffraction limited punctate images, (iii) localizing particles with high precision and limitations, and (iv) detecting and identifying single and clustered LDL-Rs. These methods are generally applicable and provide a powerful tool to visualize and measure dynamics and interactions of individual integral membrane proteins on living cell surfaces.     

Auto-accumulation method using simulated annealing enables fully automatic particle pickup completely free from a matching template or learning data

Single-particle analysis is a 3-D structure determining method using electron microscopy (EM). In this method, a large number of projections is required to create 3-D reconstruction. In order to enable completely automatic pickup without a matching template or a training data set, we established a brand-new method in which the frames to pickup particles are randomly shifted and rotated over the electron micrograph and, using the total average image of the framed images as an index, each frame reaches a particle. In this process, shifts are selected to increase the contrast of the average. By iterated shifts and further selection of the shifts, the frames are induced to shift so as to surround particles. In this algorithm, hundreds of frames are initially distributed randomly over the electron micrograph in which multi-particle images are dispersed. Starting with these frames, one of them is selected and shifted randomly, and acceptance or non-acceptance of its new position is judged using the simulated annealing (SA) method in which the contrast score of the total average image is adopted as an index. After iteration of this process, the position of each frame converges so as to surround a particle and the framed images are picked up. This method is the first unsupervised fully automatic particle picking method which is applicable to EM of various kinds of proteins, especially to low-contrasted cryo-EM protein images.     

High-throughput ballistic injection nanorheology to measure cell mechanics

High-throughput ballistic injection nanorheology is a method for the quantitative study of cell mechanics. Cell mechanics are measured by ballistic injection of submicron particles into the cytoplasm of living cells and tracking the spontaneous displacement of the particles at high spatial resolution. The trajectories of the cytoplasm-embedded particles are transformed into mean-squared displacements, which are subsequently transformed into frequency-dependent viscoelastic moduli and time-dependent creep compliance of the cytoplasm. This method allows for the study of a wide range of cellular conditions, including cells inside a 3D matrix, cell subjected to shear flows and biochemical stimuli, and cells in a live animal. Ballistic injection lasts <1 min and is followed by overnight incubation. Multiple particle tracking for one cell lasts <1 min. Forty cells can be examined in <1 h.     

Cell-permeable peptide antioxidants targeted to inner mitochondrial membrane inhibit mitochondrial swelling, oxidative cell death, and reperfusion injury

Reactive oxygen species (ROS) play a key role in promoting mitochondrial cytochrome c release and induction of apoptosis. ROS induce dissociation of cytochrome c from cardiolipin on the inner mitochondrial membrane (IMM), and cytochrome c may then be released via mitochondrial permeability transition (MPT)-dependent or MPT-independent mechanisms. We have developed peptide antioxidants that target the IMM, and we used them to investigate the role of ROS and MPT in cell death caused by t-butylhydroperoxide (tBHP) and 3-nitropropionic acid (3NP). The structural motif of these peptides centers on alternating aromatic and basic amino acid residues, with dimethyltyrosine providing scavenging properties. These peptide antioxidants are cell-permeable and concentrate 1000-fold in the IMM. They potently reduced intracellular ROS and cell death caused by tBHP in neuronal N(2)A cells (EC(50) in nm range). They also decreased mitochondrial ROS production, inhibited MPT and swelling, and prevented cytochrome c release induced by Ca(2+) in isolated mitochondria. In addition, they inhibited 3NP-induced MPT in isolated mitochondria and prevented mitochondrial depolarization in cells treated with 3NP. ROS and MPT have been implicated in myocardial stunning associated with reperfusion in ischemic hearts, and these peptide antioxidants potently improved contractile force in an ex vivo heart model. It is noteworthy that peptide analogs without dimethyltyrosine did not inhibit mitochondrial ROS generation or swelling and failed to prevent myocardial stunning. These results clearly demonstrate that overproduction of ROS underlies the cellular toxicity of tBHP and 3NP, and ROS mediate cytochrome c release via MPT. These IMM-targeted antioxidants may be very beneficial in the treatment of aging and diseases associated with oxidative stress.     

Inhibition of mitochondrial membrane bound-glutathione transferase by mitochondrial permeability transition inhibitors including cyclosporin A

AimsEffect of mitochondrial permeability transition (MPT) inhibitors on mitochondrial membrane-bound glutathione transferase (mtMGST1) activity in rat liver was investigated in vitro.Main methodsWhen mitochondria were incubated with MPT inhibitors, mtMGST1 activity was decreased dose dependently and their 50% inhibition concentration (IC₅₀) were 1.2 μM (cyclosporin A; CsA), 31 μM (bongkrekic acid; BKA), 1.8 mM (ADP), and 3.2 mM (ATP). The decrease of mtMGST1 activity by the MPT inhibitors was not observed in the presence of detergent Triton X-100. On the contrary, mtMGST1 inhibition by GST inhibitors such as cibacron blue (IC₅₀, 4.2 μM) and S-hexylglutathione (IC₅₀, 480 μM) was not affected in the presence of detergent. Although mtMGST1 resides in both the inner (IMM) and outer mitochondrial membranes (OMM), only mtMGST1 in the IMM was inhibited by the MPT inhibitors in the absence of detergent. GST inhibitors decreased mtMGST1 activity both in the IMM and OMM regardless of the presence or absence of detergent. Cytosolic GSTs and microsomal MGST1 were not inhibited by the MPT inhibitors.Key findingsThese results indicate that mtMGST1 is inhibited by MPT inhibitors through membrane components, not directly by the inhibitors.SignificanceSince CsA binds to cyclophilin D (Cyp-D) in the mitochondrial matrix whereas BKA or ADP binds to adenine nucleotide translocator (ANT) in the IMM, it was suggested that mtMGST1 in the IMM interacts with Cyp-D/ANT and the binding of MPT inhibitors to Cyp-D or ANT causes their conformational change followed by an alteration of mtMGST1 conformation, resulting in decreasing mtMGST1 activity.     

Enhanced viscoelasticity of human cystic fibrotic sputum correlates with increasing microheterogeneity in particle transport

Current biochemical characterizations of cystic fibrosis (CF) sputum do not address the high degree of microheterogeneity in the rheological properties of the mucosal matrix and only provide bulk-average particle diffusion coefficients. The viscoelasticity of CF sputum greatly reduces the diffusion rates of colloidal particles, limiting the effectiveness of gene delivery to underlying lung cells. We determine diffusion coefficients of hundreds of individual amine-modified and carboxylated polystyrene particles (diameter 100-500 nm) embedded in human CF sputum with 5 nm and 33 ms of spatiotemporal resolution. High resolution multiple particle tracking is used to calculate the effective viscoelastic properties of CF sputum at the micron scale, which we relate to its macroscopic viscoelasticity. CF sputum microviscosity, as probed by 100- and 200-nm particles, is an order of magnitude lower than its macroviscosity, suggesting that nanoparticles dispersed in CF sputum are transported primarily through lower viscosity pores within a highly elastic matrix. Multiple particle tracking provides a non-destructive, highly sensitive method to quantify the high heterogeneity of the mucus pore network. The mean diffusion coefficient becomes dominated by relatively few but fast-moving particles as particle size is reduced from 500 to 100 nm. Neutrally charged particles with a diameter <200 nm undergo more rapid transport in CF sputum than charged particles. Treatment with recombinant human DNase (Pulmozyme) reduces macroviscoelastic properties of CF sputum by up to 50% and dramatically narrows the distribution of individual particle diffusion rates but surprisingly does not significantly alter the ensemble-average particle diffusion rate.     

Analysis of Video-Based Microscopic Particle Trajectories Using Kalman Filtering

The fidelity of the trajectories obtained from video-based particle tracking determines the success of a variety of biophysical techniques, including in situ single cell particle tracking and in vitro motility assays. However, the image acquisition process is complicated by system noise, which causes positioning error in the trajectories derived from image analysis. Here, we explore the possibility of reducing the positioning error by the application of a Kalman filter, a powerful algorithm to estimate the state of a linear dynamic system from noisy measurements. We show that the optimal Kalman filter parameters can be determined in an appropriate experimental setting, and that the Kalman filter can markedly reduce the positioning error while retaining the intrinsic fluctuations of the dynamic process. We believe the Kalman filter can potentially serve as a powerful tool to infer a trajectory of ultra-high fidelity from noisy images, revealing the details of dynamic cellular processes.     

Classical detection theory and the cryo-EM particle selection problem

Particle selection is an essential but tedious step in the determination of macromolecular structures by single particle reconstruction. This paper presents an automatic, multi-reference particle detection scheme that is based on the classical matched filter principle. It makes use of a pre-whitening filter to standardize the noise, a reduced representation of the references by means of principal component analysis, and a statistic for distinguishing particles from image artifacts. Standardizing the noise allows the noise-induced false-positive frequency to be estimated, and also allows the distribution of the discrimination statistic to be calculated a priori. The method is demonstrated with an annotated dataset of cryo-EM images.     

Performance of deep learning restoration methods for the extraction of particle dynamics in noisy microscopy image sequences

Particle tracking in living systems requires low light exposure and short exposure times to avoid phototoxicity and photobleaching and to fully capture particle motion with high-speed imaging. Low-excitation light comes at the expense of tracking accuracy. Image restoration methods based on deep learning dramatically improve the signal-to-noise ratio in low-exposure data sets, qualitatively improving the images. However, it is not clear whether images generated by these methods yield accurate quantitative measurements such as diffusion parameters in (single) particle tracking experiments. Here, we evaluate the performance of two popular deep learning denoising software packages for particle tracking, using synthetic data sets and movies of diffusing chromatin as biological examples. With synthetic data, both supervised and unsupervised deep learning restored particle motions with high accuracy in two-dimensional data sets, whereas artifacts were introduced by the denoisers in three-dimensional data sets. Experimentally, we found that, while both supervised and unsupervised approaches improved tracking results compared with the original noisy images, supervised learning generally outperformed the unsupervised approach. We find that nicer-looking image sequences are not synonymous with more precise tracking results and highlight that deep learning algorithms can produce deceiving artifacts with extremely noisy images. Finally, we address the challenge of selecting parameters to train convolutional neural networks by implementing a frugal Bayesian optimizer that rapidly explores multidimensional parameter spaces, identifying networks yielding optimal particle tracking accuracy. Our study provides quantitative outcome measures of image restoration using deep learning. We anticipate broad application of this approach to critically evaluate artificial intelligence solutions for quantitative microscopy.     

Comparison of the phagocytosis of two types of cyclosporin (SDZ OXL 400 and SDZ IMM 125) by alveolar macrophages from hamsters

The aim of this study was to compare two types of cyclosporin (Cs) particles, SDZ OXL 400 and SDZ IMM 125, the latter being more hydrophilic, to understand their uptake by airway macrophages. Alveolar macrophages (AM), harvested by bronchoalveolar lavage (BAL) of hamster lungs, were cultured with two different doses (0.1 mg and 0.5 mg) for 1 h, 6 h, and 24 h. Control incubations without Cs particles or with latex particles were carried out simultaneously. Cell viability, cell activation (i.e., respiratory burst, interleukin-6 (IL-6) synthesis) and mean volume of particles phagocytosed per macrophage were measured. Both types of Cs particles did not modify the AM viability, and failed to induce IL-6 synthesis during phagocytosis but slightly decreased the cell oxidative respiratory burst. The comparison between SDZ OXL 400 and SDZ IMM 125 showed that for the lower dose the mean volume of both Cs types phagocytosed was similar at 1 h and 6 h. At 24 h an increase of the mean volume phagocytosed was seen for SDZ IMM 125 but not for SDZ OXL 400. For the higher dose the mean volume of SDZ IMM 125 phagocytosed was higher than SDZ OXL 400 at 1 h and 6 h and comparable for both types at 24 h. SDZ IMM 125 particles were phagocytosed more rapidly than SDZ OXL 400. The mean volume of phagocytosed latex particles increased with time and dose and was higher than for both Cs particle types. In conclusion, AM were seen to phagocytose particles of different physical properties (i.e., form, size, and shape), chemical properties (i.e., inert or peptidic) and degrees of hydrophilicity in a different manner.     

Accumulative Difference Image Protocol for Particle Tracking in Fluorescence Microscopy Tested in Mouse Lymphonodes

The basic research in cell biology and in medical sciences makes large use of imaging tools mainly based on confocal fluorescence and, more recently, on non-linear excitation microscopy. Substantially the aim is the recognition of selected targets in the image and their tracking in time. We have developed a particle tracking algorithm optimized for low signal/noise images with a minimum set of requirements on the target size and with no a priori knowledge of the type of motion. The image segmentation, based on a combination of size sensitive filters, does not rely on edge detection and is tailored for targets acquired at low resolution as in most of the in-vivo studies. The particle tracking is performed by building, from a stack of Accumulative Difference Images, a single 2D image in which the motion of the whole set of the particles is coded in time by a color level. This algorithm, tested here on solid-lipid nanoparticles diffusing within cells and on lymphocytes diffusing in lymphonodes, appears to be particularly useful for the cellular and the in-vivo microscopy image processing in which few a priori assumption on the type, the extent and the variability of particle motions, can be done.     

PEGylation of nanoparticles improves their cytoplasmic transport

The efficacy of nucleus-targeted drug- or gene-carrying nanoparticles may be limited by slow transport through the molecularly crowded cytoplasm following endosome escape. Cytoskeletal elements and cellular organelles may pose steric and/or adhesive obstacles to the efficient intracellular transport of nanoparticles. To potentially reduce adhesive interactions of colloids with intracellular components, the surface of model nanoparticles was coated with polyethylene glycol (PEG). Subsequently, multiple-particle tracking (MPT) was used to quantify the cytoplasmic transport rates of particles microinjected into the cytoplasm of live cells. PEGylation increased average nanoparticle diffusivities by 100% compared to unPEGylated particles (time scale of 10 s) in live cells. Faster particle transport correlated with a marked decrease in the number of particles that underwent hindered transport, from 79.2% (unmodified) to 48.8% (PEGylated). This result adds to an impressive list of positive benefits associated with PEGylation of drug and gene delivery vectors.     

Monoclonal antibodies conjugated with superparamagnetic iron oxide particles allow magnetic resonance imaging detection of lymphocytes in the mouse brain

We investigated the potential of antibody-vectorialized superparamagnetic iron oxide (SPIO) particles as cellular specific magnetic resonance contrast agents to image lymphocyte populations within the central nervous system (CNS), with the final goal of obtaining a reliable tool for noninvasively detecting and tracking specific cellular populations in vivo. We used superparamagnetic particles bound to a monoclonal antibody. The particle is the contrast agent, by means of its T?* relaxation properties; the antibody is the targeting vector, responsible for homing the particle to target a surface antigen. To investigate the efficiency of particle vectorialization by these antibodies, we compared two types of antibody-vectorialized CD3-specific particles in vivo. We successfully employed vectorialized SPIO particles to image B220? cells in a murine model of B-cell lymphoma. Likewise, we were able to identify CD3? infiltrates in a murine model of multiple sclerosis. The specificity of the technique was confirmed by immunohistochemistry and electron microscopy of corresponding sections. Our findings suggest that indirect binding of the antibody to a streptavidinated particle allows for enhanced particle vectorialization compared to covalent binding of the antibody to the particle.     

Quantifying Subpixel Accuracy: An Experimental Method for Measuring Accuracy in Image-Correlation-Based, Single-Particle Tracking

Single-particle tracking (SPT) is a range of powerful analysis techniques that measure particle motion from video microscopy image sequences. SPT is used to study the behavior of motor proteins and associated organelle transport within a cell. Many SPT algorithms deliver subpixel accurate measurements with noisy data corresponding to sub-10-nm resolution. Image-correlation techniques have been shown to be the most accurate method of tracking extended objects. However, to date, it has not been possible to determine the level of error when measuring the motion of an arbitrary particle with this method. In this article we derive a method for experimentally determining the accuracy of image-correlation-based SPT. We then apply this technique to a series of confocal fluorescence microscope image sequences of mitochondria, demonstrating the possibility of making measurements accurate to 5 nm when working with extended objects within live cells. In doing so we show that for particles with a low signal/noise ratio, the accuracy can vary by a factor of 2, corresponding to different particle shapes for a given signal/noise ratio. Use of the presented technique will allow researchers to quantify the accuracy of their measurements on a per-particle basis. This in turn will allow the selection of the most accurately tracked particles, helping to push the accuracy of spatial measurements well below the diffraction limit. This is particularly important for the study of molecular motors whose step size is a similar scale to these limits.     

Non-conventional mitochondrial permeability transition: Its regulation by mitochondrial dynamics

Mitochondrial permeability transition (MPT) is a phenomenon that the inner mitochondrial membrane (IMM) loses its selective permeability, leading to mitochondrial dysfunction and cell injury. Electrophysiological evidence indicates the presence of a mega-channel commonly called permeability transition pore (PTP) whose opening is responsible for MPT. However, the molecular identity of the PTP is still under intensive investigations and debates, although cyclophilin D that is inhibited by cyclosporine A (CsA) is the established regulatory component of the PTP. PTP can also open transiently and functions as a rapid mitochondrial Ca²⁺ releasing mechanism. Mitochondrial fission and fusion, the main components of mitochondrial dynamics, control the number and size of mitochondria, and have been shown to play a role in regulating MPT directly or indirectly. Studies by us and others have indicated the potential existence of a form of transient MPT that is insensitive to CsA. This “non-conventional” MPT is regulated by mitochondrial dynamics and may serve a protective role possibly by decreasing the susceptibility for a frequent or sustained PTP opening; hence, it may have a therapeutic value in many disease conditions involving MPT.     

Automatically detecting and tracking free‐ranging Japanese macaques in video recordings with deep learning and particle filters

Recently, automated observation systems for animals using artificial intelligence have been proposed. In the wild, animals are difficult to detect and track automatically because of lamination and occlusions. Our study proposes a new approach to automatically detect and track wild Japanese macaques (Macaca fuscata) using deep learning and a particle filter algorithm. Macaque likelihood is derived through deep learning and used as an observation model in a particle filter to predict the macaques’ position and size in an image. By using deep learning as an observation model, it is possible to simplify the observation model and improve the accuracy of the classifier. We investigated whether the algorithm could find body regions of macaques in video recordings of free‐ranging groups at Katsuyama, Japan to evaluate our model. Experimental results showed that our method with deep learning as an observation model had higher tracking accuracy than a method that uses a support vector machine. More generally, our study will help researchers to develop automatic observation systems for animals in the wild.