Applications of video mixing and digital overlay to neuroethology

Neuroethological experiments often require video images of animal behavior and recordings of physiological data to be acquired simultaneously, synchronized with each other, stored, and analyzed together. The use of inexpensive multimedia computers offers new possibilities for mixing video images, analog voltages, and computer data, storing these combined signals to videotape, and extracting quantitative data for analysis. In this paper, we summarize methods for mixing images from multiple video cameras and a Macintosh computer display to facilitate manipulation of data generated during our neurophysiological and behavioral research. These technologies enhance accuracy, speed, and flexibility during experiments, and facilitate selecting and extracting quantitative data from the videotape for further analysis. Three applications are presented: (A) we used an analog video mixer to synchronize neurophysiological recordings with ongoing behaviors of freely moving rats; (B) we used a chroma keyed digital overlay to generate positional data for the rat's face during drinking behavior; and (C) we combined a computer model of a rat's head and whiskers with videos of exploratory behaviors to better track and quantify movements in three dimensions. Although the applications described here are specific to our neuroethological work, these methods will be useful to anyone wishing to combine the signals from multiple video sources into a single image or to extract series of positional or movement data from video frames without frame grabbing.     

Analysis of the Saccharomyces cerevisiae proteome with PeptideAtlas

We present the Saccharomyces cerevisiae PeptideAtlas composed from 47 diverse experiments and 4.9 million tandem mass spectra. The observed peptides align to 61% of Saccharomyces Genome Database (SGD) open reading frames (ORFs), 49% of the uncharacterized SGD ORFs, 54% of S. cerevisiae ORFs with a Gene Ontology annotation of 'molecular function unknown', and 76% of ORFs with Gene names. We highlight the use of this resource for data mining, construction of high quality lists for targeted proteomics, validation of proteins, and software development.     

Parallel Key Frame Extraction for Surveillance Video Service in a Smart City

Surveillance video service (SVS) is one of the most important services provided in a smart city. It is very important for the utilization of SVS to provide design efficient surveillance video analysis techniques. Key frame extraction is a simple yet effective technique to achieve this goal. In surveillance video applications, key frames are typically used to summarize important video content. It is very important and essential to extract key frames accurately and efficiently. A novel approach is proposed to extract key frames from traffic surveillance videos based on GPU (graphics processing units) to ensure high efficiency and accuracy. For the determination of key frames, motion is a more salient feature in presenting actions or events, especially in surveillance videos. The motion feature is extracted in GPU to reduce running time. It is also smoothed to reduce noise, and the frames with local maxima of motion information are selected as the final key frames. The experimental results show that this approach can extract key frames more accurately and efficiently compared with several other methods.     

Computer-assisted tracking of actin filament motility.

In vitro motility assays, in which fluorescently labeled actin filaments are propelled by myosin molecules adhered to a glass coverslip, require that actin filament velocity be determined. We have developed a computer-assisted filament tracking system that reduced the analysis time, minimized investigator bias, and provided greater accuracy in locating actin filaments in video images. The tracking routine successfully tracked filaments under experimental conditions where filament density, size, and extent of photobleaching varied dramatically. Videotaped images of actin filament motility were digitized and processed to enhance filament image contrast relative to background. Once processed, filament images were cross correlated between frames and a filament path was determined. The changes in filament centroid or center position between video frames were then used to calculate filament velocity. The tracking routine performance was evaluated and the sources of noise that contributed to errors in velocity were identified and quantified. Errors originated in algorithms for filament centroid determination and in the choice of sampling interval between video frames. With knowledge of these error sources, the investigator can maximize the accuracy of the velocity calculation through access to user-definable computer program parameters.     

Optogenetic manipulation of neural activity in freely moving Caenorhabditis elegans

We present an optogenetic illumination system capable of real-time light delivery with high spatial resolution to specified targets in freely moving Caenorhabditis elegans. A tracking microscope records the motion of an unrestrained worm expressing channelrhodopsin-2 or halorhodopsin in specific cell types. Image processing software analyzes the worm's position in each video frame, rapidly estimates the locations of targeted cells and instructs a digital micromirror device to illuminate targeted cells with laser light of the appropriate wavelengths to stimulate or inhibit activity. Because each cell in an unrestrained worm is a rapidly moving target, our system operates at high speed (～50 frames per second) to provide high spatial resolution (～30 μm). To test the accuracy, flexibility and utility of our system, we performed optogenetic analyses of the worm motor circuit, egg-laying circuit and mechanosensory circuits that have not been possible with previous methods.     

Temporal organization of multi-whisker contact in rats

Previous work has established that during exploration and discrimination, rats move their whiskers at frequencies between 6 and 12 Hz and that whisking frequency changes during contact. One critical component of any tactile system is contact. In the rat whisker system, such contacts may involve one or more vibrissa in the whisker array and contact duration of each whisker may vary over a considerable range, depending upon the behavioral context. However, little is known about the variables controlling contact duration or about the temporal relationships among contacts by adjacent whiskers. To address these issues head fixed rats were trained to touch a piezo-contact-sensor with the shaft of their whiskers (Bermejo and Zeigler, Somatosens Mot Res 17: 373-377, 2000 ). During the task, whisker movements and contacts were monitored with a high-speed camera at 500 frames/s and stored on videotape. To facilitate analysis, animals had their whiskers selectively trimmed. Data are reported from animals with C1 & C2, D1 & D2, or Arc2 (E2, D2, C2, B2) whiskers intact. For both row and arc animals, when just a single whisker touched the sensor the duration of contact was significantly shorter than when multiple whiskers made contact. When multiple whiskers made contact, onset was rarely simultaneous. Furthermore, in row-intact animals, contact progressed in an orderly fashion such that the rostral whisker in a row made contact first followed 24 ms (SE = 1.9 ms) later by the caudal whisker. When contact reversed the caudal whisker lifted off first, followed by the rostral whisker. Thus, the order in which whiskers touch an object regulates contact duration: the first whisker to touch the sensor stays in contact longer than any other whisker. The temporal discharge properties of neurons in the trigeminal system are expected to reflect position of whiskers on the nose.     

Temporal organization of multi-whisker contact in rats

Previous work has established that during exploration and discrimination, rats move their whiskers at frequencies between 6 and 12 Hz and that whisking frequency changes during contact. One critical component of any tactile system is contact. In the rat whisker system, such contacts may involve one or more vibrissa in the whisker array and contact duration of each whisker may vary over a considerable range, depending upon the behavioral context. However, little is known about the variables controlling contact duration or about the temporal relationships among contacts by adjacent whiskers. To address these issues head fixed rats were trained to touch a piezo-contact-sensor with the shaft of their whiskers (Bermejo and Zeigler, Somatosens Mot Res 17: 373-377, 2000). During the task, whisker movements and contacts were monitored with a high-speed camera at 500 frames/s and stored on videotape. To facilitate analysis, animals had their whiskers selectively trimmed. Data are reported from animals with C1 & C2, D1 & D2, or Arc2 (E2, D2, C2, B2) whiskers intact. For both row and arc animals, when just a single whisker touched the sensor the duration of contact was significantly shorter than when multiple whiskers made contact. When multiple whiskers made contact, onset was rarely simultaneous. Furthermore, in row-intact animals, contact progressed in an orderly fashion such that the rostral whisker in a row made contact first followed 24 ms (SE = 1.9 ms) later by the caudal whisker. When contact reversed the caudal whisker lifted off first, followed by the rostral whisker. Thus, the order in which whiskers touch an object regulates contact duration: the first whisker to touch the sensor stays in contact longer than any other whisker. The temporal discharge properties of neurons in the trigeminal system are expected to reflect position of whiskers on the nose.     

Imputation of sequence level genotypes in the Franches-Montagnes horse breed

Background

A cost-effective strategy to increase the density of available markers within a population is to sequence a small proportion of the population and impute whole-genome sequence data for the remaining population. Increased densities of typed markers are advantageous for genome-wide association studies (GWAS) and genomic predictions.

Methods

We obtained genotypes for 54 602 SNPs (single nucleotide polymorphisms) in 1077 Franches-Montagnes (FM) horses and Illumina paired-end whole-genome sequencing data for 30 FM horses and 14 Warmblood horses. After variant calling, the sequence-derived SNP genotypes (~13 million SNPs) were used for genotype imputation with the software programs Beagle, Impute2 and FImpute.

Results

The mean imputation accuracy of FM horses using Impute2 was 92.0%. Imputation accuracy using Beagle and FImpute was 74.3% and 77.2%, respectively. In addition, for Impute2 we determined the imputation accuracy of all individual horses in the validation population, which ranged from 85.7% to 99.8%. The subsequent inclusion of Warmblood sequence data further increased the correlation between true and imputed genotypes for most horses, especially for horses with a high level of admixture. The final imputation accuracy of the horses ranged from 91.2% to 99.5%.

Conclusions

Using Impute2, the imputation accuracy was higher than 91% for all horses in the validation population, which indicates that direct imputation of 50k SNP-chip data to sequence level genotypes is feasible in the FM population. The individual imputation accuracy depended mainly on the applied software and the level of admixture.

Electronic supplementary material

The online version of this article (doi:10.1186/s12711-014-0063-7) contains supplementary material, which is available to authorized users.     

A digital image microscopy system for rare-event detection using fluorescent probes

Instrumentation for rare-event analysis should be capable of reliably detecting infrequent cells (less than 1:10,000) while both excluding false-positive signals and including true positive cells found in multicell clumps. We have developed a digital image microscopy (DIM) system in which a cytospin of 2 million cells is scanned with an intensified video camera (ISIT) using an IBM PC AT microcomputer-controlled microscope stage. PASCAL software controls the stage and analyzes video input, storing the location of positive cells to magnetic disk. The user can then "replay" each positive cell under computer control for either visual confirmation or analysis using other fluorescent probes. The computer requires 24 min to scan a cytoprep of 2 million cells, while playback for visual confirmation by the user averages 5 min. Using Hoechst-33342 premarked cells seeded into bone marrow as a model system, we found that the DIM system reliably detects one target cell per million marrow cells. With appropriate immunological markers, this system will aid in evaluating bone marrow purged of tumor cells prior to transplantation and should also be useful for detection of minimal residual disease in blood or bone marrow from patients with leukemia or solid tumors.     

Internet Video Telephony Allows Speech Reading by Deaf Individuals and Improves Speech Perception by Cochlear Implant Users

Objective

To analyze speech reading through Internet video calls by profoundly hearing-impaired individuals and cochlear implant (CI) users.

Methods

Speech reading skills of 14 deaf adults and 21 CI users were assessed using the Hochmair Schulz Moser (HSM) sentence test. We presented video simulations using different video resolutions (1280×720, 640×480, 320×240, 160×120 px), frame rates (30, 20, 10, 7, 5 frames per second (fps)), speech velocities (three different speakers), webcameras (Logitech Pro9000, C600 and C500) and image/sound delays (0–500 ms). All video simulations were presented with and without sound and in two screen sizes. Additionally, scores for live Skype™ video connection and live face-to-face communication were assessed.

Results

Higher frame rate (>7 fps), higher camera resolution (>640×480 px) and shorter picture/sound delay (<100 ms) were associated with increased speech perception scores. Scores were strongly dependent on the speaker but were not influenced by physical properties of the camera optics or the full screen mode. There is a significant median gain of +8.5%pts (p = 0.009) in speech perception for all 21 CI-users if visual cues are additionally shown. CI users with poor open set speech perception scores (n = 11) showed the greatest benefit under combined audio-visual presentation (median speech perception +11.8%pts, p = 0.032).

Conclusion

Webcameras have the potential to improve telecommunication of hearing-impaired individuals.     

TagGD: Fast and Accurate Software for DNA Tag Generation and Demultiplexing

Multiplexing is of vital importance for utilizing the full potential of next generation sequencing technologies. We here report TagGD (DNA-based Tag Generator and Demultiplexor), a fully-customisable, fast and accurate software package that can generate thousands of barcodes satisfying user-defined constraints and can guarantee full demultiplexing accuracy. The barcodes are designed to minimise their interference with the experiment. Insertion, deletion and substitution events are considered when designing and demultiplexing barcodes. 20,000 barcodes of length 18 were designed in 5 minutes and 2 million barcoded Illumina HiSeq-like reads generated with an error rate of 2% were demultiplexed with full accuracy in 5 minutes. We believe that our software meets a central demand in the current high-throughput biology and can be utilised in any field with ample sample abundance. The software is available on GitHub (https://github.com/pelinakan/UBD.git).     

Animal Scanner: Software for classifying humans,animals, and empty frames in camera trap images

Camera traps are a popular tool to sample animal populations because they are noninvasive, detect a variety of species, and can record many thousands of animal detections per deployment. Cameras are typically set to take bursts of multiple photographs for each detection and are deployed in arrays of dozens or hundreds of sites, often resulting in millions of photographs per study. The task of converting photographs to animal detection records from such large image collections is daunting, and made worse by situations that generate copious empty pictures from false triggers (e.g., camera malfunction or moving vegetation) or pictures of humans. We developed computer vision algorithms to detect and classify moving objects to aid the first step of camera trap image filtering—separating the animal detections from the empty frames and pictures of humans. Our new work couples foreground object segmentation through background subtraction with deep learning classification to provide a fast and accurate scheme for human–animal detection. We provide these programs as both Matlab GUI and command prompt developed with C++. The software reads folders of camera trap images and outputs images annotated with bounding boxes around moving objects and a text file summary of results. This software maintains high accuracy while reducing the execution time by 14 times. It takes about 6 seconds to process a sequence of ten frames (on a 2.6 GHZ CPU computer). For those cameras with excessive empty frames due to camera malfunction or blowing vegetation automatically removes 54% of the false‐triggers sequences without influencing the human/animal sequences. We achieve 99.58% on image‐level empty versus object classification of Serengeti dataset. We offer the first computer vision tool for processing camera trap images providing substantial time savings for processing large image datasets, thus improving our ability to monitor wildlife across large scales with camera traps.     

A method to synchronise video cameras using the audio band

This paper proposes and evaluates a novel method for synchronisation of video cameras using the audio band. The method consists in generating and transmitting an audio signal through radio frequency for receivers connected to the microphone input of the cameras and inserting the signal in the audio band. In a software environment, the phase differences among the video signals are calculated and used to interpolate the synchronous 2D projections of the trajectories. The validation of the method was based on: (1) Analysis of the phase difference changes as a function of time of two video signals. (2) Comparison between the values measured with an oscilloscope and by the proposed method. (3) Estimation of the improvement in the accuracy in the measurements of the distance between two markers mounted on a rigid body during movement applying the method. The results showed that the phase difference changes in time slowly (0.150 ms/min) and linearly, even when the same model of cameras are used. The values measured by the proposed method and by oscilloscope showed equivalence (R2=0.998), the root mean square of the difference between the measurements was 0.10 ms and the maximum difference found was 0.31 ms. Applying the new method, the accuracy of the 3D reconstruction had a statistically significant improvement. The accuracy, simplicity and wide applicability of the proposed method constitute the main contributions of this work.     

The video technique developed for measuring epicardial strains on guinea pig's left ventricle

Single-plain video used for measurements of epicardial strains is a technique that yields minor interference with the studied mechanical properties of the ventricle. Due to its low temporal resolution, the existing technique is, however, not appropriate for small animals. We questioned whether the technique could be improved enough to cope with higher heart rates and miniaturization necessary for experiments on rats, mice and guinea pigs. Therefore, we developed a high-speed video system and used it for measuring epicardial strains in guinea pig hearts in situ with the open chest. The improvement was achieved in video hardware (camera: Dalsa D6-0256; framegraber: EPIX PIXCI D32) and software, the markers (glowing acrylate crystals; diameter approximately 0.15 mm) and illumination (UVA light, OSRAM L). Three markers were attached onto the epicardium in the equatorial region of the left ventricular free wall, 1.5 mm apart, with fibrin glue. From their coordinates, we calculated two-dimensional finite strains with end diastole as the reference point. The accuracy of the displacement measurement of the technique and the error introduced by approximate-visual estimation of the left ventricle coordinate system were evaluated. The accuracy of the displacement measurement was +/-1.6 microm and the temporal resolution was 2 ms. Error due to approximate coordinate system orientation was +/-3% of the strain amplitude. The typical amplitude of strains was -0.06, -0.11 and 0.04 in circumferential, axial direction and in-plane shear, respectively. The improvements enable us to perform physiologically relevant measurements of epicardial deformations on guinea pig heart.     

In vivo genomic variability of human T-cell leukemia virus type I depends more upon geography than upon pathologies.

To investigate the geography- and disease-associated genomic variation of human T-cell leukemia virus type I (HTLV-I), we studied ex vivo DNA from peripheral blood lymphocytes from nine patients by polymerase chain reaction and direct DNA sequencing. For each viral strain, 1,917 bp was sequenced, including parts of the long terminal repeat, the env gene, and the px II, px III, and px IV coding frames of the px region. The number of genomic variations observed in the U3 region of the long terminal repeat was higher than that seen in the env and px genes. Very few mutations were present in the px II and px III genes. In contrast, the px IV open reading frame exhibited numerous single point mutations. While no specific mutation could be linked to any pathology (adult T-cell leukemia/lymphoma or tropical spastic paraparesis/HTLV-I-associated myelopathy), variations among HTLV-I isolates from different geographic areas (Ivory Coast, Caribbean, and Japan) existed. The Ivory Coast HTLV-I appeared to represent a group by itself.     

Design considerations and techniques for constructing video stimuli

Techniques for constructing video playback stimuli fall into five categories. The first three involve manipulating video sequences: (1) edited video is a temporal rearrangement of raw footage, (2) processed video applies global filtering algorithms to edited video, and (3) frame-manipulated video involves manually altering individual frames. The last two, (4) exemplar-based animation and (5) parameter-based animation, are synthetic models derived from visual parameters based on a single exemplar and sample data, respectively. Image-based approaches are straightforward to apply and preserve fine spatiotemporal detail. Synthetic stimuli are desirable when a large number of manipulations are called for and to ensure individual stimuli reflect population characteristics. Received: 13 December 1999 / Received in revised form: 25 February 2000 / Accepted: 1 March 2000     

Real-time multi-wavelength fluorescence imaging of living cells

We describe a new real-time fluorescence video microscope design for capturing intensified images of cells containing dual wavelength "ratio" dyes or multiple dyes. The microscope will perform real-time capture of two separate fluorescence emission images simultaneously, improving the time resolution of spatial distribution of fluorescence to video frame rates (30 frames/s or faster). Each emission wavelength is imaged simultaneously by one of two cameras, then digitized, background corrected and appropriately combined at standard video frame rates to be stored at high resolution on tape or digital video disk for further off-line analysis. Use of low noise, high sensitivity image intensifiers, coupled to CCD cameras produce stable, high contrast images using ultra low light levels with little persistence or bloom. The design has no moving parts in its optical train, which overcomes a number of technical difficulties encountered in the present filter wheel designs for multiple imaging. Coupled to compatible image processing software utilizing PC-AT computers, the new design can be built for a significantly lower cost than many presently available commercial machines. The system is ideal for ratio imaging applications; the software can calculate the ratio of fluorescence intensities pixel by pixel and provide the information to generate false-color images of the intensity data as well as other calculations based on the two images. Thus, it provides a powerful, inexpensive tool for studying the real-time kinetics of changes in living cells. Examples are presented for the kinetics of rapidly changing intracellular calcium detected by the calcium indicator probe indo-1 and the redistribution kinetics of multiple vital dyes placed in cells undergoing cell fusion.     

Characterisation of whisker control in the California sea lion (Zalophus californianus) during a complex,dynamic sensorimotor task

Studies in pinniped whisker use have shown that their whiskers are extremely sensitive to tactile and hydrodynamic signals. While pinnipeds position their whiskers on to objects and have some control over their whisker protractions, it has always been thought that head movements are more responsible for whisker positioning than the movement of the whiskers themselves. This study uses ball balancing, a dynamic sensorimotor skill that is often used in human and robotic coordination studies, to promote sea lion whisker movements during the task. For the first time, using tracked video footage, we show that sea lion whisker movements respond quickly (26.70 ms) and mirror the movement of the ball, much more so than the head. We show that whisker asymmetry and spread are both altered to help sense and control the ball during balancing. We believe that by designing more dynamic sensorimotor tasks we can start to characterise the active nature of this specialised sensory system in pinnipeds.     

A method for the assessment of sperm quality in fish

A method of computer assisted sperm analysis in fish is described. This method is used for assessing the main parameters of sperm motility by means of a microscope and video camera connected to a computer with easily available software. The results obtained during the analysis noticeably depend on the maximum velocity of spermatozoa (the maximum distance covered by a spermatozoon between two subsequent frames of the videotape, which is set in the control window of the plugin “MTrack2” included into the image analyzing software “ImageJ”). The method is illustrated with reference to the analysis of sperm activity in Zebrasoma scopas (Acanthuridae), a representative of the ichthyofauna of coral reefs.