Automated Planar Tracking the Waving Bodies of Multiple Zebrafish Swimming in Shallow Water

Zebrafish (Danio rerio) is one of the most widely used model organisms in collective behavior research. Multi-object tracking with high speed camera is currently the most feasible way to accurately measure their motion states for quantitative study of their collective behavior. However, due to difficulties such as their similar appearance, complex body deformation and frequent occlusions, it is a big challenge for an automated system to be able to reliably track the body geometry of each individual fish. To accomplish this task, we propose a novel fish body model that uses a chain of rectangles to represent fish body. Then in detection stage, the point of maximum curvature along fish boundary is detected and set as fish nose point. Afterwards, in tracking stage, we firstly apply Kalman filter to track fish head, then use rectangle chain fitting to fit fish body, which at the same time further judge the head tracking results and remove the incorrect ones. At last, a tracklets relinking stage further solves trajectory fragmentation due to occlusion. Experiment results show that the proposed tracking system can track a group of zebrafish with their body geometry accurately even when occlusion occurs from time to time.     

In vivo generation of DNA sequence diversity for cellular barcoding

Heterogeneity is a ubiquitous feature of biological systems. A complete understanding of such systems requires a method for uniquely identifying and tracking individual components and their interactions with each other. We have developed a novel method of uniquely tagging individual cells in vivo with a genetic ‘barcode’ that can be recovered by DNA sequencing. Our method is a two-component system comprised of a genetic barcode cassette whose fragments are shuffled by Rci, a site-specific DNA invertase. The system is highly scalable, with the potential to generate theoretical diversities in the billions. We demonstrate the feasibility of this technique in Escherichia coli. Currently, this method could be employed to track the dynamics of populations of microbes through various bottlenecks. Advances of this method should prove useful in tracking interactions of cells within a network, and/or heterogeneity within complex biological samples.     

Automatic detection of fish and tracking of movement for ecology

1. Animal movement studies are conducted to monitor ecosystem health, understand ecological dynamics, and address management and conservation questions. In marine environments, traditional sampling and monitoring methods to measure animal movement are invasive, labor intensive, costly, and limited in the number of individuals that can be feasibly tracked. Automated detection and tracking of small‐scale movements of many animals through cameras are possible but are largely untested in field conditions, hampering applications to ecological questions.
2. Here, we aimed to test the ability of an automated object detection and object tracking pipeline to track small‐scale movement of many individuals in videos. We applied the pipeline to track fish movement in the field and characterize movement behavior. We automated the detection of a common fisheries species (yellowfin bream, Acanthopagrus australis) along a known movement passageway from underwater videos. We then tracked fish movement with three types of tracking algorithms (MOSSE, Seq‐NMS, and SiamMask) and evaluated their accuracy at characterizing movement.
3. We successfully detected yellowfin bream in a multispecies assemblage (F1 score =91%). At least 120 of the 169 individual bream present in videos were correctly identified and tracked. The accuracies among the three tracking architectures varied, with MOSSE and SiamMask achieving an accuracy of 78% and Seq‐NMS 84%.
4. By employing this integrated object detection and tracking pipeline, we demonstrated a noninvasive and reliable approach to studying fish behavior by tracking their movement under field conditions. These cost‐effective technologies provide a means for future studies to scale‐up the analysis of movement across many visual monitoring systems.

     

Balancing exploitation of renewable resources by a robot swarm

Renewable resources like fish stock or forests should be exploited at a rate that supports regeneration and sustainability—a complex problem that requires adaptive approaches to maintain a sufficiently high exploitation while avoiding depletion. In the presence of oblivious agents that cannot keep track of all available resources—a frequent condition in swarm robotics—ensuring that the exploitation effort is correctly balanced is particularly challenging. Additionally, the possibility to exploit resources by multiple robots opens the way to focusing the effort either on a single or on multiple resources in parallel. This means that the swarm needs to collectively decide whether to remain cohesive or split among multiple resources, as a function of the ability of the available resources to replenish after exploitation. In this paper, we propose a decentralised strategy for a swarm of robots that adapts to the available resources and balances the effort among them, hence allowing to maximise the exploitation rate while avoiding to completely deplete the resources. A detailed analysis of the strategy parameters provides insights into the working principles and expected performance of the robot swarm.     

Investigating the status of biological stimuli as objects of attention in multiple object tracking

Background

Humans are able to track multiple simultaneously moving objects. A number of factors have been identified that can influence the ease with which objects can be attended and tracked. Here, we explored the possibility that object tracking abilities may be specialized for tracking biological targets such as people.

Methodology/Principal Findings

We used the Multiple Object Tracking (MOT) paradigm to explore whether the high-level biological status of the targets affects the efficiency of attentional selection and tracking. In Experiment 1, we assessed the tracking of point-light biological motion figures. As controls, we used either the same stimuli or point-light letters, presented in upright, inverted or scrambled configurations. While scrambling significantly affected performance for both letters and point-light figures, there was an effect of inversion restricted to biological motion, inverted figures being harder to track. In Experiment 2, we found that tracking performance was equivalent for natural point-light walkers and ‘moon-walkers’, whose implied direction was incongruent with their actual direction of motion. In Experiment 3, we found higher tracking accuracy for inverted faces compared with upright faces. Thus, there was a double dissociation between inversion effects for biological motion and faces, with no inversion effect for our non-biological stimuli (letters, houses).

Conclusions/Significance

MOT is sensitive to some, but not all naturalistic aspects of biological stimuli. There does not appear to be a highly specialized role for tracking people. However, MOT appears constrained by principles of object segmentation and grouping, where effectively grouped, coherent objects, but not necessarily biological objects, are tracked most successfully.     

Feature point tracking and trajectory analysis for video imaging in cell biology

This paper presents a computationally efficient, two-dimensional, feature point tracking algorithm for the automated detection and quantitative analysis of particle trajectories as recorded by video imaging in cell biology. The tracking process requires no a priori mathematical modeling of the motion, it is self-initializing, it discriminates spurious detections, and it can handle temporary occlusion as well as particle appearance and disappearance from the image region. The efficiency of the algorithm is validated on synthetic video data where it is compared to existing methods and its accuracy and precision are assessed for a wide range of signal-to-noise ratios. The algorithm is well suited for video imaging in cell biology relying on low-intensity fluorescence microscopy. Its applicability is demonstrated in three case studies involving transport of low-density lipoproteins in endosomes, motion of fluorescently labeled Adenovirus-2 particles along microtubules, and tracking of quantum dots on the plasma membrane of live cells. The present automated tracking process enables the quantification of dispersive processes in cell biology using techniques such as moment scaling spectra.     

MAD-YOLO: A quantitative detection algorithm for dense small-scale marine benthos

Marine biological resources are abundant, and the reasonable development, research and protection of marine biological resources are of great significance to marine ecological health and economic development. At present, underwater object quantitative detection plays a very important role in marine biological science research, marine species richness survey, and rare species conservation. However, the problems of a large amount of noise in the underwater environment, small object scale, dense biological distribution, and occlusion all increase the detection difficulty. In this paper, a detection algorithm MAD-YOLO (Multiscale Feature Extraction and Attention Feature Fusion Reinforced YOLO for Marine Benthos Detection) is proposed, which is based on improved YOLOv5 is proposed to solve the above problems. To improve the adaptability of the network to the underwater environment, VOVDarkNet is designed as the feature extraction backbone. It uses the intermediate features with different receptive fields to reinforce the ability to extract feature. AFC-PAN is proposed as the feature fusion network so that the network can learn correct feature information and location information of objects at various scales, improving the network's ability to perceive small objects. Label assignment strategy SimOTA and decoupled head are introduced to help the model better handles occlusion and dense distribution problems. Experiments show the MAD-YOLO algorithm increases mAP_0.5:0.95 on the URPC2020 dataset from 49.8% to 53.4% compared to the original YOLOv5. Moreover, the advantages of the model are visualized and analyzed by the method of controlling variables in the experimental part. The experiments show that MAD-YOLO is suitable for detecting blurred, dense, and small-scale objects. The model performs well in marine benthos detection tasks and can effectively promote marine life science research and marine engineering implementation. The source code is publicly available at https://github.com/JoeNan1/MAD-YOLO.     

Modelling the Effects of Current on Prey Acquisition in Planktivorous Fishes

Many planktivorous fishes forage in currents, where they actively maintain position and visually strike at current-entrained zooplankton. In general, the zooplankton are wafted by the foraging fish at a rate equivalent to the current velocity. From a fish's viewpoint the plankton approach either head-on or offset at varied distances from the fish's position. We present a model that describes the relative motion of particles as they approach and pass a foraging fish at different offset distances, and the rate of change in apparent size as they close on a fish. In addition, a series of experiments of fish feeding on plankton in a flume at increasing current velocities revealed that two basic tactics are utilized. At low current velocities (<10-14 cm s m 1), the fish swims toward the prey, whereas at higher current velocities the fish tends to fall back with the current to capture a prey item. The model and experimental results are discussed in terms of the visual problems associated with the detection and tracking of items in motion.     

Swarms of particle agents with harmonic interactions

Summary Agent-based modeling is a powerful methodology to describe the occurence of complex behavior in biological systems. The interaction of a large number of individuals (agents) may for example lead to the emergence of new forms of collective motion. In this paper, we investigate a particle-based approach to the coherent motion of a swarm with parabolic (i. e. harmonic) interactions between the agents. It is based on generalized Langevin equations for the particle agents, which take into account (i) energetic conditions for active motion, (ii) linear attractive forces between each two agents. The complex collective motion observed can be explained as the result of these different influences: the active motion of the agents, which is driven by the energy-take up, would eventually lead to a spatial dispersion of the swarm, while the mutual interaction of the agents results in a tendency of spatial concentration. In addition to particle-based computer simulations, we also provide a mathematical framework for investigating the collective dynamics. Dedicated to the memory of Michael Conrad     

How phenological tracking shapes species and communities in non-stationary environments

Climate change alters the environments of all species. Predicting species responses requires understanding how species track environmental change, and how such tracking shapes communities. Growing empirical evidence suggests that how species track phenologically – how an organism shifts the timing of major biological events in response to the environment – is linked to species performance and community structure. Such research tantalizingly suggests a potential framework to predict the winners and losers of climate change, and the future communities we can expect. But developing this framework requires far greater efforts to ground empirical studies of phenological tracking in relevant ecological theory. Here we review the concept of phenological tracking in empirical studies and through the lens of coexistence theory to show why a community-level perspective is critical to accurate predictions with climate change. While much current theory for tracking ignores the importance of a multi-species context, basic community assembly theory predicts that competition will drive variation in tracking and trade-offs with other traits. We highlight how existing community assembly theory can help understand tracking in stationary and non-stationary systems. But major advances in predicting the species- and community-level consequences of climate change will require advances in theoretical and empirical studies. We outline a path forward built on greater efforts to integrate priority effects into modern coexistence theory, improved empirical estimates of multivariate environmental change, and clearly defined estimates of phenological tracking and its underlying environmental cues.     

基于视频跟踪的竖缝式鱼道内鱼类运动行为分析

竖缝式鱼道过鱼对象运动行为与鱼道池室内水力条件是否相适应是进行鱼道设计的关键。研究通过视频跟踪法对竖缝式鱼道中目标鱼的运动轨迹进行实时跟踪, 获取鱼的运动加速度、运动速度, 并和人工手动跟踪的鱼类运动轨迹进行对比, 证明基于视频跟踪法的鱼类运动分析程序既能较好的应用于竖缝式鱼道中, 获取鱼类运动行为, 又可减少大量的人工操作, 有助于为竖缝式鱼道设计提供重要基础数据。     

Neural specializations for small target detection in insects

Despite being equipped with low-resolution eyes and tiny brains, many insects show exquisite abilities to detect and pursue targets even in highly textured surrounds. Target tracking behavior is subserved by neurons that are sharply tuned to the motion of small high-contrast targets. These neurons respond robustly to target motion, even against self-generated optic flow. A recent model, supported by neurophysiology, generates target selectivity by being sharply tuned to the unique spatiotemporal profile associated with target motion. Target neurons are likely connected in a complex network where some provide more direct output to behavior, whereas others serve an inter-regulatory role. These interactions may regulate attention and aid in the robust detection of targets in clutter observed in behavior.     

Emergent Runaway into an Avoidance Area in a Swarm of Soldier Crabs

Emergent behavior that arises from a mass effect is one of the most striking aspects of collective animal groups. Investigating such behavior would be important in order to understand how individuals interact with their neighbors. Although there are many experiments that have used collective animals to investigate social learning or conflict between individuals and society such as that between a fish and a school, reports on mass effects are rare. In this study, we show that a swarm of soldier crabs could spontaneously enter a water pool, which are usually avoided, by forming densely populated part of a swarm at the edge of the water pool. Moreover, we show that the observed behavior can be explained by the model of collective behavior based on inherent noise that is individuals’ different velocities in a directed group. Our results suggest that inherent noise, which is widely seen in collective animals, can contribute to formation and/or maintenance of a swarm and that the dense swarm can enter the pool by means of enhanced inherent noise.     

Multiple dense particle tracking in fluorescence microscopy images based on multidimensional assignment

Multiple particle tracking (MPT) has seen numerous applications in live-cell imaging studies of subcellular dynamics. Establishing correspondence between particles in a sequence of frames with high particle density, particles merging and splitting, particles entering and exiting the frame, temporary particle disappearance, and an ill-performing detection algorithm is the most challenging part of MPT. Here we propose a tracking method based on multidimensional assignment to address these problems. We combine an Interacting Multiple Model (IMM) filter, multidimensional assignment, particle occlusion handling, and merge-split event detection in a single software analysis package. The main advantage of a multidimensional assignment is that both spatial and temporal information can be used by using several later frames as reference. The IMM filter, which is used to maintain and predict the state of each track, contains several models which correspond to different types of biologically realistic movements. It works especially well with multidimensional assignment, because there tends to be a higher probability of correct particle association over time. First the method generates many particle-correspondence hypotheses, merge-split hypotheses and misdetection hypotheses within the framework of a sliding window over the frames of the image sequence. Then it builds a multidimensional assignment problem (MAP) accordingly. The particle is tracked with gap-filling, and merging and splitting events are then detected using the MAP solution. The tracking method is validated on both simulated tracks and microscopy image sequences. The results of these experiments show that the method is more accurate and robust than other "tracking from detected features" methods in dense particle situations.     

Precise particle tracking against a complicated background: polynomial fitting with Gaussian weight

We present a new particle tracking software algorithm designed to accurately track the motion of low-contrast particles against a background with large variations in light levels. The method is based on a polynomial fit of the intensity around each feature point, weighted by a Gaussian function of the distance from the centre, and is especially suitable for tracking endogeneous particles in the cell, imaged with bright field, phase contrast or fluorescence optical microscopy. Furthermore, the method can simultaneously track particles of all different sizes, and allows significant freedom in their shape. The algorithm is evaluated using the quantitative measures of accuracy and precision of previous authors, using simulated images at variable signal-to-noise ratios. To these we add new tests: the error due to a non-uniform background, and the error due to two particles approaching each other. Finally the tracking of particles in real cell images is demonstrated. The method is made freely available for non-commercial use as a software package with a graphical user-interface, which can be run within the Matlab programming environment.     

How lesioning the nucleus praeeminentialis affects electrolocation behavior in the weakly electric fish, Apteronotus leptorhynchus

The ability to orient to and track moving electrolocation targets was assessed in normal Apteronotus leptorhynchus and in those with unilateral lesions of the nucleus praeeminentialis dorsalis.

1. Each fish was trained to hover between two vertical metal rods and track their movement. Two aspects of this behavior were measured: a) the hovering position of the fish relative to stationary rods; b) the latency between the onset of rod motion and the fish's tracking response. Control fish hovered midway between stationary rods, while lesioned fish hovered closer to the rod ipsilateral to the lesion. Response latency varied negatively with rod diameter in both sets of fish, and lesioned fish exhibited shorter latencies than control fish. While the response latencies of control fish were shortest when their starting position was midway between the rods, lesioned animals' latencies were shortest when they hovered closer to the rod ipsilateral to their lesion.
2. Control fish responded to the approach of a single metal ball to either side of the body with nearly equal latencies and fish-to-object distances. After lesioning, response latency increased and fish-to-object distance decreased for approaches to the side ipsilateral to the lesion; opposite changes occurred for contralateral approaches.

     

Identification and counting of fish targets using adaptive resolution imaging sonar

Fish are a critical component of marine biology; therefore, the accurate identification and counting of fish are essential for the objective monitoring and assessment of marine biological resources. High-frequency adaptive resolution imaging sonar (ARIS) is widely used for underwater object detection and imaging, and it quickly obtains close-up video of free-swimming fish in high-turbidity water environments. Nonetheless, processing the massive data output using imaging sonars remains a major challenge. Here, the authors developed an automatic image-processing programme that fuses K-nearest neighbour background subtraction with DeepSort target tracking to automatically track and count fish. The automatic programme was evaluated using four test data sets with different target sizes and observation ranges and differently deployed sonars. According to the results, the approach successfully counted free-swimming fish targets with an accuracy index of 73% and a completeness index of 70%. Under appropriate conditions, this approach could replace time-consuming semi-automatic approaches and improve the efficiency of imaging sonar data processing, while providing technical support for future real-time data processing.     

Postural Sway and Gaze Can Track the Complex Motion of a Visual Target

Variability is an inherent and important feature of human movement. This variability has form exhibiting a chaotic structure. Visual feedback training using regular predictive visual target motions does not take into account this essential characteristic of the human movement, and may result in task specific learning and loss of visuo-motor adaptability. In this study, we asked how well healthy young adults can track visual target cues of varying degree of complexity during whole-body swaying in the Anterior-Posterior (AP) and Medio-Lateral (ML) direction. Participants were asked to track three visual target motions: a complex (Lorenz attractor), a noise (brown) and a periodic (sine) moving target while receiving online visual feedback about their performance. Postural sway, gaze and target motion were synchronously recorded and the degree of force-target and gaze-target coupling was quantified using spectral coherence and Cross-Approximate entropy. Analysis revealed that both force-target and gaze-target coupling was sensitive to the complexity of the visual stimuli motions. Postural sway showed a higher degree of coherence with the Lorenz attractor than the brown noise or sinusoidal stimulus motion. Similarly, gaze was more synchronous with the Lorenz attractor than the brown noise and sinusoidal stimulus motion. These results were similar regardless of whether tracking was performed in the AP or ML direction. Based on the theoretical model of optimal movement variability tracking of a complex signal may provide a better stimulus to improve visuo-motor adaptation and learning in postural control.     

Order and flexibility in the motion of fish schools

The coexistence of order and flexibility in the motion of fish schools was studied by using a simple numerical model and a computer simulation. The numerical model is based on behavioral rules for individuals in the school by considering attraction, repulsion, and parallel-orientation behavior. Each individual follows the same rules and makes school movements. The simulation results show that school order and flexibility are affected by the number of neighbors interacting with an individual in the school and by the randomness of individual motion. Increase in the number of interacting neighbors leads to high order, especially when the number increases from a low value (between one and three). An optimal number of interacting neighbors exists that is relatively low (two or three) for high flexibility, indicating that a fish needs only to pay attention to a few neighbors to realize both order and flexibility. The low randomness of individual motion benefits both order and flexibility. These results indicate that schooling fish have evolved specialized ability for establishing both school order and flexibility.     

The plant as a biomechatronic system

Our vision of plants is changing dramatically: from insensitive and static objects to complex living beings able to sense the environment and to use the information collected to adapt their behavior. At all times humans imitate ideas and concepts from nature to resolve technological problems. Solutions coming from plants have the potential to face challenges and difficulties of modern engineering design. Characteristic concepts of the plant world such as reiteration, modularity and swarm behavior could be of great help resolving technological problems. On the other hand a biorobotic approach would facilitate the resolution of many biological problems. In this paper, the concept of a plant-inspired robot is proposed for the investigation of both biological and technological issues.