Modeling the Contributions of Basal Ganglia and Hippocampus to Spatial Navigation Using Reinforcement Learning

A computational neural model that describes the competing roles of Basal Ganglia and Hippocampus in spatial navigation is presented. Model performance is evaluated on a simulated Morris water maze explored by a model rat. Cue-based and place-based navigational strategies, thought to be subserved by the Basal ganglia and Hippocampus respectively, are described. In cue-based navigation, the model rat learns to directly head towards a visible target, while in place-based navigation the target position is represented in terms of spatial context provided by an array of poles placed around the pool. Learning is formulated within the framework of Reinforcement Learning, with the nigrostriatal dopamine signal playing the role of Temporal Difference Error. Navigation inherently involves two apparently contradictory movements: goal oriented movements vs. random, wandering movements. The model hypothesizes that while the goal-directedness is determined by the gradient in Value function, randomness is driven by the complex activity of the SubThalamic Nucleus (STN)-Globus Pallidus externa (GPe) system. Each navigational system is associated with a Critic, prescribing actions that maximize value gradients for the corresponding system. In the integrated system, that incorporates both cue-based and place-based forms of navigation, navigation at a given position is determined by the system whose value function is greater at that position. The proposed model describes the experimental results of [1], a lesion-study that investigates the competition between cue-based and place-based navigational systems. The present study also examines impaired navigational performance under Parkinsonian-like conditions. The integrated navigational system, operated under dopamine-deficient conditions, exhibits increased escape latency as was observed in experimental literature describing MPTP model rats navigating a water maze.     

A Novel Augmented Reality Navigation System for Endoscopic Sinus and Skull Base Surgery: A Feasibility Study

Objective

To verify the reliability and clinical feasibility of a self-developed navigation system based on an augmented reality technique for endoscopic sinus and skull base surgery.

Materials and Methods

In this study we performed a head phantom and cadaver experiment to determine the display effect and accuracy of our navigational system. We compared cadaver head-based simulated operations, the target registration error, operation time, and National Aeronautics and Space Administration Task Load Index scores of our navigation system to conventional navigation systems.

Results

The navigation system developed in this study has a novel display mode capable of fusing endoscopic images to three-dimensional (3-D) virtual images. In the cadaver head experiment, the target registration error was 1.28 ± 0.45 mm, which met the accepted standards of a navigation system used for nasal endoscopic surgery. Compared with conventional navigation systems, the new system was more effective in terms of operation time and the mental workload of surgeons, which is especially important for less experienced surgeons.

Conclusion

The self-developed augmented reality navigation system for endoscopic sinus and skull base surgery appears to have advantages that outweigh those of conventional navigation systems. We conclude that this navigational system will provide rhinologists with more intuitive and more detailed imaging information, thus reducing the judgment time and mental workload of surgeons when performing complex sinus and skull base surgeries. Ultimately, this new navigational system has potential to increase the quality of surgeries. In addition, the augmented reality navigational system could be of interest to junior doctors being trained in endoscopic techniques because it could speed up their learning. However, it should be noted that the navigation system serves as an adjunct to a surgeon’s skills and knowledge, not as a substitute.     

The influence of group size and social interactions on collision risk with obstacles

In the UK there has been dramatic growth in the number of proposed wind farms, and the impact on wildlife of this expansion is largely unknown. Avian collisions with wind turbines have received wide attention but reliable predictions remain elusive. Existing predictive models consider behavioural factors such as group movement only implicitly and require accurate site-specific data to produce predictions, making them difficult to translate between locations. Here we introduce an individual-based modelling approach to describe group interactions with obstacles that incorporates aspects of collective motion to simulate and quantify likely avoidance behaviour. We quantify the effect of group size on the probability of an individual colliding with a fixed obstacle, and investigate the roles of both navigational efficiency and group cohesion. We show that, over a wide range of model assumptions and parameterisations, social interactions have a significant and potentially large effect on collision risk; in contrast to previous models, collision risk is typically a non-linear function of group size. These results show that emergent behaviour induced by social interactions can have important effects on the metrics used to inform management and policy decisions.     

Positional entropy during pigeon homing I: application of Bayesian latent state modelling

In these two companion papers, we introduce a new approach to the analysis of bird navigation which brings together several novel mathematical and technical applications. Miniaturized GPS logging devices provide track data of sufficiently high spatial and temporal resolution that considerable variation in flight behaviour can be observed remotely from the form of the track alone. We analyse a fundamental measure of bird flight track complexity, spatio-temporal entropy, and explore its state-like structure using a probabilistic hidden Markov model. The emergence of a robust three-state structure proves that the technique has analytical power, since this structure was not obvious in the tracks alone. We propose the hypothesis that positional entropy is indicative of underlying navigational uncertainty, and that familiar area navigation may break down into three states of navigational confidence. By interpreting the relationship between these putative states and features on the map, we are able to propose a number of hypothetical navigational strategies feeding into these states. The first of these two papers details the novel technical developments associated with this work and the second paper contains a navigational interpretation of the results particularly with respect to visual features of the landscape.     

Positional entropy during pigeon homing II: navigational interpretation of Bayesian latent state models

In these two companion papers we introduce a new approach to the analysis of bird navigation which brings together several novel mathematical and technical applications. Miniaturized GPS logging devices provide track data of sufficiently high spatial and temporal resolution that considerable variation in flight behaviour can be observed remotely from the form of the track alone. We analyse a fundamental measure of bird flight track complexity, spatio-temporal entropy, and explore its state-like structure using a probabilistic hidden Markov model. The emergence of a robust three-state structure proves that the technique has analytical power, since this structure was not obvious in the tracks alone. We propose the hypothesis that positional entropy is indicative of underlying navigational uncertainty, and that familiar area navigation may break down into three states of navigational confidence. By interpreting the relationship between these putative states and features on the map, we are able to propose a number of hypothetical navigational strategies feeding into these states. The first of these two papers details the novel technical developments associated with this work and the second paper contains a navigational interpretation of the results particularly with respect to visual features of the landscape.     

An optical flow-based integrated navigation system inspired by insect vision

Some insects use optic flow (OF) to perform their navigational tasks perfectly. Learning from insects' OF navigation strategies, this article proposes a bio-inspired integrated navigation system based on OF. The integrated navigation system is composed of an OF navigation system (OFNS) and an OF aided navigation system (OFAN). The OFNS uses a simple OF method to measure motion at each step along a path. The position information is then obtained by path integration. However, path integration leads to cumulative position errors which increase rapidly with time. To overcome this problem, the OFAN is employed to assist the OFNS in estimating and correcting these cumulative errors. The OFAN adopts an OF-based Kalman filter (KF) to continuously estimate the position errors. Moreover, based on the OF technique used in the OFNS, we develop a new OF method employed by the OFAN to generate the measurement input of the OF-based KF. As a result, both the OFNS and the OFAN in our integrated navigation system are derived from the same OF method so that they share input signals and some operations. The proposed integrated navigation system can provide accurate position information without interference from cumulative errors yet doing so with low computational effort. Simulations and comparisons have demonstrated its efficiency.     

Short-Term Memory Maintenance of Object Locations during Active Navigation: Which Working Memory Subsystem Is Essential?

The goal of the present study was to examine the extent to which working memory supports the maintenance of object locations during active spatial navigation. Participants were required to navigate a virtual environment and to encode the location of a target object. In the subsequent maintenance period they performed one of three secondary tasks that were designed to selectively load visual, verbal or spatial working memory subsystems. Thereafter participants re-entered the environment and navigated back to the remembered location of the target. We found that while navigation performance in participants with high navigational ability was impaired only by the spatial secondary task, navigation performance in participants with poor navigational ability was impaired equally by spatial and verbal secondary tasks. The visual secondary task had no effect on navigation performance. Our results extend current knowledge by showing that the differential engagement of working memory subsystems is determined by navigational ability.     

Orientation and navigation in Amphibia

Aquatic and terrestrial amphibians integrate acoustic, magnetic, mechanical, olfactory and visual directional information into a redundant-multisensory orientation system. The sensory information is processed to accomplish homing following active or passive displacement by either path integration, beaconing, pilotage, compass orientation or true navigation. There is evidence for two independent compass systems, a time-compensated compass based on celestial cues and a light-dependent magnetic inclination compass. Beaconing along acoustic or olfactory gradients emanating from the home site, as well as pilotage along fixed visual landmarks also form an important part in the behaviour of many species. True navigation has been shown in only one species, the aquatic salamander Notophthalmus viridescens. Evidence on the nature of the navigational map obtained so far is compatible with the magnetic map hypothesis.     

Homing pigeons only navigate in air with intact environmental odours: a test of the olfactory activation hypothesis with GPS data loggers

A large body of evidence has shown that anosmic pigeons are impaired in their navigation. However, the role of odours in navigation is still subject to debate. While according to the olfactory navigation hypothesis homing pigeons possess a navigational map based on the distribution of environmental odours, the olfactory activation hypothesis proposes that odour perception is only needed to activate a navigational mechanism based on cues of another nature. Here we tested experimentally whether the perception of artificial odours is sufficient to allow pigeons to navigate, as expected from the olfactory activation hypothesis. We transported three groups of pigeons in air-tight containers to release sites 53 and 61 km from home in three different olfactory conditions. The Control group received natural environmental air; both the Pure Air and the Artificial Odour groups received pure air filtered through an active charcoal filter. Only the Artificial Odour group received additional puffs of artificial odours until release. We then released pigeons while recording their tracks with 1 Hz GPS data loggers. We also followed non-homing pigeons using an aerial data readout to a Cessna plane, allowing, for the first time, the tracking of non-homing homing pigeons. Within the first hour after release, the pigeons in both the Artificial Odour and the Pure Air group (receiving no environmental odours) showed impaired navigational performances at each release site. Our data provide evidence against an activation role of odours in navigation, and document that pigeons only navigate well when they perceive environmental odours.     

Wüstennavigatoren en miniature

Desert navigators en miniature Cataglyphis, a strictly diurnal, heat‐tolerant, high‐speed desert ant, employs a path integrator as its main navigational means. By continually measuring directions steered and distances covered the path integrator computes a navigation vector, which can lead the ant directly back to its central place, the nest, and to any point which it has visited before. The path integration vector receives compass information from the pattern of polarized light in the sky (via a set of specialized photoreceptors at the dorsal rim of the eye), and derives information about travel distance from a stride integrator (pedometer) and an optic‐flow meter exploiting self‐induced image motion across the ventral retina. The path integrator is fully functional already at the beginning of the ant's foraging life. Later it keeps running whenever the ant is on a foraging excursion irrespective of whether other navigational tools are at work as well. Finally it provides a scaffold for landmark learning. View‐based landmark information is acquired by taking panoramic “snapshots” at certain places and routes. By comparing this memorized visual information with the actual one received during later journeys the ants are able to return to familiar places and to follow familiar routes even without the aid of the path integrator. The ant's navigational performances known to date can be simulated by designing a decentralized network, in which the individual tools are interconnected in flexible and context dependent ways.     

Orientation and navigation of migrating birds

The question of how migrating birds find their way to winter quarters and back has fascinated humans since the beginning of scientific research into avian biology. Migrating birds have been shown to possess compass systems that allow them to select and maintain certain compass directions. Three such systems are known, solar, stellar and magnetic. Their details are not quite clear and need further research. Hierarchy and interaction of compass systems of migrating birds are poorly studied; different species may vary in this respect. During migration, birds learn to use maps that make true navigation possible, i.e. to detect their position relatively to the goal of movement. The physical nature of navigational maps is an object of intensive research; currently the most promising concepts are the geomagnetic and possibly olfactory maps. A significant contribution to the study of formation of navigational maps was made by Soviet/Russian researchers, whose work was published in Zoologicheskii Zhurnal (Sokolov et al., 1984). Migrating birds have no innate map, and first-autumn individuals reach their species-specific wintering areas by using compass sense and counting time that should be spent moving in certain genetically fixed directions. However, in recent years more and more data surface that suggest that juveniles (maybe not of all species) do have some mechanism of controlling their position on the migratory route that allows them to compensate for errors of the spatio-temporal programme of migration.     

Navigational Efficiency of Nocturnal Myrmecia Ants Suffers at Low Light Levels

Insects face the challenge of navigating to specific goals in both bright sun-lit and dim-lit environments. Both diurnal and nocturnal insects use quite similar navigation strategies. This is despite the signal-to-noise ratio of the navigational cues being poor at low light conditions. To better understand the evolution of nocturnal life, we investigated the navigational efficiency of a nocturnal ant, Myrmecia pyriformis, at different light levels. Workers of M. pyriformis leave the nest individually in a narrow light-window in the evening twilight to forage on nest-specific Eucalyptus trees. The majority of foragers return to the nest in the morning twilight, while few attempt to return to the nest throughout the night. We found that as light levels dropped, ants paused for longer, walked more slowly, the success in finding the nest reduced and their paths became less straight. We found that in both bright and dark conditions ants relied predominantly on visual landmark information for navigation and that landmark guidance became less reliable at low light conditions. It is perhaps due to the poor navigational efficiency at low light levels that the majority of foragers restrict navigational tasks to the twilight periods, where sufficient navigational information is still available.     

Tracking route progression in the posterior parietal cortex

Quick and efficient traversal of learned routes is critical to the survival of many animals. Routes can be defined by both the ordering of navigational epochs, such as continued forward motion or execution of a turn, and the distances separating them. The neural substrates conferring the ability to fluidly traverse complex routes are not well understood, but likely entail interactions between frontal, parietal, and rhinal cortices and the hippocampus. This paper demonstrates that posterior parietal cortical neurons map both individual and multiple navigational epochs with respect to their order in a route. In direct contrast to spatial firing patterns of hippocampal neurons, parietal neurons discharged in a place- and direction-independent fashion. Parietal route maps were scalable and versatile in that they were independent of the size and spatial configuration of navigational epochs. The results provide a framework in which to consider parietal function in spatial cognition.     

Simulating geomagnetic bird navigation using novel high-resolution geomagnetic data

Birds rely on precise navigational mechanisms, especially for long-distance migrations. One debated mechanism is their use of the geomagnetic field. It is unclear if and how different species of birds are using intensity or inclination (or both) for navigation. Previous geomagnetic modelling research is based on static geomagnetic data despite a temporally and spatially varying geomagnetic field. Animals supposedly have a high sensitivity to those changes of the geomagnetic field. In order to understand how birds respond in real-time to its temporal variation, we need to use accurate geomagnetic information linked to the position of the bird through co-location in space and time.We developed a data-driven approach to simulate geomagnetic migratory strategies, using, for the first time, accurate contemporaneous geomagnetic data obtained from Swarm satellites of the European Space Agency. We created biased correlated random walk models which were based on both GPS data from greater white-fronted geese (Anser albifrons) during fall migration between north-west Russia and central Europe and contemporaneous satellite geomagnetic data. Different strategies of geomagnetic navigation associated with different geomagnetic values were translated into probability surfaces, built from geomagnetic data, and included into the random walk models. To evaluate which strategy was most likely, we compared the measured GPS trajectories to the simulated trajectories using different trajectory similarity measurements. We propose this as an approach to track many bird species for future comparative studies.We found that navigational strategies in these geese using magnetic intensity were closer to the observed data than those using inclination. This was the case in 80% of the best models and is an indication that it should be more beneficial for these geese to use intensity over inclination. Additionally, our results supported results from a previous study, that navigation based on taxis and compass mechanisms were more similar to the observed data than other mechanisms. We therefore suggest that these geese may use a combination of these strategies for navigation at a broad-scale. Overall, it seems likely that for successful navigation to the target location more than one mechanism is necessary; indicating a multifactorial navigation mechanism of these migratory geese in the study area. The satellite geomagnetic data are available at a higher temporal resolution and the use significantly improved the fit of the modelled simulations in comparison to the modelled geomagnetic data. Therefore, using annotated geomagnetic data could greatly improve the modelling of animal geomagnetic navigation in future research.     

Many wrongs: the advantage of group navigation

Research into the puzzling phenomena of animal navigation and aggregation has proceeded along two distinct lines. Study of navigation generally focuses on the orientation ability of the individual without reference to the implications of group membership. A simple principle (the 'many wrongs principle'), first proposed by Bergman and Donner in 1964, and developed by both Hamilton and Wallraff three decades ago, provides a link between these lines of current interest by suggesting that navigational accuracy increases with group size. With unprecedented scope for testing the hypotheses it generates, it is now time that the many wrongs principle is exhumed.     

Global navigation in migratory birds: tracks, strategies, and interactions between mechanisms

The advancing development of tracking techniques has led to fascinating new insights into avian migration, documenting the immense diversity, complexity, and flexibility of this phenomenon. Tracking studies so far have confirmed many findings from ringing recoveries and cage studies, for example, the change from flying innate compass courses in the first migration to true navigation, as experienced migrants head toward familiar goals. First attempts to analyze the navigational mechanisms by tracking manipulated migrants indicate strong parallels to those of homing pigeons. Findings suggesting that the magnetic compass of migrants is regularly calibrated by the pattern of polarized light could not be replicated with a number of other birds, pointing out differences between species and possibly region and phases of migration. Tracking has become a valuable tool, complimenting traditional methods by documenting migration behavior in the wild; whether it can be used to further unveil the navigational mechanisms of migrants and the factor used remains an open question.     

Avian navigation: from historical to modern concepts

Studies on avian navigation began at the end of the 19th century with testing various hypotheses, followed by large-scale displacement experiments to assess the capacity of the birds' navigational abilities. In the 1950s, the first theoretical concepts were published. Kramer proposed his ‘Map-and-Compass’ model, assuming that birds establish the direction to a distant goal with the help of an external reference, a compass. The model describes homing as a two-step process, with the first step determining the direction to the goal as a compass course and the second step locating this course with the help of a compass. This model was widely accepted when numerous experiments with clock-shifted pigeons demonstrated the use of the sun compass, and thus a general involvement of compass orientation, in homing. The ‘map’ step is assumed to use local site-specific information, which led to the idea of a ‘grid map’ based on environmental gradients. Kramer's model still forms the basis of our present concept on avian homing, yet route integration with the help of an external reference provides an alternative strategy to determine the home course, and the magnetic compass is a second compass mechanism available to birds. These mechanisms are interrelated by ontogenetic learning processes. A two-step process, with the first step providing the compass course and the second step locating this course with the help of a compass, appears to be a common feature of avian navigation tasks, yet the origin of the compass courses differs between tasks according to their nature, with courses acquired by experience for flights within the home range, courses based on navigational processes for returning home, and courses derived from genetically coded information in first-time migrants. Compass orientation thus forms the backbone of the avian navigational system. Copyright 2003 The Association for the Study of Animal Behaviour. Published by Elsevier Science Ltd. All rights reserved.      

Src kinase activation: A switched electrostatic network

Src tyrosine kinases are essential in numerous cell signaling pathways, and improper functioning of these enzymes has been implicated in many diseases. The activity of Src kinases is regulated by conformational activation, which involves several structural changes within the catalytic domain (CD): the orientation of two lobes of CD; rearrangement of the activation loop (A-loop); and movement of an alpha-helix (alphaC), which is located at the interface between the two lobes, into or away from the catalytic cleft. Conformational activation was investigated using biased molecular dynamics to explore the transition pathway between the active and the down-regulated conformation of CD for the Src-kinase family member Lyn kinase, and to gain insight into the interdependence of these changes. Lobe opening is observed to be a facile motion, whereas movement of the A-loop motion is more complex requiring secondary structure changes as well as communication with alphaC. A key result is that the conformational transition involves a switch in an electrostatic network of six polar residues between the active and the down-regulated conformations. The exchange between interactions links the three main motions of the CD. Kinetic experiments that would demonstrate the contribution of the switched electrostatic network to the enzyme mechanism are proposed. Possible implications for regulation conferred by interdomain interactions are also discussed.