Maintaining a cognitive map in darkness: the need to fuse boundary knowledge with path integration

Spatial navigation requires the processing of complex, disparate and often ambiguous sensory data. The neurocomputations underpinning this vital ability remain poorly understood. Controversy remains as to whether multimodal sensory information must be combined into a unified representation, consistent with Tolman's "cognitive map", or whether differential activation of independent navigation modules suffice to explain observed navigation behaviour. Here we demonstrate that key neural correlates of spatial navigation in darkness cannot be explained if the path integration system acted independently of boundary (landmark) information. In vivo recordings demonstrate that the rodent head direction (HD) system becomes unstable within three minutes without vision. In contrast, rodents maintain stable place fields and grid fields for over half an hour without vision. Using a simple HD error model, we show analytically that idiothetic path integration (iPI) alone cannot be used to maintain any stable place representation beyond two to three minutes. We then use a measure of place stability based on information theoretic principles to prove that featureless boundaries alone cannot be used to improve localization above chance level. Having shown that neither iPI nor boundaries alone are sufficient, we then address the question of whether their combination is sufficient and - we conjecture - necessary to maintain place stability for prolonged periods without vision. We addressed this question in simulations and robot experiments using a navigation model comprising of a particle filter and boundary map. The model replicates published experimental results on place field and grid field stability without vision, and makes testable predictions including place field splitting and grid field rescaling if the true arena geometry differs from the acquired boundary map. We discuss our findings in light of current theories of animal navigation and neuronal computation, and elaborate on their implications and significance for the design, analysis and interpretation of experiments.     

The Preferred Directions of Conjunctive Grid X Head Direction Cells in the Medial Entorhinal Cortex Are Periodically Organized

The discovery of speed-modulated grid, head direction, and conjunctive grid x head direction cells in the medial entorhinal cortex has led to the hypothesis that path integration, the updating of one’s spatial representation based on movement, may be carried out within this region. This hypothesis has been formalized by many computational models, including a class known as attractor network models. While many of these models propose specific mechanisms by which path integration might occur, predictions of these specific mechanisms have not been tested. Here I derive and test a key prediction of one attractor network path integration mechanism. Specifically, I first demonstrate that this mechanism predicts a periodic distribution of conjunctive cell preferred directions in order to minimize drift. Next, I test whether conjunctive cell preferred directions are in fact periodically organized. Results indicate that conjunctive cells are preferentially tuned to increments of 36°, consistent with drift minimization in this path integration mechanism. By contrast, no periodicity was observed in the preferred directions of either pure grid or pure head direction cells. These results provide the first neural evidence of a nonuniform structure in the directional preferences of any head direction representation found in the brain.     

Continuous Attractor Network Model for Conjunctive Position-by-Velocity Tuning of Grid Cells

The spatial responses of many of the cells recorded in layer II of rodent medial entorhinal cortex (MEC) show a triangular grid pattern, which appears to provide an accurate population code for animal spatial position. In layer III, V and VI of the rat MEC, grid cells are also selective to head-direction and are modulated by the speed of the animal. Several putative mechanisms of grid-like maps were proposed, including attractor network dynamics, interactions with theta oscillations or single-unit mechanisms such as firing rate adaptation. In this paper, we present a new attractor network model that accounts for the conjunctive position-by-velocity selectivity of grid cells. Our network model is able to perform robust path integration even when the recurrent connections are subject to random perturbations.     

Optimal configurations of spatial scale for grid cell firing under noise and uncertainty

We examined the accuracy with which the location of an agent moving within an environment could be decoded from the simulated firing of systems of grid cells. Grid cells were modelled with Poisson spiking dynamics and organized into multiple ‘modules’ of cells, with firing patterns of similar spatial scale within modules and a wide range of spatial scales across modules. The number of grid cells per module, the spatial scaling factor between modules and the size of the environment were varied. Errors in decoded location can take two forms: small errors of precision and larger errors resulting from ambiguity in decoding periodic firing patterns. With enough cells per module (e.g. eight modules of 100 cells each) grid systems are highly robust to ambiguity errors, even over ranges much larger than the largest grid scale (e.g. over a 500 m range when the maximum grid scale is 264 cm). Results did not depend strongly on the precise organization of scales across modules (geometric, co-prime or random). However, independent spatial noise across modules, which would occur if modules receive independent spatial inputs and might increase with spatial uncertainty, dramatically degrades the performance of the grid system. This effect of spatial uncertainty can be mitigated by uniform expansion of grid scales. Thus, in the realistic regimes simulated here, the optimal overall scale for a grid system represents a trade-off between minimizing spatial uncertainty (requiring large scales) and maximizing precision (requiring small scales). Within this view, the temporary expansion of grid scales observed in novel environments may be an optimal response to increased spatial uncertainty induced by the unfamiliarity of the available spatial cues.     

Grid alignment in entorhinal cortex

The spatial responses of many of the cells recorded in all layers of rodent medial entorhinal cortex (mEC) show mutually aligned grid patterns. Recent experimental findings have shown that grids can often be better described as elliptical rather than purely circular and that, beyond the mutual alignment of their grid axes, ellipses tend to also orient their long axis along preferred directions. Are grid alignment and ellipse orientation aspects of the same phenomenon? Does the grid alignment result from single-unit mechanisms or does it require network interactions? We address these issues by refining a single-unit adaptation model of grid formation, to describe specifically the spontaneous emergence of conjunctive grid-by-head-direction cells in layers III, V, and VI of mEC. We find that tight alignment can be produced by recurrent collateral interactions, but this requires head-direction (HD) modulation. Through a competitive learning process driven by spatial inputs, grid fields then form already aligned, and with randomly distributed spatial phases. In addition, we find that the self-organization process is influenced by any anisotropy in the behavior of the simulated rat. The common grid alignment often orients along preferred running directions (RDs), as induced in a square environment. When speed anisotropy is present in exploration behavior, the shape of individual grids is distorted toward an ellipsoid arrangement. Speed anisotropy orients the long ellipse axis along the fast direction. Speed anisotropy on its own also tends to align grids, even without collaterals, but the alignment is seen to be loose. Finally, the alignment of spatial grid fields in multiple environments shows that the network expresses the same set of grid fields across environments, modulo a coherent rotation and translation. Thus, an efficient metric encoding of space may emerge through spontaneous pattern formation at the single-unit level, but it is coherent, hence context-invariant, if aided by collateral interactions.     

A model of head direction and landmark coding in complex environments

Environmental information is required to stabilize estimates of head direction (HD) based on angular path integration. However, it is unclear how this happens in real-world (visually complex) environments. We present a computational model of how visual feedback can stabilize HD information in environments that contain multiple cues of varying stability and directional specificity. We show how combinations of feature-specific visual inputs can generate a stable unimodal landmark bearing signal, even in the presence of multiple cues and ambiguous directional specificity. This signal is associated with the retrosplenial HD signal (inherited from thalamic HD cells) and conveys feedback to the subcortical HD circuitry. The model predicts neurons with a unimodal encoding of the egocentric orientation of the array of landmarks, rather than any one particular landmark. The relationship between these abstract landmark bearing neurons and head direction cells is reminiscent of the relationship between place cells and grid cells. Their unimodal encoding is formed from visual inputs via a modified version of Oja’s Subspace Algorithm. The rule allows the landmark bearing signal to disconnect from directionally unstable or ephemeral cues, incorporate newly added stable cues, support orientation across many different environments (high memory capacity), and is consistent with recent empirical findings on bidirectional HD firing reported in the retrosplenial cortex. Our account of visual feedback for HD stabilization provides a novel perspective on neural mechanisms of spatial navigation within richer sensory environments, and makes experimentally testable predictions.     

Estimating Location without External Cues

The ability to determine one''s location is fundamental to spatial navigation. Here, it is shown that localization is theoretically possible without the use of external cues, and without knowledge of initial position or orientation. With only error-prone self-motion estimates as input, a fully disoriented agent can, in principle, determine its location in familiar spaces with 1-fold rotational symmetry. Surprisingly, localization does not require the sensing of any external cue, including the boundary. The combination of self-motion estimates and an internal map of the arena provide enough information for localization. This stands in conflict with the supposition that 2D arenas are analogous to open fields. Using a rodent error model, it is shown that the localization performance which can be achieved is enough to initiate and maintain stable firing patterns like those of grid cells, starting from full disorientation. Successful localization was achieved when the rotational asymmetry was due to the external boundary, an interior barrier or a void space within an arena. Optimal localization performance was found to depend on arena shape, arena size, local and global rotational asymmetry, and the structure of the path taken during localization. Since allothetic cues including visual and boundary contact cues were not present, localization necessarily relied on the fusion of idiothetic self-motion cues and memory of the boundary. Implications for spatial navigation mechanisms are discussed, including possible relationships with place field overdispersion and hippocampal reverse replay. Based on these results, experiments are suggested to identify if and where information fusion occurs in the mammalian spatial memory system.     

How Entorhinal Grid Cells May Learn Multiple Spatial Scales from a Dorsoventral Gradient of Cell Response Rates in a Self-organizing Map

Place cells in the hippocampus of higher mammals are critical for spatial navigation. Recent modeling clarifies how this may be achieved by how grid cells in the medial entorhinal cortex (MEC) input to place cells. Grid cells exhibit hexagonal grid firing patterns across space in multiple spatial scales along the MEC dorsoventral axis. Signals from grid cells of multiple scales combine adaptively to activate place cells that represent much larger spaces than grid cells. But how do grid cells learn to fire at multiple positions that form a hexagonal grid, and with spatial scales that increase along the dorsoventral axis? In vitro recordings of medial entorhinal layer II stellate cells have revealed subthreshold membrane potential oscillations (MPOs) whose temporal periods, and time constants of excitatory postsynaptic potentials (EPSPs), both increase along this axis. Slower (faster) subthreshold MPOs and slower (faster) EPSPs correlate with larger (smaller) grid spacings and field widths. A self-organizing map neural model explains how the anatomical gradient of grid spatial scales can be learned by cells that respond more slowly along the gradient to their inputs from stripe cells of multiple scales, which perform linear velocity path integration. The model cells also exhibit MPO frequencies that covary with their response rates. The gradient in intrinsic rhythmicity is thus not compelling evidence for oscillatory interference as a mechanism of grid cell firing. A response rate gradient combined with input stripe cells that have normalized receptive fields can reproduce all known spatial and temporal properties of grid cells along the MEC dorsoventral axis. This spatial gradient mechanism is homologous to a gradient mechanism for temporal learning in the lateral entorhinal cortex and its hippocampal projections. Spatial and temporal representations may hereby arise from homologous mechanisms, thereby embodying a mechanistic “neural relativity” that may clarify how episodic memories are learned.     

Information-Driven Active Audio-Visual Source Localization

We present a system for sensorimotor audio-visual source localization on a mobile robot. We utilize a particle filter for the combination of audio-visual information and for the temporal integration of consecutive measurements. Although the system only measures the current direction of the source, the position of the source can be estimated because the robot is able to move and can therefore obtain measurements from different directions. These actions by the robot successively reduce uncertainty about the source’s position. An information gain mechanism is used for selecting the most informative actions in order to minimize the number of actions required to achieve accurate and precise position estimates in azimuth and distance. We show that this mechanism is an efficient solution to the action selection problem for source localization, and that it is able to produce precise position estimates despite simplified unisensory preprocessing. Because of the robot’s mobility, this approach is suitable for use in complex and cluttered environments. We present qualitative and quantitative results of the system’s performance and discuss possible areas of application.     

Theta phase precession of grid and place cell firing in open environments

Place and grid cells in the rodent hippocampal formation tend to fire spikes at successively earlier phases relative to the local field potential theta rhythm as the animal runs through the cell''s firing field on a linear track. However, this ‘phase precession’ effect is less well characterized during foraging in two-dimensional open field environments. Here, we mapped runs through the firing fields onto a unit circle to pool data from multiple runs. We asked which of seven behavioural and physiological variables show the best circular–linear correlation with the theta phase of spikes from place cells in hippocampal area CA1 and from grid cells from superficial layers of medial entorhinal cortex. The best correlate was the distance to the firing field peak projected onto the animal''s current running direction. This was significantly stronger than other correlates, such as instantaneous firing rate and time-in-field, but similar in strength to correlates with other measures of distance travelled through the firing field. Phase precession was stronger in place cells than grid cells overall, and robust phase precession was seen in traversals through firing field peripheries (although somewhat less than in traversals through the centre), consistent with phase coding of displacement along the current direction. This type of phase coding, of place field distance ahead of or behind the animal, may be useful for allowing calculation of goal directions during navigation.     

Parallel and interrelated neural systems underlying adaptive navigation

The ability to process in parallel multiple forms of sensoryinformation, and link sensory-sensory associations to behavior,presumably allows for the opportunistic use of the most reliableand predictive sensory modalities in diverse behavioral contexts.Evolutionary considerations indicate that such processing mayrepresent a fundamental operating principle underlying complexsensory associations and sensory-motor integration. Here, wesuggest that animal navigation is a particularly useful modelof such opportunistic use of sensory and motor information becauseit is possible to study directly the effects of memory on neuralsystem functions. First, comparative evidence for parallel processingacross multiple brain structures during navigation is providedfrom the literatures on fish and rodent navigation. Then, basedon neurophysiological evidence of coordinated, multiregionalprocessing, we provide a neurobiological explanation of learningand memory effects on neural circuitry mediating navigation.     

How do insects use path integration for their navigation?

 We combine experimental findings on ants and bees, and build on earlier models, to give an account of how these insects navigate using path integration, and how path integration interacts with other modes of navigation. At the core of path integration is an accumulator. This is set to an initial state at the nest and is updated as the insect moves so that it always reports the insect's current position relative to the nest. Navigation that uses path integration requires, in addition, a way of storing states of the accumulator at significant places for subsequent recall as goals, and a means of computing the direction to such goals. We discuss three models of how path integration might be used for this process, which we call vector navigation. Vector navigation is the principal means of navigating over unfamiliar terrain, or when landmarks are unavailable. Under other conditions, insects often navigate by landmarks, and ignore the output of the vector navigation system. Landmark navigation does not interfere with the updating of the accumulator. There is an interesting symmetry in the use of landmarks and path integration. In the short term, vector navigation can be independent of landmarks, and landmark navigation needs no assistance from path integration. In the longer term, visual landmarks help keep path vector navigation calibrated, and the learning of visual landmarks is guided by path integration. Received: 6 June 1999 / Accepted in revised form: 20 March 2000     

A novel somatosensory spatial navigation system outside the hippocampal formation

Spatially selective firing of place cells, grid cells, boundary vector/border cells and head direction cells constitutes the basic building blocks of a canonical spatial navigation system centered on the hippocampal-entorhinal complex. While head direction cells can be found throughout the brain, spatial tuning outside the hippocampal formation is often non-specific or conjunctive to other representations such as a reward. Although the precise mechanism of spatially selective firing activity is not understood, various studies show sensory inputs, particularly vision, heavily modulate spatial representation in the hippocampal-entorhinal circuit. To better understand the contribution of other sensory inputs in shaping spatial representation in the brain, we performed recording from the primary somatosensory cortex in foraging rats. To our surprise, we were able to detect the full complement of spatially selective firing patterns similar to that reported in the hippocampal-entorhinal network, namely, place cells, head direction cells, boundary vector/border cells, grid cells and conjunctive cells, in the somatosensory cortex. These newly identified somatosensory spatial cells form a spatial map outside the hippocampal formation and support the hypothesis that location information modulates body representation in the somatosensory cortex. Our findings provide transformative insights into our understanding of how spatial information is processed and integrated in the brain, as well as functional operations of the somatosensory cortex in the context of rehabilitation with brain-machine interfaces.Subject terms: Biological techniques, Cell biology     

Self-organised path formation in a swarm of robots

In this paper, we study the problem of exploration and navigation in an unknown environment from an evolutionary swarm robotics perspective. In other words, we search for an efficient exploration and navigation strategy for a swarm of robots, which exploits cooperation and self-organisation to cope with the limited abilities of the individual robots. The task faced by the robots consists in the exploration of an unknown environment in order to find a path between two distant target areas. The collective strategy is synthesised through evolutionary robotics techniques, and is based on the emergence of a dynamic structure formed by the robots moving back and forth between the two target areas. Due to this structure, each robot is able to maintain the right heading and to efficiently navigate between the two areas. The evolved behaviour proved to be effective in finding the shortest path, adaptable to new environmental conditions, scalable to larger groups and larger environment size, and robust to individual failures.     

Spiking Neurons in a Hierarchical Self-Organizing Map Model Can Learn to Develop Spatial and Temporal Properties of Entorhinal Grid Cells and Hippocampal Place Cells

Medial entorhinal grid cells and hippocampal place cells provide neural correlates of spatial representation in the brain. A place cell typically fires whenever an animal is present in one or more spatial regions, or places, of an environment. A grid cell typically fires in multiple spatial regions that form a regular hexagonal grid structure extending throughout the environment. Different grid and place cells prefer spatially offset regions, with their firing fields increasing in size along the dorsoventral axes of the medial entorhinal cortex and hippocampus. The spacing between neighboring fields for a grid cell also increases along the dorsoventral axis. This article presents a neural model whose spiking neurons operate in a hierarchy of self-organizing maps, each obeying the same laws. This spiking GridPlaceMap model simulates how grid cells and place cells may develop. It responds to realistic rat navigational trajectories by learning grid cells with hexagonal grid firing fields of multiple spatial scales and place cells with one or more firing fields that match neurophysiological data about these cells and their development in juvenile rats. The place cells represent much larger spaces than the grid cells, which enable them to support navigational behaviors. Both self-organizing maps amplify and learn to categorize the most frequent and energetic co-occurrences of their inputs. The current results build upon a previous rate-based model of grid and place cell learning, and thus illustrate a general method for converting rate-based adaptive neural models, without the loss of any of their analog properties, into models whose cells obey spiking dynamics. New properties of the spiking GridPlaceMap model include the appearance of theta band modulation. The spiking model also opens a path for implementation in brain-emulating nanochips comprised of networks of noisy spiking neurons with multiple-level adaptive weights for controlling autonomous adaptive robots capable of spatial navigation.     

Coordinated learning of grid cell and place cell spatial and temporal properties: multiple scales,attention and oscillations

A neural model proposes how entorhinal grid cells and hippocampal place cells may develop as spatial categories in a hierarchy of self-organizing maps (SOMs). The model responds to realistic rat navigational trajectories by learning both grid cells with hexagonal grid firing fields of multiple spatial scales, and place cells with one or more firing fields, that match neurophysiological data about their development in juvenile rats. Both grid and place cells can develop by detecting, learning and remembering the most frequent and energetic co-occurrences of their inputs. The model''s parsimonious properties include: similar ring attractor mechanisms process linear and angular path integration inputs that drive map learning; the same SOM mechanisms can learn grid cell and place cell receptive fields; and the learning of the dorsoventral organization of multiple spatial scale modules through medial entorhinal cortex to hippocampus (HC) may use mechanisms homologous to those for temporal learning through lateral entorhinal cortex to HC (‘neural relativity’). The model clarifies how top-down HC-to-entorhinal attentional mechanisms may stabilize map learning, simulates how hippocampal inactivation may disrupt grid cells, and explains data about theta, beta and gamma oscillations. The article also compares the three main types of grid cell models in the light of recent data.     

Robust path integration in the entorhinal grid cell system with hippocampal feed-back

Animals are able to update their knowledge about their current position solely by integrating the speed and the direction of their movement, which is known as path integration. Recent discoveries suggest that grid cells in the medial entorhinal cortex might perform some of the essential underlying computations of path integration. However, a major concern over path integration is that as the measurement of speed and direction is inaccurate, the representation of the position will become increasingly unreliable. In this paper, we study how allothetic inputs can be used to continually correct the accumulating error in the path integrator system. We set up the model of a mobile agent equipped with the entorhinal representation of idiothetic (grid cell) and allothetic (visual cells) information and simulated its place learning in a virtual environment. Due to competitive learning, a robust hippocampal place code emerges rapidly in the model. At the same time, the hippocampo-entorhinal feed-back connections are modified via Hebbian learning in order to allow hippocampal place cells to influence the attractor dynamics in the entorhinal cortex. We show that the continuous feed-back from the integrated hippocampal place representation is able to stabilize the grid cell code. This research was supported by the EU Framework 6 ICEA project (IST-4-027819-IP).     

Robotic and neuronal simulation of the hippocampus and rat navigation.

The properties of hippocampal place cells are reviewed, with particular attention to the nature of the internal and external signals that support their firing. A neuronal simulation of the firing of place cells in open-field environments of varying shape is presented. This simulation is coupled with an existing model of how place-cell firing can be used to drive navigation, and is tested by implementation as a miniature mobile robot. The sensors on the robot provide visual, odometric and short-range proximity data, which are combined to estimate the distance of the walls of the enclosure from the robot and the robot''s current heading direction. These inputs drive the hippocampal simulation, in which the robot''s location is represented as the firing of place cells. If a goal location is encountered, learning occurs in connections from the concurrently active place cells to a set of ''goal cells'', which guide subsequent navigation, allowing the robot to return to an unmarked location. The system shows good agreement with actual place-cell firing, and makes predictions regarding the firing of cells in the subiculum, the effect of blocking long-term synaptic changes, and the locus of search of rats after deformation of their environment.     

Environment Recognition System Based on Multiple Classification Analyses for Mobile Robots

Various mechanisms have recently been developed that combine linkage mechanisms and wheels. In particular, the combination of passive linkage mechanisms and small wheels is a main research trend because standard wheeled mobile mechanisms find it difficult to move on rough terrain. In our previous research, a six-wheel mobile robot employing a passive linkage mechanism has been developed to enhance maneuverability and was able to climb over a 0.20 m bump and stairs. We designed a hybrid velocity and torque controller using a neural network since simple velocity controllers fail to climb up. In this paper, we propose an environment recognition system for a wheeled mobile robot that consists of multiple classification analyses to make the robot more adaptive to various environments by selecting a suitable system such as decision making, navigation and controller using the result of the environment recognition system. We evaluate the recognition performance in operation environments; slopes, bumps and stairs by comparing principle component, k-means and self-organizing map analyses.     

Navigating sensory conflict in dynamic environments using adaptive state estimation

Most conventional robots rely on controlling the location of the center of pressure to maintain balance, relying mainly on foot pressure sensors for information. By contrast, humans rely on sensory data from multiple sources, including proprioceptive, visual, and vestibular sources. Several models have been developed to explain how humans reconcile information from disparate sources to form a stable sense of balance. These models may be useful for developing robots that are able to maintain dynamic balance more readily using multiple sensory sources. Since these information sources may conflict, reliance by the nervous system on any one channel can lead to ambiguity in the system state. In humans, experiments that create conflicts between different sensory channels by moving the visual field or the support surface indicate that sensory information is adaptively reweighted. Unreliable information is rapidly down-weighted, then gradually up-weighted when it becomes valid again. Human balance can also be studied by building robots that model features of human bodies and testing them under similar experimental conditions. We implement a sensory reweighting model based on an adaptive Kalman filter in a bipedal robot, and subject it to sensory tests similar to those used on human subjects. Unlike other implementations of sensory reweighting in robots, our implementation includes vision, by using optic flow to calculate forward rotation using a camera (visual modality), as well as a three-axis gyro to represent the vestibular system (non-visual modality), and foot pressure sensors (proprioceptive modality). Our model estimates measurement noise in real time, which is then used to recompute the Kalman gain on each iteration, improving the ability of the robot to dynamically balance. We observe that we can duplicate many important features of postural sway in humans, including automatic sensory reweighting, effects, constant phase with respect to amplitude, and a temporal asymmetry in the reweighting gains.