Coordinated learning of grid cell and place cell spatial and temporal properties: multiple scales,attention and oscillations

A neural model proposes how entorhinal grid cells and hippocampal place cells may develop as spatial categories in a hierarchy of self-organizing maps (SOMs). The model responds to realistic rat navigational trajectories by learning both grid cells with hexagonal grid firing fields of multiple spatial scales, and place cells with one or more firing fields, that match neurophysiological data about their development in juvenile rats. Both grid and place cells can develop by detecting, learning and remembering the most frequent and energetic co-occurrences of their inputs. The model''s parsimonious properties include: similar ring attractor mechanisms process linear and angular path integration inputs that drive map learning; the same SOM mechanisms can learn grid cell and place cell receptive fields; and the learning of the dorsoventral organization of multiple spatial scale modules through medial entorhinal cortex to hippocampus (HC) may use mechanisms homologous to those for temporal learning through lateral entorhinal cortex to HC (‘neural relativity’). The model clarifies how top-down HC-to-entorhinal attentional mechanisms may stabilize map learning, simulates how hippocampal inactivation may disrupt grid cells, and explains data about theta, beta and gamma oscillations. The article also compares the three main types of grid cell models in the light of recent data.     

Spiking Neurons in a Hierarchical Self-Organizing Map Model Can Learn to Develop Spatial and Temporal Properties of Entorhinal Grid Cells and Hippocampal Place Cells

Medial entorhinal grid cells and hippocampal place cells provide neural correlates of spatial representation in the brain. A place cell typically fires whenever an animal is present in one or more spatial regions, or places, of an environment. A grid cell typically fires in multiple spatial regions that form a regular hexagonal grid structure extending throughout the environment. Different grid and place cells prefer spatially offset regions, with their firing fields increasing in size along the dorsoventral axes of the medial entorhinal cortex and hippocampus. The spacing between neighboring fields for a grid cell also increases along the dorsoventral axis. This article presents a neural model whose spiking neurons operate in a hierarchy of self-organizing maps, each obeying the same laws. This spiking GridPlaceMap model simulates how grid cells and place cells may develop. It responds to realistic rat navigational trajectories by learning grid cells with hexagonal grid firing fields of multiple spatial scales and place cells with one or more firing fields that match neurophysiological data about these cells and their development in juvenile rats. The place cells represent much larger spaces than the grid cells, which enable them to support navigational behaviors. Both self-organizing maps amplify and learn to categorize the most frequent and energetic co-occurrences of their inputs. The current results build upon a previous rate-based model of grid and place cell learning, and thus illustrate a general method for converting rate-based adaptive neural models, without the loss of any of their analog properties, into models whose cells obey spiking dynamics. New properties of the spiking GridPlaceMap model include the appearance of theta band modulation. The spiking model also opens a path for implementation in brain-emulating nanochips comprised of networks of noisy spiking neurons with multiple-level adaptive weights for controlling autonomous adaptive robots capable of spatial navigation.     

Functional connectivity of the entorhinal–hippocampal space circuit

The mammalian space circuit is known to contain several functionally specialized cell types, such as place cells in the hippocampus and grid cells, head-direction cells and border cells in the medial entorhinal cortex (MEC). The interaction between the entorhinal and hippocampal spatial representations is poorly understood, however. We have developed an optogenetic strategy to identify functionally defined cell types in the MEC that project directly to the hippocampus. By expressing channelrhodopsin-2 (ChR2) selectively in the hippocampus-projecting subset of entorhinal projection neurons, we were able to use light-evoked discharge as an instrument to determine whether specific entorhinal cell groups—such as grid cells, border cells and head-direction cells—have direct hippocampal projections. Photoinduced firing was observed at fixed minimal latencies in all functional cell categories, with grid cells as the most abundant hippocampus-projecting spatial cell type. We discuss how photoexcitation experiments can be used to distinguish the subset of hippocampus-projecting entorhinal neurons from neurons that are activated indirectly through the network. The functional breadth of entorhinal input implied by this analysis opens up the potential for rich dynamic interactions between place cells in the hippocampus and different functional cell types in the entorhinal cortex (EC).     

Hippocampal Remapping Is Constrained by Sparseness rather than Capacity

Grid cells in the medial entorhinal cortex encode space with firing fields that are arranged on the nodes of spatial hexagonal lattices. Potential candidates to read out the space information of this grid code and to combine it with other sensory cues are hippocampal place cells. In this paper, we investigate a population of grid cells providing feed-forward input to place cells. The capacity of the underlying synaptic transformation is determined by both spatial acuity and the number of different spatial environments that can be represented. The codes for different environments arise from phase shifts of the periodical entorhinal cortex patterns that induce a global remapping of hippocampal place fields, i.e., a new random assignment of place fields for each environment. If only a single environment is encoded, the grid code can be read out at high acuity with only few place cells. A surplus in place cells can be used to store a space code for more environments via remapping. The number of stored environments can be increased even more efficiently by stronger recurrent inhibition and by partitioning the place cell population such that learning affects only a small fraction of them in each environment. We find that the spatial decoding acuity is much more resilient to multiple remappings than the sparseness of the place code. Since the hippocampal place code is sparse, we thus conclude that the projection from grid cells to the place cells is not using its full capacity to transfer space information. Both populations may encode different aspects of space.     

How Entorhinal Grid Cells May Learn Multiple Spatial Scales from a Dorsoventral Gradient of Cell Response Rates in a Self-organizing Map

Place cells in the hippocampus of higher mammals are critical for spatial navigation. Recent modeling clarifies how this may be achieved by how grid cells in the medial entorhinal cortex (MEC) input to place cells. Grid cells exhibit hexagonal grid firing patterns across space in multiple spatial scales along the MEC dorsoventral axis. Signals from grid cells of multiple scales combine adaptively to activate place cells that represent much larger spaces than grid cells. But how do grid cells learn to fire at multiple positions that form a hexagonal grid, and with spatial scales that increase along the dorsoventral axis? In vitro recordings of medial entorhinal layer II stellate cells have revealed subthreshold membrane potential oscillations (MPOs) whose temporal periods, and time constants of excitatory postsynaptic potentials (EPSPs), both increase along this axis. Slower (faster) subthreshold MPOs and slower (faster) EPSPs correlate with larger (smaller) grid spacings and field widths. A self-organizing map neural model explains how the anatomical gradient of grid spatial scales can be learned by cells that respond more slowly along the gradient to their inputs from stripe cells of multiple scales, which perform linear velocity path integration. The model cells also exhibit MPO frequencies that covary with their response rates. The gradient in intrinsic rhythmicity is thus not compelling evidence for oscillatory interference as a mechanism of grid cell firing. A response rate gradient combined with input stripe cells that have normalized receptive fields can reproduce all known spatial and temporal properties of grid cells along the MEC dorsoventral axis. This spatial gradient mechanism is homologous to a gradient mechanism for temporal learning in the lateral entorhinal cortex and its hippocampal projections. Spatial and temporal representations may hereby arise from homologous mechanisms, thereby embodying a mechanistic “neural relativity” that may clarify how episodic memories are learned.     

Path integration and the neural basis of the 'cognitive map'

The hippocampal formation can encode relative spatial location, without reference to external cues, by the integration of linear and angular self-motion (path integration). Theoretical studies, in conjunction with recent empirical discoveries, suggest that the medial entorhinal cortex (MEC) might perform some of the essential underlying computations by means of a unique, periodic synaptic matrix that could be self-organized in early development through a simple, symmetry-breaking operation. The scale at which space is represented increases systematically along the dorsoventral axis in both the hippocampus and the MEC, apparently because of systematic variation in the gain of a movement-speed signal. Convergence of spatially periodic input at multiple scales, from so-called grid cells in the entorhinal cortex, might result in non-periodic spatial firing patterns (place fields) in the hippocampus.     

Grid cells use HCN1 channels for spatial scaling

Entorhinal grid cells have periodic, hexagonally patterned firing locations that scale up progressively along the dorsal-ventral axis of medial entorhinal cortex. This topographic expansion corresponds with parallel changes in cellular properties dependent on the hyperpolarization-activated cation current (Ih), which is conducted by hyperpolarization-activated cyclic nucleotide-gated (HCN) channels. To test the hypothesis that grid scale is determined by Ih, we recorded grid cells in mice with forebrain-specific knockout of HCN1. We find that, although the dorsal-ventral gradient of the grid pattern was preserved in HCN1 knockout mice, the size and spacing of the grid fields, as well as the period of the accompanying theta modulation, was expanded at all dorsal-ventral levels. There was no change in theta modulation of simultaneously recorded entorhinal interneurons. These observations raise the possibility that, during self-motion-based navigation, Ih contributes to the gain of the transformation from movement signals to spatial firing fields.     

Theta phase precession of grid and place cell firing in open environments

Place and grid cells in the rodent hippocampal formation tend to fire spikes at successively earlier phases relative to the local field potential theta rhythm as the animal runs through the cell''s firing field on a linear track. However, this ‘phase precession’ effect is less well characterized during foraging in two-dimensional open field environments. Here, we mapped runs through the firing fields onto a unit circle to pool data from multiple runs. We asked which of seven behavioural and physiological variables show the best circular–linear correlation with the theta phase of spikes from place cells in hippocampal area CA1 and from grid cells from superficial layers of medial entorhinal cortex. The best correlate was the distance to the firing field peak projected onto the animal''s current running direction. This was significantly stronger than other correlates, such as instantaneous firing rate and time-in-field, but similar in strength to correlates with other measures of distance travelled through the firing field. Phase precession was stronger in place cells than grid cells overall, and robust phase precession was seen in traversals through firing field peripheries (although somewhat less than in traversals through the centre), consistent with phase coding of displacement along the current direction. This type of phase coding, of place field distance ahead of or behind the animal, may be useful for allowing calculation of goal directions during navigation.     

The mechanism of rate remapping in the dentate gyrus

Rate remapping is a recently revealed neural code in which sensory information modulates the firing rate of hippocampal place cells. The mechanism underlying rate remapping is unknown. Its characteristic modulation, however, must arise from the interaction of the two major inputs to the hippocampus, the medial entorhinal cortex (MEC), in which grid cells represent the spatial position of the rat, and the lateral entorhinal cortex (LEC), in which cells represent the sensory properties of the environment. We have used computational methods to elucidate the mechanism by which this interaction produces rate remapping. We show that the convergence of LEC and MEC inputs, in conjunction with a competitive network process mediated by feedback inhibition, can account quantitatively for this phenomenon. The same principle accounts for why different place fields of the same cell vary independently as sensory information is altered. Our results show that rate remapping can be explained in terms of known mechanisms.     

Grid cells, place cells, and geodesic generalization for spatial reinforcement learning

Reinforcement learning (RL) provides an influential characterization of the brain's mechanisms for learning to make advantageous choices. An important problem, though, is how complex tasks can be represented in a way that enables efficient learning. We consider this problem through the lens of spatial navigation, examining how two of the brain's location representations--hippocampal place cells and entorhinal grid cells--are adapted to serve as basis functions for approximating value over space for RL. Although much previous work has focused on these systems' roles in combining upstream sensory cues to track location, revisiting these representations with a focus on how they support this downstream decision function offers complementary insights into their characteristics. Rather than localization, the key problem in learning is generalization between past and present situations, which may not match perfectly. Accordingly, although neural populations collectively offer a precise representation of position, our simulations of navigational tasks verify the suggestion that RL gains efficiency from the more diffuse tuning of individual neurons, which allows learning about rewards to generalize over longer distances given fewer training experiences. However, work on generalization in RL suggests the underlying representation should respect the environment's layout. In particular, although it is often assumed that neurons track location in Euclidean coordinates (that a place cell's activity declines "as the crow flies" away from its peak), the relevant metric for value is geodesic: the distance along a path, around any obstacles. We formalize this intuition and present simulations showing how Euclidean, but not geodesic, representations can interfere with RL by generalizing inappropriately across barriers. Our proposal that place and grid responses should be modulated by geodesic distances suggests novel predictions about how obstacles should affect spatial firing fields, which provides a new viewpoint on data concerning both spatial codes.     

Robust path integration in the entorhinal grid cell system with hippocampal feed-back

Animals are able to update their knowledge about their current position solely by integrating the speed and the direction of their movement, which is known as path integration. Recent discoveries suggest that grid cells in the medial entorhinal cortex might perform some of the essential underlying computations of path integration. However, a major concern over path integration is that as the measurement of speed and direction is inaccurate, the representation of the position will become increasingly unreliable. In this paper, we study how allothetic inputs can be used to continually correct the accumulating error in the path integrator system. We set up the model of a mobile agent equipped with the entorhinal representation of idiothetic (grid cell) and allothetic (visual cells) information and simulated its place learning in a virtual environment. Due to competitive learning, a robust hippocampal place code emerges rapidly in the model. At the same time, the hippocampo-entorhinal feed-back connections are modified via Hebbian learning in order to allow hippocampal place cells to influence the attractor dynamics in the entorhinal cortex. We show that the continuous feed-back from the integrated hippocampal place representation is able to stabilize the grid cell code. This research was supported by the EU Framework 6 ICEA project (IST-4-027819-IP).     

Development of cue integration in human navigation

Mammalian navigation depends both on visual landmarks and on self-generated (e.g., vestibular and proprioceptive) cues that signal the organism's own movement [1-5]. When these conflict, landmarks can either reset estimates of self-motion or be integrated with them [6-9]. We asked how humans combine these information sources and whether children, who use both from a young age [10-12], combine them as adults do. Participants attempted to return an object to its original place in an arena when given either visual landmarks only, nonvisual self-motion information only, or both. Adults, but not 4- to 5-year-olds or 7- to 8-year-olds, reduced their response variance when both information sources were available. In an additional "conflict" condition that measured relative reliance on landmarks and self-motion, we predicted behavior under two models: integration (weighted averaging) of the cues and alternation between them. Adults' behavior was predicted by integration, in which the cues were weighted nearly optimally to reduce variance, whereas children's behavior was predicted by alternation. These results suggest that development of individual spatial-representational systems precedes development of the capacity to combine these within a common reference frame. Humans can integrate spatial cues nearly optimally to navigate, but this ability depends on an extended developmental process.     

A feedforward model for the formation of a grid field where spatial information is provided solely from place cells

Grid cells (GCs) in the medial entorhinal cortex (mEC) have the property of having their firing activity spatially tuned to a regular triangular lattice. Several theoretical models for grid field formation have been proposed, but most assume that place cells (PCs) are a product of the grid cell system. There is, however, an alternative possibility that is supported by various strands of experimental data. Here we present a novel model for the emergence of gridlike firing patterns that stands on two key hypotheses: (1) spatial information in GCs is provided from PC activity and (2) grid fields result from a combined synaptic plasticity mechanism involving inhibitory and excitatory neurons mediating the connections between PCs and GCs. Depending on the spatial location, each PC can contribute with excitatory or inhibitory inputs to GC activity. The nature and magnitude of the PC input is a function of the distance to the place field center, which is inferred from rate decoding. A biologically plausible learning rule drives the evolution of the connection strengths from PCs to a GC. In this model, PCs compete for GC activation, and the plasticity rule favors efficient packing of the space representation. This leads to gridlike firing patterns. In a new environment, GCs continuously recruit new PCs to cover the entire space. The model described here makes important predictions and can represent the feedforward connections from hippocampus CA1 to deeper mEC layers.     

Neuronal rebound spiking,resonance frequency and theta cycle skipping may contribute to grid cell firing in medial entorhinal cortex

Data show a relationship of cellular resonance and network oscillations in the entorhinal cortex to the spatial periodicity of grid cells. This paper presents a model that simulates the resonance and rebound spiking properties of entorhinal neurons to generate spatial periodicity dependent upon phasic input from medial septum. The model shows that a difference in spatial periodicity can result from a difference in neuronal resonance frequency that replicates data from several experiments. The model also demonstrates a functional role for the phenomenon of theta cycle skipping in the medial entorhinal cortex.     

Estimating Location without External Cues

The ability to determine one''s location is fundamental to spatial navigation. Here, it is shown that localization is theoretically possible without the use of external cues, and without knowledge of initial position or orientation. With only error-prone self-motion estimates as input, a fully disoriented agent can, in principle, determine its location in familiar spaces with 1-fold rotational symmetry. Surprisingly, localization does not require the sensing of any external cue, including the boundary. The combination of self-motion estimates and an internal map of the arena provide enough information for localization. This stands in conflict with the supposition that 2D arenas are analogous to open fields. Using a rodent error model, it is shown that the localization performance which can be achieved is enough to initiate and maintain stable firing patterns like those of grid cells, starting from full disorientation. Successful localization was achieved when the rotational asymmetry was due to the external boundary, an interior barrier or a void space within an arena. Optimal localization performance was found to depend on arena shape, arena size, local and global rotational asymmetry, and the structure of the path taken during localization. Since allothetic cues including visual and boundary contact cues were not present, localization necessarily relied on the fusion of idiothetic self-motion cues and memory of the boundary. Implications for spatial navigation mechanisms are discussed, including possible relationships with place field overdispersion and hippocampal reverse replay. Based on these results, experiments are suggested to identify if and where information fusion occurs in the mammalian spatial memory system.     

Place cells, neocortex and spatial navigation: a short review

Hippocampal place cells are characterized by location-specific firing, that is each cell fires in a restricted region of the environment explored by the rat. In this review, we briefly examine the sensory information used by place cells to anchor their firing fields in space and show that, among the various sensory cues that can influence place cell activity, visual and motion-related cues are the most relevant. We then explore the contribution of several cortical areas to the generation of the place cell signal with an emphasis on the role of the visual cortex and parietal cortex. Finally, we address the functional significance of place cell activity and demonstrate the existence of a clear relationship between place cell positional activity and spatial navigation performance. We conclude that place cells, together with head direction cells, provide information useful for spatially guided movements, and thus provide a unique model of how spatial information is encoded in the brain.     

Increased size and stability of CA1 and CA3 place fields in HCN1 knockout mice

Hippocampal CA1 and CA3 pyramidal neuron place cells encode the spatial location of an animal through localized firing patterns called "place fields." To explore the mechanisms that control place cell firing and their relationship to spatial memory, we studied mice with enhanced spatial memory resulting from forebrain-specific knockout of the HCN1 hyperpolarization-activated cation channel. HCN1 is strongly expressed in CA1 neurons and in entorhinal cortex grid cells, which provide spatial information to the hippocampus. Both CA1 and CA3 place fields were larger but more stable in the knockout mice, with the effect greater in CA1 than CA3. As HCN1 is only weakly expressed in CA3 place cells, their altered activity likely reflects loss of HCN1 in grid cells. The more pronounced changes in CA1 likely reflect the intrinsic contribution of HCN1. The enhanced place field stability may underlie the effect of HCN1 deletion to facilitate spatial learning and memory.     

How to build a grid cell

Neurons in the medial entorhinal cortex fire action potentials at regular spatial intervals, creating a striking grid-like pattern of spike rates spanning the whole environment of a navigating animal. This remarkable spatial code may represent a neural map for path integration. Recent advances using patch-clamp recordings from entorhinal cortex neurons in vitro and in vivo have revealed how the microcircuitry in the medial entorhinal cortex may contribute to grid cell firing patterns, and how grid cells may transform synaptic inputs into spike output during firing field crossings. These new findings provide key insights into the ingredients necessary to build a grid cell.     

Spatial cognition and neuro-mimetic navigation: a model of hippocampal place cell activity

 A computational model of hippocampal activity during spatial cognition and navigation tasks is presented. The spatial representation in our model of the rat hippocampus is built on-line during exploration via two processing streams. An allothetic vision-based representation is built by unsupervised Hebbian learning extracting spatio-temporal properties of the environment from visual input. An idiothetic representation is learned based on internal movement-related information provided by path integration. On the level of the hippocampus, allothetic and idiothetic representations are integrated to yield a stable representation of the environment by a population of localized overlapping CA3-CA1 place fields. The hippocampal spatial representation is used as a basis for goal-oriented spatial behavior. We focus on the neural pathway connecting the hippocampus to the nucleus accumbens. Place cells drive a population of locomotor action neurons in the nucleus accumbens. Reward-based learning is applied to map place cell activity into action cell activity. The ensemble action cell activity provides navigational maps to support spatial behavior. We present experimental results obtained with a mobile Khepera robot. Received: 02 July 1999 / Accepted in revised form: 20 March 2000     

Solving navigational uncertainty using grid cells on robots

To successfully navigate their habitats, many mammals use a combination of two mechanisms, path integration and calibration using landmarks, which together enable them to estimate their location and orientation, or pose. In large natural environments, both these mechanisms are characterized by uncertainty: the path integration process is subject to the accumulation of error, while landmark calibration is limited by perceptual ambiguity. It remains unclear how animals form coherent spatial representations in the presence of such uncertainty. Navigation research using robots has determined that uncertainty can be effectively addressed by maintaining multiple probabilistic estimates of a robot's pose. Here we show how conjunctive grid cells in dorsocaudal medial entorhinal cortex (dMEC) may maintain multiple estimates of pose using a brain-based robot navigation system known as RatSLAM. Based both on rodent spatially-responsive cells and functional engineering principles, the cells at the core of the RatSLAM computational model have similar characteristics to rodent grid cells, which we demonstrate by replicating the seminal Moser experiments. We apply the RatSLAM model to a new experimental paradigm designed to examine the responses of a robot or animal in the presence of perceptual ambiguity. Our computational approach enables us to observe short-term population coding of multiple location hypotheses, a phenomenon which would not be easily observable in rodent recordings. We present behavioral and neural evidence demonstrating that the conjunctive grid cells maintain and propagate multiple estimates of pose, enabling the correct pose estimate to be resolved over time even without uniquely identifying cues. While recent research has focused on the grid-like firing characteristics, accuracy and representational capacity of grid cells, our results identify a possible critical and unique role for conjunctive grid cells in filtering sensory uncertainty. We anticipate our study to be a starting point for animal experiments that test navigation in perceptually ambiguous environments.