Spiking Neurons in a Hierarchical Self-Organizing Map Model Can Learn to Develop Spatial and Temporal Properties of Entorhinal Grid Cells and Hippocampal Place Cells

Medial entorhinal grid cells and hippocampal place cells provide neural correlates of spatial representation in the brain. A place cell typically fires whenever an animal is present in one or more spatial regions, or places, of an environment. A grid cell typically fires in multiple spatial regions that form a regular hexagonal grid structure extending throughout the environment. Different grid and place cells prefer spatially offset regions, with their firing fields increasing in size along the dorsoventral axes of the medial entorhinal cortex and hippocampus. The spacing between neighboring fields for a grid cell also increases along the dorsoventral axis. This article presents a neural model whose spiking neurons operate in a hierarchy of self-organizing maps, each obeying the same laws. This spiking GridPlaceMap model simulates how grid cells and place cells may develop. It responds to realistic rat navigational trajectories by learning grid cells with hexagonal grid firing fields of multiple spatial scales and place cells with one or more firing fields that match neurophysiological data about these cells and their development in juvenile rats. The place cells represent much larger spaces than the grid cells, which enable them to support navigational behaviors. Both self-organizing maps amplify and learn to categorize the most frequent and energetic co-occurrences of their inputs. The current results build upon a previous rate-based model of grid and place cell learning, and thus illustrate a general method for converting rate-based adaptive neural models, without the loss of any of their analog properties, into models whose cells obey spiking dynamics. New properties of the spiking GridPlaceMap model include the appearance of theta band modulation. The spiking model also opens a path for implementation in brain-emulating nanochips comprised of networks of noisy spiking neurons with multiple-level adaptive weights for controlling autonomous adaptive robots capable of spatial navigation.     

Coordinated learning of grid cell and place cell spatial and temporal properties: multiple scales,attention and oscillations

A neural model proposes how entorhinal grid cells and hippocampal place cells may develop as spatial categories in a hierarchy of self-organizing maps (SOMs). The model responds to realistic rat navigational trajectories by learning both grid cells with hexagonal grid firing fields of multiple spatial scales, and place cells with one or more firing fields, that match neurophysiological data about their development in juvenile rats. Both grid and place cells can develop by detecting, learning and remembering the most frequent and energetic co-occurrences of their inputs. The model''s parsimonious properties include: similar ring attractor mechanisms process linear and angular path integration inputs that drive map learning; the same SOM mechanisms can learn grid cell and place cell receptive fields; and the learning of the dorsoventral organization of multiple spatial scale modules through medial entorhinal cortex to hippocampus (HC) may use mechanisms homologous to those for temporal learning through lateral entorhinal cortex to HC (‘neural relativity’). The model clarifies how top-down HC-to-entorhinal attentional mechanisms may stabilize map learning, simulates how hippocampal inactivation may disrupt grid cells, and explains data about theta, beta and gamma oscillations. The article also compares the three main types of grid cell models in the light of recent data.     

Place cells,navigational accuracy,and the human hippocampus.

The hippocampal formation in both rats and humans is involved in spatial navigation. In the rat, cells coding for places, directions, and speed of movement have been recorded from the hippocampus proper and/or the neighbouring subicular complex. Place fields of a group of the hippocampal pyramidal cells cover the surface of an environment but do not appear to do so in any systematic fashion. That is, there is no topographical relation between the anatomical location of the cells within the hippocampus and the place fields of these cells in an environment. Recent work shows that place cells are responding to the summation of two or more Gaussian curves, each of which is fixed at a given distance to two or more walls in the environment. The walls themselves are probably identified by their allocentric direction relative to the rat and this information may be provided by the head direction cells. The right human hippocampus retains its role in spatial mapping as demonstrated by its activation during accurate navigation in imagined and virtual reality environments. In addition, it may have taken on wider memory functions, perhaps by the incorporation of a linear time tag which allows for the storage of the times of visits to particular locations. This extended system would serve as the basis for a spatio-temporal event or episodic memory system.     

Neural dynamics of the cognitive map in the hippocampus

The rodent hippocampus has been thought to represent the spatial environment as a cognitive map. In the classical theory, the cognitive map has been explained as a consequence of the fact that different spatial regions are assigned to different cell populations in the framework of rate coding. Recently, the relation between place cell firing and local field oscillation theta in terms of theta phase precession was experimentally discovered and suggested as a temporal coding mechanism leading to memory formation of behavioral sequences accompanied with asymmetric Hebbian plasticity. The cognitive map theory is apparently outside of the sequence memory view. Therefore, theoretical analysis is necessary to consider the biological neural dynamics for the sequence encoding of the memory of behavioral sequences, providing the cognitive map formation. In this article, we summarize the theoretical neural dynamics of the real-time sequence encoding by theta phase precession, called theta phase coding, and review a series of theoretical models with the theta phase coding that we previously reported. With respect to memory encoding functions, instantaneous memory formation of one-time experience was first demonstrated, and then the ability of integration of memories of behavioral sequences into a network of the cognitive map was shown. In terms of memory retrieval functions, theta phase coding enables the hippocampus to represent the spatial location in the current behavioral context even with ambiguous sensory input when multiple sequences were coded. Finally, for utilization, retrieved temporal sequences in the hippocampus can be available for action selection, through the process of reverting theta rhythm-dependent activities to information in the behavioral time scale. This theoretical approach allows us to investigate how the behavioral sequences are encoded, updated, retrieved and used in the hippocampus, as the real-time interaction with the external environment. It may indeed be the bridge to the episodic memory function in human hippocampus.     

Spatial cognition and neuro-mimetic navigation: a model of hippocampal place cell activity

 A computational model of hippocampal activity during spatial cognition and navigation tasks is presented. The spatial representation in our model of the rat hippocampus is built on-line during exploration via two processing streams. An allothetic vision-based representation is built by unsupervised Hebbian learning extracting spatio-temporal properties of the environment from visual input. An idiothetic representation is learned based on internal movement-related information provided by path integration. On the level of the hippocampus, allothetic and idiothetic representations are integrated to yield a stable representation of the environment by a population of localized overlapping CA3-CA1 place fields. The hippocampal spatial representation is used as a basis for goal-oriented spatial behavior. We focus on the neural pathway connecting the hippocampus to the nucleus accumbens. Place cells drive a population of locomotor action neurons in the nucleus accumbens. Reward-based learning is applied to map place cell activity into action cell activity. The ensemble action cell activity provides navigational maps to support spatial behavior. We present experimental results obtained with a mobile Khepera robot. Received: 02 July 1999 / Accepted in revised form: 20 March 2000     

Independence of landmark and self-motion-guided navigation: a different role for grid cells

Recent interest in the neural bases of spatial navigation stems from the discovery of neuronal populations with strong, specific spatial signals. The regular firing field arrays of medial entorhinal grid cells suggest that they may provide place cells with distance information extracted from the animal''s self-motion, a notion we critically review by citing new contrary evidence. Next, we question the idea that grid cells provide a rigid distance metric. We also discuss evidence that normal navigation is possible using only landmarks, without self-motion signals. We then propose a model that supposes that information flow in the navigational system changes between light and dark conditions. We assume that the true map-like representation is hippocampal and argue that grid cells have a crucial navigational role only in the dark. In this view, their activity in the light is predominantly shaped by landmarks rather than self-motion information, and so follows place cell activity; in the dark, their activity is determined by self-motion cues and controls place cell activity. A corollary is that place cell activity in the light depends on non-grid cells in ventral medial entorhinal cortex. We conclude that analysing navigational system changes between landmark and no-landmark conditions will reveal key functional properties.     

From Rapid Place Learning to Behavioral Performance: A Key Role for the Intermediate Hippocampus

Rapid place encoding by hippocampal neurons, as reflected by place-related firing, has been intensely studied, whereas the substrates that translate hippocampal place codes into behavior have received little attention. A key point relevant to this translation is that hippocampal organization is characterized by functional–anatomical gradients along the septotemporal axis: Whereas the ability of hippocampal neurons to encode accurate place information declines from the septal to temporal end, hippocampal connectivity to prefrontal and subcortical sites that might relate such place information to behavioral-control processes shows an opposite gradient. We examined in rats the impact of selective lesions to relevant parts of the hippocampus on behavioral tests requiring place learning (watermaze procedures) and on in vivo electrophysiological models of hippocampal encoding (long-term potentiation [LTP], place cells). We found that the intermediate hippocampus is necessary and largely sufficient for behavioral performance based on rapid place learning. In contrast, a residual septal pole of the hippocampus, although displaying intact electrophysiological indices of rapid information encoding (LTP, precise place-related firing, and rapid remapping), failed to sustain watermaze performance based on rapid place learning. These data highlight the important distinction between hippocampal encoding and the behavioral performance based on such encoding, and suggest that the intermediate hippocampus, where substrates of rapid accurate place encoding converge with links to behavioral control, is critical to translate rapid (one-trial) place learning into navigational performance.     

Hippocampal "time cells" bridge the gap in memory for discontiguous events

The hippocampus is critical to remembering the flow of events in distinct experiences and, in doing so, bridges temporal gaps between discontiguous events. Here, we report a robust hippocampal representation of sequence memories, highlighted by "time cells" that encode successive moments during an empty temporal gap between the key events, while also encoding location and ongoing behavior. Furthermore, just as most place cells "remap" when a salient spatial cue is altered, most time cells form qualitatively different representations ("retime") when the main temporal parameter is altered. Hippocampal neurons also differentially encode the key events and disambiguate different event sequences to compose unique, temporally organized representations of specific experiences. These findings suggest that hippocampal neural ensembles segment temporally organized memories much the same as they represent locations of important events in spatially defined environments.     

Neural basis for spatial memory in animals: what do hippocampal neurons tell us?

Recent studies relying on the recording of neuronal unit activity in freely moving rats show the existence of two populations of neurons signalling the animal's location or head direction: place cells found primarily in the hippocampus and head direction cells found in brain areas anatomically and functionally related to the hippocampus. The properties of these two neuronal populations suggest that their activity strongly depends upon information cues stemming from the spatial environment, and also suggest their involvement in spatial memory. Place cells and head direction cells would jointly participate in a neural network allowing the animal to orient in space and to store spatial locations in memory. This network would also be operating in humans, in particular for encoding specific events in episodic memory.     

Animat navigation using a cognitive graph

 This article describes a computational model of the hippocampus that makes it possible for a simulated rat to navigate in a continuous environment containing obstacles. This model views the hippocampus as a “cognitive graph”, that is, a hetero-associative network that learns temporal sequences of visited places and stores a topological representation of the environment. Calling upon place cells, head direction cells, and “goal cells”, it suggests a biologically plausible way of exploiting such a spatial representation for navigation that does not require complicated graph-search algorithms. Moreover, it permits “latent learning” during exploration, that is, the building of a spatial representation without the need of any reinforcement. When the rat occasionally discovers some rewarding place it may wish to rejoin subsequently, it simply records within its cognitive graph, through a series of goal and sub-goal cells, the direction in which to move from any given start place. Accordingly, the model implements a simple “place-recognition-triggered response” navigation strategy. Two implementations of place cell management are studied in parallel. The first one associates place cells with place fields that are given a priori and that are uniformly distributed in the environment. The second one dynamically recruits place cells as exploration proceeds and adjusts the density of such cells to the local complexity of the environment. Both implementations lead to identical results. The article ends with a few predictions about results to be expected in experiments involving simultaneous recordings of multiple cells in the rat hippocampus. Received: 25 June 1999 / Accepted in revised form: 20 March 2000     

Dynamic Grouping of Hippocampal Neural Activity During Cognitive Control of Two Spatial Frames

Cognitive control is the ability to coordinate multiple streams of information to prevent confusion and select appropriate behavioral responses, especially when presented with competing alternatives. Despite its theoretical and clinical significance, the neural mechanisms of cognitive control are poorly understood. Using a two-frame place avoidance task and partial hippocampal inactivation, we confirmed that intact hippocampal function is necessary for coordinating two streams of spatial information. Rats were placed on a continuously rotating arena and trained to organize their behavior according to two concurrently relevant spatial frames: one stationary, the other rotating. We then studied how information about locations in these two spatial frames is organized in the action potential discharge of ensembles of hippocampal cells. Both streams of information were represented in neuronal discharge—place cell activity was organized according to both spatial frames, but almost all cells preferentially represented locations in one of the two spatial frames. At any given time, most coactive cells tended to represent locations in the same spatial frame, reducing the risk of interference between the two information streams. An ensemble''s preference to represent locations in one or the other spatial frame alternated within a session, but at each moment, location in the more behaviorally relevant spatial frame was more likely to be represented. This discharge organized into transient groups of coactive neurons that fired together within 25 ms to represent locations in the same spatial frame. These findings show that dynamic grouping, the transient coactivation of neural subpopulations that represent the same stream of information, can coordinate representations of concurrent information streams and avoid confusion, demonstrating neural-ensemble correlates of cognitive control in hippocampus.     

Place Cells,Grid Cells,and Memory

The hippocampal system is critical for storage and retrieval of declarative memories, including memories for locations and events that take place at those locations. Spatial memories place high demands on capacity. Memories must be distinct to be recalled without interference and encoding must be fast. Recent studies have indicated that hippocampal networks allow for fast storage of large quantities of uncorrelated spatial information. The aim of the this article is to review and discuss some of this work, taking as a starting point the discovery of multiple functionally specialized cell types of the hippocampal–entorhinal circuit, such as place, grid, and border cells. We will show that grid cells provide the hippocampus with a metric, as well as a putative mechanism for decorrelation of representations, that the formation of environment-specific place maps depends on mechanisms for long-term plasticity in the hippocampus, and that long-term spatiotemporal memory storage may depend on offline consolidation processes related to sharp-wave ripple activity in the hippocampus. The multitude of representations generated through interactions between a variety of functionally specialized cell types in the entorhinal–hippocampal circuit may be at the heart of the mechanism for declarative memory formation.The scientific study of human memory started with Herman Ebbinghaus, who initiated the quantitative investigation of associative memory processes as they take place (Ebbinghaus 1885). Ebbinghaus described the conditions that influence memory formation and he determined several basic principles of encoding and recall, such as the law of frequency and the effect of time on forgetting. With Ebbinghaus, higher mental functions were brought to the laboratory. In parallel with the human learning tradition that Ebbinghaus started, a new generation of experimental psychologists described the laws of associative learning in animals. With behaviorists like Pavlov, Watson, Hull, Skinner, and Tolman, a rigorous program for identifying the laws of animal learning was initiated. By the middle of the 20th century, a language for associative learning processes had been developed, and many of the fundamental relationships between environment and behavior had been described. What was completely missing, though, was an understanding of the neural activity underlying the formation of the memory. The behaviorists had deliberately shied away from physiological explanations because of the intangible nature of neural activity at that time.Then the climate began to change. Karl Lashley had shown that lesions in the cerebral cortex had predictable effects on behavior in animals (Lashley 1929, 1950), and Donald Hebb introduced concepts and ideas to account for complex brain functions at the neural circuit level, many of which have retained a place in modern neuroscience (Hebb 1949). Both Lashley and Hebb searched for the engram, but they found no specific locus for it. A significant turning point was reached when Scoville and Milner (1957) reported severe loss of memory in an epileptic patient, patient H.M., after bilateral surgical removal of the hippocampal formation and the surrounding medial temporal lobe areas. “After operation this young man could no longer recognize the hospital staff nor find his way to the bathroom, and he seemed to recall nothing of the day-to-day events of his hospital life.” This tragic misfortune inspired decades of research on the function of the hippocampus in memory. H.M.’s memory impairment could be reproduced in memory tasks in animals and studies of H.M., as well as laboratory animals, pointed to a critical role for the hippocampus in declarative memory—memory, which, in humans, can be consciously recalled and declared, such as memories of experiences and facts (Milner et al. 1968; Mishkin 1978; Cohen and Squire 1980; Squire 1992; Corkin 2002). What was missing from these early studies, however, was a way to address the neuronal mechanisms that led information to be stored as memory.The aim of this article is to show how studies of hippocampal neuronal activity during the past few decades have brought us to a point at which a mechanistic basis of memory formation is beginning to surface. An early landmark in this series of investigations was the discovery of place cells, cells that fire selectively at one or few locations in the environment. At first, these cells seemed to be part of the animal’s instantaneous representation of location, independent of memory, but gradually, over the course of several decades, it has become clear that place cells express current as well as past and future locations. In many ways, place cells can be used as readouts of the memories that are stored in the hippocampus. More recent work has also shown that place cells are part of a wider network of spatially modulated neurons, including grid, border, and head direction cells, each with distinct roles in the representation of space and spatial memory. In this article, we shall discuss potential mechanisms by which these cell types, particularly place and grid cells, in conjunction with synaptic plasticity, may form the basis of a mammalian system for fast high-capacity declarative memory.     

Memory of sequential experience in the hippocampus during slow wave sleep

Rats repeatedly ran through a sequence of spatial receptive fields of hippocampal CA1 place cells in a fixed temporal order. A novel combinatorial decoding method reveals that these neurons repeatedly fired in precisely this order in long sequences involving four or more cells during slow wave sleep (SWS) immediately following, but not preceding, the experience. The SWS sequences occurred intermittently in brief ( approximately 100 ms) bursts, each compressing the behavioral sequence in time by approximately 20-fold. This rapid encoding of sequential experience is consistent with evidence that the hippocampus is crucial for spatial learning in rodents and the formation of long-term memories of events in time in humans.     

Tracking route progression in the posterior parietal cortex

Quick and efficient traversal of learned routes is critical to the survival of many animals. Routes can be defined by both the ordering of navigational epochs, such as continued forward motion or execution of a turn, and the distances separating them. The neural substrates conferring the ability to fluidly traverse complex routes are not well understood, but likely entail interactions between frontal, parietal, and rhinal cortices and the hippocampus. This paper demonstrates that posterior parietal cortical neurons map both individual and multiple navigational epochs with respect to their order in a route. In direct contrast to spatial firing patterns of hippocampal neurons, parietal neurons discharged in a place- and direction-independent fashion. Parietal route maps were scalable and versatile in that they were independent of the size and spatial configuration of navigational epochs. The results provide a framework in which to consider parietal function in spatial cognition.     

Learning Navigational Maps Through Potentiation and Modulation of Hippocampal Place Cells

We analyze a model of navigational map formation based oncorrelation-based, temporally asymmetric potentiation anddepression of synapses between hippocampal place cells. We showthat synaptic modification during random exploration of anenvironment shifts the location encoded by place cell activityin such a way that it indicates the direction from any locationto a fixed target avoiding walls and other obstacles. Multiplemaps to different targets can be simultaneously stored if weintroduce target-dependent modulation of place cell activity.Once maps to a number of target locations in a given environmenthave been stored, novel maps to previously unknown targetlocations are automatically constructed by interpolation betweenexisting maps.     

Functional connectivity of the entorhinal–hippocampal space circuit

The mammalian space circuit is known to contain several functionally specialized cell types, such as place cells in the hippocampus and grid cells, head-direction cells and border cells in the medial entorhinal cortex (MEC). The interaction between the entorhinal and hippocampal spatial representations is poorly understood, however. We have developed an optogenetic strategy to identify functionally defined cell types in the MEC that project directly to the hippocampus. By expressing channelrhodopsin-2 (ChR2) selectively in the hippocampus-projecting subset of entorhinal projection neurons, we were able to use light-evoked discharge as an instrument to determine whether specific entorhinal cell groups—such as grid cells, border cells and head-direction cells—have direct hippocampal projections. Photoinduced firing was observed at fixed minimal latencies in all functional cell categories, with grid cells as the most abundant hippocampus-projecting spatial cell type. We discuss how photoexcitation experiments can be used to distinguish the subset of hippocampus-projecting entorhinal neurons from neurons that are activated indirectly through the network. The functional breadth of entorhinal input implied by this analysis opens up the potential for rich dynamic interactions between place cells in the hippocampus and different functional cell types in the entorhinal cortex (EC).     

New experiences enhance coordinated neural activity in the hippocampus

The acquisition of new memories for places and events requires synaptic plasticity in the hippocampus, and plasticity depends on temporal coordination among neurons. Spatial activity in the hippocampus is relatively disorganized during the initial exploration of a novel environment, however, and it is unclear how neural activity during the initial stages of learning drives synaptic plasticity. Here we show that pairs of CA1 cells that represent overlapping novel locations are initially more coactive and more precisely coordinated than are cells representing overlapping familiar locations. This increased coordination occurs specifically during brief, high-frequency events (HFEs) in the local field potential that are similar to ripples and is not associated with better coordination of place-specific neural activity outside of HFEs. As novel locations become more familiar, correlations between cell pairs decrease. Thus, hippocampal neural activity during learning has a unique structure that is well suited to induce synaptic plasticity and to allow for rapid storage of new memories.     

Increased size and stability of CA1 and CA3 place fields in HCN1 knockout mice

Hippocampal CA1 and CA3 pyramidal neuron place cells encode the spatial location of an animal through localized firing patterns called "place fields." To explore the mechanisms that control place cell firing and their relationship to spatial memory, we studied mice with enhanced spatial memory resulting from forebrain-specific knockout of the HCN1 hyperpolarization-activated cation channel. HCN1 is strongly expressed in CA1 neurons and in entorhinal cortex grid cells, which provide spatial information to the hippocampus. Both CA1 and CA3 place fields were larger but more stable in the knockout mice, with the effect greater in CA1 than CA3. As HCN1 is only weakly expressed in CA3 place cells, their altered activity likely reflects loss of HCN1 in grid cells. The more pronounced changes in CA1 likely reflect the intrinsic contribution of HCN1. The enhanced place field stability may underlie the effect of HCN1 deletion to facilitate spatial learning and memory.     

Encoding new episodes and making them stick

How do we encode, store, and retrieve new episodic memories, and what are the computations performed by the hippocampus during this process? One system that has been used to model the brain basis of episodic memory in humans is the study of spatial navigation by path integration in rodents. Here I discuss three exciting new findings focused on encoding or replay of spatial sequences in the rat hippocampus. These findings not only provide important new insight into the computations associated with encoding and consolidation of spatial trajectories, but may also have implications for understanding key aspects of human episodic memory.     

Evidence for hippocampal role in place avoidance other than merely memory storage

Spatial navigation is used as a popular animal model of higher cognitive functions in people. The data suggest that the hippocampus is important for both storing spatial memories and for performing spatial computations necessary for navigation. Animals use multiple behavioral strategies to solve spatial tasks often using multiple memory systems. We investigated how inactivation of the rat hippocampus affects performance in a place avoidance task to determine if the role of the hippocampus in this task could be attributed to memory storage/retrieval or to the computations needed for navigation. Injecting tetrodotoxin (TTX) into both hippocampi impaired conditioned place avoidance, but after injecting only one hippocampus, the rats learned the place avoidance as well as without any injections. Retention of the place avoidance learned with one hippocampus was not impaired when the injection was switched to the hippocampus that had not been injected during learning. The result suggests that during learning, the hippocampus did not store the place avoidance memory.