Navigation Using Sensory Substitution in Real and Virtual Mazes

Under certain specific conditions people who are blind have a perception of space that is equivalent to that of sighted individuals. However, in most cases their spatial perception is impaired. Is this simply due to their current lack of access to visual information or does the lack of visual information throughout development prevent the proper integration of the neural systems underlying spatial cognition? Sensory Substitution devices (SSDs) can transfer visual information via other senses and provide a unique tool to examine this question. We hypothesize that the use of our SSD (The EyeCane: a device that translates distance information into sounds and vibrations) can enable blind people to attain a similar performance level as the sighted in a spatial navigation task. We gave fifty-six participants training with the EyeCane. They navigated in real life-size mazes using the EyeCane SSD and in virtual renditions of the same mazes using a virtual-EyeCane. The participants were divided into four groups according to visual experience: congenitally blind, low vision & late blind, blindfolded sighted and sighted visual controls. We found that with the EyeCane participants made fewer errors in the maze, had fewer collisions, and completed the maze in less time on the last session compared to the first. By the third session, participants improved to the point where individual trials were no longer significantly different from the initial performance of the sighted visual group in terms of errors, time and collision.     

Sleep modulates the neural substrates of both spatial and contextual memory consolidation

It is known that sleep reshapes the neural representations that subtend the memories acquired while navigating in a virtual environment. However, navigation is not process-pure, as manifold learning components contribute to performance, notably the spatial and contextual memory constituents. In this context, it remains unclear whether post-training sleep globally promotes consolidation of all of the memory components embedded in virtual navigation, or rather favors the development of specific representations. Here, we investigated the effect of post-training sleep on the neural substrates of the consolidation of spatial and contextual memories acquired while navigating in a complex 3D, naturalistic virtual town. Using fMRI, we mapped regional cerebral activity during various tasks designed to tap either the spatial or the contextual memory component, or both, 72 h after encoding with or without sleep deprivation during the first post-training night. Behavioral performance was not dependent upon post-training sleep deprivation, neither in a natural setting that engages both spatial and contextual memory processes nor when looking more specifically at each of these memory representations. At the neuronal level however, analyses that focused on contextual memory revealed distinct correlations between performance and neuronal activity in frontal areas associated with recollection processes after post-training sleep, and in the parahippocampal gyrus associated with familiarity processes in sleep-deprived participants. Likewise, efficient spatial memory was associated with posterior cortical activity after sleep whereas it correlated with parahippocampal/medial temporal activity after sleep deprivation. Finally, variations in place-finding efficiency in a natural setting encompassing spatial and contextual elements were associated with caudate activity after post-training sleep, suggesting the automation of navigation. These data indicate that post-training sleep modulates the neural substrates of the consolidation of both the spatial and contextual memories acquired during virtual navigation.     

Multisensory Control of Multimodal Behavior: Do the Legs Know What the Tongue Is Doing?

Understanding of adaptive behavior requires the precisely controlled presentation of multisensory stimuli combined with simultaneous measurement of multiple behavioral modalities. Hence, we developed a virtual reality apparatus that allows for simultaneous measurement of reward checking, a commonly used measure in associative learning paradigms, and navigational behavior, along with precisely controlled presentation of visual, auditory and reward stimuli. Rats performed a virtual spatial navigation task analogous to the Morris maze where only distal visual or auditory cues provided spatial information. Spatial navigation and reward checking maps showed experience-dependent learning and were in register for distal visual cues. However, they showed a dissociation, whereby distal auditory cues failed to support spatial navigation but did support spatially localized reward checking. These findings indicate that rats can navigate in virtual space with only distal visual cues, without significant vestibular or other sensory inputs. Furthermore, they reveal the simultaneous dissociation between two reward-driven behaviors.     

Easy Rider: Monkeys Learn to Drive a Wheelchair to Navigate through a Complex Maze

The neurological bases of spatial navigation are mainly investigated in rodents and seldom in primates. The few studies led on spatial navigation in both human and non-human primates are performed in virtual, not in real environments. This is mostly because of methodological difficulties inherent in conducting research on freely-moving monkeys in real world environments. There is some incertitude, however, regarding the extrapolation of rodent spatial navigation strategies to primates. Here we present an entirely new platform for investigating real spatial navigation in rhesus monkeys. We showed that monkeys can learn a pathway by using different strategies. In these experiments three monkeys learned to drive the wheelchair and to follow a specified route through a real maze. After learning the route, probe tests revealed that animals successively use three distinct navigation strategies based on i) the place of the reward, ii) the direction taken to obtain reward or iii) a cue indicating reward location. The strategy used depended of the options proposed and the duration of learning. This study reveals that monkeys, like rodents and humans, switch between different spatial navigation strategies with extended practice, implying well-conserved brain learning systems across different species. This new task with freely driving monkeys provides a good support for the electrophysiological and pharmacological investigation of spatial navigation in the real world by making possible electrophysiological and pharmacological investigations.     

Linking active sensing and spatial learning in weakly electric fish

Weakly electric fish can learn the spatial layout of their environment using only their short-range electric sense. During spatial learning, active sensing motions are used to memorize landmark locations so that they can serve as anchors for idiothetic-based navigation. A hindbrain feedback circuit selectively amplifies the electrosensory input arising from these motions. The ascending electrolocation pathway preferentially transmits this information to the pallial regions involved in spatial learning and navigation. Similarities in both behavioral patterns and hindbrain circuitry of gymnotiform and mormyrid fish, two families that independently evolved their electrosense, suggest that amplification and transmission of active sensing motion inputs are fundamental mechanisms for spatial memory acquisition.     

Learning, attentional control, and action video games

While humans have an incredible capacity to acquire new skills and alter their behavior as a result of experience, enhancements in performance are typically narrowly restricted to the parameters of the training environment, with little evidence of generalization to different, even seemingly highly related, tasks. Such specificity is a major obstacle for the development of many real-world training or rehabilitation paradigms, which necessarily seek to promote more general learning. In contrast to these typical findings, research over the past decade has shown that?training on 'action video games' produces learning that transfers well beyond the training task. This has led to substantial interest among those interested in rehabilitation, for instance, after stroke or to treat amblyopia, or?training for various precision-demanding jobs, for instance, endoscopic surgery or piloting unmanned aerial drones. Although the predominant focus of the field has been on outlining the breadth of possible action-game-related enhancements, recent work has concentrated on uncovering the mechanisms that underlie these changes, an important first step towards the goal of designing and using video games for more definite purposes. Game playing may not convey an immediate advantage on new tasks (increased performance from the very first trial), but rather the true effect of action video game playing may be to enhance the ability to learn new tasks. Such a mechanism may serve as a signature of training regimens that are likely to produce transfer of learning.     

Function of a fly motion-sensitive neuron matches eye movements during free flight

Sensing is often implicitly assumed to be the passive acquisition of information. However, part of the sensory information is generated actively when animals move. For instance, humans shift their gaze actively in a sequence of saccades towards interesting locations in a scene. Likewise, many insects shift their gaze by saccadic turns of body and head, keeping their gaze fixed between saccades. Here we employ a novel panoramic virtual reality stimulator and show that motion computation in a blowfly visual interneuron is tuned to make efficient use of the characteristic dynamics of retinal image flow. The neuron is able to extract information about the spatial layout of the environment by utilizing intervals of stable vision resulting from the saccadic viewing strategy. The extraction is possible because the retinal image flow evoked by translation, containing information about object distances, is confined to low frequencies. This flow component can be derived from the total optic flow between saccades because the residual intersaccadic head rotations are small and encoded at higher frequencies. Information about the spatial layout of the environment can thus be extracted by the neuron in a computationally parsimonious way. These results on neuronal function based on naturalistic, behaviourally generated optic flow are in stark contrast to conclusions based on conventional visual stimuli that the neuron primarily represents a detector for yaw rotations of the animal.     

Sex Differences in Exploration Behavior and the Relationship to Harm Avoidance

Venturing into novel terrain poses physical risks to a female and her offspring. Females have a greater tendency to avoid physical harm, while males tend to have larger range sizes and often outperform females in navigation-related tasks. Given this backdrop, we expected that females would explore a novel environment with more caution than males, and that more-cautious exploration would negatively affect navigation performance. Participants explored a novel, large-scale, virtual environment in search of five objects, pointed in the direction of each object from the origin, and then navigated back to the objects. We found that females demonstrated more caution while exploring as reflected in the increased amounts of pausing and revisiting of previously traversed locations. In addition, more pausing and revisiting behaviors led to degradation in navigation performance. Finally, individual levels of trait harm avoidance were positively associated with the amount of revisiting behavior during exploration. These findings support the idea that the fitness costs associated with long-distance travel may encourage females to take a more cautious approach to spatial exploration, and that this caution may partially explain the sex differences in navigation performance.     

Effects of ageing on allocentric and egocentric spatial strategies in the Wistar rat

This study was designed to assess the effect of ageing on spatial (allocentric and egocentric) strategies in rats. Two different tasks were designed for this purpose: one involving Morris' circular pool (distal extramaze cues) and another using the T water maze (egocentric cues). In the first task, the aged rats showed some difficulty in acquiring allocentric spatial learning skills. After increasing the number of trials in this task, there was no significant improvement in the performance of the aged group of rats compared to the adult group. However, in the second spatial task (using egocentric cues), both age groups gave a similar performance. Therefore, the effect of ageing on spatial learning depends on the strategy required to acquire this learning.     

Human spatial navigation: cognitive maps, sexual dimorphism, and neural substrates.

Recent research on navigation has been particularly notable for the increased understanding of the factors affecting human navigation and the neural networks supporting it. The use of virtual reality environments has made it possible to explore the effect of environment layout and content on way-finding performance, and it has shown that these effects may interact with the sex and age of subjects. Functional brain imaging, combined with the use of virtual environments, has revealed strong parallels between humans and other animals in the neural basis of navigation.     

Development of maze navigation by tufted capuchins (Cebus apella)

Theories of spatial navigation hypothesize that animals use vector or topological information to choose routes, often including detours, to move objects or themselves to goals. We assessed adult capuchin monkeys’ (Cebus apella) navigation through 192 virtual two-dimensional mazes that incorporated detour problems. Six monkeys initially were significantly less likely to choose the correct paths when detours were required than when not. Three of the six monkeys repeatedly practiced the 192 mazes to asymptotic mastery; the other three did not practice the mazes again. In a subsequent transfer test, each monkey made correct choices equivalently often on familiar and novel mazes, which suggests that they used general planning skills for maze navigation. Of the three monkeys that practiced the 192 maze-set repeatedly, one efficiently detoured and the other two significantly improved detouring compared to their initial performance. Two monkeys, contrary to their performance when completing the 192 maze-set for the first time, made correct choices at the same rate as chimpanzees. Some evidence suggested that two monkeys used topological information, but utilization of vector information was obvious for all monkeys. Our findings suggest that the boundaries of any individual's navigational abilities are not predicted by species, but depend on experience.     

Autonomous Navigation for Unmanned Aerial Vehicles Based on Chaotic Bionics Theory

In this paper a new reactive mechanism based on perception-action bionics for multi-sensory integration applied to Un-manned Aerial Vehicles (UAVs) navigation is proposed.The strategy is inspired by the olfactory bulb neural activity observed inrabbits subject to external stimuli.The new UAV navigation technique exploits the use of a multiscroll chaotic system which isable to be controlled in real-time towards less complex orbits,like periodic orbits or equilibrium points,considered as perceptiveorbits.These are subject to real-time modifications on the basis of environment changes acquired through a Synthetic ApertureRadar (SAR) sensory system.The mathematical details of the approach are given including simulation results in a virtual en-vironment.The results demonstrate the capability of autonomous navigation for UAV based on chaotic bionics theory in com-plex spatial environments.     

Student test scores are improved in a virtual learning environment

This study evaluates the effectiveness of delivering the core curriculum of an introductory neuroscience course using a software application referred to as a virtual learning interface (VLI). The performance of students in a virtual learning environment (VLE) is compared with that of students in a conventional lecture hall in which the same lecturer presented the same material. This study was not designed to determine whether grades are improved by augmenting a lecture with other information. The VLI takes advantage of audio, video, animation, and text in a multimedia computer environment. Our results indicate that raw average scores on weekly examinations were 14 percentage points higher for students in the VLE compared with those for students in a conventional lecture hall setting. Moreover, normalized test scores were over 5 points higher for students in the VLE. This analysis suggest that a core curriculum can be effectively presented to students using the VLE, thereby making it possible for faculty to spend less class time relaying facts and more time engaging students in discussion of scientific theory.     

Short-Term Memory Maintenance of Object Locations during Active Navigation: Which Working Memory Subsystem Is Essential?

The goal of the present study was to examine the extent to which working memory supports the maintenance of object locations during active spatial navigation. Participants were required to navigate a virtual environment and to encode the location of a target object. In the subsequent maintenance period they performed one of three secondary tasks that were designed to selectively load visual, verbal or spatial working memory subsystems. Thereafter participants re-entered the environment and navigated back to the remembered location of the target. We found that while navigation performance in participants with high navigational ability was impaired only by the spatial secondary task, navigation performance in participants with poor navigational ability was impaired equally by spatial and verbal secondary tasks. The visual secondary task had no effect on navigation performance. Our results extend current knowledge by showing that the differential engagement of working memory subsystems is determined by navigational ability.     

When do objects become landmarks? A VR study of the effect of task relevance on spatial memory

We investigated how objects come to serve as landmarks in spatial memory, and more specifically how they form part of an allocentric cognitive map. Participants performing a virtual driving task incidentally learned the layout of a virtual town and locations of objects in that town. They were subsequently tested on their spatial and recognition memory for the objects. To assess whether the objects were encoded allocentrically we examined pointing consistency across tested viewpoints. In three experiments, we found that spatial memory for objects at navigationally relevant locations was more consistent across tested viewpoints, particularly when participants had more limited experience of the environment. When participants' attention was focused on the appearance of objects, the navigational relevance effect was eliminated, whereas when their attention was focused on objects' locations, this effect was enhanced, supporting the hypothesis that when objects are processed in the service of navigation, rather than merely being viewed as objects, they engage qualitatively distinct attentional systems and are incorporated into an allocentric spatial representation. The results are consistent with evidence from the neuroimaging literature that when objects are relevant to navigation, they not only engage the ventral "object processing stream", but also the dorsal stream and medial temporal lobe memory system classically associated with allocentric spatial memory.     

Increasing Accessibility to the Blind of Virtual Environments,Using a Virtual Mobility Aid Based On the "EyeCane": Feasibility Study

Virtual worlds and environments are becoming an increasingly central part of our lives, yet they are still far from accessible to the blind. This is especially unfortunate as such environments hold great potential for them for uses such as social interaction, online education and especially for use with familiarizing the visually impaired user with a real environment virtually from the comfort and safety of his own home before visiting it in the real world. We have implemented a simple algorithm to improve this situation using single-point depth information, enabling the blind to use a virtual cane, modeled on the “EyeCane” electronic travel aid, within any virtual environment with minimal pre-processing. Use of the Virtual-EyeCane, enables this experience to potentially be later used in real world environments with identical stimuli to those from the virtual environment. We show the fast-learned practical use of this algorithm for navigation in simple environments.     

Capuchin monkeys (Cebus nigritus) use spatial and visual information during within-patch foraging

Foraging in large-scale (navigation between patches), small-scale (choice of within-patch feeding sites), and micro-scale (close inspection of food items) space presents variable cognitive challenges. The reliability and usefulness of spatial memory and perceptual cues during food search in a forest environment vary among these spatial scales. This research applied an experimental field design to test the ability of a free-ranging group composed of eight black-horned capuchin monkeys, Cebus nigritus, inhabiting a forest fragment in Porto Alegre, State of Rio Grande do Sul, Brazil, to use food-associated spatial, visual, olfactory, and quantitative (amount of food) cues during small-scale foraging decisions. The experimental design involved the establishment of a feeding station composed of eight feeding platforms distributed in a circular arrangement. A series of six experiments, each lasting 20 days, was conducted from March to August 2005. Two feeding platforms in each experimental session contained a food reward (real banana), whereas the remaining six platforms contained either a sham banana or an inaccessible real banana. Data on capuchin monkey foraging behavior at the feeding stations were collected by the "all occurrences" sampling method. The performance of the capuchins in the experiments was analyzed based on the first two platforms inspected in each session. The study group inspected feeding platforms in 571 occasions during 113 sessions. Capuchins used visual cues and spatial information (and adopted a win-return strategy) for finding the platforms baited with real bananas and showed weak evidence of the integration of spatial and quantitative cues, but failed to show evidence of using olfactory cues. In addition, individual differences in social rank and foraging behavior affected opportunities for learning and the performance in the cognitive tasks.     

Differential Recruitment of Brain Networks following Route and Cartographic Map Learning of Spatial Environments

An extensive neuroimaging literature has helped characterize the brain regions involved in navigating a spatial environment. Far less is known, however, about the brain networks involved when learning a spatial layout from a cartographic map. To compare the two means of acquiring a spatial representation, participants learned spatial environments either by directly navigating them or learning them from an aerial-view map. While undergoing functional magnetic resonance imaging (fMRI), participants then performed two different tasks to assess knowledge of the spatial environment: a scene and orientation dependent perceptual (SOP) pointing task and a judgment of relative direction (JRD) of landmarks pointing task. We found three brain regions showing significant effects of route vs. map learning during the two tasks. Parahippocampal and retrosplenial cortex showed greater activation following route compared to map learning during the JRD but not SOP task while inferior frontal gyrus showed greater activation following map compared to route learning during the SOP but not JRD task. We interpret our results to suggest that parahippocampal and retrosplenial cortex were involved in translating scene and orientation dependent coordinate information acquired during route learning to a landmark-referenced representation while inferior frontal gyrus played a role in converting primarily landmark-referenced coordinates acquired during map learning to a scene and orientation dependent coordinate system. Together, our results provide novel insight into the different brain networks underlying spatial representations formed during navigation vs. cartographic map learning and provide additional constraints on theoretical models of the neural basis of human spatial representation.     

Finding landmarks - An investigation of viewing behavior during spatial navigation in VR using a graph-theoretical analysis approach

Vision provides the most important sensory information for spatial navigation. Recent technical advances allow new options to conduct more naturalistic experiments in virtual reality (VR) while additionally gathering data of the viewing behavior with eye tracking investigations. Here, we propose a method that allows one to quantify characteristics of visual behavior by using graph-theoretical measures to abstract eye tracking data recorded in a 3D virtual urban environment. The analysis is based on eye tracking data of 20 participants, who freely explored the virtual city Seahaven for 90 minutes with an immersive VR headset with an inbuild eye tracker. To extract what participants looked at, we defined “gaze” events, from which we created gaze graphs. On these, we applied graph-theoretical measures to reveal the underlying structure of visual attention. Applying graph partitioning, we found that our virtual environment could be treated as one coherent city. To investigate the importance of houses in the city, we applied the node degree centrality measure. Our results revealed that 10 houses had a node degree that exceeded consistently two-sigma distance from the mean node degree of all other houses. The importance of these houses was supported by the hierarchy index, which showed a clear hierarchical structure of the gaze graphs. As these high node degree houses fulfilled several characteristics of landmarks, we named them “gaze-graph-defined landmarks”. Applying the rich club coefficient, we found that these gaze-graph-defined landmarks were preferentially connected to each other and that participants spend the majority of their experiment time in areas where at least two of those houses were visible. Our findings do not only provide new experimental evidence for the development of spatial knowledge, but also establish a new methodology to identify and assess the function of landmarks in spatial navigation based on eye tracking data.     

Modelling human visual navigation using multi-view scene reconstruction

It is often assumed that humans generate a 3D reconstruction of the environment, either in egocentric or world-based coordinates, but the steps involved are unknown. Here, we propose two reconstruction-based models, evaluated using data from two tasks in immersive virtual reality. We model the observer’s prediction of landmark location based on standard photogrammetric methods and then combine location predictions to compute likelihood maps of navigation behaviour. In one model, each scene point is treated independently in the reconstruction; in the other, the pertinent variable is the spatial relationship between pairs of points. Participants viewed a simple environment from one location, were transported (virtually) to another part of the scene and were asked to navigate back. Error distributions varied substantially with changes in scene layout; we compared these directly with the likelihood maps to quantify the success of the models. We also measured error distributions when participants manipulated the location of a landmark to match the preceding interval, providing a direct test of the landmark-location stage of the navigation models. Models such as this, which start with scenes and end with a probabilistic prediction of behaviour, are likely to be increasingly useful for understanding 3D vision.