Development of a Data Management Tool for Investigating Multivariate Space and Free Will Experiences in Virtual Reality

Virtual reality (VR) has become mature enough to be successfully used in clinical applications such as exposure therapy, pain distraction, and neuropsychological assessment. However, we now need to go beyond the outcome data from this research and conduct the detailed scientific investigations required to better understand what factors influence why VR works (or doesn’t) in these types of clinical applications. This knowledge is required to guide the development of VR applications in the key areas of education, training, and rehabilitation and to further evolve existing VR approaches. One of the primary assets obtained with the use of VR is the ability to simulate the complexity of real world environments, within which human performance can be tested and trained. But this asset comes with a price in terms of the capture, quantification and analysis of large, multivariate and concurrent data sources that reflect the naturalistic behavioral interaction that is afforded in a virtual world. As well, while achieving realism has been a main goal in making convincing VR environments, just what constitutes realism and how much is needed is still an open question situated firmly in the research domain. Just as in real “reality,” such factors in virtual reality are complex and multivariate, and the understanding of this complexity presents exceptional challenges to the VR researcher. For certain research questions, good behavioral science often requires consistent delivery of stimuli within tightly controlled lab-based experimental conditions. However, for other important research questions we do not want to constrain naturalistic behavior and limit VR’s ability to replicate real world conditions, simply because it is easier to study human performance with traditional lab-based methodologies. By doing so we may compromise the very qualities that comprise VR’s unique capacity to mimic the experiences and challenges that exist in everyday life. What is really needed to address scientific questions that require natural exploration of a simulated environment are more usable and robust tools to instrument, organize, and visualize the complex data generated by measurements of participant behaviors within a virtual world. This paper briefly describes the rationale and methodology of an initial study in an ongoing research program that aims to investigate human performance within a virtual environment where unconstrained “free will” exploratory behavior is essential to research questions that involve the relationships between physiology, emotion, and memory. After a discussion of the research protocol and the types of data that were collected, we describe a novel tool that was borne from our need to more efficiently capture, manage, and explore the complex data that was generated in this research. An example of a research participant’s annotated display from this data management and visualization tool is then presented. It is our view that this tool provides the capacity to better visualize and understand the complex data relationships that may arise in VR research that investigates naturalistic free will behavior and its impact on other human performance variables.     

Integration of consonant and pitch processing as revealed by the absence of additivity in mismatch negativity

Consonants, unlike vowels, are thought to be speech specific and therefore no interactions would be expected between consonants and pitch, a basic element for musical tones. The present study used an electrophysiological approach to investigate whether, contrary to this view, there is integrative processing of consonants and pitch by measuring additivity of changes in the mismatch negativity (MMN) of evoked potentials. The MMN is elicited by discriminable variations occurring in a sequence of repetitive, homogeneous sounds. In the experiment, event-related potentials (ERPs) were recorded while participants heard frequently sung consonant-vowel syllables and rare stimuli deviating in either consonant identity only, pitch only, or in both dimensions. Every type of deviation elicited a reliable MMN. As expected, the two single-deviant MMNs had similar amplitudes, but that of the double-deviant MMN was also not significantly different from them. This absence of additivity in the double-deviant MMN suggests that consonant and pitch variations are processed, at least at a pre-attentive level, in an integrated rather than independent way. Domain-specificity of consonants may depend on higher-level processes in the hierarchy of speech perception.     

规律短音中极短间隔短音诱发的失匹配负电位

失匹配负电位是听觉事件相关电位的一个成分,它由一系列重复的,同性质的“标准刺激”的物理性质稍有偏离的“偏差刺激”所诱发,在规律性的标准刺激中,偶然的物理性质稍有偏离的刺激,如频率,强度,久度等的些微变化均可诱发ＭＭＮ。偶然地给于时间上“过早出现”的同样刺激,即频率,强度,久度完全相同,只是在规律性的标准刺激中过早地出现的刺激,作为偏差刺激,也可以诱发出ＮＮＭ。本研究在恒定刺激间隔ＩＳＩ＝６００ｍｓ     

Mechanisms of generation of mismatch negativity and their role in discrimination of short acoustical stimuli

The effect of stimulus duration on the mismatch negativity in the auditory event-related potentials was used to study the role of mismatch negativity (MMN) in discrimination of short acoustical stimuli. We compared discrimination of different short acoustical stimuli in active variant of "odd ball" paradigm. It was shown that it is possible to discriminate between standard and deviant acoustical stimuli which do not produce MMN in passive condition. It makes possible to estimate behavioural significance of MMN in active discrimination task. If the MMN had not been recorded in passive condition, that leads to an increase of reaction time in active paradigm approximately by 50 ms.     

Beat Processing Is Pre-Attentive for Metrically Simple Rhythms with Clear Accents: An ERP Study

The perception of a regular beat is fundamental to music processing. Here we examine whether the detection of a regular beat is pre-attentive for metrically simple, acoustically varying stimuli using the mismatch negativity (MMN), an ERP response elicited by violations of acoustic regularity irrespective of whether subjects are attending to the stimuli. Both musicians and non-musicians were presented with a varying rhythm with a clear accent structure in which occasionally a sound was omitted. We compared the MMN response to the omission of identical sounds in different metrical positions. Most importantly, we found that omissions in strong metrical positions, on the beat, elicited higher amplitude MMN responses than omissions in weak metrical positions, not on the beat. This suggests that the detection of a beat is pre-attentive when highly beat inducing stimuli are used. No effects of musical expertise were found. Our results suggest that for metrically simple rhythms with clear accents beat processing does not require attention or musical expertise. In addition, we discuss how the use of acoustically varying stimuli may influence ERP results when studying beat processing.     

Mismatch negativity in the brain evoked potentials induced by short duration acoustic stimuli in adolescents in the norm and those with attention deficit

The effect of stimulus duration on the mismatch negativity of the auditory event-related potentials was studied in healthy children and those with attention-deficit symptoms. The minimum stimulus duration with which a MMN was elicited by frequency deviant was 50 ms in healthy children. The MMN was absent with shorter stimuli (11 ms and 30 ms). These parametres are close to those in adults. As to children with attention-deficit symptoms the MMN was insignificant with all tested stimulus durations (11 ms, 30 ms, 50 ms).     

Auditory stimulus discrimination recorded in dogs, as indicated by mismatch negativity (MMN)

Dog cognition research tends to rely on behavioural response, which can be confounded by obedience or motivation, as the primary means of indexing dog cognitive abilities. A physiological method of measuring dog cognitive processing would be instructive and could complement behavioural response. Electroencephalogram (EEG) has been used in humans to study stimulus processing, which results in waveforms called event-related potentials (ERPs). One ERP component, mismatch negativity (MMN), is a negative deflection approximately 160-200 ms after stimulus onset, which may be related to change detection from echoic sensory memory. We adapted a minimally invasive technique to record MMN in dogs. Dogs were exposed to an auditory oddball paradigm in which deviant tones (10% probability) were pseudo-randomly interspersed throughout an 8 min sequence of standard tones (90% probability). A significant difference in MMN ERP amplitude was observed after the deviant tone in comparison to the standard tone, t₅ = −2.98, p = 0.03. This difference, attributed to discrimination of an unexpected stimulus in a series of expected stimuli, was not observed when both tones occurred 50% of the time, t₁ = −0.82, p > 0.05. Dogs showed no evidence of pain or distress at any point. We believe this is the first illustration of MMN in a group of dogs and anticipate that this technique may provide valuable insights in cognitive tasks such as object discrimination.     

Preattentive auditory information processing under exposure to the 902 MHz GSM mobile phone electromagnetic field: A mismatch negativity (MMN) study

Previous studies on the effects of the mobile phone electromagnetic field (EMF) on various event‐related potential (ERP) components have yielded inconsistent and even contradictory results, and often failed in replication. The mismatch negativity (MMN) is an auditory ERP component elicited by infrequent (deviant) stimuli differing in some physical features from the repetitive frequent (standard) stimuli in a sound sequence. The MMN provides a sensitive measure for cortical auditory stimulus feature discrimination, regardless of attention and other contaminating factors. In this study, MMN responses to duration, intensity, frequency, and gap changes were recorded in healthy young adults (n = 17), using a multifeature paradigm including several types of auditory change in the same stimulus sequence, while a GSM mobile phone was placed on either ear with the EMF (902 MHz pulsed at 217 Hz; SAR_1g = 1.14 W/kg, SAR_10g = 0.82 W/kg, peak value = 1.21 W/kg, measured with an SAM phantom) on or off. An MMN was elicited by all deviant types, while its amplitude and latency showed no significant differences due to EMF exposure for any deviant types. In the present study, we found no conclusive evidence that acute exposure to GSM mobile phone EMF affects cortical auditory change detection processing reflected by the MMN. Bioelectromagnetics 30:241–248, 2009. © 2009 Wiley‐Liss, Inc.     

Mismatch negativity of ERP in cross-modal attention

Event-related potentials were measured in 12 healthy youth subjects aged 19-22 using the paradigm "cross-modal and delayed response" which is able to improve unattended purity and to avoid the effect of task target on the deviant components of ERP. The experiment included two conditions: (i) Attend visual modality, ignore auditory modality; (ii) attend auditory modality, ignore visual modality. The stimuli under the two conditions were the same. The difference wave was obtained by subtracting ERPs of the standard stimuli from that of the deviant stim-uli. The present results showed that mismatch negativity (MMN), N2b and P3 components can be produced in the auditory and visual modalities under attention condition. However, only MMN was observed in the two modalities un-der inattention condition. Auditory and visual MMN have some features in common: their largest MMN wave peaks were distributed respectively over their primary sensory projection areas of the scalp under attention condition, but over front     

Human brain evoked potentials and discrimination of short acoustic stimuli with various frequency

We found that the mismatch negativity (MMM) was absent in evoked potentials in passive condition for frequency deviant acoustical stimuli when stimulus duration was only 11-30 ms. But it was shown that it is possible for participants to fairly well discriminate these stimuli and the component N2b in RP was generated without preceding MMN. Processing negativity also was absent in potentials by the minimal stimulus duration (11 ms). Auditory discrimination nevertheless was still possible, but reaction time and number of committed errors increased significantly.