Rapid Expectation Adaptation during Syntactic Comprehension

When we read or listen to language, we are faced with the challenge of inferring intended messages from noisy input. This challenge is exacerbated by considerable variability between and within speakers. Focusing on syntactic processing (parsing), we test the hypothesis that language comprehenders rapidly adapt to the syntactic statistics of novel linguistic environments (e.g., speakers or genres). Two self-paced reading experiments investigate changes in readers’ syntactic expectations based on repeated exposure to sentences with temporary syntactic ambiguities (so-called “garden path sentences”). These sentences typically lead to a clear expectation violation signature when the temporary ambiguity is resolved to an a priori less expected structure (e.g., based on the statistics of the lexical context). We find that comprehenders rapidly adapt their syntactic expectations to converge towards the local statistics of novel environments. Specifically, repeated exposure to a priori unexpected structures can reduce, and even completely undo, their processing disadvantage (Experiment 1). The opposite is also observed: a priori expected structures become less expected (even eliciting garden paths) in environments where they are hardly ever observed (Experiment 2). Our findings suggest that, when changes in syntactic statistics are to be expected (e.g., when entering a novel environment), comprehenders can rapidly adapt their expectations, thereby overcoming the processing disadvantage that mistaken expectations would otherwise cause. Our findings take a step towards unifying insights from research in expectation-based models of language processing, syntactic priming, and statistical learning.     

Ready…Go: Amplitude of the fMRI Signal Encodes Expectation of Cue Arrival Time

What happens when the brain awaits a signal of uncertain arrival time, as when a sprinter waits for the starting pistol? And what happens just after the starting pistol fires? Using functional magnetic resonance imaging (fMRI), we have discovered a novel correlate of temporal expectations in several brain regions, most prominently in the supplementary motor area (SMA). Contrary to expectations, we found little fMRI activity during the waiting period; however, a large signal appears after the “go” signal, the amplitude of which reflects learned expectations about the distribution of possible waiting times. Specifically, the amplitude of the fMRI signal appears to encode a cumulative conditional probability, also known as the cumulative hazard function. The fMRI signal loses its dependence on waiting time in a “countdown” condition in which the arrival time of the go cue is known in advance, suggesting that the signal encodes temporal probabilities rather than simply elapsed time. The dependence of the signal on temporal expectation is present in “no-go” conditions, demonstrating that the effect is not a consequence of motor output. Finally, the encoding is not dependent on modality, operating in the same manner with auditory or visual signals. This finding extends our understanding of the relationship between temporal expectancy and measurable neural signals.     

Ready…Go: Amplitude of the fMRI Signal Encodes Expectation of Cue Arrival Time

What happens when the brain awaits a signal of uncertain arrival time, as when a sprinter waits for the starting pistol? And what happens just after the starting pistol fires? Using functional magnetic resonance imaging (fMRI), we have discovered a novel correlate of temporal expectations in several brain regions, most prominently in the supplementary motor area (SMA). Contrary to expectations, we found little fMRI activity during the waiting period; however, a large signal appears after the “go” signal, the amplitude of which reflects learned expectations about the distribution of possible waiting times. Specifically, the amplitude of the fMRI signal appears to encode a cumulative conditional probability, also known as the cumulative hazard function. The fMRI signal loses its dependence on waiting time in a “countdown” condition in which the arrival time of the go cue is known in advance, suggesting that the signal encodes temporal probabilities rather than simply elapsed time. The dependence of the signal on temporal expectation is present in “no-go” conditions, demonstrating that the effect is not a consequence of motor output. Finally, the encoding is not dependent on modality, operating in the same manner with auditory or visual signals. This finding extends our understanding of the relationship between temporal expectancy and measurable neural signals.     

The Relation between Thematic Role Computing and Semantic Relatedness Processing during On-Line Sentence Comprehension

Sentence comprehension involves timely computing different types of relations between its verbs and noun arguments, such as morphosyntactic, semantic, and thematic relations. Here, we used EEG technique to investigate the potential differences in thematic role computing and lexical-semantic relatedness processing during on-line sentence comprehension, and the interaction between these two types of processes. Mandarin Chinese sentences were used as materials. The basic structure of those sentences is “Noun+Verb+‘le’+a two-character word”, with the Noun being the initial argument. The verb disambiguates the initial argument as an agent or a patient. Meanwhile, the initial argument and the verb are highly or lowly semantically related. The ERPs at the verbs revealed that: relative to the agent condition, the patient condition evoked a larger N400 only when the argument and verb were lowly semantically related; however, relative to the high-relatedness condition, the low-relatedness condition elicited a larger N400 regardless of the thematic relation; although both thematic role variation and semantic relatedness variation elicited N400 effects, the N400 effect elicited by the former was broadly distributed and reached maximum over the frontal electrodes, and the N400 effect elicited by the latter had a posterior distribution. In addition, the brain oscillations results showed that, although thematic role variation (patient vs. agent) induced power decreases around the beta frequency band (15–30 Hz), semantic relatedness variation (low-relatedness vs. high-relatedness) induced power increases in the theta frequency band (4–7 Hz). These results suggested that, in the sentence context, thematic role computing is modulated by the semantic relatedness between the verb and its argument; semantic relatedness processing, however, is in some degree independent from the thematic relations. Moreover, our results indicated that, during on-line sentence comprehension, thematic role computing and semantic relatedness processing are mediated by distinct neural systems.     

Perceptual Compensation Is Correlated with Individuals' “Autistic” Traits: Implications for Models of Sound Change

Variation is a ubiquitous feature of speech. Listeners must take into account context-induced variation to recover the interlocutor''s intended message. When listeners fail to normalize for context-induced variation properly, deviant percepts become seeds for new perceptual and production norms. In question is how deviant percepts accumulate in a systematic fashion to give rise to sound change (i.e., new pronunciation norms) within a given speech community. The present study investigated subjects'' classification of /s/ and // before /a/ or /u/ spoken by a male or a female voice. Building on modern cognitive theories of autism-spectrum condition, which see variation in autism-spectrum condition in terms of individual differences in cognitive processing style, we established a significant correlation between individuals'' normalization for phonetic context (i.e., whether the following vowel is /a/ or /u/) and talker voice variation (i.e., whether the talker is male or female) in speech and their “autistic” traits, as measured by the Autism Spectrum Quotient (AQ). In particular, our mixed-effect logistic regression models show that women with low AQ (i.e., the least “autistic”) do not normalize for phonetic coarticulation as much as men and high AQ women. This study provides first direct evidence that variability in human''s ability to compensate for context-induced variations in speech perceptually is governed by the individual''s sex and cognitive processing style. These findings lend support to the hypothesis that the systematic infusion of new linguistic variants (i.e., the deviant percepts) originate from a sub-segment of the speech community that consistently under-compensates for contextual variation in speech.     

Placebo in emotional processing--induced expectations of anxiety relief activate a generalized modulatory network

Placebo analgesia and reward processing share several features. For instance, expectations have a strong influence on the subsequent emotional experience of both. Recent imaging data indicate similarities in the underlying neuronal network. We hypothesized that placebo analgesia is a special case of reward processing and that placebo treatment could modulate emotional perception in the same way as does pain perception. The behavioral part of this study indicates that placebo treatment has an effect on how subjects perceive unpleasant pictures. Furthermore, event-related fMRI demonstrated that the same modulatory network, including the rostral anterior cingulate cortex and the lateral orbitofrontal cortex, is involved in both emotional placebo and placebo analgesia. These effects were correlated with the reported placebo effect and were predicted by the amount of treatment expectation induced on a previous day. Thus, the placebo effect may be considered to be a general process of modulation induced by the subjects' expectations.     

Memory for Expectation-Violating Concepts: The Effects of Agents and Cultural Familiarity

Previous research has shown that ideas which violate our expectations, such as schema-inconsistent concepts, enjoy privileged status in terms of memorability. In our study, memory for concepts that violate cultural (cultural schema-level) expectations (e.g., “illiterate teacher”, “wooden bottle”, or “thorny grass”) versus domain-level (ontological) expectations (e.g., “speaking cat”, “jumping maple”, or “melting teacher”) was examined. Concepts that violate cultural expectations, or counter-schematic, were remembered to a greater extent compared with concepts that violate ontological expectations and with intuitive concepts (e.g., “galloping pony”, “drying orchid”, or “convertible car”), in both immediate recall, and delayed recognition tests. Importantly, concepts related to agents showed a memory advantage over concepts not pertaining to agents, but this was true only for expectation-violating concepts. Our results imply that intuitive, everyday concepts are equally attractive and memorable regardless of the presence or absence of agents. However, concepts that violate our expectations (cultural-schema or domain-level) are more memorable when pertaining to agents (humans and animals) than to non-agents (plants or objects/artifacts). We conclude that due to their evolutionary salience, cultural ideas which combine expectancy violations and the involvement of an agent are especially memorable and thus have an enhanced probability of being successfully propagated.     

The Brain’s Dorsal Route for Speech Represents Word Meaning: Evidence from Gesture

The dual-route model of speech processing includes a dorsal stream that maps auditory to motor features at the sublexical level rather than at the lexico-semantic level. However, the literature on gesture is an invitation to revise this model because it suggests that the premotor cortex of the dorsal route is a major site of lexico-semantic interaction. Here we investigated lexico-semantic mapping using word-gesture pairs that were either congruent or incongruent. Using fMRI-adaptation in 28 subjects, we found that temporo-parietal and premotor activity during auditory processing of single action words was modulated by the prior audiovisual context in which the words had been repeated. The BOLD signal was suppressed following repetition of the auditory word alone, and further suppressed following repetition of the word accompanied by a congruent gesture (e.g. [“grasp” + grasping gesture]). Conversely, repetition suppression was not observed when the same action word was accompanied by an incongruent gesture (e.g. [“grasp” + sprinkle]). We propose a simple model to explain these results: auditory and visual information converge onto premotor cortex where it is represented in a comparable format to determine (in)congruence between speech and gesture. This ability of the dorsal route to detect audiovisual semantic (in)congruence suggests that its function is not restricted to the sublexical level.     

Benevolent Ideology and Women’s Economic Decision-Making: When Sexism Is Hurting Men’s Wallet

Can ideology, as a widespread “expectation creator,” impact economic decisions? In two studies we investigated the influence of the Benevolent Sexism (BS) ideology (which dictates that men should provide for passive and nurtured women) on women’s economic decision-making. In Study 1, using a Dictator Game in which women decided how to share amounts of money with men, results of a Generalized Linear Mixed Model analysis show that higher endorsement of BS and contextual expectations of benevolence were associated with more very unequal offers. Similarly, in an Ultimatum Game in which women received monetary offers from men, Study 2’s Generalized Linear Mixed Model’s results revealed that BS led women to reject more very unequal offers. If women’s endorsement of BS ideology and expectations of benevolence prove contrary to reality, they may strike back at men. These findings show that BS ideology creates expectations that shape male-female relationships in a way that could be prejudicial to men.     

Left inferior frontal activations depending on the canonicity determined by the argument structures of ditransitive sentences: an MEG study

To elucidate the relationships between syntactic and semantic processes, one interesting question is how syntactic structures are constructed by the argument structure of a verb, where each argument corresponds to a semantic role of each noun phrase (NP). Here we examined the effects of possessivity [sentences with or without a possessor] and canonicity [canonical or noncanonical word orders] using Japanese ditransitive sentences. During a syntactic decision task, the syntactic structure of each sentence would be constructed in an incremental manner based on the predicted argument structure of the ditransitive verb in a verb-final construction. Using magnetoencephalography, we found a significant canonicity effect on the current density in the left inferior frontal gyrus (IFG) at 530-550 ms after the verb onset. This effect was selective to canonical sentences, and significant even when the precedent NP was physically identical. We suggest that the predictive effects associated with syntactic processing became larger for canonical sentences, where the NPs and verb were merged with a minimum structural distance, leading to the left IFG activations. For monotransitive and intransitive verbs, in which structural computation of the sentences was simpler than that of ditransitive sentences, we observed a significant effect selective to noncanonical sentences in the temporoparietal regions during 480-670 ms. This effect probably reflects difficulty in semantic processing of noncanonical sentences. These results demonstrate that the left IFG plays a predictive role in syntactic processing, which depends on the canonicity determined by argument structures, whereas other temporoparietal regions would subserve more semantic aspects of sentence processing.     

Do I Have My Attention? Speed of Processing Advantages for the Self-Face Are Not Driven by Automatic Attention Capture

We respond more quickly to our own face than to other faces, but there is debate over whether this is connected to attention-grabbing properties of the self-face. In two experiments, we investigate whether the self-face selectively captures attention, and the attentional conditions under which this might occur. In both experiments, we examined whether different types of face (self, friend, stranger) provide differential levels of distraction when processing self, friend and stranger names. In Experiment 1, an image of a distractor face appeared centrally – inside the focus of attention – behind a target name, with the faces either upright or inverted. In Experiment 2, distractor faces appeared peripherally – outside the focus of attention – in the left or right visual field, or bilaterally. In both experiments, self-name recognition was faster than other name recognition, suggesting a self-referential processing advantage. The presence of the self-face did not cause more distraction in the naming task compared to other types of face, either when presented inside (Experiment 1) or outside (Experiment 2) the focus of attention. Distractor faces had different effects across the two experiments: when presented inside the focus of attention (Experiment 1), self and friend images facilitated self and friend naming, respectively. This was not true for stranger stimuli, suggesting that faces must be robustly represented to facilitate name recognition. When presented outside the focus of attention (Experiment 2), no facilitation occurred. Instead, we report an interesting distraction effect caused by friend faces when processing strangers’ names. We interpret this as a “social importance” effect, whereby we may be tuned to pick out and pay attention to familiar friend faces in a crowd. We conclude that any speed of processing advantages observed in the self-face processing literature are not driven by automatic attention capture.     

Task-induced neural covariability as a signature of approximate Bayesian learning and inference

Perception is often characterized computationally as an inference process in which uncertain or ambiguous sensory inputs are combined with prior expectations. Although behavioral studies have shown that observers can change their prior expectations in the context of a task, robust neural signatures of task-specific priors have been elusive. Here, we analytically derive such signatures under the general assumption that the responses of sensory neurons encode posterior beliefs that combine sensory inputs with task-specific expectations. Specifically, we derive predictions for the task-dependence of correlated neural variability and decision-related signals in sensory neurons. The qualitative aspects of our results are parameter-free and specific to the statistics of each task. The predictions for correlated variability also differ from predictions of classic feedforward models of sensory processing and are therefore a strong test of theories of hierarchical Bayesian inference in the brain. Importantly, we find that Bayesian learning predicts an increase in so-called “differential correlations” as the observer’s internal model learns the stimulus distribution, and the observer’s behavioral performance improves. This stands in contrast to classic feedforward encoding/decoding models of sensory processing, since such correlations are fundamentally information-limiting. We find support for our predictions in data from existing neurophysiological studies across a variety of tasks and brain areas. Finally, we show in simulation how measurements of sensory neural responses can reveal information about a subject’s internal beliefs about the task. Taken together, our results reinterpret task-dependent sources of neural covariability as signatures of Bayesian inference and provide new insights into their cause and their function.     

Evolutionary Stability in the Asymmetric Volunteer's Dilemma

It is often assumed that in public goods games, contributors are either strong or weak players and each individual has an equal probability of exhibiting cooperation. It is difficult to explain why the public good is produced by strong individuals in some cooperation systems, and by weak individuals in others. Viewing the asymmetric volunteer''s dilemma game as an evolutionary game, we find that whether the strong or the weak players produce the public good depends on the initial condition (i.e., phenotype or initial strategy of individuals). These different evolutionarily stable strategies (ESS) associated with different initial conditions, can be interpreted as the production modes of public goods of different cooperation systems. A further analysis revealed that the strong player adopts a pure strategy but mixed strategies for the weak players to produce the public good, and that the probability of volunteering by weak players decreases with increasing group size or decreasing cost-benefit ratio. Our model shows that the defection probability of a “strong” player is greater than the “weak” players in the model of Diekmann (1993). This contradicts Selten''s (1980) model that public goods can only be produced by a strong player, is not an evolutionarily stable strategy, and will therefore disappear over evolutionary time. Our public good model with ESS has thus extended previous interpretations that the public good can only be produced by strong players in an asymmetric game.     

Pathological States and Criminal Responsibility

The doctor is embarrassed in court when asked to testify to the effects of illness on a defendant''s capability “of appreciating the nature and quality of an act or omission or of knowing that an act or omission is wrong”. The source of his difficulty is traced to the legal concepts of “guilt”, “crime” and “punishment” which imply a legal view of man at variance with our modern biological view. To abolish this discrepancy we need not accept a medical model for criminal law where “crime” is analogous to “disease”, and “punishment” to “treatment”. A pragmatic approach to the handling of the criminal could exclude the notions of “guilt” and “punishment” and yet fulfil the rational goals of protecting society from the criminal and of compensating his victims.     

Temporal expectancy in the context of a theory of visual attention

Temporal expectation is expectation with respect to the timing of an event such as the appearance of a certain stimulus. In this paper, temporal expectancy is investigated in the context of the theory of visual attention (TVA), and we begin by summarizing the foundations of this theoretical framework. Next, we present a parametric experiment exploring the effects of temporal expectation on perceptual processing speed in cued single-stimulus letter recognition with unspeeded motor responses. The length of the cue–stimulus foreperiod was exponentially distributed with one of six hazard rates varying between blocks. We hypothesized that this manipulation would result in a distinct temporal expectation in each hazard rate condition. Stimulus exposures were varied such that both the temporal threshold of conscious perception (t₀ ms) and the perceptual processing speed (v letters s⁻¹) could be estimated using TVA. We found that the temporal threshold t₀ was unaffected by temporal expectation, but the perceptual processing speed v was a strikingly linear function of the logarithm of the hazard rate of the stimulus presentation. We argue that the effects on the v values were generated by changes in perceptual biases, suggesting that our perceptual biases are directly related to our temporal expectations.     

No,There Is No 150 ms Lead of Visual Speech on Auditory Speech,but a Range of Audiovisual Asynchronies Varying from Small Audio Lead to Large Audio Lag

An increasing number of neuroscience papers capitalize on the assumption published in this journal that visual speech would be typically 150 ms ahead of auditory speech. It happens that the estimation of audiovisual asynchrony in the reference paper is valid only in very specific cases, for isolated consonant-vowel syllables or at the beginning of a speech utterance, in what we call “preparatory gestures”. However, when syllables are chained in sequences, as they are typically in most parts of a natural speech utterance, asynchrony should be defined in a different way. This is what we call “comodulatory gestures” providing auditory and visual events more or less in synchrony. We provide audiovisual data on sequences of plosive-vowel syllables (pa, ta, ka, ba, da, ga, ma, na) showing that audiovisual synchrony is actually rather precise, varying between 20 ms audio lead and 70 ms audio lag. We show how more complex speech material should result in a range typically varying between 40 ms audio lead and 200 ms audio lag, and we discuss how this natural coordination is reflected in the so-called temporal integration window for audiovisual speech perception. Finally we present a toy model of auditory and audiovisual predictive coding, showing that visual lead is actually not necessary for visual prediction.     

Lexical Planning in Sentence Production Is Highly Incremental: Evidence from ERPs

The scope of lexical planning, which means how far ahead speakers plan lexically before they start producing an utterance, is an important issue for research into speech production, but remains highly controversial. The present research investigated this issue using the semantic blocking effect, which refers to the widely observed effects that participants take longer to say aloud the names of items in pictures when the pictures in a block of trials in an experiment depict items that belong to the same semantic category than different categories. As this effect is often interpreted as a reflection of difficulty in lexical selection, the current study took the semantic blocking effect and its associated pattern of event-related brain potentials (ERPs) as a proxy to test whether lexical planning during sentence production extends beyond the first noun when a subject noun-phrase includes two nouns, such as “The chair and the boat are both red” and “The chair above the boat is red”. The results showed a semantic blocking effect both in onset latencies and in ERPs during the utterance of the first noun of these complex noun-phrases but not for the second noun. The indication, therefore, is that the lexical planning scope does not encompass this second noun-phrase. Indeed, the present findings are in line with accounts that propose radically incremental lexical planning, in which speakers plan ahead only one word at a time. This study also provides a highly novel example of using ERPs to examine the production of long utterances, and it is hoped the present demonstration of the effectiveness of this approach inspires further application of ERP techniques in this area of research.     

The Wallpaper Effect: The Contact Hypothesis Fails for Minority Group Members Who Live in Areas with a High Proportion of Majority Group Members

We aim to provide one explanation for why the link between contact and prejudice is consistently less strong for minority group members than it is for majority group members. Specifically, we propose a “wallpaper effect” such that contact works to increase minority group members'' positivity towards majority groups when they live in areas densely populated with other minority group members. Conversely, we suggest that when minority group members live in neighborhoods patterned with majority group faces (as is so often the case), contact will be less transformative. We test this assumption using a large sample of both New Zealander minority (Māori; N = 925) and majority (European; N = 3805) group members. In line with predictions, Māori who lived in minority dense neighborhoods showed the traditional association between contact and increased warmth towards New Zealander Europeans. This relationship, however, was weak or non-existent when they lived in primarily European neighborhoods. Contact effects in majority group members were unaffected by neighborhood composition. The interaction held when controlling for, and was not explained by: gender, income, experiences of harm, cognitions of race-based rejection, or realistic threat. We provide the first evidence to suggest that when it comes to minority group members'' intergroup attitudes, contact with majority group members may be a relatively ineffective predictor unless the “wallpaper” of their lives is minority-dense.     

Differential Entrainment of Neuroelectric Delta Oscillations in Developmental Dyslexia

Oscillatory entrainment to the speech signal is important for language processing, but has not yet been studied in developmental disorders of language. Developmental dyslexia, a difficulty in acquiring efficient reading skills linked to difficulties with phonology (the sound structure of language), has been associated with behavioural entrainment deficits. It has been proposed that the phonological ‘deficit’ that characterises dyslexia across languages is related to impaired auditory entrainment to speech at lower frequencies via neuroelectric oscillations (<10 Hz, ‘temporal sampling theory’). Impaired entrainment to temporal modulations at lower frequencies would affect the recovery of the prosodic and syllabic structure of speech. Here we investigated event-related oscillatory EEG activity and contingent negative variation (CNV) to auditory rhythmic tone streams delivered at frequencies within the delta band (2 Hz, 1.5 Hz), relevant to sampling stressed syllables in speech. Given prior behavioural entrainment findings at these rates, we predicted functionally atypical entrainment of delta oscillations in dyslexia. Participants performed a rhythmic expectancy task, detecting occasional white noise targets interspersed with tones occurring regularly at rates of 2 Hz or 1.5 Hz. Both groups showed significant entrainment of delta oscillations to the rhythmic stimulus stream, however the strength of inter-trial delta phase coherence (ITC, ‘phase locking’) and the CNV were both significantly weaker in dyslexics, suggestive of weaker entrainment and less preparatory brain activity. Both ITC strength and CNV amplitude were significantly related to individual differences in language processing and reading. Additionally, the instantaneous phase of prestimulus delta oscillation predicted behavioural responding (response time) for control participants only.