Bayesian natural selection and the evolution of perceptual systems

In recent years, there has been much interest in characterizing statistical properties of natural stimuli in order to better understand the design of perceptual systems. A fruitful approach has been to compare the processing of natural stimuli in real perceptual systems with that of ideal observers derived within the framework of Bayesian statistical decision theory. While this form of optimization theory has provided a deeper understanding of the information contained in natural stimuli as well as of the computational principles employed in perceptual systems, it does not directly consider the process of natural selection, which is ultimately responsible for design. Here we propose a formal framework for analysing how the statistics of natural stimuli and the process of natural selection interact to determine the design of perceptual systems. The framework consists of two complementary components. The first is a maximum fitness ideal observer, a standard Bayesian ideal observer with a utility function appropriate for natural selection. The second component is a formal version of natural selection based upon Bayesian statistical decision theory. Maximum fitness ideal observers and Bayesian natural selection are demonstrated in several examples. We suggest that the Bayesian approach is appropriate not only for the study of perceptual systems but also for the study of many other systems in biology.     

A new perceptual bias reveals suboptimal population decoding of sensory responses

Several studies have reported optimal population decoding of sensory responses in two-alternative visual discrimination tasks. Such decoding involves integrating noisy neural responses into a more reliable representation of the likelihood that the stimuli under consideration evoked the observed responses. Importantly, an ideal observer must be able to evaluate likelihood with high precision and only consider the likelihood of the two relevant stimuli involved in the discrimination task. We report a new perceptual bias suggesting that observers read out the likelihood representation with remarkably low precision when discriminating grating spatial frequencies. Using spectrally filtered noise, we induced an asymmetry in the likelihood function of spatial frequency. This manipulation mainly affects the likelihood of spatial frequencies that are irrelevant to the task at hand. Nevertheless, we find a significant shift in perceived grating frequency, indicating that observers evaluate likelihoods of a broad range of irrelevant frequencies and discard prior knowledge of stimulus alternatives when performing two-alternative discrimination.     

Adaptive Sampling of Information in Perceptual Decision-Making

In many perceptual and cognitive decision-making problems, humans sample multiple noisy information sources serially, and integrate the sampled information to make an overall decision. We derive the optimal decision procedure for two-alternative choice tasks in which the different options are sampled one at a time, sources vary in the quality of the information they provide, and the available time is fixed. To maximize accuracy, the optimal observer allocates time to sampling different information sources in proportion to their noise levels. We tested human observers in a corresponding perceptual decision-making task. Observers compared the direction of two random dot motion patterns that were triggered only when fixated. Observers allocated more time to the noisier pattern, in a manner that correlated with their sensory uncertainty about the direction of the patterns. There were several differences between the optimal observer predictions and human behaviour. These differences point to a number of other factors, beyond the quality of the currently available sources of information, that influences the sampling strategy.     

Posterior Probability Matching and Human Perceptual Decision Making

Probability matching is a classic theory of decision making that was first developed in models of cognition. Posterior probability matching, a variant in which observers match their response probabilities to the posterior probability of each response being correct, is being used increasingly often in models of perception. However, little is known about whether posterior probability matching is consistent with the vast literature on vision and hearing that has developed within signal detection theory. Here we test posterior probability matching models using two tools from detection theory. First, we examine the models’ performance in a two-pass experiment, where each block of trials is presented twice, and we measure the proportion of times that the model gives the same response twice to repeated stimuli. We show that at low performance levels, posterior probability matching models give highly inconsistent responses across repeated presentations of identical trials. We find that practised human observers are more consistent across repeated trials than these models predict, and we find some evidence that less practised observers more consistent as well. Second, we compare the performance of posterior probability matching models on a discrimination task to the performance of a theoretical ideal observer that achieves the best possible performance. We find that posterior probability matching is very inefficient at low-to-moderate performance levels, and that human observers can be more efficient than is ever possible according to posterior probability matching models. These findings support classic signal detection models, and rule out a broad class of posterior probability matching models for expert performance on perceptual tasks that range in complexity from contrast discrimination to symmetry detection. However, our findings leave open the possibility that inexperienced observers may show posterior probability matching behaviour, and our methods provide new tools for testing for such a strategy.     

A wavelength discrimination function for the hummingbirdArchilochus alexandri

Summary Free-flying black-chinned hummingbirds (Archilochus alexandri) at a site in southeastern Arizona were attracted to artificial feeders displaying narrow spectral bands of light (7 nm half band width). The birds were taught to discriminate between pairs of wavelengths of approximately equal brightness but with a spectral separation of 10 nm. After training, performance of the birds was not significantly changed by alterations in the relative intensities of the two lights. Moreover, when the spectral composition of the test and training lights was made identical, the birds did not learn to make a discrimination on the basis of intensity differences of 0.5 or 1 log unit. In the learned foraging behavior of these hummingbirds, the salience of brightness is therefore inconsequential relative to hue.Discrimination scores for a constant 10 nm separation of test and training wavelengths were determined between 410 and 650 nm. This measure of the spectral dependence of wavelength discrimination shows a deterioration of performance at the red end of the spectrum but not in the blue and violet. Moreover, the minima at 585 and 555 nm indicate more structure than is present in this region of the spectrum in the human hue discrimination curve, and are similar but not identical to data on pigeon. These results are consistent with a growing body of evidence suggesting that the color space of birds may be more than three dimensional.This work was supported by NIH grants EY03266 and EY00222. We are indebted to Sally and Walter Spofford, who generously allowed us to work at their home, Aguila-Rancho, during May and June of 1980, and without whose kind help these experiments could not have been performed.     

Nociceptive inputs to C3, a motoneuron of the tentacle withdrawal reflex in Helix aspersa

The tentacle withdrawal reflex of snails is perhaps the fastest, most sensitive reflex in the animals' repertoire. We have investigated the sensory inputs to a major motoneuron (C3) mediating the reflex. The cell C3 is sensitive to both chemical and mechanical stimulation, but there is little or no discrimination of quality in chemical stimuli. Small increments in the concentration of chemical stimuli produce large changes in neuronal responses. When chemicals are applied to the afferent nerve, the effects are comparable to those caused by applications to the olfactory epithelium, suggesting that the transducing elements are unspecialized. The afferent pathway is independent of the procerebrum, which is the primary olfactory lobe. Two excitatory synaptic inputs are identified, both of which originate in the tentacle, propagate centrally and synapse directly onto C3. A small, low threshold input is assigned to dendritic sites distant from the soma. A larger, higher threshold input is assigned to proximal dendritic sites. The latter input is largely responsible for the strong activation of C3 following noxious stimulation of the tentacle. The sensory inputs to C3 have properties similar to those of fibres in the nasal branch of the vertebrate trigeminal nerve.     

Adaptive Spike Threshold Enables Robust and Temporally Precise Neuronal Encoding

Neural processing rests on the intracellular transformation of information as synaptic inputs are translated into action potentials. This transformation is governed by the spike threshold, which depends on the history of the membrane potential on many temporal scales. While the adaptation of the threshold after spiking activity has been addressed before both theoretically and experimentally, it has only recently been demonstrated that the subthreshold membrane state also influences the effective spike threshold. The consequences for neural computation are not well understood yet. We address this question here using neural simulations and whole cell intracellular recordings in combination with information theoretic analysis. We show that an adaptive spike threshold leads to better stimulus discrimination for tight input correlations than would be achieved otherwise, independent from whether the stimulus is encoded in the rate or pattern of action potentials. The time scales of input selectivity are jointly governed by membrane and threshold dynamics. Encoding information using adaptive thresholds further ensures robust information transmission across cortical states i.e. decoding from different states is less state dependent in the adaptive threshold case, if the decoding is performed in reference to the timing of the population response. Results from in vitro neural recordings were consistent with simulations from adaptive threshold neurons. In summary, the adaptive spike threshold reduces information loss during intracellular information transfer, improves stimulus discriminability and ensures robust decoding across membrane states in a regime of highly correlated inputs, similar to those seen in sensory nuclei during the encoding of sensory information.     

Predicting human perceptual decisions by decoding neuronal information profiles

Perception relies on the response of populations of neurons in sensory cortex. How the response profile of a neuronal population gives rise to perception and perceptual discrimination has been conceptualized in various ways. Here we suggest that neuronal population responses represent information about our environment explicitly as Fisher information (FI), which is a local measure of the variance estimate of the sensory input. We show how this sensory information can be read out and combined to infer from the available information profile which stimulus value is perceived during a fine discrimination task. In particular, we propose that the perceived stimulus corresponds to the stimulus value that leads to the same information for each of the alternative directions, and compare the model prediction to standard models considered in the literature (population vector, maximum likelihood, maximum-a-posteriori Bayesian inference). The models are applied to human performance in a motion discrimination task that induces perceptual misjudgements of a target direction of motion by task irrelevant motion in the spatial surround of the target stimulus (motion repulsion). By using the neurophysiological insight that surround motion suppresses neuronal responses to the target motion in the center, all models predicted the pattern of perceptual misjudgements. The variation of discrimination thresholds (error on the perceived value) was also explained through the changes of the total FI content with varying surround motion directions. The proposed FI decoding scheme incorporates recent neurophysiological evidence from macaque visual cortex showing that perceptual decisions do not rely on the most active neurons, but rather on the most informative neuronal responses. We statistically compare the prediction capability of the FI decoding approach and the standard decoding models. Notably, all models reproduced the variation of the perceived stimulus values for different surrounds, but with different neuronal tuning characteristics underlying perception. Compared to the FI approach the prediction power of the standard models was based on neurons with far wider tuning width and stronger surround suppression. Our study demonstrates that perceptual misjudgements can be based on neuronal populations encoding explicitly the available sensory information, and provides testable neurophysiological predictions on neuronal tuning characteristics underlying human perceptual decisions.     

Dissociation of Detection and Discrimination of Pure Tones following Bilateral Lesions of Auditory Cortex

It is well known that damage to the peripheral auditory system causes deficits in tone detection as well as pitch and loudness perception across a wide range of frequencies. However, the extent to which to which the auditory cortex plays a critical role in these basic aspects of spectral processing, especially with regard to speech, music, and environmental sound perception, remains unclear. Recent experiments indicate that primary auditory cortex is necessary for the normally-high perceptual acuity exhibited by humans in pure-tone frequency discrimination. The present study assessed whether the auditory cortex plays a similar role in the intensity domain and contrasted its contribution to sensory versus discriminative aspects of intensity processing. We measured intensity thresholds for pure-tone detection and pure-tone loudness discrimination in a population of healthy adults and a middle-aged man with complete or near-complete lesions of the auditory cortex bilaterally. Detection thresholds in his left and right ears were 16 and 7 dB HL, respectively, within clinically-defined normal limits. In contrast, the intensity threshold for monaural loudness discrimination at 1 kHz was 6.5±2.1 dB in the left ear and 6.5±1.9 dB in the right ear at 40 dB sensation level, well above the means of the control population (left ear: 1.6±0.22 dB; right ear: 1.7±0.19 dB). The results indicate that auditory cortex lowers just-noticeable differences for loudness discrimination by approximately 5 dB but is not necessary for tone detection in quiet. Previous human and Old-world monkey experiments employing lesion-effect, neurophysiology, and neuroimaging methods to investigate the role of auditory cortex in intensity processing are reviewed.     

Perceptual learning via modification of cortical top-down signals

The primary visual cortex (V1) is pre-wired to facilitate the extraction of behaviorally important visual features. Collinear edge detectors in V1, for instance, mutually enhance each other to improve the perception of lines against a noisy background. The same pre-wiring that facilitates line extraction, however, is detrimental when subjects have to discriminate the brightness of different line segments. How is it possible to improve in one task by unsupervised practicing, without getting worse in the other task? The classical view of perceptual learning is that practicing modulates the feedforward input stream through synaptic modifications onto or within V1. However, any rewiring of V1 would deteriorate other perceptual abilities different from the trained one. We propose a general neuronal model showing that perceptual learning can modulate top-down input to V1 in a task-specific way while feedforward and lateral pathways remain intact. Consistent with biological data, the model explains how context-dependent brightness discrimination is improved by a top-down recruitment of recurrent inhibition and a top-down induced increase of the neuronal gain within V1. Both the top-down modulation of inhibition and of neuronal gain are suggested to be universal features of cortical microcircuits which enable perceptual learning.     

Is discrimination enhanced at the boundaries of perceptual categories? A negative case

The human visual system imposes discrete perceptual categories on the continuous input space that is represented by the ratios of excitations of the cones in the retina. Is discrimination enhanced at the boundaries between perceptual hues, in the way that discrimination may be enhanced at the boundaries between speech sounds in hearing? In the chromaticity diagram, the locus of unique green separates colours that appear yellowish from those that appear bluish. Using a two-alternative spatial forced choice and an adapting field equivalent to the Daylight Illuminant D65, we measured chromatic discrimination along lines orthogonal to the locus of unique green. In experimental runs interleaved with these performance measurements, we obtained estimates of the phenomenological boundary from the same observers. No enhancement of objectively measured discrimination was observed at the category boundary between yellowish and bluish hues. Instead, thresholds were minimal at chromaticities where the ratio of long-wave to middle-wave cone excitation was the same as that for the background adapting field.     

Multisensory Oddity Detection as Bayesian Inference

A key goal for the perceptual system is to optimally combine information from all the senses that may be available in order to develop the most accurate and unified picture possible of the outside world. The contemporary theoretical framework of ideal observer maximum likelihood integration (MLI) has been highly successful in modelling how the human brain combines information from a variety of different sensory modalities. However, in various recent experiments involving multisensory stimuli of uncertain correspondence, MLI breaks down as a successful model of sensory combination. Within the paradigm of direct stimulus estimation, perceptual models which use Bayesian inference to resolve correspondence have recently been shown to generalize successfully to these cases where MLI fails. This approach has been known variously as model inference, causal inference or structure inference. In this paper, we examine causal uncertainty in another important class of multi-sensory perception paradigm – that of oddity detection and demonstrate how a Bayesian ideal observer also treats oddity detection as a structure inference problem. We validate this approach by showing that it provides an intuitive and quantitative explanation of an important pair of multi-sensory oddity detection experiments – involving cues across and within modalities – for which MLI previously failed dramatically, allowing a novel unifying treatment of within and cross modal multisensory perception. Our successful application of structure inference models to the new ‘oddity detection’ paradigm, and the resultant unified explanation of across and within modality cases provide further evidence to suggest that structure inference may be a commonly evolved principle for combining perceptual information in the brain.     

Chromatic and achromatic stimulus discrimination of long wavelength (red) visual stimuli by the honeybee Apis mellifera

It has long been assumed that bees cannot see red. However, bees visit red flowers, and the visual spectral sensitivity of bees extends into wavelengths to provide sensitivity to such flowers. We thus investigated whether bees can discriminate stimuli reflecting wavelengths above 560 nm, i.e., which appear orange and red to a human observer. Flowers do not reflect monochromatic (single wavelength) light; specifically orange and red flowers have reflectance patterns which are step functions, we thus used colored stimuli with such reflectance patterns. We first conditioned honey bees Apis mellifera to detect six stimuli reflecting light mostly above 560 nm and found that bees learned to detect only stimuli which were perceptually very different from a bee achromatic background. In a second experiment we conditioned bees to discriminate stimuli from a salient, negative (un-rewarded) yellow stimulus. In subsequent unrewarded tests we presented the bees with the trained situation and with five other tests in which the trained stimulus was presented against a novel one. We found that bees learned to discriminate the positive from the negative stimulus, and could unambiguously discriminate eight out of fifteen stimulus pairs. The performance of bees was positively correlated with differences between the trained and the novel stimulus in the receptor contrast for the long-wavelength bee photoreceptor and in the color distance (calculated using two models of the honeybee colors space). We found that the differential conditioning resulted in a concurrent inhibitory conditioning of the negative stimulus, which might have improved discrimination of stimuli which are perceptually similar. These results show that bees can detect long wavelength stimuli which appear reddish to a human observer. The mechanisms underlying discrimination of these stimuli are discussed. Handling Editor: Lars Chittka.     

Generalization and categorization of spectral colors in goldfish I. Experiments with one training wavelength

Goldfish have a tetrachromatic color vision with a high discrimination ability for spectral colors as well as for object colors. We investigate the question whether goldfish organize the high number of discriminable colors in terms of color categories, i.e. in a few larger groups of colors independent of wavelength discrimination. Twenty-four goldfish were trained with food reward, each fish on one out of 13 wavelengths between 371 nm and 630 nm. In transfer tests two different wavelengths were presented, one shorter and one longer than the training wavelength, and the choice behavior was determined. Choice frequencies of ≥50% were assumed to indicate similarity to the training color. The wavelength ranges ≥50% were about 100 nm and twice as large as the just noticeable differences measured in wavelength discrimination tests (Fig. 7). The ranges were surprisingly about the same for all training wavelengths, provided the data were plotted on a wavelength scale weighted according to discrimination ability (Fig. 4). Thus, with the training method chosen goldfish showed a kind of categorization which, however, depends on training wavelength and discrimination ability. Generalization tests in which training wavelength and test wavelengths were shown separately for 2 min each gave the same results as wavelength discrimination tests (Figs. 5 and 6) and are, therefore, not indicative for color categories.     

On the perception of probable things: neural substrates of associative memory, imagery, and perception

Perception is influenced both by the immediate pattern of sensory inputs and by memories acquired through prior experiences with the world. Throughout much of its illustrious history, however, study of the cellular basis of perception has focused on neuronal structures and events that underlie the detection and discrimination of sensory stimuli. Relatively little attention has been paid to the means by which memories interact with incoming sensory signals. Building upon recent neurophysiological/behavioral studies of the cortical substrates of visual associative memory, I propose a specific functional process by which stored information about the world supplements sensory inputs to yield neuronal signals that can account for visual perceptual experience. This perspective represents a significant shift in the way we think about the cellular bases of perception.     

Perceived Loudness of Self-Generated Sounds Is Differentially Modified by Expected Sound Intensity

Performing actions with sensory consequences modifies physiological and behavioral responses relative to otherwise identical sensory input perceived in a passive manner. It is assumed that such modifications occur through an efference copy sent from motor cortex to sensory regions during performance of voluntary actions. In the auditory domain most behavioral studies report attenuated perceived loudness of self-generated auditory action-consequences. However, several recent behavioral and physiological studies report enhanced responses to such consequences. Here we manipulated the intensity of self-generated and externally-generated sounds and examined the type of perceptual modification (enhancement vs. attenuation) reported by healthy human subjects. We found that when the intensity of self-generated sounds was low, perceived loudness is enhanced. Conversely, when the intensity of self-generated sounds was high, perceived loudness is attenuated. These results might reconcile some of the apparent discrepancies in the reported literature and suggest that efference copies can adapt perception according to the differential sensory context of voluntary actions.     

Sensory signals during active versus passive movement

Our sensory systems are simultaneously activated as the result of our own actions and changes in the external world. The ability to distinguish self-generated sensory events from those that arise externally is thus essential for perceptual stability and accurate motor control. Recently, progress has been made towards understanding how this distinction is made. It has been proposed that an internal prediction of the consequences of our actions is compared to the actual sensory input to cancel the resultant self-generated activation. Evidence in support of this hypothesis has been obtained for early stages of sensory processing in the vestibular, visual and somatosensory systems. These findings have implications for the sensory-motor transformations that are needed to guide behavior.     

Suprathreshold perceptual decisions constrain models of confidence

Perceptual confidence is an important internal signal about the certainty of our decisions and there is a substantial debate on how it is computed. We highlight three confidence metric types from the literature: observers either use 1) the full probability distribution to compute probability correct (Probability metrics), 2) point estimates from the perceptual decision process to estimate uncertainty (Evidence-Strength metrics), or 3) heuristic confidence from stimulus-based cues to uncertainty (Heuristic metrics). These metrics are rarely tested against one another, so we examined models of all three types on a suprathreshold spatial discrimination task. Observers were shown a cloud of dots sampled from a dot generating distribution and judged if the mean of the distribution was left or right of centre. In addition to varying the horizontal position of the mean, there were two sensory uncertainty manipulations: the number of dots sampled and the spread of the generating distribution. After every two perceptual decisions, observers made a confidence forced-choice judgement whether they were more confident in the first or second decision. Model results showed that the majority of observers were best-fit by either: 1) the Heuristic model, which used dot cloud position, spread, and number of dots as cues; or 2) an Evidence-Strength model, which computed the distance between the sensory measurement and discrimination criterion, scaled according to sensory uncertainty. An accidental repetition of some sessions also allowed for the measurement of confidence agreement for identical pairs of stimuli. This N-pass analysis revealed that human observers were more consistent than their best-fitting model would predict, indicating there are still aspects of confidence that are not captured by our modelling. As such, we propose confidence agreement as a useful technique for computational studies of confidence. Taken together, these findings highlight the idiosyncratic nature of confidence computations for complex decision contexts and the need to consider different potential metrics and transformations in the confidence computation.     

Delayed perceptual awareness in rapid perceptual decisions

The flourishing of studies on the neural correlates of decision-making calls for an appraisal of the relation between perceptual decisions and conscious perception. By exploiting the long integration time of noisy motion stimuli, and by forcing human observers to make difficult speeded decisions--sometimes a blind guess--about stimulus direction, we traced the temporal buildup of motion discrimination capability and perceptual awareness, as assessed trial by trial through direct rating. We found that both increased gradually with motion coherence and viewing time, but discrimination was systematically leading awareness, reaching a plateau much earlier. Sensitivity and criterion changes contributed jointly to the slow buildup of perceptual awareness. It made no difference whether motion discrimination was accomplished by saccades or verbal responses. These findings suggest that perceptual awareness emerges on the top of a developing or even mature perceptual decision. We argue that the middle temporal (MT) cortical region does not confer us the full phenomenic depth of motion perception, although it may represent a precursor stage in building our subjective sense of visual motion.     

Predictive Coding or Evidence Accumulation? False Inference and Neuronal Fluctuations

Perceptual decisions can be made when sensory input affords an inference about what generated that input. Here, we report findings from two independent perceptual experiments conducted during functional magnetic resonance imaging (fMRI) with a sparse event-related design. The first experiment, in the visual modality, involved forced-choice discrimination of coherence in random dot kinematograms that contained either subliminal or periliminal motion coherence. The second experiment, in the auditory domain, involved free response detection of (non-semantic) near-threshold acoustic stimuli. We analysed fluctuations in ongoing neural activity, as indexed by fMRI, and found that neuronal activity in sensory areas (extrastriate visual and early auditory cortex) biases perceptual decisions towards correct inference and not towards a specific percept. Hits (detection of near-threshold stimuli) were preceded by significantly higher activity than both misses of identical stimuli or false alarms, in which percepts arise in the absence of appropriate sensory input. In accord with predictive coding models and the free-energy principle, this observation suggests that cortical activity in sensory brain areas reflects the precision of prediction errors and not just the sensory evidence or prediction errors per se.