Learning of Multi-Modal Stimuli in Hawkmoths

The hawkmoth, Manduca sexta, uses both colour and odour to find flowers when foraging for nectar. In the present study we investigated how vision and olfaction interact during learning. Manduca sexta were equally attracted to a scented blue coloured feeding target (multimodal stimulus) as to one that does not carry any scent (unimodal stimulus; visual) or to an invisible scented target (unimodal stimulus; odour). This naive attraction to multimodal as well as to unimodal stimuli could be manipulated through training. Moths trained to feed from a blue, scented multimodal feeding target will, when tested in a set-up containing all three feeding targets, select the multimodal target as well as the scented, unimodal target, but ignore the visual target. Interestingly, moths trained to feed from a blue, unimodal visual feeding target will select the visual target as well as the scented, multimodal target, but ignore the unimodal odour target. Our results indicate that a multimodal target is perceived as two separate modalities, colour and odour, rather than as a unique fused target. These findings differ from earlier studies of desert ants that perceive combined visual and odour signals as a unique fused stimulus following learning trials.     

Discrimination training with multimodal stimuli changes activity in the mushroom body of the hawkmoth Manduca sexta

Background

The mushroom bodies of the insect brain play an important role in olfactory processing, associative learning and memory. The mushroom bodies show odor-specific spatial patterns of activity and are also influenced by visual stimuli.

Methodology/Principal Findings

Functional imaging was used to investigate changes in the in vivo responses of the mushroom body of the hawkmoth Manduca sexta during multimodal discrimination training. A visual and an odour stimulus were presented either together or individually. Initially, mushroom body activation patterns were identical to the odour stimulus and the multimodal stimulus. After training, however, the mushroom body response to the rewarded multimodal stimulus was significantly lower than the response to the unrewarded unimodal odour stimulus, indicating that the coding of the stimuli had changed as a result of training. The opposite pattern was seen when only the unimodal odour stimulus was rewarded. In this case, the mushroom body was more strongly activated by the multimodal stimuli after training. When no stimuli were rewarded, the mushroom body activity decreased for both the multimodal and unimodal odour stimuli. There was no measurable response to the unimodal visual stimulus in any of the experiments. These results can be explained using a connectionist model where the mushroom body is assumed to be excited by olfactory stimulus components, and suppressed by multimodal configurations.

Conclusions

Discrimination training with multimodal stimuli consisting of visual and odour cues leads to stimulus specific changes in the in vivo responses of the mushroom body of the hawkmoth.     

Reproducibility and discriminability of brain patterns of semantic categories enhanced by congruent audiovisual stimuli

One of the central questions in cognitive neuroscience is the precise neural representation, or brain pattern, associated with a semantic category. In this study, we explored the influence of audiovisual stimuli on the brain patterns of concepts or semantic categories through a functional magnetic resonance imaging (fMRI) experiment. We used a pattern search method to extract brain patterns corresponding to two semantic categories: "old people" and "young people." These brain patterns were elicited by semantically congruent audiovisual, semantically incongruent audiovisual, unimodal visual, and unimodal auditory stimuli belonging to the two semantic categories. We calculated the reproducibility index, which measures the similarity of the patterns within the same category. We also decoded the semantic categories from these brain patterns. The decoding accuracy reflects the discriminability of the brain patterns between two categories. The results showed that both the reproducibility index of brain patterns and the decoding accuracy were significantly higher for semantically congruent audiovisual stimuli than for unimodal visual and unimodal auditory stimuli, while the semantically incongruent stimuli did not elicit brain patterns with significantly higher reproducibility index or decoding accuracy. Thus, the semantically congruent audiovisual stimuli enhanced the within-class reproducibility of brain patterns and the between-class discriminability of brain patterns, and facilitate neural representations of semantic categories or concepts. Furthermore, we analyzed the brain activity in superior temporal sulcus and middle temporal gyrus (STS/MTG). The strength of the fMRI signal and the reproducibility index were enhanced by the semantically congruent audiovisual stimuli. Our results support the use of the reproducibility index as a potential tool to supplement the fMRI signal amplitude for evaluating multimodal integration.     

Stereoscopic Offset Makes Objects Easier to Recognize

Binocular vision is obviously useful for depth perception, but it might also enhance other components of visual processing, such as image segmentation. We used naturalistic images to determine whether giving an object a stereoscopic offset of 15-120 arcmin of crossed disparity relative to its background would make the object easier to recognize in briefly presented (33-133 ms), temporally masked displays. Disparity had a beneficial effect across a wide range of disparities and display durations. Most of this benefit occurred whether or not the stereoscopic contour agreed with the object’s luminance contour. We attribute this benefit to an orienting of spatial attention that selected the object and its local background for enhanced 2D pattern processing. At longer display durations, contour agreement provided an additional benefit, and a separate experiment using random-dot stimuli confirmed that stereoscopic contours plausibly contributed to recognition at the longer display durations in our experiment. We conclude that in real-world situations binocular vision confers an advantage not only for depth perception, but also for recognizing objects from their luminance patterns and bounding contours.     

Accurate memory for colour but not pattern contrast in chicks

The visual displays of animals and plants often look dramatic and colourful to us, but what information do they convey to their intended, non-human, audience [1] [2]? One possibility is that stimulus values are judged accurately - so, for example, a female might choose a suitor if he displays a specific colour [3]. Alternatively, as for human advertising, displays may attract attention without giving information, perhaps by exploiting innate preferences for bright colours or symmetry [2] [4] [5]. To address this issue experimentally, we investigated chicks' memories of visual patterns. Food was placed in patterned paper containers which, like seed pods or insect prey, must be manipulated to extract food and their patterns learnt. To establish what was learnt, birds were tested on familiar stimuli and on alternative stimuli of differing colour or contrast. For colour, birds selected the trained stimulus; for contrast, they preferred high contrast patterns over the familiar. These differing responses to colour and contrast show how separate components of display patterns could serve different roles, with colour being judged accurately whereas pattern contrast attracts attention.     

Chromatic signals control proboscis movements during hovering flight in the hummingbird hawkmoth Macroglossum stellatarum

Most visual systems are more sensitive to luminance than to colour signals. Animals resolve finer spatial detail and temporal changes through achromatic signals than through chromatic ones. Probably, this explains that detection of small, distant, or moving objects is typically mediated through achromatic signals. Macroglossum stellatarum are fast flying nectarivorous hawkmoths that inspect flowers with their long proboscis while hovering. They can visually control this behaviour using floral markings known as nectar guides. Here, we investigate whether this is mediated by chromatic or achromatic cues. We evaluated proboscis placement, foraging efficiency, and inspection learning of naïve moths foraging on flower models with coloured markings that offered either chromatic, achromatic or both contrasts. Hummingbird hawkmoths could use either achromatic or chromatic signals to inspect models while hovering. We identified three, apparently independent, components controlling proboscis placement: After initial contact, 1) moths directed their probing towards the yellow colour irrespectively of luminance signals, suggesting a dominant role of chromatic signals; and 2) moths tended to probe mainly on the brighter areas of models that offered only achromatic signals. 3) During the establishment of the first contact, naïve moths showed a tendency to direct their proboscis towards the small floral marks independent of their colour or luminance. Moths learned to find nectar faster, but their foraging efficiency depended on the flower model they foraged on. Our results imply that M. stellatarum can perceive small patterns through colour vision. We discuss how the different informational contents of chromatic and luminance signals can be significant for the control of flower inspection, and visually guided behaviours in general.     

Multimodal cues improve prey localization under complex environmental conditions

Predators often eavesdrop on sexual displays of their prey. These displays can provide multimodal cues that aid predators, but the benefits in attending to them should depend on the environmental sensory conditions under which they forage. We assessed whether bats hunting for frogs use multimodal cues to locate their prey and whether their use varies with ambient conditions. We used a robotic set-up mimicking the sexual display of a male túngara frog (Physalaemus pustulosus) to test prey assessment by fringe-lipped bats (Trachops cirrhosus). These predatory bats primarily use sound of the frog''s call to find their prey, but the bats also use echolocation cues returning from the frog''s dynamically moving vocal sac. In the first experiment, we show that multimodal cues affect attack behaviour: bats made narrower flank attack angles on multimodal trials compared with unimodal trials during which they could only rely on the sound of the frog. In the second experiment, we explored the bat''s use of prey cues in an acoustically more complex environment. Túngara frogs often form mixed-species choruses with other frogs, including the hourglass frog (Dendropsophus ebraccatus). Using a multi-speaker set-up, we tested bat approaches and attacks on the robofrog under three different levels of acoustic complexity: no calling D. ebraccatus males, two calling D. ebraccatus males and five D. ebraccatus males. We found that bats are more directional in their approach to the robofrog when more D. ebraccatus males were calling. Thus, bats seemed to benefit more from multimodal cues when confronted with increased levels of acoustic complexity in their foraging environments. Our data have important consequences for our understanding of the evolution of multimodal sexual displays as they reveal how environmental conditions can alter the natural selection pressures acting on them.     

Aggressive Bimodal Communication in Domestic Dogs,Canis familiaris

Evidence of animal multimodal signalling is widespread and compelling. Dogs’ aggressive vocalisations (growls and barks) have been extensively studied, but without any consideration of the simultaneously produced visual displays. In this study we aimed to categorize dogs’ bimodal aggressive signals according to the redundant/non-redundant classification framework. We presented dogs with unimodal (audio or visual) or bimodal (audio-visual) stimuli and measured their gazing and motor behaviours. Responses did not qualitatively differ between the bimodal and two unimodal contexts, indicating that acoustic and visual signals provide redundant information. We could not further classify the signal as ‘equivalent’ or ‘enhancing’ as we found evidence for both subcategories. We discuss our findings in relation to the complex signal framework, and propose several hypotheses for this signal’s function.     

Multimodal Floral Signals and Moth Foraging Decisions

Background

Combinations of floral traits – which operate as attractive signals to pollinators – act on multiple sensory modalities. For Manduca sexta hawkmoths, how learning modifies foraging decisions in response to those traits remains untested, and the contribution of visual and olfactory floral displays on behavior remains unclear.

Methodology/Principal Findings

Using M. sexta and the floral traits of two important nectar resources in southwestern USA, Datura wrightii and Agave palmeri, we examined the relative importance of olfactory and visual signals. Natural visual and olfactory cues from D. wrightii and A. palmeri flowers permits testing the cues at their native intensities and composition – a contrast to many studies that have used artificial stimuli (essential oils, single odorants) that are less ecologically relevant. Results from a series of two-choice assays where the olfactory and visual floral displays were manipulated showed that naïve hawkmoths preferred flowers displaying both olfactory and visual cues. Furthermore, experiments using A. palmeri flowers – a species that is not very attractive to hawkmoths – showed that the visual and olfactory displays did not have synergistic effects. The combination of olfactory and visual display of D. wrightii, however – a flower that is highly attractive to naïve hawkmoths – did influence the time moths spent feeding from the flowers. The importance of the olfactory and visual signals were further demonstrated in learning experiments in which experienced moths, when exposed to uncoupled floral displays, ultimately chose flowers based on the previously experienced olfactory, and not visual, signals. These moths, however, had significantly longer decision times than moths exposed to coupled floral displays.

Conclusions/Significance

These results highlight the importance of specific sensory modalities for foraging hawkmoths while also suggesting that they learn the floral displays as combinatorial signals and use the integrated floral traits from their memory traces to mediate future foraging decisions.     

A neural network model of ventriloquism effect and aftereffect

Presenting simultaneous but spatially discrepant visual and auditory stimuli induces a perceptual translocation of the sound towards the visual input, the ventriloquism effect. General explanation is that vision tends to dominate over audition because of its higher spatial reliability. The underlying neural mechanisms remain unclear. We address this question via a biologically inspired neural network. The model contains two layers of unimodal visual and auditory neurons, with visual neurons having higher spatial resolution than auditory ones. Neurons within each layer communicate via lateral intra-layer synapses; neurons across layers are connected via inter-layer connections. The network accounts for the ventriloquism effect, ascribing it to a positive feedback between the visual and auditory neurons, triggered by residual auditory activity at the position of the visual stimulus. Main results are: i) the less localized stimulus is strongly biased toward the most localized stimulus and not vice versa; ii) amount of the ventriloquism effect changes with visual-auditory spatial disparity; iii) ventriloquism is a robust behavior of the network with respect to parameter value changes. Moreover, the model implements Hebbian rules for potentiation and depression of lateral synapses, to explain ventriloquism aftereffect (that is, the enduring sound shift after exposure to spatially disparate audio-visual stimuli). By adaptively changing the weights of lateral synapses during cross-modal stimulation, the model produces post-adaptive shifts of auditory localization that agree with in-vivo observations. The model demonstrates that two unimodal layers reciprocally interconnected may explain ventriloquism effect and aftereffect, even without the presence of any convergent multimodal area. The proposed study may provide advancement in understanding neural architecture and mechanisms at the basis of visual-auditory integration in the spatial realm.     

Pattern discrimination in a hawkmoth: innate preferences,learning performance and ecology

Spatial patterns are important cues for flower detection and recognition by nectar-feeding insects. Pattern vision has been studied in much detail in bees and flies but rarely in butterflies and moths. In this paper, I present a first proof of pattern-learning abilities in a moth, and discuss reasons for the limitations to their pattern learning. The diurnal hawkmoth Macroglossum stellatarum spontaneously prefers patterned to uniformly coloured stimuli but can be trained to choose the uniform stimulus. By contrast, experience does not override the innate preferences for radial over tangential patterns, and for tangential over striped patterns. These results do not reflect bad visual discrimination but rather a lack of learning ability and motivation to abolish innate preferences. I propose that radial and tangential flower patterns are good predictors of nectar reward, a condition under which learning is unlikely to evolve. These patterns serve not only as cues for flower detection but also as guides to the reward. Hovering pollinators strongly depend on these guides and should therefore: (i). have rigid pattern preferences; and (ii). not be motivated to abolish these preferences as easily as their innate preferences for colours.     

Activity patterns of the serotine bat (Eptesicus serotinus) at a roost in southern England

Activity patterns and emergence times of a colony of serotine bats, Eptesicus serotinus , were studied in southern England. Time of emergence from the day roost varied over the summer but was strongly correlated with sunset. Mean emergence time was 11.6 ± 7.7 min after sunset. Early in summer, activity patterns were unimodal, becoming bimodal during mid- to late pregnancy and multimodal in early to mid-lactation. When juveniles were volant, activity patterns became unimodal again. Periods of low ambient temperature were associated with reduced activity. The duration of the first foraging flight decreased as pregnancy progressed, possibly as a result of the greater wing-loading caused by increased body mass. However, the first foraging flight increased in duration during the course of lactation, probably in response to a combination of increased night length and the increased energetic demands of milk production. It is concluded that seasonal variation in the length of time spent away from the roost in the serotine is related to reproductive status, night length and ambient temperature. It is suggested that the more northerly distribution of this bat in continental Europe may be due to differences in habitat use and diet.     

Book reviews

The cichlid fishes of Lake Malawi represent a prime example of a rapid evolutionary radiation via sexual selection. We show that in this group, acoustic communication significantly coincides with visual courtship displays. Specifically, individuals tend to use both multimodal and unimodal displays in successive courtship bouts. Behavioral analyses on six species of Malawian cichlids from two divergent genera revealed that five of the species use displays containing both an auditory and a visual component. Metriaclima zebra “katale” was the notable exception, along with a single individual of Metriaclima lombardoi, which utilized unimodal and multimodal displays with equal frequency. Taken together, these results highlight the importance of behavioral lability in multiple sensory signals at both the intra- and interspecific levels.     

Temporal polyethism in Korean yellowjacket foragers,Vespula koreensis (Hymenoptera,Vespidae)

We examined the foraging behavior of the Korean yellowjacket, Vespula koreensis, to determine whether this species displays temporal polyethism. Using video-recordings of the entrances of artificial nest boxes installed in the field, we investigated the association between the tasks performed by workers and age. We identified three foraging tasks (pulp, nectar and prey foraging). Pulp foraging was performed by younger foragers, while nectar and prey foraging were performed by older foragers. We measured the foraging time (time spent outside of the nest during a single foraging bout) and the weight of the materials that foragers brought into the nest for each task to estimate the cost of the task. Pulp foraging was less costly than nectar or prey foraging by both measures. Taken together, the results suggest that yellowjacket foragers tend to perform low-cost task in their early foraging days and high-cost task later. Our results add to a growing literature showing temporal polyethism in social insects.     

Crossmodal spatial influences of touch on extrastriate visual areas take current gaze direction into account

Recent results indicate that crossmodal interactions can affect activity in cortical regions traditionally regarded as "unimodal." Previously we found that combining touch on one hand with visual stimulation in the anatomically corresponding hemifield could boost responses in contralateral visual cortex. Here we manipulated which visual hemifield corresponded to the location of the stimulated hand, by changing gaze direction such that right-hand touch could now arise in either the left or right visual field. Crossmodal effects on visual cortex switched from one hemisphere to the other, depending on gaze direction, regardless of whether the hand was seen. This indicates that crossmodal influences of touch upon visual cortex depend on spatial alignment for the multimodal stimuli, with gaze posture taken into account.     

Issues in the classification of multimodal communication signals

Communication involves complex behavior in multiple sensory channels, or "modalities." We provide an overview of multimodal communication and its costs and benefits, place examples of signals and displays from an array of taxa, sensory systems, and functions into our signal classification system, and consider issues surrounding the categorization of multimodal signals. The broadest level of classification is between signals with redundant and nonredundant components, with finer distinctions in each category. We recommend that researchers gather information on responses to each component of a multimodal signal as well as the response to the signal as a whole. We discuss the choice of categories, whether to categorize signals on the basis of the signal or the response, and how to classify signals if data are missing. The choice of behavioral assay may influence the outcome, as may the context of the communicative event. We also consider similarities and differences between multimodal and unimodal composite signals and signals that are sequentially, rather than simultaneously, multimodal.     

Reversal Learning and Risk‐Averse Foraging Behavior in the Monarch Butterfly,Danaus plexippus (Lepidoptera: Nymphalidae)

Learning ability allows insects to respond to a variable environment, and to adjust their behaviors in response to positive or negative experiences. Pollinating insects readily learn to associate floral characteristics, such as color, shape, or pattern, with appetitive stimuli, such as the presence of a nectar reward. However, in nature pollinators may also encounter flowers that contain distasteful or toxic nectar, or offer highly variable nectar volumes, providing opportunities for aversive learning or risk‐averse foraging behavior. Whereas some bees learn to avoid flowers with unpalatable or unreliable nectar rewards, little is known about how Lepidoptera respond to such stimuli. We used a reversal learning paradigm to establish that monarch butterflies learn to discriminate against colored artificial flowers that contain salt solution, decreasing both number of probes and probing time on flowers of a preferred color and altogether avoiding artificial flowers of a non‐preferred color. In addition, when we offered butterflies artificial flowers of two different colors, both of which contained the same mean nectar volume but which differed in variance, the monarchs exhibited risk‐averse foraging: they probed the constant flowers significantly more than the variable ones, regardless of flower color or butterfly sex. Our results add to our understanding of butterfly foraging behavior, as they demonstrate that monarchs can respond to aversive as well as appetitive stimuli, and can also adjust their foraging behavior to avoid floral resources with high variance rewards.     

Proximate Costs of Competition for Nectar

Food competition among coexisting nectarivorous birds is conspicuousand often intense, affecting patterns of flower choice, dailybehavior budgets, and timing of successful reproduction. Exploitativecompetition involves loss of accumulated nectar to other individualsthat visited a flower first. Preliminary data support the useof Poisson models of the frequencies of point-source visitationand overlap for determining the probabilities of actual competitiveevents. Nectar losses from monitored flowers can be estimatedin terms of time intervals between visits weighted by flower-specificnectar production and bird-specific nectar removal capabilities.Foraging time budgets then provide a meaningful common denominatorfor assessing the impacts of competitive nectar losses, becausecompensatory increases in foraging time are required to maintaina balanced energy budget. Flexibility in foraging time budgetsmade possible by high efficiency foraging and predictably lowcompetitive losses may be an important determinant of reproductivetiming and success in nectar feeding birds. Aggressive displacement of competitors and territorial defenseof flowers are common forms of interference competition in nectar-feedingbirds. Aggression has definable caloric costs that ultimatelymust relate to caloric gains. Defense of flowers increases theaggressor's exclusive use of nectar, increases the predictabilityof a nectar supply, and increases the average amount of nectarobtained per flower. Simple cost-benefit models of territorialitydefine conditions when net benefits of territoriality are greaterthan those of alternatives.     

Floral signposts: testing the significance of visual 'nectar guides' for pollinator behaviour and plant fitness

Nectar guides, contrasting patterns on flowers that supposedly direct pollinators towards a concealed nectar reward, are taxonomically widespread. However, there have been few studies of their functional significance and effects on plant fitness. Most previous studies focused on pollinator behaviour and used artificial flowers in laboratory settings. We experimentally investigated the role of putative nectar guides in a natural system: the South African iris Lapeirousia oreogena, whose flowers have a clearly visible pattern of six white arrow-markings pointing towards the narrow entrance of the long corolla tube, and its sole pollinator, a long-proboscid nemestrinid fly. We painted over none, some or all of the white arrow-markings with ink that matched the colour of the corolla background. Although arrow-marking removal had little effect on the approaches by flies to flowers from a distance, it dramatically reduced the likelihood of proboscis insertion. Export of pollen dye analogue (an estimate of male fitness) was reduced to almost zero in flowers from which all nectar guides had been removed, and fruit set (a measure of female fitness) was also significantly reduced. Our results confirm that the markings on L. oreogena flowers serve as nectar guides and suggest that they are under strong selective maintenance through both male and female fitness components in this pollination system.     

Foraging strategy predicts foraging economy in a facultative secondary nectar robber

In mutualistic interactions, the decision whether to cooperate or cheat depends on the relative costs and benefits of each strategy. In pollination mutualisms, secondary nectar robbing is a facultative behavior employed by a diverse array of nectar‐feeding organisms, and is thought to be a form of cheating. Primary robbers create holes in floral tissue through which they feed on nectar, whereas secondary robbers, which often lack chewing mouthparts, feed on nectar through existing holes. Because primary robbers make nectar more readily available to secondary robbers, primary robbers facilitate the behaviors of secondary robbers. However, the net effect of facilitation on secondary robber fitness has not been empirically tested: it is unknown whether the benefit secondary robbers receive is strong enough to overcome the cost of competing with primary robbers for a shared resource. We conducted foraging experiments using the bumble bee Bombus bifarius, which can alternatively forage ‘legitimately’ (from the floral opening) or secondary‐rob. We measured the relative foraging efficiencies (handling time per flower, flowers visited per minute, proportion of foraging bout spent consuming nectar) of these alternative behaviors, and tested whether the frequency of primary robbing and nectar standing crop in primary‐robbed flowers of Linaria vulgaris (Plantaginaceae) affected foraging efficiency. Surprisingly, there was no effect of primary robbing frequency on the foraging efficiency of secondary‐robbing B. bifarius. Instead, foraging strategy was a major predictor of foraging efficiency, with legitimate foraging being significantly more efficient than secondary robbing. Legitimate foraging was the more common strategy used by B. bifarius in our study; however, it is rarely used by B. bifarius foraging on L. vulgaris in nature, despite indications that it is more efficient. Our results suggest the need for deeper investigations into why bees adopt secondary robbing as a foraging strategy, specifically, the environmental contexts that promote the behavior.