Multistability in perception: binding sensory modalities, an overview

This special issue presents research concerning multistable perception in different sensory modalities. Multistability occurs when a single physical stimulus produces alternations between different subjective percepts. Multistability was first described for vision, where it occurs, for example, when different stimuli are presented to the two eyes or for certain ambiguous figures. It has since been described for other sensory modalities, including audition, touch and olfaction. The key features of multistability are: (i) stimuli have more than one plausible perceptual organization; (ii) these organizations are not compatible with each other. We argue here that most if not all cases of multistability are based on competition in selecting and binding stimulus information. Binding refers to the process whereby the different attributes of objects in the environment, as represented in the sensory array, are bound together within our perceptual systems, to provide a coherent interpretation of the world around us. We argue that multistability can be used as a method for studying binding processes within and across sensory modalities. We emphasize this theme while presenting an outline of the papers in this issue. We end with some thoughts about open directions and avenues for further research.     

Coherent activity between auditory and visual modalities during the induction of peacefulness

Multisensory integration involves combining information from different senses to create a perception. The diverse characteristics of different sensory systems make it interesting to determine how cooperation and competition contribute to emotional experiences. Therefore, the aim of this study were to estimate the bias from the match attributes of the auditory and visual modalities and to depict specific brain activity frequency (theta, alpha, beta, and gamma) patterns related to a peaceful mood by using magnetoencephalography. The present study provides evidence of auditory domination in perceptual bias during multimodality processing of peaceful consciousness. Coherence analysis suggested that the theta oscillations are a transmitter of emotion signals, with the left and right brains being active in peaceful and fearful moods, respectively. Notably, hemispheric lateralization was also apparent in the alpha and beta oscillations, which might govern simple or pure information (e.g. from single modality) in the right brain but complex or mixed information (e.g. from multiple modalities) in the left brain.     

Towards omni-tomography--grand fusion of multiple modalities for simultaneous interior tomography

We recently elevated interior tomography from its origin in computed tomography (CT) to a general tomographic principle, and proved its validity for other tomographic modalities including SPECT, MRI, and others. Here we propose "omni-tomography", a novel concept for the grand fusion of multiple tomographic modalities for simultaneous data acquisition in a region of interest (ROI). Omni-tomography can be instrumental when physiological processes under investigation are multi-dimensional, multi-scale, multi-temporal and multi-parametric. Both preclinical and clinical studies now depend on in vivo tomography, often requiring separate evaluations by different imaging modalities. Over the past decade, two approaches have been used for multimodality fusion: Software based image registration and hybrid scanners such as PET-CT, PET-MRI, and SPECT-CT among others. While there are intrinsic limitations with both approaches, the main obstacle to the seamless fusion of multiple imaging modalities has been the bulkiness of each individual imager and the conflict of their physical (especially spatial) requirements. To address this challenge, omni-tomography is now unveiled as an emerging direction for biomedical imaging and systems biomedicine.     

The role of various modalities in breast imaging

Background: Breast cancer is the most common type of cancer in women worldwide. Mammography is considered the "gold standard" in the evaluation of the breast from an imaging perspective. Apart from mammography, ultrasound examination and magnetic resonance imaging are being offered as adjuncts to the preoperative workup. Recently, other new modalities like positron emission tomography, 99mTc-sestamibi scintimammography, and electrical impedance tomography (EIT) are also being offered. However, there is still controversy over the most appropriate use of these new modalities. Based on the literature, this review evaluates the role of various modalities used in the screening and diagnosis of breast cancer. Methods and Results: Based on relevant literatures this article gives an overview of the old and new modalities used in the field of breast imaging. A narrative literature review of all the relevant papers known to the authors was conducted. The search of literatures was done using pubmed and ovid search engines. Additional references were found through bibliography reviews of relevant articles. It was clear that though various new technics and methods have emerged, none have substituted mammography and it is still the only proven screening method for the breast as of date. Conclusion: From the literature it is clear that apropos modern radiology's impact on diagnosis, staging and patient follow-up, only one imaging technique has had a significant impact on screening asymptomatic individuals for cancer i.e.; low-dose mammography. Mammography is the only screening test proven in breast imaging. Positron emission tomography (PET) also plays an important role in staging breast cancer and monitoring treatment response. As imaging techniques improve, the role of imaging will continue to evolve with the goal remaining a decrease in breast cancer morbidity and mortality. Progress in the development and commercialisation of EIT breast imaging system will definitely help to promote other systems and applications based on the EIT and similar visualization methods. Breast ultrasound and breast magnetic resonance imaging (MRI) are frequently used adjuncts to mammography in today's clinical practice and these techniques enhance the radiologist's ability to detect cancer and assess disease extent, which is crucial in treatment planning and staging.     

Spatiotemporal mapping of brain activity by integration of multiple imaging modalities

Functional magnetic resonance imaging (fMRI) and positron emission tomography measure local changes in brain hemodynamics induced by cognitive or perceptual tasks. These measures have a uniformly high spatial resolution of millimeters or less, but poor temporal resolution (about 1s). Conversely, electroencephalography (EEG) and magnetoencephalography (MEG) measure instantaneously the current flows induced by synaptic activity, but the accurate localization of these current flows based on EEG and MEG data alone remains an unsolved problem. Recently, techniques have been developed that, in the context of brain anatomy visualized with structural MRI, use both hemodynamic and electromagnetic measures to arrive at estimates of brain activation with high spatial and temporal resolution. These methods range from simple juxtaposition to simultaneous integrated techniques. Their application has already led to advances in our understanding of the neural bases of perception, attention, memory and language. Further advances in multi-modality integration will require an improved understanding of the coupling between the physiological phenomena underlying the different signal modalities.     

Serum vanadium concentrations in different sports modalities

AimsThe aim of this study was to compare serum vanadium (V) concentrations between athletes of different sports modalities and people who did not practise physical exercise regularly.MethodsOne hundred and twenty-one subjects divided into a control group (CG; n = 37; 1.75 ± 0.03 m; 79.45 ± 10.20 kg; 24.72 ± 6.06 years) and an athletes’ group (AG; n = 84; 1.77 ± 0.05 m; 66.34 ± 6.12 kg; 19.57 ± 1.95 years) participated in this research. AG were classified by sports modality: aerobic (AE; n = 26), anaerobic (ANA; n = 22); aerobic-anaerobic (AE-ANA; n = 36). Serum V concentrations were analysed by inductively coupled plasma mass spectrometry.ResultsAG showed higher V concentrations compared to CG (p < 0.01). AE obtained higher concentrations compared to ANA and AE-ANA (p < 0.05).ConclusionsPhysical training could increase serum V levels. Specifically, aerobic sports modalities could increase serum V levels to a greater extent than other sports modalities.     

Recent advances in induced proximity modalities

Challenging disease targets necessitate new approaches for therapeutic intervention. Rewiring protein-biomolecule interactions with proximity-inducing agents extends intervention opportunities beyond target agonism or inhibition. Spanning varied molecular phenotypes and diverse target classes, proximity-inducing agents demonstrate immense potential across target degradation, cleavage, and post-translational editing. Here, we review a selection of exciting developments in the concepts and mechanisms of induced proximity-driven strategies from the last two years. Key technological advances that enable these discoveries and expand the scope of targets and machinery for induced-proximity modalities are highlighted.     

Imaging of brain activation by odorants in humans

Application of positron emission tomography and magnetic resonance imaging has provided several new insights into various olfactory functions. One is that sniffing and smelling engage separate subsystems in the human olfactory cortex. Another is that perception of odorous compounds (odorants) is mediated by a set of core regions, which are partly different for pure olfactory than for olfactory plus trigeminal odorants. Depending on the task associated with odor perception, the core regions are recruited together with other circuits, in a parallel and hierarchical manner. The sense of smell seems, therefore, to be organized similarly to other sensory modalities, and the specific psychophysical characteristics of olfaction should be attributed to an early involvement of the limbic system rather than to a conceptually different mode of processing.     

Can the use of physical modalities for pain control be rationalized by the research evidence?

Physical modalities, including cold and heat, are widely used in the conservative management of pain associated with musculoskeletal disorders. This review has critically appraised the literature supporting the use of these modalities in the treatment of musculoskeletal pain. It was concluded that, apart from a few exceptions and in a few types of disorders, existing evidence does not support the use of these modalities in long-term pain control. There was, however, evidence that several modalities, specifically cold and a form of deep heat (shortwave diathermy), do have short-lived analgesic effects and so may contribute to more painfree function in the short term. Further research is clearly warranted to define the short- and long-term therapeutic efficacy of physical modalities in the treatment of musculoskeletal pain to justify their continued use in clinical practice.     

Temporal dynamics of auditory and visual bistability reveal common principles of perceptual organization

When dealing with natural scenes, sensory systems have to process an often messy and ambiguous flow of information. A stable perceptual organization nevertheless has to be achieved in order to guide behavior. The neural mechanisms involved can be highlighted by intrinsically ambiguous situations. In such cases, bistable perception occurs: distinct interpretations of the unchanging stimulus alternate spontaneously in the mind of the observer. Bistable stimuli have been used extensively for more than two centuries to study visual perception. Here we demonstrate that bistable perception also occurs in the auditory modality. We compared the temporal dynamics of percept alternations observed during auditory streaming with those observed for visual plaids and the susceptibilities of both modalities to volitional control. Strong similarities indicate that auditory and visual alternations share common principles of perceptual bistability. The absence of correlation across modalities for subject-specific biases, however, suggests that these common principles are implemented at least partly independently across sensory modalities. We propose that visual and auditory perceptual organization could rely on distributed but functionally similar neural competition mechanisms aimed at resolving sensory ambiguities.     

Integrating across neuroimaging modalities boosts prediction accuracy of cognitive ability

Variation in cognitive ability arises from subtle differences in underlying neural architecture. Understanding and predicting individual variability in cognition from the differences in brain networks requires harnessing the unique variance captured by different neuroimaging modalities. Here we adopted a multi-level machine learning approach that combines diffusion, functional, and structural MRI data from the Human Connectome Project (N = 1050) to provide unitary prediction models of various cognitive abilities: global cognitive function, fluid intelligence, crystallized intelligence, impulsivity, spatial orientation, verbal episodic memory and sustained attention. Out-of-sample predictions of each cognitive score were first generated using a sparsity-constrained principal component regression on individual neuroimaging modalities. These individual predictions were then aggregated and submitted to a LASSO estimator that removed redundant variability across channels. This stacked prediction led to a significant improvement in accuracy, relative to the best single modality predictions (approximately 1% to more than 3% boost in variance explained), across a majority of the cognitive abilities tested. Further analysis found that diffusion and brain surface properties contribute the most to the predictive power. Our findings establish a lower bound to predict individual differences in cognition using multiple neuroimaging measures of brain architecture, both structural and functional, quantify the relative predictive power of the different imaging modalities, and reveal how each modality provides unique and complementary information about individual differences in cognitive function.     

Imaging modalities for monitoring acute therapeutic effects after near-infrared photoimmunotherapy in vivo

Near-infrared photoimmunotherapy (NIR-PIT) induces immediate cell death after irradiation with near-infrared (NIR) light. Acute therapeutic effects caused by NIR-PIT before the change of tumor size is essential to be monitored by imaging modalities. We summarized and compared the imaging modalities for evaluating acute therapeutic effects after NIR-PIT, and aimed to provide a better understanding of advantages and disadvantages of each modality for evaluation in clinical applications. Fluorescence imaging and fluorescence lifetime, with high resolution, remains high accumulation of fluorescence dyes in the normal organs. High resolution and noninvasiveness are the major advantages of magnetic resonance imaging, while ¹⁸F-fluorodeoxyglucose positron emission tomography provides information about the glucose metabolism. Optical coherence tomography provided more information about the blood vessels. Thus, all of the imaging modalities play an important role in evaluating acute therapeutic effects after NIR-PIT. Clinicians should choose suitable modality according to specific purpose and conditions in clinical application.     

The role of signals of different modalities in initiating vibratory communication in Nezara viridula

Signals of different modalities are involved in the behaviour of the green stink bug, Nezara viridula (L.) (Pentatomidae, Heteroptera). Long range attraction is mediated by male pheromones, resulting in aggregation of bugs on the same plant where vibratory signals, vision and various chemical signals become important. Both males and females sing spontaneously. When both are on the plant, males start vibratory communication as often as females. Females induce the exchange of vibratory signals spontaneously or triggered by the male pheromone while males initiate the duet either spontaneously or after seeing the female. Males and females sing spontaneously and respond to signals of different modalities more often in the daylight than in the dark. Long lasting autonomous emission of the female calling song is present when triggered by the male pheromone and males respond to female calling predominantly by the emission of the courtship song.     

Multiple modalities converge on a common gate to control K2P channel function

Members of the K_2P potassium channel family regulate neuronal excitability and are implicated in pain, anaesthetic responses, thermosensation, neuroprotection, and mood. Unlike other potassium channels, K_2Ps are gated by remarkably diverse stimuli that include chemical, thermal, and mechanical modalities. It has remained unclear whether the various gating inputs act through separate or common channel elements. Here, we show that protons, heat, and pressure affect activity of the prototypical, polymodal K_2P, K_2P2.1 (KCNK2/TREK‐1), at a common molecular gate that comprises elements of the pore‐forming segments and the N‐terminal end of the M4 transmembrane segment. We further demonstrate that the M4 gating element is conserved among K_2Ps and is employed regardless of whether the gating stimuli are inhibitory or activating. Our results define a unique gating mechanism shared by K_2P family members and suggest that their diverse sensory properties are achieved by coupling different molecular sensors to a conserved core gating apparatus.     

The role of lipolysis in human orosensory fat perception

Taste perception elicited by food constituents and facilitated by sensory cells in the oral cavity is important for the survival of organisms. In addition to the five basic taste modalities, sweet, umami, bitter, sour, and salty, orosensory perception of stimuli such as fat constituents is intensely investigated. Experiments in rodents and humans suggest that free fatty acids represent a major stimulus for the perception of fat-containing food. However, the lipid fraction of foods mainly consists of triglycerides in which fatty acids are esterified with glycerol. Whereas effective lipolysis by secreted lipases (LIPs) liberating fatty acids from triglycerides in the rodent oral cavity is well established, a similar mechanism in humans is disputed. By psychophysical analyses of humans, we demonstrate responses upon stimulation with triglycerides which are attenuated by concomitant LIP inhibitor administration. Moreover, lipolytic activities detected in minor salivary gland secretions directly supplying gustatory papillae were correlated to individual sensitivities for triglycerides, suggesting that differential LIP levels may contribute to variant fat perception. Intriguingly, we found that the LIPF gene coding for lingual/gastric LIP is not expressed in human lingual tissue. Instead, we identified the expression of other LIPs, which may compensate for the absence of LIPF.     

Mimicry for all modalities

Mimicry is a canonical example of adaptive signal design. In principle, what constitutes mimicry is independent of the taxonomic identity of the mimic, the ecological context in which it operates, and the sensory modality through which it is expressed. However, in practice the study of mimicry is inconsistent across research fields, with theoretical and empirical advances often failing to cross taxonomic and sensory divides. We propose a novel conceptual framework whereby mimicry evolves if a receiver perceives the similarity between a mimic and a model and as a result confers a selective benefit onto the mimic. Here, misidentification and/or deception are no longer formal requirements, and mimicry can evolve irrespective of the underlying proximate mechanisms. The centrality of receiver perception in this framework enables us to formally distinguish mimicry from perceptual exploitation and integrate mimicry and multicomponent signalling theory for the first time. In addition, it resolves inconsistencies in our understanding of the role of learning in mimicry evolution, and shows that imperfect mimicry is expected to be the norm. Mimicry remains a key model for understanding signal evolution and cognition, and we recommend the adoption of a unified approach to stimulate future interdisciplinary developments in this fascinating area of research.     

Minor and unsystematic cortical topographic changes of attention correlates between modalities

In this study we analyzed the topography of induced cortical oscillations in 20 healthy individuals performing simple attention tasks. We were interested in qualitatively replicating our recent findings on the localization of attention-induced beta bands during a visual task [1], and verifying whether significant topographic changes would follow the change of attention to the auditory modality. We computed corrected latency averaging of each induced frequency bands, and modeled their generators by current density reconstruction with Lp-norm minimization. We quantified topographic similarity between conditions by an analysis of correlations, whereas the inter-modality significant differences in attention correlates were illustrated in each individual case. We replicated the qualitative result of highly idiosyncratic topography of attention-related activity to individuals, manifested both in the beta bands, and previously studied slow potential distributions [2]. Visual inspection of both scalp potentials and distribution of cortical currents showed minor changes in attention-related bands with respect to modality, as compared to the theta and delta bands, known to be major contributors to the sensory-related potentials. Quantitative results agreed with visual inspection, supporting to the conclusion that attention-related activity does not change much between modalities, and whatever individual changes do occur, they are not systematic in cortical localization across subjects. We discuss our results, combined with results from other studies that present individual data, with respect to the function of cortical association areas.     

Focus on the research utility of intravascular ultrasound - comparison with other invasive modalities

Intravascular ultrasound (IVUS) is an invasive modality which provides cross-sectional images of a coronary artery. In these images both the lumen and outer vessel wall can be identified and accurate estimations of their dimensions and of the plaque burden can be obtained. In addition, further processing of the IVUS backscatter signal helps in the characterization of the type of the plaque and thus it has been used to study the natural history of the atherosclerotic evolution. On the other hand its indigenous limitations do not allow IVUS to assess accurately stent struts coverage, existence of thrombus or exact site of plaque rupture and to identify some of the features associated with increased plaque vulnerability. In order this information to be obtained, other modalities such as optical coherence tomography, angioscopy, near infrared spectroscopy and intravascular magnetic resonance imaging have either been utilized or are under evaluation. The aim of this review article is to present the current utilities of IVUS in research and to discuss its advantages and disadvantages over the other imaging techniques.