Multi-view features fusion for birdsong classification

As important members of the ecosystem, birds are good monitors of the ecological environment. Bird recognition, especially birdsong recognition, has attracted more and more attention in the field of artificial intelligence. At present, traditional machine learning and deep learning are widely used in birdsong recognition. Deep learning can not only classify and recognize the spectrums of birdsong, but also be used as a feature extractor. Machine learning is often used to classify and recognize the extracted birdsong handcrafted feature parameters. As the data samples of the classifier, the feature of birdsong directly determines the performance of the classifier. Multi-view features from different methods of feature extraction can obtain more perfect information of birdsong. Therefore, aiming at enriching the representational capacity of single feature and getting a better way to combine features, this paper proposes a birdsong classification model based multi-view features, which combines the deep features extracted by convolutional neural network (CNN) and handcrafted features. Firstly, four kinds of handcrafted features are extracted. Those are wavelet transform (WT) spectrum, Hilbert-Huang transform (HHT) spectrum, short-time Fourier transform (STFT) spectrum and Mel-frequency cepstral coefficients (MFCC). Then CNN is used to extract the deep features from WT, HHT and STFT spectrum, and the minimal-redundancy-maximal-relevance (mRMR) to select optimal features. Finally, three classification models (random forest, support vector machine and multi-layer perceptron) are built with the deep features and handcrafted features, and the probability of classification results of the two types of features are fused as the new features to recognize birdsong. Taking sixteen species of birds as research objects, the experimental results show that the three classifiers obtain the accuracy of 95.49%, 96.25% and 96.16% respectively for the features of the proposed method, which are better than the seven single features and three fused features involved in the experiment. This proposed method effectively combines the deep features and handcrafted features from the perspectives of signal. The fused features can more comprehensively express the information of the bird audio itself, and have higher classification accuracy and lower dimension, which can effectively improve the performance of bird audio classification.     

Enhancing oscillations in intracranial electrophysiological recordings with data-driven spatial filters

In invasive electrophysiological recordings, a variety of neural oscillations can be detected across the cortex, with overlap in space and time. This overlap complicates measurement of neural oscillations using standard referencing schemes, like common average or bipolar referencing. Here, we illustrate the effects of spatial mixing on measuring neural oscillations in invasive electrophysiological recordings and demonstrate the benefits of using data-driven referencing schemes in order to improve measurement of neural oscillations. We discuss referencing as the application of a spatial filter. Spatio-spectral decomposition is used to estimate data-driven spatial filters, a computationally fast method which specifically enhances signal-to-noise ratio for oscillations in a frequency band of interest. We show that application of these data-driven spatial filters has benefits for data exploration, investigation of temporal dynamics and assessment of peak frequencies of neural oscillations. We demonstrate multiple use cases, exploring between-participant variability in presence of oscillations, spatial spread and waveform shape of different rhythms as well as narrowband noise removal with the aid of spatial filters. We find high between-participant variability in the presence of neural oscillations, a large variation in spatial spread of individual rhythms and many non-sinusoidal rhythms across the cortex. Improved measurement of cortical rhythms will yield better conditions for establishing links between cortical activity and behavior, as well as bridging scales between the invasive intracranial measurements and noninvasive macroscale scalp measurements.     

Motor control of birdsong

One of the challenges when considering the motor control of birdsong is to understand how such a wide variety of temporally and spectrally diverse vocalizations are learned and produced. A better understanding of central neural processing, together with direct endoscopic observations and physiological studies of peripheral motor function during singing, has resulted in the formation of new theoretical models of song production. Recent work suggests that it may be more profitable to focus on the temporal relationship between control parameters than to attempt to directly correlate neural processing with details of the acoustic output.     

Automated classification of bat echolocation call recordings with artificial intelligence

Acoustic recorders are commonly used to remotely monitor and collect data on bats (Order Chiroptera). These efforts result in many acoustic recordings that must be classified by a bat biologist with expertise in call classification in order to obtain useful information. The rarity of this expertise and time constraints have prompted efforts to automatically classify bat species in acoustic recordings using a variety of learning methods. There are several software programs available for this purpose, but they are imperfect and the United States Fish and Wildlife Service often recommends that a qualified acoustic analyst review bat call identifications even if using these software programs. We sought to build a model to classify bat species using modern computer vision techniques. We used images of bat echolocation calls (i.e., plots of the pulses) to train deep learning computer vision models that automatically classify bat calls to species. Our model classifies 10 species, five of which are protected under the Endangered Species Act. We evaluated our models using standard model validation procedures, and performed two external tests. For these tests, an entire dataset was withheld from the procedure before splitting the data into training and validation sets. We found that our validation accuracy (92%) and testing accuracy (90%) were higher than when we used Kaleidoscope Pro and BCID software (65% and 61% accuracy, respectively) to evaluate the same calls. Our results suggest that our approach is effective at classifying bat species from acoustic recordings, and our trained model will be incorporated into new bat call identification software: WEST-EchoVision.     

The neural basis of birdsong

Songbirds represent an excellent model system for understanding the neural mechanisms underlying learning.     

Neural mechanisms of birdsong memory

The process through which young male songbirds learn the characteristics of the songs of an adult male of their own species has strong similarities with speech acquisition in human infants. Both involve two phases: a period of auditory memorization followed by a period during which the individual develops its own vocalizations. The avian 'song system', a network of brain nuclei, is the probable neural substrate for the second phase of sensorimotor learning. By contrast, the neural representation of song memory acquired in the first phase is localized outside the song system, in different regions of the avian equivalent of the human auditory association cortex.     

Peripheral control and lateralization of birdsong

Recent studies on several species of oscine songbirds show that they achieve their varied vocal performances through coordinated activity of respiratory, syringeal, and other vocal tract muscles in ways that take maximum advantage of the acoustic flexibility made possible by the presence of two independently controlled sound sources in their bipartite syrinx (vocal organ). During song, special motor programs to respiratory muscles alter the pattern of ventilation to maintain the supply of respiratory air and oxygen to permit songs of long duration, high syllable repetition rates, or maximum spectral complexity. Each side of the syrinx receives its own motor program that, together with that sent to respiratory muscles, determines the acoustic properties of the ipsilaterally produced sound. The acoustic expression of these bilaterally distinct, phonetic motor patterns depends on the action of dorsal syringeal adductor muscles that, by opening or closing the ipsilateral side of the syrinx to airflow, determine the amount each side contributes to song. The syringeally generated sound is further modified by muscles that control the shape of the vocal tract. Different species have adopted different motor strategies that use the left and right sides of the syrinx in patterns of unilateral, bilateral, alternating, or sequential phonation to achieve the differing temporal and spectral characteristics of their songs. As a result, the degree of song lateralization probably varies between species to form a continuum from unilateral dominance to bilateral equality. © 1997 John Wiley & Sons, Inc. J Neurobiol 33: 632–652, 1997     

The evolution of birdsong on islands

Islands are simplified, isolated ecosystems, providing an ideal set‐up to study evolution. Among several traits that are expected to change on islands, an interesting but poorly understood example concerns signals used in animal communication. Islands are typified by reduced species diversity, increased population density, and reduced mate competition, all of which could affect communication signals. We used birdsong to investigate whether there are systematic changes in communication signals on islands, by undertaking a broad comparison based on pairs of closely related island‐mainland species across the globe. We studied song traits related to complexity (number of different syllables, frequency bandwidth), to vocal performance (syllable delivery rate, song duration), and also three particular song elements (rattles, buzzes, and trills) generally implicated in aggressive communication. We also investigated whether song complexity was related to the number of similar sympatric species. We found that island species were less likely to produce broadband and likely aggressive song elements (rattles and buzzes). By contrast, various aspects of song complexity and performance did not differ between island and mainland species. Species with fewer same‐family sympatric species used wider frequency bandwidths, as predicted by the character release hypothesis, both on continents and on islands. Our study supports the hypothesis of a reduction in aggressive behavior on islands and suggests that discrimination against closely related species is an important factor influencing birdsong evolution.     

The neuromuscular control of birdsong.

Birdsong requires complex learned motor skills involving the coordination of respiratory, vocal organ and craniomandibular muscle groups. Recent studies have added to our understanding of how these vocal subsystems function and interact during song production. The respiratory rhythm determines the temporal pattern of song. Sound is produced during expiration and each syllable is typically followed by a small inspiration, except at the highest syllable repetition rates when a pattern of pulsatile expiration is used. Both expiration and inspiration are active processes. The oscine vocal organ, the syrinx, contains two separate sound sources at the cranial end of each bronchus, each with independent motor control. Dorsal syringeal muscles regulate the timing of phonation by adducting the sound-generating labia into the air stream. Ventral syringeal muscles have an important role in determining the fundamental frequency of the sound. Different species use the two sides of their vocal organ in different ways to achieve the particular acoustic properties of their song. Reversible paralysis of the vocal organ during song learning in young birds reveals that motor practice is particularly important in late plastic song around the time of song crystallization in order for normal adult song to develop. Even in adult crystallized song, expiratory muscles use sensory feedback to make compensatory adjustments to perturbations of respiratory pressure. The stereotyped beak movements that accompany song appear to have a role in suppressing harmonics, particularly at low frequencies.     

Sex steroids and their actions on the birdsong system

It is probably not surprising to most of us that the endocrine system plays a significant role in controlling the singing behavior of birds. We are familiar with the song of birds as a conspicuous acoustic feature of our environment during the avian breeding season. We often witness song when it is produced by birds (males) that are aggressively establishing and defending territories and that are advertising to available females. Thus, it is easy to imagine that song is likely to be stimulated by gonadal hormones. However, the ways in which gonadal sex steroids influence the various parts of the brain at various stages of the bird's life to influence song are complex and far from being completely understood. In this review, I will highlight some of the significant discoveries that have contributed to our view that the songbird brain is a significant and dynamic target of sex steroids. I will also describe what we have learned about properties of the endocrine system and the brain and how they each contribute to making androgens or estrogens available to particular parts of the songbird brain. Finally, I will describe some new research directions that may help answer some unresolved issues about hormonal effects on the songbird brain. © 1997 John Wiley & Sons, Inc. J Neurobiol 33: 619–631, 1997     

Identifying COVID-19 by using spectral analysis of cough recordings: a distinctive classification study

Sound signals from the respiratory system are largely taken as tokens of human health. Early diagnosis of respiratory tract diseases is of great importance because, if delayed, it exerts irreversible effects on human health. The Coronavirus pandemic, which is deeply shaking the world, has revealed the importance of this diagnosis even more. During the pandemic, it has become the focus of researchers to differentiate symptoms from similar diseases such as influenza. Among these symptoms, the difference in cough sound played a distinctive role in research. Clinical data collected under the supervision of doctors in a reliable environment were used as the dataset consisting of 16 subjects suspected of COVID-19 with a specific patient demographic. Using the polymerase chain reaction test, the suspected subjects were divided into two groups as negative and positive. The negative and positive labels represent the patients with non-COVID and with a COVID-19 cough, respectively. Using the 3D plot or waterfall representation of the signal frequency spectrum, the salient features of the cough data are revealed. In this way, COVID-19 can be differentiated from other coughs by applying effective feature extraction and classification techniques. Power spectral density based on short-time Fourier transform and mel-frequency cepstral coefficients (MFCC) were chosen as the efficient feature extraction method. From among the classification techniques, the support vector machine (SVM) algorithm was applied to the processed signals in order to identify and classify COVID-19 cough. In terms of results evaluation, the cough of subjects with COVID-19 was detected with 95.86% classification accuracy thanks to the radial basis function (RBF) kernel function of SVM and the MFCC method. The diagnosis of COVID-19 coughs was performed with 98.6% and 91.7% sensitivity and specificity, respectively.     

Spike timing and synaptic plasticity in the premotor pathway of birdsong

The neural circuits of birdsong appear to utilize specific time delays in their operation. In particular, the anterior forebrain pathway (AFP) is implicated in an approximately 40- to 50- ms time delay, T, playing a role in the relative timing of premotor signals from the nucleus HVc to the nucleus robust nucleus of the archistratium (RA) and control/learning signals from the nucleus lateral magnocellular nucleus of the anterior neostratium (lMAN) to RA. Using a biophysical model of synaptic plasticity based on experiments on mammalian hippocampal and neocortical pyramidal neurons, we propose an understanding of this 40- to 50- ms delay. The biophysical model describes the influence of Ca²⁺ influx into the postsynaptic RA cells through NMDA and AMPA receptors and the induction of LTP and LTD through complex metabolic pathways. The delay, T, between HVc RA premotor signals and lMAN RA control/learning signals plays an essential role in determining if synaptic plasticity is induced by signaling from each pathway into RA. If T is substantially larger than 40 ms, no plasticity is induced. If T is much less than 40 ms, only potentiation is expected. If T 40 ms, the sign of synaptic plasticity is sensitive to T. Our results suggest that changes in T may influence learning and maintenance of birdsong. We investigate the robustness of this result to noise and to the removal of the Ca²⁺ contribution from lMAN RA NMDA receptors.     

Behavioral discrimination and time-series phenotyping of birdsong performance

Variation in the acoustic structure of vocal signals is important to communicate social information. However, relatively little is known about the features that receivers extract to decipher relevant social information. Here, we took an expansive, bottom-up approach to delineate the feature space that could be important for processing social information in zebra finch song. Using operant techniques, we discovered that female zebra finches can consistently discriminate brief song phrases (“motifs”) from different social contexts. We then applied machine learning algorithms to classify motifs based on thousands of time-series features and to uncover acoustic features for motif discrimination. In addition to highlighting classic acoustic features, the resulting algorithm revealed novel features for song discrimination, for example, measures of time irreversibility (i.e., the degree to which the statistical properties of the actual and time-reversed signal differ). Moreover, the algorithm accurately predicted female performance on individual motif exemplars. These data underscore and expand the promise of broad time-series phenotyping to acoustic analyses and social decision-making.     

Phylogeny and mechanisms of shared hierarchical patterns in birdsong

1. Download : Download high-res image (237KB)
2. Download : Download full-size image

     

Rapamycin blocks the neuroprotective effects of sex steroids in the adult birdsong system

In adult songbirds, the telencephalic song nucleus HVC and its efferent target RA undergo pronounced seasonal changes in morphology. In breeding birds, there are increases in HVC volume and total neuron number, and RA neuronal soma area compared to nonbreeding birds. At the end of breeding, HVC neurons die through caspase‐dependent apoptosis and thus, RA neuron size decreases. Changes in HVC and RA are driven by seasonal changes in circulating testosterone (T) levels. Infusing T, or its metabolites 5α‐dihydrotestosterone (DHT) and 17 β‐estradiol (E2), intracerebrally into HVC (but not RA) protects HVC neurons from death, and RA neuron size, in nonbreeding birds. The phosphoinositide 3‐kinase (PI3K)‐Akt (a serine/threonine kinase)‐mechanistic target of rapamycin (mTOR) signaling pathway is a point of convergence for neuroprotective effects of sex steroids and other trophic factors. We asked if mTOR activation is necessary for the protective effect of hormones in HVC and RA of adult male Gambel's white‐crowned sparrows (Zonotrichia leucophrys gambelii). We transferred sparrows from breeding to nonbreeding hormonal and photoperiod conditions to induce regression of HVC neurons by cell death and decrease of RA neuron size. We infused either DHT + E2, DHT + E2 plus the mTOR inhibitor rapamycin, or vehicle alone in HVC. Infusion of DHT + E2 protected both HVC and RA neurons. Coinfusion of rapamycin with DHT + E2, however, blocked the protective effect of hormones on HVC volume and neuron number, and RA neuron size. These results suggest that activation of mTOR is an essential downstream step in the neuroprotective cascade initiated by sex steroid hormones in the forebrain.     

Enhancing navigation in biomedical databases by community voting and database-driven text classification

Background  

The breadth of biological databases and their information content continues to increase exponentially. Unfortunately, our ability to query such sources is still often suboptimal. Here, we introduce and apply community voting, database-driven text classification, and visual aids as a means to incorporate distributed expert knowledge, to automatically classify database entries and to efficiently retrieve them.