Unsupervised classification to improve the quality of a bird song recording dataset

Open audio databases such as Xeno-Canto are widely used to build datasets to explore bird song repertoire or to train models for automatic bird sound classification by deep learning algorithms. However, such databases suffer from the fact that bird sounds are weakly labelled: a species name is attributed to each audio recording without timestamps that provide the temporal localization of the bird song of interest. Manual annotations can solve this issue, but they are time consuming, expert-dependent, and cannot run on large datasets. Another solution consists in using a labelling function that automatically segments audio recordings before assigning a label to each segmented audio sample. Although labelling functions were introduced to expedite strong label assignment, their classification performance remains mostly unknown. To address this issue and reduce label noise (wrong label assignment) in large bird song datasets, we introduce a data-centric novel labelling function composed of three successive steps: 1) time-frequency sound unit segmentation, 2) feature computation for each sound unit, and 3) classification of each sound unit as bird song or noise with either an unsupervised DBSCAN algorithm or the supervised BirdNET neural network. The labelling function was optimized, validated, and tested on the songs of 44 West-Palearctic common bird species. We first showed that the segmentation of bird songs alone aggregated from 10% to 83% of label noise depending on the species. We also demonstrated that our labelling function was able to significantly reduce the initial label noise present in the dataset by up to a factor of three. Finally, we discuss different opportunities to design suitable labelling functions to build high-quality animal vocalizations with minimum expert annotation effort.     

Classifying and ranking audio clips to support bird species richness surveys

Advances in programmable field acoustic sensors provide immense data for bird species study. Manually searching for bird species present in these acoustic data is time-consuming. Although automated techniques have been used for species recognition in many studies, currently these techniques are prone to error due to the complexity of natural acoustics.In this paper we propose a smart sampling approach to help identify the maximum number of bird species while listening to the minimum amount of acoustic data. This approach samples audio clips in a manner that can direct bird species surveys more efficiently. First, a classifier is built to remove audio clips that are unlikely to contain birds; second, the remaining audio clips are ranked by a proxy for the number of species. This technique enables a more efficient determination of species richness.The experimental results show that the use of a classifier enables to remove redundant acoustic data and make our approach resilient to various weather conditions. By ranking audio clips classified as “Birds”, our method outperforms the currently best published strategy for finding bird species after 30 one-minute audio clip samples. Particularly after 60 samples, our method achieves 10 percentage points more species. Despite our focus on bird species, the proposed sampling approach is applicable to the search of other vocal species.     

Western Mediterranean Wetland Birds dataset: A new annotated dataset for acoustic bird species classification

The deployment of an expert system running over a wireless acoustic sensors network made up of bioacoustic monitoring devices that recognize bird species from their sounds would enable the automation of many tasks of ecological value, including the analysis of bird population composition or the detection of endangered species in areas of environmental interest. Endowing these devices with accurate audio classification capabilities is possible thanks to the latest advances in artificial intelligence, among which deep learning techniques stand out. To train such algorithms, data from the sources to be classified is required. For this reason, this paper presents the Western Mediterranean Wetland Birds (WMWB) dataset, consisting of 201.6 min and 5795 annotated audio excerpts of 20 endemic bird species of the Aiguamolls de l'Empordà Natural Park. The main objective of this work is to describe and analyze this new dataset. Moreover, this work presents the results of bird species classification experiments using four well- known deep neural networks fine-tuned on our dataset, whose models are also made public along with the dataset. These results are aimed to serve as a performance baseline reference for the community when using the WMWB dataset for their experiments.     

Animal Sound Identifier (ASI): software for automated identification of vocal animals

Automated audio recording offers a powerful tool for acoustic monitoring schemes of bird, bat, frog and other vocal organisms, but the lack of automated species identification methods has made it difficult to fully utilise such data. We developed Animal Sound Identifier (ASI), a MATLAB software that performs probabilistic classification of species occurrences from field recordings. Unlike most previous approaches, ASI locates training data directly from the field recordings and thus avoids the need of pre‐defined reference libraries. We apply ASI to a case study on Amazonian birds, in which we classify the vocalisations of 14 species in 194 504 one‐minute audio segments using in total two weeks of expert time to construct, parameterise, and validate the classification models. We compare the classification performance of ASI (with training templates extracted automatically from field data) to that of monitoR (with training templates extracted manually from the Xeno‐Canto database), the results showing ASI to have substantially higher recall and precision rates.     

Assessing the cross platform performance of marine mammal indicators between two collocated acoustic recorders

Equipment and deployment strategies for remote passive acoustic sensing of marine environments must balance memory capacity, power requirements, sampling rate, duty-cycle, deployment duration, instrument size, and environmental concerns. The impact of different parameters on the data and applicability of the data to the specific questions being asked should be considered before deployment. Here we explore the effect of recording and detection parameters on marine mammal acoustic data across two platforms. Daily classifications of marine mammal vocalizations from two passive acoustic monitors with different subsampling parameters, an AURAL and a Passive Aquatic Listener (PAL), collocated in the Bering Sea were compared. The AURAL subsampled on a pre-set schedule, whereas the PAL sampled via an adaptive protocol. Detected signals of interest were manually classified in each dataset independently. The daily classification rates of vocalizations were similar. Detections from the higher duty-cycle but lower sample rate AURAL were limited to species and vocalizations with energy below 4 kHz precluding detection of echolocation signals. Temporal coverage from the PAL audio files was limited by the adaptive sub-sampling protocol. A method for classifying ribbon (Histriophoca fasciata) and bearded seal (Erignathus barbatus) vocalizations from the sparse spectral time histories of the PAL was developed. Although application of the acoustic entropy as a rapid assessment of biodiversity was not reflective of the number of species detected, acoustic entropy was robust to changes in sample rate and window length.     

Adaptive frequency scaled wavelet packet decomposition for frog call classification

Environmental changes have put great pressure on biological systems leading to the rapid decline of biodiversity. To monitor this change and protect biodiversity, animal vocalizations have been widely explored by the aid of deploying acoustic sensors in the field. Consequently, large volumes of acoustic data are collected. However, traditional manual methods that require ecologists to physically visit sites to collect biodiversity data are both costly and time consuming. Therefore it is essential to develop new semi-automated and automated methods to identify species in automated audio recordings. In this study, a novel feature extraction method based on wavelet packet decomposition is proposed for frog call classification. After syllable segmentation, the advertisement call of each frog syllable is represented by a spectral peak track, from which track duration, dominant frequency and oscillation rate are calculated. Then, a k-means clustering algorithm is applied to the dominant frequency, and the centroids of clustering results are used to generate the frequency scale for wavelet packet decomposition (WPD). Next, a new feature set named adaptive frequency scaled wavelet packet decomposition sub-band cepstral coefficients is extracted by performing WPD on the windowed frog calls. Furthermore, the statistics of all feature vectors over each windowed signal are calculated for producing the final feature set. Finally, two well-known classifiers, a k-nearest neighbour classifier and a support vector machine classifier, are used for classification. In our experiments, we use two different datasets from Queensland, Australia (18 frog species from commercial recordings and field recordings of 8 frog species from James Cook University recordings). The weighted classification accuracy with our proposed method is 99.5% and 97.4% for 18 frog species and 8 frog species respectively, which outperforms all other comparable methods.     

Low-cost open-source recorders and ready-to-use machine learning approaches provide effective monitoring of threatened species

Passive acoustic monitoring is a powerful tool for monitoring vocally active taxa. Automated signal recognition software reduces the expert time needed for recording analyses and allows researchers and managers to manage large acoustic datasets. The application of state-of-the-art techniques for automated identification, such as Convolutional Neural Networks, may be challenging for ecologists and managers without informatics or engineering expertise. Here, we evaluated the use of AudioMoth — a low-cost and open-source sound recorder — to monitor a threatened and patchily distributed species, the Eurasian bittern (Botaurus stellaris). Passive acoustic monitoring was carried out across 17 potential wetlands in north Spain. We also assessed the performance of BirdNET — an automated and freely available classifier able to identify over 3000 bird species — and Kaleidoscope Pro — a user-friendly recognition software — to detect the vocalizations and the presence of the target species. The percentage of presences and vocalizations of the Eurasian bittern automatically detected by BirdNET and Kaleidoscope Pro software was compared to manual annotations of 205 recordings. The species was effectively recorded up to distances of 801–900 m, with at least 50% of the vocalizations uttered within that distance being manually detected; this distance was reduced to 601–700 m when considering the analyses carried out using Kaleidoscope Pro. BirdNET detected the species in 59 of the 63 (93.7%) recordings with known presence of the species, while Kaleidoscope detected the bittern in 62 recordings (98.4%). At the vocalization level, BirdNet and Kaleidoscope Pro were able to detect between 76 and 78%, respectively, of the vocalizations detected by a human observer. Our study highlights the ability of AudioMoth for detecting the bittern at large distances, which increases the potential of that technique for monitoring the species at large spatial scales. According to our results, a single AudioMoth could be useful for monitoring the species' presence in wetlands of up to 150 ha. Our study proves the utility of passive acoustic monitoring, coupled with BirdNET or Kaleidoscope Pro, as an accurate, repeatable, and cost-efficient method for monitoring the Eurasian bittern at large spatial and temporal scales. Nonetheless, further research should evaluate the performance of BirdNET on a larger number of species, and under different recording conditions (e.g., more closed habitats), to improve our knowledge about BirdNET's ability to perform bird monitoring. Future studies should also aim to develop an adequate protocol to perform effective passive acoustic monitoring of the Eurasian bittern.     

Automated audio recording as a means of surveying tinamous (Tinamidae) in the Peruvian Amazon

1. The use of machine learning technologies to process large quantities of remotely collected audio data is a powerful emerging research tool in ecology and conservation.
2. We applied these methods to a field study of tinamou (Tinamidae) biology in Madre de Dios, Peru, a region expected to have high levels of interspecies competition and niche partitioning as a result of high tinamou alpha diversity. We used autonomous recording units to gather environmental audio over a period of several months at lowland rainforest sites in the Los Amigos Conservation Concession and developed a Convolutional Neural Network‐based data processing pipeline to detect tinamou vocalizations in the dataset.
3. The classified acoustic event data are comparable to similar metrics derived from an ongoing camera trapping survey at the same site, and it should be possible to combine the two datasets for future explorations of the target species'' niche space parameters.
4. Here, we provide an overview of the methodology used in the data collection and processing pipeline, offer general suggestions for processing large amounts of environmental audio data, and demonstrate how data collected in this manner can be used to answer questions about bird biology.

     

Automatic classification of electrocardiogram signals based on transfer learning and continuous wavelet transform

Classification and subsequent diagnosis of cardiac arrhythmias is an important research topic in clinical practice. Confirmation of the type of arrhythmia at an early stage is critical for reducing the risk and occurrence of cardiovascular events. Nevertheless, diagnoses must be confirmed by a combination of specialist experience and electrocardiogram (ECG) examination, which can lead to delays in diagnosis. To overcome such obstacles, this study proposes an automatic ECG classification algorithm based on transfer learning and continuous wavelet transform (CWT). The transfer learning method is able to transfer the domain knowledge and features of images to a EGG, which is a one-dimensional signal when a convolutional neural network (CNN) is used for classification. Meanwhile, CWT is used to convert a one-dimensional ECG signal into a two-dimensional signal map consisting of time-frequency components. Considering that morphological features can be helpful in arrhythmia classification, eight features related to the R peak of an ECG signal are proposed. These auxiliary features are integrated with the features extracted by the CNN and then fed into the fully linked arrhythmia classification layer. The CNN developed in this study can also be used for bird activity detection. The classification experiments were performed after converting the two types of audio files containing songbird sounds and those without songbird sounds from the NIPS4Bplus bird song dataset into the Mel spectrum. Compared to the most recent methods in the same field, the classification results improved accuracy and recognition by 11.67% and 11.57%, respectively.     

A first test of unattended,acoustic recorders for monitoring Capercaillie Tetrao urogallus lekking activity

Capsule: Automated acoustic recording can be used as a valuable survey technique for Capercaillie Tetrao urogallus leks, improving the quality and quantity of field data for this endangered bird species. However, more development work and testing against traditional methods are needed to establish optimal working practices.

Aims: This study aims to determine whether Capercaillie vocalizations can be recognized in lek recordings, whether this can be automated using readily available software, and whether the number of calls resulting varies with location, weather conditions, date and time of day.

Methods: Unattended recording devices and semi-automated call classification software were used to record and analyse the display calls of Capercaillie at three known lek sites in Scotland over a two-week period.

Results: Capercaillie calls were successfully and rapidly identified within a data set that included the vocalizations of other bird species and environmental noise. Calls could be readily recognized to species level using a combination of unsupervised software and manual analysis. The number of calls varied by time and date, by recorder/microphone location at the lek site, and with weather conditions. This information can be used to better target future acoustic monitoring and improve the quality of existing traditional lek surveys.

Conclusion: Bioacoustic methods provide a practical and cost-effective way to determine habitat occupancy and activity levels by a vocally distinctive bird species. Following further testing alongside traditional counting methods, it could offer a significant new approach towards more effective monitoring of local population levels for Capercaillie and other species of conservation concern.     

Acoustic localization of terrestrial wildlife: Current practices and future opportunities

Autonomous acoustic recorders are an increasingly popular method for low‐disturbance, large‐scale monitoring of sound‐producing animals, such as birds, anurans, bats, and other mammals. A specialized use of autonomous recording units (ARUs) is acoustic localization, in which a vocalizing animal is located spatially, usually by quantifying the time delay of arrival of its sound at an array of time‐synchronized microphones. To describe trends in the literature, identify considerations for field biologists who wish to use these systems, and suggest advancements that will improve the field of acoustic localization, we comprehensively review published applications of wildlife localization in terrestrial environments. We describe the wide variety of methods used to complete the five steps of acoustic localization: (1) define the research question, (2) obtain or build a time‐synchronizing microphone array, (3) deploy the array to record sounds in the field, (4) process recordings captured in the field, and (5) determine animal location using position estimation algorithms. We find eight general purposes in ecology and animal behavior for localization systems: assessing individual animals' positions or movements, localizing multiple individuals simultaneously to study their interactions, determining animals' individual identities, quantifying sound amplitude or directionality, selecting subsets of sounds for further acoustic analysis, calculating species abundance, inferring territory boundaries or habitat use, and separating animal sounds from background noise to improve species classification. We find that the labor‐intensive steps of processing recordings and estimating animal positions have not yet been automated. In the near future, we expect that increased availability of recording hardware, development of automated and open‐source localization software, and improvement of automated sound classification algorithms will broaden the use of acoustic localization. With these three advances, ecologists will be better able to embrace acoustic localization, enabling low‐disturbance, large‐scale collection of animal position data.     

Automated classification of bird and amphibian calls using machine learning: A comparison of methods

We compared the ability of three machine learning algorithms (linear discriminant analysis, decision tree, and support vector machines) to automate the classification of calls of nine frogs and three bird species. In addition, we tested two ways of characterizing each call to train/test the system. Calls were characterized with four standard call variables (minimum and maximum frequencies, call duration and maximum power) or eleven variables that included three standard call variables (minimum and maximum frequencies, call duration) and a coarse representation of call structure (frequency of maximum power in eight segments of the call). A total of 10,061 isolated calls were used to train/test the system. The average true positive rates for the three methods were: 94.95% for support vector machine (0.94% average false positive rate), 89.20% for decision tree (1.25% average false positive rate) and 71.45% for linear discriminant analysis (1.98% average false positive rate). There was no statistical difference in classification accuracy based on 4 or 11 call variables, but this efficient data reduction technique in conjunction with the high classification accuracy of the SVM is a promising combination for automated species identification by sound. By combining automated digital recording systems with our automated classification technique, we can greatly increase the temporal and spatial coverage of biodiversity data collection.     

Time-series forecasting offers novel quantitative measure to assess loud sound event in an urban park with restored prairie

Soundscape ecology and ecoacoustics study the spatiotemporal dynamics of a soundscape and how the dynamics reflect and influence ecological processes in the environment. Soundscape analysis methods employ acoustic recording units (ARUs) that collect acoustic data in study areas over time. Analyzing these data includes computation of several acoustic diversity indices developed to quantify species abundance, richness, or habitat condition through digital audio processing and algorithm adaptations for within-group populations. However, the success of specific indices is often dependent on habitat type and biota richness present and analyzing these data can be challenging due to temporal pseudo-replication. Time-series analytical methods address the inherent problems of temporal autocorrelation for soundscape analyses challenges. Animal population dynamics fluctuate in a variety of ways due to changes in habitat or natural patterns of a landscape and chronic noise exposure, with soundscape phenology patterns evident in terrestrial and aquatic environments. Historical phenological soundscape patterns have been used to predict expected soundscape patterns in long-term studies but limited work has explored how forecasting can quantify changes in short-term studies. We evaluate how forecasting from an acoustic index can be used to quantify change in an acoustic community response to a loud, acute noise. We found that the acoustic community of a Midwestern restored prairie had decreased acoustic community activity after a loud sound event (LSE), a Civil War Reenactment, mainly driven by observed changes in the bird community and quantified using two methods: an automated acoustic index and species richness. Time-series forecasting maybe considered an underutilized tool in analyzing acoustic data whose experimental design can be flawed with temporal autocorrelation. Forecasting using auto ARIMA with acoustic indices could benefit decision makers who consider ecological questions at different time scales.     

A comparison of supervised learning techniques in the classification of bat echolocation calls

Today's acoustic monitoring devices are capable of recording and storing tremendous amounts of data. Until recently, the classification of animal vocalizations from field recordings has been relegated to qualitative approaches. For large-scale acoustic monitoring studies, qualitative approaches are very time-consuming and suffer from the bias of subjectivity. Recent developments in supervised learning techniques can provide rapid, accurate, species-level classification of bioacoustics data. We compared the classification performances of four supervised learning techniques (random forests, support vector machines, artificial neural networks, and discriminant function analysis) for five different classification tasks using bat echolocation calls recorded by a popular frequency-division bat detector. We found that all classifiers performed similarly in terms of overall accuracy with the exception of discriminant function analysis, which had the lowest average performance metrics. Random forests had the advantage of high sensitivities, specificities, and predictive powers across the majority of classification tasks, and also provided metrics for determining the relative importance of call features in distinguishing between groups. Overall classification accuracy for each task was slightly lower than reported accuracies using calls recorded by time-expansion detectors. Myotis spp. were particularly difficult to separate; classifiers performed best when members of this genus were combined in genus-level classification and analyzed separately at the level of species. Additionally, we identified and ranked the relative contributions of all predictor features to classifier accuracy and found measurements of frequency, total call duration, and characteristic slope to be the most important contributors to classification success. We provide recommendations to maximize accuracy and efficiency when analyzing acoustic data, and suggest an application of automated bioacoustics monitoring to contribute to wildlife monitoring efforts.     

An automated multispecies bioacoustics sound classification method based on a nonlinear pattern: Twine-pat

Categorizing the bioacoustic and ecoacoustic properties of animals is great interest to biologists and ecologists. Also, multidisciplinary studies in engineering have significantly contributed to the development of acoustic analysis. Observing the animals living in the ecological environment provides information in many areas such as global warming, climate changes, monitoring of endangered animals, agricultural activities. However, the classification of bioacoustics sounds by manually is very hard. Therefore, automated bioacoustics sound classification is crucial for ecological science. This work presents a new multispecies bioacoustics sound dataset and novel machine learning model to classify bird and anuran species with sounds automatically. In this model, a new nonlinear textural feature generation function is presented by using twine cipher substitution box(S-box), and this feature generation function is named twine-pat. By using twine-pat and tunable Q-factor wavelet transform, a multilevel feature generation network is presented. Iterative ReliefF(IRF) is employed to select the most effective/valuable features. Two shallow classifiers are used to calculate results. Our presented model reached 98.75% accuracy by using k-nearest neighbor(kNN) classifier. The results obviously demonstrated the success of the presented model.     

The effect of soundscape composition on bird vocalization classification in a citizen science biodiversity monitoring project

There is a need for monitoring biodiversity at multiple spatial and temporal scales to aid conservation efforts. Autonomous recording units (ARUs) can provide cost-effective, long-term and systematic species monitoring data for sound-producing wildlife, including birds, amphibians, insects and mammals over large areas. Modern deep learning can efficiently automate the detection of species occurrences in these sound data with high accuracy. Further, citizen science can be leveraged to scale up the deployment of ARUs and collect reference vocalizations needed for training and validating deep learning models. In this study we develop a convolutional neural network (CNN) acoustic classification pipeline for detecting 54 bird species in Sonoma County, California USA, with sound and reference vocalization data collected by citizen scientists within the Soundscapes to Landscapes project (www.soundscapes2landscapes.org). We trained three ImageNet-based CNN architectures (MobileNetv2, ResNet50v2, ResNet100v2), which function as a Mixture of Experts (MoE), to evaluate the usefulness of several methods to enhance model accuracy. Specifically, we: 1) quantify accuracy with fully-labeled 1-min soundscapes for an assessment of real-world conditions; 2) assess the effect on precision and recall of additional pre-training with an external sound archive (xeno-canto) prior to fine-tuning with vocalization data from our study domain; and, 3) assess how detections and errors are influenced by the presence of coincident biotic and non-biotic sounds (i.e., soundscape components). In evaluating accuracy with soundscape data (n = 37 species) across CNN probability thresholds and models, we found acoustic pre-training followed by fine-tuning improved average precision by 10.3% relative to no pre-training, although there was a small average 0.8% reduction in recall. In selecting an optimal CNN architecture for each species based on maximum F(β = 0.5), we found our MoE approach had total precision of 84.5% and average species precision of 85.1%. Our data exhibit multiple issues arising from applying citizen science and acoustic monitoring at the county scale, including deployment of ARUs with relatively low fidelity and recordings with background noise and overlapping vocalizations. In particular, human noise was significantly associated with more incorrect species detections (false positives, decreased precision), while physical interference (e.g., recorder hit by a branch) and geophony (e.g., wind) was associated with the classifier missing detections (false negatives, decreased recall). Our process surmounted these obstacles, and our final predictions allowed us to demonstrate how deep learning applied to acoustic data from low-cost ARUs paired with citizen science can provide valuable bird diversity data for monitoring and conservation efforts.     

Literature based species occurrence data of birds of northeast India

The northeast region of India is one of the world's most significant biodiversity hotspots. One of the richest bird areas in India, it is an important route for migratory birds and home to many endemic bird species. This paper describes a literature-based dataset of species occurrences of birds of northeast India. The occurrence records documented in the dataset are distributed across eleven states of India, viz.: Arunachal Pradesh, Assam, Bihar, Manipur, Meghalaya, Mizoram, Nagaland, Sikkim, Tripura, Uttar Pradesh and West Bengal. The geospatial scope of the dataset represents 24 to 29 degree North latitude and 78 to 94 degree East longitude, and it comprises over 2400 occurrence records. These records have been collated from scholarly literature published between1915 and 2008, especially from the Journal of the Bombay Natural History Society (JBNHS). The temporal scale of the dataset represents bird observations recorded between 1909 and 2007. The dataset has been developed by employing MS Excel. The key elements in the database are scientific name, taxonomic classification, temporal and geospatial details including geo-coordinate precision, data collector, basis of record and primary source of the data record. The temporal and geospatial quality of more than 50% of the data records has been enhanced retrospectively. Where possible, data records are annotated with geospatial coordinate precision to the nearest minute. This dataset is being constantly updated with the addition of new data records, and quality enhancement of documented occurrences. The dataset can be used in species distribution and niche modeling studies. It is planned to expand the scope of the dataset to collate bird species occurrences across the Indian peninsula.     

Applying bioacoustic methods for long-term monitoring of a nocturnal wetland bird

Bioacoustic monitoring is becoming more and more popular as a non-invasive method to study populations and communities of vocalizing animals. Acoustic pattern recognition techniques allow for automated identification of species and an estimation of species composition within ecosystems. Here we describe an approach where on the basis of long term acoustic recordings not only the occurrence of a species was documented, but where the number of vocalizing animals was also estimated. This approach allows us to follow up changes in population density and to define breeding sites in a changing environment. We present the results of five years of continuous acoustic monitoring of Eurasian bittern (Botaurus stellaris) in a recent wetland restoration area. Using a setup consisting of four four-channel recorders equipped with cardioid microphones we recorded vocal activity during entire nights. Vocalizations of bitterns were detected on the recordings by spectrogram template matching. On basis of time differences of arrival (TDOA) of the acoustic signals at different recording devices booming bitterns could be mapped using hyperbolic localization. During the study period not only changes in the number of calling birds but also changes in their spatial distribution connected with changes in habitat structure could be documented. This semi-automated approach towards monitoring birds described here could be applied to a wide range of monitoring tasks for animals with long distance vocalizations.