Method for passive acoustic monitoring of bird communities using UMAP and a deep neural network

An effective practice for monitoring bird communities is the recognition and identification of their acoustic signals, whether simple, complex, fixed or variable. A method for the passive monitoring of diversity, activity and acoustic phenology of structural species of a bird community in an annual cycle is presented. The method includes the semi-automatic elaboration of a dataset of 22 vocal and instrumental forms of 16 species. To analyze bioacoustic richness, the UMAP algorithm was run on two parallel feature extraction channels. A convolutional neural network was trained using STFT-Mel spectrograms to perform the task of automatic identification of bird species. The predictive performance was evaluated by obtaining a minimum average precision of 0.79, a maximum equal to 1.0 and a mAP equal to 0.97. The model was applied to a huge set of passive recordings made in a network of urban wetlands for one year. The acoustic activity results were synchronized with climatological temperature data and sunlight hours. The results confirm that the proposed method allows for monitoring a taxonomically diverse group of birds that nourish the annual soundscape of an ecosystem, as well as detecting the presence of cryptic species that often go unnoticed.     

Who shall I say is calling? Validation of a caller recognition procedure in Bornean flanged male orangutan (Pongo pygmaeus wurmbii) long calls

Acoustic individual discrimination has been demonstrated for a wide range of animal taxa. However, there has been far less scientific effort to demonstrate the effectiveness of automatic individual identification, which could greatly facilitate research, especially when data are collected via an acoustic localization system (ALS). In this study, we examine the accuracy of acoustic caller recognition in long calls (LCs) emitted by Bornean male orangutans (Pongo pygmaeus wurmbii) derived from two data-sets: the first consists of high-quality recordings taken during individual focal follows (N = 224 LCs by 14 males) and the second consists of LC recordings with variable microphone-caller distances stemming from ALS (N = 123 LCs by 10 males). The LC is a long-distance vocalization. We therefore expect that even the low-quality test-set should yield caller recognition results significantly better than by chance. Automatic individual identification was accomplished using software originally developed for human speaker recognition (i.e. the MSR identity toolbox). We obtained a 93.3% correct identification rate with high-quality recordings, and 72.23% with recordings stemming from the ALS with variable microphone-caller distances (20–420 m). These results show that automatic individual identification is possible even though the accuracy declines compared with the results of high-quality recordings due to severe signal degradations (e.g. sound attenuation, environmental noise contamination, and echo interference) with increasing distance. We therefore suggest that acoustic individual identification with speaker recognition software can be a valuable tool to apply to data obtained through an ALS, thereby facilitating field research on vocal communication.     

Identification of Behaviour in Freely Moving Dogs (Canis familiaris) Using Inertial Sensors

Monitoring and describing the physical movements and body postures of animals is one of the most fundamental tasks of ethology. The more precise the observations are the more sophisticated the interpretations can be about the biology of a certain individual or species. Animal-borne data loggers have recently contributed much to the collection of motion-data from individuals, however, the problem of translating these measurements to distinct behavioural categories to create an ethogram is not overcome yet. The objective of the present study was to develop a “behaviour tracker”: a system composed of a multiple sensor data-logger device (with a tri-axial accelerometer and a tri-axial gyroscope) and a supervised learning algorithm as means of automated identification of the behaviour of freely moving dogs. We collected parallel sensor measurements and video recordings of each of our subjects (Belgian Malinois, N=12; Labrador Retrievers, N=12) that were guided through a predetermined series of standard activities. Seven behavioural categories (lay, sit, stand, walk, trot, gallop, canter) were pre-defined and each video recording was tagged accordingly. Evaluation of the measurements was performed by support vector machine (SVM) classification. During the analysis we used different combinations of independent measurements for training and validation (belonging to the same or different individuals or using different training data size) to determine the robustness of the application. We reached an overall accuracy of above 90% perfect identification of all the defined seven categories of behaviour when both training and validation data belonged to the same individual, and over 80% perfect recognition rate using a generalized training data set of multiple subjects. Our results indicate that the present method provides a good model for an easily applicable, fast, automatic behaviour classification system that can be trained with arbitrary motion patterns and potentially be applied to a wide range of species and situations.     

Genetic fingerprinting proves cross‐correlated automatic photo‐identification of individuals as highly efficient in large capture–mark–recapture studies

Capture–mark–recapture (CMR) approaches are the backbone of many studies in population ecology to gain insight on the life cycle, migration, habitat use, and demography of target species. The reliable and repeatable recognition of an individual throughout its lifetime is the basic requirement of a CMR study. Although invasive techniques are available to mark individuals permanently, noninvasive methods for individual recognition mainly rest on photographic identification of external body markings, which are unique at the individual level. The re‐identification of an individual based on comparing shape patterns of photographs by eye is commonly used. Automated processes for photographic re‐identification have been recently established, but their performance in large datasets (i.e., > 1000 individuals) has rarely been tested thoroughly. Here, we evaluated the performance of the program AMPHIDENT, an automatic algorithm to identify individuals on the basis of ventral spot patterns in the great crested newt (Triturus cristatus) versus the genotypic fingerprint of individuals based on highly polymorphic microsatellite loci using GENECAP. Between 2008 and 2010, we captured, sampled and photographed adult newts and calculated for 1648 samples/photographs recapture rates for both approaches. Recapture rates differed slightly with 8.34% for GENECAP and 9.83% for AMPHIDENT. With an estimated rate of 2% false rejections (FRR) and 0.00% false acceptances (FAR), AMPHIDENT proved to be a highly reliable algorithm for CMR studies of large datasets. We conclude that the application of automatic recognition software of individual photographs can be a rather powerful and reliable tool in noninvasive CMR studies for a large number of individuals. Because the cross‐correlation of standardized shape patterns is generally applicable to any pattern that provides enough information, this algorithm is capable of becoming a single application with broad use in CMR studies for many species.     

AUTOMATIC FEATURE EXTRACTION AND CLASSIFICATION OF CROSSBILL (LOXIA SPP.) FLIGHT CALLS

ABSTRACT

In this paper a new method for the automatic classification of bird sounds is presented. Our method is based on acoustic parameters (features) taken from the first harmonic component computed from the sound spectrogram. The features are based on a line segment approximation of the first harmonic component. The final feature vectors, consisting of 16 real numbers, are then classified using a self-organizing map (SOM) neural network. Flight calls of four crossbill species (Loxia spp.) are used as a test example. In the first phase, an unsupervised network was trained and tested using common crossbill L. curvirostra flight calls recorded mainly in the Netherlands. The network was tested using two-barred L. leucoptera, Scottish L. scotica and parrot L. pytyopsittacus crossbill flight calls in the second phase. Finally, the results were validated applying the same network to flight calls of common crossbills and parrot crossbills recorded in Finland. The method automatically separated common crossbill flight calls from those of parrot crossbills. The classification accuracy of the Dutch recordings was 58% in the first phase and 54% in the second phase. The Finnish recordings were classified with 54% accuracy.     

染色体易位重组位点的自动识别方法研究

染色体易位重组位点的识别对很多染色体遗传性疾病的诊断有着重要的意义.本文基于实际诊断中采集到的24类染色体数据和9号正常与异常染色体数据,构建了一套自动识别染色体易位重组位点的模型和方法.首先,对染色体图像进行预处理,得到了方向梯度直方图特征(HOG)和局部二值模式特征(LBP),构建了基于纹理特征的染色体24分类多通...     

Unsupervised classification to improve the quality of a bird song recording dataset

Open audio databases such as Xeno-Canto are widely used to build datasets to explore bird song repertoire or to train models for automatic bird sound classification by deep learning algorithms. However, such databases suffer from the fact that bird sounds are weakly labelled: a species name is attributed to each audio recording without timestamps that provide the temporal localization of the bird song of interest. Manual annotations can solve this issue, but they are time consuming, expert-dependent, and cannot run on large datasets. Another solution consists in using a labelling function that automatically segments audio recordings before assigning a label to each segmented audio sample. Although labelling functions were introduced to expedite strong label assignment, their classification performance remains mostly unknown. To address this issue and reduce label noise (wrong label assignment) in large bird song datasets, we introduce a data-centric novel labelling function composed of three successive steps: 1) time-frequency sound unit segmentation, 2) feature computation for each sound unit, and 3) classification of each sound unit as bird song or noise with either an unsupervised DBSCAN algorithm or the supervised BirdNET neural network. The labelling function was optimized, validated, and tested on the songs of 44 West-Palearctic common bird species. We first showed that the segmentation of bird songs alone aggregated from 10% to 83% of label noise depending on the species. We also demonstrated that our labelling function was able to significantly reduce the initial label noise present in the dataset by up to a factor of three. Finally, we discuss different opportunities to design suitable labelling functions to build high-quality animal vocalizations with minimum expert annotation effort.     

The Mass Distance Fingerprint: a statistical framework for de novo detection of predominant modifications using high-accuracy mass spectrometry

We describe a statistical measure, Mass Distance Fingerprint, for automatic de novo detection of predominant peptide mass distances, i.e., putative protein modifications. The method's focus is to globally detect mass differences, not to assign peptide sequences or modifications to individual spectra. The Mass Distance Fingerprint is calculated from high accuracy measured peptide masses. For the data sets used in this study, known mass differences are detected at electron mass accuracy or better. The proposed method is novel because it works independently of protein sequence databases and without any prior knowledge about modifications. Both modified and unmodified peptides have to be present in the sample to be detected. The method can be used for automated detection of chemical/post-translational modifications, quality control of experiments and labeling approaches, and to control the modification settings of protein identification tools. The algorithm is implemented as a web application and is distributed as open source software.     

Bioacoustics for species management: two case studies with a Hawaiian forest bird

The management of animal endangered species requires detailed information on their distribution and abundance, which is often hard to obtain. When animals communicate using sounds, one option is to use automatic sound recorders to gather information on the species for long periods of time with low effort. One drawback of this method is that processing all the information manually requires large amounts of time and effort. Our objective was to create a relatively “user‐friendly” (i.e., that does not require big programming skills) automatic detection algorithm to improve our ability to get basic data from sound‐emitting animal species. We illustrate our algorithm by showing two possible applications with the Hawai'i ‘Amakihi, Hemignathus virens virens, a forest bird from the island of Hawai'i. We first characterized the ‘Amakihi song using recordings from areas where the species is present in high densities. We used this information to train a classification algorithm, the support vector machine (SVM), in order to identify ‘Amakihi songs from a series of potential songs. We then used our algorithm to detect the species in areas where its presence had not been previously confirmed. We also used the algorithm to compare the relative abundance of the species in different areas where management actions may be applied. The SVM had an accuracy of 86.5% in identifying ‘Amakihi. We confirmed the presence of the ‘Amakihi at the study area using the algorithm. We also found that the relative abundance of ‘Amakihi changes among study areas, and this information can be used to assess where management strategies for the species should be better implemented. Our automatic song detection algorithm is effective, “user‐friendly” and can be very useful for optimizing the management and conservation of those endangered animal species that communicate acoustically.     

Reconstruction of cell-electrode-adjacencies on multielectrode arrays

The multichannel recordings of signals of many cells cultivated on a multielectrode array (MEA) impose some challenging problems. A meanwhile classic problem is the separation of the recordings of a single electrode into classes of recordings where each class is caused by a single cell. This is the well-known spike sorting. A “dual” problem is the determination of the set of electrodes that record signals of a single cell. This set is called the neighborhood of the cell and has often more than one element if the MEA has a large number of electrodes with high density. A method for the reconstruction of the neighborhoods from the multichannel recordings is presented. Special effort is directed to a precise peak detection. For the evaluation of the algorithm, artificial data, obtained from an appropriate model of MEA recordings, are used. Because the artificial data provide a ground truth, an evaluation of the accuracy of the algorithm is possible. The algorithm works well for realistic parameters.     

A novel machine learning unsupervised algorithm for sleep/wake identification using actigraphy

ABSTRACT

Actigraphy is widely used in sleep studies but lacks a universal unsupervised algorithm for sleep/wake identification. An unsupervised algorithm is useful in large-scale population studies and in cases where polysomnography (PSG) is unavailable, as it does not require sleep outcome labels to train the model but utilizes information solely contained in actigraphy to learn sleep and wake characteristics and separate the two states. In this study, we proposed a machine learning unsupervised algorithm based on the Hidden Markov Model (HMM) for sleep/wake identification. The proposed algorithm is also an individualized approach that takes into account individual variabilities and analyzes each individual actigraphy profile separately to infer sleep and wake states. We used Actiwatch and PSG data from 43 individuals in the Multi-Ethnic Study of Atherosclerosis study to evaluate the method performance. Epoch-by-epoch comparisons and sleep variable comparisons were made between our algorithm, the unsupervised algorithm embedded in the Actiwatch software (AS), and the pre-trained supervised UCSD algorithm. Using PSG as the reference, the accuracy was 85.7% for HMM, 84.7% for AS, and 85.0% for UCSD. The sensitivity was 99.3%, 99.7%, and 98.9% for HMM, AS, and UCSD, respectively, and the specificity was 36.4%, 30.0%, and 31.7%, respectively. The Kappa statistic was 0.446 for HMM, 0.399 for AS, and 0.311 for UCSD, suggesting fair to moderate agreement between PSG and actigraphy. The Bland–Altman plots further show that the total sleep time, sleep latency, and sleep efficiency estimates by HMM were closer to PSG with narrower 95% limits of agreement than AS and UCSD. All three methods tend to overestimate sleep and underestimate wake compared to PSG. Our HMM approach is also able to differentiate relatively active and sedentary individuals by quantifying variabilities in activity counts: individuals with higher estimated activity variabilities tend to show more frequent sedentary behaviors. Our unsupervised data-driven HMM algorithm achieved better performance than the commonly used Actiwatch software algorithm and the pre-trained UCSD algorithm. HMM can help expand the application of actigraphy in cases where PSG is hard to acquire and supervised methods cannot be trained. In addition, the estimated HMM parameters can characterize individual activity patterns and sedentary tendencies that can be further utilized in downstream analysis.     

Individual identification from genetic marker data: developments and accuracy comparisons of methods

Genetic marker‐based identification of distinct individuals and recognition of duplicated individuals has important applications in many research areas in ecology, evolutionary biology, conservation biology and forensics. The widely applied genotype mismatch (MM) method, however, is inaccurate because it relies on a fixed and suboptimal threshold number (T_M) of mismatches, and often yields self‐inconsistent pairwise inferences. In this study, I improved MM method by calculating an optimal T_M to accommodate the number, mistyping rates, missing data and allele frequencies of the markers. I also developed a pairwise likelihood relationship (LR) method and a likelihood clustering (LC) method for individual identification, using poor‐quality data that may have high and variable rates of allelic dropouts and false alleles at genotyped loci. The 3 methods together with the relatedness (RL) method were then compared in accuracy by analysing an empirical frog data set and many simulated data sets generated under different parameter combinations. The analysis results showed that LC is generally one or two orders more accurate for individual identification than the other methods. Its accuracy is especially superior when the sampled multilocus genotypes have poor quality (i.e. teemed with genotyping errors and missing data) and highly replicated, a situation typical of noninvasive sampling used in estimating population size. Importantly, LC is the only method that guarantees to produce self‐consistent results by partitioning the entire set of multilocus genotypes into distinct clusters, each cluster containing one or more genotypes that all represent the same individual. The LC and LR methods were implemented in a computer program COLONY for free download from the Internet.     

A review of automatic recognition technology for bird vocalizations in the deep learning era

Birds are considered critical indicators of ecosystem condition. Automatic recording devices have emerged as a trending tool to assist field observations, contributing to biodiversity monitoring on large spatio-temporal scales. However, manually processing huge volumes of recordings is challenging. Consequently, there has been a growing interest in automatic bird vocalization recognition in recent years. Automatic bird vocalization recognition technology has advanced from classical pattern recognition to deep learning (DL), with significantly improved recognition performance. This paper reviews related works on DL-based automatic bird vocalization recognition technology in the last decade. In this review, we present the current state of research in the three key areas of pre-processing, feature extraction and recognition methods involved in automatic bird vocalization recognition. The related datasets, evaluation metrics and software are also summarized. Finally, existing challenges along with opportunities for future work are highlighted. We conclude that, while DL-based automatic bird vocalization recognition has made recent advances in specific species, more robust denoising approaches, larger public datasets, and stronger generalization capabilities of feature extraction and recognition are required to achieve reliable and general bird recognition in the wild. We expect that this review will serve as a firm foundation for new researchers working in the field of DL-based automatic bird vocalization recognition technologies, as well as become an insightful guide for computer science and ecology experts.     

Automatic recording of individual oestrus vocalisation in group-housed dairy cattle: development of a cattle call monitor

Oestrus detection remains a problem in the dairy cattle industry. Therefore, automatic detection systems have been developed to detect specific behavioural changes at oestrus. Vocal behaviour has not been considered in such automatic oestrus detection systems in cattle, though the vocalisation rate is known to increase during oestrus. The main challenge in using vocalisation to detect oestrus is correctly identifying the calling individual when animals are moving freely in large groups, as oestrus needs to be detected at an individual level. Therefore, we aimed to automate vocalisation recording and caller identification in group-housed dairy cows. This paper first presents the details of such a system and then presents the results of a pilot study validating its functionality, in which the automatic detection of calls from individual heifers was compared to video-based assessment of these calls by a trained human observer, a technique that has, until now, been considered the ‘gold standard’. We developed a collar-based cattle call monitor (CCM) with structure-borne and airborne sound microphones and a recording unit and developed a postprocessing algorithm to identify the caller by matching the information from both microphones. Five group-housed heifers, each in the perioestrus or oestrus period, were equipped with a CCM prototype for 5 days. The recorded audio data were subsequently analysed and compared with audiovisual recordings. Overall, 1404 vocalisations from the focus heifers and 721 vocalisations from group mates were obtained. Vocalisations during collar changes or malfunctions of the CCM were omitted from the evaluation. The results showed that the CCM had a sensitivity of 87% and a specificity of 94%. The negative and positive predictive values were 80% and 96%, respectively. These results show that the detection of individual vocalisations and the correct identification of callers are possible, even in freely moving group-housed cattle. The results are promising for the future use of vocalisation in automatic oestrus detection systems.     

基于目标检测的东北虎个体自动识别

东北虎个体的自动识别是种群数量评估和制定有效保护策略的重要基础。以东北虎林园和怪坡虎园38 只虎为研究对象,将目标检测方法首次应用到东北虎个体识别研究中,采用多种深度卷积神经网络模型,以实现虎个体的自动识别。首先通过相机在不同角度对 38 只东北虎进行拍摄取样,建立包含13579张图像的虎样本数据集。由于虎的体侧条纹信息不具有对称性,所以运用单次多盒目标检测（Single Shot MultiBox Detector, SSD）方法,对虎的躯干左侧条纹、右侧条纹以及脸部等不同部位图像,进行自动检测并分割提取,极大节省手工截取时间。在检测分割出的左右侧及脸部不同部位图片基础上,运用上、下、左、右平移变换进行数据增强,使图片数目扩大为原来的5 倍。采用LeNet、AlexNet、ZFNet、VGG16、ResNet34共5 种卷积神经网络模型进行个体自动识别。为了提高识别准确率,运用平均值和最大值不同组合方式来优化池化操作,并在全连接层引入概率分别为0.1、0.2、0.3、0.4的丢弃（Dropout）操作防止过拟合。实验表明,目标检测模型耗时较少,截取分割老虎不同部位条纹能达到0.6 s/张,远快于人工截取速度,并且在测试集上准确率能达到97.4%。不同姿态下的目标部位都能正确识别并分割。ResNet34模型的准确率优于其他网络模型,左右侧条纹以及脸部图像识别准确率分别为93.75%、97.01%和 86.28%,右侧条纹识别准确率优于左侧条纹和脸部图像。研究为野生虎自动相机影像的识别提供技术参考。在未来研究中,对东北虎个体影响数据进行扩充,选取更多影像数据进行训练,使网络具有更强的适应性,从而实现更准确的个体识别。     

RECENT BIOACOUSTIC PUBLICATIONS (2002 and earlier)

To more easily and non-invasively monitor urban Eastern Screech-Owl populations, we developed a method of distinguishing individual owls using their calls. A set of seven variables derived from recordings of ‘bounce’ calls taken from 10 known (either free-ranging birds recorded at a single site on a single night or identifiable captive owls) owls was tested using a model-based clustering analysis (Mclust) as a method of discriminating individual owls. The cluster analysis correctly classified these calls with 98% accuracy. A second set of calls from nine owls was used to further test the method and correctly classified 84% of the calls using the same variables. Four owls were recorded repeatedly from 2008 to 2010 to determine the extent to which calls changed over time; the cluster analysis correctly assigned 89% of the calls to the correct owl regardless of the year the recordings were made. Based on these results, we are confident that the Mclust analysis can be used to reliably and safely estimate abundance and survival of Eastern Screech-Owls within the time frame of a few years and of population sizes < 15 owls.     

Identification of Cichlid Fishes from Lake Malawi Using Computer Vision

Background

The explosively radiating evolution of cichlid fishes of Lake Malawi has yielded an amazing number of haplochromine species estimated as many as 500 to 800 with a surprising degree of diversity not only in color and stripe pattern but also in the shape of jaw and body among them. As these morphological diversities have been a central subject of adaptive speciation and taxonomic classification, such high diversity could serve as a foundation for automation of species identification of cichlids.

Methodology/Principal Finding

Here we demonstrate a method for automatic classification of the Lake Malawi cichlids based on computer vision and geometric morphometrics. For this end we developed a pipeline that integrates multiple image processing tools to automatically extract informative features of color and stripe patterns from a large set of photographic images of wild cichlids. The extracted information was evaluated by statistical classifiers Support Vector Machine and Random Forests. Both classifiers performed better when body shape information was added to the feature of color and stripe. Besides the coloration and stripe pattern, body shape variables boosted the accuracy of classification by about 10%. The programs were able to classify 594 live cichlid individuals belonging to 12 different classes (species and sexes) with an average accuracy of 78%, contrasting to a mere 42% success rate by human eyes. The variables that contributed most to the accuracy were body height and the hue of the most frequent color.

Conclusions

Computer vision showed a notable performance in extracting information from the color and stripe patterns of Lake Malawi cichlids although the information was not enough for errorless species identification. Our results indicate that there appears an unavoidable difficulty in automatic species identification of cichlid fishes, which may arise from short divergence times and gene flow between closely related species.     

How common is avian hybridization on an individual level?*

Hybridization among different bird species is relatively common, but the hybridization rate of individuals is not well known. Justyn et al. use data from the citizen science project eBird to assess the individual hybridization rate in birds, showing that 0.064% of individuals are hybrids. The accuracy of this new estimate is affected by potential biases introduced by birdwatchers, such as over-reporting of rare hybrids and under-reporting of difficult-to-identify hybrids.     

Vocal individuality,but not stability,in wild palm cockatoos (Probosciger aterrimus)

The ability to identify individuals within a population is often essential for a detailed understanding of the ecology and conservation of a species. However, some species, including large parrots, are notoriously difficult to catch and mark for individual identification. Palm cockatoos (Probosciger aterrimus) are a large, poorly understood species of parrot which are likely in severe decline within the eastern part – and possibly the western part – of their range on Cape York Peninsula, Australia. Here, we investigated whether three different palm cockatoo call types are sufficiently individually distinctive to function as a non-invasive “marker” for identifying individuals over time. Using Discriminant Function Analysis, overall identification accuracy among 12 putative individuals for all call types was 81% (i.e. 148 out of 183 calls were assigned to the correct individual) on the basis of multiple temporal, energy (amplitude) and frequency measurements on the spectrogram. For three different call types, individual identification accuracy among males and females ranged from 69 to 95%. However, based on a limited sample sizes of five putative individuals between years, our data suggest that individual call structure, as quantified by call parameters, was not stable between years. We discuss the applicability of these results for future studies of palm cockatoos and other parrot species.