The effect of soundscape composition on bird vocalization classification in a citizen science biodiversity monitoring project

There is a need for monitoring biodiversity at multiple spatial and temporal scales to aid conservation efforts. Autonomous recording units (ARUs) can provide cost-effective, long-term and systematic species monitoring data for sound-producing wildlife, including birds, amphibians, insects and mammals over large areas. Modern deep learning can efficiently automate the detection of species occurrences in these sound data with high accuracy. Further, citizen science can be leveraged to scale up the deployment of ARUs and collect reference vocalizations needed for training and validating deep learning models. In this study we develop a convolutional neural network (CNN) acoustic classification pipeline for detecting 54 bird species in Sonoma County, California USA, with sound and reference vocalization data collected by citizen scientists within the Soundscapes to Landscapes project (www.soundscapes2landscapes.org). We trained three ImageNet-based CNN architectures (MobileNetv2, ResNet50v2, ResNet100v2), which function as a Mixture of Experts (MoE), to evaluate the usefulness of several methods to enhance model accuracy. Specifically, we: 1) quantify accuracy with fully-labeled 1-min soundscapes for an assessment of real-world conditions; 2) assess the effect on precision and recall of additional pre-training with an external sound archive (xeno-canto) prior to fine-tuning with vocalization data from our study domain; and, 3) assess how detections and errors are influenced by the presence of coincident biotic and non-biotic sounds (i.e., soundscape components). In evaluating accuracy with soundscape data (n = 37 species) across CNN probability thresholds and models, we found acoustic pre-training followed by fine-tuning improved average precision by 10.3% relative to no pre-training, although there was a small average 0.8% reduction in recall. In selecting an optimal CNN architecture for each species based on maximum F(β = 0.5), we found our MoE approach had total precision of 84.5% and average species precision of 85.1%. Our data exhibit multiple issues arising from applying citizen science and acoustic monitoring at the county scale, including deployment of ARUs with relatively low fidelity and recordings with background noise and overlapping vocalizations. In particular, human noise was significantly associated with more incorrect species detections (false positives, decreased precision), while physical interference (e.g., recorder hit by a branch) and geophony (e.g., wind) was associated with the classifier missing detections (false negatives, decreased recall). Our process surmounted these obstacles, and our final predictions allowed us to demonstrate how deep learning applied to acoustic data from low-cost ARUs paired with citizen science can provide valuable bird diversity data for monitoring and conservation efforts.     

Plants 'cry' for help through acoustic signals

Plants perceive sounds, while responses to these sounds were already known. A breakthrough is the discovery by Khait et al. that stressed plants emit various informative ultrasonic sound signals, which can be categorized according to plant species, stress type, and stress severity. This discovery may change how plants are cultivated.     

Fish Sound Production in the Presence of Harmful Algal Blooms in the Eastern Gulf of Mexico

This paper presents the first known research to examine sound production by fishes during harmful algal blooms (HABs). Most fish sound production is species-specific and repetitive, enabling passive acoustic monitoring to identify the distribution and behavior of soniferous species. Autonomous gliders that collect passive acoustic data and environmental data concurrently can be used to establish the oceanographic conditions surrounding sound-producing organisms. Three passive acoustic glider missions were conducted off west-central Florida in October 2011, and September and October 2012. The deployment period for two missions was dictated by the presence of red tide events with the glider path specifically set to encounter toxic Karenia brevis blooms (a.k.a red tides). Oceanographic conditions measured by the glider were significantly correlated to the variation in sounds from six known or suspected species of fish across the three missions with depth consistently being the most significant factor. At the time and space scales of this study, there was no detectable effect of red tide on sound production. Sounds were still recorded within red tide-affected waters from species with overlapping depth ranges. These results suggest that the fishes studied here did not alter their sound production nor migrate out of red tide-affected areas. Although these results are preliminary because of the limited measurements, the data and methods presented here provide a proof of principle and could serve as protocol for future studies on the effects of algal blooms on the behavior of soniferous fishes. To fully capture the effects of episodic events, we suggest that stationary or vertically profiling acoustic recorders and environmental sampling be used as a complement to glider measurements.     

饲养状态真鲂(鱼弗)发声的昼夜与季节变化

本文研究了受水温和光周期等自然变化影响的饲养状态真鲂鮄发声的昼夜与季节变化,并研究了实验鱼活动(鱼类游泳次数)的日变化。声音信号的昼夜节律记录发声活动的日常水平(摄食之外的时期),但是每月变化的记录(季节性式型)则在摄食期间进行,因为摄食时声音信号增加,而日常发声活动较不频繁。实验鱼包括雌雄两性,且未达性成熟。真鲂鮄在白天发声多一些,也更活跃。声音为阵发式的,较不频繁(平均值=0.04发声/min每鱼每天)。最少的发声活动出现在晚上,凌晨和黄昏居中(声音的阵发更频繁,但是声音更少),最多的发声活动出现在白天(声音的阵发更频繁,并且含更多数目的声音)。竞争摄食时声音信号的比率不呈现季节性变化(平均值=3.98发声/min每鱼),与温度也不相关,显示出竞争摄食时声音的发出以最大比率进行。敲击声和呼噜声的某些声学特征与温度相关,特别是在较高的温度下呼噜声的节拍间隔急剧下降。敲击声和呼噜声的声音参数中的季节性变化,多数可以解释为发声肌肉和中央声音控制回路的温度效应。     

SILIC: A cross database framework for automatically extracting robust biodiversity information from soundscape recordings based on object detection and a tiny training dataset

• 1.Passive acoustic monitoring (PAM) offers many advantages comparing with other survey methods and gains an increasing use in terrestrial ecology, but the massive effort needed to extract species information from a large number of recordings limits its application. The convolutional neural network (CNN) has been demonstrated with its high performance and effectiveness in identifying sound sources automatically. However, requiring a large amount of training data still constitutes a challenge.
• 2.Object detection is used to detect multiple objects in photos or videos and is effective at detecting small objects in a complex context, such as animal sounds in a spectrogram and shows the opportunity to build a good performance model with a small training dataset. Therefore, we developed the Sound Identification and Labeling Intelligence for Creatures (SILIC), which integrates online animal sound databases, PAM databases and an object detection-based model, for extracting information on the sounds of multiple species from complex soundscape recordings.
• 3.We used the sounds of six owl species in Taiwan to demonstrate the effectiveness, efficiency and application potential of the SILIC framework. Using only 786 sound labels in 133 recordings, our model successfully identified the species' sounds from the recordings collected at five PAM stations, with a macro-average AUC of 0.89 and a mAP of 0.83. The model also provided the time and frequency information, such as the duration and bandwidth, of the sounds.
• 4.To our best knowledge, this is the first time that the object detection algorithm has been used to identify sounds of multiple wildlife species. With an online sound-labeling platform embedded and a novel data preprocessing approach (i.e., rainbow mapping) applied, the SILIC shows its good performance and high efficiency in identifying wildlife sounds and extracting robust species, time and frequency information from a massive amount of soundscape recordings based on a tiny training dataset acquired from existing animal sound databases. The SILIC can help expand the application of PAM as a tool to evaluate the state of and detect the change in biodiversity by, for example, providing high temporal resolution and continuous information on species presence across a monitoring network.

     

First attempt to use a remotely operated vehicle to observe soniferous fish behavior in the Gulf of Maine, Western Atlantic Ocean

Underwater sound and video observations were made at noon, sunset, and midnight in sand, gravel, and boulder habitat in the Stellwagen Bank National Marine Sanctuary, Gulf of Maine, USA in October 2001 using a remotely operated vehicle (ROV). Seventeen species of fish and squid were observed with clear habitat and time differences. Observations of feeding behavior, disturbance behavior, and both interspecific and intraspecific interactions provided numerous opportunities for potential sound production; however, sounds were recorded only during a single dive. Although high noise levels generated by the ROV and support ship may have masked some sounds, we conclude that fish sound production in the Gulf of Maine during the fall is uncommon. The recorded fish sounds are tentatively attributed to the cusk Brosme brosme. Cusk sounds consisted variously of isolated thumps, widely spaced thump trains, drumrolls, and their combinations. Frequency peaks were observed at 188, 539, and 1195 Hz. Use of a remotely operated vehicle (ROV) as a passive acoustic observation platform was problematic due to high ROV self-noise and the ROV's inability to maintain a fixed position on the bottom without thruster power. Some fishes were clearly also disturbed by ROV noise, indicating a potential ROV sampling bias. Based on our observations, we suggest that new instruments incorporating both optic and passive acoustic technologies are needed to provide better tools for in situ behavioral studies of cusk and other fishes [Current Zoology 56 (1): 90-99 2010].     

Ontogeny of Acoustic and Feeding Behaviour in the Grey Gurnard, Eutrigla gurnardus

Although sound production in teleost fish is often associated with territorial behaviour, little is known of fish acoustic behaviour in other agonistic contexts such as competitive feeding and how it changes during ontogeny. The grey gurnard, Eutrigla gurnardus, frequently emits knock and grunt sounds during competitive feeding and seems to adopt both contest and scramble tactics under defensible resource conditions. Here we examine, for the first time, the effect of fish size on sound production and agonistic behaviour during competitive feeding. We have made sound (alone) and video (synchronized image and sound) recordings of grey gurnards during competitive feeding interactions. Experimental fish ranged from small juveniles to large adults and were grouped in four size classes: 10–15, 15–20, 25–30 and 30–40 cm in total length. We show that, in this species, both sound production and feeding behaviour change with fish size. Sound production rate decreased in larger fish. Sound duration, pulse duration and the number of pulses increased whereas the peak frequency decreased with fish size, in both sound types (knocks and grunts). Interaction rate and the frequency of agonistic behaviour decreased with increasing fish size during competitive feeding sessions. The proportion of feeding interactions accompanied by sound production was similar in all size classes. However, the proportion of interactions accompanied by knocks (less aggressive sounds) and by grunts (more aggressive) increased and decreased with fish size, respectively. Taken together, these results suggest that smaller grey gurnards compete for food by contest tactics whereas larger specimens predominantly scramble for food, probably because body size gives an advantage in locating, capturing and handling prey. We further suggest that sounds emitted during feeding may potentially give information on the motivation and ability of the individual to compete for food resources.     

Learned or innate production of acoustic signals in fishes: a test using a cyprinid

No information on the inheritance of the ability to produce sounds exists for fishes. In birds, which usually provide extensive post-hatching parental care, acoustic signals are learned in some species but are innate in others. Almost no fishes provide extensive post-hatching parental care and, consequently, the offspring have little opportunity to hear and learn sounds produced by the parents (usually the male in fishes); they may, however, be exposed to acoustic signals of conspecifics in the same habitat. We used a cyprinid, Codoma ornata, to test whether sound production is learned from the parents or whether it is innate. Fertilized eggs of this species were raised in isolation from adults. Upon maturity, these fish were tested for sound production in aggressive and reproductive contexts. Fish which had no contact with adults, and therefore no opportunity to hear the acoustic signals of their species, produced sounds that were similar to those produced by their parents, and they produced these in the same contexts. Significant differences were observed in dominant frequency for one context, with the smaller F₁ fish having signals of higher frequency than parental fish. Since no opportunity for learning existed, this provided evidence that the ability to produce sounds is innate in this minnow species.     

Mysterious bio-duck sound attributed to the Antarctic minke whale (Balaenoptera bonaerensis)

For decades, the bio-duck sound has been recorded in the Southern Ocean, but the animal producing it has remained a mystery. Heard mainly during austral winter in the Southern Ocean, this ubiquitous sound has been recorded in Antarctic waters and contemporaneously off the Australian west coast. Here, we present conclusive evidence that the bio-duck sound is produced by Antarctic minke whales (Balaenoptera bonaerensis). We analysed data from multi-sensor acoustic recording tags that included intense bio-duck sounds as well as singular downsweeps that have previously been attributed to this species. This finding allows the interpretation of a wealth of long-term acoustic recordings for this previously acoustically concealed species, which will improve our understanding of the distribution, abundance and behaviour of Antarctic minke whales. This is critical information for a species that inhabits a difficult to access sea-ice environment that is changing rapidly in some regions and has been the subject of contentious lethal sampling efforts and ongoing international legal action.     

Ontogenesis of agonistic vocalizations in the cichlid fish Metriaclima zebra

While acoustic communication has been described in adults of various fish species, our knowledge about the ontogeny of fish sound production is limited. In adults, sound signals are known to be involved during aggressive interactions. However, aggressive behaviour may appear early in the life of fishes due to the possible competition for food and space. If acoustic signals are used to send information to competitors, sounds are likely to play a role during interactions between juvenile fish as well. The apparition and evolution of sound production were monitored in a group of juveniles of the cichlid fish Metriaclima zebra from hatching to 4months of age. In addition, the link between vocalizations and agonistic behaviour was studied during dyadic interactions at three different ages. Sounds production appeared to be present early in the development of this fish and increased along with the number of aggressive behaviours. Recorded sounds consisted, in juveniles, in isolated pulses showing a decrease in frequency and duration as the fish grew. In adults, sounds became bursts of pulses but the transition from isolated to repetitive pulses was not observed. These results are compared to the existing literature on sound production ontogeny in fishes.     

Temporal and spatial mapping of red grouper Epinephelus morio sound production

The goals of this project were to determine the daily, seasonal and spatial patterns of red grouper Epinephelus morio sound production on the West Florida Shelf (WFS) using passive acoustics. An 11 month time series of acoustic data from fixed recorders deployed at a known E. morio aggregation site showed that E. morio produce sounds throughout the day and during all months of the year. Increased calling (number of files containing E. morio sound) was correlated to sunrise and sunset, and peaked in late summer (July and August) and early winter (November and December). Due to the ubiquitous production of sound, large‐scale spatial mapping across the WFS of E. morio sound production was feasible using recordings from shorter duration‐fixed location recorders and autonomous underwater vehicles (AUVs). Epinephelus morio were primarily recorded in waters 15–93 m deep, with increased sound production detected in hard bottom areas and within the Steamboat Lumps Marine Protected Area (Steamboat Lumps). AUV tracks through Steamboat Lumps, an offshore marine reserve where E. morio hole excavations have been previously mapped, showed that hydrophone‐integrated AUVs could accurately map the location of soniferous fish over spatial scales of <1 km. The results show that passive acoustics is an effective, non‐invasive tool to map the distribution of this species over large spatial scales.     

Acoustic localization of terrestrial wildlife: Current practices and future opportunities

Autonomous acoustic recorders are an increasingly popular method for low‐disturbance, large‐scale monitoring of sound‐producing animals, such as birds, anurans, bats, and other mammals. A specialized use of autonomous recording units (ARUs) is acoustic localization, in which a vocalizing animal is located spatially, usually by quantifying the time delay of arrival of its sound at an array of time‐synchronized microphones. To describe trends in the literature, identify considerations for field biologists who wish to use these systems, and suggest advancements that will improve the field of acoustic localization, we comprehensively review published applications of wildlife localization in terrestrial environments. We describe the wide variety of methods used to complete the five steps of acoustic localization: (1) define the research question, (2) obtain or build a time‐synchronizing microphone array, (3) deploy the array to record sounds in the field, (4) process recordings captured in the field, and (5) determine animal location using position estimation algorithms. We find eight general purposes in ecology and animal behavior for localization systems: assessing individual animals' positions or movements, localizing multiple individuals simultaneously to study their interactions, determining animals' individual identities, quantifying sound amplitude or directionality, selecting subsets of sounds for further acoustic analysis, calculating species abundance, inferring territory boundaries or habitat use, and separating animal sounds from background noise to improve species classification. We find that the labor‐intensive steps of processing recordings and estimating animal positions have not yet been automated. In the near future, we expect that increased availability of recording hardware, development of automated and open‐source localization software, and improvement of automated sound classification algorithms will broaden the use of acoustic localization. With these three advances, ecologists will be better able to embrace acoustic localization, enabling low‐disturbance, large‐scale collection of animal position data.     

First evidence of fish larvae producing sounds

The acoustic ecology of marine fishes has traditionally focused on adults, while overlooking the early life-history stages. Here, we document the first acoustic recordings of pre-settlement stage grey snapper larvae (Lutjanus griseus). Through a combination of in situ and unprovoked laboratory recordings, we found that L. griseus larvae are acoustically active during the night, producing ‘knock’ and ‘growl’ sounds that are spectrally and temporally similar to those of adults. While the exact function and physiological mechanisms of sound production in fish larvae are unknown, we suggest that these sounds may enable snapper larvae to maintain group cohesion at night when visual cues are reduced.     

Plants can talk: a new era in plant acoustics

Plants release chemical signals to interact with their environment when exposed to stress. Khait and colleagues unveiled that plants ‘verbalize’ stress by emitting airborne sounds. These can train machine learning models to identify plant stressors. This unlocks a new path in plant-environment interactions research with multiple possibilities for future applications.     

Spatio-temporal heterogeneity in river sounds: Disentangling micro- and macro-variation in a chain of waterholes

1. Passive acoustic monitoring is gaining momentum as a viable alternative method to surveying freshwater ecosystems. As part of an emerging field, the spatio-temporal replication levels of these sampling methods need to be standardised. However, in shallow waters, acoustic spatio-temporal patchiness remains virtually unexplored.
2. In this paper, we specifically investigate the spatial heterogeneity in underwater sounds observed within and between waterholes of an ephemeral river at different times of the day and how it could affect sampling in passive acoustic monitoring.
3. We recorded in the Einasleigh River, Queensland in August 2016, using a linear transect of hydrophones mounted on frames. We recorded four times a day: at dawn, midday, dusk, and midnight. To measure different temporal and spectral attributes of the recorded sound, we investigated the mean frequency spectrum and computed acoustic indices.
4. Both mean frequency spectrum and index analyses revealed that the site and diel activity patterns significantly influenced the sounds recorded, even for adjacent sites with similar characteristics along a single river. We found that most of the variation was due to temporal patterns, followed by between-site differences, while within-site differences had limited influence.
5. This study demonstrates high spatio-temporal acoustic variability in freshwater environments, linked to different species or species groups. Decisions about sampling design are vital to obtain adequate representation. This study thus emphasises the need to tailor spatio-temporal settings of a sampling design to the aim of the study, the species and the habitat.

     

The stomatopod rumble: Low frequency sound production in Hemisquilla californiensis

Stomatopods (mantis shrimp), numbering over 450 species, are renowned for their exceptional visual and chemosensory abilities and yet virtually nothing is known about their use of acoustic signals. We present acoustic recordings and analyses of the sounds of a stomatopod, Hemisquilla californiensis. This species generates tonal, low frequency sounds, which we term 'rumble', that are spectrally similar to those produced by African and Asian elephants. The fundamental frequency of the stomatopod rumble ranges from 20 to 60 Hz, with a strong second harmonic. Hemisquilla californiensis appears to generate these sounds through contractions of the posterior mandibular remotor muscle which is coupled to a stiff, lateral extension of the carapace. The sides of the carapace are covered by large, polarized, red spots which vibrate during sound production. Thus, the animals may generate a multi-modal signal by coupling vibrational and visual signals. Hemisquilla californiensis generates the rumble while interacting with potential predators and burrow intruders, suggesting a defensive or territorial function for the sound.     

Stridulatory Sound-Production and Its Function in Females of the Cicada Subpsaltria yangi

Acoustic behavior plays a crucial role in many aspects of cicada biology, such as reproduction and intrasexual competition. Although female sound production has been reported in some cicada species, acoustic behavior of female cicadas has received little attention. In cicada Subpsaltria yangi, the females possess a pair of unusually well-developed stridulatory organs. Here, sound production and its function in females of this remarkable cicada species were investigated. We revealed that the females could produce sounds by stridulatory mechanism during pair formation, and the sounds were able to elicit both acoustic and phonotactic responses from males. In addition, the forewings would strike the body during performing stridulatory sound-producing movements, which generated impact sounds. Acoustic playback experiments indicated that the impact sounds played no role in the behavioral context of pair formation. This study provides the first experimental evidence that females of a cicada species can generate sounds by stridulatory mechanism. We anticipate that our results will promote acoustic studies on females of other cicada species which also possess stridulatory system.     

Applications of bioacoustics in animal ecology

In communication animals use a full range of signals: acoustic, visual, chemical, electrical and tactile. The processes involved in how and why animals communicate have long held veritable fascination for scientists. A branch of science concerned with the production of sound and its effects on living organisms is bioacoustics.The main purpose of the present study is to raise and discuss some issues related to the relationship between animals, their sounds and ecology, including presentation of methods of analysis of sound recordings. A better understanding of the relationship between the studied animals will allow for development of a better framework for future research, as well as a better grasp of interactions between different organisms, including humans. The paper discusses the significance of acoustic research in animal ecology and its possible applications in the future. The author also summarizes previous research in the field of sound communication of various animal species.The paper proves that vocalizations of every acoustically communicating animal are threatened by climate change. For marine animals, the source of changes in vocalization abilities is ocean acidification and increased ambient noise, which can affect communication and foraging behavior. For terrestrial animals, changes in precipitation and temperature may result in modifications of the sounds emitted, as well as certain modifications to the auditory system. Together with changes in species distribution due to environmental parameters, cumulatively these factors can cause changes in the entire landscape of acoustics ecosystems. Thanks to acoustic biomonitoring, we can understand how the sounds of entire habitats and acoustic ecosystems will change in response to the changing climate and how it will affect bioacoustics on a global scale.