Audio Data
Audio data is primarily produced by hydrophones (underwater microphones). A number of hydrophones are producing audio files with sounds at a wide range of frequencies, having applications in seismology, marine mammal studies, ship noise and more.
Given the sensitive nature of hydrophone data, the military has the ability to divert the data as required. Diverted data is then reviewed by military authorities, if it does not contain sensitive recordings it is returned to the ONC archive. Prior to August 2016, instead of diverting the entire frequency range of the data, the military diverted only a low frequency band of the data. When this filtering occurred, the file-name was appended with 'LPF' for low-pass filtering. Once filtered, this data remained in separate frequency bands in separate files, so data returned to ONC was labelled 'HPF' for high-pass filtered. See here for more information on the diversion of hydrophone and seismometer data. Data Search and the DataProductDelivery API provide options for filtering out HPF and LPF files, with the "original data" filter in the data diversion mode options (see the Data Product Options section below, one can also select the HPF or LPF files for download). If you happen to download a dataset that has LPF or HPF files and you want to segregate the files, follow these instructions: How to remove LPF and HPF data from an acoustic data set.
For hydrophones located on remote observatories with limited internet bandwidth (Cambridge Bay, Brentwood Bay, etc), live hydrophone audio files may not be available. In this case, any missing audio data is stored on site to be retrieved during regular maintenance (yearly or less). Live the spectrogram data (FFT) files will normally be available they are much smaller and can be sent over limited connections.
When requested format of audio data already exists in the archive, the search will complete immediately, otherwise some processing will occur to convert the formats. Processing times vary, but in general, expect it would take approximately 15 seconds to process one 5 minute file, so one hour to process a day's worth of data.
Some software applications that employ good error checking may reject some of our wav files for erroneous header information. The cause of this is in the military diversion drivers and is outside of ONC's direct control (they hardcoded these values). Only a select few customized applications will reject the files. Audacity, MATLAB and many others read these files fine. If you get an error like this:
and you are an active MATLAB user, you can fix the wav files with with this MATLAB code: oncwavfilefix.m and it's dependency: wavchunksizefix.m. If not, please contact us and we'll be happy to help.
Oceans 2.0 API filter: dataProductCode=AD
Revision History
- 20100217: Hydrophone files initially made publicly available
- 20130123: searches handled by MATLAB search to process data on the fly if needed; all formats made available to all devices
Data Product Options
Hydrophone Channel
H1
This option will cause the search to return results for hydrophone channel H1 only. The hydrophone arrays consist of multiple hydrophones connected to a single data acquisition computer, which collects the data into single files that have multiple channels (nominally raw hydrophone array files, although other formats can handle multiple channels). Data products may be produced from these files on a per channel basis and returned as specified.
This is the default option.
Oceans 3.0 API filter: dpo_hydrophoneChannel=H1
File-name mode field
'H1' is added to the file-name when the hydrophone channel option is set to H1, i.e. IOS3HYDARR02_20111211T152404.000Z-spect-H1.pdf.
H2
This option will cause the search to return results for hydrophone channel H2 only.
Oceans 3.0 API filter: dpo_hydrophoneChannel=H2
File-name mode field
'H2' is added to the file-name when the hydrophone channel option is set to H2, i.e. IOS3HYDARR02_20111211T152404.000Z-spect-H2.png.
H3
This option will cause the search to return results for hydrophone channel H3 only.
Oceans 3.0 API filter: dpo_hydrophoneChannel=H3
File-name mode field
'H2' is added to the file-name when the hydrophone channel option is set to H3, i.e. IOS3HYDARR02_20120801T090939.000Z-H3.mp3.
All
This option will cause the search to return results for all available hydrophone channels.
Oceans 3.0 API filter: dpo_hydrophoneChannel=All
File-name mode field
'H1', 'H2', 'H3', etc are added to the file-name.
Hydrophone Data Diversion Mode
Diversion Mode
For security reasons, the military occasionally diverts seismic and acoustic data. Over time how this diversion is performed has changed. Currently, when diverted the entire data set is removed. Diverted data is then reviewed by military authorities, if it does not contain sensitive recordings it is returned to the ONC archive.
Standard practice prior to August 2016: instead of diverting the entire data stream, the military diverted only a low frequency band of the data. When this filtering occurred, the remaining data's file-name was appended with 'HPF' for high-pass filtering, while the low-pass data was held for review. Usually that withheld/diverted data was returned, after a delay of 3 days to 2 months; those files are appended with 'LPF' for low-pass filtered. To further confuse matters, sometimes the file-name appending was not complete - half of the data stream was not appended with the LPF or HPF moniker (usually the HPF side), however, our data product software now detects this via time overlaps and handles the other half of the LPF/HPF even if it isn't named so. After 2016, diversions tended to be all or nothing and no low-pass diversion occurred. Recently, the LPF/HPF data splitting has occurred again.
Data diversion is further explained in the data diversion page. Feel free to contact us for support.
Original Data
This option will cause the search to return results for original data only. Files labelled with "-HPF" or "-LPF" are excluded as well as any files that overlap in time with "-HPF" or "-LPF" files. For spectral probability density plots and spectrograms, 'Data Diversion Mode: Original Data' will appear in the plot title.
This is the default option.
Oceans 3.0 API filter: dpo_hydrophoneDataDiversionMode=OD
Low Pass Filtered
Applies to pre-August 2016 data (with some exceptions). This option will cause the search to return results for diverted data that has been low pass filtered only (only files with "-LPF" in the their file-names). For spectral probability density plots and spectrograms, 'Data Diversion Mode: Low Pass Filtered' will appear in the plot title.
Oceans 3.0 API filter: dpo_hydrophoneDataDiversionMode=LPF
High Pass Filtered
Applies to pre-August 2016 data (with some exceptions). This option will cause the search to return results for diverted data that has been high pass filtered only (only files with "-HPF" in the their file-names). For spectral probability density plots and spectrograms, 'Data Diversion Mode: High Pass Filtered' will appear in the plot title.
Oceans 3.0 API filter: dpo_hydrophoneDataDiversionMode=HPF
All
This option will cause the search to return results for all data. For spectral probability density plots and spectrograms, 'Data Diversion Mode: High Pass Filtered' will appear in the plot title. This is only way to see data that overlaps in time with files labelled "-LPF" or "-HPF".
Oceans 3.0 API filter: dpo_hydrophoneDataDiversionMode=All
File-name mode field
"-LPF" or "-HPF" is added to the file-name when the quality option is set to high or low pass filtered data, i.e. ICLISTENHF1234_20110101T000000Z-HPF.wav. For spectral probability density data products, 'All' may be added to the file-name, as these plots can join LPF, Original and HPF data together into one plot if the spectral frequency bins are the same (data with different frequency content will make addition plots with labels indicating the frequency range). For brevity, 'Original' does not get added to the file-name.
Acquisition Mode
For hydrophones operating with a duty cycle that includes high and low frequency sample rates (the hydrophones alternate between low and high sample rates periodically, to save battery and memory storage in autonomous deployments). The low sample frequency data will likely have a sample frequency of 16 kHz and the high sample frequency data will likely have a sample frequency greater or equal to then 128 kHz.
Low Sample Frequency
This option will cause the search to return results for the low sample frequency data only (files with "-16KHZ" in their file-names). For spectral probability density plots and spectrograms, "Data Acquisition Mode: Low Frequency" will appear in the plot title.
Oceans 3.0 API filter: dpo_hydrophoneAcquisitionMode=LF
High Sample Frequency
This option will cause the search to return results for the high sample frequency data only (files with "-128KHZ" or similar in their file-names). For spectral probability density plots and spectrograms, "Data Acquisition Mode: High Frequency" will appear in the plot title.
Oceans 3.0 API filter: dpo_hydrophoneAcquisitionMode=HF
All
This option will cause the search to return results for both the low and high sample frequency data or other mode data. For spectral probability density plots and MAT files, the low and high frequency data will be segregated regardless of option.
Oceans 3.0 API filter: dpo_hydrophoneAcquisitionMode=All
File-name mode field
The sample frequency is added to the file-name for each data acquisition mode option, i.e. ICLISTENHF1234_20110101T000000Z-16KHZ.wav. The Spectrogram_ModeDurationDPO device attribute is populated on devices with a duty cycle, it is used to link the low frequency (LF) and high frequency (HF) acquisition modes with the exact file-name mode modifier string - if this link is not correct, the data acquisition mode option will not properly filter the data products.
Format
Hydrophone data is available in WAV and MP3 audio files. For normal hydrophones, WAV files are the base data type, for hydrophone arrays, HYD files are the base data type. MP3s are generated from WAV files and are stored in the archive or may be generated on the fly.
For WAV format data, the files will be accompanied by a calibration file (except for FileDownloadService requests). This file is a text file, comma delimited with one descriptive header row. It is named following the usual standard and ending with '-hydrophoneCalibration.txt'. The dateFrom / dateTo is taken from the calibration date range of the first sensitivity bin attribute. If a search extends over multiple calibrations, multiple files will be produced. A currently applicable calibration will produce a file named with a dateTo that is midnight tomorrow. Here is an example of the first few lines of a hydrophone calibration text file:
#Hydrophone calibration sensitivities. The file contains one header line followed by comma delimited data. First column is the centre frequency of each frequency bin(Hz). Second column is the sensitivity calibration for each bin (dB/uPa). Data is from device attributes: http://qaweb2.neptune.uvic.ca/DeviceListing?DeviceId=1230 . Device attribute HydrophoneSensitivityVectorPart1 last modified: 02-Dec-2015 20:59:52. File created: 29-Mar-2016 15:36:06. 1, -33.1853 2, -30.6233 3, -29.8957 4, -29.6103
Oceans 2.0 API filter: extension={wav,mp3,flac}
Discussion
To comment on this product, click Add Comment below.