Data Products
Here's the place to learn and talk about our data products – both current and future.
New Features and Highlights
- New data products available for all hydrophones: audio data (wav, MP3 files), array raw data, spectral data (PNG, PDF spectrograms, FFT data for remote hydrophones). New data product options enable users to sort out diverted data and channels.
- Hydrophone, COVIS and BioSonics searches can now access and retrieve files directly from the archive. This speeds up processing, reliability and, above all, greatly speeds up searches on this data.
- Update for the COVIS DIFFUSE and IMAGING data products, new data products for COVIS DOPPLER mat, png, pdf files.
- A new time series scalar data product has been introduced: Ocean Data View Spreadsheet text files. These files are similar to CSV files, however, they load directly in to the popular Ocean Data View visualization software. See the ODV data product documentation
. - Time series scalar data and plots now utilize data product options for averaging, max/min resampling, clean and raw filtering, data gap filling, and plot type. For more details see the data product options main page here. The default option for time series scalar plots is the new automatic min/max+average resampling on raw data (mirrors the default plots in plotting utility). The default option for time series scalar data (CSV, MAT, ODV txt files) is clean data with gaps filled by NaNs.
- See the latest release notes here.
Current Data Products
ID |
NEPTUNE Canada |
|
ID |
Interoperability Partners |
|---|---|---|---|---|
1 |
|
15 |
||
2 |
|
16 |
||
3 |
|
17 |
||
4 |
|
|
|
|
5 |
|
|
|
|
7 |
|
|
|
|
9 |
|
|
|
|
10 |
|
|
|
|
11 |
|
|
|
|
12 |
|
|
|
|
14 |
|
|
|
|
18 |
|
|
|
|
19 |
|
|
|
|
20 |
|
|
|
|
21 |
|
|
|
|
22 |
|
|
|
|
23 |
|
|
|
|
24 |
|
|
|
|
25 |
|
|
|
|
26 |
|
|
|
|
27 |
|
|
|
|
30 |
|
|
|
|
33 |
|
|
|
|
34 |
|
|
|
|
35 |
|
|
|
|
36 |
|
|
|
|
38 |
|
|
|
|
40 |
|
|
|
|
41 |
|
|
|
|
42 |
|
|
|
|
43 |
|
|
|
|
44 |
|
|
|
|
45 |
|
|
|
|
48 |
|
|
|
|
49 |
|
|
|
|
52 |
|
|
|
|
56 |
|
|
|
|
57 |
|
|
|
|
58 |
|
|
|
|
59 |
|
|
|
|
60 |
|
|
|
|
61 |
|
|
|
|
62 |
|
|
|
|
63 |
|
|
|
|
64 |
|
|
|
|
65 |
|
|
|
|
66 |
|
|
|
|
67 |
|
|
|
|
68 |
|
|
|
|
76 |
|
|
|
Data Product Options
For all scalar data products and some complex data products, users will be presented with options to customize their data products. These options are described in the individual data product pages. A compilation of the options is presented in the data product options page.
Data Quality
Data quality information is supplied by way of data quality flags and comments in the data products, as well as annotations listed in the metadata reports. See the Quality Assurance Quality Control page for more information.
Metadata
Metadata reports available with nearly all different data products. These PDF reports are produced automatically when a data search is completed and are made available via a link adjacent to the data, see step 3 in data search help. The reports contain extensive information about the data, including instrument location, deployment, calibration, data quality and data gaps.
Conventions
Time-stamps: Time-stamps are always in UTC. For file-names and string dates, the format conforms to the ISO8601 convention: yyyymmddTHHMMSS. In some cases, the millisecond portion may be added: yyyymmddTHHMMSS.FFFZ. Numerical time-stamps within data product files may follow a different format as noted on the data product pages. For instances, numeric time-stamps within MAT files are in the MATLAB serial date format. When [resampling], the time-stamps are generally taken from the the centre of the resample interval.
File-names: Note that the underscore character, "_", is used to separate the components of the names. It is also used to replace any spaces in the name to maintain compatibility. File breaks are avoided as much as possible, but do occur for many reasons, including configuration or device changes, plus some data products have daily file breaks.
For an instrument search, files are named as DEVICECODE_SENSORNAME_yyyymmddTHHMMSSZ_yyyymmddTHHMMSSZ-MODE.EXT where:
- DEVICECODE is a descriptive string unique to each instrument.
- SENSORNAME is the sensor name as it appears in data search, and is only included if a single-sensor data product was requested.
- The first yyyymmddTHHMMSSZ is the time-stamp (ISO8601 format) of the first data record in the file. The second yyyymmddTHHMMSSZ is the last time-stamp of data in the file (not yet mandatory, being phased in). Notes: the time span of the data within the file may be considerably less than than the requested search time span. Instrument site-level searches are limited to the selected instrument deployment: use the 'All Available' option when selecting the time range to search for all of the data for the selected instrument and site. Note that file-name time-stamps may also have millisecond information: yyyymmddTHHMMSS.FFFZ, where 'FFF' are the milliseconds (this is being phased out).
- MODE is optional text which allows files to be differentiated. It is used for different operation modes (Kongsberg scan or sweep for instance) or different data product options. Data product option mode strings are only used on scalar data products, examples: '-NaN', '-clean', '-NaN_clean_avg15minute', '-MinMax1hour', see here for more details. In addition, the VPS cast products supply the cast time range and direction here.
- EXT is the file extension.
For a location search, files are named as STATIONNAME_DEVICECATEGORY_SENSORNAME_yyyymmddTHHMMSSZ_yyyymmddTHHMMSSZ-MODE.EXT where
- STATIONNAME is the station name, including node and station names separated by dashes, for example: BarkleyCanyon-VPSUpperSlope.
- DEVICECATEGORY is the device category, such as 'CTD'. If there is more than one device in the category, the file will contain multiple devices combined together for a long record of data.
- SENSORNAME is the sensor name and is omitted for a device-level data product that contains multiple sensors.
- yyyymmddTHHMMSSZ, MODE and EXT are as above.
File formats
Additional resources for available file formats is available here.
If you have any data product related questions or would like to see additional data products, please [let us know].