Versions Compared

Old Version 26

changes.mady.by.user Ben Biffard

Saved on

compared with

New Version Current

changes.mady.by.user Spencer Plovie

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Please consult individual instruments' documentation for more information.

Oceans 23.0 API filter: dataProductCode=CSPPCSPPD

Revision History

  1. 20160901: Initial Release
  2. 20180507: Updated with cast delineation, options for cast type selection and temperature/conductivity lag correction

Data Product Options

Include Page
Cast Scalar Profile Plot and Data
Cast Scalar Profile Plot and Data

Formats

MAT / netCDF: Cast Scalar Profile Data

This data is available as MAT or netCDF data products. The MAT file is structured as an MxN structure where M is the cast field including sensor data (cond, temp, psal, etc.) and cast details (cast start date, end data, duration, depth, direction, etc.), and N is the cast number. Sensor data is stored in a structure identical to its original data structure (sensorID, samplePeriod, time, data, qaqcflags, etc.).

PNG / PDF: Cast Scalar Profile Plot

This data is available as PNG or PDF format plots.Oceans 3.0 API filter: extension={png,pdf}

Cast Scalar Profile plots are plots of depth versus the variable or sensor being plotted and are available as PNG or PDF format plots. These are the standard CTD cast-type plots. Multiple sensors maybe plotted on multiple x-axes for device-level plots.

For device level plots, particularly for CTDs, we put density, sigma-t, and conductivity vs depth in the first plot, with specific colours while the remaining sensors are plotted in subsequent plots. The second plot contains practical salinity, temperature, sigma-theta, and sound speed vs depth. Each plot is colour coded to improve readability. The second plot will have “_2” added on the end of the filename. Here is an example of a device level plot search for down casts on the BPS, resulting in two plots:

Image RemovedImage RemovedImage AddedImage Added

Here is an example of single sensor plot for up casts on the VPS:

Image Added


MAT / netCDF:

...

Cast Scalar Profile Data

This data is available as MAT or netCDF data products. 

MAT Files

Oceans 3

Image Removed

Oceans 2.0 API filter: extension={pngmat,pdf}nc}

MAT files (v7) can be opened using MathWorks MATLAB 7.0 or later. The file contains two structures: ProfileData and metadata.

ProfileData: structured as an 1 x N structure with M fields where M includes the sensor data fields (conductivity, temperature, salinity, etc.) and cast details (cast start date, end data, duration, depth, direction, etc.), and N is the cast number:

Image Added

  • Sensor Data (example: seawatertemperature, depth, etc.): see below for structure details 
  • startDate: Start date of the profile in the format 'dd-mmm-yyyy HH:MM:SS.FFF' 
  • endDate: End date of the profile in the format 'dd-mmm-yyyy HH:MM:SS.FFF' 
  • duration: Duration of the profile, in minutes.
  • meanDate: Mean date of the profile (mean of startDate and endDate)
  • minDepth: Minimum depth of the profile (in meters)
  • maxDepth: Maximum depth of the profile (in meters) 
  • depthRange: Total depth of the profile (in meters) 
  • direction: Direction of cast (either 'Down Casts', 'Up Casts', or 'Stationary')

Sensor data is stored in a structure identical to its original data structure (the original being the standard scalar data structure defined in Time Series Scalar Data).

Image Added

  • sensorID: Unique identifier number for sensor.

  • sensorName: Name of sensor.

  • sensorCode: Unique string for the sensor.

  • sensorDescription: Description of sensor.

  • sensorType: Type of sensor as classified in the ONC data model.

  • sensorTypeID: ONC ID given to sensor type.

  • units: Unit of measure for the sensor data.

  • isEngineeringSensor: boolean (flag) to determine if sensor is an engineering sensor.

  • sensorDerivation: String describing the source of the sensor data: derived from calibration formula (dmas-derived), calculated on the device (instrument-derived), calculated by an external process (externally-derived), or direct from the instrument.

  • propertyCode: Unique string for the sensor using only lowercase letters and no unique characters

  • isMobilePositionSensor: boolean (flag) to determine if sensor is a mobile sensor. Note, this will only be flagged true if this data was added in addition to the requested data. For example, if the user requests a device-level mat product from a GPS device, then the latitude sensor is not flagged. Conversely, if the user requests temperature data from a mobile platform like a ship, then the latitude data from the GPS is added and interpolated to match the time stamps of the temperature sensor. See Positioning and Attitude for Mobile Devices for more information.

  • deviceID: Unique identifier number for the parent device.

  • searchDateNumFrom: Start date of the specific search in MATLAB datenum format - searches are truncated by availability and deployment dates.

  • searchDateNumTo: End date of the specific search MATLAB datenum format - searches are truncated by availability and deployment dates.

  • samplePeriod: Vector of sample periods in seconds.

  • samplePeriodDateFrom: Vector of the start date of each sample period (MATLAB datenum format).

  • samplePeriodDateTo: Vector of the end date of each of sample period (MATLAB datenum format).

  • sampleSize: The size of the data sample. 

  • resampleType: Type of resampling used.

  • resampleDescription: Description of the resampleing used.

  • resamplePeriod_sec: Resample period in seconds.

  • resampleTypeID: Unique identifier of the subsample type used: 0/NaN - none, 1 - average, 2 - decimated (not offered), 3 - min/max, 4 - linear interpolation (VPS pressure only).

  • dataProductOptions: A string describing the data product options selected for this data product. This information is reflected in the file name.

  • qaqcFlagDescription: A string describing the flags. See the QAQC page for more information.

  • time: A vector of data timestamps in MATLAB datenum format.

  • dat: A vector of sensor values corresponding to each timestamp. When resampling by averaging, this becomes the average value. (May make a separate field for this in the future, especially if users prefer that option).

  • qaqcFlags: A vector indicating the quality of the data, matching the time and dat vectors. See the QAQC page for more information.

  • dataDateNumFrom: First time-stamp of the time series.

  • dataDateNumTo: Last time-stamp of the time series.

  • samplesExpected: The number of valid samples expected from the minimum returned data to the maximum returned data, accounting for variations in sample period.

  • samplesReceived: The number of raw samples received, maybe less than length(data.time) when data gaps are being filled with the NaN option.

  • Calibration: Structure containing information on the calibration formula applied to the data, as a it appears on the sensor listing page, in the JEP langauage. Fields include: dateFrom, dateTo, sensorID, name, formula.

Image Added

Include Page
MAT file Metadata structure - Scalar
MAT file Metadata structure - Scalar

NETCDF Files

Oceans 3.0 API filter: extension=nc

NetCDF is a machine-independent data format offered by numerous institutions, particularly within the earth and ocean science communities. Additional resources are noted here.

Two NetCDF files are created with this Data Product: TimeSeriesNetCDF and BinnedNetCDF. The NetCDF files are extracted from the data contained in the above MAT file.

The TimeSeriesNetCDF file contains the following variables: 

  • time: Time of measurement in days since 1970-01-01 00:00:00
  • start_stop_indx: Start and stop indices for each profile (index corresponding to time series)
  • direction: Direction of cast (up = -1, down = 1, stationary = 0)
  • variable1*: Time series of sensor data (ie seawatertemperature, depth, etc.). The time series only includes data from desired profile (ie only down casts), and the start and stop indices of each profile are stored in the variable start_stop_indx
  • variable2*: Time series of next sensor (ie seawatertemperature, depth, etc.).
  • variable1_qaqcFlags: Time series of sensor qaqc flag
  • variable2_qaqcFlags: Time series of next sensor qaqc flag

* these will be labelled using the sensor's propertyCode (ie depth, depth_qaqcFlags)

Example text file to display contents: InshoreProfilingSystem_ProfilingInstrumentPackage_CTD_Temperature_20160821T000000Z_20160822T000000Z-clean_Profile_DownCasts_TimeSeries.txt

Example NetCDF file: InshoreProfilingSystem_ProfilingInstrumentPackage_CTD_Temperature_20160821T000000Z_20160822T000000Z-clean_Profile_DownCasts_TimeSeries.nc

The BinnedNetCDF file contains the following variables: 

  • time_prof: A single date for each profile (the mean profile date)
  • depth_bin: Water depth (in m) of the edges of the measurement bin (ie depth_bin(1) = 0.5 m and depth_bin(2) = 1.5 m so the first bin covers the depths 0.5 to 1.5 m, in other words binned to a 1 m grid centered around integer depths)
  • direction: Direction of cast (up = -1, down = 1, stationary = 0)
  • variable1*: Binned profile of sensor data (ie seawatertemperature, depth, etc., see DP61 for details on binning method)
  • variable2*: Binned profile of next sensor (ie seawatertemperature, depth, etc.)
  • variable1*_qaqcFlags: Qaqc flag of each bin for sensor (see “Documentation for Integrating QAQC Flags in VENUS search” for method details)
  • variable2*_qaqcFlags: Qaqc flag of each bin for next sensor

* these will be labelled using the sensor's propertyCode (i.e. depth, depth_qaqcFlags)

Unfortunately, our NetCDF files are not CF-Compliant due to the use of ONC variable names, but this will change in the future.

Example text file to display contents: InshoreProfilingSystem_ProfilingInstrumentPackage_CTD_Temperature_20160821T030720Z_20160821T212313Z-clean_Binned.txt

Example NetCDF file: InshoreProfilingSystem_ProfilingInstrumentPackage_CTD_Temperature_20160821T030720Z_20160821T212313Z-clean_Binned.nc

Temperature/Conductivity Lag Correction

...

The newly calculated ‘_aligned’ sensors do not have unique sensor ID’s. Therefore, they cannot be explicitly searched for in Oceans 23.0 and will not show up in Cast Scalar Profile plots. They will however show up in Time Series Scalar Profile plots and both MAT or netCDF data files, but only if the user selects data that includes both temperature and conductivity sensors.

...