Skip to end of metadata
Go to start of metadata

RDI ADCP Time Series

Data files for Teledyne RDI ADCPs are described here. Ancillary data (e.g., temperature, pitch, roll) are also available independently, through scalar data products, see time series scalar data and time series scalar plots.

Oceans 2.0 API filterdataProductCode=RADCPTS

Revision History

  • 20180501: New options added for three-beam solutions, screening and fish detection
  • 20150903: RDI files now use corrected heading/pitch/roll as specified by site and sitedevice meta data. RDI files with Earth co-ord data are rotated to the corrected heading or apply magnetic declination. These changes do not change the MAT/PNG/NC/PDF data products produced from RDI files other than the compassHeading/pitch/roll values within the MAT/NC files. These changes will allow RDI files to be made available.
  • 20150601: Added bin-mapping options (None, nearest vertical, linear).
  • 20150228: Depth cell / bin mapping added to correct range and velocity for tilt.
  • 20141222: Heading correction rework (affecting ENU / uvw velocities): handle sites with null positions, mobile sensors, autonomous deployments, and better documentation of processing was done to the data.
  • 20130317: Bug fixes, including making the number of dimensions of data structures fixed (trailing singleton dimensions are always dropped in MATLAB however).
  • 20110328: NetCDF, CF1.5 compliant product released.
  • 20110202: Correction for determination of ENU (with respect to true North) velocities. Prior code assumed ADCP was downward-facing. Updated code uses orientation or instrument to convert to ENU coordinates.
  • 20100521: Derived backscatter added to MAT contents.
  • 20100513: Initial NetCDF product released for RDI ADCPs, available in CF-1.4 compliant format.
  • 20100218: MAT contents made more complete and descriptive
  • 20091208: Initial RDI and MAT products released

Formats

This data is available as raw data in RDI files, or as processed data in MAT and NETCDF formats. Content descriptions and example files are provided below.

Manufacturer's Raw Binary Format: .RDI Files

This binary format is specific to the manufacturer. When using Teledyne-RDI data acquisition software, data is normally stored in this way. Although we use custom-built drivers to communicate with our instruments, we can use the raw data in the log file to produce the RDI file which can be interpreted by Teledyne-RDI post-processing software.

To produce the file, the following requirements apply:

  • A new RDI file is started at the beginning of each day, when the maximum records per file is exceeded, or when the driver is restarted (this should account for configuration changes, site changes, etc).
  • Only records with valid checksums are included.
  • The instrument date/time field is replaced using the NEPTUNE timestamp at the beginning of the log file (since this timestamp is more accurate than the instrument clock), and the checksum is recalculated.
  • For Beam or Instrument co-ordinate data, heading/pitch/roll (EH/EP/ER) data in the variable length data header are replaced as specified by site and sitedevice metadata (replacing the often unreliable onboard compass) (the EZ sensor source value is changed to manual for each). For Earth co-ordinate data, only the heading is updated with the value from the site and sitedevice metadata, if the internal compass sensor is used, it is corrected for magnetic declination, else if the internal heading value is set manually, it is left alone. Also for Earth data, because the heading has changed, the North/South current velocities are rotated to the new heading. These changes do not change the velocities in the MAT/PNG/NC/PDF data products produced from RDI files, other than the compassHeading/pitch/roll values within the MAT/NC files. The original magnetic compass, pitch and roll maybe replaced by these fixed or external sensor values, however, users can access the original sensor data via scalar data products and services. 

For users' information and interest, the site and sitedevice information is available through the device details pages, for example: http://dmas.uvic.ca/DeviceListing?DeviceId=20003. This information is also included in the metadata report that accompanies most data products.

This format is further described in the manufacturer's documentation.

Example: RDIADCP75WH17432_20131113T000000.000Z.rdi

Oceans 2.0 API filter: extension=rdi

Processed Data

Ensemble Period (ping averaging)

For RDI ADCP data file products (MAT and netCDF formats)

Ensemble Period:

When selecting any of the ensemble periods, this option will cause the search to perform the standard box-car average resampling on the data. 'Boxes' of time are defined based on the ensemble period, e.g. starting every 15 minutes on the 15s, with the time stamp given as the center of the 'box'. Acoustic pings that occur within that box are averaged and the summary statistics are updated. This process is often called 'ping averaging'. The process uses log scale averaging on the intensity data, which involves backing out the logarithmic scale, compute the weighted average, and then compute the logarithmic scale again. Weighted averages are used when raw files bridge an ensemble period and when the data is already an ensemble or ping average.

New files are started when the maximum records per file is exceeded (usually set to make files that will use less than 1 GB of memory when loaded), or when there is a configuration, device or site changes. In the case where there is data from either side of a configuration change within the one ensemble period, two files will be produced with the same ensemble period, with the same time stamps, but different data. Users may use the ensemble statistics on the number of pings or samples per ensemble to filter out ensembles that do not have enough data. (As an aside, we do this by default with clean averaged scalar data - each ensemble period needs to have at least 70% of it's expected data to be reported as good.)

The default value is no averaging, meaning the data is not altered. This option is only available for MAT and NETCDF files.

File-name mode field

Selecting an ensemble period will add 'Ensemble' followed by the ensemble period. For example '-Ensemble600s'.

Velocity Bin-mapping (tilt compensation EX)

For all RDI ADCP data products (PNG/PNG and MAT and netCDF formats)

This option specifies the bin-mapping processing method to be applied. Bin-mapping is also known as 'depth cell mapping' or 'tilt compensation' or even 'map to vertical'. There are two methods, both correct for tilt effects on ADCP velocity data, while the none option leaves the velocity data as is. For details on the two methods, see the section on correction and rotation of velocities (included below). The 'None' option is the default for Nortek ADCPs since the free version of the manufacturer's software does not apply bin-mapping (a core goal of our data products is to replicate the functionality offered by the manufacturer's software). The 'Nearest vertical bin' is the default for RDI ADCPs as winADCP applies this method for Instrument or Beam co-ordinate data. The 'As configured on the device' option uses the configuration onboard to determine whether to apply bin-mapping, this matches processing on-board the device (for Earth-co-ordinate data, while for Instrument or Beam co-ordinate data winADCP ignores the device configuration and always uses 'Nearest vertical bin'). The best method has been shown to be the linear interpolation method (Ott, 1992).

File-name mode field

The velocity bin-mapping option will be appended to the filename. For example: '-binMapNone', 'binMapLinearInterp', 'binMapNearest'.

Three-beam Solutions (EX)

For all RDI ADCP data products (PNG/PNG and MAT and netCDF formats) 

Three-beam solutions allow computation of velocity from three beams when the fourth beam has been masked or screened to NaN, as described in ADCP Velocity Computation: Correction and Rotation to East-North-Up Co-ordinate System, Three-beam Solutions and Screening (included below). This option allows the user to use the on-device configured value or override it and select whether or not to use three-beam solutions. The default value retains the previous behaviour of ONC data products: off. Only available on Instrument or Beam co-ordinate data. 

File-name mode field

If a value other than the default is used, a '-3beam'<value> will be appended to the file-name, where <value> is the value of the option matching the API filter.

Low Correlation Screen Threshold (WC)

For all RDI ADCP data products (PNG/PNG and MAT and netCDF formats) 

This option allows the user to control the RDI correlation screening step. The default value retains the previous behaviour of ONC data products: a threshold of 64 counts. Beam-velocities that have associated correlation values lower than this threshold are are masked / screened to NaN values. Only available on Instrument or Beam co-ordinate data. The WC command configures this value on-board the device which is then used for on-board processing (Earth co-ordinate data only). ONC data products can use the WC set value to match winADCP output or the user can override it.

File-name mode field

If a value other than the default is used, a '-corr'<value> will be appended to the file-name, where <value> is the value of the option matching the API filter.

Error Velocity Screen Threshold (WE)

For all RDI ADCP data products (PNG/PNG and MAT and netCDF formats) 

This option allows the user to control the RDI error veloctiy screening step. The default value retains the previous behaviour of ONC data products: a threshold of 2 m/s. Final East-North-Up co-ordinate velocities that have associated error velocity values higher than this threshold are masked / screened to NaN values (lower values are more stringent). Available on all data, except for velocities are the result of a three-beam solution, see ADCP Velocity Computation: Correction and Rotation to East-North-Up Co-ordinate System, Three-beam Solutions and Screening (included below) for more information on how three-beam solutions and the error velocites are related. The WE command configures this value on-board the device which is then used for on-board processing (Earth co-ordinate data only). ONC data products can use the WE set value to match winADCP output or the user can override it.

File-name mode field

If a value other than the default is used, a '-errVal'<value> will be appended to the file-name, where <value> is the value of the option matching the API filter.

False Target Threshold Maximum (WA)

For all RDI ADCP data products (PNG/PNG and MAT and netCDF formats) 

This option controls the False Target Detection algorithm, which is also known as the Fish Rejection algorithm. See chapter 7 in the adcp coordinate transformation_Jan10.pdf documentation from RDI. Essentially, the algorithm looks at the echo levels from bins at the same depth/range and if there is a large difference in their levels, it rejects them in two steps: reject one bin (then the 3-beam solution may apply, so it is suggested to use Fish Rejection and Three-beam solutions together), and if that does not resolve the difference, reject all bins at that depth/range. Lower values of the threshold are more stringent. Available on Beam and Instrument co-ordinate data only, Earth co-ordinate data may have had this algorithm to it onboard the device, see the WA command and configuration value. ONC data products can use the WA configuration value to match winADCP output or the user can override it.

File-name mode field

If a value other than the default is used, a '-falseTar'<value> will be appended to the file-name, where <value> is the value of the option matching the API filter.


ADCP Velocity Computation: Correction and Rotation to East-North-Up Co-ordinate System, Three-beam Solutions and Screening

Unable to render {include} The included page could not be found.

MAT Files

MAT files (v7) can be opened using MathWorks MATLAB 7.0 or later. The file contains four structures: meta, adcp, config, and units.

Meta: a structure array containing the following metadata fields:
  • deviceID: A unique identifier to represent the instrument within the Ocean Networks Canada data management and archiving system.
  • creationDate:Date and time (using ISO8601 format) that the data product was produced. This is a valuable indicator for comparing to other revisions of the same data product.
  • deviceName: A name given to the instrument.
  • deviceCode: A unique string for the instrument which is used to generate data product filenames.
  • deviceCategory: Device category to list under data search ('Echosounder').
  • deviceCategoryCode: Code representing the device category. Used for accessing webservices, as described here: API / webservice documentation (log in to see this link).
  • lat: Fixed value obtained at time of deployment. Will be NaN if mobile or if both site latitude and device offset are null. If mobile, sensor information will be available in mobilePositionSensor structure..
  • lon: Fixed value obtained at time of deployment. Will be NaN if mobile or if both site longitude and device offset are null. If mobile, sensor information will be available in mobilePositionSensor structure.
  • depth: Fixed value obtained at time of deployment. Will be NaN if mobile or if both site depth and device offset are null. If mobile, sensor information will be available in mobilePositionSensor structure.
  • deviceHeading: Fixed value obtained at time of deployment. Will be NaN if mobile or if both site heading and device offset are null. If mobile, sensor information will be available in mobilePositionSensor structure.
  • devicePitch: Fixed value obtained at time of deployment. Will be NaN if mobile or if both site pitch and device offset are null. If mobile, sensor information will be available in mobilePositionSensor structure.
  • deviceRoll: Fixed value obtained at time of deployment. Will be NaN if mobile or if both site roll and device offset are null. If mobile, sensor information will be available in mobilePositionSensor structure.
  • siteName: Name corresponding to its latitude, longitude, depth position.
  • stationCode: Code representing the station or site. Used for accessing webservices, as described here: API / webservice documentation (log in to see this link).
  • locationName: The node of the Ocean Networks Canada observatory. Each location contains many sites.
  • dataQualityComments: In some cases, there are particular quality-related issues that are mentioned here. This is distinct from QAQC information contained in the data structure.
  • MobilePositionSensor.name: A cell array of sensor names for mobile position sensors. If not a mobile device, this will be an empty cell string.
  • MobilePositionSensor.sensorID: An array of unique identifiers of sensors that provide position data for mobile devices - this data may be used in this data product.
  • MobilePositionSensor.deviceID: An array of unique identifiers of sensors that provide position data for mobile devices - this data may be used in this data product.
  • MobilePositionSensor.dateFrom: An array of datenums denoting the range of applicability of each mobile position sensor - this data may be used in this data product.
  • MobilePositionSensor.dateTo: An array of datenums denoting the range of applicability of each mobile position sensor - this data may be used in this data product.
  • MobilePositionSensor.typeName: A cell array of sensor names for mobile position sensors. If not a mobile device, this will be an empty cell string. One of: Latitude, Longitude, Depth, COMPASS_SENSOR, Pitch, Roll.
  • MobilePositionSensor.offset: An array of offsets between the mobile position sensors' values and the position of the device (for instance, if cabled profiler has a depth sensor that is 1.2 m above the device, the offset will be -1.2m).
  • MobilePositionSensor.sensorTypeID: An array of unique identifiers for the sensor type.
  • MobilePositionSensor.correctedSensorID: An array of unique identifiers of sensors that provide corrected mobile positioning data. This is generally used for profiling deployments where the latency is corrected for: CTD casts primarily.
  • deploymentDateFrom: The date of the deployment on which the data was acquired.
  • deploymentDateTo: The date of the end of the deployment on which the data was acquired (may be omitted if still deployed).
  • samplingPeriod: Sampling rate of the instrument in seconds (maybe omitted on some devices that have no scalar sensors).
  • samplingPeriodDateFrom: matlab datenum of the start of the corresponding sample period (maybe omitted on some devices that have no scalar sensors).
  • samplingPeriodDateTo: matlab datenum of the end of the corresponding sample period (maybe omitted on some devices that have no scalar sensors).
  • searchID: unique number tracking this search request (not normally included).
  • Attribution: A structure array with Attribution information, ordered by importance and date. For internal users, go to the Network Console to configure the attributions. If an organization has more than one role it will be collated. If there are gaps in the date ranges, they are filled in with the default Ocean Networks Canada citation. If the "Attribution Required?" field is set to "No" on the Network Console then the citation will not appear. For data products with a attribution (except MAT files) and for users making products from a MAT file, if the special attribution is blank/null, then the company default attribution will be used and if it is also blank/null, then the final attribution will consist of the organization name and role: "Ocean Networks Canada (Owner, Collaborator)". Here are the fields:
    • acknowledgement: the acknowledgement text, note that if the special acknowledgement blank/null, the default acknowledgement is used.
    • startDate: datenum format
    • endDate: datenum format
    • organizationName
    • organizationRole


adcp: structure containing the ADCP data, having the following fields.

  • range: vector of distance to each bin centre. If bin mapping compensation for tilt is active, then the bins are moved so that ADCP.range is now the vertical range to the corrected bin centres (ADCP.range values are not changed by bin-mapping). See adcp.processingComments to see if bin mapping was applied. To calculate the depth of each velocity bin, use the following:  

    if strcmp(Config.orient, 'Up')
        binDepth = Meta.depth - ADCP.range;
    else
        binDepth = Meta.depth + ADCP.range;
    end

  • corr: 3D matrix, correlation time-series for each bin
  • intens: 3D matrix, intensity time-series for each bin (also known as received signal strength intensity or RSSI, multiply by 0.45 dB/count to convert to a relative dB scale)
  • velocity: 3D matrix, corresponds directly to output of instrument and so depends on configuration coordinate system
  • percentGood: 3D matrix, percent good time-series for each bin
  • ens: vector, ensemble number (replaced ensFirst for averaged products)
  • ensFirst: vector of the first ensemble number in an ensemble average (averaged products only, replaces ens).
  • compassHeading: vector, containing the EH values from the RDI files. This is either the fixed value from site and sitedevice headings (if not null), or values from an assigned true heading mobile position sensor. If neither, then it defaults to the magnetic compass heading data.
  • pitch: vector, containing the EP values from the RDI files. This is either the fixed value from site and sitedevice headings (if not null), or values from an assigned true heading mobile position sensor. If neither, then it defaults to the onboard pitch sensor data.
  • roll: vector, containing the ER values from the RDI files. This is either the fixed value from site and sitedevice headings (if not null), or values from an assigned true heading mobile position sensor. If neither, then it defaults to the onboard roll sensor data.
  • time: vector, timestamp in datenum format (obtained from time the reading reached the shore station)
  • temperature: vector, temperature time-series
  • salinity: vector salinity time-series, (may contain constant values depending on device configuration)
  • pressure: vector, pressure time-series. From RDI manual: "Contains the pressure of the water at the transducer head relative to one atmosphere (sea level). Output is in decapascals". Converted to decibar.
  • depth: vector, depth of the device below the water surface as measured by the device for each ping. Can be set to a fixed value by the ED command / device attribute. For fixed deployments, this value is replaced in the RDI file by our code with the site depth plus offset, which is also reported as meta.depth. If not a fixed location deployment where the device is autonomous or mobile, this value is unaltered. It will vary with the tide and movement.
  • soundSpeed: vector speed of sound time-series, (may contain constant values depending on device configuration)
  • uMagnetic (optional): 2D matrix, East velocity relative to magnetic North
  • vMagnetic (optional): 2D matrix, North velocity relative to magnetic North
  • processingCommments: a string documenting all of the processing steps done to produce adcp.u/v/w. Use char(adcp.processingComments) to print out the text.
  • processingOptions: a structure with the data product option set that was requested and what was actually done (not all options are available and maybe overriden at run time, this will be explained in the processingComments).
  • velocity_sourceInfo: a string describing the source of data in adcp.velocity
  • uvw_sourceInfo: a string describing the resulting final, rotated data in adcp.u/v/w
  • u: 2D matrix, East velocity relative to True North
  • v: 2D matrix, North velocity relative to True North
  • w: 2D matrix, Upward Velocity
  • u_std, v_std, w_std: standard deviations for ensemble averaged u,v,w (averaged products only)
  • u_count, v_count, w_count: the number of ensembles/pings that contributed to u,v,w (averaged products only). Differs from pingsPerEnsemble in that it accounts for filtered / NaN-ed data
  • velocityError: 2D matrix, computed using RDI algorithm
  • backscatter: 3D matrix based on received signal strength intensity (adcp.intensity), compensated for two-way spreading (20LogR) and absorption. Equation based on Gostiaux and van Haren ("Extracting Meaningful Information from Uncalibrated Backscattered Echo Intensity Data, Journal of Atomsheric and Oceanic Technology, 72, 943-949, 2010). The absorption computation follows Ainslie and Malcolm ("A simplified formula for viscous and chemical absorption in sea water", Journal of the Acoustical Society of America, 103(3), 1671-1672, 1998). Absorption coefficient based on mean depth, temperature and salinity in adcp structure.
  • meanBackscatter: same as above, except averaged over the four beams to create a 2D matrix. Averaging is done by converting to standard intensity, averaging, then converting back to decibels.

config: structure containing ADCP configuration details, where some field names are appended with '_XX' to represent the corresponding configuration command (beneficial for experienced RDI ADCP users).

  • fwVer: CPU firmware version
  • fwRev: CPU firmware revision
  • sysCfg: hardware configuration definition
  • freq: frequency
  • beamPattern: convex or concave
  • orient: orientation (e.g.,'Up' indicates transducers are looking upward)
  • beamAngle: beam angle
  • janusCfg: description of Janus Configuration
  • lagLength: time period between sound pulses
  • nbeams: number of beams
  • nbins_WN: number of bins
  • npings_WP: number of pings per ensemble
  • cellSize_WS: length per cell
  • profilingMode: sigal processing mode
  • corrThresh_WC: correlation threshold
  • codeReps: code repetitions in transmit pulse
  • percentGoodMin_WG: percent good threshold
  • errVelThreshold_WE: error velocity threshold
  • timePing_TP: time between pings within ensemble
  • coord_EX: coordinate transformation processing parameters
  • coordSys: coordinate system (evaluated from coord_EX)
  • headingAlign_EA: correction factor for physical heading misalignment
  • headingBias_EB: correction factor for electrical/magnetic heading bias
  • heading_EH: manual setting of the heading from device attributes (not in RDI file, added after)
  • sensorSrc_EZ: defines selected source of environmental sensor data
  • sensorAvail: defines available sources of environmental sensor data
  • bin1Dist: distance to the middle of the first depth cell
  • transmitLength_WT: length of the transmit pulse
  • falseTrgt_WA: false target threshold
  • transmitLagDistance: distance between pulse repetitions
  • cpuSN: CPU board SN
  • sysBndwdth_WB: bandwidth setting (narrow or wide)
  • sysPower_CQ: system power setting
  • instSN: instrument serial number
  • ensInterval: ensemble interval
  • soundAbsorptionCoefficient: sound absorption coefficient used for computing backscatter
  • ambiguityVelocity_WV: radial ambiguity velocity (not in RDI, added after)

units: structure containing unit of measure for fields in structures above. For instance, units.pressure='decibar'.

For details about the configuration parameters, refer to the manufacturer documentation (especially the WorkHorse Commands and Output Data Format manual).

Example: RDIADCP75WH3808_20101106T000225.mat

Oceans 2.0 API filter: extension=mat

NETCDF Files

NetCDF is a machine-independent data format offered by numerous institutions, particularly within the earth and ocean science communities. Additional resources are noted here.

The NetCDF file is extracted from the data contained in the above MAT file. The NetCDF file contains the following variables: u, v, w, time, depth, latitude, longitude and temperature. Important: the depth variable here is distinct from the pressure sensor measurement in the MAT file (adcp.depth). The NetCDF depth variable is is calculated as meta.depth - adcp.range; it is the approximate water depth at which the measurement bins are located. It designated as the Z-axis for plotting purposes. The advantage of using the near-constants meta.depth and adcp.range in the calculation is that plots made will be consistent. For mobile RDI ADCPs, it is necessary to use the adcp.depth and adcp.range variables for plotting. Unfortunately, the CF convention does not permit these additional variables, so use the MAT file in this rare case.

Example: BC_POD3_ADCP_20120214T175028.546Z.nc

Oceans 2.0 API filter: extension=nc

Discussion

To comment on this product, click Add Comment below.

  • No labels