Nortek Time Series
Data files for Nortek devices (ADCPs, current meters, velocimeters) are described here. Daily current plots are also available for some Nortek devices. Parsed scalar ancillary data (e.g., temperature, pitch, roll) are also available as time series scalar data and time series scalar plots.
Oceans 3.0 API filter: dataProductCode=NTS
Revision History
- 20211006: netCDF file format update: added ancillary data (e.g. Beam Velocity, Correlations, Intensity), apply compression, and change format to netcdf4 classic.
- 20161101: Support for Signature 55 ADCPs
- 20150601: Added bin-mapping option (none, nearest vertical, linear).
- 20150226: Change range to be to the beam centre, add bin mapping to correct velocities and range for tilt. Add all other Nortek raw binary formats (VEC, VNO, AQD, WPR).
- 20150202: Heading correction rework (affecting ENU / uvw velocities): improve handing of mobile sensors, autonomous deployments and better documentation of processing was done to the data.
- 20111019: Initial NetCDF, CF1.5 compliant product released.
- 20110302: Correction to algorithm for obtaining ENU (with respect to true North velocities). If instrument is mobile, a combination of instrument compass and local magnetic declination is used. If instrument is fixed position, the orientation determined at instrument deployment is used.
- 20110110: Initial MAT product released
- 20101130: Initial PRF product released
Formats
This data is available in manufacturers' raw binary data files (PRF / AQD / VEC / VNO / WPR / AD2CP), and parsed and processed time series data files (MAT and NETCDF). Content descriptions are provided below.
To produce these files, the following requirements apply:
- A new file is started at the start of each day, when the maximum records per file is exceeded (86400 for MAT and NETCDF, 86400 for PRF / AQD / VEC / VNO / WPR / AD2CP, or when the driver is restarted (this should account for configuration changes, site changes, etc).
- Only records with valid checksums are included.
- Only records that have a valid Nortek data structure ID, including configuration data structures and structures that may not normally be associated with file extension (Nortek software appears to handle this ok. We don't filter out valid data structures as firmware updates may cause a structure to be used by a different device). List of valid data structures was taken from the system integrator manual, version Dec. 2014 and the Signature Series System Integrator Guide.
- The instrument date/time field is replaced by the ONC timestamp at the beginning of the log file (since this timestamp is more accurate than the instrument clock).
All Nortek devices are sensitive to orientation. Current meters and velocimeters' data are parsed into scalar data, and part of that process includes a rotation from instrument or Earth relative velocities to true East, North and Up velocities. Data from the manufacturers' raw binary files (PRF / AQD / VEC / VNO / WPR) is as is: no rotation or processing is done. Users should use the appropriate post-processing software to rotate the data to correct for orientation. The MAT and NETCDF processed data products described further below are generated correction for orientation that is described in detail below. If any configurations are changed, files will break. In particular, Nortek Signature 55 ADCP may operate at two different frequencies, so two sets of processed time series products will be generated, while the raw binary data file supports the two frequencies.
Raw Binary Data Files: PRF / AQD / VEC / VNO / WPR / AD2CP
These very similar binary formats are specific to Nortek acoustic devices. When using Nortek data acquisition software, data is normally stored in this way. Although we use custom-built drivers to communicate with our instruments, we can use the raw data in the log file to produce these files. The aforementioned requirements apply, as well as:
- After replacing the instrument date/time field with the ONC timestamp, the checksum is recalculated.
- For all Nortek devices except Signature series: Instrument hardware, head and user configuration strings are inserted as the first three lines of the each raw binary file and important parameters are parsed into MAT file. However, any data existing previous to accompanying configuration details being saved in the database cannot be interpreted (earlier than Fall 2009). We are actively back-filling the configurations, contact us or press the help button if your data product request indicates that the configuration does not exist, even though there is data available.
- For Nortek Signature series: the configuration and command information that you'd normally find at at the start of an AD2CP file are reconstructed from the device configuration which is managed and stored in the device attributes. See here http://dmas.uvic.ca/DeviceListing?DeviceId=22925 and go the additional attributes for an example. This configuration and command data is presented in the AD2CP file as a string data record, as per the Signature Series System Integrator Guide.
VEC is the format for Nortek Vector Current Meters, VNO is the format for Nortek Vectrino Current Meters, AQD is the format for Nortek Aquadopp Current Meters, PRF is the format for Nortek ADCPs, both the Aquadopp Current Profiler and the Aquadopp-HR Current Profiler, WPR is the format for Nortek AWAC (measures currents and waves, see Nortek's description here), AD2CP is the format for Nortek Signature Series ADCPs (viewable in Signature Viewer is a license key). Go to each of those links for the appropriate manufacturer's software for each. Each type of Nortek device has it's own software, however, they all (except Signature Series) have the same data conversion function that appears to read all of the above formats and data structures (however, it's probably best to use the appropriate software). In the appropriate Nortek device interface software, go to the 'Deployment' menu, select 'Data Conversion...', then 'Add file' and then press the arrow. Text file conversions of the raw binary data will be produced. Please note, the raw binary formats are not the .dep files the Nortek device interface software will attempt to load on click of the open file icon or on selecting File -> Open. The .dep files are deployment configuration files useful for field operations. The raw binary files are produced from live devices when the operator selects Deployment -> Recorder Data Retrieval, which is the functionality that our system emulates.
Nortek also offers post-processing software, such as Surge, Prof2NDP, bin2mat, etc., that will also read these formats, see here for downloads (some require license): http://www.nortek-as.com/en/support/software The Surge software will analyze and plot the velocity data, including doing bin-mapping correction, however it is not free software. The free post-processing option, other than our processed MAT and NetCDF files and plot products, is a combination of Prof2NDP and ExploreP. To post-process the data with Prof2NDP and ExploreP, begin by downloading and installing Prof2NDP and ExploreP from the Nortek website. Get the PRF or other raw binary files from data search, convert the file with Prof2NDP and open the resulting .ADP file in ExploreP. To perform the co-ordinate transform on the velocity data from beam or instrument co-ordinates, go to Contour Plot -> Plot Param... choose East/West or N/S or Up/Down, then click on the icon that says 'insert profile graph' or 'insert time series graph'. Users can then click on the graph to chose the time or bin of the profile, then export the post-processed velocity data via the file menu.
These formats are further described in the manufacturer's System Integrator Guide and Signature Series System Integrator Guide.
Oceans 3.0 API filter: extension={prf,aqd,vec,vno,wpr,ad2cp}
Example: BH_POD2_AD2M_20101018T160041.704.prf
Errors and Warnings when Processing Raw Binary Files with Nortek Device Interface Software (Except for Signature Series)
When reading and converting the raw binary data files with Nortek Software, users may encounter some warnings. In our testing, we've come across the following warnings and messages:
In the .ssl file that was was generated by Aquadopp from the AQD file, we saw some warnings about out of range pitch and roll sensor data. This could be caused by a bad configuration or orientation, most likely test data, or this can occur if the orientation of the device is too extreme, as could happen if the device was deployed to a mooring or mobile deployment. In the case of the latter, users may want to ignore or mask out this data.
In the Vector software, we often see this warning. It will create a small subset VEC file that appears to duplicate data that was processed normally. The processed data appears to be ok; it is not out of order or otherwise scrambled.
Contact us if you have any questions about these errors.
Processed Time Series Data Files: MAT / NETCDF
Processed time series data files are available for Nortek ADCP-type devices: Aquadopp Current Profiler, Aquadopp-HR Current Profiler, Nortek Signature Series ADCPs. The Nortek current meter type devices are parsed as scalar sensors. The processed data for those devices are available in generic time series scalar data and time series scalar plot data products, and from other scalar data portals such as Plotting Utility.
Data Product Options For ADCPs (Ensemble averaging, bin-mapping and plan)
Note that this included page (Ensemble Averaging, Bin-mapping, Three-beam Solutions and Filtering Options For RDI ADCPs) covers both Nortek and RDI ADCPs
MAT
MAT files (v7) can be opened using MathWorks MATLAB 7.0 or later. The file contains four structures: meta, data, config, and units. For short duration searches (less than one day) or short duration data returned, the files will be concatenated, otherwise, expect one file per day.
data: structure containing the ADCP data, having the following fields.
- time: vector, timestamp in datenum format (obtained from time the reading reached the shore station)
- range: vector of distance to each bin centre. If bin mapping compensation for tilt is active, this is then a vertical range to the bin centres, see data.processingComments to see if bin mapping was applied. (Prior to Feb 26, 2015: vector of distance to start of each bin)
- cellDepth: vector, starting depth of each cell calculated from range and deployment depth (may be inaccurate if instrument is vertically mobile)
- u,v,w: 2D matrices, East/North/Up velocities relative to True North, derived using raw velocity matrix, orientation, coordinate system, transformation matrix and one of a) calculated magnetic declination based on time and location (for mobile adcps) or b) meta.deviceHeading (fixed position ADCPs)
- u_std, v_std, w_std: standard deviations for ensemble averaged u,v,w (averaged products only).
- pingsPerEnsemble: vector, number of pings that contributed to the ensemble (averaged products only).
- velocity: 3D matrix, corresponds directly to output of instrument and so depends on configuration coordinate system
- amplitude: 3D matrix, amplitude time-series for each of the three receivers. From Nortek Technical Note No. 3: " All Nortek systems use the same component family to measure the amplitude for the echo. This component outputs a signal that is referred to as the RSSI and that is proportional to logarithm of the echo strength. The dynamic range of this signal is about 90 dB and it is linear within an accuracy of about 1-2 dB over a range of 70 dB. Inside this range, the scaling factor is about 0.45 counts/dB but with some variation (about 0.40 to 0.47)."
- correlation: 3D matrix, signal correlation coefficient time-series for each of the three receivers (only available for HR-Profilers)
- compassHeading: vector, magnetic compass heading time-series. (Strictly from onboard sensor. See data.processingComments to determine which source was used for rotation: one of internal or external sensor or fixed value.)
- pitch: vector, pitch time-series. (Strictly from onboard sensor. See data.processingComments to determine which source was used for rotation: one of internal or external sensor or fixed value.)
- roll: vector, roll time-series. (Strictly from onboard sensor. See data.processingComments to determine which source was used for rotation: one of internal or external sensor or fixed value.)
- pressure: vector, pressure time-series
- temperature: vector, temperature time-series
- soundSpeed: vector, sound speed time-series (often constant, depending on configuration)
- voltage: vector, battery voltage time-series
- error: vector, error code (see the manufacturer's system integrator guides to decode)
- backscatter: 3D matrix based on received signal strength intensity (adcp.intensity), compensated for two-way spreading (20LogR) and absorption. Equation based on Gostiaux and van Haren ("Extracting Meaningful Information from Uncalibrated Backscattered Echo Intensity Data, Journal of Atmospheric and Oceanic Technology, 72, 943-949, 2010). The absorption computation follows Ainslie and Malcolm ("A simplified formula for viscous and chemical absorption in sea water", Journal of the Acoustical Society of America, 103(3), 1671-1672, 1998). Absorption coefficient is based on mean depth, temperature and salinity in adcp structure.
- meanBackscatter: same as above, except averaged over the four beams to create a 2D matrix. Averaging is done by converting to standard intensity, averaging, then converting back to decibels.
- processingComments: cell array char field of comments, documenting all of the processing steps taken on the data (such as transform from beam co-ordinates to East-North-Up) and the sources of positioning data applied (heading, pitch, roll, depth)
- magneticDeclination: value that can be applied to correct the compassHeading to true North.
- corScreenThresholdUsed: value used for the correlation screen, i.e. data with correlation less than this are screened (replaced with NaN) from the final u,v,w velocities.
- amplitudeVerticalRange: tilt-corrected range for plotting backscatter (calculated as cos( average tilt ) * range).
config: structure containing configuration details, parsed from the binary configuration or string data structures that are placed at the beginning of the raw binary files (as described earlier). For details about the configuration parameters, refer to the manufacturer documentation. * - denotes that these fields are not (yet) populated for Signature Series.
- SN: instrument type and serial number*
- headSN: head serial number*
- boardFrequency: board frequency*
- headFrequency: head frequency
- fwVersion: firmware version*
- hwRevision: hardware revision*
- PICversion: PIC code version*
- avgInterval: time the instrument is actively measuring within the profile interval
- measurementInterval: time between each measurement (i.e., data output rate)
- compassUpdateRate: rate at which compass readings are updated
- nbeams: number of transducers/beams.
- ncells: number of cells
- npings: number of pings per profile
- velocityScaling: scale conversion from raw data to data in mat file - always converts all velocities to m/s, from either mm/s or 0.1 mm/s, by multiplying by 10^(config.velocityScaling). (Nortek Software does this as well).
- beamAngle: angle of beams
- blankingDistance: distance from the sensor head to the start of the first measurement cell. See Nortek forum discussion for calculation details.
- cellSize: size of each cell. See Nortek forum discussion for calculation details.
- coordSys: coordinates system of raw data (BEAM, XYZ or ENU)
- orientation: Up or Down indicates the direction transducers are facing
- transformationMatrix: matrix, used to convert from beam to xyz coordinates
- correlationThreshold: correlation threshold for resolving ambiguities
- rawDataVelocityUnits: records the original unit of measurement for the raw velocity, can be either 'mm/s' or '0.1 mm/s', see velocityScaling above
- soundAbsorptionCoefficient: : sound absorption coefficient used for computing backscatter
- configBit: various configuration flags, see the Nortek documentation for details.
units: structure containing unit of measure for fields in structures above. For instance, units.pressure='decibar'.
Oceans 3.0 API filter: extension=mat
Example: NortekADCP1504_20101022T000021Z.mat
NETCDF
NetCDF is a machine-independent data format offered by numerous institutions, particularly within the earth and ocean science communities. Additional resources are noted here. For short duration searches (less than one day) or short duration data returned, the files will be concatenated, otherwise, expect one file per day.
The Nortek NetCDF file is extracted from the data contained in the above MAT file. The NetCDF file contains the following main variables: u, v, w, time, binmap_depth, where the binmap_depth variable here is distinct from the pressure sensor measurement in the MAT file (data.depth). When there is bin-mapping performed on the data, the binmap_depth is calculated as meta.depth - data.range; it is the approximate water depth at which the measurement bins are located. If bin-mapping is 'none', then the binmap_depth and the depth are equal. depth is calculated as meta.depth - data.amplitudeVerticalRange, which is the range corrected for tilt. depth is used to plot the beam-averaged backscatter and any other beam-averaged values that are not bin-mapped. The structure and content of the netCDF file is very similar to that of the MAT file, including the variable naming. The RDI and Nortek netCDF files are intended to be very similar, even interchangeable with the same structure and definitions.
The NetCDF files are intended to be in compliance with the CF-1.7 standard. Depending on the compliance checker that is used the data may not meet this standard for the following reasons: The units dB are not defined, and the feature type of the data is not defined.
Oceans 3.0 API filter: extension=nc
Example: NORTEKSIGNATURE200066_20191027T000000Z_20191028T000000Z-Ensemble60s_binMapNone_PLAN0.nc
>> ncdisp('C:\Users\ONC\Test\NORTEKSIGNATURE200066_20191027T000000Z_20191028T000000Z-Ensemble60s_binMapNone_PLAN0.nc')
Source:
C:\Users\ONC\Test\NORTEKSIGNATURE200066_20191027T000000Z_20191028T000000Z-Ensemble60s_binMapNone_PLAN0.nc
Format:
netcdf4_classic
Global Attributes:
Conventions = 'CF-1.7'
title = 'Ocean Networks Canada Nortek Profiler Data'
institution = 'Ocean Networks Canada'
source = 'Nortek Profiler 55 kHz'
history = 'Includes data extracted from raw output, minimally processed, and processed data'
references = 'http://www.oceannetworks.ca/'
comment = 'Detailed documentation: https://wiki.oceannetworks.ca/display/DP/22'
processing_comments = 'Using a fixed true heading of 297 degrees from metadata. Using internal pitch sensor. Using internal roll sensor. Raw data matrix, data.velocity, is the beam radial velocities. A correlation screen of 50% was applied. Number of NaN beam-pings before: 0, number after: 259633 (1883520 total). Bin-mapping option set to 'None'. Average or fixed tilt is 2.7376 degrees. data.range is instrument normal range to each bin centre. Transformed and rotated bin-mapped beam radial velocities to true East-North-Up velocities, using instrument transform matrix and afforementioned heading/pitch/roll data for rotation. adcp.backscatter (relative volume backscattter) calculated from 0.45*adcp.amplitude + 20*log10(adcp.range) + 2*config.soundAbsorptionCoefficient*adcp.range, adcp.meanBackscatter calculated by beam-averaging the volume backscatter (accounting for dB scale). adcp.meanBackscatter and adcp.backscatter have units of relative dB. Ensemble file created by box-car average resampling to a ensemble interval of 60 seconds (raw data ensemble interval: 6 seconds). data.amplitude was averaged normally, while data.backscatter and data.meanBackscatter were averaged by first converting the logarithmic dB scale to linear, averaging, then converting back.For information on the various processing steps applied, see https://wiki.oceannetworks.ca/display/DP/22 '
CREATION_DATE = '20210927T201637Z in QA'
time_coverage_start = '20191027T000130Z'
time_coverage_end = '20191027T235530Z'
device_id = 24420
device_heading = 297
platform_depth = 981
site_name = 'CanyonAxis_IP_2018-06'
device_name = 'Nortek Signature55 Current Profiler 200066'
location_name = 'Barkley Canyon'
search_id = 10439788
firmware_version = ''
hardware_revision =
frequency = 55
serial_number = ''
orientation = 'Up'
beam_angle = 20
adcp_setup_measurement_interval_sec = 6
adcp_setup_average_interval_sec = 240
adcp_setup_number_beams = 3
adcp_setup_number_cells = 109
adcp_setup_number_pings = 40
adcp_setup_blanking_distance_meters = 0.2
adcp_setup_cell_size_meters = 10
Dimensions:
time = 1435
depth = 109
latitude = 1
longitude = 1
Variables:
time
Size: 1435x1
Dimensions: time
Datatype: double
Attributes:
sdn_parameter_name = 'time'
long_name = 'time of measurement'
units = 'days since 19700101T000000Z'
axis = 'T'
calendar = 'gregorian'
binmap_depth
Size: 109x1
Dimensions: depth
Datatype: single
Attributes:
sdn_parameter_name = 'depth'
long_name = 'water depth of final velocity measurement bins'
units = 'meters'
axis = 'Z'
positive = 'down'
comment = 'Water depth for the u (east), v (north), w (up), velocityError seawater velocity bins, accounting for bin-mapping'
depth
Size: 109x1
Dimensions: depth
Datatype: single
Attributes:
sdn_parameter_name = 'depth'
long_name = 'water depth of beam-averaged measurement bins, tilt-corrected'
units = 'meters'
axis = 'Z'
positive = 'down'
comment = 'Water depth of measurement bins corrected for tilt, applies to meanBackscatter (beam-averaged backscatter)'
range
Size: 109x1
Dimensions: depth
Datatype: single
Attributes:
sdn_parameter_name = 'range'
long_name = 'range from transducer'
units = 'meters'
axis = 'Z'
comment = 'Range of measurement bins from the transducer, applies to all parameters except: u, v, w, velocityError when bin-mapping is on and meanBackscatter'
latitude
Size: 1x1
Dimensions: latitude
Datatype: single
Attributes:
sdn_parameter_name = 'latitude'
long_name = 'latitude'
units = 'degrees_north'
axis = 'Y'
longitude
Size: 1x1
Dimensions: longitude
Datatype: single
Attributes:
sdn_parameter_name = 'longitude'
long_name = 'longitude'
units = 'degrees_east'
axis = 'X'
u
Size: 109x1435
Dimensions: depth,time
Datatype: single
Attributes:
sdn_parameter_name = 'eastward_sea_water_velocity'
long_name = 'eastward sea water velocity'
units = 'meters/second'
_FillValue = -9999999
v
Size: 109x1435
Dimensions: depth,time
Datatype: single
Attributes:
sdn_parameter_name = 'northward_sea_water_velocity'
long_name = 'northward sea water velocity'
units = 'meters/second'
_FillValue = -9999999
w
Size: 109x1435
Dimensions: depth,time
Datatype: single
Attributes:
sdn_parameter_name = 'upward_sea_water_velocity'
long_name = 'upward sea water velocity'
units = 'meters/second'
_FillValue = -9999999
meanBackscatter
Size: 109x1435
Dimensions: depth,time
Datatype: single
Attributes:
sdn_parameter_nam = 'sound_intensity_level_in_water'
long_name = 'acoustic doppler current profiler beam-averaged corrected acoustic backscatter'
units = 'dB'
_FillValue = -9999999
comment = 'beam averaged and corrected for two-way beam spreading and attenuation, not bin-mapped, use with tilt-corrected water depth'
intens_beam1
Size: 109x1435
Dimensions: depth,time
Datatype: single
Attributes:
sdn_parameter_name = 'signal_intensity_from_multibeam_acoustic_doppler_velocity_sensor_in_sea_water'
long_name = 'acoustic doppler current profiler return signal strength intensity beam 1'
units = 'counts'
_FillValue = -9999999
intens_beam2
Size: 109x1435
Dimensions: depth,time
Datatype: single
Attributes:
sdn_parameter_name = 'signal_intensity_from_multibeam_acoustic_doppler_velocity_sensor_in_sea_water'
long_name = 'acoustic doppler current profiler return signal strength intensity beam 2'
units = 'counts'
_FillValue = -9999999
intens_beam3
Size: 109x1435
Dimensions: depth,time
Datatype: single
Attributes:
sdn_parameter_name = 'signal_intensity_from_multibeam_acoustic_doppler_velocity_sensor_in_sea_water'
long_name = 'acoustic doppler current profiler return signal strength intensity beam 3'
units = 'counts'
_FillValue = -9999999
corr_beam1
Size: 109x1435
Dimensions: depth,time
Datatype: single
Attributes:
sdn_parameter_name = 'beam_consistency_indicator_from_multibeam_acoustic_doppler_velocity_profiler_in_sea_water'
long_name = 'acoustic doppler current profiler correlation magnitude beam 1'
units = 'percent'
_FillValue = -9999999
corr_beam2
Size: 109x1435
Dimensions: depth,time
Datatype: single
Attributes:
sdn_parameter_name = 'beam_consistency_indicator_from_multibeam_acoustic_doppler_velocity_profiler_in_sea_water'
long_name = 'acoustic doppler current profiler correlation magnitude beam 2'
units = 'percent'
_FillValue = -9999999
corr_beam3
Size: 109x1435
Dimensions: depth,time
Datatype: single
Attributes:
sdn_parameter_name = 'beam_consistency_indicator_from_multibeam_acoustic_doppler_velocity_profiler_in_sea_water'
long_name = 'acoustic doppler current profiler correlation magnitude beam 3'
units = 'percent'
_FillValue = -9999999
velocity_beam1
Size: 109x1435
Dimensions: depth,time
Datatype: single
Attributes:
sdn_parameter_name = 'radial_sea_water_velocity_away_from_instrument'
long_name = 'radial velocity beam 1'
units = 'meters/second'
_FillValue = -9999999
velocity_beam2
Size: 109x1435
Dimensions: depth,time
Datatype: single
Attributes:
sdn_parameter_name = 'radial_sea_water_velocity_away_from_instrument'
long_name = 'radial velocity beam 2'
units = 'meters/second'
_FillValue = -9999999
velocity_beam3
Size: 109x1435
Dimensions: depth,time
Datatype: single
Attributes:
sdn_parameter_name = 'radial_sea_water_velocity_away_from_instrument'
long_name = 'radial velocity beam 3'
units = 'meters/second'
_FillValue = -9999999
temperature
Size: 1435x1
Dimensions: time
Datatype: single
Attributes:
sdn_parameter_name = 'sea_water_temperature'
long_name = 'sea water temperature'
units = 'K'
_FillValue = -9999999
pitch
Size: 1435x1
Dimensions: time
Datatype: single
Attributes:
sdn_parameter_name = 'platform_pitch_angle'
long_name = 'pitch'
units = 'degrees'
_FillValue = -9999999
roll
Size: 1435x1
Dimensions: time
Datatype: single
Attributes:
sdn_parameter_name = 'platform_roll_angle'
long_name = 'roll'
units = 'degrees'
_FillValue = -9999999
pressure
Size: 1435x1
Dimensions: time
Datatype: single
Attributes:
sdn_parameter_name = 'sea_water_pressure'
long_name = 'sea water pressure'
units = 'dbar'
_FillValue = -9999999 |
Above example may be out of date. Recent changes include: added compass heading, updated units. |
Discussion
To comment on this product, click Add Comment below.