CODAR Quality Controlled Surface Currents

About CODAR

CODAR (Coastal Ocean Dynamics Applications Radar) is a technology that allows the measurement of surface ocean current velocities at a distance using the Doppler shift of reflected radio waves. The term “CODAR” is frequently used as a generic term, but it is strongly associated with CODAR Ocean Sensors, a California company that manufactures them. A more general term for CODAR is “HF (High-Frequency) Coastal Radar”.

CODAR works on the principle of Bragg scattering from ocean surface waves. Bragg scattering returns a signal to the transmitting antenna only when the ocean waves are moving either directly towards or directly away from the antenna.

Bragg scattering from ocean waves is also highly wavelength-specific; any given system is sensitive only to ocean waves with wavelengths one-half the wavelength of the radio waves it transmits. The Salish Sea / Strait of Georgia CODAR system (device details) used by ONC is a 25 MHz model. A radio frequency of 25 MHz corresponds to a radio wavelength of 12 metres, so the CODAR system is sensitive to ocean wavelengths of 6 metres. If ocean waves of this wavelength are entirely absent, the CODAR system is unable to measure surface currents. In practice, there is virtually always some wave energy in the 6-metre band, so this is generally not a problem. In extremely calm conditions, however, the data quality can be reduced because of the resulting lower signal-to-noise ratio.

The principle of Bragg scattering dictates that virtually all of the radio signal received by a CODAR antenna is being reflected back from ocean waves of a particular wavelength. The reflected signal is Doppler-shifted to higher or lower frequencies, depending on whether the ocean wave is moving, respectively, toward or away from the antenna. In the absence of ocean currents, this Doppler shift gives a measurement of the wave speed, just as a police radar gun measures the speed of a vehicle on the highway. If ocean currents are present, however, the Doppler shift gives a measurement of the wave speed PLUS the speed of the ocean current towards or away from the antenna. Because the speed of deep-water waves is known for a given wavelength, the wave speed can be subtracted from the total measured speed, resulting in a value for the speed of the ocean current alone.

The equations used in calculating ocean current velocities from the Bragg-scattering signal assume that the ocean waves being measured are “deep-water” waves--that is, that their wavelengths are less than twice the water depth. As depth decreases beyond this point, the waves will increasingly take on the character of “shallow-water” waves, and the assumptions used in calculating the ocean currents will become less and less valid. For a 25 MHz system, then, ocean currents measured in water depths of less than 3 metres should be viewed with suspicion. Note that the daily tidal range in the Strait of Georgia is on the order of 3 metres, so this effect will be time-varying for the current deployment.

CODAR Stations and Combiners

The individual CODAR stations are listed as individual devices. For example, the original two CODAR stations were at the Westshore Coal Terminal in Tsawwassen and at the Iona Wastewater Treatment Plant, near Vancouver Airport. The stations are displayed in the search tree under “Westshore Coal Terminal (VCOL)” and “Iona (VION)”, respectively. Each station measures the radial velocities of ocean surface currents; that is, towards and away from the station's antenna. The stations have been placed in locations such that their areas of coverage overlap considerably. In the overlapping region, it is possible to combine the radial data from the two stations into “total” velocities that may be resolved into their true north-south and east-west components. In this way, the stations have radial and raw data, while we list the combined data products on a separate, virtual, device, nominally found in the search tree as "Oceanographic Radar System" at the same level as the station search tree locations (for example: https://data.oceannetworks.ca/DataSearch?location=SOGCS&deviceCategory=OCEANOGRAPHICRADAR).

CODAR Data Products Overview

The CODAR stations and combiner virtual device have different data products available to them. The CODAR stations have manufacturer produced .RUV files (radial surface velocity data) and raw data formats: CODAR Raw RNG Data, CODAR Raw CSS Data, CODAR Raw RNG Data. The CODAR combiner virtual devices have manufacturer produced .TUV files (total surface current data) and CODAR grid files. Both types of devices have manufacturer produced PNG and GIF plots of their respective radial and total surface velocities. Also available are the ONC-developed CODAR Quality-Controlled Surface Currents data products for all stations and combiners, plus a related CODAR Data Availability Plot, showing the areal coverage of data with or without ONC quality control.

Oceans 3.0 API filter: dataProductCode=CODARQCSC

The data formats and processing algorithms are described in detail below.

Revision History

  1. 20191001: Initial CODAR Quality Controlled Surface Currents data product released

Data Product Options

This option is provided for MAT and NetCDF formats, for CODAR Totals/Combiner virtual devices only.

Include Radial Data: Yes

This is the default. All MAT and NetCDF files will include Radial data structures from all applicable CODAR stations.

Oceans 3.0 API filter: dpo_includeRadials=1

Include Radial Data: No (makes files much smaller)

Alternatively, users may use this option if they are only interested in the Totals data and wish to exclude the Radial data from the contributing CODAR stations, The advantage here is file size. Each Totals structure is about the same size as the Radials, so for a 4 station array, the file size will be reduced by 80%, causing the file to break far less often. (Files break automatically at 1 GB of memory occupied, which is larger than size on disk.) With this option selected, MAT and NetCDF files will have "_exRad" appended to their file-names.

Oceans 3.0 API filter: dpo_includeRadials=0

Formats

The CODAR quality controlled surface current data products are processed on-the-fly from the source RUV files (from CODAR stations with radial surface velocities). The formats are described below. All of these data products have filenames that end with _CLEAN.

PNG, PDF and GIF plots

PNG and PDF files contain a graphic representation of both station radial and combined total velocities, depending on what device/station the data product was requested from. Station files with radial currents end in the four letter CODAR site code (e.g. -VCOL_CLEAN), while the combined files (with total velocities) end in -TOTALS_CLEAN. The PDF variant bundles all the PNGs (up to a file size limit) in a single vector-graphics PDF with a filename corresponding to the extents of the PNGs. The GIF variant also bundles the PNG images into an animation with a 1 second frame rate and pause at the end of the animation. The GIF breaks into multiple files at UTC midnight unless the entire set spans less than 24 hours or has 6 or fewer files.

Oceans 3.0 API filter: extension=png,pdf,gif

An example radials plot:

An example totals plot:

An example animated totals GIF:

Note that the above examples are based on test data. Real data is usually better.

MAT and NetCDF processed data files

The processed, quality controlled data used to create the plots are available in these files. Care has been taken to ensure the MAT file structure is compatible with the HFRProgs structures, version 2.1.2. The modifications (detailed far below) and extensions on HFRProgs only add structures/arrays, while maintaining the original HFRProgs structure. (One exception is the Flag sub-structure that was the most recent addition to HFRProgs and was empty, it has been replaced by the QC sub-structure.) Both Totals and Radials, the main structures, are indexed by time, each instance is a separate reading. ONC processing code carefully handles and aligns the Radials time stamps to create the Totals, processing one at time (it's not slow). Consideration is made for time-varying source files: .tuv and .grd files as  well as device attributes that change within or between deployments. The MAT and netCDF files contain an accumulation of data; the file is broken at a size limit based on standard memory limits (~1 GB), similar to how the GIF file accumulates up to 1 day or the PDF accumulates up to ~100 MB of data. Radials structures may be excluded optionally. MAT files can contain both Totals and Radials structures, whereas the netCDF file cannot easily contain both, so separate files are created with -Radials and -Totals modifiers in the file-names. The structure of the netCDF file is closely related to the structure of the MAT file.

Oceans 3.0 API filter: extension=mat,nc

MAT file data structures and definitions

This section describes the MAT file structure in detail. HFRProgs's README and code also documents these structures, so only relevant ones will be listed in detail.

Radials: a structure array containing radial surface current data. Each instance of the structure is a scan at a particular time. Details as follows:

  • U/V: surface current radial velocities as read from the .ruv files and subsequently tested and cleaned
  • LonLat: :,2 vector containing longitude and latitude pairs for each U/V value. Note: U/V are not on a grid, but are evenly spaced along radial beams.
  • Error: can be taken from the ETMP or STDV column from the RUV file (may be deprecated in the future)
  • temporalQuality: is the ETMP column from the RUV file, so it's usually a duplicate of Error
  • spatialQuality: is the ESPC column from the RUV file
  • OtherMetadata: this structure was modified from HFRProgs. We moved the ruv file Header and RawData into the Radials structure, see the next two entries:
  • ruvHeader: cell strings of the ruv file header
  • ruvRawData: matrix corresonding to the raw data body of the ruv file
  • ProcessingSteps: a cell array of char vectors (strings) reporting on the various details of the processing that was done. Tip: to get a print out of the comments, try this: char(Radials(1).ProcessingSteps) 

Radials.QC: a structure containing the overallFlag QC flag and any other flag generated by the radial tests and cleaning. Flag values of 1 are a pass, 4 is a failure/bad. The flag vectors all match the size of the U/V vectors. We've removed the Radials.Flag vector in our code as the QC structure supersedes it. QC also contains all of the test parameters, named with the same test name as the flag appended by 'TestThreshold'. Unless noted in the documentation, the tests pass if the value is less than this threshold. All test thresholds are stored as device attributes and vary by device and time. A sub-structure contains the results of the .ruv file syntax test: CheckRDLsyntax, each value in contained within describes each check and it's result, all of which must pass for the data to be included in the Radials and Totals structures. Failures are noted in the search status (visible in the Data Search cart). Syntax failures should not happen.


Totals: a structure containing total surface data:

  • U/V: surface current total velocities calculated from the Radials structure
  • LonLat: :,2 vector containing longitude and latitude pairs for each U/V value. Note: in-order to calculate Totals, a grid is required, as such these values are on a regular spatial grid defined by the accompanying grid structure. Values are only located the Grid.nodesOcean grid centres.
  • Uraw/Vraw: surface current total velocities as generated by loading radials, compiling by time and passing to the same configuration of makeTotals.m as done for the clean U/V. So no cleaning is done on Uraw/Vraw and the values should be very similar to the manufacturer's U/V, but may not be exactly the same due to different configuration values in makeTotals.m. These velocities may be used as a comparison to see the difference the cleaning/QC process has made to the data.

Totals.ErrorEstimates: struct array of the test results for each algorithm. One note-able difference here is that the GDOP test has a value in OtherMatrixVars. These are the new test values calculated as GDOPerror = sqrt(T.ErrorEstimates(i).Uerr + T.ErrorEstimates(i).Verr) while the test values are taken as the TotalErrors for all other tests.

Totals.OtherMetadata.makeTotals.parameters: this structure contains the parameter values used by makeTotals.m. This is unchanged from HFRProgs, other than all the 'WhichErrors' are used, the MinNumSites, MinNumRads, spatthresh, tempthresh are provided by the Oceans 3.0 device attributes, which are variable for each device and each time stamp.

Totals.QC: a structure containing the overallFlag QC flag and any other flag generated by the Totals tests and cleaning. Flag values of 1 are a pass, 4 is a failure/bad. The flag vectors all match the size of the U/V vectors. We've removed the Totals.Flag vector in our code as the QC structure supersedes it. QC also contains all of the test parameters, named with the same test name as the flag appended by 'TestThreshold'. Unless noted in the documentation, the tests pass if the value is less than this threshold. Note the test thresholds with values of 'Inf' are effectively not used, but are available if ONC Data Quality Specialists modify the device attributes with values to make use of these tests. All test thresholds are stored as device attributes and vary by device and time.

Grid: a structure containing all the information from accompanying .grid files primarily for CODAR combiner stations. It will also appear for CODAR radial stations where the .grid files are available as described in the CODAR grid files documentation.

  • headerNames/Values: pairs of parameter names and values, containing information such as grid spacing, grid origin, etc.
  • flags: indicate if a node is on the ocean or otherwise not used
  • nodes: longitude, latitude values at each grid node
  • nodesIndex: grid index for each node
  • nodesOcean: sub-scale grid containing only nodes that aren't flagged, i.e. nodes that are applicable and on the Ocean.
  • lat/lonLimitsOceans: extents of applicable nodes on the Ocean.

Meta: a structure array containing the following metadata fields:

  • deviceID: A unique identifier to represent the instrument within the Ocean Networks Canada data management and archiving system.
  • creationDate:Date and time (using ISO8601 format) that the data product was produced. This is a valuable indicator for comparing to other revisions of the same data product.
  • deviceName: A name given to the instrument.
  • deviceCode: A unique string for the instrument which is used to generate data product filenames.
  • deviceCategory: Device category to list under data search ('Echosounder').
  • deviceCategoryCode: Code representing the device category. Used for accessing webservices, as described here: API / webservice documentation (log in to see this link).
  • lat: Fixed value obtained at time of deployment. Will be NaN if mobile or if both site latitude and device offset are null. If mobile, sensor information will be available in mobilePositionSensor structure..
  • lon: Fixed value obtained at time of deployment. Will be NaN if mobile or if both site longitude and device offset are null. If mobile, sensor information will be available in mobilePositionSensor structure.
  • depth: Fixed value obtained at time of deployment. Will be NaN if mobile or if both site depth and device offset are null. If mobile, sensor information will be available in mobilePositionSensor structure.
  • deviceHeading: Fixed value obtained at time of deployment. Will be NaN if mobile or if both site heading and device offset are null. If mobile, sensor information will be available in mobilePositionSensor structure.
  • devicePitch: Fixed value obtained at time of deployment. Will be NaN if mobile or if both site pitch and device offset are null. If mobile, sensor information will be available in mobilePositionSensor structure.
  • deviceRoll: Fixed value obtained at time of deployment. Will be NaN if mobile or if both site roll and device offset are null. If mobile, sensor information will be available in mobilePositionSensor structure.
  • siteName: Name corresponding to its latitude, longitude, depth position.
  • locationName: The node of the Ocean Networks Canada observatory. Each location contains many sites.
  • stationCode: Code representing the station or site. Used for accessing webservices, as described here: API / webservice documentation (log in to see this link).
  • dataQualityComments: In some cases, there are particular quality-related issues that are mentioned here.
  • MobilePositionSensor: A structure with information about sensors that provide additional scalar data on positioning and attitude (latitude, longitidue, depth below sea surface, heading, pitch, yaw, etc).

    • name: A cell array of sensor names for mobile position sensors. If not a mobile device, this will be an empty cell string.
    • sensorID: An array of unique identifiers of sensors that provide position data for mobile devices - this data may be used in this data product.
    • deviceID: An array of unique identifiers of devices that provide position data for mobile devices - this data may be used in this data product.
    • dateFrom: An array of datenums denoting the range of applicability of each mobile position sensor - this data may be used in this data product.
    • dateTo: An array of datenums denoting the range of applicability of each mobile position sensor - this data may be used in this data product.
    • typeName: A cell array of sensor names for mobile position sensors. If not a mobile device, this will be an empty cell string. One of: Latitude, Longitude, Depth, COMPASS_SENSOR, Pitch, Roll.
    • offset: An array of offsets between the mobile position sensors' values and the position of the device (for instance, if cabled profiler has a depth sensor that is 1.2 m above the device, the offset will be -1.2m).
    • sensorTypeID: An array of unique identifiers for the sensor type.
    • correctedSensorID: An array of unique identifiers of sensors that provide corrected mobile positioning data. This is generally used for profiling deployments where the latency is corrected for: CTD casts primarily.
  • deploymentDateFrom: The date of the deployment on which the data was acquired.
  • deploymentDateTo: The date of the end of the deployment on which the data was acquired (will be NaN if still deployed).
  • samplingPeriod: Sample period / data rating of the device in seconds, this is the sample period that controls the polling or reporting rate of the device (some parsed scalar sensors may report faster, some devices report in bursts) (may be omitted for some data products).
  • samplingPeriodDateFrom: matlab datenum of the start of the corresponding sample period (may be omitted for some data products).
  • samplingPeriodDateTo: matlab datenum of the end of the corresponding sample period (may be omitted for some data products).
  • sampleSize: the number of readings per sample period, normally 1, except for instruments that report in bursts. Will be zero for intermittent devices (may be omitted for some data products).
  • SamplePeriodSensor: A structure array with an entry for each scalar sensor on the device (even though this metadata is for complex data products that don't use scalar sensors).

    • sp: sample period in seconds (array), unless sensorid is NaN then this is the device sample period
    • dateFrom: array of date from / start date (inclusive) for each sample period in MATLAB datenum format.
    • dateTo: array of date to / end date (exclusive) for each sample period in MATLAB datenum format.
    • sampleSize: the number of readings per sample period (array). Normally 1, except for instruments that report in bursts. Will be zero for intermittent devices.
    • deviceID: array of unique identifiers of devices (should all be the same).
    • sensorID: array of unique identifiers of sensors on this device.
    • isDeviceLevel: flag (logical) that indicates, when true or 1, if the corresponding sample period/size is from the device-level information (i.e. applies to all sensors and the device driver's poll rate).
    • sensorName: the name of the sensor for which the sample period/size applies (much more user friendly than a sensorID).
  • citation: a char array containing the DOI citation text as it appears on the Dataset Landing Page. The citation text is formatted as follows: <Author(s) in alphabetical order>. <Publication Year>. <Title, consisting of Location Name (from searchTreeNodeName or siteName in ONC database) Deployed <Deployment Date (sitedevicedatefrom in ONC database)>. <Repository>. <Persistent Identifier, which is either a DOI URL or the queryPID (search_dtlid in ONC database)>. Accessed Date <query creation date (search.datecreated in ONC database)>
  • Attribution: A structure array with information on any contributors, ordered by importance and date. If an organization has more than one role it will be collated. If there are gaps in the date ranges, they are filled in with the default Ocean Networks Canada citation. If the "Attribution Required?" field is set to "No" on the Network Console then the citation will not appear. Here are the fields:
    • acknowledgement: the acknowledgement text, usually formatted as "<organizationName> (<organizationRole>)", except for when there are no attributions and the default is used (as shown above).
    • startDate: datenum format
    • endDate: datenum format
    • organizationName
    • organizationRole: comma separated list of roles
    • roleComment: primarily for internal use, usually used to reference relevant parts of the data agreement (may not appear)

netCDF File Description

NetCDF is a machine-independent data format offered by numerous institutions, particularly within the earth and ocean science communities. Additional resources are noted here. NetCDF is intended to be self-describing so only limited additional information is required here.

The CODAR QC netCDF data product largely mirrors the MAT file described in detail above. The differences include the separation of radial and total data into separate files (both may be provided in the default option for combiner stations), naming and metadata reporting. The totals files will have the lat/lon variables and x,y Easting/Northing dimensions as these values come from the CODAR combiner grid (see the Grid structure above). Note that the grid used is the full grid, including flagged locations (so the size of U/V variables is larger in the netCDF format compared to the MAT format or Manufacturer's TUV files). In the radials files, the grid information is only provided if available and is for use in gridding the data (radial data is not gridded). The totals files organize the data into grids with y,x and time dimensions, while the radials files use arrays for the current data, with dimensions of time and index, where index links the currents to their lat/lon sampling locations. The x,y dimensions are the grid centers relative to the grid origin. The totals files will also contain additional QC tests and quality indicators. The radials files from combiner stations will contain data from multiple radials stations; the platform_name variable will supply the station name / call sign (i.e. VION) for each time step. The global attributes will contain many useful fields, including processing_steps that will describe all of the steps and tests applied to the data. 

Source:
           C:\BenTestData\StraitofGeorgia_StraitofGeorgiaCODARSystem_OceanographicRadarSystem_20210603T220000.000Z_20210603T230000.000Z-Radials_Clean.nc
Format:
           netcdf4_classic
Global Attributes:
           acknowledgement                = 'Ocean Networks Canada is an initiative of the University of Victoria'
           id                             = 'BLANK'
           naming_authority               = 'Ocean Networks Canada'
           project                        = 'BLANK'
           processing_level               = 'Quality controlled surface current data'
           time_coverage_duration         = 'BLANK'
           time_coverage_resolution       = 'BLANK'
           standard_name_vocabulary       = 'BLANK'
           publisher_name                 = 'Ocean Networks Canada'
           publisher_url                  = 'https://data.oceannetworks.ca'
           publisher_email                = 'info@oceannetworks.ca'
           title                          = 'Surface Ocean Velocity; Strait of Georgia'
           institution                    = 'Ocean Networks Canada; University of Victoria'
           creator_institution            = 'Ocean Networks Canada'
           creator_name                   = 'Ocean Networks Canada'
           creator_type                   = 'institution'
           creator_email                  = 'info@oceannetworks.ca'
           creator_url                    = 'https://data.oceannetworks.ca'
           date_created                   = 'see searchID 20383822'
           history                        = 'CODAR radial data (hourly, usually) combined into one file containing ocean surface velocity vector'
           references                     = 'https://wiki.oceannetworks.ca/display/DP/148'
           source                         = 'CODAR Seasonde HF-Radar ocean surface current mapping device'
           Conventions                    = 'CF-1.7, ACDD-1.3'
           Metadata_conventions           = 'Unidata Dataset Discovery v1.0'
           cdm_data_type                  = 'Grid'
           geospatial_bounds              = 'BLANK'
           geospatial_bounds_vertical_crs = 'BLANK'
           geospatial_vertical_min        = 0
           geospatial_vertical_max        = 0
           geospatial_vertical_positive   = 'down'
           sea_name                       = 'Salish Sea'
           instrument                     = 'VATK, VCOL, VGPT, VION'
           summary                        = 'Radial velocity data within a specified radius of each grid coordinate are combined by an unweighted least-squares method using a ONC-modified HFR_Progs (v2.1.2) MATLAB package. Duty cycle is usually hourly. A number of QC tests are applied to the total current velocity data to produce a clean (filtered) data set, building on the tests applied to the radial velocity data sets. See https://wiki.oceannetworks.ca/display/DP/148 for complete documentation. In the final product, raw and quality controlled total current velocity data and other variable data are placed on a geospatial grid with easting and northing distance to site origin as dimensions. Processing parameters and related quality control information are also provided.'
           keywords                       = 'BLANK'
           comment                        = 'Detailed documentation: https://wiki.oceannetworks.ca/display/DP/148'
           geospatial_lat_units           = 'degrees east'
           geospatial_lat_min             = -124.003
           geospatial_lat_max             = 49.4272
           geospatial_lon_units           = 'degrees north'
           geospatial_lon_min             = -124.003
           geospatial_lon_max             = -122.7348
           time_coverage_start            = '29590526T235959Z'
           time_coverage_end              = '29690404T000000Z'
           attrbution                     = 'Ocean Networks Canada Data Archive, http://www.oceannetworks.ca, University of Victoria, Canada, Oceanographic Radar System data from 03-Jun-2021 22:00:00 to 03-Jun-2021 23:59:59'
           processing_steps               = 'Radial velocity measurements are obtained from individual radar sites via manufacturer's software and archived in Oceans 3.0 (usually hourly). Upon a user requesting data, a number of QC tests are applied to the raw data to produce a clean (filtered) data set. The test thresholds/parameters are recorded below and in the Radials. QC structure. See https://wiki.oceannetworks.ca/display/DP/148 for complete documention. 
                                            loadRDLfile loaded radial velocity measurements from ruv files, obtained from individual radar sites via manufacturer's software and archived in Oceans 3.0. 
                                            QARTOD Test 6: Syntax - a collection of tests ensuring proper formatting and existence of fields within a radial file. checkRDLsyntax.m passed all checks on the RUV file header. 
                                            QARTOD Test 7: Max Threshold (radials) - Flags radial surface current speed exceeding the maximum threshold of 150 cm/s. cleanRadialsVel.m cleaned * data points exceeding the maximum radial current speed. 
                                            QARTOD Test 8: Valid Location (radials) - Flags radial surface current velocities placed over land or other invalid measurement areas. cleanRadialsLoc.m cleaned * data points with a VectorFlag value of 128. 
                                            Additional test: Temporal Quality - Flags radial surface current velocities with Radials.Error/temporalQuality (EMTP or STDV in the raw data) equal to a value of 999. cleanRadialsTemp.m cleaned * data points. 
                                            Additional test: Spatial Quality - Flags radial surface current velocities with Radials.spatialQuality (ESPT in the raw data) equal to a value of 999. cleanRadialsSpat.m cleaned * data points. 
                                            * see the relevant flag variable to determine the number of data points cleaned at each step. '
Dimensions:
           time           = 8
           x              = 93
           y              = 74
           index          = 1511
           siteNameLength = 4
Variables:
    time        
           Size:       8x1
           Dimensions: time
           Datatype:   double
           Attributes:
                       _FillValue         = -999
                       sdn_parameter_name = 'time'
                       long_name          = 'Time of measurement UTC'
                       units              = 'seconds since 1970-01-01 00:00:00'
                       axis               = 'T'
                       calendar           = 'gregorian'
    x           
           Size:       93x1
           Dimensions: x
           Datatype:   double
           Attributes:
                       long_name = 'Easting distance from the CODAR grid origin'
                       units     = 'km'
                       valid_min = '-46'
                       valid_max = '46'
                       comment   = 'Grid origin coordinates: 49.0675 degrees latitude, -123.3689 degrees longitude. CODAR total ocean surface current velocity grid spacing (in km): 1'
    y           
           Size:       74x1
           Dimensions: y
           Datatype:   double
           Attributes:
                       long_name = 'Northing distance from the CODAR grid origin'
                       units     = 'km'
                       valid_min = '-33'
                       valid_max = '40'
                       comment   = 'Grid origin coordinates: 49.0675 degrees latitude, -123.3689 degrees longitude. CODAR total ocean surface current velocity grid spacing (in km): 1'
    index       
           Size:       1511x1
           Dimensions: index
           Datatype:   int32
           Attributes:
                       long_name = 'Index of radial velocity samples'
                       units     = ''
                       comment   = 'Dimension representing the sample number for the radial data arrays'
    site_name   
           Size:       8x4
           Dimensions: time,siteNameLength
           Datatype:   char
           Attributes:
                       sdn_parameter_name = 'platform_name'
                       long_name          = 'Site name at time index'
                       units              = ''
                       comment            = 'Provides the siteName to distinguish the source of combined radial data at each time index'
    lon         
           Size:       74x93
           Dimensions: y,x
           Datatype:   double
           Attributes:
                       sdn_parameter_name = 'longitude'
                       long_name          = 'Longitude'
                       units              = 'degrees_east'
                       axis               = 'X'
                       valid_min          = -180
                       valid_max          = 180
                       reference_datum    = 'geographical coordinates, WGS84 datum'
    lat         
           Size:       74x93
           Dimensions: y,x
           Datatype:   double
           Attributes:
                       sdn_parameter_name = 'latitude'
                       long_name          = 'Latitude'
                       units              = 'degrees_north'
                       axis               = 'Y'
                       valid_min          = -90
                       valid_max          = 90
                       reference_datum    = 'geographical coordinates, WGS84 datum'
    lon_sample  
           Size:       8x1511
           Dimensions: time,index
           Datatype:   double
           Attributes:
                       sdn_parameter_name = 'longitude'
                       long_name          = 'Longitude at each radial velocity sampling location'
                       units              = 'degrees_east'
                       axis               = 'X'
                       valid_min          = -180
                       valid_max          = 180
                       reference_datum    = 'geographical coordinates, WGS84 datum'
    lat_sample  
           Size:       8x1511
           Dimensions: time,index
           Datatype:   double
           Attributes:
                       sdn_parameter_name = 'latitude'
                       long_name          = 'Latitude at each radial velocity sampling location'
                       units              = 'degrees_north'
                       axis               = 'Y'
                       valid_min          = -90
                       valid_max          = 90
                       reference_datum    = 'geographical coordinates, WGS84 datum'
    u           
           Size:       8x1511
           Dimensions: time,index
           Datatype:   double
           Attributes:
                       _FillValue         = -999
                       sdn_parameter_name = 'surface_eastward_sea_water_velocity'
                       long_name          = 'Quality controlled eastward component of ocean surface radial velocity'
                       units              = 'cm/s'
                       valid_min          = -150
                       valid_max          = 150
                       ancillary_variable = 'quality_flag'
                       coordinates        = 'time lon lat'
                       comment            = 'Quality controlled eastward component of ocean surface radial velocity.'
    v           
           Size:       8x1511
           Dimensions: time,index
           Datatype:   double
           Attributes:
                       _FillValue         = -999
                       sdn_parameter_name = 'surface_northward_sea_water_velocity'
                       long_name          = 'Quality controlled northward component of ocean surface radial velocity'
                       units              = 'cm/s'
                       valid_min          = -150
                       valid_max          = 150
                       ancillary_variable = 'quality_flag'
                       coordinates        = 'time lon lat'
                       comment            = 'Quality controlled northward component of ocean surface radial velocity.'
    u_raw       
           Size:       8x1511
           Dimensions: time,index
           Datatype:   double
           Attributes:
                       _FillValue         = -999
                       sdn_parameter_name = 'surface_eastward_sea_water_velocity'
                       long_name          = 'Eastward component of ocean surface raw radial velocity'
                       units              = 'cm/s'
                       ancillary_variable = 'quality_flag'
                       coordinates        = 'time lon lat'
                       comment            = 'Eastward component of ocean surface raw radial velocity.'
    v_raw       
           Size:       8x1511
           Dimensions: time,index
           Datatype:   double
           Attributes:
                       _FillValue         = -999
                       sdn_parameter_name = 'surface_northward_sea_water_velocity'
                       long_name          = 'Northward component of ocean surface raw radial velocity'
                       units              = 'cm/s'
                       ancillary_variable = 'quality_flag'
                       coordinates        = 'time lon lat'
                       comment            = 'Northward component of ocean surface raw radial velocity.'
    quality_flag
           Size:       8x1511
           Dimensions: time,index
           Datatype:   double
           Attributes:
                       _FillValue         = -999
                       sdn_parameter_name = 'quality_flag'
                       long_name          = 'Overall quality flags for total ocean surface current velocity'
                       flag_meanings      = 'No_quality_control_performed Good_Data Data_probably_good Data_bad_but_potentially_correctable Bad_data Insufficient_valid_data_for_reliable_down_sampling Averaged_value Interpolated_value Missing_value'
                       valid_min          = 0
                       valid_max          = 9
                       flag_values        = [0  1  2  3  4  6  7  8  9]
                       coordinates        = 'time lon lat'
                       comment            = 'Overall quality flags for eastward and northward ocean surface current velocities. For details about quality control tests applied see processing_parameters variable. With more details on ONC's flag meanings: 0 - No quality control performed, 1 - Good. Data passes all tests, 2 - Data probably good, 3 - Data bad but potentially correctable, 4 - Bad data, 6 - Insufficient valid data for reliable down-sampling, 7 - Averaged value, 8 - Interpolated value, 9 - Missing value.'
    crs         
           Size:       8x1
           Dimensions: time
           Datatype:   int32
           Attributes:
                       long_name          = 'Coordinate Reference System'
                       grid_mapping_name  = 'latitude_longitude'
                       epsg_code          = 'EPSG:4326'
                       semi_major_axis    = 6378137
                       inverse_flattening = 298.2572
                       comment            = 'http://www.opengis.net/def/crs/EPSG/0/4326'


Source:
           C:\Users\bbiffard\Downloads\StraitofGeorgia_StraitofGeorgiaCODARSystem_OceanographicRadarSystem_20210603T220000.000Z_20210603T230000.000Z-Totals_Clean.nc
Format:
           netcdf4_classic
Global Attributes:
           acknowledgement                = 'Ocean Networks Canada is an initiative of the University of Victoria'
           id                             = 'BLANK'
           naming_authority               = 'Ocean Networks Canada'
           project                        = 'BLANK'
           processing_level               = 'Quality controlled surface current data'
           time_coverage_duration         = 'BLANK'
           time_coverage_resolution       = 'BLANK'
           standard_name_vocabulary       = 'BLANK'
           publisher_name                 = 'Ocean Networks Canada'
           publisher_url                  = 'https://data.oceannetworks.ca'
           publisher_email                = 'info@oceannetworks.ca'
           title                          = 'Surface Ocean Velocity; Strait of Georgia'
           institution                    = 'Ocean Networks Canada; University of Victoria'
           creator_institution            = 'Ocean Networks Canada'
           creator_name                   = 'Ocean Networks Canada'
           creator_type                   = 'institution'
           creator_email                  = 'info@oceannetworks.ca'
           creator_url                    = 'https://data.oceannetworks.ca'
           date_created                   = 'see searchID 20383823'
           history                        = 'CODAR radial data (hourly, usually) combined into one file containing ocean surface velocity vector'
           references                     = 'https://wiki.oceannetworks.ca/display/DP/148'
           source                         = 'CODAR Seasonde HF-Radar ocean surface current mapping device'
           Conventions                    = 'CF-1.7, ACDD-1.3'
           Metadata_conventions           = 'Unidata Dataset Discovery v1.0'
           cdm_data_type                  = 'Grid'
           geospatial_bounds              = 'BLANK'
           geospatial_bounds_vertical_crs = 'BLANK'
           geospatial_vertical_min        = 0
           geospatial_vertical_max        = 0
           geospatial_vertical_positive   = 'down'
           sea_name                       = 'Salish Sea'
           instrument                     = 'Network includes active CODAR sites: VATK, VCOL, VGPT, VION'
           summary                        = 'Radial velocity data within a specified radius of each grid coordinate are combined by an unweighted least-squares method using a ONC-modified HFR_Progs (v2.1.2) MATLAB package. Duty cycle is usually hourly. A number of QC tests are applied to the total current velocity data to produce a clean (filtered) data set, building on the tests applied to the radial velocity data sets. See https://wiki.oceannetworks.ca/display/DP/148 for complete documentation. In the final product, raw and quality controlled total current velocity data and other variable data are placed on a geospatial grid with easting and northing distance to site origin as dimensions. Processing parameters and related quality control information are also provided.'
           keywords                       = 'BLANK'
           comment                        = 'Detailed documentation: https://wiki.oceannetworks.ca/display/DP/148'
           geospatial_lat_units           = 'degrees east'
           geospatial_lat_min             = -124.003
           geospatial_lat_max             = 49.4272
           geospatial_lon_units           = 'degrees north'
           geospatial_lon_min             = -124.003
           geospatial_lon_max             = -122.7348
           time_coverage_start            = '29590526T235959Z'
           time_coverage_end              = '29690404T000000Z'
           attrbution                     = 'Ocean Networks Canada Data Archive, http://www.oceannetworks.ca, University of Victoria, Canada, Oceanographic Radar System data from 03-Jun-2021 22:00:00 to 03-Jun-2021 23:59:59'
           processing_steps               = 'Total ocean surface current velocities are computed from clean radial data upon a user request of the data. See the Grid structure for grid spacing and extents on which total velocities are calculated. Hourly radial data within a specified radius of each grid coordinate are combined by an unweighted least-squares method using a ONC-modified HFR_Progs (v2.1.2) MATLAB package. A number of QC tests are applied to the totals data to produce a clean (filtered) data set, building on the tests applied to the Radial data sets. See the Radials structure and Radials.ProcessingSteps for details on those tests. The test thresholds/parameters are recorded below and in the Totals.QC structure. See https://wiki.oceannetworks.ca/display/DP/148 for complete documention. 
                                            makeTotals.m created the Totals data structure with the following parameters: spatthresh: 1.5 km, tempthresh: 0.041667 days, MinNumSites: 2, MinNumRads: 3. 
                                            QARTOD Test 16: Max Threshold (totals) - Flags total surface current speeds exceeding the maximum threshold of 220 cm/s. cleanTotalsVel.m cleaned * data points exceeding the maximum radial current speed. 
                                            QARTOD Test 15: GDOP Threshold (totals) - Flags total surface current velocities with poor geometric relationship between contributing radials. Totals with GDOP > 1.25 are flagged. cleantotalsGDOP.m cleaned * data points. 
                                            Additional test: GDOPMaxOrthog - Flags total surface current velocities exceeding the GDOPMaxOrthog error estimation (TotalErrors) values and a test threshold value of Inf. cleantotalsGDOP.m cleaned * data points. 
                                            Additional test: GDOPRadErr - Flags total surface current velocities exceeding the GDOPRadErr error estimation (TotalErrors) values and a test threshold value of Inf. cleantotalsGDOP.m cleaned * data points. 
                                            Additional test: GDOPSites - Flags total surface current velocities exceeding the GDOPSites error estimation (TotalErrors) values and a test threshold value of Inf. cleantotalsGDOP.m cleaned * data points. 
                                            Additional test: FitDif - Flags total surface current velocities exceeding the FitDif error estimation (TotalErrors) values and a test threshold value of Inf. cleantotalsGDOP.m cleaned * data points. 
                                            Additional test: TUV Quality - Flags total surface current velocities with UQAL and VQUAL equal to 999. UQAL and VQAL values are read from the total file (.tuv) produced by Seasonde software. cleanTUVqual.m cleaned * data points. 
                                            * see the relevant flag variable to determine the number of data points cleaned at each step. '
Dimensions:
           time = 2
           x    = 93
           y    = 74
Variables:
    time             
           Size:       2x1
           Dimensions: time
           Datatype:   double
           Attributes:
                       _FillValue         = -999
                       sdn_parameter_name = 'time'
                       long_name          = 'Time of measurement UTC'
                       units              = 'seconds since 1970-01-01 00:00:00'
                       axis               = 'T'
                       calendar           = 'gregorian'
    x                
           Size:       93x1
           Dimensions: x
           Datatype:   double
           Attributes:
                       long_name = 'Easting distance from the CODAR grid origin'
                       units     = 'km'
                       valid_min = '-46'
                       valid_max = '46'
                       comment   = 'Grid origin coordinates: 49.0675 degrees latitude, -123.3689 degrees longitude. CODAR total ocean surface current velocity grid spacing (in km): 1'
    y                
           Size:       74x1
           Dimensions: y
           Datatype:   double
           Attributes:
                       long_name = 'Northing distance from the CODAR grid origin'
                       units     = 'km'
                       valid_min = '-33'
                       valid_max = '40'
                       comment   = 'Grid origin coordinates: 49.0675 degrees latitude, -123.3689 degrees longitude. CODAR total ocean surface current velocity grid spacing (in km): 1'
    lon              
           Size:       74x93
           Dimensions: y,x
           Datatype:   double
           Attributes:
                       sdn_parameter_name = 'longitude'
                       long_name          = 'Longitude'
                       units              = 'degrees_east'
                       axis               = 'X'
                       valid_min          = -180
                       valid_max          = 180
                       reference_datum    = 'geographical coordinates, WGS84 datum'
    lat              
           Size:       74x93
           Dimensions: y,x
           Datatype:   double
           Attributes:
                       sdn_parameter_name = 'latitude'
                       long_name          = 'Latitude'
                       units              = 'degrees_north'
                       axis               = 'Y'
                       valid_min          = -90
                       valid_max          = 90
                       reference_datum    = 'geographical coordinates, WGS84 datum'
    u                
           Size:       74x93x2
           Dimensions: y,x,time
           Datatype:   double
           Attributes:
                       _FillValue         = -999
                       sdn_parameter_name = 'surface_eastward_sea_water_velocity'
                       long_name          = 'Quality controlled eastward ocean surface current velocity'
                       units              = 'cm/s'
                       valid_min          = -220
                       valid_max          = 220
                       ancillary_variable = 'quality_flag'
                       coordinates        = 'time lon lat'
                       comment            = 'Quality controlled eastward ocean surface current velocity resulting from the unweighted least-squares fit of quality controlled radial velocity measurements.'
    v                
           Size:       74x93x2
           Dimensions: y,x,time
           Datatype:   double
           Attributes:
                       _FillValue         = -999
                       sdn_parameter_name = 'surface_northward_sea_water_velocity'
                       long_name          = 'Quality controlled northward ocean surface current velocity'
                       units              = 'cm/s'
                       valid_min          = -220
                       valid_max          = 220
                       ancillary_variable = 'quality_flag'
                       coordinates        = 'time lon lat'
                       comment            = 'Quality controlled northward ocean surface current velocity resulting from the unweighted least-squares fit of quality controlled radial velocity measurements.'
    u_raw            
           Size:       74x93x2
           Dimensions: y,x,time
           Datatype:   double
           Attributes:
                       _FillValue         = -999
                       sdn_parameter_name = 'surface_eastward_sea_water_velocity'
                       long_name          = 'Eastward ocean surface current velocity'
                       units              = 'cm/s'
                       ancillary_variable = 'quality_flag'
                       coordinates        = 'time lon lat'
                       comment            = 'Eastward ocean surface current velocity resulting from the unweighted least-squares fit of raw radial velocity measurements.'
    v_raw            
           Size:       74x93x2
           Dimensions: y,x,time
           Datatype:   double
           Attributes:
                       _FillValue         = -999
                       sdn_parameter_name = 'surface_northward_sea_water_velocity'
                       long_name          = 'Northward ocean surface current velocity'
                       units              = 'cm/s'
                       ancillary_variable = 'quality_flag'
                       coordinates        = 'time lon lat'
                       comment            = 'Northward ocean surface current velocity resulting from the unweighted least-squares fit of raw radial velocity measurements.'
    u_err            
           Size:       74x93x2
           Dimensions: y,x,time
           Datatype:   double
           Attributes:
                       _FillValue  = -999
                       long_name   = 'Uncertainty error associated with eastward velocity component'
                       units       = 'unitless_velocity^2'
                       coordinates = 'time lon lat'
                       comment     = 'Eastward variance based on directional information of eastward ocean surface current velocity (u_raw).'
    v_err            
           Size:       74x93x2
           Dimensions: y,x,time
           Datatype:   double
           Attributes:
                       _FillValue  = -999
                       long_name   = 'Uncertainty error associated with northward velocity component'
                       units       = 'unitless_velocity^2'
                       coordinates = 'time lon lat'
                       comment     = 'Northward variance based on directional information of northward ocean surface current velocity (v_raw).'
    uv_covariance    
           Size:       74x93x2
           Dimensions: y,x,time
           Datatype:   double
           Attributes:
                       _FillValue  = -999
                       long_name   = 'Eastward and northward covariance directional information of u and v'
                       units       = 'unitless_velocity^2'
                       coordinates = 'time lon lat'
                       comment     = 'Eastward and northward covariance based on directional information of eastward (u_raw) northward (v_raw) ocean surface current velocities.'
    gdop             
           Size:       74x93x2
           Dimensions: y,x,time
           Datatype:   double
           Attributes:
                       _FillValue  = -999
                       long_name   = 'Geometric dilution of precision'
                       units       = '1'
                       coordinates = 'time lon lat'
                       comment     = 'The GDOP (Geometric Dilution Of Precision) terminology is borrowed from satellite navigation and represents the effect that measurement errors due to low angular separation between radial vectors have on the precision of the total current velocity solution. Higher values of GDOP indicate higher uncertainty in the total current velocity vector due to poor geometry.'
    number_of_radials
           Size:       74x93x2
           Dimensions: y,x,time
           Datatype:   double
           Attributes:
                       _FillValue  = -999
                       long_name   = 'Number of radial measurements used to calculate each totals velocity'
                       units       = 1
                       coordinates = 'time lon lat'
                       comment     = 'The number of radial measurements used to calculate each total current velocity are based on quality controlled radial velocities. Radial velocity measurements are obtained from individual radar sites via manufacturer's software and archived in Oceans 3.0 (https://data.oceannetworks.ca). Upon requesting data, a number of QC tests are applied to the raw data to produce a clean (filtered) data set. The test thresholds/parameters are recorded in the processingSteps and releated attributes. See https://wiki.oceannetworks.ca/display/DP/148 for complete documentation.'
    quality_flag     
           Size:       74x93x2
           Dimensions: y,x,time
           Datatype:   double
           Attributes:
                       _FillValue         = -999
                       sdn_parameter_name = 'quality_flag'
                       long_name          = 'Overall quality flags for total ocean surface current velocity'
                       flag_meanings      = 'No_quality_control_performed Good_Data Data_probably_good Data_bad_but_potentially_correctable Bad_data Insufficient_valid_data_for_reliable_down_sampling Averaged_value Interpolated_value Missing_value'
                       valid_min          = 0
                       valid_max          = 9
                       flag_values        = [0  1  2  3  4  6  7  8  9]
                       coordinates        = 'time lon lat'
                       comment            = 'Overall quality flags for eastward and northward ocean surface current velocities. For details about quality control tests applied see processing_parameters variable. With more details on ONC's flag meanings: 0 - No quality control performed, 1 - Good. Data passes all tests, 2 - Data probably good, 3 - Data bad but potentially correctable, 4 - Bad data, 6 - Insufficient valid data for reliable down-sampling, 7 - Averaged value, 8 - Interpolated value, 9 - Missing value.'
    crs              
           Size:       2x1
           Dimensions: time
           Datatype:   int32
           Attributes:
                       long_name          = 'Coordinate Reference System'
                       grid_mapping_name  = 'latitude_longitude'
                       epsg_code          = 'EPSG:4326'
                       semi_major_axis    = 6378137
                       inverse_flattening = 298.2572
                       comment            = 'http://www.opengis.net/def/crs/EPSG/0/4326'



CODAR QARTOD Tests and Procedures

Flag values used, follow our QAQC flag scheme (more details here Quality Assurance Quality Control, although this page is intended primarily for scalar data), where, simply:

  • 0 - No test
  • 1 - Pass
  • 4 - Fail

QARTOD guidelines

From the QARTOD “Manual for Real-Time Quality Control of High Frequency Radar Surface Current Data” – Version 1 (May 2016). The summary table below presents the QARTOD QC tests for HF radar surface current data:

Test Type

Test Name

Status

Test Control

Signal (or Spectral) Processing

Signal-to-Noise Ratio (SNR) for Each Antenna (Test 1)

Required

Embedded

Cross Spectra Covariance Matrix Eigenvalues (Test 2)

Suggested

Embedded

Single and Dual Angle Solution - Direction of Arrival (DOA) Metrics (magnitude) (Test 3)

Suggested

Embedded

Single and Dual Angle Solution - Direction of Arrival (DOA) Function Widths (3 dB) (Test 4)

Suggested

Embedded

Positive Definiteness of 2×2 Signal Matrix (Test 5)

Suggested

Embedded

Radial Components

Syntax (Test 6)

Required

National

Max Threshold (Test 7)

Required

Local and National

Valid Location (Test 8)

Required

Local and National

Radial Count* (Test 9)

In development

Local and National

Spatial Median Filter (Test 10)

Suggested

Local and National

Temporal Gradient (Test 11)

Suggested

Local and National

Average Radial Bearing (Test 12)

Suggested

Local and National

Synthetic Radial (Test 13)

In development

Local and National

Total Vectors

Data Density Threshold (Test 14)

Required

Local and National

GDOP Threshold (Test 15)

Required

Local and National

Max Speed Threshold (Test 16)

Required

Local and National

Spatial Median Comparison (Test 17)

Required

Local and National

Further Details on Required Radial Velocity Tests (Section 3.4.2 Radial Tests of the QARTOD manual)

This set of tests is conducted during the development of the radial velocities, or upon the resultant radial velocities. These tests may be carried out at the local, regional and/or national network levels.

Test 6 – Syntax (Required) A collection of tests ensuring proper formatting and existence of fields within a radial file.

The radial file may be tested for proper parsing and content, for file format (hfrweralluv1.0, for example), site code, appropriate time stamp, site coordinates, antenna pattern type (measured or ideal, for DF systems), and internally consistent row/column specifications.

Flags

Condition

Codable Instructions

Fail = 4

One or more fields are corrupt or contain invalid data.

If “File Format” ≠ “hfrweralluv1.0”, flag = 4

Suspect = 3

N/A

N/A

Pass = 1

Applies for test pass condition.

N/A

Test Exception: None.

Test specifications to be established by operator. Acceptable files types, site codes, coordinates, APM names, etc., must be presented. For example, the national network performs the following suite of tests:

• All radial files acquired by HFRNet portals report the data timestamp in the filename. The filename timestamp must not be any more than 72 hours in the future relative to the portals’ system time.

• The file name timestamp must match the timestamp reported within the file.

• Radial data tables (Lon, Lat, U, V, ...) must not be empty.

• Radial data table columns stated must match the number of columns reported for each row (a useful test for catching partial or corrupted files).

• The site location must be within range: − 180 ≤ Longitude ≤ 180 − 90 ≤ Latitude ≤ 90.

• As a minimum, the following metadata must be defined:

o File type (LLUV)

o Site code

o Timestamp

o Site coordinates

o Antenna pattern type (measured or idealized)

o Time zone (only Coordinated Universal Time or Greenwich Mean Time accepted)

Test 7 - Max Threshold (Required) Ensures that a radial current speed is not unrealistically high.

The maximum radial speed threshold (RSPDMAX) represents the maximum reasonable surface radial velocity for the given domain.

Flags

Condition

Codable Instructions

Fail = 4

Radial current speed exceeds the maximum radial speed threshold.

If RSPD > RSPDMAX, flag = 4

Suspect = 3

N/A

N/A

Pass = 1

Radial current speed is less than or equal to the maximum radial speed threshold.

If RSPD ≤ RSPDMAX, flag = 1

Test Exception: None.

Test specifications to be established by operator. The maximum total speed threshold is 1 m/s for the West Coast of the United States and 3 m/s for the East/Gulf Coast domain. The threshold must vary by region. For example, the presence of the Gulf Stream dictates the higher threshold on the East Coast.

Test 8 – Valid Location (Required) Removes radial vectors placed over land or in other unmeasurable areas.

Radial vector coordinates are checked against a reference file containing information about which locations are over land or in an unmeasurable area (for example, behind an island or point of land). Radials in these areas will be flagged with a code (FLOC) in the radial file (+128 in CODAR radial files) and are not included in total vector calculations.

Flags

Condition

Codable Instructions

Fail = 4

Radial contains a user-defined location flag code in the radial file.

If FLOC exists, flag = 4

Suspect = 3

N/A

None

Pass = 1

Radial does not contain a user-defined location flag code in the radial file.

If FLOC does not exist, flag = 1

Test Exception: None.

Test specifications to be established by operator. For CODAR systems, the reference file is called AngSeg_XXXX.txt, where XXXX is the four-letter site code of the station and is located in the “RadialConfigs” folder. These vectors receive a code of +128 in the flag column of the radial text file. BF systems use pre-set grid locations for radials.

Further Details on Required Total Velocity Tests (section 3.4.3 Total Vectors of the QARTOD manual)

Test 14 - Data Density Threshold* (Required) Tests that a sufficient number of radial velocities exist to compute a total velocity vector.

A minimum number of radial velocities (RV_MIN) are required to construct a total velocity vector. RV_CNT is the number of radial velocities available to be used in the calculation.

Flags

Condition

Codable Instructions

Fail = 4

Insufficient number of radial velocities exist.

If RV_CNT < RV_MIN, flag = 4

Suspect = 3

N/A

N/A

Pass = 1

A sufficient number of radial velocities exist.

If RV_CNT ≥ RV_MIN, flag = 1

Test Exception: Does not apply to BF systems.

Test specifications to be established by operator. Recommend RV_MIN = 3

In CODAR software, this is set in line 1 of the AnalysisOptions.txt configuration file; the default value is 2.

Test 15 - GDOP Threshold (Required) Tests that the uncertainty in velocity due to the geometric relationship between radials is low enough for the vector to be considered valid.

GDOP (Geometric Dilution of Precision) is a scalar representing the contribution of the radial (bearing) geometry to uncertainty in velocity at a given gridpoint. Higher GDOP values indicate larger co-variances associated with the least square’s fit used in obtaining the solution. GDOP must be less than a maximum allowed value of GDOP_MAX to pass and less than a GDOP_HIGH value to not be considered suspect.

Flags

Condition

Codable Instructions

Fail = 4

Poor geometric relationship between radials yields a total vector with too much uncertainty to be valid.

If GDOP ≥ GDOP_MAX, flag = 4

Suspect = 3

The GDOP value associated with a total vector solution may be acceptable.

If GDOP < GDOP_MAX and GDOP ≥ GDOP_HIGH, flag = 3

Pass = 1

The GDOP associated with the total vector solution is sufficient.

If GDOP < GDOP_HIGH, flag = 1

Test Exception: None.

Test specifications to be established by operator.

The national network uses a GDOP_MAX of 10 and a more conservative value of 1.25 for near-real time applications such as Web display.

Test 16 - Max Speed Threshold (Required) Ensures that a total current speed is not unrealistically high.

Like the maximum radial velocity threshold, the maximum total speed threshold TSPDMAX represents the maximum reasonable surface velocity for the given domain.

Flags

Condition

Codable Instructions

Fail = 4

Total current speed exceeds the maximum total speed threshold.

If TSPD > TSPDMAX, flag = 4

Suspect = 3

N/A

None.

Pass = 1

Total current speed is below or equal to the maximum total speed threshold.

If TSPD ≤ TSPDMAX, flag = 1

Test Exception: None.

Test specifications to be established by operator.

The maximum total speed threshold is 1 m/s for the West Coast of the United States and 3 m/s for the East/Gulf Coast domain. The threshold must vary by region. For example, the presence of the Gulf Stream dictates the higher threshold on the East Coast.

ONC CODAR QARTOD Overall Implementation

This section describes the implementation of the QARTOD tests and procedures from the previous section to create the CODAR Quality Controlled Surface Current data products. The ONC implementation uses a custom extended version of an open source MATLAB® toolbox called HFR_PROGS (High Frequency Radar PROGramming Suite), found here: https://github.com/rowg/hfrprogs (slightly updated from the old 2.1.2 version found here: https://cencalarchive.org/~cocmpmb/COCMP-wiki/index.php/HFR_Progs_download_page. Our implementation of HFR_PROGS has significant extensions, which should be, at some future date, shared back to the github rowg group repository or our own. If not, contact us for the code. 

The workflow consists of reading SeaSonde .RUV (radial u & v) files containing hourly radial data for each CODAR station. The QARTOD tests prescribed in the previous section are applied to first to the radial data, as described below:

Radial data QA/QC workflow for each station:

  1. Read hourly .RUV file
  2. Apply Test 6 - Syntax Test: Verify the syntax in the header matches the expected metadata for each site.
    • File type = RUV
    • Site code
    • Timestamp
    • Site coordinates
    • Antenna pattern type (measured or idealized)
    • Time zone
  3. Apply Test 7 - Max Threshold Test:
    • Flag radial data exceeding RSPDMAX.
  4. Apply Test 8 - Valid Location Test:
    • Flag radial data over land (VFLG = 128).
  5. Spatial/Temporal Quality Test:
    • Flag data where ESPC or ETMP = 999.
  6. Output new radial RUV data including QA/QC test results.

Note that an additional test has been applied (Spatial/Temporal Quality Test) not defined in the QARTOD manual to flag radial data having a Spatial Quality (ESPC) or Temporal Quality (ETMP) flag of 999. These flags indicate non-calculable data points as determined by the SeaSonde software and are included as columns in the .RUV files.

The output from the radial data QA/QC workflow are modified hourly .RUV files for each CODAR site that include columns containing the QA/QC test results. After the hourly radial data from each CODAR station has undergone QA/QC, a new set of total vectors are generated (combined) using the ‘clean’ radial data. The workflow for generating total current vectors consists of inputting the ‘clean’ radial data, inputting raw total data files (.TUV) and total grid positions from the SeaSonde combiner computer, calculating a new grid of total vectors with HFR_PROGS/makeTotals, applying the prescribed QA/QC tests, then outputting a new set of total (.TUV) data.

Total data QA/QC workflow:

  1. Input ‘clean’ radial data from radial data QAQC workflow for each station and time stamp.
  2. Input .TUV total vector files from SeaSonde.
  3. Input combine_<LOCATION_CODE>.grd grid file from SeaSonde combiner computer for each network, where location code is VSOG for Strait of Georgia, VRUP for Prince Rupert, etc. The grid file applied is also selected from a time range of applicability
  4. Calculate new set of totals from clean radial data using HFR_PROGS. In this step, the spatial grid is provided from the combine_LOCN.grd file. Pre-determined temporal and spatial windows around each grid point are used to determine which radial vectors contribute to each total vector.
    • A spatial threshold (a search radius) is supplied to makeTotals for griding the radial data into total: SPATTHRES
    • A minimum number of contributing CODAR sites is required: MIN_NUM_SITES
  5. Apply Test 14 - Data Density Test:
    • Flag total vectors with RV_MIN < 3.
  6. Calculate GDOP values at each grid point using HFR_PROGS.
  7. Apply Test 15 - GDOP Threshold Test:
    • Flag total vectors with GDOP ≥ GDOP_MAX.
    • Optional GDOP tests are available but currently are not used. HFR_Progs produces four other error estimates in addition to the standard GDOP: GDOPMaxOrthog, GDOPRadErr, GDOPSites, FitDif. These are available in the MAT file structure if users wish to do additional filtering. ONC Data Specialists can also enable these in the live data products by changing the device attributes in Oceans 3.0 (current values are empty so that tests are not applied).
  8. Apply Test 16 - Max Speed Threshold:
    • Flag total vectors with TSPD > TSPD_MAX.
  9. Spatial Quality Test:
    • Flag data where UQUAL or VQUAL = 999.
  10. Output new total data (.TUV) including QA/QC test results.

Step 9 is an additional QA/QC test not prescribed by QARTOD that flags total data having a U or V standard deviation quality indicator (UQUAL/VQUAL) of 999 as indicated in the .TUV files produced by SeaSonde. UQUAL and VQUAL values of 999 indicate a non-calculable result for the grid point.

The outputs of these steps are available in the MAT, netCDF and other file products. In the MAT file product, the Radials structure is the cleaned RUV data, The raw RUV data is available as:

Radials(1).Uraw = Radials(1).OtherMetadata.RawData(3);
Radials(1).Vraw = Radials(1).OtherMetadata.RawData(4);


The Totals structure is the cleaned total surface velocities. The 'raw' totals U and V velocity components are available as Totals(1).Uraw  and Totals(1).Vraw, with raw meaning that these values are as makeTotals.m  has calculated in step 4, prior to step 5 thru 9 that test and clean the data, so 'raw' in this case includes the testing and cleaning done in the calculation of the radial velocities. All of the processing is all done on demand and on-the-fly and the data processing is quite quick. The bolded parameters above are tune-able via the Oceans 3.0 device attributes, and the values used stored in additional fields in the MAT file structure so that are easily available. The parameters used in the various functions are in the QC structure and in Totals.OtherMetadata.makeTotals.parameters. The Totals.ProcessingSteps cell array contains summary information on the cleaning steps in particular. The QC sub-structure is an additional structure that exists within both the Radials and Totals that captures the flags and test thresholds generated by the tests.

ONC CODAR Software Implementation

As noted above, these data products are generated by a modified version of HFRProgs suite foe Matlab, version 2.1.2. The products also utilize Rich Pawlowicz's M_Map, A mapping package for MATLAB", version 1.4k, 2019, available online at www.eoas.ubc.ca/~rich/map.html. Many thanks to Rich for his great software. In the remainder of this section, the specific modified and extended functions from HFRProgs are detailed. The list below is not an exhaustive list of the full implementation, but is intended to give insight on the implementation.

File Inputs

Input

Description

Implementation

Raw radial files

.ruv files from Seasonde coming from each CODAR sites (antenna) and for the same time stamp

For each CODAR regions, contributing sites are properly identified

Master Grid file

The grid file that exists on the combiner computer.

These files are archived as data products and may made be available via data search. They are location and time tagged so that the appropriate file is used for each time stamp of data

The combiner device is one-level up in the Oceans2.0 search-tree from each radial station so that the processing code can easily associate them.

Raw total file

.tuv file produced by Seasonde for the same time stamp. This is needed for the Spatial Quality Test.

Load the file from the correct array location.

Device Attribute QC parameter

For the CODAR station devices and radial data products:

For the CODAR combiner virtual devices and the total data products:

Function Descriptions

getGridHeader – New function

The function getGridHeader was created to parse the header file using getRDLHeader as a model. It outputs the metadata names and values contained in the header. The creation date of the file can then easily be extracted using the HFRProgs function getNameValuePair, a function developed to select the elements from a cell array of names and values that matches a given string.

loadRDLFile – HFRProgs modified function

This function conveniently loads multiple radial files into one radial MATLAB structure.

Rorig = loadRDLFile(fnames, 'true');

The function loadRDLFile loads the core variables such as Longitude, Latitude, U, V, Velocity (renamed RadComp by loadRDLFile), Range, Bearing, Heading, Temporal Quality (renamed Error by loadRDLFile), and VectorFlag from the raw Seasonde radial file.

Some variables (e.g. ESPC - Spatial Quality) from the Seasonde radial file are not imported along with the other core variables in the radial MATLAB structure created by the loadRDLFile function. However, it is possible to keep the original raw data in the final MATLAB structure by setting the input variable ‘saverawdata’ as true (the default is false). The original raw data (‘RawData’) is then saved under the field ‘OtherMetadata’ in the radial MATLAB structure.

Note: values of Range, Bearing, and Heading in the radial MATLAB structure differs from the raw Seasonde radial file, since as stated in the loadRDLFile function a conversion is applied using the HFRProgs true2math function:

% Change direction to cartesian convention.  Note, however, that bearing

% will still point away from radar and heading will still point towards radar.

TOTAL.RangeBearHead(:,2:3) = true2math( TOTAL.RangeBearHead(:,2:3) );

The true2math function converts angles measured in degrees from North (North = 0 degrees) to angles measured in degrees from East (East = 0 degrees).

The ‘ProcessingSteps’ is a useful section of the radial MATLAB structure that keeps a record of the processing functions applied to the radial data in a cell array.

This function was modified to store Seasonde variables Spatial Quality (ESPC) and Temporal Quality (ETMP, with its original Seasonde name and instead of ‘Error’) in the Radials structure. The lines of code in loadRDLFile replacing any 999 values from in the RADIALmatvars variables with NaN were commented out, in keeping with the function’s primary goal to load radial files into MATLAB. Any processing of the raw data will be taken care of by different functions in subsequent steps.

checkRDLSyntax – New function

This function was created to do the syntax checks required in the QARTOD manual for CODAR data. Specifically, checkRDLSyntax checks for:

  • File type is LLUV
  • Site code exists and is consistent with the site code reported in the file name
  • Time stamp exists
  • Time stamp matches time stamp reported in the file name
  • Core variable data exist
  • Number of columns matches number of column names
  • Site location exist
  • Site location is within proper range
  • Antenna pattern type
  • Time zone exists and is either UTC or GMT

The HFRProgs function getRDLHeader is used to parse the header of the radial files. The HFRProgs function getNameValuePair is used to extract the relevant information (e.g. the value ‘measured’ of the variable ‘AntennaPatternType’) from the parsed radial file header. Subsequent lines of code verify that the information exists and/or that the information is correct. If a condition is not respected an warning is written to the log and search result (visible in the Data Search cart) and the file is excluded. 

All the different syntax checks are saved in the Radials.QC structure under ‘CleanRDLSyntax’ field and a processing step is added in the ‘ProcessingSteps’ cell array.

cleanRadialsVel, cleanRadialsLoc, cleanRadialsTemp, cleanRadialsSpat – Modified and New functions

These functions were developed using the function cleanRadials from the HFRProgs as a model. For each modified and new functions the list of variables affected by the result of the test was reduced to the following variables:

RADIALmatvars = {'U','V','RadComp'}

The RADIALmatvars list is directly defined inside each functions. Any data point failing a test values of the variables in RADIALmatvars are replaced by NaN and the corresponding value in the vector Flag is replaced by 4.

The HFRProgs function cleanRadials is a speed test. The function cleanRadialsVel is the essentially the same function as cleanRadials but it uses an updated list of variables. The threshold value, RSPDMAX, is an input variable as it is site specific and should be determined by ONC Science. From the ONC “Summary of Coastal Ocean Dynamics Application Radar (CODAR): Operation, Data Quality and Products”:

…

The CODAR manual recommends a maximum threshold of 1 m/s for the West Coast of the United States, and 3 m/s for the East/Gulf Coast; however, thresholds may vary by region. The RSPDMAX value used to test the radial velocity was 1.3 m/s for the Strait of Georgia CODAR data; however, the “best” value to be used operationally in real-time should be determined by considering observed and modeled surface current data for each operational region. Note that the SeaSonde software has a “Maximum Radial Velocity” threshold in the settings for determining the first order limits. This parameter limits the radial velocity magnitude in the output data by changing the Bragg spectral boundary limits used in processing, thus works in a different way than the QARTOD test.

…

cleanRadialsLoc is the function created to clean out the vector values 128 values from the ‘VectorFlag’ vector. The VectorFlag informs about the validity of location of the vector (on land, behind a structure such as an island, etc.). As described in the Seasonde ‘File_LLUV’ document (Sept 24, 2016):

…

Vector Flag

The Vector Indicator Flag (VFLG) in the fifth column is a composite value used by the Combine Grid and the LLUV files to indicate extra information about individual current vectors.

Bit0 (+1) disabled grid point and not outputted in LLUV files.

Bit1 (+2) grid point is near the coastline.

Bit2 (+4) grid point contains a point measurement like an ADCP.

Bit4 (+16) current vector was the result of interpolating across a baseline area.

Bit5 (+32) vector result exceeded the Maximum current limit.

Bit7 (+128) vector is out of bounds. For radial vectors this means that the vector was outside the Angular filter area. Not used for Total Vectors.

Bit8 (+256) Total vector does not have enough angular resolution from the contributing radials.

Bit9 (+512) vector is marked as hidden and should not be normally displayed.

Bit11(+2048) vector was created by interpolation.

Bit12(+4096) vector is of dubious quality and should not normally be used or displayed.

…

Currently the QARTOD manual only specifies that the vector flag VFLG = 128 need to be flagged as bad data and should not be included in the total vector calculation. The value 128 is the input variable FLOC in this function so it can easily be configured if QARTOD decides to include (for example) more or other VectorFlag values in the future.

cleanRadialsTemp and cleanRadialsSpat are the functions created to clean out the 999 values from, respectively, the Temporal Quality (ETMP) and the Spatial Quality (ESPT) vectors. As described in the Seasonde File_LLUV document (Sept 24, 2016):

…

Uncertainties/Quality Factors

There are three to six columns for Quality Factors. Their usage is dependent on the type of LLUV: radial, elliptical or total. While we typically call these Quality Factors, it's really a misnomer; they are really measurement uncertainties but it is simpler to label them as quality. They indicate measurement uncertainty due to noise, a changing surface current pattern, and/or horizontal shear over each measured area. They represent a lower bound on the actual uncertainty in the current velocity. For example, errors due to contaminated radar spectra or incorrect first-order region boundaries will not be included in the estimate. We believe that most spatial uncertainty is due to horizontal current shear and most temporal uncertainty is due to the current pattern changing over time. A resulting Quality Factor of 999 indicates a non-calculable result. For Radials & Ellipticals, there are 7 possible columns: Temporal (ETMP), Spatial (ESPC), Maximum Velocity (MAXV), Minimum Velocity (MINV), Velocity Doppler Count (EDVC), Spatial Count (ERSC), and Spatial Count (ERTC)

…

The 999 value is not likely to change and is hardcoded inside both cleanRadialsTemp and cleanRadialsSpat functions.

cleanRadialsTemp the function works on the temporalQuality vector (ERROR or ETMP in the ruv header), while cleanRadialsSpat works on the spatialQuality vector (ESPT in the ruv header). 

Each test threshold issaved in the QC structure with the respective name append by 'TestThreshold' and a processing step is added in the ‘ProcessingSteps’ cell array.

For each QC tess, a resulting vector of flags is saved under ‘QC’: a field ‘radialSpeedflag’, ‘locationFlag’, ‘temporalQualityFlag’, and ‘spatialQualityFlag’ are created after respectively, cleanRadialsVel, cleanRadialsLoc, cleanRadialsTemp and cleanRadialsSpat functions are used. For any data point failing one of these test, the corresponding value in the ‘overallFlag’ is changed to 4.


makeTotals - HFRProgs modified function

This function makeTotals uses the HFRProgs function tuvls that calculates the totals and different GDOP errors and output a total MATLAB structure.

[Totals, ~] = makeTotals(Radials, 'Grid', Grid.nodesOcean, 'TimeStamp', Radials(1).TimeStamp, 'spatthresh', spatialThreshold,...
'tempthresh', TEMPTHRESH, 'MinNumSites', minNumSites, 'MinNumRads', minNumRadials, ...
'WhichErrors', {'GDOPMaxOrthog', 'GDOP', 'GDOPRadErr', 'GDOPSites', 'FitDif'});

makeTotals also performs some density checks. There are two input variables relating to the density checks: minNumSites for minimum number of contributing sites, and MinNumRads for the minimum number of radials. If the number of sites is less than the threshold value a warning is issued. If the number of radials is less than the threshold value, calculation of totals from the given radials is skipped with the use of continue (no error or warning are generated).

The different GDOP calculated from the function makeTotals are: ’GDOPMaxOrthog', 'GDOPRadErr', 'GDOPSites', 'GDOP’, ‘FitDif'. For each GDOP type, Uerr, Verr, UVCovariance and TotalsErrors values are calculated and stored under the field ‘ErrorEstimates’ in the total MATLAB structure.

After contacting different members of the HF Radar community, Mark Otero from HFRNet group send us the piece of code National Network uses to calculate the GDOP. Following this, it was decided that ONC will calculate GDOP using all angles (named ‘GDOP’, in HFRProgs). Code used by the National Network to calculate the GDOP:

% UWLSTOTAL Computes a total velocity from radials using least-squares
%
%    [ u, v, dopx, dopy, hdop ] = uwlsTotal( rSpeed, rHeading ) returns the 
%    eastward (u) and northward (v) components of the total velocity 
%    resulting from the unweighted least squares fit of radial velocity 
%    measurements provided by rSpeed and rHeading.  The horizontal dilution
%    of precision (hdop) and corresponding eastward (dopx) and northward
%    (dopy) components are also returned as a metric for assessing the 
%    quality of the solution.
%
%    INPUT:
%        rSpeed   - Column vector (n* x 1) of radial velocity magnitude
%        rHeading - Column vector (n* x 1) of radial velocity heading in 
%        degrees counterclockwise from +x (east)
%
%        *n >= 2
%
%    OUTPUT:
%        u    - Eastward total velocity
%        v    - Northward total velocity
%        dopx - Eastward dilution of precision
%        dopy - Northward dilution of precision
%        hdop - Horizontal dilution of precision
%
%    All output values are scalars and the units of eastward and northward 
%   velocity are the same as the units for rSpeed.
%
%    Radial velocities are related to the total velocity by projection of 
%   the eastward and northward components of the total
%   velocity onto the radial heading:
%        rSpeed = u*cosd(rHeading) + v*sind(rHeading)
%
%    and in matrix form as
%        rSpeed = X*b
%
%    where
%        X = [ cosd(rHeading) sind(rHeading) ]
%        b = [ u; v ]
%
%    The unweighted least squares solution for b is given by
%        b = inv( X'*X ) * X' * rSpeed
%
%    The geometric portion of the covariance matrix for b is
%        C = inv( X'*X )
%
%    which provides the dilution of precision along the diagonal where
%        dopx = sqrt( C(1,1) )
%        dopy = sqrt( C(2,2) )
%        hdop = sqrt( C(1,1) + C(2,2) )
%
%    References
%    -   Least-squares methods for the extraction of surface currents from 
%    CODAR crossed-loop data: Application at ARSLOE, Lipa, B. J., D. E. 
%    Barrick, IEEE J. Ocean. Eng., 1983.
%    -  Shipborne measurement of surface current fields by HF radar, Gurgel,
%    K. -W., L'Onde Electr., 1994.
%    -   On the accuracy of HF radar suface current measurements: 
%    Intercomparisons with ship-based sensors, Chapman, R. D., L. K.  Shay,
%    H. C. Graber, J. B. Edson, A. Karachintsev, C. L. Trump, D.  B. Ross, 
%    J. Geophys. Res., 1997.
%    -  Guide to GPS Positioning, Wells, D. E., et al., Can. GPS Assoc., 
%    Fredericton, N. B., Canada, 1986.
%
%    Acknowledgement
%    This function was adapted from the TOTCALC function (ver. 3.0, Apr 
%    2004) included in the HFRadarmap toolbox (ver. 4.1, Sept 2004) written
%    by Mike Cook while at the Naval Postgraduate School Department of
%    Oceanography.
%
%    HF-Radar Network
%    Mark Otero
%    Scripps Institution of Oceanography
%    Coastal Observing Research and Development Center


makeTotals is basically the same function as the original makeTotals with the following modifications:

  • It was modified to accept the Temporal Quality ‘ETMP’ variable (line 180, 181 of the function)
  • The function will throw an error instead of a warning if the number of sites is less than the threshold (line 195).
  • The function creates a vector Flag of nines (missing data) to be able to store flags after QC processing. (line 107)
  • The value in the vector Flag is replaced by 4 where the number of site is less than MINSITES or the number of radials is less than MINRAD (line 309 to 311). 
  • Else, the value in the vector Flag is 1 if U,V are not NaN, otherwise, the Flag is 9 (missing data).

Same as the radial QC functions, some information for ‘book keeping’ are saved. For this function some parameters (such as MINSITES and MINRAD thresholds, time stamp and other input parameters) are saved in the ‘OtherMetadata’ section in the total MATLAB structure, and a processing step with the name of the function is added to the ‘ProcessingSteps’ cell array.

A structure ‘QC’ was created containing two fields: ‘overallFlag’ and ‘numSitesRadialsFlag’. ‘overallFlag’ is the vector containing the overall flag results after different QC tests are applied. The ‘numSitesRadialsFlag’ is the vector containing the results from the QC test based on the minimum number of sites and minimum number of radials contributing to the calculation of the total vector. Since this test is needed at (and integrated in) the makeTotals function level, the function output both ‘overallFlag’ and ‘numSitesRadialsFlag’ with both vectors being identical at that step.


cleanTotals - HFRProgs function, replaced by cleanTotalsVel - Modified function

The function cleanTotals is a speed test for totals. U and V values failing the test are replaced by NaN. The threshold value TSPDMAX is an input variable as it is site specific and should be determined by ONC Science. From the ONC “Summary of Coastal Ocean Dynamics Application Radar (CODAR): Operation, Data Quality and Products”:

…

The recommended threshold (TSPD_MAX) is 1 m/s for the West Coast of the United States and 3 m/s for the East and Gulf Coasts. The threshold will vary by region and appropriate values will be determined by operational region for ONC CODAR data.

…

The function cleanTotalsVel is the same function as cleanTotals except that for any given point failing the test the corresponding values in the vector ‘totalSpeedFlag’ and in the ‘overallFlag’ vectors under ‘QC’ are replaced by 4. The max speed threshold is Totals.QC.totalSpeedMaxTestThreshold

Relevant information (TSPDMAX,etc.) are saved in the ‘OtherMetadata’ section and a processing step with the name of the function is added to the ‘ProcessingSteps’ cell array in the total MATLAB structure.


cleanGDOP – New function

This function was created to calculate the GDOP value using the Uerr and Verr calculated by the makeTotals function, as follow:

sqrt(Uerr+Verr)

The threshold GDOPMAX is an input variable so it can easily be configured. For each GDOP values failing the test, corresponding U and V data points are replaced by NaN and the corresponding values in the ‘gdopFlag’ and in the ‘overallFlag’ vectors under ‘QC’ are replaced by 4.

QARTOD recommends two values for the GDOPMAX. GDOPMAX = 10 is used by the National network and GDOPMAX = 1.25 is used for near real time application and web display. HFRNet uses 1.25.

The GDOP calculated error is saved in the ‘OtherMatrixVars’ section. A processing step with the name of the function is added to the ‘ProcessingSteps’ cell array in the total MATLAB structure.

This function also implements the other GDOP tests, as specified by device attributes: 'GDOPMaxOrthogErrorMax', 'GDOPRadErrErrorMax', 'GDOPSitesErrorMax', 'FitDifErrorMax'. Currently, these are not used (default values in device attributes are null, values in the MAT file structure are set to Inf). The test metric is the TotalError value in the struct array Totals.ErrorEstimates and the test is simply if the test metric exceeds the threshold, the test fails, surface currents are NaN'ed and the corresponding QC flag is set to 4.


cleanTUVQual – New function

This function was created to clean out the data points with corresponding UQAL and VQUAL equal to 999 in the total file (.tuv) produced by Seasonde. Prior using this function, the Seasonde total file needs to be loaded using the loadTUVFile, an HFRProgs function similar to loadRDLFile but for totals.

The function cleanTUVQual matches the data points by longitudes and latitudes between the Seasonde total file and the total MATLAB structure. U and V data points with corresponding UQAL and VQUAL values of 999 are replaced by NaN and the corresponding values in the ‘tuvQualityFlag’ and in the ‘overallFlag’ vectors under ‘QC’ are replaced by 4.

A processing step with the name of the function is added to the ‘ProcessingSteps’ cell array in the total MATLAB structure..


Data Availability

CODAR radial and total-current data is also available from HFRNet, a continent-wide repository of HF Radar measurements. An interactive mapping application there provides a graphical way to explore ocean currents, not only from Oceans 3.0, but from CODAR stations across the United States (and a single station in Mexico) as well.
Straight of Georgia Deployment

Radial data is available from the Westshore Coal Terminal station (VCOL) beginning on on November 24, 2011 and from the Iona Wastewater Treatment Plant (VION) beginning August 1, 2012. “Total” current data files are also available beginning August 1, 2012. Radial and total data files are generated hourly.

CSS and CSQ/Range data are available for each station for the same time period as the corresponding radial data. The exception to this is the CSQ files from the VCOL station in the five and half months between the date that acquisition started and June 10, 2012. An unnoticed default setting on the VCOL computer resulted in the CSQ files from this period being overwritten (apart from the CSQ files from February 6-7, 2012, which were recovered).

The VION station was switched from retaining CSQ files to retaining Range files on December 4, 2012. VCOL was switched to retaining Range files on December 19, 2012. Because VCOL is on a wireless router with limited bandwidth, CSQ/Range files are not fetched over the network, but rather are physically transferred via portable hard drive when site visits are made. As a result, there may be a lag of several months between acquisition and availability of CSQ/Range files from VCOL.

The range of high-frequency radio waves is greatly reduced over fresh water compared to the range over salty, highly-conductive water. For this reason, during the spring freshet, when the surface water in the Strait of Georgia is much less salty, one or both of the CODAR stations may return little or no current data in the area of overlap, resulting in no total current vectors being available. This will vary from year to year, but at the time of writing (May, 2013) it appears that total current vectors may be unavailable from early May to as late as July each year.

As of May 2016, the data acquisition framework was switched to the stations/combiners model. All of the formats will be available from then onward. The older files will migrate from the old scheme to the new devices over the next few months. The old scheme used a single device with multiple file mode fields to distinguish the stations. The new scheme is more flexible for expansion (we have added CODAR stations in Prince Rupert and there's potential for much more).

Discussion

To comment on this product, click Write a comment... below.

  • No labels