Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Release notes summaries on this page:

...

  • Implemented "QAQC" in scalar data products. Data will start to be analyzed for invalid values and flagged. It will take some time before all instrument data is being analyzed, but the infrastructure is now in place to support it.** Data can be analyzed "manually" (by hand) or automatically by software.** Automatic tests include (not an exhaustive list) out of range tests, spike tests, high gradient tests, and stuck value tests.** Scalar data products (CSV, MAT, PNG, PDF) now show a QAQC flag for each record. CSV and MAT list the flag value, ranging from 0 to 9.PNG and PDF plots highlight potentially invalid data with yellow or red colours, and with a yellow or red flag in the center of each group of flagged data.** Flag values: 0 means no analysis has been done, 1 means good data, 2 - probably good, 3 - use with caution, 4 - bad data, 7 - averaged data, 8 - interpolated value, 9 - data gap. See here for more details: http://venus.uvic.ca/2011/05/qaqc-for-venus-scalar-data/ ** In a later release the user will have the ability to select "clean" or "raw" data in their data products. For now all non-resampled data is "raw", meaning that invalid data is still present but is flagged. All resampled data is "clean", meaning that invalid data has been removed; for the CSV NaN option it will be replaced with "NaN" (not a number); for the other options it will be removed entirely.** In the case of averaged data, if less than 80% of the expected data values for a given averaging point are not present (either due to data gaps or failing a QAQC test), that averaged point is treated as a failed QAQC data point.
  • Data Search - "sort by Instrument" now has an additional choice level. Besides selecting an instrument you now also have to select where that instrument was deployed. Note that "MTC" means Marine Technology Center, our on-shore testing labs (which in some cases have data). The data availability graph will only show data obtained from the selected deployed site. Regardless of the data range you select, the data products will only contain data from your selected site. If you want to see data from a given site regardless of which instrument was deployed, use "sort by Location".
  • Plotting Utility - has the same "sort by Instrument" changes mentioned above.
  • Data Search - the "min/max" resampling option has been temporarily removed. It will return in a future release but only for certain data products.
  • Improvements were made to the internal "Device Workflow" tool.
  • Modified SensorDataService, DeviceAttributeService
  • Added more information in search failure e-mails if an Instrument search.
  • Improved the performance of Data Search for generating matlab scalar data products by up to 3X.
  • Modified the Nortek data products to detect velocity-phase wrapping.
  • In CSV files that reference data from more than one instrument (such as on the VPS), it is made more clear which sensors are external to the main instrument.

...