Release Notes
Detailed release notes are available on internal wiki.
Release notes summaries on this page:
3 July 2014
- Added two new scheduled jobs (157 and 158) to run specified data searches on a regular basis. These can be used to produce, for example, daily plots for the same instrument. Plots like this will be added to the ONC website soon.
- Changed Plotting Utility to make the default plot "Clean Min/Max+Avg" rather than a "raw" plot that includes data that is probably not valid.
- Modified the DeviceAttributesService to return Nortek attributes within a specified time range.
- Added ancillary tilt data and Prosoft support files to Satlantic Radiometers data products.
- "Derived sensors" are pseudo sensors whose values are derived mathematically from other sensors, typically on the same device. The data products and metadata for derived sensors was changed to include QAQC (data quality) flags inherited from their parent sensors.
- We are now storing quarter scalar data in the Cassandra database in addition to what was already being stored. At this time the Oracle database remains the primary database. A decision to switch to Cassandra has not yet been made, but in the meantime it is possible for users to be added to the "Cassandra Group" in User Management. This will cause all plots made by Plotting Utility to use data from Cassandra.
- Added a parser for IC Listen hydrophones.
- Improved the performance of the QuarterScalarDataSummary job.
4 June 2014
Plotting Utility
- QAQC (data quality) selection has been moved from Options >> Plot Properties to Options >> QAQC. The default for new plots is "Raw - min/max" and there are no other plot types that are supported for raw data. If you want a different plot type (line, bar, etc) you must select "clean" first using Options >> QAQC. Remember that the data quality and plot type options are noted in the legend below the plot.
- There may be a few cases where certain types of plots that were saved more than a few months ago (before data quality was considered when making plots) may now display differently than before.
Dive Log
- In the Dive Log page, we now display the device ID in the Dive Log page, and allow the administrator to enable dive videos for playing in SeaTube.
- It is now possible to take a series of 9 photo snapshots from the Dive Log page from any of the dive log entries.
- Added a "Quick Observation" feature on the Dive Log page.
- Modified the Dive Log page to use tabs.
- Added the ability to export dive listings and dive log entries to CSV files.
SeaTube
- Added a "permalink". Whenever a video is playing, if you click on the Permalink icon in the upper right section of the playback window (the icon that looks like a chain) it will give you a link that you can, for example, e-mail to someone. Anyone using this link will be directed to SeaTube where the same video will start playing at the same point in time where the link was created, including the dive log.
WARN
- Progressed with the development of the tsunami-detection section of the WARN (Web-enabled Awareness Research Network) project. The DART detider algorithm is now in place to remove the effects of tides on BPR (bottom pressure recorder) data in order to detect tsunamis. In this release the de-tided signal will now be computed as each data sample is received, and the results stored as new derived sensors called "DART Detided Pressure". At this time this is only for data collection purposes. The only BPRs this applies to are:
- 22790 Folger Deep
- 21301 Cascadia Basin (CORK)
- 21503 Barkley Upper Slope
- 22503 Clayoquot Slope
- Some analysis of the de-tided signal is being done using an STA/LTA algorithm (short term average vs. long term average) but the results are not being stored and no actions will be taken yet as a result of the analysis.
Miscellaneous
- Reprocessing now handles mismatches between DMAS timestamps and instrument timestamps in the data.
- Developed a generic parser framework that will make it easier and more consistent to develop individual sensor data parsers from now on.
- In order to support the Nikon DSLR camera recently deployed at Endeavour, we added the ability to see still pictures quickly in an FTP folder (go to More >> FTP directory, open the Parent Directory, then open the folder "NikonCameraDeviceId23090").
- Added commands to our internal scripting language "Seascript" to support automated observations using the Nikon camera.
- Developed a driver for the Satlantic Nitrate SUNA sensor on the VPS (vertical profile system or "Pogo").
- Developed a LOKI driver so we can start archiving navigation data from the ROPOS ROV.
15 May 2014
Digital Fishers
On May 15, 2014 a new Digital Fishers campaign ("Sablefish II") was released. This involved some (temporary) changes to the DMAS software:
- Allowing videos to play in their entirety (rather than 15 seconds). All videos chosen for this campaign are approximately 1 minute long.
- Not allowing videos to be annotated until after they had played to the end.
- Requiring the sablefish count to be a mandatory annotation.
28 April 2014
Data Search
- Improved the performance of searches for complex data products in cases where the file already exists in the archive.
- Reduced the size of hydrophone spectrogram and ADCP PDF files, which will allow them to be created more quickly.
- Modified ADCP current plots to explain that black areas indicate areas where there is no data.
Plotting Utility
- The Plotted Values table now displays the QAQC (quality control) flag for each data point.
- You can now choose between "Clean" and "Raw" data when plotting data using Options >> Plot Properties. "Raw" displays all data even if some values may not be valid (eg. temporary spikes). "Clean" displays only data that has passed the QAQC tests. NOTE that QAQC tests have not yet been defined for all instruments and all time ranges; if no tests are defined for your instrument and time range, "Clean" will display the same as "Raw". At this time the only way you can tell if QAQC tests are defined is to check the "Flag" column in the Plotted Values table - a 0 value means no tests defined, 1 means all QAQC tests passed, 4 means one or more QAQC tests failed. For complete definitions see the CSV or ODV (TXT) data products in Data Search or the Time Series Scalar Data documentation.
- The information icon in Plot Properties (letter "i" in a blue dot) now gives some information about "Clean" and "Raw".
- The label under each plot now indicates whether "Clean" or "Raw" data is being displayed (again, "Clean" assumes QAQC tests have been defined for your instrument and time range).
- "Average" and "All" has been renamed to "Downsampled" and "Not Downsampled" in Plot Properties.
Internal
- Made some minor improvements to the Cruise maintenance pages.
- Created an FTP job to archive images captured by the Nikon D800 DSLR camera (which is not yet deployed).
- Added a "Take Event Sample" button for Sediment Traps in the device attribute page.
- Created a new group of Observatory level QAQC tests for mobile instruments.
2 April 2014
Digital Fishers
- Developed a new way of creating new Digital Fishers campaigns. See the Digital Fishers Campaign List item on the Tools menu. This should make it easier for non-programmers to create and test campaigns in the future before deploying them. Up to now it was not possible to view a new campaign without actually deploying it but now it can be previewed and corrected if necessary. Further improvements may be coming later. The Digital Fishers Video Management page has been hidden but is still available if needed.
- Relocated the videos played by Digital Fishers from Amazon to our internal servers.
- Upgraded SeaTube and Digital Fishers to use the latest version of JWPlayer (6.5) for streaming videos. This will allow videos from these applications to be viewed on iPhones and iPads.
Data Search
- Searches for most data products should now be faster and more reliable due to file system improvements.
- Added a hydrophone spectrograms data product choice to Data Search for the Arctic network.
- Rebranded all Data Search plots (PNG and PDF) to use the ONC logo.
- Greatly improved the speed and reliability of searches for BioSonics Echosounder .MAT data products.
- Most Arctic network data products now include metadata reports.
Data and Instruments
- Oxygen saturation data is now available from Aanderaa Oxygen Optodes located on ferries.
- Developed a parser to handle data from the navigation device used on ships we use on our maintenance cruises, and also from the navigation device used on our ROV submersibles.
Internal Features
- When a queued task is canceled, the name of the host that was running the task is now displayed.
- Did some planning and initial software development for the WARN project (Web-enabled Awareness Research Network) which will eventually provide notifications that the start of an earthquake and/or tsunami has just been detected in our area.
- Made the canceling of running tasks more robust.
- Scientific data is now written to the Cassandra database as well as to Oracle. This does not include quarterscalar data. Many other tasks were completed in the migration to Cassandra.
- Modified the driver for the Omegabus to make it more flexible.
- Migrated all Data Search data products (including metadata reports), quick plot, and Plotting Utility to use the new quarterscalardata table for all cases where data averaged over periods of 15 minutes and above are needed. This should improve performance, simplify the software, and reduce the chances of errors.
- The Simulator for the RM Young meteorological station now supports both push and pull modes.
18 February 2014
Data Search:
- Improved the rendering of hydrophone spectrograms when using the PDF format.
- Improved the contents of Data Search failure e-mails to include data options and site information.
- When data products are generated, now the status will change to "Completed" only after both the data product and metadata report have been generated.
- Sped up production of COVIS and BioSonics data products where the files already exist in the archives.
QA/QC
- Improved the QAQC section of metadata reports to show the test limits (attribute values) in some cases.
- Finished implementing Observatory level QAQC tests for all observatories (rather than just VENUS).
Other:
- In Task Monitor, tasks that cannot be cancelled while running now have their check box disabled so you cannot attempt to cancel them. As well, jobs that are in the process of being cancelled can be found by checking the new "Cancelling" checkbox in the filter area.
- Changed SensorCode to SensorName in SNMP traps from junction boxes.
- In the Junction Box Alarm page, the tabs are now in order of J number rather than breaker number.
- Made additional improvements to support the eventual switch to the Cassandra database.
- Developed a parser for the SCIMPI device.
- Modified the Site Maintenance page to handle mobile devices, such as devices mounted on ferries and drifting buoys.
- Improved the Omegabus transmitter driver/parser.
- Developed a UDP simulator for the Nikon camera.
- Developed a new parser for the Wetlabs ECO-FLS.
23 January 2014
Data Search:
- Prepared Data Search to handle the data products for the new SCIMPI device deployed to Clayoquot Slope (running autonomously for now) but will not be made searchable until the data is ready.
- Added additional data products for hydrophones, including .wav and .mp3 audio, .png and .pdf spectrograms, and .mp4 videos to show moving spectrograms with sound. Note that only limited videos are available at this time.
QAQC:
- Added a new feature to allow data files to be annotated with QAQC (quality flag) information. This can be done using the File Upload page (not yet on the menus but available to privileged users at dmas.uvic.ca/FileUpload) or by FTP. At this time the flags are only internal but will affect the QAQC information in data products in a later release.
- Added the initial code to support allowing Observatory level QAQC tests to be created for any observatory (currently only supports VENUS).
- Created a "Station Area Range Test" formula for Station level QAQC tests.
Other:
- Created a new post-processing job for hydrophone data (147).
- Created a new scheduled job (145) for hydrophone spectrogram video generation.
- Changes were made to support the archival of Slocum glider files.