This page contains release notes from 2018, primarily for Oceans 2.0 / DMAS, as well as any applications created/maintained by the ONC software development group, including mobile Apps and desktop/PC applications. In progress project status may be presented as well.

December 6, 2018

Major release. Released a major revision for the Community Fishers App.

Instruments

  • Moved more parsers from java into declarative parsers: 26 more, all but 5 remaining.

  • Added modify by to parser history

  • Released the Community Fishers App to support community-based CTD casts. Major functionality added to support AML CTDs. Additional features to more easily manage and swap devices, online updates, add bluetooth support, improved wifi support, added manual GPS entry.

Web Presence

  • Working on Geospatial search and Data Search UI improvements (no features added yet)

Infrastructure and Internal Tools

  • Continued improvements to web service automation, added servlet testing.

  • Added JVM, service-level and data acquisition metrics.
  • Added ability to turn on/off and configure metrics gathering.
  • Added metrics to system dashboards: graphana.

Maintenance

  • Various bug fixes, including but not limited to: 
  • Allow sensorCategoryCode to be case insensitive
  • Improved cache loading times for ferries
  • Task management parameter modification

Data Products

  • State of Ocean data products are gap filled, handle cases where deployments occur within a resample time period
  • State of Ocean plot maintenance for VENUS cruise
  • Added option to plot both PLAN0 and PLAN1 data for Nortek Signature ADCPs, added both plots to Data Preview
  • Added option for a correlation screen for Nortek Signature ADCPs
  • Added a fail over for matlab webservice calls
  • Added chunking in matlab QAQC result retrieve to prevent running out of memory
  • Allow ice buoy data products to access all sensors
  • Fixed bugs: water column data products, ADCP edge case of one record, handle inconsistent creation and deployment bad dates

Earthquake Early Warning

  • Made deployments faster
  • Used October 22 earthquake events as test cases and made associated improvements

SeaTubeV3 - CANARIE

  • Investigation, various bug fixes, requirements and planning (no features)
  • Map scale in SeaTube
  • Export annotations by dive

Dashboards - CANARIE

  • Investigation and requirements: user interviews (no features)

MINTED - CANARIE

  • Project start up (no features)

Tickets Released: Oceans 2.0: 18.11.000 (accessible to internal users only). Total: 143.

                                Community Fishers App: CS.18.12.000 (accessible to internal users only). Total: 43.


November 5, 2018

Major release.

Instruments

  • Moved more parsers from java into declarative parsers: 38 devicetypes

  • Continuing progress on the Community Fishers App to support community-based CTD casts.

  • Added ability to view previous parser versions

Web Presence

  • Liferay investigation and training

Infrastructure and Internal Tools

  • Continued improvements on web service automation tests: added numerous new test cases, update tests to improve pass/fail.

Maintenance

  • Various bug fixes, including but not limited to:
  • Added modification history to ERDDAP dataset maintenance
  • Added sensorCategoryCode to scalardata service so that users can access a specific sensor's data more easily
  • Improved display of drifters in Data Search map, improved performance, fixed icon sizes

Data Products

  • Live Raw Data / Log files now available through Data Search
  • SIMBA / ice buoy plots to plot by sensor number by default, with new option to plot by sensor elevation
  • Added feedback to users when they're plotting too much data in ferry plots

Earthquake Early Warning

  • Data acquisition support for RBR tilt meters
  • Updated CAP message to meet standard
  • CAP messages available via ftp server

SeaTubeV3 - CANARIE

  • Investigation, various bug fixes, requirements and planning
  • Added sorting by organization to SeaTube search tree.
  • Added hover over to provide more information

Dashboards - CANARIE

  • Investigation, requirements and planning (no features)

Tickets Released: Oceans 2.0: 18.10.000 (accessible to internal users only). Total: 87.

.

October 10, 2018

Major release, branched as normal on September 30th, deployment delayed due to inshore maintenance expedition.

Instruments

  • Moved more parsers from java into declarative parsers.

  • Improved declarative parser UI, including history. Parsers can now be easily updated in real-time from the device details pages.

  • Good progress on the Community Fishers App to support community-based CTD casts.

Web Presence

  • Updates to oceannetworks.ca: community pages, wiring the abyss, science landing page.
  • Conference pages template in liferay 7.1.

Infrastructure and Internal Tools

  • Big push on web service automation tests: 95% now passing due to improvements to the response comparison and checking (uses a lot of regex). This will improve quality of deployed software and speed our release cycle.

Maintenance

  • Various bug fixes, including but not limited to: jwPlayer license fix, google maps API fix.
  • Completed switch to https. Can now add features like JIRA service desk. 
  • Rinko parser update in real-time.
  • Improved Data Search display of drifters and improved loading time.

Data Products

  • HydroCal software delivered to client on September 5th.
  • HydroCal bug fixes and client acceptance feedback fixes (still included in the first release). 
  • State of Ocean plot maintenance

Earthquake Early Warning

  • Testing, bug fixes, support for field deployments. Correlator/associator deployed live. 
  • Automated testing improvements

SeaTubeV3 - CANARIE

  • Investigation, various bug fixes, requirements and planning (no features)

Dashboards - CANARIE

  • Investigation, requirements and planning (no features)

Tickets Released: Oceans 2.0: 18.09.000 (accessible to internal users only). Total: 74.

                                HydroCal: HC.18.08.000 (accessible to internal users only). Total: 7.


September 4, 2018

Major release. 

Instruments

  • Created two new parsers and drivers for both Pro-Oceanus Gas Tension device and SBE 39 device for Queen of Alberni.

  • Moved parser definitions into the database and added a parser definition tab to device listings page for viewing and editing parser definitions from the UI.

  • Updated the Watchman500 and Folger Pinnacle mini-jb parsers to support recent changes.

  • Fixed a bug with SubC cameras and their laser not working correctly; other bug fixes.

Empower Sandbox and API

  • Added a share feature.
  • Bug fixes.
  • Final demonstration took place in August and project was closed.

Web Presence

  • Progress on new pages for annual reporting

Infrastructure and Internal Tools

  • Modularized core infrastructure.

  • Continued to improve handling of dependencies.

  • Began modularizing driver framework.

  • Modularized the device-related code and parts of earthquake correlation code

Maintenance

  • Various bug fixes.
  • Separate Data Preview searches into a new queue for better control and higher priority for user searches.
  • Removed filedownloadservice and several SOAP services.

Data Products

Earthquake Early Warning

  • Completed additions to the associator / correlator event detection, such as using maximum peak displacement for magnitude estimates.
  • Improvements to Kalman Filter.
  • Work on the iPhone App - new protocol

SeaTubeV2

  • Improve permissions handling

  • Improvements to annotations search and filtering. Added annotations export.

  • WMS servers integration to show bathymetry, added a page to configure the bathymetry

  • Improvements to search function

  • Completion NOAA SeaTube project "SeaTube V2"

Tickets Released: 18.08.000 (accessible to internal users only). Total: 130.


August 8, 2018

Major release. 

Instruments

  • Archive Gascoyne Bay data directly for improved performance
  • New commands added to SeaScript (device command language) to clear and flush device memory and gather statistics
  • Bug fixes

Empower Sandbox and API

  • Task Monitor is now Task Management and was moved to the More menu
  • Improve permissions and access on the Task Management page
  • Last minute changes and fixes for the final delivery and demonstration

Infrastructure and Internal Tools

  • Finished refactoring/cleanup of infrastructure code to prepare for modularization.

  • Investigate loosely coupling the driver framework to driver implementations.

  • Improve handling of external dependencies.

Maintenance

  • Various bug fixes.
  • New user interface for data attributions
  • Improve data attributions to remove the citation fields and simplify

Data Products

  • Improvements to daily spectrograms
  • Progress on HydroCal, stand-alone software for ONC's hydrophone calibration system
  • Bug fixes

Earthquake Early Warning

  • Added client software test message sending on a schedule
  • Added the backup datacenter that will receive all data and be able to replace the primary in few seconds or less if the primary fails.
  • Fixes to automated tests

NOAA SeaTubeV2

  • Interactivity between dive trace and video and annotation

  • Annotation export is now a data product

  • Numerous bug fixes for previous releases

Tickets Released: 18.07.000 (accessible to internal users only). Total: 108.

July 3, 2018

Major release. First release using GIT.

Instruments

  • Updated Data Search page to support multiple map providers.
  • Finished and tested drivers and parsers for the four devices related to the SIDSSE project.
  • Finished and tested drivers and parsers for both POCAM’s, SDOM’s, and Mini Junction Boxes  for the STRAW project.
  • Further updates to parsers for Wally devices.
  • Added photo download support for SubC cameras.
  • Finished and tested drivers & parsers for DBoxes.
  • Updates to Loki driver and parser.

Empower Sandbox and API

  • Added support for MATLAB to the sandbox (not yet connected in PROD)
  • Improvements to the Task Monitor UI
  • Added Empower API to CANARIE registry
  • Lots of clean-up and completion of various issues to close the project

Web Presence

  • Numerous updates to support the cruises: banners, modifications to the Wiring the Abyss and expedition pages
  • Prince Rupert Regional Ocean Observatory page (community page meant for a display for the Prince Rupert Port Authority) 

Infrastructure and Internal Tools

  • Migrated to use GIT / Bitbucket as our versioning system
  • Metrics page on search automation
  • Moved search automation into the test tree

Maintenance

  • Various bug fixes.
  • Fixed bug in the CORK parser.
  • Rolled out first iteration of Data Search clustering for Drifter icons.

Data Products

  • Fixed a bug that affected some cases of RDI ADCP bin-mapping, causing an error in values when compared to winADCP of about 0.3% overall on average.
  • Re-write of seismometer data products to handle new seismometer metadata and configurations.
  • New daily collated spectrograms for hydrophones. These new plots were also added to Data Preview.
  • Additional bug fixes.

Earthquake Early Warning

  • Added functionality for EEW machines to reload and update the caches over activeMQ. 
  • Added a web service to allow beta testers to generate CAP messages and verify that their software correctly receives and reacts to a cap message.
  • Driver and parser for RBR tiltmeters.

NOAA SeaTubeV2

  • Hide video for a dive.

  • Display attributes last used for selected taxon.

  • Export sensor values for each annotation.

  • Filter annotations by sensor values (NOAA only).

  • Set livestream URLs per cruise, values not replicated so they can be defined differently for different machines.

  • New Taxon Button Set Configuration screen which displays a preview of button sets.

Tickets Released: 18.06.000 (accessible to internal users only). Total: 175.


June 6, 2018

Major release.

Instruments

  • DAF testing, driver and parser development and support for upcoming cruises, in particular, support for new devices for the STRAW project and new installations at Endeavour

Empower Sandbox and API

  • Added automated testing for Sandbox
  • Improvements to RawData service

Infrastructure and Internal Tools

  • GIT migration research

Maintenance

  • Various bug fixes

Data Products

Earthquake Early Warning

  • Improvements to correlator queue

NOAA SeaTubeV2 and Fisheries Science Reporting System

  • Numerous improvements for the taxonomy system, including UI elements
  • Added WoRMS common names
  • Export annotations with associated environmental and positional data, export annotations for a single dive with taxon defintion
  • Add the ability to view, edit, create taxons and taxonomy, including creating button sets and previewing their layout
  • Added some support for multiple simultaneous streams

Tickets Released: 18.05.000 (accessible to internal users only). Total: 166.


May 7, 2018

Major release, several new features and official releases.

Instruments

  • Drivers for POCAM Camera, Mk. 5 + 6 (STRAW project)
  • Updated Camera Controls
  • BPR wakes more reliably
  • PVCS (ferry value control system) and QAQC changes (quality assurance quality control flagging)

Empower Sandbox and API

  • Officially released the API to the public
  • Fixed a number of automation tests
  • Added new automation tests for sandbox services
  • Added a parameter to get the latest reading for external web services
  • Split sandbox tasks from scheduled tasks and added permission checking to TaskMonitor
  • Fixed an issue where row limits for scalardata were not being applied correctly
  • Added support for compiled (C/C++ code)
  • Did an iteration on the UI design of TaskMonitor
  • Checked the acceptance criteria for the API
  • Modified archivefiles service to have a similar structure as other services

Infrastructure

  • Continuing cleanup and modularization (OSGi-fication) of core infrastructure code: extracted observatory and searchtree modules, preparing to extract device, sensor and deps modules
  • Improved cache handling

Maintenance

  • Various bug fixes
  • Instrument report page (for data stewardship team, but could be made accessible)

Data Products

Earthquake Early Warning

  • Progress on epicentre and magnitude 'correlator', notification and message routing architecture
  • Added a new calibration feature: external device triggering
  • Can now send a serialized cache over ActiveMQ to remote stations (allows easier deployments and updates to remote stations)

NOAA SeaTubeV2

Completed almost all NOAA requested enhancements including:

  • Live dive annotation and display
  • Annotations include standardized vocabulary (taxonomies)
  • Annotation quick entry
  • Optimized screen use
  • Environmental values display

Fisheries Science Reporting System

  • Finishing up event scheduling including automated scheduling using Liferay’s quartz functionality

Tickets Released: 18.04.000 (accessible to internal users only). Total: 289.


April 3, 2018

Larger-than-normal release in preparation for several official releases and demonstrations of significant projects in the next month or so.

Instruments

  • Parsing data from two additional Placentia Bay Buoys for Memorial University

Empower Sandbox and API

  • Standardized date parameters for web services
  • Added a second output format for the RawDataService
  • Standardized archivefiles service to conform to the ONC API standard
  • Sandbox is ready for user testing in the QA (limited access) environment ("alpha testing" phase)
  • Improvements to the Task Monitor and Scheduler page

Infrastructure

  • Continuing cleanup and modularization (OSGi-fication) of core infrastructure code

Maintenance

  • Various bug fixes
  • SeaTube, Digital Fishers and the live Cameras pages now support html5 playback - this is a big improvement for mobile devices and browsers like Chrome.
  • Frame grab improvements in SeaTube, including not having to log in. 

Data Products

  • Drifter data products: new track plot data products are available to support DFO drifters. Formats include PNG/PDF, and animated GIF and a Google Earth KML file.

Earthquake Early Warning

  • Progress on epicentre and magnitude 'correlator', notification and message routing architecture
  • Updates to calculated values for magnitude estimation for the updated 'correlator' event detection
  • Completion of the routing slip infrastructure. The routing slip allows data processing on shorestations to be dynamic and go to go across multiple machines. In the case of EEW it allows us to send data from one shorestation to a second central shorestation for event detection.

DFO and NOAA projects

  • Released a preliminary version of SeaTubeV2, which is an enhanced version SeaTube combining SeaScribe and SeaTube functionality targeted for NOAA. This isn’t the official release, that will happen at the end of April.

  • Continued work on our Event Management functionality including full automation coverage. Most functionality is complete; events can be created and run and notifications generated. 
  • Modifications to handle thousands of drifters (Data Search step 1 primarily).

Tickets Released: 18.03.000 (accessible to internal users only). Total: 232



March 7, 2018

Normal release. Ramping up towards completing several large projects.

Instruments

  • Parsing data for the Red Island Shoal Buoy at Memorial University
  • DFO Drifter jobs implemented (transfer, archiving, parsing data, plus cloning to generate new drifter devices)
  • DFO Drifter parser and QAQC

Empower Sandbox and API

  • Progress on building a sandbox for user to upload and run scripts (nothing released yet)
  • Upgraded our old scheduler and put the functionality into the current Task Monitor page
  • Standardized API calls that use time parameters
  • Fixed a bug in the API where our log was filling up with useless messages
  • Fixed a bug where certain API calls would cause a Null Pointer Exception

Infrastructure

  • Partial cleanup and modularization of core infrastructure code

Maintenance

  • Updated SeaTube to stream only non-Flash video (html5)
  • Updated SeaTube’s JWPlayer instance which added a few new things like 2X and 0.5X speed playback
  • Added the ability to cancel individual queued reprocessing files on the Reprocess Console
  • Added ERDDAP Dataset information to the Site, SiteDevice and Device pages
  • Added an automated abstract submission confirmation email feature to the ISSMMT conference page
  • Created a site for the CVAUI2018 Workshop
  • Various bug fixes

Data Access and Products

  • Data Preview now uses https for embedded images (plots)
  • State of Oceans / Environment plots will now always have yearly ticks
  • Water column data products in final review and automated testing phase

Earthquake Early Warning

  • Progress on epicentre and magnitude 'correlater', notification and message routing architecture
  • Automated testing improvements, reliability

DFO and NOAA projects

  • For NOAA SeaTube, continuing progress, lots of great features in progress but no visible changes for users
  • For the fisheries data repository, completed much of the Event Management functionality along with good automated test coverage. It is now possible to create events and test them to determine if the content and recipients are correct.

Tickets Released: 18.02.000 (accessible to internal users only). Total: 135.

February 5, 2018

Normal release, now using the fix version feature in JIRA (all released tickets will now be linked below).

Instruments

  • Support ASCII for CORK devices

  • Add BPR data import as a scheduled job

  • Data acquisition for Memorial University Buoys and DFO drifters (in progress)

  • Filtering out bad data (QAQC) for some ferry instruments when vessel is in port and/or pump flow is reduced

Empower Sandbox and API

  • Progress on building a sandbox for user to upload and run scripts (nothing released yet)
  • Modifications to API for standardized nomenclature and new methods (rawdataservice)

Infrastructure

  • OSGi modularization (on-going)
  • Performance improvements

Data Access and Products

  • Improvements to data access restrictions and attribution
  • Links added to State of Oceans / Environment plots in Data Preview: users can get the data and file behind these plots
  • fileSize reporting for API
  • Water column data product preparation (PropertyCode now appears in MAT files, more features to come)

Earthquake Early Warning

  • Progress on EEW heartbeats, notification architecture, iPhone app
  • Automated testing improvements, reliability

DFO and NOAA projects

  • Continuing progress, lots of great features in progress but no visible changes for users

Tickets Released: 18.01.000 (accessible to internal users only). Total: 231.

January 3, 2018

This was a smaller-than-normal release, due to the holiday break.

Instruments

  • Underwater Listening Station (ULS) parser for V2 AMAR hydrophones

  • Improvements to the Pump Control and Valve System (PVCS) for the ferries

  • Filtering out bad data (QAQC) for some ferry instruments when vessel is in port and/or pump flow is reduced

Empower Sandbox and API

  • Progress on building a sandbox for user to upload and run scripts (nothing released yet)
  • API returns a 503 error when server down for scheduled maintenance

Data Access and Products

  • Improvements to data access restrictions (applies to both API and front-end apps like Data Search)
  • Updates and bug fixes for State of Ocean plots
  • Bug fixes for seismometer, video QAQC (corrects for time index mis-match ..finally!), ODV with many deployments
  • Bug fixed by adding a new feature for spectrograms: an audio data only option

Earthquake Early Warning

  • Reduced the size/bandwidth of remote station data traffic
  • Improved reliability and maintainability of remote stations
  • Improve infrastructure to better support notifications (in progress)

DFO and NOAA projects

  • NOAA Seatube initial release: prototype for NOAA annotation-makers (this is an html5 enabled version of SeaTube, specific for NOAA)
  • DFO project sites permission and user management updated to more closely match liferay