Release Notes

Detailed release notes are available on internal wiki .

Release notes summaries on this page:

• 28 Aug 2014
• 30 July 2014
• 3 July 2014
• 4 June 2014
• 15 May 2014
• 28 April 2014
• 2 April 2014
• 18 February 2014
• 23 January 2014

2013 Release Notes

2012 Release Notes

2011 Release Notes

2010 Release Notes

29 September 2014

Administrative and Internal Features

• Developed a tool to manage search trees, which are the hierarchical lists of instruments and locations used in Data Search and Plotting Utility and other web pages.
• Created a 30 second summary table in the database, similar to the existing 15 minute summary table. This will reduce the processing required, for example, when plotting sensors with a high sampling rate when it is necessary to subsample between 1 minute and 10 minutes per data point plotted. This will allow the system to support a larger number of concurrent users as well as improve the performance.

 Other Applications

 Cambridge Bay Lite page

• Created a hidden web page on Oceans 2.0 called CambridgeBayLite that displays the latest camera images and key sensor readings (like air and water temperature, wind speed, ice thickness). All text is in both English and in the Inuiannaqtun language. It will be displayed with no login required at the local community center.

ONC Website (oceannetworks.ca, Learning >> For Educators >> Community Observatories pages)

• Added the ability to view the most recent underwater and (where applicable) dock camera videos for the community observatories (currently Mill Bay, British Columbia and Cambridge Bay, Nunavut.)
• Added "quick plots" to show the data from user-selectable sensors and data ranges. These plots automatically update over time, and it is not necessary to have an account or to log in in order to see the data. They can be copied and pasted into a user's document.
•  Added the ability to copy and embed HTML code into a user's web page in order to plot data of their choosing from one of the community observatories.

28 August 2014

Data Products

• Added "pressure VPS" data (ie. pressure from the CTD on the VPS in order to establish depth) to the data products for the SBE63 dissolved oxygen sensor on the VPS (Vertical Profile System).
• Modified all data products for all mobile devices to include whatever mobile position data is available for each reading (latitude, longitude, depth/altitude, pitch, roll, heading). This applies to all mobile devices such as Buoy Profile System (ie. Saanich Inlet Profiler), Vertical Profile System (ie. Barkley Upper Slope Profiler), ferry instruments, gliders, ROVs, ice buoys, etc.
• Improved profile plots and added daily inshore profile plots to the website.
• Initial changes to ADCP plots to make them more consistent across networks. Eg. changed intensity plots from compensated backscatter to RSSI, smarter ranges, removed extra time stamps, etc.
• Implemented standardized spectrogram data products for all hydrophones, enables posting live data to the website (will be brought online mid-september pending military driver changes).
• Implemented a tool to bulk import annotations. This will enable us to import libraries of hydrophone annotations, among other applications.
• Added clean option for quick plots.
• Finalized Kongsberg Rotary sonar data products, including handling of big files, speed and plotting improvements.

WARN

Progressed with the development of the tsunami-detection section of the WARN (Web-enabled Awareness Research Network) project:

• Added a UI page to subscribe to events such as the detection of a tsunami or earthquake. (Eventually subscribers will be notified that an event has occurred).
• Improved the way events (in general) can be defined.
• Added peak tsunami wave height to the Watcher output.

Miscellaneous

• Added a log message to system commands sent when using System Console.
• Added the ability to view the Sensor Serial Number on the Device Attributes page.
• Updated the driver for the Vertical Profile System.
• A driver was developed for the VENUS SIIM (Scientific Instrument Interface Module) so we can start to collect Engineering data for these devices.
• Developed post processing software to create spectrograms for hydrophone audio data.
• Created a new scalardata_30 table in the database to improve performance of Plotting Utility and Data Search. It is similar to the quarterscalardata table in that it summarizes scalar data on a 30 second basis (vs. 15 minutes for the quarterscalardata table). It will not be used until it has been fully populated with old data.
• Started storing all Engineering sensor data in the Cassandra database instead of in Oracle. Made other modifications to implement more of the database functionality in Cassandra.
• The summary tables now included Engineering data as well as Scientific data. This will improve the performance of Plotting Utility and CSV searches in some cases.
• Made additional improvements to the Hydrophone Viewer (still a hidden page).

Other Applications

• Made significant progress on the development of an Android cell phone app that will be used by fishermen to record data from instruments deployed over the side of the fishing boat. The data will be GPS tagged and can be sent to ONC once a cell or WiFi signal is available.

30 July 2014

• Added a termination date option to driver schedules.
• For Data Search plot data products (PNG and PDF), changed the default quality type to "clean" (from "raw").
• In Plotting Utility and in Data Download, when Sort by Instrument, the list of deployed sites will no longer include sites that are marked "not searchable". This will eliminate sites where no data has been collected.
• Added Kongsberg rotary MATLAB and plot data products for both sweep and scan mode.
• Improved the performance when using the "double-click" function in Plotting Utility to redraw the plot in order to rescale the vertical axis.
• Added the ability to write and manage SeaScript scripts to control instrument via a SeaScript tab on the Device Details page.
• Modified the CORK and BPR calibration formulas to read the instrument coefficients. No change in functionality but should be easier to change when necessary.

Vertical Profilers

• New Cast Status and Cast Type derived sensors have been added to the Barkley Upper Slope Profiler to allow the data products to produce more relevant results because they will be able to more accurately determine when casts start and stop.
• Data products for the Oxygen Sensor devices (piggybacks) on the Barkley Upper Slope Profiler will now include pressure data.
• Created a scheduled job to create daily plots for the Barkley Upper Slope Profiler and Saanich Inlet Profiler. These plots will soon be added to the oceannetworks.ca website as a new page for vertical profiler data. 
• Created a SeaScript script to control the Saanich Inlet Profiler on an automatic schedule.

Hydrophone Viewer

Made the following improvements to this new tool, which is currently only available on a hidden web page:

• Created a zoom feature for use on the spectrogram preview.
• Clicking on a spectrogram displays it full screen.
• Select individual hydrophone segments to the cart for download.
• Ability to select an entire day's worth of spectrograms for download.
• Ability to deselect a previously selected segment if download hasn't been requested yet.
• Ability to move to "next day" or "previous day".
• Display an icon when a spectrogram preview is not available but other data formats are available to download.

 WARN

Progressed with the development of the tsunami-detection section of the WARN (Web-enabled Awareness Research Network) project. Added the following derived sensors to record the output from several tsunami detectors:

• Direct Detider Weighted Threshold changes state if the detided pressure ("DART Pressure Residual") value exceeds a threshold.
• Signal to Noise Ratio (using the STA/LTA algorithm) - outputs the ratio of the short term average to the long term average of the detided pressure.
• Signal to Noise Ratio Weighted Threshold - changes state if the Signal to Noise Ratio exceeds a threshold.
• WARN Kertosis - outputs the result of the Kertosis statistical analysis algorithm.

In addition the WARN Threshold Watcher derived sensor amalgamates the outputs from all the tsunami detectors. All of these derived sensors have been added only to the following bottom pressure recorders (BPRs):

• 22790 Folger Deep
• 21301 Cascadia Basin (CORK)
• 22503 Clayoquot Slope
  

3 July 2014

•  Added two new scheduled jobs (157 and 158) to run specified data searches on a regular basis. These can be used to produce, for example, daily plots for the same instrument. Plots like this will be added to the ONC website soon.
• Changed Plotting Utility to make the default plot "Clean Min/Max+Avg" rather than a "raw" plot that includes data that is probably not valid.
• Modified the DeviceAttributesService to return Nortek attributes within a specified time range.
• Added ancillary tilt data to Satlantic Radiometers data products and improved how the data products work with Satlantic's analysis software Prosoft.
• "Derived sensors" are pseudo sensors whose values are derived mathematically from other sensors, typically on the same device. The data products and metadata for derived sensors was changed to include QAQC (data quality) flags inherited from their parent sensors.
• We are now storing quarter scalar data in the Cassandra database in addition to what was already being stored. At this time the Oracle database remains the primary database. A switch to Cassandra is forthcoming, but in the meantime it is possible for users to be added to the "Cassandra Group" in User Management. This will cause all plots made by Plotting Utility to use data from Cassandra.
• Added a parser for IC Listen hydrophones.
• Improved the performance of the QuarterScalarDataSummary job.

4 June 2014

Plotting Utility

• QAQC (data quality) selection has been moved from Options >> Plot Properties to Options >> QAQC. The default for new plots is "Raw - min/max" and there are no other plot types that are supported for raw data. If you want a different plot type (line, bar, etc) you must select "clean" first using Options >> QAQC. Remember that the data quality and plot type options are noted in the legend below the plot.
• There may be a few cases where certain types of plots that were saved more than a few months ago (before data quality was considered when making plots) may now display differently than before.

Dive Log

• In the Dive Log page, we now display the device ID in the Dive Log page, and allow the administrator to enable dive videos for playing in SeaTube.
• It is now possible to take a series of 9 photo snapshots from the Dive Log page from any of the dive log entries.
• Added a "Quick Observation" feature on the Dive Log page.
• Modified the Dive Log page to use tabs.
• Added the ability to export dive listings and dive log entries to CSV files.

SeaTube

• Added a "permalink". Whenever a video is playing, if you click on the Permalink icon in the upper right section of the playback window (the icon that looks like a chain)  it will give you a link that you can, for example, e-mail to someone. Anyone using this link will be directed to SeaTube where the same video will start playing at the same point in time where the link was created, including the dive log.

WARN

• Progressed with the development of the tsunami-detection section of the WARN (Web-enabled Awareness Research Network) project. The DART detider algorithm is now in place to remove the effects of tides on BPR (bottom pressure recorder) data in order to detect tsunamis. In this release the de-tided signal will now be computed as each data sample is received, and the results stored as new derived sensors called "DART Detided Pressure". At this time this is only for data collection purposes. The only BPRs this applies to are:
  • 22790 Folger Deep
  • 21301 Cascadia Basin (CORK)
  • 21503 Barkley Upper Slope
  • 22503 Clayoquot Slope
• Some analysis of the de-tided signal is being done using an STA/LTA algorithm (short term average vs. long term average) but the results are not being stored and no actions will be taken yet as a result of the analysis.

Miscellaneous

• Reprocessing now handles mismatches between DMAS timestamps and instrument timestamps in the data.
• Developed a generic parser framework that will make it easier and more consistent to develop individual sensor data parsers from now on.
• In order to support the Nikon DSLR camera recently deployed at Endeavour, we added the ability to see still pictures quickly in an FTP folder (go to More >> FTP directory, open the Parent Directory, then open the folder "NikonCameraDeviceId23090").
• Added commands to our internal scripting language "Seascript" to support automated observations using the Nikon camera.
• Developed a driver for the Satlantic Nitrate SUNA sensor on the VPS (vertical profile system or "Pogo").
• Developed a LOKI driver so we can start archiving navigation data from the ROPOS ROV.

15 May 2014

Digital Fishers

On May 15, 2014 a new Digital Fishers campaign ("Sablefish II") was released. This involved some (temporary) changes to the DMAS software:

• Allowing videos to play in their entirety (rather than 15 seconds). All videos chosen for this campaign are approximately 1 minute long.
• Not allowing videos to be annotated until after they had played to the end.
• Requiring the sablefish count to be a mandatory annotation.

28 April 2014

Data Search

• Improved the performance of searches for complex data products in cases where the file already exists in the archive.
• Reduced the size of hydrophone spectrogram and ADCP PDF files, which will allow them to be created more quickly.
• Modified ADCP current plots to explain that black areas indicate areas where there is no data.

Plotting Utility

• The Plotted Values table now displays the QAQC (quality control) flag for each data point.
• You can now choose between "Clean" and "Raw" data when plotting data using Options >> Plot Properties. "Raw" displays all data even if some values may not be valid (eg. temporary spikes). "Clean" displays only data that has passed the QAQC tests. NOTE that QAQC tests have not yet been defined for all instruments and all time ranges; if no tests are defined for your instrument and time range, "Clean" will display the same as "Raw". At this time the only way you can tell if QAQC tests are defined is to check the "Flag" column in the Plotted Values table - a 0 value means no tests defined, 1 means all QAQC tests passed, 4 means one or more QAQC tests failed. For complete definitions see the CSV or ODV (TXT) data products in Data Search or the Time Series Scalar Data documentation.
• The information icon in Plot Properties (letter "i" in a blue dot) now gives some information about "Clean" and "Raw".
• The label under each plot now indicates whether "Clean" or "Raw" data is being displayed (again, "Clean" assumes QAQC tests have been defined for your instrument and time range).
• "Average" and "All" has been renamed to "Downsampled" and "Not Downsampled" in Plot Properties.

Internal

• Made some minor improvements to the Cruise maintenance pages.
• Created an FTP job to archive images captured by the Nikon D800 DSLR camera (which is not yet deployed).
• Added a "Take Event Sample" button for Sediment Traps in the device attribute page.
• Created a new group of Observatory level QAQC tests for mobile instruments.
   
  
  

2 April 2014 

Digital Fishers

• Developed a new way of creating new Digital Fishers campaigns. See the Digital Fishers Campaign List item on the Tools menu. This should make it easier for non-programmers to create and test campaigns in the future before deploying them. Up to now it was not possible to view a new campaign without actually deploying it but now it can be previewed and corrected if necessary. Further improvements may be coming later. The Digital Fishers Video Management page has been hidden but is still available if needed.
• Relocated the videos played by Digital Fishers from Amazon to our internal servers.
• Upgraded SeaTube and Digital Fishers to use the latest version of JWPlayer (6.5) for streaming videos. This will allow videos from these applications to be viewed on iPhones and iPads.

Data Search

• Searches for most data products should now be faster and more reliable due to file system improvements.
• Added a hydrophone spectrograms data product choice to Data Search for the Arctic network.
• Rebranded all Data Search plots (PNG and PDF) to use the ONC logo.
• Greatly improved the speed and reliability of searches for BioSonics Echosounder .MAT data products.
• Most Arctic network data products now include metadata reports.

Data and Instruments

• Oxygen saturation data is now available from Aanderaa Oxygen Optodes located on ferries.
• Developed a parser to handle data from the navigation device used on ships we use on our maintenance cruises, and also from the navigation device used on our ROV submersibles.

Internal Features

• When a queued task is canceled, the name of the host that was running the task is now displayed.
• Did some planning and initial software development for the WARN project (Web-enabled Awareness Research Network) which will eventually provide notifications that the start of an earthquake and/or tsunami has just been detected in our area.
• Made the canceling of running tasks more robust.
• Scientific data is now written to the Cassandra database as well as to Oracle. This does not include quarterscalar data. Many other tasks were completed in the migration to Cassandra.
• Modified the driver for the Omegabus to make it more flexible.
• Migrated all Data Search data products (including metadata reports), quick plot, and Plotting Utility to use the new quarterscalardata table for all cases where data averaged over periods of 15 minutes and above are needed. This should improve performance, simplify the software, and reduce the chances of errors.
•  The Simulator for the RM Young meteorological station now supports both push and pull modes.
  

18 February 2014 

Data Search:

• Improved the rendering of hydrophone spectrograms when using the PDF format.
• Improved the contents of Data Search failure e-mails to include data options and site information.
• When data products are generated, now the status will change to "Completed" only after both the data product and metadata report have been generated.
• Sped up production of COVIS and BioSonics data products where the files already exist in the archives.

QA/QC

• Improved the QAQC section of metadata reports to show the test limits (attribute values) in some cases.
• Finished implementing Observatory level QAQC tests for all observatories (rather than just VENUS).

Other:

• In Task Monitor, tasks that cannot be cancelled while running now have their check box disabled so you cannot attempt to cancel them. As well, jobs that are in the process of being cancelled can be found by checking the new "Cancelling" checkbox in the filter area.
• Changed SensorCode to SensorName in SNMP traps from junction boxes.
• In the Junction Box Alarm page, the tabs are now in order of J number rather than breaker number.
• Made additional improvements to support the eventual switch to the Cassandra database.
• Developed a parser for the SCIMPI device.
• Modified the Site Maintenance page to handle mobile devices, such as devices mounted on ferries and drifting buoys.
• Improved the Omegabus transmitter driver/parser.
• Developed a UDP simulator for the Nikon camera.
• Developed a new parser for the Wetlabs ECO-FLS.

23 January 2014 

Data Search:

• Prepared Data Search to handle the data products for the new SCIMPI device deployed to Clayoquot Slope (running autonomously for now) but will not be made searchable until the data is ready.
• Added additional data products for hydrophones, including .wav and .mp3 audio, .png and .pdf spectrograms, and .mp4 videos to show moving spectrograms with sound. Note that only limited videos are available at this time.

QAQC:

• Added a new feature to allow data files to be annotated with QAQC (quality flag) information. This can be done using the File Upload page (not yet on the menus but available to privileged users at dmas.uvic.ca/FileUpload) or by FTP. At this time the flags are only internal but will affect the QAQC information in data products in a later release.
• Added the initial code to support allowing Observatory level QAQC tests to be created for any observatory (currently only supports VENUS).
• Created a "Station Area Range Test" formula for Station level QAQC tests.

Other:

• Created a new post-processing job for hydrophone data (147).
• Created a new scheduled job (145) for hydrophone spectrogram video generation.
• Changes were made to support the archival of Slocum glider files.