This page contains the current year's release notes and new features, primarily for Oceans 3.0 / 2.0 / DMAS, as well as any applications created/maintained by the ONC software engineering group, including mobile Apps and desktop/PC applications. In progress project status may be presented as well. The information below is written in plain language as much as possible. "Major" releases include all completed items at that time (monthly), while "minor" releases are updates to the previous major release with bug fixes and specific features. Detailed release notes are available on internal confluence pages (internal users only).
[DMAS-73853] - FIle postprocessors fail if the source directory contains unexpected files
Task
[DMAS-72998] - Add database check constraints to taxonbuttonsetline table
[DMAS-73774] - Add category for new turbidity piggyback device to list of possible community fishers device category's
[DMAS-73775] - Consolidate definitions for possible community fishers device category's to one location
[DMAS-73798] - As a dev, I would like to see some improvements about pop-up warning when data would be better accessed through the API
[DMAS-73808] - Don't pass empty strings as comments to TaxonButtonSetService
Implementation
[DMAS-73746] - Update service call to return DMASUsername and Id instead of just Id
[DMAS-73796] - UI display for dmasuser name and id
[DMAS-73832] - Turbidity data from piggyback device is written into data file for CTD
September 8, 2022
Major release
CIOOS
Upgraded CKAN production version and updated documentation
Dashboards
Widgets now display a descriptive message when required configuration details are missing (part 1 of error handling, part 2 deals with missing data)
Data Policy
Ongoing, work has progressed on page for creating licences and licence management
SeaTube
continue on WoRMS update
continue on Deck Log implementation
continue on Map Layer improvements
continue on UI tests implementation
bring the search field values to advance search
don't allow anonymous users to create playlist
click edit button will not set video seek time (fix)
add highlight for annotation editing
improve live/historical mode for SeaTube
add SeaTube deep link for SeaTube exports
other bug fixes
Internal Projects
Bugfix: production crashes from reading junction box data with /api/scalardata
Bugfix: possible production crashes from aborting requests to /api/dataProductDelivery
Improvements to web service logging
ActiveMQ client upgraded to 5.16.5
Cleanup of code dealing with ArchiveFile filename parsing
Cleanup of Cassandra scalardata/rawdata archiving code
Maintenance Sprint
Data Search
Internal email notification for when a data search is cancelled or killed
Pop-up warning to suggest user to use API
Remind user that he/she does not have to have browser open for searches to complete
User receive an email even if a search fails
New feature or improvement
New Device Category page written with React
Filter to display list of active events on event detection page
Show maximum number of characters and the count in a text field if there exists a maximum
Re-organized fields in user profile
Add confirmation dialog to geospatial map if close selected while still drawing
Bug fixes
wiki link for Digital Fishers on the landing page
Dataset Landing page error when QueryPIDs have no DOIs
JunctionBoxalarmSetting page if one JB have port 11 or 12
Instruments with a hyphen in the name are not added correctly in user management page
Click "search data product" buttons causes screen to go black in search map when fullscreen
EEW
In-progress: simulation work, new message formats
API
OpenAPI project initiation
Data Products
BioSonics Echosounder data products: in addition to the existing DT4 manufacturer's binary format and hourly raw MAT files, added calibrated and resampled MAT files, plotting (PNG and PDF formats) on daily and multi-day bases, daily plots for Data Preview, a CSV format for EchoView import, plus documentation
Seismometer data product improvements: use the "All" option in Data Preview so all channels with be plotted, plot title improvement, improve error handling
Minor updates for documentation, links, Sound Metrics speed improvement, final search result update, one bug fix, add turbidity device to CF products
Community Fishers
Restrict date range to 1 hour when creating a manual cast
Download XML reports from the map
Added the ability to show CF cast history to search tree maintenance
Removed Cruise Information popup on Community Fisher App
Improved response time for getAll siteDeviceSubset with expanded option
Stopped App from sending cast data (3 sets of files) multiple times on poor connections
Instruments
Updated Didson postprocessor to produce DDF files with more accurate file/frame headers, including device attribute values
Driver update for ONC lights and other driver maintenance
[DMAS-73527] - Fix Popup for Deleting Subsets when Editing Dates
Task
[DMAS-69513] - update job 177-seismometer-SOH-batch-ftp-job to take in new seismometer file names produced by John D's driver
Implementation
[DMAS-73396] - Copy existing parserdefinition rows without current transaction records into parserdefinition_trx
[DMAS-73484] - script for a job to move DMAS files to Archive folder
August 10, 2022
Major release
Instruments
Didson driver fixes, improvements:
Set parameters command has been fixed and will now set all parameters (went out in earlier minor release).
A new acquisition mode 'burst mode' has been added (capable for higher frame rates) and some changes to driver behaviour were implemented.
Driver now sets the device's time on driver start. No other commands are sent on driver start.
When under driver control (not using seascript), the driver will send the following commands before polling: Set Time, Set Parameters using attribute settings
New command burstModeCapture. This command is the same as startCapture except retrieves data using burst mode.
When polling under seascript control, the driver will send the set time command whenever executing the startCapture or burstModeCapture command.
During testing the focus setting would sometimes drift during the acquisition cycle. The driver now checks the focus setting and will adjust if necessary once every 5 seconds.
Rayfin and camera control fixes/improvements (went out in earlier minor release).
Community Fishers
Added MATLAB memory usage logging
Added check to warn of a possible cast deletion when creating a new new site device subset
Fixed missing aligned and depth binned cast data for T'Sou-ke on June 20 2022
Created separate directories for the scheduled and manual pipeline jobs
Upgrade creating overlapping subsets alert to not allow the creation
Fixed Data Search to download Tablet Annotations
Added ability to view annotations for a station's most recent cast on the GeospatialMap
Handle overlapping empty castdata files in ExternallyDerivedSensorDataIngester
Updated toolkit and protocols into Android app
Upgraded sitedevicesubset maintenance page with better workflow, can no longer enter a siteid for a sitedeviceid
Created and tested App version CF.3.2.7
Added 'MANUAL' OR 'MEASURED' GPS flag to App raw data upload
Added APP feature to view the latitude and longitude data from the tablet after data has been collected with the CTD
Changed App Aanderaa oxygen units from µm/L to µmol/L
Corrected validation for 3D WKT on SiteDeviceSubsetMaintenace page
Modified the MethodToCreateCast to use deviceId and Date Range to get siteDeviceId
Removed im_sitedevicesubsetcastdefinitionid_fkey on im_SiteDeviceSubsetCastDefinitionTransaction table so SiteDeviceSubsetCastDefinitions can be deleted
Added check to see siteDeviceSubsetId has siteDeviceSubsetCastDefinition in MethodToDeleteCast
Set default site device subset type to "downcast" instead of upcast on SDS maintenance page
Prod-error: deleted SiteDeviceSubsets with 10 year date range
Data Products
Update for seismometer data products:
Improve option sets and code to support accelerometer channels, add an all velocity option
Improved handling of filter options and overrides, improved plot title and comments in that case
Fix display of Maris accelerometer data in Data Preview
Fix scaling issues, add a new scaling parameter for Maris "C" channel
Improve feedback to user when selected channel option is not available
Documentation updates for new Data Search time/size estimates, new annotation UI, all data product options now have a link to their documentation
Updated time/date formats for internal services
Prepare for BioSonics data products: add to Data Preview, configure option sets and documentation
Bug fixes for: bad RDI data, ferry and profile product edge cases, old hydrophone data
Maintenance Sprint
File Management page refactored to REACT standard with a new ingestion ready feature
Remove Budget module code
Improvements: autocomplete deviceid for Device Sync page; not allow future date for datefrom in deviceattributeservice
Fix bugs:
PU: downsampling issue, download button, no datarating situation,
data search: timeout stuck and response queue for cancelled search, help icon for send email, data availability for all available option,
Device Details: data error in sensor additional attribute tab
API: RawData service return null when should return error message
Digital Fishers: "about us" link
System log: flexjson.JSONException: Error trying to deepSerialize
Camera control page
Date agreement tab on Network Console
Selenium tests
EEW
Fix auto refresh issues for eew dashboard
Fixed convergence check in Kalman filter
Full earthquake simulations working and mostly automated
[DMAS-73037] - QA web server failing to find missing resource "ResourceManager : unable to find resource 'ca/uvic/dmas/core/system/serviceframework/externalapi/proxy/ApiProxy.vm' in any resource loader."
[DMAS-73094] - Rawdata service returning null when it should be returning an error message
[DMAS-73097] - Oceans Next CI builds fail with "@types/lodash not accessible from @storybook/mdx1-csf"
[DMAS-73135] - RawData service not properly validating date parameters
[DMAS-73157] - Data preview not available for Maris
[DMAS-73222] - As a annotation creator a negative sign is now appearing at time-stamp
[DMAS-73244] - Search automation user can't rerun searches
[DMAS-73255] - ERDDAP Currents Records don't seem to resolve when added to erddap table
[DMAS-73266] - SiteDeviceSubsetMaintenace Page does not validate 3D WKT correctly
[DMAS-73275] - Some searches aren't being queued after 22.06.002
[DMAS-69106] - Rayfin driver bugs related to takePicture
[DMAS-72975] - Prevent bad RDI ADCP SoG East data from getting into the RDI files
[DMAS-72990] - Data Search shows data availability when there is none
[DMAS-73014] - QA stops responding at the same time every night
[DMAS-73040] - Missing aligned and depth binned cast data for T'Sou-ke on June 20 2022
[DMAS-73137] - Fix mmi:MD_MaintenanceInformation not expected in ISO 19115 XML
[DMAS-73150] - The data availability bar is not showing the data availability correct for both DDL Sonars- search by location ( see ticket body for details)
[DMAS-73152] - Camera Control Page Commands failing
[DMAS-73171] - New data products not appearing under Attributions in Data Agreements in Network Console
[DMAS-73233] - Default attributes not populating when taxon buttons used
July 12, 2022
Minor release: fix excess logging and ISO 19115 extra field (needed to restart web server for a start up issue anyway)
Infrastructure Management: performance improvement for data agreement on Network Console; fix spelling typo on device detail page; improvement on device attribute upload feature; AdFile return message for no file case; Remove Budget module (half way, still need keep some features); re-write File management page (half way)
Plotting Utility: fix plot title, Support Sensor level data ratings (also applied for all web services); save/download all plots in one file; show/hide plot legends for all plots at once; data gap when no exist raw data;
UI Test Automation: fix failing tests and add new test cases
CF
Geospatial Map now places multiple downloads lasso selection into one search results folder
Added a manual reprocessed flag (M) to Site Device Subset Maintenance and to Search Tree Maintenance
Built tests for CF date.From and Date.To Task Job parameters
Removed "live" reviewed check boxes from Search Tree Maintenance, replaced with flag (R)
No longer render Geospatial Area Field in Form for casts in Site Device Subset Maintenance (data model update)
Updated stale Contact Us link on Android App
SeaTube
In-progress: Bounding Box with animation
In-progress: WORMS taxonomy update
In-progress: configurable annotation table demo for deck log
In-progress: add ONC internal deck log page to display the annotations with filters (supports both AND/OR logic)
UI testing improvements
Bug fixes and small usability improvements
Internal projects
Java 17 upgrade (in progress)
Simplify date parsing/formatting code
Automatic dependency upgrades (in progress)
Annotation UI Updates
Annotation V2 fully integrated, this includes using Annotations V2 in the following:
More menu
Hydrophone Search
Plotting Utility
Data Search
Device Listing
and more
Tweaked Annotation Entry display so it scrolls if content doesn't fit vertically
CIOOS
National updated so deleted CKAN sets will be removed after deleted in RAs
Translation upload detects character encoding preventing incorrect characters from being saved
ISO 19115 now include RORs for organizations
CKAN changes
Correct ISO 19115 XML format
Temporarilyremove spaces from EOV keywords
Upload job now saves ISO 19115 files instead of using the CKAN API
Replace use of datasetIdString with siteDeviceId in job
Use siteDeviceId instead of datasetIdString in im_iso19115datasetrecord
ERDDAP
Fix bug where ERDDAP titles had the incorrect format
OBIS - in progress
meta.xml now supported
EEW
Removed tau algorithm from magnitude calculation
Support sensor level data ratings (now the mmi sensor will show up as clean data with a sensor level data rating of 10 seconds)
[DMAS-69886] - Search Tree Maintenance Cast Reviewed check boxes should not be live
[DMAS-69899] - Master build is failing due to compile error in TaxonomyAttributeServiceTest.java
[DMAS-69905] - DMAS Search Failed Notification: 08-Jun-2022 15:10:06 UTC, Strait of Georgia East-Strait of Georgia East VENUS Instrument Platform, RDI ADCP Time Series, SearchHdrId 12082529, UserId 4377
[DMAS-69906] - Plotting Utility shows all plots as being downsampled for any time range
[DMAS-69937] - Please investigate "Data Error" in Sensor Additional Attributes tab after adding a new attribute to table in QAQC Auto Test Detail Page
[DMAS-69960] - Widget resize changes on Seatube Dashboard
[DMAS-69961] - Unable to vertically shrink video widget in videocentric layout
[DMAS-69963] - Camera not setting in ExpeditionManagment edit dive
[DMAS-69968] - CameraControl SeaScripts are failing
[DMAS-70000] - Plotting Utility Plots have minor display issues
[DMAS-70009] - Earthquake Catalog page associated two usgs events to the same earthquake
[DMAS-70015] - Creating Attribute Button without Attribute causes error
[DMAS-72903] - A few bugs related Taxon Button Set
[DMAS-72909] - Min/Max dates aren't set correctly in the Dive Form's date pickers when a date hasn't been selected yet
[DMAS-72919] - pgd and pd sensors always fail to derive in prod
[DMAS-72923] - Running datacentre in Dev fails with BundleException: missing package ca.uvic.dmas.core.infrastructure.dwc.text.domain.manager
[DMAS-72982] - Generating a Plotting Utility image with legend hidden causes overlap
[DMAS-72991] - Deleting an empty attribute deletes the button
[DMAS-73002] - Data Search Data Availability is Never Available in QA
[DMAS-73006] - Date Time Picker Field Validation broken in Form
[DMAS-73008] - Unable to save annotations in SeaTubeV3
[DMAS-73010] - DMAS Search Failed Notification: 29-Jun-2022 22:50:29 UTC, AML CTD Plus X 50328, Aligned and Depth-Binned Profile Data (On-Demand), SearchHdrId 12293071, UserId 112720
[DMAS-73015] - Unable to clear taxonomy field in seatube dashboard filter
[DMAS-73017] - Deleting a taxon button set button doesn't fully remove it from the UI
[DMAS-73028] - Dataset titles are showing up incorrectly on ERDDAP
[DMAS-73069] - Fix concatenatedTitle included in maintenance diff
[DMAS-73073] - CSV/json data products have bad date strings in QA
[DMAS-73083] - Exempt user list is still adding to the response queue
[DMAS-73091] - Plotting Utility download button does not work in QA
[DMAS-73100] - Cancelled tasks create messages in response queue which are never read
[DMAS-73108] - Taxon Button Set list is overlapping its search bar
[DMAS-73134] - Messages can be Improperly Sent to Response Queue on Startup
User Story
[DMAS-57948] - create back-end infrastructure to support ability to track changes in the ISO 19115 metadata record
[DMAS-65501] - As a sys operator, limit the number of searches running per user
[DMAS-66934] - Export dive and cruise information to event.csv
[DMAS-67070] - Add taxon information associated with dive WoRMS annotations to occurrence.csv
[DMAS-67181] - Only export annotations with enough up-votes to ensure only valid annotations are submitted to OBIS
[DMAS-67249] - As a data steward, I'd like to upload translation files without needing to check if the file is saved with UTF-8 or ISO 8859 or another encoding, so replacement characters (eg �) don't appear in translations
[DMAS-69034] - As a dev, I would like to remove deprecated columns in dmasuser and dmasuser_hist tables
[DMAS-69447] - As a Data Steward, I would like to see improvement on Network Console Performance - Adding/Editing Data Agreements takes years
[DMAS-69678] - As a data specialist, it would be great to have the UI display if a cast had its ref lat and long manually set
[DMAS-69835] - As a hydrophone data steward, I need improvements to the sensitivity vector device attributes upload feature
[DMAS-69864] - As a Plotting Utility user it would be a time-saver to have a button that saves/downloads all current plots into one file
[DMAS-73012] - DFO would like to change 'Dive Chief' to 'Dive Lead'
Task
[DMAS-53943] - Build tests for CF date.From and Date.To Task Job parameters
[DMAS-57371] - Add/update formsections to support Maris accelerometer / seismometers
[DMAS-57387] - Improve seismometer data products to support channels that are 1/2/3 instead of Z/E/N
[DMAS-69327] - Have all Annotation Entry options enabled by default (Quick Entry, Capture Time, & Attributes)
[DMAS-69747] - Implement units tag on AllowAny data product options
[DMAS-69776] - Implement using ckanext-spatial (cioos/siooc fork) and continuing to use API to upload CKAN datasets
[DMAS-69834] - Support Sensor level data ratings when getting scalar data
[DMAS-69840] - Dont use the Tau algorithm for determining earthquake magnitude
[DMAS-69860] - Restore Annotation V2 changes that were reverted
[DMAS-69870] - Delete orphaned classes DataSearchJSON and JSONHelper
[DMAS-69885] - Make parserdefinitionname column consistent
[DMAS-69893] - As a community fishers data user, I would prefer that downloaded COR/PNG files from geospatial map result in one folder for all files instead of individual folders for each file.
[DMAS-69950] - Fix formatting issues in ISO 19115 XML
[DMAS-69959] - Add skipSubfolders parameter to SearchResultsService in Geospatial Map laasso tool
[DMAS-69969] - Provide infrastructure to restrict allow any dataproduct options to integer values if desired
[DMAS-69992] - Do not render Geospatial Area Field in Form for casts in SiteDeviceSubset Maintenance
[DMAS-70014] - Drop support for deploying to /usr/local/gemini_driver
[DMAS-72897] - Don't save null attributes to taxon button set Attribute buttons
[DMAS-72905] - Add primary sensors for Cambridge bay community page
[DMAS-72910] - Dataset Manager and related code often uses ERDDAP dataset type, but should only be dependent on the site device id
[DMAS-72922] - As a Data Steward, I would like a deviceattribute_trx table to capture changes to device attributes
[DMAS-72929] - update acceleration download task to write files to sub directories for each seachtreenode code and date
[DMAS-72935] - Don't save an empty button into database right away after clicking "ADD Button"
[DMAS-72946] - Add MTC test site topology setup to qa post refresh script
[DMAS-72951] - Create scheduled job for continually downloading acceleration data from some EEW land sites
[DMAS-72954] - Minor improvement to handle bad hydrophone calibrations
[DMAS-72959] - As a Dev, I would like to add a warning to AdFile Service when a file does not exist
[DMAS-72994] - Set documentation link on seismometer channel options
[DMAS-73019] - Add confirmation dialogs for deleting buttons and attributes on Taxon Button Set Config page
Bug [DMAS-42666] - The "link" link on the start page of PU is broken [DMAS-52110] - Plotting Utility not displaying correct plot title [DMAS-64433] - Required permissions for adding and saving dives are different [DMAS-67410] - Call to ScalarDataAPIService linked to a production crash [DMAS-67738] - editing a batch task to remove the parameter value doesnt remove the key [DMAS-69125] - Oceans Next builds are timing out in multiple branches [DMAS-69174] - Editing an Attribute's Data Type Breaks Annotation List Edit Functionality and Crashes STv3 Page [DMAS-69495] - updating playlist description results in 'playlist already with name "<name>" error. [DMAS-69566] - Frontend changes for preventing attribute datatype changes if attribute is in use [DMAS-69616] - Banner message is not shown up on landingpage [DMAS-69641] - As a system operator, I want to prevent calls to DataSearchScalarDataService from crashing the web server [DMAS-69666] - Missing space in file archiver warning message [DMAS-69672] - Casts correctly being attributed to station in COR files, but the Sitedevicesubset name still has the generic "CAST" prefix [DMAS-69696] - Fix oceans next loading on search tree maintenance [DMAS-69709] - QAQC Create page has incorrect fields [DMAS-69713] - Seatube - taxonomy required to add attribute - behavior not consistent [DMAS-69732] - In dive detail widget the Active flag is not checked even it is in the edit mode [DMAS-69751] - Save and Cancel button showing in Taxon Button Set Config for RO users [DMAS-69768] - DMAS Search Failed Notification: 23-May-2022 20:44:23 UTC, Clayoquot Slope-ODP 1364A, Time Series Scalar Plot, SearchHdrId 12036575, UserId 19343 [DMAS-69771] - All CSV data products showing $DateUtil.dateRangeToScienceFormat($citation.startDate $citation.endDate) instead of a date in header in QA [DMAS-69772] - Earthquake Catalog page shows some detections as unconfirmed when they should be confirmed [DMAS-69773] - Fix bug in metadata information causing object loading to be difficult when there are null values [DMAS-69791] - I can't update some field in Dive Log using the popup in Expedition Management page [DMAS-69793] - DMAS Search Failed Notification: 26-May-2022 04:14:07 UTC, ROV Data-Odysseus, Time Series Scalar Data, SearchHdrId 12056870, UserId 48660 [DMAS-69813] - Device ID does not automatically fill in when editing on Driver Schedule Page [DMAS-69823] - scheduled job 378-DOI-Registration-job-deviceTypes has failed the last 3 days [DMAS-69829] - Fix issue where bndrun files were generated incorrectly [DMAS-69843] - Can't save multiple attribute in a button during one operation User Story [DMAS-54177] - As a passive acoustic scientist interested in low frequency signals, I would like downsampled 2 kHz versions of high sampling rate data to be available [DMAS-67032] - As a data specialist, I would like changes to the Sensor QAQC Attribute UI that allow us to modify historical values and datefroms [DMAS-67182] - As a biological expert reviewer I want to vote on biological annotations to give my opinion [DMAS-67999] - As a Didson SONAR user, I would like value-added data products to visualize and analyze the data [DMAS-69243] - As a Scientist, I'd like a pH property filter in DataSearch [DMAS-69367] - Modify the TelnetConsole page to not require device permissions [DMAS-69670] - As a Device Console user I would like to see a field that displays the latest driver start time-stamp [DMAS-69781] - Add searching ability for pages using Device ID links [DMAS-69784] - As a dev, I would like to see traceability for the configproperty table Task [DMAS-58198] - Document services used in existing Annotation Search [DMAS-58203] - Define high level component structure [DMAS-67641] - Update external documentation for Oceans 3 - data products [DMAS-68007] - Enable parsing of zoom values for Axis cameras, devicetype 158 [DMAS-69331] - It's impossible to clear the Taxonomy field in Annotation Entry without clearing all fields [DMAS-69485] - Don't embed unnecessary data in the DeviceListing velocity template [DMAS-69510] - Investigate 2022-04-23 earthquake [DMAS-69535] - Investigate new PDF append function for MATLAB data products [DMAS-69586] - Fix new checkstyle warnings from updating to 9.3.0 [DMAS-69594] - Update external documentation (DP and HydroCal) to fix links broken by the new ONC website [DMAS-69601] - As a scientist using SeaTubePro, I would like to see a closed "Cruise"-tree [DMAS-69615] - Remove the old TaxonButtonSetConfig servlet [DMAS-69642] - Support deploying storybook for branches with new dependencies [DMAS-69668] - Update data policy URLs [DMAS-69669] - Update configproperty data policy url reference [DMAS-69673] - Update siteDeviceSubset reference fields when re-post-processing a cast with scalar lat long [DMAS-69694] - Remove character limit from expedition table fields [DMAS-69728] - clarify 'dateTo' or empty sensorLoad black functionality in Watchman declarative parser to prevent data getting assigned to wrong devices parsing of [DMAS-69730] - Don't require unit test coverage on typescript files in the storybook [DMAS-69734] - Improve test coverage of TestHistoryTable.jsx [DMAS-69737] - Improve test coverage in TaxonButtonConfig [DMAS-69743] - Increase Test Coverage of AvailabilityChart.jsx [DMAS-69846] - Revert changes to AnnotationsV2 as not complete (was: Clicking "Query Annotations" in Device Details opens AnnotationsV2 in *two* new tabs) [DMAS-69862] - As a Data Steward, I would like a nameplate_trx table to capture changes to nameplate metadata Requirement [DMAS-69549] - Document requirements for the video preprocessor job Design [DMAS-69550] - Document video preprocessor job design Implementation [DMAS-67402] - Create Simulator for Valeport Current Meter [DMAS-69177] - Don't allow unit tests to make API calls [DMAS-69226] - Implement react code for Attribute table [DMAS-69245] - Add "(how to do this)" link to instructions on Plotting Utility splash screen [DMAS-69267] - Use a file preprocess job to set exact start/end timestamps in video files [DMAS-69403] - Create a matlab reader for the Didson sonar .ddf files [DMAS-69409] - Add jaxb generated EML classes [DMAS-69416] - Add raw icon for DIDSON DDF files [DMAS-69458] - Update AnnotationsManager to support V2 querying [DMAS-69476] - Implement Java service to update/delete attribute values [DMAS-69491] - Update PlottingUtilityPlot.js to support querying annotations [DMAS-69499] - Create packages to generate meta.xml file [DMAS-69508] - Switch out Default Lasso Tool Icon for a better looking one [DMAS-69525] - Implement new Sound Metrics (Didson) sonar data product formats [DMAS-69537] - Change modifyBy field to name instead of ID [DMAS-69584] - Update data search annotations link to use new V2 URL [DMAS-69587] - Fix new checkstyle warnings in dmas [DMAS-69607] - Revise instructions block on Plotting Utility opening page [DMAS-69656] - Allow any user with RO privleges on the telnet console to see device data streams [DMAS-69679] - Create new options set for audio data products [DMAS-69681] - Set up Renovate in Jenkins [DMAS-69689] - Implement audio file downsampling using ffmpeg [DMAS-69692] - Get Annotations Menu Items working [DMAS-69701] - Remove and confirm removal of Budget Item and related modules from DMAS code [DMAS-69703] - Remove and confirm removal of Budget Junction Box and related modules from DMAS code [DMAS-69708] - Modify ARIS postprocess job and add device attributes movement detection configuration [DMAS-69721] - Add a field that displays the latest driver start time [DMAS-69723] - Update Sound Metrics postprocessor email recipients [DMAS-69740] - Add tests for QaqcAttributesTable.jsx and QaqcAttributesConfig.jsx [DMAS-69741] - Modify payload when click save qaqc [DMAS-69742] - Modify code when update a qaqc [DMAS-69748] - Write script to give Taxonomy RW group write permission on Attribute Management page [DMAS-69801] - Create DB script (to be run after each DB refresh) to support test cases made in DMAS-69712 [DMAS-69804] - DMAS-69804 Create a transaction table for the configproperty table [DMAS-69805] - Add better logging to help diagnose annotation form errors [DMAS-69807] - Update DMAS code to trace configproperty table changes Test [DMAS-69406] - Add test cases for Didson value-added data products [DMAS-69691] - Create automated test cases for audio downsampling data product options Bug (ST) [DMAS-69636] - AIS Source - filter out command responses [DMAS-69752] - Driver Scheduler doesn't search on Device ID [DMAS-69783] - QaqcAutotestsFinder doesn't search on Description, Device Id, or Sensor Id [DMAS-69785] - Device Listing Details IP Tab doesn't search on IP [DMAS-69787] - Device Actions does not search on Device Action ID [DMAS-69788] - Device Listing Page contains multiple tabs with inability to search link text [DMAS-69821] - Search failure in QA for new downsample audio data option [DMAS-69822] - Search failure in QA automation: out of memory in new append_pdfs function [DMAS-69830] - Fix add attribute crashing QAQC attributes page [DMAS-69872] - Didson test case generating duplicate new video where there's pre-generated video Investigation [DMAS-67559] - Investigate differences and changes needed to support MATLAB R2021b (or the version we upgrade to) [DMAS-69179] - Profile some of the slowest unit tests Documentation [DMAS-67997] - External data product documentation for Sound Metrics [DMAS-69404] - Internal documentation and requirements for Didson value-added data products [DMAS-69690] - Document new audio downsampling options in the DP wiki
May 26, 2022
Minor release: driver/seascript update for Didson sonar (no web downtime)
Support associating taxons from Search Tree Maintenance (pending review)
Annotation UI Updates
In Progress - not in this release: open new Annotation V2 search and entry pages from Device/Sensor, etc links
Instruments
SeapHOx (PH sensor) was completed last month
Modifications were made to some of the acoustic drivers to allow configuration files to be uploaded/deployed
Improvements to make use of existing infrastructure to more easily take photos from devices
Current project for AIS device is finishing up
EEW
Improvements made to the earthquake catalog page
Old earthquakes from 2018 onwards have been reprocessed, the data is now available
Leaflet map on earthquake page has a new tool to give the lengths in kilometers for drawn lines
Support for EEW simulations
Data Products
Added a new option for deployments with mobile position sensors to default to only integrating depth, lat, lon. Leaving out heading, pitch, roll makes the searches much faster (users can still opt in)
Speed improvement for mobile position sensor integration for some searches
[DMAS-67664] - As a creator of published plots on plotting utility, I would like to see my published plots listed separately from everyone else's and see names of who published other plots
[DMAS-69143] - As a user, I would like the Sensor tab in Oceans 3.0 to be sortable
[DMAS-69339] - As a dev, I would like to remove "Status Console" from list of resource name on User Management page
[DMAS-69364] - As a community fisher user, I would like to be able to lasso unassigned casts on the geospatial map
[DMAS-69470] - As a data specialist, I would like the Cast Re-postprocess Management UI to have the same filter options as Cast Review UI
Task
[DMAS-53807] - when the timestamp is empty and a user clicks a button (without autosave) the timestamp should populate
[DMAS-55748] - Investigate and improve performance for mobile position sensor scalar data searches (MATLAB)
[DMAS-56572] - Export button should be disabled once it is clicked. Otherwise users will click twice.
[DMAS-66754] - As a data steward, I want to see the Dive ID in SeaTube V3
[DMAS-66907] - Investigate the 2021-10-12 missed earthquake detection
[DMAS-67706] - Investigate adding a unique constraint on the sitedevicesubsetId column in searchtreenodesitedevicesubset
[DMAS-67825] - Add the mail.from configuration for emails
[DMAS-69274] - Filename change in ADCP data products (non-averaged) that cross midnight
User Story
[DMAS-42635] - As a developer and data steward, I would like to remove archivefiletype database table and remove relations of other database table with it
[DMAS-57234] - As a Data Specialist, I would like to be able to search on the /QaqcAutotestsFinder page for tests by device
[DMAS-64295] - As a data specialist, I would like the ability to flag a binned sensor as '4' and for the data products to be populated with NaNs for that bin
[DMAS-65487] - As a hydrophone data product user, include post-calibration txt files with hydrophone data products
[DMAS-66619] - As a NOAA ST admin, I would like to ingest archived chat log messages into ST
[DMAS-67068] - Export WoRMS annotations to event.csv
[DMAS-67910] - As a data specialist, I would like the Cast Review UI to have the functionality to add multiple sitedevicesubsets to review through filtering options
[DMAS-67912] - As a data specialist, I would like a to see a cast review status checkmark for each SDS in searchtreemaintenance
[DMAS-67929] - As a data specialist, I would like data products that are pre generated for the geospatial map to stay up to date with edits to casts
[DMAS-67990] - As a community fishers end-user, I would like an option to download COR files for individual casts instead of one large file with many casts
[DMAS-68969] - Select STv3 dive loggers should be able to delete annotations
[DMAS-69136] - As a data steward/operator, I would like a device attribute entry when there is only a default value
Task
[DMAS-64706] - Data search does not populate if archive file datefrom precedes sitedevice datefrom
[DMAS-67028] - Please investigate the response of seatub/map service
[DMAS-67230] - driver change for RBR BPRZero to have valve commands as device readings
[DMAS-67809] - For non-averaged ADCP MAT/netCDF in searches of 24 hours or less, concatenate the files over midnight
Fixed issues with rounding in the sensitivity and frequency values, enables narrow sweeps
Improve the drive amplitude user interface, add quick scaling parameters, improve and clarify use of drive amplitude max and bias values
Improve handling of files and file-naming, remove the save button
Improve handling of time stamps - in the metadata UI, in file-naming and in keeping track of acquisition and post-process times
Workflow and data file I/O improvements, improve switching between files / switching configurations, loading (particularly for old files) and saving, fix bugs in this area
Output data increasing order for better integration with Oceans 3.0
Minor improvements such as spelling mistakes, user manual updates, plot labels, improved diagnostic logging, add more wait/busy indicators, bug fixes
Improvements for main user interface
March 16, 2022
Minor release: database optimizations, bug fixes for Data Search, Dataset Landing pages, SeaTubeV3 chat log, Didson driver update.
[DMAS-69021] - Check for and fix missing deployments in SOO/E searches
Implementation
[DMAS-67893] - Add front end support for deleting chat log operation
[DMAS-67973] - Update UI to match with the requested mockups
[DMAS-68014] - Don't extend data rating datefrom/to in MATLAB metadata struct
March 1, 2022
Major release
Maintenance
Data Search data availability plot zoom in/zoom out/interaction now populates the datefrom/dateto
delete published save plots
improve PU axis auto-scale configuration
GIFs controls and links in Data Preview easier to click on
Dataset landing page landing time fix (woohoo)
Device Synchronization UI improvements
Various bugs and small improvements
Instruments
Nortek profiler driver modified to improve timestamp jitter. Change was made for device 64167, may apply to other deployed Nortek profiler instruments (device type 59 and 130)
ProOceanus CO2 Pro driver – fixed bug related to using the device controlled zero interval. This does not impact the driver controlled zero interval feature
Axis Camera driver – setFocusWindow command added for seascript and front end
Earthquake Early Warning
Update the EEW Report Job to expand how it relates organization detections
Fixed caching issue specific to correlator shore station
Support for historical/reprocessing earthquakes on the earthquake detail page
Added EEW performance metrics through jmx beans
improve speed/ease of deploying code to remote stations (in-progress)
SeaTube
Add back end support for Chat Log deletion
Support ST admin / dive chiefs synchronization
ST layout support for displaying Chat Log
User controlled index number display option for Quick Button Set
Some bug fixes
Data Products
sensor level data rating integration
bug fixes
Added a ASL echoview csv file-mode modifier
device-level data product mappings for nodes and SIIMs
CIOOS
Improved EOV keyword allocation for ISO19115 metadata
Created a front end prototype for biological expert review
Add checks to EOV and Marine Gazetter vocabularies to ensure incorrect attributes can not be added
Community Fishers
New page to review multiple casts at once
Bug fixes
Search page for site device subsets
Selectable sensors plotted on cast scalar multi profile plot
Name of plots on plotting utility no longer named with element id
Changes to plotting utility to fit long sensor unit names
Handle errors during creation of site device subsets
Ability to tell manual GPS apart of tablet GSP in tablet log file
Added community fishers site device subset management pages to DMAS menu
Contact information and support form added to the geospatial map
[DMAS-67777] - Manually created Site device subset id 6359 is missing reference information and name
[DMAS-67804] - Many NullPointExceptions in production from api calls
[DMAS-67811] - SiteDeviceSubsetTable Sorting on ID is broken
[DMAS-67824] - EEW correlator machine (eewshore1.dc.onc) - Cache loading errors due to NullPointerExceptions
[DMAS-67827] - ProOceanus Driver not setting idle timeout correctly
[DMAS-67859] - Creating new child taxon inherits sibling as child in taxonomy management
[DMAS-67861] - CODARContiguousConfigJob fails if a deviceid has no matching archivefiles
[DMAS-67870] - Oceans Next builds fail in Jenkins with "TS2307: Cannot find module 'assets/images/API-Oceans3.jpg' or its corresponding type declarations"
[DMAS-67915] - Test sometimes fails on Oceans 3.0's master branch
[DMAS-67921] - Fix test failing in DOIDatasetMinterImplTest in master branch
[DMAS-67922] - Video player not loading in SeaTubeV3
[DMAS-67924] - Any predefined layout other than Public is being compressed on refresh
[DMAS-37860] - As a user, I would like to utilize sensor-level data ratings in data products generated by matlab code
[DMAS-56796] - As a community user of the Geospatial map and CF app, we would like the ability to provide user feedback from these interfaces
[DMAS-64667] - As a data specialist, I would like to be able to search for a siteDeviceSubset ID
[DMAS-64684] - As a system admin, I would like the ST logger admins can be synced.
[DMAS-64772] - As a Community Fishers Data specialist, I want to be able to adjust parameters plotted in on-the-fly plots i.e. turbidity, CDOM etc. or even expand to 4+ plots, so that I can easily customize graphs (ONC data products) to include in reports.
[DMAS-67195] - As a data specialist, I would like a web page for displaying earthquake information
[DMAS-67577] - As a dev, I would like to see small UI improvement for Device Synchronization page
Task
[DMAS-10675] - Usability report suggests allowing click-drag and single click on a shaded area in Data Availability to populate the Date From and Date To
[DMAS-44797] - please add unique constraint for devicecode field in database table device
[DMAS-54120] - Please disable the delete button icon on User Management page for RO user
[DMAS-19662] - As an operator, I would like to be able to stop and start drivers on a schedule
[DMAS-25072] - As a creator of published plots on plotting utility, I would like the ability to be able to delete the published plots if I created them
[DMAS-50989] - As a Community Fishers data user, I would like configurable cast data products from Data Search
[DMAS-56294] - As a data steward, I would like a better date/time select tool (especially time)
[DMAS-56514] - As an Iqaluit Community Fishers Data Partner, I want to have the crude oil and refined fuels sensor added into suite (CTD, Oceans 2.0 data base and Pipeline data processing jobs)
[DMAS-57132] - As a user from NOAA, I would like to display chat logs into Ocean's 2
[DMAS-57835] - As a data specialist, I want cast data products to include annotations that they have been verified after being reviewed
[DMAS-64661] - As a data stewardship member, I would like to automatically update CODAR config file datetos when new files are archived
[DMAS-66713] - As a software developer, I want to extract session-handling boilerplate from DAOs to reduce the surface for bugs and reduce the number of tests needed
[DMAS-66862] - Import CSV with vocabularies and vocabulary terms to Taxonomies
[DMAS-66947] - As a Data Specialist, I want to re-post-process multiple Site Device Subset Ids at the same time.
Task
[DMAS-54910] - please remove two unused sequences (tm_taskgroup_seq and tm_taskdetail_seq)
[DMAS-56792] - jobs without names should not be created through the TaskManagement page
[DMAS-57135] - fix dmasuserid for quarantine operations
[DMAS-57851] - Add Monitoring to send a Notification/e-mail if a CF Pipeline job fails with an error
[DMAS-57933] - Add unassigned casts layer to Geospatial map
[DMAS-64392] - In PU only show CF nodes that have data
[DMAS-65593] - Can we include dmasuserid in dmas.log to tell who start or stop machine in reprocess?
[DMAS-66468] - Remove extra UUID column from the database
[DMAS-66701] - Write a script to update some incorrect tasktypeid in prod database
[DMAS-66730] - As a Data specialist, I would like extend the search field option to include deviceCode
[DMAS-66848] - Check on magnetic declination calculation
[DMAS-66995] - the state of devicetype DOI minting job 378 indicates completed when it is actually still running, causing problems when running job again before it is actually finished
[DMAS-67006] - document job 370 for DOI Tombstone Job
[DMAS-67177] - please create hist or trx table for groupprivilege database table
[DMAS-67225] - Allow Resource type to be changed when opening Annotation Entry from Device and Sensor
[DMAS-67264] - Add dateto to hydrophone sensitivity device attributes when adding new attributes through the device attribute upload
[DMAS-67282] - Return unassigned casts from GeospatialAreaService
[DMAS-67317] - as a Community Fishers data user, I would like Data Search to automatically apply the CF Sampling station filter when using a deep link to a CF Node
[DMAS-67326] - Update CF preview job to generate plots for unassigned casts
[DMAS-67386] - Investigate December 9th earthquake detection
[DMAS-67452] - Update all "Oceans 2.0" strings to "Oceans 3.0" in java code
[DMAS-67457] - Change links to DeviceListing in Oceans Next code to use OpenInNewLink
[DMAS-67472] - Update matlab code to use "Oceans 3.0" instead of "Oceans 2.0"
[DMAS-67493] - Apply changes to community fishers cast detection to VPS and BPS as well
[DMAS-67496] - EEW Event expression - move to JSON format only
[DMAS-67497] - Add job.failed.email.recipients job parameter for CF pipeline jobs
[DMAS-67498] - Add device-level MATLAB data products for JBs
[DMAS-67499] - move "migrated" earthquake files to Bitbucket
[DMAS-67532] - Investigate why we have different codecs for one ROV camera
[DMAS-67537] - new JEP formula for Rinko ARO-FT Oxygen Concentration Corrected
[DMAS-67547] - Do not send corrected cast emails from Cast Data (3) Correction Matlab Postprocess Job if no casts processed
[DMAS-67597] - New driver attributes for P/Swave_forgetting_factor and amplitude_step_size device attributes for EEW Accelerometer (350), Nanometrics Titan EA (156) and RBR Tilt Meter (430)
[DMAS-67601] - Replace references to qapgsql2 with qapgsql1 in Oceans2 code
[DMAS-67603] - Replace references to qapgsql2 with qapgsql1 in MATLAB code
[DMAS-67607] - Migrate Drupal Earthquake Dashboard code for new IG website