This page contains the year's release notes and new features, primarily for Oceans 2.0 / DMAS, as well as any applications created/maintained by the ONC software engineering group, including mobile Apps and desktop/PC applications. In progress project status may be presented as well. The information below is written in plain language as much as possible. Detailed release notes are available on internal confluence pages (requires login).
December 20, 2021
Minor release: Bug fix for SeaTube V3 annotation export
Bug fixes, in these areas: Data Preview, Plotting Utility, Device Console, Reprocess Console
Support for sensorattributes and deployment of a new LDAP server
Update map in SeaTube Pro for compatibility
Bug and testing fixes for DriverSchedule UI page
Annotation UI
Support table updates after annotation created or updated
Support deep-link to annotations which includes the filter and selected annotation
Add accordion filter behaviour to minimise vertical space used
Add annotation export
Update Annotation Entry behaviour so only uses resource and resource type are defaulted
Other bug fixes
CIOOS: Marine Gazetteer Support
Added fields to Taxonomies and Taxons to support vocabulary fields
Added support for upload of Taxons to a taxonomy
Added support for mapping Taxons to Resources to allow Marine Gazetteer terms to be associated with search tree nodes
CIOOS: ERDDAP
Ensure datasets.xml is compatible with ERDDAP 2.14
SeaTube
Chat log story book in Demo server
Time code development
Many automated UI tests added
Bug fixes
Community Fishers
Refactored out cast definition out of the comment field into its own
Added email notification if a cast ingestion failed
Added millisecond precision when gathering parameters for cast detection
Fixed missing deployments out of .COR files
Acoustic Instruments Data Viewer
Added time range selection for daily and weekly
Created some MBeans for Matlab as as service monitoring
Data Products
Bug fixes: IOS hydrophone post-process, hydrophone calibration order, profile product and track plot search failures, Nortek daily time range and a few more
CODAR typos and data rescue
Several in-progress features and fixes/updates (sensor-level data ratings, BioSonics plots, HydroCal updates)
Instruments
EEW: two new jobs to automate and replace manual earthquake quality checks. First job compares earthquake detections to NRCan and USGS detections, second job that keeps the USGS/NRcan detections up to date. Working on a webpage to display the results.
Multi-queue: final testing with live data into QA, bug to fix before going live
[DMAS-66698] - Caching for the acoustic dataviewer does not return results if previous request was made by another user
[DMAS-67079] - Unreadable file errors on CODAR FTP job after latest preview feature added
[DMAS-67080] - DMAS Search Failed Notification: 04-Nov-2021 02:38:50 UTC, Insite Pacific Zeus Plus HD Camera on Hercules ROV, MP4 Video, SearchHdrId 10755970, UserId 2603
[DMAS-67087] - CODARDownloadTask is creating searches for wrong dataproductformat
[DMAS-67088] - The incorrect url for ERDDAP datasets is showing up on ckan
[DMAS-67093] - Fix compile error in DateFilter.spec.tsx
[DMAS-67098] - Replace occurances of "datasetsLandgingPage" with proper spelling
[DMAS-67143] - scalarDataManager.countScalarData() does not return accurate count in QA
[DMAS-67147] - DataProductDeliveryService returning errorCode 71 incorrectly in QA
[DMAS-67150] - dataProductDelivery download method throws null pointer in QA
Task
[DMAS-64431] - Re-running searches disables the search complete notification emails
[DMAS-67029] - Change Acoustic Date Viewer Front End to only Cache DataProductDelivery Request method
[DMAS-67059] - Write script to delete all existing im_iso19115datasetrecord rows
November 3, 2021
Major release
Maintenance
Update Community pages on ONC website (quick plot and State of Ocean/Environment plots)
Bug fixes (e.g., qaqc detail page, SMB postprocessor, data search email, DOI, device action, qaqcreprocess, Latest Data Preview)
Small features (e.g., API related, header of CSV data search, items on Oceans 2.0 menu)
Sea Tube
Performance improvements for Oceans 2 backend
Bug fixes for SeaTube
Instruments
Extensive update to the Camera pages and controls, including improved integration with SeaScript and drivers, better light and strobe controls, new custom function button
Support for video recording to internal memory, particularly for Rayfin cameras
[DMAS-66971] - Skip time period info is missing in the zoom bar from ST V3 video player
[DMAS-66983] - Cannot get Hydrophone Data In Acoustic DataViewer
[DMAS-67020] - Tons of INFO messages for Titan devices in dmas.log of mtcshore1.dc.onc
[DMAS-67025] - SeaTube - switching to Historical mode results in blank page
[DMAS-67050] - Correct null pointer in CODARDownloadTask
[DMAS-67051] - After zooming on datasearch map the Drifters search tree node does not display after clicking Display Drifters until map is zoomed again
[DMAS-67058] - Sea Tube filtering is not working for date range
User Story
[DMAS-22725] - As a data specialist, I would like to be notified when a cast is uploaded via the PSF app so that I can promptly assess and process the data
[DMAS-33861] - As a software engineer, I would like to reduce the deployment time
[DMAS-64698] - As a user, I would like to see a menu link from Oceans 2.0 to the API documentation
[DMAS-66506] - As a software developer, I want to resolve the Hibernate n + 1 problem throughout Oceans 2 so that system performance isn't catastrophically affected by load
[DMAS-66747] - Update Axis Camera Driver to support optical zoom
[DMAS-67011] - As a user, I would like to replace projects with Science Hub from our dmas menu
Task
[DMAS-53692] - rename externally-derived in the SensorListing UI to be more semantically correct
[DMAS-56819] - As a Data Specialist I would like the ability to modify the datefrom of a QAQC test attribute through the UI
[DMAS-57175] - Update CSV/JSON product to ignore the specialcitation_deprecated text for the attribute/acknowledgement
[DMAS-57937] - Integrate complex dataviewer spectrogram dataproduct with normal spectrogram generation.
[DMAS-58178] - Add colour map options for ASL echosounder plots
[DMAS-64402] - Scalardata service method getByDevice has no sensorCategoryCodes parameter
[DMAS-64474] - As a data steward, I would like to extend the data acquisition system for AML Metrecs to use the newer xml format
[DMAS-64518] - Design UI options for adding vocab terms or just creating a relation between a vocab term and STNs
[DMAS-65496] - Improve performance of CODAR QC products
[DMAS-66395] - Refactor "Trigger Re-Post-Processing" code to remove side effects
[DMAS-66502] - As a DataSearch user, I would like improved emails for searches with no data found
[DMAS-66651] - Apply checkstyle rules from Google Checks
[DMAS-66724] - Support a non-default set of resource types in resource selector
[DMAS-66733] - As an API user, I'd like a button to copy my token to my clipboard
[DMAS-66768] - Update dataviewer to set Spectrogram Concatenation from option selected on configure screen
[DMAS-66771] - CI builds that require the controller node ("master") should run on an "integration" node instead
[DMAS-66845] - Improve modified HFRProgs loadRDLFile.m function so the ruv header is parsed in our MATLAB data product
[DMAS-66847] - Include samplePeriod info in Nortek MAT file meta structure
[DMAS-53639] - Time Axis Issues in Nortek Signature 55 ADCP Data Products
[DMAS-57390] - When in fullscreen, widgets' embed button does not produce an embed window
[DMAS-57627] - Data Availability Chart does not show availability for a piggy-back
[DMAS-64778] - Many simultaneous calls to DataPreviewService from GeospatialMap
[DMAS-66234] - DMAS Search Failed Notification: 19-Jul-2021 07:48:25 UTC, Barkley Canyon Upper Slope-Upper Slope Vertical Profiler, Time Series Scalar Profile Plot and Gridded Data, SearchHdrId 8883869, UserId 106680
[DMAS-66488] - what is the difference between SearchTreeNodeUI and SearchTreeNodeUIs caches?
[DMAS-66521] - Error occurs trying to save layout with RO access
[DMAS-66528] - qaqcreprocess failed for DI 24119 on date 20200401
[DMAS-66529] - deviceId and deviceName are not shown in email for qaqcreprocess
[DMAS-66530] - status of qaqcreprocess task should be completed_with_error if there is error message during process
[DMAS-66582] - Viewing a larger image from the Image Widget doesn't work when dashboard is full screen
[DMAS-66606] - Unauthorized users can view the Create Dive form, but can't save it
[DMAS-66607] - DMAS Search Failed Notification: 05-Sep-2021 02:05:29 UTC, Vancouver Island-Strathcona Park, Time Series Scalar Plot, SearchHdrId 9669183, UserId 15356
[DMAS-66638] - Data Player is not functioning for RDI ADCPs
[DMAS-66853] - CODAR netCDF pulls lon data for geospatial_lat_min field
User Story
[DMAS-54234] - As a DataViewer user I want matlab to be provided as a service.
[DMAS-55065] - As a software developer, I would like to automatically deploy a given feature with backend, frontend, and UI test code to the QA environment so UI tests can be run on it before merge
[DMAS-64627] - QAQC result should merge with existing result when CF late data merge
[DMAS-65449] - As a Data Specialist, I would like the ability to delete a sitedevicesubset from the UI
[DMAS-66244] - As a Data Specialist, I would like an option in SiteDeviceSubset Maintenance to use reference lat/long when repostprocessing without edits, and repostprocessing should be forced on reference lat long or start end date change
Task
[DMAS-58059] - As a Data Steward, I would like the columns in Task Monitor to be sortable
[DMAS-66288] - Leverage the option to set instrument clock in driver initialization for Pro-Oceanus CO2-Pro CV devices
[DMAS-66338] - When editing sitedevicesubsets (for CF), the UI should have the option to input milliseconds for start/end time
[DMAS-66557] - As a data steward, I would like the dive chief drop-down in the dive metadata UI to be alphabetically by last name
[DMAS-66615] - Hide initial data available loading image
[DMAS-66617] - Display the circular progress image while waiting for the cast image to be posted
[DMAS-66673] - Disable CORs in cypress browser, update cypress commands
[DMAS-66682] - Support material-ui 4.12 and higher in oceans next
[DMAS-66710] - Improve Kongsberg rotary plotting so that it doesn't plot a single ping in a new sweep
[DMAS-66732] - Allow lists of search tree node codes in the event definition json
[DMAS-66738] - Improve production build time for Oceans Next
[DMAS-66762] - Move Annotation Table pagination bar to top
Design
[DMAS-66431] - Create UI design for using Ref Lat Long when reprocessing without changes on SiteDeviceSubsetMaintenance UI
[DMAS-66454] - Design UI to require repostprocessing of cast if altered in SiteDeviceSubestMaintenance
Implementation
[DMAS-54233] - Implement Matlab always running (Matlab as a service)
[DMAS-58181] - Add ancillary data to Nortek netCDF data product- correlation, error, percent good, etc and boost compression and format for RDI/Nortek netCDF
[DMAS-64208] - Create Owner Filter for Annotation Search
[DMAS-66430] - Create new option on SiteDeviceSubsetMaintenance UI to use Ref Lat Lon when Reprocessing without changes, also force repostprocessing if Ref Lat Lon, or Start End date is altered
[DMAS-66432] - Update Matlab castdatacorrection code to use ref lat long when reprocessing a cast without changes when flag is encountered
[DMAS-66433] - Modify SiteDeviceSubset service to allow for repostprocessing a cast without changes but use ref lat lon
[DMAS-66472] - Add new operation to SiteDeviceSubsetService to get SiteDeviceSubsetTypes that are casts
[DMAS-66485] - Add faster service call to get annotationId and timestamp for a dive
[DMAS-66587] - CKAN API call is generating random UUIDs for Citation Identifiers (AKA DOIs) for CKAN records - even when there are DOIs associated
Bug
[DMAS-64777] - Data Preview searches are no longer tasktypeid 16 "Data Product Preview Search"
[DMAS-66640] - AnnotationList doesn't scroll to the correct annotation when parameters are provided in the SeaTube url
[DMAS-66659] - Site Device Subset edit bug for CF126-4082
[DMAS-66672] - Check to see if device is Community fishers device does not function if device was decommissioned and search range ends before decommission date or location based search
[DMAS-65620] - Fix SubCImagingCam driver to send correct commands from flashTime and flashDelay functions
User Story
[DMAS-58369] - As a data specialist, I would like modify a SiteDeviceSubsets's referenceLat and/or referenceLon and re-postprocess the data
[DMAS-58372] - As a data specialist, I would like modify a SiteDeviceSubsets's startDate and/or endDate and re-postprocess the data
[DMAS-64851] - As a Data Specialist, I would like an automated geofencing QAQC test
[DMAS-65458] - As a data specialist, I would like to be able to post-process a cast from the sitedevicesubset maintenance page WITHOUT making any changes to reference data
Task
[DMAS-64696] - Update McLane Phytoplankton Sampler driver to accommodate changes in adaptive sampling commands
[DMAS-65493] - After editing site device subsets, the reference depth should display the End depth, not the Start depth
[DMAS-66228] - sql to insert a sensorAttribute for QAQC attribute for PVCS test
Implementation
[DMAS-64828] - Remove autosave from Button Set Config
[DMAS-64859] - dctsk04 - DMAS' log - Failed to send DataSyncMessage to MQ. Shutting down and writing to file instead
Task
[DMAS-64658] - As a dive logger, I would like ST validates the annotation date within the dive range
[DMAS-65382] - Remove unnecessary error snackbars, or update their messages to be more informative
[DMAS-65385] - As a NOAA ST user, I would like to keep my last selected taxonomy after an annotation is saved
[DMAS-65400] - Extend manual trigger of samples for McLane Phytoplankton Sampler (devicetype: 341)
[DMAS-65401] - TaskManagement page task table queries too many tasks
Implementation
[DMAS-64779] - Cast Data Ingestion should ingest empty versions of EDS files before versions with data and check for aligned and depth binned data between castdata files before merging them
[DMAS-64800] - Track the number of cached objects over time
[DMAS-64836] - Update service call and check service to prevent nulls
[DMAS-65097] - Modify Nortek Signature 55 driver to recover when device stops
[DMAS-65333] - Cabled Fetch driver - modifications for AZA
[DMAS-65355] - Re-post-process casts when their reference latitudes and/or longitudes are modified
[DMAS-65374] - Interpolate navigation data from cast file when reprocessing flag encounted
[DMAS-65391] - Handle varying number of values in sitedevicesubset comment
Bug (ST)
[DMAS-64588] - Pamguard PGDF files are not going into FTP on PROD
[DMAS-65375] - Only the old CTD scalardata is being deleted, not the piggybacks and tablet scalardata
[DMAS-65388] - Ingesting new casts creates duplicate sitedevicesubsets
[DMAS-65390] - Error injecting constructor, java.lang.NullPointerException when start driver of BPRZero in QA
Multiple queues project (almost ready, not deployed)
Driver / parser bug fixes and support
New BPR driver
Nortek Signature 55 driver bug (in-progress)
PAMguard bug fixes
Earthquake Early Warning
OSGi modularization: eventdetection in separate module, shorestation metrics (in-progress)
Data Products
CODAR documentation and minor updates
Search automation improvements: test coverage at 100% with new metrics that include sensor level mappings, fix for reprocessing automation
Commission the first of a new generation of task machine - now online!
Data Player
Caching and performance (in-progress)
Community Fishers Phase 2
Facility to manually edit or add cast detections, including lat/lon (sitedevicesubsets), which then triggers re-post-processing of aligned/binned data (in-progress)
SeaTubeV3
Dive logger can play historical videos
Customization support - store layout in cookies
changes to the live video page
investigating memory issues (root cause found - fix in next minor release)
Bug fixes
Annotations & Dashboards
Initial new annotation search in limited release
Transition to public services instead of internal services: now using archivefiles service instead of internal service in audio widget
[DMAS-64657] - All QA hosts logged errors while loading the DafTest cache after deployment with release/21.05.000
[DMAS-64670] - DmasGroupManager calls ServiceUtil too often
[DMAS-64689] - Task machines running 21.05.000 can't deserialize EventDetectedContainer, sent in messages on the archiver topic from an archiver running on master
[DMAS-64710] - Failed to generate a DeviceService response: java.lang.NullPointerException
[DMAS-64714] - PA Shorestation logging ERRORs that it can't handle an ActiveMQ object - java.lang.ClassNotFoundException
[DMAS-64740] - Can't see Batch tasks in TaskManagement in Prod
[DMAS-64750] - NOAA EX2103 expedition and dives are using ONC logo.
[DMAS-64775] - "Error: Invalid Parameter: liveStream1DeviceId. Invalid Integer format" on saving a cruise in QA
[DMAS-64782] - update coverage to exclude untestable functional component packages
[DMAS-64825] - Cannot update any cruise on Expedition Maintenance page
[DMAS-64302] - "Add User defined Taxonomy" is visible to anonymous users
[DMAS-64335] - When a UI test suite finishes, an empty Chrome window with the URL "data;," is left open
[DMAS-64346] - Inactive ERDDAP dataset metadata should not be ingested to CKAN and CIOOS portals
[DMAS-64429] - PSF data (IS-stations) not plotting in Geospatial Map (May 11th)
[DMAS-64475] - fix failing tests in TaskDefinition
[DMAS-64486] - Data Preview Links covering State of Ocean plots
[DMAS-64539] - DMAS Search Failed Notification: 26-May-2021 19:25:44 UTC, AML CTD Plus X 50334, Aligned and Depth-Binned Profile Data (On-Demand), SearchHdrId 8677842, UserId 48660
[DMAS-64543] - NOAA Logger - Admin (df21) does not have write permission to Attribute Management Page
[DMAS-64547] - Add the DFO station and JVM to the database
[DMAS-64576] - Acoustic Data Viewer ADCP Toolbox Options onSubmit not working
[DMAS-64578] - qaeewshore1 fails to start on release branch
Citizen Scientist Android version CF.3.2.5 for Community Fishers Project
Improved backup to Google Drive, support for Android 10 and new tablets, improved date and GPS handling, release note viewing and update notifications, bug fixes
Tickets Released: CF.3.2.5 (accessible to internal users only). Total: 17.
Bug
[DMAS-57748] - Community Fisher Tablet outputs future date in commandData
[DMAS-57840] - AML-6 Data not downloading - :error data not found"
[DMAS-64457] - GPS Date Comparison Dialog box inconsistent bolding of date in message
[DMAS-64460] - CF App Google Account Sign is not working
User Story
[DMAS-23434] - As a Pacific Salmon foundation person, I would like to have enhanced data security
[DMAS-57262] - As a Citizen Scientist, I would like to have a notification when the app is ready to be updated, and be able to view release notes
[DMAS-64427] - Start Date displayed on Qaqc Auto Test Details page shows current date, not start date of test
[DMAS-64446] - Request Support link throws an error
Task
[DMAS-64298] - Please investigate why the scheduled job 340 (Cast Ingestion) is throwing up an error once it was enabled to run on schedule yesterday
[DMAS-64336] - Return searchresultstatusid 4 when providing matlab-generated products in the case of no data found
Requirement
[DMAS-64089] - Create requirements for no-video annotation user choices
Design
[DMAS-64090] - Design dialog for no-video annotation user options
Implementation
[DMAS-64389] - Reduce the data overhead in the searchtree by excluding non-searchable nodes/data
[DMAS-64394] - Reduce the data overhead in the searchtree by excluding unused metadata
[DMAS-64425] - Improve DeviceService Instruments by Category loading time by eliminating excess cache iteration
[DMAS-64440] - Improve ONCKmlService search tree loading time by eliminating excess cache iteration
May 11, 2021
Minor release: performance improvements for search trees (Data Search and Plotting Utility), instrument search failures, permissions fix for geospatial map and more
[DMAS-64283] - Add permissions check for cast operation in DataPreviewService
[DMAS-64323] - Do Not Display Error snackbar if DataPreviewService returns an error
[DMAS-64390] - Improve searchtree loading time by eliminating excess manager startup
[DMAS-64393] - Improve search tree filter loading time by eliminating excessive cache iteration
Bug (ST)
[DMAS-64269] - Ingester does not set creation and modification information with the identifier's
[DMAS-64340] - Ingester test sometimes fails on Jenkins
May 5, 2021
Major release
Maintenance and Production Developer
Email notification for Data Search
QAQC related tickets
Bug / Production Developer tickets
Migration of several pages/UI to modern REACT framework
Instruments
Multiple queues project (significant progress, not deployed)
PAMguard features: batching, external user access, improvement performance
Earthquake Early Warning
Added a second protocol for EEW client and other improvements for modularization
Data Products
Nortek current plot ensemble averaging - fixes a number of recently detected issues with the plots
FLAC support in PAMguard pre-processor
CODAR improvements: mercator projection, plot limits and titles
CF support: multiple deployment bug, descriptive attributes, crossing midnight, re-arrange of cor file columns and labels
Search automation related improvements: fix out of memory errors for disjoint scalar data, very slow plotting for min/max plots, support for 2nd QA, exclusion list for devicecategories
Data Player
Changes to persist Toolbox Options between mods
Changes to update Toolbox Options after device change
Caching (in-progress)
Testing and operationalizing Matlab as a service (in-progress)
Community Fishers (Now in Phase 2)
Clock issues in Android tablets
Remove auto-generation of tickets for bad or unassigned casts
Allow map polygons be id-able for testing
Tweek cast endpoints in cast detection MATLAB code
Data restrictions (in-progress)
Bug fixes + support
SeaTubeV3
Oceans 2.0 / SeaTube Help Desk improvements
Capstone annotation ingestion
Video play choice dialog for missing video
Bug fixes
API & Annotations & Dashboards
External API improvements
TypeScript pilot project in REACT framework
Migrate and build a new & improved annotations search page in REACT (in-progress)
Automated UI test improvements/experiments with Cypress
Dashboards bug fixes
CIOOS
Greatly improve performance of ERDDAP Management
Synchronous service for Data Products (a one-call alternative to dataproductdelivery) (in-progress)
[DMAS-58131] - As an engineer, I am struggling to use the new Device Search interface
[DMAS-58142] - Buggy save behaviour in Demo sometimes causing page to hang
[DMAS-58204] - In SeaTubeV3, annotations that aren't within a video file can be clicked, but clicking doesn't do anything except highlight the annotation
[DMAS-64187] - Investigate sporadic search failures in QA search automation with high volume min/max data
[DMAS-64210] - Fix failing ui tests on sensor maintenance
[DMAS-64217] - Fix failing ui tests in SiteDeviceMaintenanceTest
[DMAS-64224] - Data Search for COR files By Location not working, even though these data are populating geospatial map
[DMAS-64241] - Fix failing UI tests in Plotting Utility
[DMAS-64243] - Error when generating multiple data products
[DMAS-64249] - Internal Users in QA are not able to create annotations
[DMAS-64253] - Bug in videoFileQAQC job caused by new code for 2nd QA
[DMAS-64255] - Test failing in SearchTreeMaintenance
[DMAS-64260] - Fix failing tests in generationAndDisplay
[DMAS-64288] - Can't add derived sensor definition if none exist
User Story
[DMAS-25060] - As a Nortek daily current plot user, I would like the plots to be resampled properly to prevent errors in data gaps, backscatter averaging and anti-aliasing
[DMAS-56819] - As a Data Specialist I would like the ability to modify the datefrom of a QAQC test attribute through the UI
[DMAS-57154] - As a Data Specialist, I would like an automated PVCS QAQC test
[DMAS-57343] - As a user from NOAA, I would like to easily report and keep track issues found in Ocean's 2.0
[DMAS-57761] - As a data specialist, I would like to Improve the maintainability of the correlator event expression
[DMAS-57819] - As a Data Search user, I would like an email notifying me when my searches are complete
[DMAS-57822] - Keep files in /ftp/pub folder 21 days, instead of 14 days
[DMAS-57970] - As a software developer i'd like the tabs on Sensor Maintenance to be updated to React
[DMAS-58263] - Fail to add DateTo information for existing IP
[DMAS-58273] - DMAS Search Failed Notification: 09-Apr-2021 16:46:17 UTC, AML CTD Plus X 50241, Aligned and Depth-Binned Profile Data (On-Demand), SearchHdrId 8536670, UserId 94330
[DMAS-58276] - Video files at the start of dives are not played in SeaTubeV3
User Story
[DMAS-58233] - As a DFO CODAR Data Preview user, I would like to load lower resolution gifs for faster load times
[DMAS-58245] - As a dive logger, I would like a ST layout to display larger video.
Task
[DMAS-58196] - Create locations for DMAS + ActiveMQ for new ferries
[DMAS-58242] - Add small GIFs for Data Preview to work with the new GIF player
[DMAS-58393] - As a Data Steward, I would like to stop the automatic generation of JIRA tickets for Unassigned and Bad casts
Design
[DMAS-58264] - Create design for a video-focused predefined layout
Implementation
[DMAS-58248] - Add new video-focused predefined layout into STV3
[DMAS-58252] - Front-end code change for gif player to use smaller sized gifs
Test
[DMAS-57976] - Add web service tests to cover resampling changes
[DMAS-58207] - Create regression test for Gif Player
Bug (ST)
[DMAS-58221] - Update styling and functionality following testing
April 7, 2021
Major release
Maintenance and Production Developer
Bug fixes including QAQC inheritance
ONC website support, including Wiring The Abyss updates for the maintenance expedition
Migration of several pages/UI to modern REACT framework: extension + item pages
State of Ocean/Environment/Climate plots now configurable
Animated GIF player and image tiling for Data Preview
Instruments
Minor modifications to straw drivers and parsers.
Fine tuning new compass feature on Kongsberg driver to work with data products.
Minor modification to ASL MUD driver, check and turn off verbose mode.
Cancel blocked processing on shore station: Systems can now cancel blocked processes on the shore station such as the parsing routine blocking. This applies to all processes after a message is dequeued and before it is queued to the archiver.
Minor improvement to logging at shore station.
New features for binary parsers. Used in ASL MUD. Binary parsers now work with variable length records and allow specifying sensor positions relative to the payload instead of the packet. Changes wrt to ASL Mud documented here Parsing Binary Data With Dynamic Packet Size, general documentation for declarative parsers updated: Declarative Data Parser Definition File Format (internal links)
Improved logging on shore station event detection routine.
Multiple queues project (in-progress)
Users can now create PAMguard batches through the task management screen, Batch section.
Earthquake Early Warning
Correlator event expression simplification. EEW event detection expression now uses a more generalized device selector expression.
Titan driver modified so build date can be parsed. This feature requires deployment to all titan locations, not sure of the priority for deployment.
Data Products
NEW CODAR netCDF products for Combiner + Radial stations
RDI fix for correlation screen option being less than onboard correlation screen
ASL mud improvements: .01a files to use DMAS time not device time, handle varied orientation, improve handling of non-standard data, improve logging
Search automated testing improvements (internal use only)
Bug fixes + support
Data Player
No longer uses archived hydrophone images on initial load (to avoid old incompatible images)
Toolbox now programmatically gets dataproduct options for a given device
Fixed bug: plot times on ADCP current plot do not align with timeline in dataviewer
Initial changes in place for image caching (will work with Systems to activate in April)
Testing and operationalizing Matlab as a service (in-progress)
Community Fishers
Pipeline now triggered to run cast detections by raw data ingestion (not activated)
Data products to use search tree node name for station name instead of search tree node code
Geospatial Map returns most recently completed cast preview data products, not the most recent
Only generates Cast Preview Plots if older than three days or new cast is at SearchTreeNode
Fixed bug: automated creation of JIRA tickets by job #341
Androidlog files are no longer being archived
Pipeline now sends e-mail when a bad file is encountered
SeaTubeV3
ONC SeaTube support
Oceans 2 Help Desk improvements
Sensor plot now uses the dashboard plotting widget with support for plotting multiple sensors at the same time
Bug fixes
Dashboards
Fixed bug where it wasn't possible to select a run folder in a User Defined Task Widget
Can share a Dashboard to a user with Edit permissions
Correct title in Latest Readings Widget
API & CIOOS
Fixed bug where an incorrect end date was put in a CKAN record
Return data set Id from the ErddapManagementService create operation
[DMAS-50249] - Search automation falsely reports failure for AquaTec CSV data product
[DMAS-54242] - EEW Test Notification service logging error
[DMAS-56636] - failed export json format with image in QA
[DMAS-56757] - Search automation report shows data product .json expected files in the result.json column
[DMAS-57319] - CKAN record Temporal Extent End dates are inaccurate
[DMAS-57337] - jobid link on CameraControl page should be linked to taskmanagement page, not schedulerconsole
[DMAS-57340] - In some cases, QAQC reprocessing fixes incorrect QAQC flags for all sensors but Density and Sound Speed
[DMAS-57534] - How do we delete qaqcResult dateTo is beginning of next date in reprocess or qaqcreprocess?
[DMAS-57546] - qaqcreprocess does not handle "duplicate" qaqcresults correctly
[DMAS-57555] - question about how qaqcreprocess handle overlap for station level qaqc
[DMAS-57638] - User defined task widget doesn't show list of runs
[DMAS-57724] - Test automation is logging lots of vague NullPointerExceptions in GetWebServiceMetricsMVCResourceCommand
[DMAS-57734] - When running automation tests, the tool prints a number that is implied to be the number of pending tasks, but is actually the total of historical + pending tasks
[DMAS-57790] - need support for millisecond while request the video stream
[DMAS-57795] - Cache load of event definitions throwing null pointer
[DMAS-57800] - The automation tool fails to deploy and logs a hard-to-understand exception if it can't connect to qapgsql
[DMAS-57802] - Fix logging of configuration change ASL data products
[DMAS-57830] - Need update dateto of current ip when add a new ip for the same device
[DMAS-57833] - should not allow one ip active for mutliple devices
[DMAS-57868] - Changes to a Taxon cause its children to vanish from the taxonomy tree until the page is refreshed
[DMAS-57940] - Back-end changes to support taxonomy delete functionality
[DMAS-57941] - Back-end changes to support taxon button set delete functionality
[DMAS-57978] - get SitePosition from database only when cacheloading
Bug (ST)
[DMAS-57634] - Editable inherited comment attribute always saves on focus loss
[DMAS-57947] - Logging in redirects to /domain/Apps/theme/colors.css instead of the home page
[DMAS-57949] - Can't create issues from the new issue collector when not logged in
[DMAS-57957] - Not getting good compass heading values in new SMB files
March 4, 2021
Major release
Maintenance and Production Developer
Bug fixes including QAQC reprocessing, QAQC inheritance
ONC website support
Migration of several pages/UI to modern REACT framework (DeviceAction, new page Cables) (on-going)
Several other small improvements/features (in-progress)
Instruments
Wally - driver modified to match devices telemetry change in motor logging
Rayfin - take picture function in seascript now available
Straw - new status commands added, refinements to parsing.
Pro Oceanus - Burst mode enhanced to be more reliable
Seabird CTD with O2 piggyback - Driver enhanced to handle piggyback problems better
RBR Concerto Tilt Meter - Additional start up setting
BPR - Driver now support binary mode
AIS Source bug resolved
New syntax in calibration formulas where sensors can be referenced by sensor code and device attributes can be referenced by attribute code, instead of by sensorid and attributeid
New screen for uploading PAMguard configuration files
PAMguard batches are automatically scheduled, system will limit the number of concurrent PAMguard tasks
Earthquake Early Warning
jep refactored as a separate bundle. This is an optimization for EEW deployments and moves us closer to exposing the jep code publicly
Shore station - device being processed can be viewed through jconsole. Allows for quicker troubleshooting when the shore station is blocked
Improvements to PGD algorithm
Shore station threading and cache load issues
Data Products
Search automation improvements (in-progress)
Added ancillary data to RDI netCDF plus improvements
Kongsberg compass support
Support for ASL mud variant + overall AZFP improvements
Data Player
Fixed misalignment between image and timeline
Add more instruments to selection dropdown
Testing Matlab as a service (in-progress)
Community Fishers
ALM-6 with AMLx formatting
Help button with wiki links in Geospatial Map
Now able to save data off tablet to Google drive
Geospatial Map now uses Open Street Map tiles as default
Updated data search to handle multiple casts for .COR file data product
Multiple minor changes to Android Tablet App
SeaTubeV3
support adding multi attributes in a single quick button, Taxonomy Attribute Management page.
NOAA help desk rewrites in React and adding auto fill for names/email and organization information
Bug fixes
Dashboards
permissions:
don't allow display of dashboard using a URL if don't have permission
only allow access to Dashboards if logged in
only allow access to embed if logged in
only auto-refresh if tab is active, only refresh every 30s
fix issue with playing video play lists
API & CIOOS
adcpdata service - add getByLocation
scalardata service - check for resample period
archivefiles service - support download of compressed
Bug [DMAS-32109] - Step 2 of data search providing incorrect link to IRIS. [DMAS-38396] - Send Help Request from search results shows "undefined [DMAS-46794] - DeviceActionForUI class has a misspelled variable (EnginneringListId) [DMAS-48286] - Can't create a Device Action for Devices that aren't on the Observatory [DMAS-56546] - Tests failing in UserReplication [DMAS-57350] - QA hosts with DATACENTRE role become unresponsive after launching too many threads [DMAS-57442] - sitedevices are not being saved [DMAS-57533] - qaqcreprocess does not handle qaqc inheritance correctly [DMAS-57610] - Fix failing UI tests in DataViewer [DMAS-57619] - AML6 Cor dataproduct sometimes has turbidity column for cast in cor dataproduct [DMAS-57631] - Prevent a dashboard from being viewed if the user has not been given permission [DMAS-57644] - Using latest in Audio widget doesn't get the most recent file [DMAS-57657] - CKAN data records link to DataSearch but the name suggests they link to ERDDAP [DMAS-57671] - Permissions for data specialists editing device attributes not working [DMAS-57672] - Error when starting a fitlet on the latest branch [DMAS-57676] - update PGD and PD algorithms to use most recent displacement values [DMAS-57697] - Some automatic cache reloads over JMS dont work [DMAS-57725] - Search automation detected a change in header metadata for CSV products on the VPS - related to siteposition? [DMAS-57727] - the device action table doesn't show all columns within the width of page if the comment is long and no scroll bar [DMAS-57737] - BundleUtil Cannot Find DataEnrichmentCacheModule [DMAS-57743] - AIS Source driver is not outputting anything [DMAS-57760] - qaweb is unable to find sos.properties [DMAS-57771] - Requests for dashboards 115 and 137 causing significant load on web server [DMAS-57778] - Could not get french translation for device category description. Will not generate French abstract [DMAS-57792] - Latest Readings Widget isn't using the correct sensor when using location source [DMAS-57798] - Cast Scalar Profile Plot is unable to find cast location for device level searches [DMAS-57811] - 404 error caused by references to qaqc in neptune_header_light.vm [DMAS-57812] - Playlist data source in Video Widget does not select a playlist [DMAS-57831] - Plot times on ADCP current plot do not align with timeline in dataviewer. [DMAS-57834] - Error Running tests in onc-web [DMAS-57836] - master branch of Oceans 2.0 build failed [DMAS-57877] - I can't rename a customized taxon name [DMAS-57880] - hydrophoneCalibration.txt file missing from FLAC search result [DMAS-57881] - SystemConsole in QA with release/21.02.000 is broken [DMAS-57885] - Unable to create new searches with DataSearch. [DMAS-57886] - Bug causing DiveServlet not to load properly [DMAS-57894] - Fix TypeError when adding a Properties by Location data product to cart [DMAS-57898] - error saving ServiceLogState on qa web [DMAS-57899] - clicking either of the root nodes on the tree in Taxonomy crashes page [DMAS-57901] - Fix Create.instrumentsByCategory failure [DMAS-57913] - Task reference name is unknown until finishing and cant cancel running task [DMAS-57921] - Map is not loading in QA data search. Also, card pop out when selecting a device is not occurring. [DMAS-57922] - Can't delete attribute and attribute group in Taxonomy Attribute management page User Story [DMAS-45728] - Passwords can be read using inspect element in Task Management [DMAS-56773] - As a dev, I would like to see qaqc auto tests finder page rewritten in react [DMAS-56775] - As a dev, I would like to see qaqc auto test details page rewritten in react [DMAS-56889] - As a Data Search user, I would like to be able to cancel any running of my running or pending searches [DMAS-57230] - As a software developer, I'd like the /DeviceStatus page updated to React [DMAS-57379] - Cloning a device also clones the serial number of the sensors in the additional attributes tab Task [DMAS-52454] - Search Automation failing comparison on new .cor files: add to text comparator and skip 'ProcessingDate' [DMAS-56007] - New driver attribute for Nanometrics titan (350) and tiltmeter (430) for controlling sample rate [DMAS-57116] - Add DOI Attributes to ERDDAP records through dataset.xml [DMAS-57400] - Update UI tests for /FileUpload [DMAS-57449] - Make "Data Availability" title look the same as the other config titles [DMAS-57456] - simplify PPP calibration formulas [DMAS-57459] - Add UI component tests for the DataAvailabilityChart [DMAS-57512] - Add ancillary data to RDI netCDF data product- correlation, error, percent good [DMAS-57543] - Change DatasetAttributeService from using hard coded versions of the organization statement abstracts and move them to GlobalProperties [DMAS-57630] - Only allow Embed Dashboard and Widget when user signed in [DMAS-57698] - As a Task Management user, I would like the windows for the job parameters to be expandable [DMAS-57738] - Update kalman filter to use the accelerometer's sample rate attribute [DMAS-57739] - Correct code for creating default values of "Expected search failures" field in search automation configuration [DMAS-57751] - change the default sorting of the Cruise Listing to be reverse datefrom chronologically [DMAS-57766] - As ONC who doesn't have anonymous API access I want to ensure only logged in users can access Dashboards management and dashboard pages [DMAS-57789] - Write FTP Job to Transfer androidlog files from FTP directory to ArchiverlowPriority [DMAS-57844] - new item "System Status" on dmas menu [DMAS-57849] - Change 'No Data' to 'No Records Found' for React Tables Requirement [DMAS-57223] - Gather requirements [DMAS-57327] - Document general requirements for Java 9+ support [DMAS-57402] - Document requirements Design [DMAS-57224] - Design the system for associating a set of bundles to a DMAS host [DMAS-57477] - Design the new support request UI [DMAS-57816] - Need a story book mock up Implementation [DMAS-28469] - Driver for ASL Acoustic Scintillation Device [DMAS-43988] - Move the web layer to a new bundle [DMAS-56280] - Add a new react page for user defined classifiers in PAMGuard [DMAS-56860] - Write QAQC Finder front end in React [DMAS-56914] - Update QAQC Finder back end for updating to React [DMAS-57015] - Write QAQC Details front end in React [DMAS-57016] - Update QAQC Details back end for updating to React [DMAS-57024] - Automatic scheduling of Pamguard batch jobs [DMAS-57100] - When a user cancels a search they should be linked to the data search help page [DMAS-57240] - Extract JepCache into its own OSGi bundle [DMAS-57258] - Update services for use with react page of DeviceListing [DMAS-57259] - Write new React page for /DeviceListing [DMAS-57310] - Create Help button in Geospatial Map [DMAS-57347] - Do compass check in kongsberg rotary SCAN mode [DMAS-57348] - Use the compass data in the SMB post-processor heading calculation [DMAS-57403] - Use an alternate issue collector in Dev/QA [DMAS-57436] - Update backend for /DeviceStatus [DMAS-57439] - Support both Java 8 and 11 in UI tests [DMAS-57447] - Add AvailabilityChart cypress test [DMAS-57470] - Update ui tests for /DeviceStatus [DMAS-57508] - Create an issue collector form [DMAS-57510] - Update backend for DeviceSearch [DMAS-57511] - Write new react page for DeviceSearch [DMAS-57528] - Update selenium scripts for QAQC Details change to React [DMAS-57535] - Add functionality to getByDevice [DMAS-57536] - Add functionality to getByLocation [DMAS-57544] - Creating database scripts [DMAS-57572] - Update UI tests for DeviceSearch [DMAS-57623] - Update TaxonButtonSetService to handle multiple attributes for a single button [DMAS-57637] - Modify archiveFile service to accept compression parameter for accessing compressed files [DMAS-57654] - Update new Taxon Button Set Config and Manual Entry pages to handle service call changes [DMAS-57682] - Create page in the front end for Taxonomy Attribute Management [DMAS-57688] - Add getByLocation method [DMAS-57696] - Change DeviceCloner code to not clone sensorattribute values [DMAS-57700] - Add unimplemented CRUD operations to TaxonomyAttributeService [DMAS-57701] - Add method to get list of devices at a location with date ranges [DMAS-57702] - Create a taxonomy attribute group service [DMAS-57741] - Create a taxonomy attribute line service [DMAS-57752] - implement react code [DMAS-57755] - Increase Equinox's framework start level [DMAS-57765] - Update ui tests [DMAS-57773] - Add french and english organization statements for abstract to database [DMAS-57774] - Change code to use config property organization statement in creating Dataset object [DMAS-57794] - rollback java code to recover those two columns [DMAS-57810] - Refactor camera take picture driver file transfer [DMAS-57825] - Display all devices in the title and e-mail subject [DMAS-57826] - code change to make Cruise Listing be reverse datefrom chronologically sorted by default [DMAS-57854] - Update text following demo meeting [DMAS-57912] - Add page to the SeaTube menu in the navigation [DMAS-57914] - Add Attribute Management page to entry.jsx [DMAS-57920] - Allow nullable attributes for ONC expedition loggers and data stewardship Bug (ST) [DMAS-57732] - Fix misspellings in the back end code [DMAS-57733] - Fix misspellings in the React code [DMAS-57740] - master branch with Revision: 0010036539 in Oceans 2.0 deploy failed in QA [DMAS-57749] - No web pages are visible after qaweb2 finishes starting [DMAS-57772] - Any logged-in user can access TaxonomyMatrixAttributeService's POST operations [DMAS-57904] - Some OceanNext features are not working in QA Documentation [DMAS-57329] - Update development environment installation documentation
February 17, 2021
Minor release: FLAC file transfer support, SeaTube and Community Fishers improvements.
[DMAS-57395] - Fix failing test in dataPreviewGeneration
[DMAS-57415] - The datacite abstract is adding unwanted French content to the end of the abtract
[DMAS-57418] - CKAN ISO 19115 records not downloading
[DMAS-57422] - DevicePort Maintenance page has an error when trying to get newly added port
[DMAS-57465] - Link from siteDevice page to ERDDAP record is broken
[DMAS-57472] - Error occurred trying to reprocess qaqc results on dateRange DateRange [Mon Jan 14 00:00:00 UTC 2019 to Tue Jan 15 00:00:00 UTC 2019] for deviceId506
[DMAS-57480] - memory leaking from qaqcreprocess job