New Features Release Notes

Bug

• [DMAS-39745] - Investigate potential error in orientation of upward/downward ADCP data for Endeavour moorings
• [DMAS-39831] - Telnet Console not filtering data to device id, showing every device
• [DMAS-40078] - East Node Junction Box view gives wrong information
• [DMAS-41956] - QAQC autotest UI is not updating to match what's in the database
• [DMAS-42007] - Provide fileSizes in searchresult for files from AD (matlab)
• [DMAS-42204] - two searches (searchid 5633033 and 5633034) in prod are not able to be completed within 14 days
• [DMAS-42300] - Data Preview "day" plots slow, causing scans of qaqcresult
• [DMAS-42340] - Automation failing to handle the case where raw data service returns a reading as an array of integers
• [DMAS-42359] - Unable to edit/delete rules in Data Preview Management
• [DMAS-42450] - Automated DAF Web Service throwing null pointer exception on devices without IP configuration
• [DMAS-42469] - Device Console's RowExpandingDataTable Has an Infinite Event Loop
• [DMAS-42472] - DeviceSensorService and DeviceElectricalRatingService are not available to anonymous users
• [DMAS-42545] - VideoFileQAQC keeps failing on the same 13 files
• [DMAS-42556] - RDI ADCP current plot for the last 24 hr doesn't show data past midnight
• [DMAS-42557] - DMAS Search Failed Notification: 18-Jan-2018 19:39:01 UTC, Folger Passage-Folger Deep, RDI Daily Current Plot, SearchHdrId 2590660, UserId 15223
• [DMAS-42559] - MatlabFilePostProcessor - One or more files not found
• [DMAS-42646] - Thomson detided mat file missing at Cascadia, default filters hide Jan 23 Alaska tsunami
• [DMAS-42680] - Some complex MAT files fail automation due to a samplePeriodDateTo that is set to 'now'
• [DMAS-42686] - Error while downloading files from searches - QA automation
• [DMAS-42721] - hydrophonepostprocesssjob failing due to empty deviceID field
• [DMAS-42734] - DMAS Search Failed Notification: 29-Jan-2018 16:23:28 UTC, East-Strait of Georgia East VENUS Instrument Platform, Time Series Scalar Plot, SearchHdrId 2603770, UserId 48400
• [DMAS-42763] - unable to find files that exist due to overspecification of data product options, but error when not specified
• [DMAS-42765] - Reprocessing failing to downloading files after completing ~30 files (mostly in QA)
• [DMAS-42774] - Reprocessing for ASL SWIP (Ice Profiler, deviceID 524, Oct-Nov 2012) fails due calibration service error with new formulae
• [DMAS-42825] - DMAS Search Failed Notification: 01-Feb-2018 22:34:25 UTC, Patricia Bay-Saanich Inlet VENUS Instrument Platform, ASL Acoustic Profiler Time Series, SearchHdrId 2609469, UserId 42732
• [DMAS-42830] - DMAS Search Failed Notification: 01-Feb-2018 23:19:34 UTC, JASCO M36-V35-100 Hydrophone C000682, Hydrophone Spectral Data, SearchHdrId 2609472, UserId 4143
• [DMAS-42847] - parser.q is missed in mq.queues.UVIC[TEST] of mq.properties file
• [DMAS-42848] - SeaTubeV2 - rendering a blank comment breaks the annotation format and functionality
• [DMAS-42849] - SeaTubeV2 - recording a clip and saving it will not populate playlist till refresh
• [DMAS-42870] - Matlab Search Failure Due to VM File Failure

User Story

• [DMAS-34635] - As a Nortek Signature 55 user, I would like the processed data products to match output from the SignatureViewer software
• [DMAS-38426] - As an engineer I would like DAF for the Memorial Buoy and it's instruments
• [DMAS-39979] - As a scientist, I would like a parser for the ROPOS temperature probe (devicetypeid 475)
• [DMAS-42156] - Remove the VENUS left over ArchiveFileType table and relations from DMAS and database
• [DMAS-42265] - As a Science User I would like the Data Preview page to include obvious links to the data behind the plots
• [DMAS-42321] - As a tester, I would like to have a dashboard that summarizes test results from all sections of automation
• [DMAS-42333] - Generate test result database entries for automated web services
• [DMAS-42334] - As a tester, I want data search automation to write results to the database
• [DMAS-42348] - As an Oceans 2.0 User I want the Device Console Page's Performance to be Investigated
• [DMAS-42357] - As an logged-in ISSMMTC website user, I would like to see only one Registration tab
• [DMAS-42403] - As a data steward, I would like parsers for devicetypes 356, 357 for DFO-Drifter
• [DMAS-42404] - As a scientist, I would like data parsed from DFO-Drifter files to be archived
• [DMAS-42410] - As an Oceans 2.0 User, I want the Device Console's Jira Issues Tab to Show the Latest Open Tickets
• [DMAS-42498] - As a software engineer, I would like to transfer knowledge to metadata team to configure WARN Event Maintenance
• [DMAS-42585] - As a tester, I want automated web service tests to be stored in a database instead of a file system
• [DMAS-42601] - As a tester, I want data search automation to store and read test cases from a database
• [DMAS-42622] - As an ISSMMT participant, I would like create an account and successfully submit a poster abstract
• [DMAS-42647] - As a data steward, I would like post-process job for DFO Files

Task

• [DMAS-24281] - Need better wording for "log in to access this feature" dialog
• [DMAS-31989] - Investigate which decoding method is correct for the videos
• [DMAS-39571] - Clean up assemblers unnecessarily implemented as services in usermanagement
• [DMAS-41880] - SCUV2 Driver Improvements
• [DMAS-41993] - Digital Fishers current campaign behaviour not as intended
• [DMAS-42076] - Shorestation for Marine Institute needs to authenticate to activemq
• [DMAS-42331] - Have Reprocess Automation select a random day between site devices
• [DMAS-42332] - Store automated reprocess results in a table instead of clearing them each run
• [DMAS-42443] - Add new .sip files for SAStracker data products
• [DMAS-42460] - Switch automation over from using raw data service method getByDevice2 to getByDevice
• [DMAS-42535] - Calculate EEW rupture scenario lead times for BC leg engineers
• [DMAS-42652] - Add new parameter "convertHexToDecimal" to raw data web service call in Automated DAF Testing
• [DMAS-42729] - Nanometrics Titan at Bullseye is not visible in Data Preview
• [DMAS-42851] - Document ULS data products
• [DMAS-42912] - Fix unit test bugs in GenericLocalDDMPostProcessorTest

Requirement

• [DMAS-28554] - ASL Acoustic Scintillation Requirements
• [DMAS-38912] - Gather Requirements
• [DMAS-39228] - Gather requirements for EEW notifications
• [DMAS-39709] - Meet with Systems to discuss how memorial buoys should connect to ActiveMQ
• [DMAS-42465] - Gather requirements

Design

• [DMAS-28607] - ASL Acoustic Scintillation design for File Acquisition
• [DMAS-39229] - Design EEW notification infrastructure
• [DMAS-39497] - Meeting to discuss transformation of Light DDM to proper DDM
• [DMAS-39710] - Have a meeting with senior devs to discuss DAF design changes to support memorial buoys
• [DMAS-42067] - Design cloning utility
• [DMAS-42322] - Design screens and functionality of the automation dashboard
• [DMAS-42405] - Design schedule job to archive DFO-Drifter files
• [DMAS-42454] - Design QAQC for DFO Drifter
• [DMAS-42586] - Design the table structure to match what web services currently use in files
• [DMAS-42648] - Design post-process job for DFO drifter

Implementation

• [DMAS-28705] - Create a schedule job for the ASL sftp
• [DMAS-29168] - Modify the driver to set the ASL instrument time once the command is available
• [DMAS-29174] - Create a sql script to create a clock sensor
• [DMAS-29221] - Add 3-beam solution to RDI ADCP beam to XYZ transform to fix disagreement with winADCP results (matlab)
• [DMAS-32277] - Implement corrected screening (matlab)
• [DMAS-32609] - Refactor BPR Import as Scheduled Job
• [DMAS-38321] - As a WHOI engineer, I would like parsers for the SIIM
• [DMAS-38540] - Create a web service to kick off an automated DAF test
• [DMAS-38587] - Create a tape file from the Mexico data
• [DMAS-38684] - Update WARNNotificationClientAPI to support heartbeat messages
• [DMAS-38693] - Add pd and pgd calculation support to matlab
• [DMAS-38747] - Add Support for ASCII Mode for New CORKs
• [DMAS-38794] - Create a parser and unit test for the Thrane and Thrane device type (189)
• [DMAS-38997] - Investigate how the Declarative Data Parser uses sensor load.
• [DMAS-39191] - As a WHOI engineer, I would like parsers for the BIP
• [DMAS-39321] - Upsample tohuku accelerometer data to 200 Hz
• [DMAS-39378] - Create a unit test for the GSC_Seismometer_Source parser
• [DMAS-39407] - Add expected results of the KalmanFilterAlgorithm unit test
• [DMAS-39500] - Meet with Systems about setting up ActiveMQ at Memorial Institute
• [DMAS-39527] - Separate the Event Detection from Qaqc
• [DMAS-39586] - Add screening options for RDI ADCP data products
• [DMAS-39978] - Extend Receiver Message Service to support External Device Messages
• [DMAS-41658] - Create Parser for ROPOS Temperature Probe
• [DMAS-41916] - Support NaN in the calibration formula
• [DMAS-42078] - Adjust styling of list to better match page
• [DMAS-42101] - Implement cloning utility
• [DMAS-42134] - Adding an event through the maintenance page sets requiresAllVariables to false
• [DMAS-42143] - Modify the UI so when an event is created it has all global reactions mapped to it
• [DMAS-42144] - Convert the WARN Correlations formula from the CalculationService into the EventDefinition factory
• [DMAS-42145] - Change how the EventDefinition Cache to support a single sensor to have multiple Event Definitions
• [DMAS-42146] - Change the input of the eventreaction to use the eventdetected message instead of the payload
• [DMAS-42147] - The eventreaction should output the EventReactionLog instead of logging it directly
• [DMAS-42148] - Create a queue interface for the EventReaction input
• [DMAS-42149] - Create a queue output interface for the EventReaction
• [DMAS-42150] - Create an Activator to support archiving of EventDetected message
• [DMAS-42168] - Remove the EventDefinitionManager from the EventDetectionContainerService
• [DMAS-42176] - Update WARNNotificationApiClient to repeatedly call getEarthquakeDetectionStatus for heartbeat
• [DMAS-42194] - Create Unit Testing for Notifiers
• [DMAS-42220] - Remove the EventDetection from the Qaqc Service
• [DMAS-42224] - Implement script for scheduled job
• [DMAS-42225] - Event Detection and Event Reaction should have separate package
• [DMAS-42226] - The DataMessageProcessor is using the QaqcCacheEntry to create EventDefinition, this needs to be updated with the new framework
• [DMAS-42233] - Create an Activator to support archiving of EventReactionLog
• [DMAS-42234] - Separate out the Qaqc Archiving and Event Archiving from the EventArchiver class
• [DMAS-42304] - Create Notification Module
• [DMAS-42323] - Create new dashboard module in Liferay
• [DMAS-42343] - Remove EventReactionLogManager from the EventReactionThread
• [DMAS-42347] - Modify DMAS to be able to handle server notifications from the DATACENTER
• [DMAS-42384] - Work with Data Stewardship to setup device mappings
• [DMAS-42389] - Set the event detection / reaction log queues to not use mq.properties
• [DMAS-42397] - When annotation list is filtered, seeking functionality doesn't work correctly
• [DMAS-42400] - Create a new table to store data search results
• [DMAS-42401] - Make the data search automation tests write results to the database
• [DMAS-42402] - Add the ability to display and re-run the most recently run DAF tests
• [DMAS-42406] - Implement schedule job to archive DFO-Drifter files
• [DMAS-42408] - Create a new table to store automated web service results
• [DMAS-42409] - Have web service tests write results to the database
• [DMAS-42419] - Add the ability to display and re-run Task tests
• [DMAS-42424] - Add Links to State of the Oceans Plot Data Files to Data Preview
• [DMAS-42431] - Join the two sets of tabs on the left into one set.
• [DMAS-42435] - Add the ability to display and re-run automated reprocess tests
• [DMAS-42436] - Add the ability to see automated web service tests
• [DMAS-42437] - Add the ability to see automated data search results
• [DMAS-42444] - EventDetectedIds will have to be created when EventDetected object is save into the database
• [DMAS-42446] - Create a script to add global reactions to all existing events
• [DMAS-42448] - Add a location field to the EventDetected and EventReactionLog table also add a timestamp to EventReactionLog
• [DMAS-42456] - Remove unused methods of Device
• [DMAS-42466] - Remove the selector and send event detected messages only to the task machine
• [DMAS-42496] - Make sure that only the WARN event detection server notification will be process in the DataCenter
• [DMAS-42504] - The EventReactionFactory need to be a single instance
• [DMAS-42505] - The reaction mapped to an event definition must be retrieved from the cache
• [DMAS-42509] - Implement getChildrenDevices in the Topoplogy Manager
• [DMAS-42524] - Add the EVENT_DEFINITIONS cache to be save in the serialize cache
• [DMAS-42527] - Implement clone QAQC
• [DMAS-42534] - Implement method of skipping data validation for aggregator device.
• [DMAS-42568] - add the ability to query the webservice for specific device types availability
• [DMAS-42587] - Write a script to create the new web service test case table and subcase table
• [DMAS-42588] - Convert current web service test cases to database entries
• [DMAS-42589] - Change functionality for creating test cases to use the database table
• [DMAS-42595] - Change functionality of running test cases to use the database table instead of reading files
• [DMAS-42627] - The NotificationActivator is not shutting down the Notification Listener thread during shutdown
• [DMAS-42637] - write unit tests to support new functionality
• [DMAS-42642] - Add a TreeView for taxonomies
• [DMAS-42645] - Branch Reintegration
• [DMAS-42649] - Implement DFO Drifter Post-Process job
• [DMAS-42663] - Dont allow events to be saved unless they have a name, description, and expression added to them
• [DMAS-42670] - Add the propertyCode to the matlab data structure
• [DMAS-42675] - Implement QAQC formula for DFO Drifter
• [DMAS-42690] - Write Documentation for the Receiver Message Service Activator
• [DMAS-42740] - Modify jobs to support local time in DFO-Drifter files
• [DMAS-42746] - Remove the dependency from Device to DeviceActionManager
• [DMAS-42750] - Remove the dependency of DeviceManager on TopologyManager
• [DMAS-42757] - Convert folder names containing output and expected files to use testId instead of testNumber
• [DMAS-42760] - Remove the dependency from SiteDevice to DeviceManager
• [DMAS-42798] - Remove unused methods of SiteManager
• [DMAS-42882] - Parser for BOOTS navigation data
• [DMAS-42883] - Parser Test for BOOTS navigation data parser

Test

• [DMAS-39428] - Import the upsample tohuko data into the DAF automated test database
• [DMAS-41659] - Create Unit Test for ROPOS Temperature Probe
• [DMAS-42119] - Create unit testing for the LightWeightDataMessageProcessorService and the LightWeightDataMessageProcessorServiceActivator
• [DMAS-42328] - Test the changes to the notification system
• [DMAS-42335] - Create Unit tests for Event Reaction Queues and the Activators reading from the Queues
• [DMAS-42350] - Create Unit Test for DataMessageProcessorService
• [DMAS-42426] - Create Unit Tests for the event definition service and JMSDestinationService
• [DMAS-42570] - Create DataPreviewService Automation Test Cases to Test New Link Configurations
• [DMAS-42572] - Create Automation Tests to Cover Changes Made to Device Console
• [DMAS-42581] - Update/Add TicketManagerService Tests
• [DMAS-42614] - Create Unit test for EventDetectionService
• [DMAS-42807] - Add TicketManagerService Unit tests

Bug (ST)

• [DMAS-39519] - Create DecPar's for memU Bonavista Buoy device
• [DMAS-39690] - Simulator for CORK (devicetype 68) not support Ascii mode
• [DMAS-39873] - The Titan DAF Autotest case 286 has some data missing data in the RawData table
• [DMAS-42133] - Fix getEarthquakeDetectionStatus so that it works properly with simulator earthquake data
• [DMAS-42167] - Unexpected bytes prepended to midnight data line
• [DMAS-42246] - Change button colour to match dmas buttons
• [DMAS-42324] - Write documentation for QAQC formula for ferry speed/PVCS.
• [DMAS-42360] - After creating the notification module the subscription page for events doesn't work
• [DMAS-42364] - send test notification does not work
• [DMAS-42391] - TicketManagerService Takes 5+ Minutes to Return
• [DMAS-42398] - Seatube map doesnt handle resize properly
• [DMAS-42447] - Find and fix the source of the svn bugs with eewnotification
• [DMAS-42449] - Device Console Tab Switching is Slow
• [DMAS-42479] - Device Console Imports Two Different RowExpansionDataTable.js Files
• [DMAS-42511] - the notification activator is processing messages slowly with the selector in place causing the event detected queue to build up
• [DMAS-42513] - Error saving EventReactionLog
• [DMAS-42514] - Fetch Latest Archived File Query Takes 30 Seconds and Slows Down Device Console
• [DMAS-42521] - Emails not being saved
• [DMAS-42544] - DAF automated test 273 The ppp and titan combined doesnt work
• [DMAS-42546] - TitanAccelerometerDevice tests are failing after fixes to its serializability test
• [DMAS-42578] - Move Filtering of Jira Tickets to Backend Via Parameter to the Web Service
• [DMAS-42580] - Move Sorting of Tickets to Jira Query
• [DMAS-42611] - subscriptions are being saved with empty strings rather than null
• [DMAS-42618] - All the event notifications are being sent to the notifiers regardless of which notifier (email/server/apple) they should have gone to.
• [DMAS-42631] - When trying to filter on the dashboard, table is empty when there should be results
• [DMAS-42632] - Remove Second Registration Tab
• [DMAS-42640] - Fix IGCP 640 Liferay Registration
• [DMAS-42650] - getEarthquakeDetectionStatus fails
• [DMAS-42664] - Fix Accessors in Data Preview POJOs
• [DMAS-42684] - modules from webpack are only accessible in liferay as their module definition
• [DMAS-42718] - In QA the location of event detected did not get populated, but in DEV it was working
• [DMAS-42726] - PVCS events dont show up in the event log
• [DMAS-42756] - qaeewdrv1 not showing up in status console and does not receive driver commands
• [DMAS-42758] - QA activemq shows reactionlog not being dequeued
• [DMAS-42771] - Nullpointer exception occur when new eventdefinition are created
• [DMAS-42809] - Auto reload qaqc cache after clone job
• [DMAS-42834] - fix unit tests
• [DMAS-42853] - Task machine events not triggering
• [DMAS-42878] - Investigate CAP Messages not working

Investigation

• [DMAS-39225] - Investigate and evaluate different methods of notification
• [DMAS-39318] - Investigate in detail how the current notification system work, inorder to propose some architecture changes
• [DMAS-42029] - Investigate options on how to approach heartbeat
• [DMAS-42086] - Investigate how other organizations send heartbeat with redundant servers
• [DMAS-42087] - Investigate possible protocols that we could use to receive response from client that subscribes to EEW notifications
• [DMAS-42088] - Investigate the scope involve in harmonizing the event notification system
• [DMAS-42142] - Investigate the use of reprocess column in the eventdefinition table
• [DMAS-42151] - Investigate package dependencies for event detection and reaction inorder to modularize
• [DMAS-42153] - Convert the notification module as a type of event reaction
• [DMAS-42232] - Investigate if Notification can be created into its own module
• [DMAS-42354] - Automate the integration testing of the email notification
• [DMAS-42383] - Investigate Device Console Page's Performance Issues
• [DMAS-42388] - Investigate Adding Data Source Links to Data Preview
• [DMAS-42407] - Investigate DFO-Drifter archive job
• [DMAS-42579] - Investigate how much work still needs to be done for the heartbeat service

Documentation

• [DMAS-41660] - Create Documentation for ROPOS Temperature Probe
• [DMAS-42195] - Create a design document for the changes in Archiver to support EventDetection and EventReaction archiving
• [DMAS-42626] - Document how to use the Event Maintenance page for EEW
• [DMAS-42695] - Update documentation for post-process job
• [DMAS-42782] - Document DFO Drifter Archive Job
• [DMAS-42783] - Document DFO Drifter Clone Script

Sub-task

• [DMAS-41728] - Refactor DeviceFactory.java
• [DMAS-41770] - Work with Systems to setup ActiveMQ to support Memorial Buoy data
• [DMAS-41859] - Create declarative data parsers and unit tests
• [DMAS-42258] - Test device cloning tool implementation