Versions Compared

Old Version 21

changes.mady.by.user Ben Biffard

Saved on

compared with

New Version Current

changes.mady.by.user Spencer Plovie

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Kongsberg Mesotech Rotary Sonar Data Product - SWEEP Mode

Data files for Kongsberg Kongsberg Mesotech Rotary Sonar instruments are described here. In the sweep mode, the sonar does a 360 degrees sweep of the nearby water and seabed. For SCAN mode, see this page.

Oceans 3.0 API filter: dataProductCode=KSWDP

Revision History

  • 20120519: Initial SMB product released
  • 20121002: Scan Mode enabled
  • 20140730: MAT-file and plotting products added

...

This data is available in LOG, SMB and MAT format (the log LOG file is a internal Neptune Canada ONC format). Log LOG files are created daily at midnight UTC from data acquired throughout the day. Shortly thereafter, the data products are processed from the log LOG files and are made available to data search. Content descriptions are provided below.

Plots are available in PNG and PDF format.

SMB

This binary format is specific to the manufacturer. When using Kongsberg Mesotech data acquisition software, data is normally stored in this way. Although we use custom-built drivers to communicate with our instruments, we can use the raw data in the log file to produce the SMB file which can be interpreted by Kongsberg Mesotech MS1000 MS 1000 post-processing software.

This format is further described in the manufacturer's documentation: 900-00007904-2.1.pdf. (Here's a more recent version: Singlebeam_SMB_V0230.pdf. A MATLAB library is available upon request to read the SMB file. This code will be used to create a MAT file data product in the future., however, our SMB format is based on the older format specification.) The MATLAB library we use to produce MAT file and plots from SMB files is available upon request. In the replication of the SMB file, the sonarAngle, also known as the startAngle from EXTRA_HIGH_FREQ_SONAR_TUPLE_DATA is corrected to a bearing relative to true North. The correction from instrument relative bearing to true bearing uses the following, in descending order:

  • device true heading from Oceans 3.0 metadata, specified using a fixed heading or a mobile position sensor plus the sitedevice offset. For example: https://data.oceannetworks.ca/DeviceListing?DeviceId=11401 → site tab → https://data.oceannetworks.ca/Sites?siteId=1000659 where offset heading 181 degrees (site fixed heading is not required - only one of site heading or offset heading are, but are additive if both provided).
  • onboard compass, if equipped (new for spring 2021). If the device attribute compass is "ON", the instrument driver software will poll the compass at the start of each SWEEP or SCAN and that data is passed into the SMB file as a DATA_TYPE_HDM tuple, specifying the magnetic heading. A declination correction is applied for the latitude of the device.

This differs from the operation of a rotary sonar onboard a moving vehicle using the MS1000 software: the compass or heading isn't applied to the startAngle, but instead MS1000 rotates the compass rose on the display, keeping the instrument axis pointed up. The compass data appears in the DATA_TYPE_HEAD_SENSORS tuple. In our use case, we're interested in the plot relative to true North, so North is always up. When running an ONC SMB file through MS1000 playback, the software fortunately ignores the compass data, plotting only the sonarAngle/startAngle, which is a true bearing so that the plot on screen is correct.

This data product has been tested mostly with MS 1000 version 4.51, while the new version, 5.2, also tests well. Historic data has been reprocessed and sweep SMB files should be fully availablePlease note, this is a brand new data product. It has only been tested in MS1000 version 4.51 (testing is in progress with version 5.0). It's availability may be limited, there may be data gaps. Daily data processing will begin around July 11, 2012. Some recent data is currently available. Historic data will be reprocessed and will become available in the near future. If a data search returns with 'No data found' when the data availability shows that there is data, please please contact NEPTUNE support us and we will make that data a priority for processing.be happy to investigate.

Oceans 3.0 API filter: extension=smb

Viewing SMB Files

The MS1000 MS 1000 software (originally developed by Kongsberg Mesotech, http://www.kongsberg-mesotech.com-Mesotech) is free to redistribute for playback of the data. Download the MS the software via request here (go to the bottom of that page), or direct download an older Windows version here: MS1000_V0520_unlocked.zip. Then  (unzip and run MS1000_V0520_Setup.exe). To view the data, run the MS1000 software and press the play button. Alternatively, double-click an SMB file to launch the software in playback mode. The fast-forward function is useful for viewing the data quickly.

Previous versions of the MS1000 software did not directly support the high sample high rate that our rotary sonars operate at. To make older versions work, right-click on the image and select 'Fit Image To Screen'. To avoid doing this every time, best to "Save Settings" when prompted while closing the software. Some combinations of Windows OS and MS1000 may not work together; we've had some success with "Run as administrator" (right-click on the MS1000 icon to see that option) and/or running in compatibility mode (right-click on the icon, go to properties and then the compatibility tab).

In some cases, the software does not update the image promptly. Wiggle your mouse and the display will update. (It appears to be a limitation of the software and graphical capability of one's computer.) The fast forward function of the software is very useful to view a large amount of data quickly.

 

MAT

MAT files (v7) can be opened using MathWorks MATLAB 7.0 or later. The file contains two structures: Meta and Data.

Include Page

...

MAT file Metadata structure - Complex

...

MAT file Metadata structure - Complex

Data: structure containing the ADCP data, having the following fields.

...

the Kongsberg rotary sonar data, tightly based on the tuple structure of SMB files. Only the important fields are listed below. For more information, refer to the SMB file specfication: 900-00007904-2.1.pdf. Please note that not all structures and fields available in the SMB files are converted and provided in the MAT file all of the time. If there is anything missing or something to be improved, please contact us, we'll be happy to make improvements.

These structures are from the sonar ping tuple:

  • overloadCounter: logical value that indicates signal saturation in the corresponding sonar ping (one for each sonar ping / profile).
  • sonarData: data matrix, with the rows representing each sonar ping, columns are range. Format is uint16 or uint8, as controlled by extraHdr.BitsPerSample - '8' meand 16 bit, '16' means 8 bit!
  • sonarTime: array of times for each ping in matlab datenum format (days since year 0)
  • sonarAngle: sonar pointing angle for each ping in degrees, copied from extraHdr.StartAngle for clarity
  • sampleRate: sampling rate in Hz.
  • samplingDelay: sampling delay or hold off (to blank transmit) in samples.
    The following structures are part of the sonar ping tuple, but are only provided when the SMB source file is relatively small. These structures are slow to process, consume memory, and duplicate data that's already captured in the preceding structures.
  • hdr: a structure representing the standard SMB tuple header (one for each sonar ping / profile):
    • DataType: type of data contained within this tuple, 12 is single ping data
    • DataTime: time at which data was collected; milliseconds since midnight UTC.
  • extraHdr: a structure representing a common header used to provide extra information for sonar and profile data tuples (one for each sonar ping / profile):
    • ScanDirection: 0 = clockwise, 1 = counterclockwise
    • StepSize: step size of head in 0.225 degree increments
    • StartAngle: start angle of scan in 0.225 degree increments. This is sonar pointing angle copied to Data.sonarAngle below.
    • BitsPerSample: Number of bits in each sample
    • SampleRate: sampling rate in Hz.
    • SamplingDelay: sampling delay or hold off (to blank transmit) in samples

Here is the settings tuple:

  • settings: a structure containing information on the sonar settings
    • hdr: header for the settings tuple
    • extraHdr: a copy of the first extraHdr structure
    • config: a structure of settings with many fields, the important ones are:
      • Range: range of data to display in meters
      • SampleCount: number of samples recorded in each ping - not accurate, use size(Data.sonarData, 2) to determine this
      • Gain: sonar gain
      • PulseLength: sonar transmit pulse duration in microseconds
      • SoundSpeedDefault: sound speed in meters per seconds
    • ftr: a structure representing the settings footer tuple

Here is the date tuple:

  • date: a structure with the following date/time fields, modified from the SMB time tuple:
    • dateFrom: start time of source SMB files in MATLAB datenum format: days since year 0.
    • dateTo: end time of source SMB files in MATLAB datenum format: days since year 0.
    • Version:
    • TimeUTC: start time of source SMB file in seconds since 1970.
    • hdr: header for the date tuple

Additional tuples for heading, location maybe added in the future.

Oceans 3.0 API filter: extension=mat

PNG / PDF

The PNG / PDF plot data products mirror the display of the Kongsberg MS1000 software, but break each rotation into separate plots so that no data is overwritten or has to be averaged. The raw backscatter amplitude is plotted as contained in the MAT file: 16 bit data ranges from 0 to 65535, 8 bit ranges from 0 to 255. The backscatter data is not calibrated. For each source SMB file, there are two types of plots returned: a plot of the sonar pointing / heading angle as a function of time (called 'SonarAngle') and multiple backscatter polar plots (called 'Backscatter'). The SonarAngle plot has numbered annotations showing the start of each corresponding numbered Backscatter plot. In the examples below, the SonarAngle plot shows that five Backscatter plots will be created for this SMB source file. The PDF version of this data product will group this plots into a single PDF file with multiple pages. Please note that these examples are of test tank data; real data looks much better.

  Image Added

  Image Added

The algorithms for detecting each rotation and for making the plots are somewhat complex. Contact us if you are interested in the source code for these plots, which would be useful for analyzing the data from the MAT files.

Oceans 3.0 API filter: extension={png,pdf}

GIF 

The GIF is an animation of the PNG plots seen above excluding the sonar pointing / heading angle as a function of time plot. The GIF plots are broken up daily so they run through every PNG in a day.

Image Added

 Oceans 3.0 API filter: extension=gif

...

Discussion

To comment on this product, click Add Comment below.