27

Satlantic Radiometer Time Series

The Satlantic Radiometers are light sensing devices that measure wavelengths and intensities of light in the ocean. The are two variations supported by this data product: profiling and the surface acquisition system (SAS or SAStracker). The profilering radiometers are usually deployed as a pair to measure light from above (downwelling light) and light from below (upwelling light). The SAStracker is deployed onboard moving vessels, most notably on the BC ferries.

Oceans 3.0 API filter: dataProductCode=SRTS

Revision History

1. 20110302: Beta product released
2. 20140602: Major revision to better include ancillary data (depths, tilts), filtering, reliability fixes, improved documentation
3. 20160624: Added support for SAStracker

Formats

This data is available in a binary .raw format. Content descriptions and example files are provided below.

RAW

This format corresponds to the data files produced by the Satlantic data acquisition software. This file format, together with associated calibration files, can be interpreted by Satlantic post-processing software (e.g., SatCon, ProSoft).

New RAW files are started at the start of each day or when the driver is restarted (this should account for configuration changes, site changes, etc).

RAW files for profiling Radiometers (located on the vertical or bouy profiling systems)

In operation with the vertical profiling system (VPS), the radiometers are usually started and stopped at the beginning and end of each cast segment, 'up' or 'down', through the water column. This effectively breaks RAW files into cast segments. A cast specific data product, that explicitly segments the data into casts, will be made available with future improvements. For now, it is recommended that users compare the start and end dates of the files returned with the cast times that they can determine from plotting utility or from time series scalar plots of CTD pressure on the VPS. (To use the reference to plotting utility, change the date as needed and scroll down to see the depth data). When a file spans multiple casts, Prosoft's higher level analysis may not be useful or correct, see below.

The radiometers do not have on-board pressure or depth sensors. Because of this, the RAW file data product requires data to be available from a co-located depth or pressure sensors such as a CTD. If no reference data is available, the data product request will return with a 'no data found' message, even though there may be radiometer source data. Without reference depth or pressure data, the radiometer data is not useable. Pressure sensor data from a co-located CTD is is converted to water depths using the International Thermodynamic Equation of SeaWater (TEOS-10) and then integrated into the RAW files as an ancillary sensor.

Prior to February 2014, the radiometers on the VPS did not have tilt sensors (Satlantic Downwelling Radiometer HOCR-ICSW (12106) Details | Documentation, Satlantic Upwelling Radiometer HOCR-RO8 (12107) Details | Documentation). Ancillary tilt data is very useful, but not necessary, so data products will still be generated without tilt data. For all radiometers, when available, data for tilt (pitch and roll) is added as ancillary sensors into the RAW files. For the VPS radiometers, when necessary, tilts are provided by the co-located MEMS 3- axis accelerometer and gyroscope, normally a Crossbow 440 Inertial System.

To ingest the ancillary data, the calibration files and instrument parameters need to document what has been added. A calibration file named MPR001.cal provides this information (for more on the RAW and cal file formats, see the file format specification). The analysis of the RAW files also requires the relative positions of the sensors; this information is built into the .cfs parameters file (provided below). The relative positions are discussed in the Prosoft manual, see page 15: '6.5.2 Configuring Sensor Distances'.

RAW files for non-profiling Radiometers / SAStracker (located on BC ferries)

The SAStracker raw files are simpler than the profiler units; they do not need ancillary inserted from external devices.

Oceans 3.0 API filter: extension=raw

Calibration Files (.cal, .sip, .cfs)

Calibration files have several formats that all may be used with Satlantic's post-processing software ProSoft. CAL and TDF files are the raw format. They contain necessary metadata on the RAW file structure so they must accompany the RAW files in order to be read by any software. CAL files also contain information on the wavelengths measured and the intensity calibrations. Because a single radiometer may have multiple sensors and a RAW file may contain data from several devices, several CAL files may be needed for a RAW file. Therefore, to ease file handling, CAL files may be zipped together in a single .zip file; for emailing, security reasons, etc., these zip files are renamed to a SIP file. However! for the profiling radiometers only, the SIP file does not contain all the information needed to read and process a RAW file: we still need the relative positions and the 'Frame Type' may need to be modified. This information is captured in a CFS file. Unlike a SIP file, CFS files only contain references to the source CAL files. To use a CFS file, you will need to have the CFS and referenced CAL files together in the same directory, otherwise an error will occur when importing the CFS files. To that end for the profiling radiometers, the necessary CFS and CAL files are provided to users with RAW files together in a zip file. The necessary CFS and CAL files are determined dynamically by data availability as the RAW files are generated. For the non-profiling radiometers, such as the SAStracker, the SIP files are provided for the data requested. In this way, users will always have the calibration files they need to process the data; expand the zip file that you download from Data Search and the data is ready to process in that directory with ProSoft.

For the profiling radiometers, the original SIP files are available in the instrument documentation pages (see the child pages of the radiometer home page).

Quick Start and Tips for processing the RAW file data using ProSoft

(Underlined text refers to user interface elements in ProSoft)

1. Visit the Satlantic ProSoft webpage to download and install the post-processing software.
2. Request Radiometer RAW data products from data search, download and unzip, creating a new working directory where ProSoft can create new files as it processes the raw data.
3. Start ProSoft.
4. Import instrument configuration and calibration: in the Processing Context / Current Instrument: 
   for profiling radiometers click Import, navigate to the working directory, select a CFS file;
   for non-profiling radiometers click New, navigate to the working directory, select a SIP file
   then press save once it's done reading the file. (If you've done this step before, select the Current Instrument you will be working on.)
5. Create a new/default Current Parameters, click New under current parameters, then Save As..., name it something like 'Default'. (You may create additional instances of Current Parameters and edit as necessary) 
6. For non-profiling radiometers, follow the normal processing flow, except you will probably need to exclude the SATMSG.tdf from the loaded calibration files (it was causing failures to read the data). You'll also need to choose the parameters in the both Current Instrument and Current Parameters to match the instrument and processing objectives, see the ProSoft User Manual for more information. When setting up the Current Instrument you can save and import a CFS file. Here's an example CFS file for the SAStracker: test3.cfs. Once everything is set up, select Multi-Level Processing Level 1 --> 3a and select the RAW files in working directory that you wish to process (we tend to go to Level 3a because that's as far as a default parameters will take you).
   
   For profiling radionmeters, the processing flow is more complex: select Multi-Level Processing Level 1 --> 4. This will bring up a window to select the RAW file to process (locate the working directory, then 'Add' a file to process. Recommend only working on one file at a time, as each file is likely from a different cast, unless the files span midnight. This is essential: make sure that the name of the RAW file you select corresponds to the selected Current Instrument you wish to process - check the date range and serial number in the CFS file-name you loaded in step 4, select a RAW file that fits within the time range and matches the serial number (i.e. HOCR-R08W). For a download of a single cast segment, there are normally 2 CFS files and 2 RAW files, one pair for upwelling and downwelling radiometers. Additional CFS files are added only when necessary. If ProSoft asks to confirm the pressure tare initial value in a pop-up dialogue box, enter '0.0'. The pressure tare is always zero because our pressure values come from a calibrated CTD.
7. If there is a problem with the processing (ProSoft freezes), read the manual, see the following tips or contact us. We encourage our users to get familiar with the operation of ProSoft, see the ProSoft User Manual for more information. In addition, here are some tips to attempt to get ProSoft to work. Following the steps above, but instead selecting Single Level Processing, users should always be able to get to level 1b where calibrations have been applied. With the level 1b data, users can visualize and export the calibrated data (see the steps below). Level 1b --> 2 processing involves referencing, sorting and sequencing (deglitching). For a problem in reaching level 2, try turning off the 'Profile Edit' (this will disable exclusion of bad data). Also try turning off 'Deglitch Profiler Data'. Further, try restarting ProSoft, then investigate and experiment with the Current Parameters. For problems in level 3, experiment with the bin intervals - larger can work. For level 4, some products are not supported (especially for non-profiling radiometers and SAStracker). We've also found some problems with the progression of data in profiling radiometers; ProSoft appears to prefer individual casts where the radiometers are consistently moving up or down. Water depth variation due to surface waves may cause problems. Users may need to cut out some data at the top of casts by limiting the data search time range. User may also need to combine RAW files in ProSoft by processing these files together. As noted above, we may improve this data product by integrating cast delimitation so that we can provide RAW files that are much easier to process; including joining good data together, while eliminating difficult data.
8. View or extract data from the HDF files produced, see the 'Tools' menu. Create ASCII data products, see the 'Ascii' menu. A quick look with the HDF data viewer is a great way to get an initial sense of the data.
9. Repeat steps 4 thru 9 for the remaining RAW files.

Following the steps above, but instead, .

Discussion

To comment on this product, click Add Comment below.