You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 35 Next »

The ONC MatLab client library contains a number of classes and functions that make access to the ONC data, through the Oceans 2.0 API, quick and easy to use.

Requires

Currently, only R2017a is supported, due to updates to the HTTP library that support payloads with non-200 status codes

Installation

The ONC client library is a Add-On Toolbox that can be installed in MatLab by performing the following

  1. Download the latest version of the ONC Toolbox
  2. Open MATLAB R2017a and click the 'Open' button.



  3. Select the downloaded file and click the 'Open' button.



  4. When presented with the 'Install onc' dialog, click the 'Install' button.



  5. To view the ONC Add-On toolbox click on the 'Add-On' button on the Home ribbon and select 'Manage Add-Ons'






Versions

version
Date
Description
Toolbox
Source
1.06/28/2017Beta version

onc-1.0.mltbx

 
1.017/18/2017Added getDirectScalar, getDirectRaw, getDirectFilesonc-1.01.mltbxonc-1.02-source.zip
1.027/28/2017

Resolve unhandled exception with invalid parameters.

Resolved naming inconsistencies

onc-1.02.mltbxonc-1.02-source.zip
1.038/01/2017Added getDataProductUrls(), downloadFile() and decodeJsonFromUrl() functionsonc-1.03.mltbxonc-1.03-source.zip
1.048/03/2017Updated production url from http://dmas.uvic.ca to http://data.oceannetworks.caonc-1.04.mltbxonc-1.04-source.zip
1.058/11/2017Rename 'parameters' parameter to 'filters' in all applicable onc class functionsonc-1.05.mltbxonc-1.04-source.zip
1.068/15/2017Added file parameter defult to the download() function.onc-1.06.mltbxonc-1.06-source.zip

To download an attachment, please right click on the link and select Save Link AsSave Target As or a similar option provided by your browser. This will open a 'Save' dialog.

New versions are created to resolve known client library issues, add new functionality or handle changes to the underlying Web Services.

The latest version works with the current deployment of the underlying web services. See API Reference

Classes

 

class
description
oncA wrapper class for access to Oceans 2.0 API web services

 

ONC Class

Description

This class provides a wrapper for some for the most common Oceans 2.0 API requests, including the discovery services (locations, devices, deviceCategories, properties) and delivery (dataProductDelivery, scalardata and rawdata).

Constructor

onc()

onc(token, production, showInfo, outPath, timeout)
Parameter
Type
Description
Example
Required   
tokenstringAll Web Services require a token. This can be generated at http://dmas.uvic.ca/Profile. Click on the "Web Services" tab and click "Generate Token"token=5ba514e8-1203-428c-8835-879c8173e387
Optional   
productionboolean

Indicates if the ONC Production server URL is used for service requests.

  • Default is true.
  • false is intended for internal ONC testing only.
  • All external users should use false or leave blank.
true or false
showInfoboolean

Indicates if verbose script messages are displayed, such as request url and processing time information.

  • Default is false.
  • True is intended for script debugging.
true or false
outPathstring

The path that downloaded files are saved to.

  • Default is 'c:/Temp'.
  • Full path will be created if it does not already exist.
'c:/ONC/Download'
timeoutint

The web service request timeout in seconds

  • Default is 60
30

 

Usage

o = onc('YOUR_TOKEN_HERE');

or

isProduction = true;
showInfo = false;
outPath = 'c:/ONC/Data';
timeout = 30;
o = onc('YOUR_TOKEN_HERE',isProduction,showInfo,outPath,timeout);

 

Discovery methods

getLocations()

getLocations(filters)

 

Returns a list of locations, as a list of struct objects that meet the criteria defined by the input filters.

 

Parameter
Type
Description
Example
Optional   
filtersstruct or array

A comman separated list of one or more filter criteria, used to return an exclusive set of location elements from the ONC locations endpoint.

struct('locationCode','BACAX')

or

{'locationCode','BACAX'}

 

Returns

A list of location objects.

        deployments: 50
       locationName: 'Axis '
              depth: 984.3076
               bbox: [1×1 struct]
        description: 'Depth: 985 m Latitude: 48.3167 Longitude: -126.0501 Type: Stationary platform Description: Canyon axis: benthic processes, biodiversity, sediment dynamics.'
      hasDeviceData: 'true'
                lon: -126.0503
       locationCode: 'BACAX'
    hasPropertyData: 'true'
                lat: 48.3167
      dataSearchURL: 'http://data.oceannetworks.ca/DataSearch?location=BACAX'
Property
Type
Description
location[]listA list of location dictionaries

location[].deployments

intThe number of instrument deployments that meet the filter criteria
location[].locationNamestringThe full name of the location
location[].depthdoubleThe average depth of the deployments
location[].bboxdictionaryA Bounding Box dictionary
location[].bbox.maxDepthdoubleThe maximum depth of the deployments
location[].bbox.maxLatdoubleThe maximum Latitude of bounding box of the deployments
location[].bbox.maxLondoubleThe maximum Longitude of the bounding box of the deployments
location[].bbox.minDepthdoubleThe minimum Depth of the deployments
location[].bbox.minLatdoubleThe minimum Latitude of bounding box of the deployments
location[].bbox.minLondoubleThe minimum Longitude of the bounding box of the deployments
location[].descriptionstringThe description of the location
location[].hasDeviceDatastringIndicates that data products can be requrested using a device category for the location
location[].londoubleThe average longitude of the deployments
location[].locationCodestringA code that unquely identifies a location
location[].hasPropertyDatastringIndicates that data products can be requested using property code for the location
location[].latdoubleThe average latitude of the deployments
location[].dataSearchUrlstringThe location specific Data Search web page URL
Example - Print all locations with 2 MHz ADCP deployed
o = onc('YOUR_TOKEN_HERE');

locations = o.getLocations({'deviceCategoryCode','ADCP2MHZ'});
for i=1:numel(locations)
    disp(locations(i));
end

 

getDevices()

getDevices(filters)

 

Returns a list of devices, as a list of struct objects that meet the criteria defined by the input filters.

 

Parameter
Type
Description
Example
Optional   
filtersstruct or array

A comman separated list of one or more filter criteria, used to return an exclusive set of device elements from the ONC devices endpoint.

struct('deviceCode','NORTEKADCP9917')

or

{'deviceCode','NORTEKADCP9917'}

 

Returns

A list of device objects

        cvTerm: [1×1 struct]
				device: [1×1 struct]
			           uri: 'http://vocab.nerc.ac.uk/collection/L22/current/TOOL0888/'
			    vocabulary: 'SeaVoX Device Catalogue'

    dataRating: [2×1 struct]
		        dateFrom: '2008-11-01T00:00:00.000Z'
		          dateTo: '2010-05-27T19:27:04.000Z'
		    samplePeriod: 1
		      sampleSize: 1
 
		        dateFrom: '2010-05-27T19:27:04.000Z'
		          dateTo: []
		    samplePeriod: 10
		      sampleSize: 1


    deviceCode: 'BC_POD1_AD2M'
      deviceId: 11302
    deviceLink: 'http://data.oceannetworks.ca/DeviceListing?DeviceId=11302'
    deviceName: 'Nortek Aquadopp HR-Profiler 2965'
Property
Type
Description
device[]listA list of devices
device[].cvTermdictionaryA dictionary of controlled vocabulary terms
device[].cvTerm.device[]dictionaryA list of dictionaries of controlled vocabulary term information for the device
device[].cvTerm.device[].uristringThe Uri of the controlled vocabulary term
device[].cvTerm.device[].vocabularystringThe name of the controlled vocabulary that the term belongs to
device[].dataRating[]listA list of data ratings dictionaries
device[].dataRating[].dateFromdatetime string

The starting datetime for a data rating

device[].dataRating[].dateTodatetime string

The ending datetime for a data rating.

  • If None, the data rating was still in affect when the request was made
  • Date Time format: yyyy-MM-dd'T'HH:mm:ss.SSS'Z'
  • datetime is represented in Coordinated Universal Time (UTC) 

device[].dataRating[].samplePerioddoubleThe sample period, in seconds for the data rating
device[].dataRating[].sampleSizeintThe size of the data sample for the data rating
device[].deviceCodestringA code that unquely identifes a device
device[].deviceLinkstringThe Url link to the Devices Listing page for the specific device
device[].deviceNamestringThe name of the device
Example - Print all devices deployed at Barkley Canyon - Axis, between June 1, 2016 and May 31, 2017
o = onc('YOUR_TOKEN_HERE');

devices = o.getDevices({'locationCode','BACAX', ... 				%BACAX - Barkley Canyon / Axis
						'deploymentBegin','2016-06-01T00:00:00.000Z', ...
						'deploymentEnd':'2017-05-31T23:59:59.999Z'});  

devices = o.devices({'deviceCategoryCode','ADCP2MHZ'});
for i=1:numel(devices)
    disp(devices(i));
end

 

getDeviceCategories()

getDeviceCategories(filters)

 

Returns a list of deviceCategories, as a list of struct objects that meet the criteria defined by the input filters.

 

Parameter
Type
Description
Example
Optional   
filtersstruct or array

A comman separated list of one or more filter criteria, used to return an exclusive set of deviceCategory elements from the ONC deviceCategories endpoint.

struct('deviceCategoryCode','HYDROPHONE')

or

{'deviceCategoryCode','HYDROPHONE'}

 

Returns

A list of device category objects.

                cvTerm: [1×1 struct]
			    deviceCategory: [1×1 struct]
						           uri: 'http://vocab.nerc.ac.uk/collection/L05/current/130/'
    						vocabulary: 'SeaDataNet device categories'

           description: 'Conductivity Temperature (and Depth Sensor)'
    deviceCategoryCode: 'CTD'
    deviceCategoryName: 'CTD'
         hasDeviceData: 'true'
       longDescription: ' Conductivity Temperature Depth (CTD) is an abbreviated name for an instrument package that contains sensors for measuring the conductivity, temperature, and pressure of seawater. Salinity, sound velocity, and density are variables derived from sensor measurements. CTDs can carry additional instruments and sensors such as oxygen sensors, turbidity sensors and fluorometers. '
Property
Type
Description
deviceCategory[]listA list of device categories
deviceCategory[].cvTermdictionaryA dictionary of controlled vocabulary terms
deviceCategory[].cvTerm.deviceCategory[]dictionaryA list of dictionaries of controlled vocabulary term information for the device category
deviceCategory[].cvTerm.deviceCategory[].uristringThe Uri of the controlled vocabulary term
deviceCategory[].cvTerm.deviceCategory[].vocabularystringThe name of the controlled vocabulary that the term belongs to
deviceCategory[].descriptionstringThe short description of the device category
deviceCategory[].deviceCategoryCodestringA code to uniquely identify a device category
deviceCategory[].deviceCategoryNamestringThe name of the device category
deviceCategory[].hasDeviceDatastringIndicates that data products can be requested using the device category for a location
deviceCategory[].longDescriptionstringThe long description of the device category
Example - Print all device categories available at Barkley Canyon - Upper Slope
o = ONC('YOUR_TOKEN_HERE');

deviceCategories = o.getDeviceCategories({'locationCode':'NCBC'});    #NCBC - Barkely Canyon / Upper Slope

for i=1:numel(deviceCategories)
    disp(deviceCategories(i));
end

 

getProperties()

getProperties(filters)

 

Returns a list of properties, as a list of struct objects that meet the criteria defined by the input filters.

 

Parameter
Type
Description
Example
Optional   
filtersstruct or array

A comman separated list of one or more filter criteria, used to return an exclusive set of properties elements from the ONC properties endpoint.

struct('propertyCode','seawatertemperature')

or

{'propertyCode','seawatertemperature'}

 

Returns

A list of property objects.

             cvTerm: [1×1 struct]
				    property: []
				         uom: [1×1 struct]
						           uri: 'http://vocab.nerc.ac.uk/collection/P06/current/UPDB/'
						    vocabulary: 'BODC data storage units'

        description: 'Pressure'
      hasDeviceData: 'true'
    hasPropertyData: 'true'
       propertyCode: 'pressure'
       propertyName: 'Pressure'
                uom: 'decibar'
Property
Type
Description
property[]listA list of properties
property[].cvTermdictionaryA dictionary of controlled vocabulary terms
property[].cvTerm.property[]dictionaryA list of dictionaries of controlled vocabulary term information for the property
property[].cvTerm.property[].uristringThe Uri of the controlled vocabulary term
property[].cvTerm.property[].vocabularystringThe name of the controlled vocabulary that the term belongs to
property[].cvTerm.uom[]dictionaryA list of dictionaries of controlled vocabulary term information for the unit of measure
property[].cvTerm.uom[].uristringThe Uri of the controlled vocabulary term
property[].cvTerm.uom[].vocabularystringThe name of the controlled vocabulary that the term belongs to
property[].descriptionstringThe short description of the device category
property[].hasDeviceDatastringIndicates that data products can be requrested using the property code, along with a device category for a location
property[].hasPropertyDatastringIndicates that data products can be requested using the property code for a location
property[].propertyCodestringA code that uniquely identifies the property
property[].propertyNamestringThe name of the property
property[].uomstringThe Unit of Measure that the property measurements are in.
Example - Print all properties that are available at Barkley Canyon - Axis
o = onc('YOUR_TOKEN_HERE');

properties = o.getProperties({'locationCode':'BACAX'});  #BACAX - Barkley Canyon / Axis

for i=1:numel(properties)
    disp(properties(i));
end

 

getDataProducts()

getDataProducts(filters)

 

Returns a list of data products as a list of struct objects that meet the criteria defined by the input filters.

 

Parameter
Type
Description
Example
Optional   
filtersstruct or array

A comman separated list of one or more filter criteria, used to return an exclusive set of data product & extension elements from the ONC dataProducts endpoint.

struct('dataProductCode','TSSD', ...
       'extension','json')
or
{'dataProductCode','TSSD', ...
 'extension','json'}

 

Returns

A list of data product objects.

    dataProductCode: 'TSSD'
    dataProductName: 'Time Series Scalar Data'
          extension: 'json'
      hasDeviceData: 1
    hasPropertyData: 1
       helpDocument: 'https://wiki.oceannetworks.ca/display/DP/1'
Property
Type
Description
dataProduct[]listA list of data products
dataProduct[].dataProductCodestringA code that uniquely identifies a data product
dataProduct[].dataProductNamestringThe name of the data product
dataProduct[].extensionstringThe extension for a data product
dataProduct[].hasDeviceDatastringIndicates that the data product can be requrested using a device category code and a location code
dataProduct[].hasPropertyDatastringIndicates that the data product can be requested using a property code and a location code or device code

 

 

Example - Print all MatLab data product
o = onc('YOUR_TOKEN_HERE');

dataProducts = o.getDataProducts({'extension','mat'}); %mat - MatLab

for i=1:numel(dataProducts)
    disp(dataProducts(i));
end

 

Data Product Download methods

orderDataProduct()

orderDataProduct(filters, maxRetries, downloadResultsOnly, includeMetadataFile)

 

Requests a data product and downloads the generated files to the class's outPath.

This method performs the complete data product download workflow by calling additional methods to perform the following steps:

 1. Request a data product, which includes estimates for download size and time.

 2. Run a requested data product, which kicks off a process on the task machine to generate the data product.

 3. Download the data product to disk. The process will continue to poll the web service to determine if the product is ready to download. If it is not ready, status messages will be provided. Once it is ready, it will download the data product to disk

 

Parameter
Type
Description
Example
Required   
filtersstruct or array

A comma separated list of filter criteria parameters, used to request data as a data product from the ONC dataProductDelivery endpoint.

struct('locationCode','BACAX',...
'deviceCategoryCode','ADCP2MHZ',...
'dataProductCode','TSSD',...
'extension','csv',...
'begin','2016-07-27T00:00:00.000Z',...
'end','2016-08-01T00:00:00.000Z',...
'dpo_qualityControl',1,...
'dpo_resample','none',...
'dpo_dataGaps',0)

or

{'locationCode','BACAX',...
'deviceCategoryCode','ADCP2MHZ',...
'dataProductCode','TSSD',...
'extension','csv',...
'begin','2016-07-27T00:00:00.000Z',...
'end','2016-08-01T00:00:00.000Z',...
'dpo_qualityControl',1,...
'dpo_resample','none',...
'dpo_dataGaps',0}

Option   
maxRetriesint

The number of times to retry the service before the function aborts.

  • If excluded, the default of 100 is used
 
downloadResultsOnlyboolean

Indicates if the files will be downloaded or if only the url to the file will be returned

  • false - Files are downloaded to the ONC class outPath property location
  • true - Files will not be downloaded. The download URLs are available from the url property for each downloadResult in downloadResults in the returned struct
  • If excluded, the default of false is used
 
includeMetadataFileboolean

Indicates if the metadata file associated with the data product request will be downloaded

  • false - Metadata file is not downloaded
  • true - Metadata file is downloaded
  • If excluded, the default of false is used.
 

 

Returns

A dictionary of download results and run time statistics

           url: "http://data.oceannetworks.ca/api/dataProductDelivery?method=download&token=b6ede000-1865-4ac3-94ad-e87d8bdfd307&dpRunId=4658962&index=1"
      messages: {1×2 cell}
        status: 'complete'
          size: 3060227
          file: 'c:\ONC\data\BarkleyCanyon_Axis_ADCP2MHz_20160727T000005Z_20160731T235958Z-clean.csv'
         index: 1
    downloaded: 1

           url: "http://data.oceannetworks.ca/api/dataProductDelivery?method=download&token=b6ede000-1865-4ac3-94ad-e87d8bdfd307&dpRunId=4658962&index=meta"
      messages: {}
        status: 'complete'
          size: 27504
          file: 'c:\ONC\data\BarkleyCanyon_Axis_ADCP2MHz_20160727T000005Z_20160731T235958Z-clean_CSV_META.pdf'
         index: 'meta'
    downloaded: 1
Property
Type
Description
downloadResultslistA cell array of download results struct objects.
downloadResults{}.urlstringThe URL used to make the download request.
downloadResults{}.messagelistA cell array of the messages that were returned from the dataProductDelivery download method when it was polled.
downloadResults{}.statusstring

The status of the download process at the request. Possible values are:

  • "running" - The data product request is running on the Task server.
  • "complete" - The data product request Task has completed and file has been downloaded or url has been returned.
  • "error" - An error, preventing the task execution or download from completing successfully, has occurred.
downloadResults{}.sizefloatThe actual size of the file in bytes.
downloadResults{}.filestringThe full path of the file that was downloaded.
downloadResults{}.indexstringThe index of the file downloaded. The index can either be a number 1-* or "meta" for the data product metadata file.
downloadResults{}.downloadedbooleanIndicates if the file was downloaded. If the filter downloadResultsOnly=True is used, the data product is not downloaded and the value will be False
Example - Download Time Series Scalar Data Product in CSV format for ADCP 2 MHZ at Barkley Canyon - Axis
o = onc('YOUR_TOKEN_HERE');

results = o.orderDataProduct(struct('locationCode','BACAX',...
                				    'deviceCategoryCode','ADCP2MHZ',...
                				    'dataProductCode','TSSD',...
                				    'extension','csv',...
                				    'begin','2016-07-27T00:00:00.000Z',...
                				    'end','2016-08-01T00:00:00.000Z',...
                				    'dpo_qualityControl',1,...
                				    'dpo_resample','none',...
                				    'dpo_dataGaps',0),100,false,true);
            
for i=1:numel(results)
    disp(results{i});
end

 

requestDataProduct()
requestDataProduct(filters)

 

Requests a data product using the dataProductDelivery service and provides request information so that the calling function can decide if should run the data product.
The struct of information it returns includes a requestID, which is used to run the data product, and estimates of the expected download times and file sizes.

 

Parameter
Type
Description
Example
Required   
filters

struct or array

A comma separated list of filter criteria parameters, used to request data as a data product from the ONC dataProductDelivery endpoint.

struct('locationCode','BACAX',...
'deviceCategoryCode','ADCP2MHZ',...
'dataProductCode','TSSD',...
'extension','csv',...
'begin','2016-07-27T00:00:00.000Z',...
'end','2016-08-01T00:00:00.000Z',...
'dpo_qualityControl',1,...
'dpo_resample','none',...
'dpo_dataGaps',0)
or
{'locationCode','BACAX', ...
'deviceCategoryCode','ADCP2MHZ', ...
'dataProductCode','TSSD', ...
'extension','csv', ...
'begin','2016-07-27T00:00:00.000Z', ...
'end','2016-08-01T00:00:00.000Z', ...
'dpo_qualityControl',1, ...
'dpo_resample','none', ...
'dpo_dataGaps',0}

 

Returns

returns a data product request struct

for archived files (AD):

           dpRequestId: 2046404
    compressedFileSize: 12563408
              fileSize: 70766230
              numFiles: 4
         downloadTimes: {1x3 cell}
 
 
                         10Mbps: 7.076623
                         50Mbps: 1.4153247
                         150Mbps: 0.47177488

 

for generated files:

                dpRequestId: 2046404
          estimatedFileSize: 8348187
    estimatedProcessingTime: 16
Property
Type
Description
Example
Always contains
dpRequestIdintA unique id for a data product request
dpRequestId: 2046404
May contain
compressedFileSizeintThe compressed size of the known file(s) in bytes.compressedFileSize: 12563408
fileSizeintThe size of known file(s) in bytes.fileSize: 70766230
numFilesintThe number of files.numFiles: 4
downloadTimescell arrayA dictionary of estimated download times
downloadTimes: {1x3 cell}
              10Mbps: 7.076623
              50Mbps: 1.4153247
              150Mbps: 0.47177488
estimatedFileSizeintThe estimated file size of generated data productestimatedFileSize: 8348187
estimatedProcessingTimeintThe estimated time, in seconds, that it will take to run the data product request.estimatedProcessingTime: 16
Example - Download Time Series Scalar Data Product in CSV format for ADCP 2 MHZ at Barkley Canyon - Axis
o = onc('YOUR_TOKEN_HERE');

requestInfo = o.requestDataProduct(struct('locationCode','BACAX',...
                				          'deviceCategoryCode','ADCP2MHZ',...
                				          'dataProductCode','TSSD',...
                				          'extension','csv',...
                				          'begin','2016-07-27T00:00:00.000Z',...
                				          'end','2016-08-01T00:00:00.000Z',...
                				          'dpo_qualityControl',1,...
                				          'dpo_resample','none',...
                				          'dpo_dataGaps',0));
            
disp(requestInfo);

 

runDataProduct()
runDataProduct(requestId)

 

Runs a data product request using the dataProductDelivery service with a Request Id and returns a list of run ids

 

Parameter
Type
Description
Example
Required   
requestIdint

A dataProductDelivery dpRequestId, which is returned by the dataProductDelivery service request method, or as the 'dpRequestId' property in the dpRequest returned by the requestDataProduct() method.

2046404

 

Returns

a list of runIds

    dpRunId: 4659102
     status: 'data product running'
Example - Run a requested data product
o = onc('YOUR_TOKEN_HERE');

runs = o.runDataProduct(YOUR_REQUEST_ID_HERE);
 
disp(runs);

 

downloadDataProduct()
downloadDataProduct(runId, fileCount, estimatedProcessingTime, maxRetries, downloadResultsOnly, includeMetadataFile)

 

Polls the dataProductDelivery service until data product generation task is complete and downloads all of the files and/or retrieves the file information (url, file and download status).

 

Parameter
Type
Description
Example
Required   
runIdint

A dataProductDelivery dpRunId, which is returned by the dataProductDelivery service run method, or as the dpRunId property in the list returned by the runDataProduct() method.

4629218
Option   
fileCountint

The actual or estimated file count, which is returned from the dataProductDelivery request method

  • If there is no estimated file count, use 0.
  • If excluded, the default of 0 is used.
1
estimatedProcessingTimeint

The estimated time in seconds, that it should take to run the request on the task machine.

  • Used to determine how often the dataProductDelivery service is called to see if the task processing is complete.
  • Estimated processing time is provided by the dataProductDelivery request method
2
maxRetriesint

The number of times to retry the service before the function aborts.

  • If excluded, the default of 100 is used
10
downloadResultsOnlyboolean

Determines if the files will be downloaded or if only the url to the file will be returned

  • false - Files are downloaded to the ONC class outPath property location
  • true - Files will not be downloaded. The download URLs are available from the url property for each downloadResult in downloadResults in the returned struct
  • If excluded, the default of false is used
true
includeMetadataFileboolean

Indicates if the metadata file associated with the data product request will be downloaded

  • false - Metadata file is not downloaded
  • true - Metadata file is downloaded
  • If excluded, the default of false is used.

true

 

Returns

returns a list of download results struct objects 

           url: "http://data.oceannetworks.ca/api/dataProductDelivery?method=download&token=<YOUR_TOKEN>&dpRunId=<YOUR_RUN_ID>&index=1"
      messages: {1×2 cell}
        status: 'complete'
          size: 3060227
          file: 'c:\ONC\data\BarkleyCanyon_Axis_ADCP2MHz_20160727T000005Z_20160731T235958Z-clean.csv'
         index: 1
    downloaded: 1

           url: "http://data.oceannetworks.ca/api/dataProductDelivery?method=download&token=<YOUR_TOKEN>&dpRunId=<YOUR_RUN_ID>&index=meta"
      messages: {}
        status: 'complete'
          size: 27504
          file: 'c:\ONC\data\BarkleyCanyon_Axis_ADCP2MHz_20160727T000005Z_20160731T235958Z-clean_CSV_META.pdf'
         index: 'meta'
    downloaded: 1
Property
Type
Description
urlstringThe URL used to make the download request.
messagelistA list of the messages that were returned from the dataProductDelivery download method when it was polled.
statusstring

The status of the download process at the request. Possible values are:

  • "running" - The data product request is running on the Task server.
  • "complete" - The data product request Task has completed and file has been downloaded or url has been returned.
  • "error" - An error, preventing the task execution or download from completing successfully, has occurred.
sizefloatThe actual size of the file in bytes.
filestringThe full path of the file that was downloaded.
indexstringThe index of the file downloaded. The index can either be a number 1-* or "meta" for the data product metadata file.
downloadedbooleanIndicates if the file was downloaded. If the filter downloadResultsOnly=True is used, the data product is not downloaded and the value will be False
Example - Download the results of requested data product run
o = onc('YOUR_TOKEN_HERE');

results = o.downloadDataProduct(YOUR_RUN_ID_HERE, 1);
 
for i=1:numel(results)
    disp(results{i});
end

 

getDataProductUrls()

getDataProductUrls(filters, maxRetries=100)

 

Orders a data product and return only the urls, for download at a later time. Urls can be used with the downloadFile() or getJsonFromUrl() methods

 

Parameter
Type
Description
Example
Required   
filtersstruct or array

A comma separated list of filter criteria parameters, used to request data as a data product from the ONC dataProductDelivery endpoint.

struct('locationCode','BACAX',...
'deviceCategoryCode','ADCP2MHZ',...
'dataProductCode','TSSD',...
'extension','csv',...
'begin','2016-07-27T00:00:00.000Z',...
'end','2016-08-01T00:00:00.000Z',...
'dpo_qualityControl',1,...
'dpo_resample','none',...
'dpo_dataGaps',0)
or
{'locationCode','BACAX', ...
'deviceCategoryCode','ADCP2MHZ', ...
'dataProductCode','TSSD', ...
'extension','csv', ...
'begin','2016-07-27T00:00:00.000Z', ...
'end','2016-08-01T00:00:00.000Z', ...
'dpo_qualityControl',1, ...
'dpo_resample','none', ...
'dpo_dataGaps',0}
Option   
maxRetriesint

The number of times to retry the service before the function aborts.

  • If excluded, the default of 100 is used
10

 

Returns

returns a list of urls

["http://data.oceannetworks.ca/api/dataProductDelivery?method=download&token=<YOUR_TOKEN>&dpRunId=<YOUR_RUN_ID>&index=1"]
Example - Order a data product and get a list of the download urls
o = onc('YOUR_TOKEN_HERE');
 
urls = o.getDataProductUrls(struct('locationCode','BACAX',...
							       'deviceCategoryCode','ADCP2MHZ',...
                                   'dataProductCode','TSSD',...
							       'extension','json',...
							       'begin','2016-07-27T00:00:00.000Z',...
							       'end','2016-08-01T00:00:00.000Z',...
							       'dpo_qualityControl',1,...
							       'dpo_resample','none',...
							       'dpo_dataGaps',0,...
							       'dpo_jsonOutputEncoding','OM'),100);


for i=1:numel(urls)
    disp(urls{i});
end

 

downloadFile()

downloadFile(url, file, multiThreadMessages=False)

 

Downloads a file from a URL, write it to a file and return download results information (url, file, message and download status).

 

Parameter
Type
Description
Example
Required   
urlstring

The url to be downloaded

http://data.oceannetworks.ca/api/dataProductDelivery?method=download&token=<YOUR_TOKEN_HERE>
&dpRunId=<YOUR_RUN_ID_HERE>&index=<YOUR_INDEX_HERE>
filestring

The full path of the file to be downloaded to.

  • If excluded, the file name from the content disposition in the header is used, along with the onc class object outPath to create a full file name path.
c:/temp/myDownload.csv
Option   
multiThreadMessagesboolean

Determines how the method and called method should print messages to the console. If downloading data products in a multi-threaded pattern, messages written to the console can overlap and progress dots can be written out of context

  • If excluded, the default of False is used.
True

 

returns a downloadResults dictionary

           url: "http://data.oceannetworks.ca/api/dataProductDelivery?method=download&token=b6ede000-1865-4ac3-94ad-e87d8bdfd307&dpRunId=4722272&index=1"
      messages: {}
          file: 'c:\ONC\data\matlab\myDownload.json'
    downloaded: 1
        status: 'complete'
Property
Type
Description
urlstringThe URL used to make the download request.
filestringThe full path of the file that was downloaded.
messagelistA list of the messages that were returned from the dataProductDelivery download method when it was polled.
downloadedbooleanIndicates if the file was downloaded.
statusstring

The status of the download process at the request. Possible values are:

  • "running" - The data product request is running on the Task server.
  • "complete" - The data product request Task has completed and file has been downloaded or url has been returned.
  • "error" - An error, preventing the task execution or download from completing successfully, has occurred.
Example - Download a file from a url
o = onc('YOUR_TOKEN_HERE');
 
url = 'http://data.oceannetworks.ca/api/dataProductDelivery?method=download&token=<YOUR_TOKEN_HERE>&dpRunId=<YOUR_RUN_ID_HERE>&index=<YOUR_INDEX_HERE>';
file = 'c:/temp/myDownload.csv';

downloadResult = o.downloadFile(url, file);
 
disp(downloadResult);

 

decodeJsonFromUrl()

decodeJsonFromUrl(url)

 

Returns a dictionary from the JSON returned from a URL

 

Parameter
Type
Description
Example
Required   
urlstring

The url to a JSON data product delivery result

http://data.oceannetworks.ca/api/dataProductDelivery?method=download&token=<YOUR_TOKEN_HERE>&dpRunId=<YOUR_RUN_ID_HERE>&index=<YOUR_INDEX_HERE>
Example - Get dictionary of results from json data product request
o = onc('YOUR_TOKEN_HERE');
 
urls = o.getDataProductUrls(struct('locationCode','BACAX',...
							       'deviceCategoryCode','ADCP2MHZ',...
                                   'dataProductCode','TSSD',...
							       'extension','json',...
							       'begin','2016-07-27T00:00:00.000Z',...
							       'end','2016-08-01T00:00:00.000Z',...
							       'dpo_qualityControl',1,...
							       'dpo_resample','none',...
							       'dpo_dataGaps',0,...
							       'dpo_jsonOutputEncoding','OM'),100);
 
for i=1:numel(urls)
	object = o.decodeJsonFromUrl(url);
 	disp(object);
end

 

Near Real-Time data product methods

getDirectScalar()

getDirectScalar(filters, outputFormat, metadata, rowLimit)

 

Returns scalar data, in JSON Object or Array format, in the response payload that meet the criteria defined by the input filters.

 

Parameter
Type
Description
Example
Required   
filtersstruct or array

A comma separated list of filter criteria parameters, used to request data and metadata from the scalardata endpoint.

struct('locationCode','TWDP', ...

'deviceCategoryCode','TSG', ...
'begin','2016-09-01T00:00:00.000Z', ...
'end','2016-09-01T08:00:00.000Z')

or

{'locationCode','TWDP', ...
'deviceCategoryCode','TSG', ...
'begin','2016-09-01T00:00:00.000Z', ...
'end','2016-09-01T08:00:00.000Z'}

Option   
outputFormatstring

The encoding of the response output.

  • Object - Data is a list of dictionaries with sampleTime, value and qaqcFlag for a given sensor.
  • Array - Data has 3 arrays for a given sensor. Array of sampleTime, array of values and array of qaqcFlags
  • If excluded, the default 'Object' is used.
"outputFormat","Array"
metadatastring

The amount of metadata detail that is included in the response output.

  • Minimum - Provides only basic property information. Response includes serviceMetadata dictionary only.
  • Full - Provides all property information. Response includes both metadata and serviceMetadata dictionaries.
  • If excluded, the default 'Minimum' is used.
"metadata","Full"
rowLimitinteger

The maximum number of rows of scalar data that will be returned for each sensor

  • Maximum is 100,000
  • If excluded, the default 100,000 is used
"rowLimit",5000

 

Returns

A structure that contains sensor data that matches the filter criteria defined in the input parameters, metadata and service metadata.

 metadata: [1×1 struct]
 
                  bbox: [1×1 struct]
				    	maxDepth: 3
					      maxLat: 49.1139
					      maxLon: -123.5264
					    minDepth: 3
					      minLat: 49.1139
				      	  minLon: -123.5264
                 depth: 3
    deviceCategoryName: 'TSG'
              latitude: 49.1139
             longitude: -123.5264
                 begin: '2016-09-01T00:00:00.000Z'
                   end: '2016-09-01T00:00:30.000Z'
          locationCode: 'TWDP'
 
sensorData: [3×1 struct]
 
        actualSamples: 1
                 data: [1×1 struct]
     				 qaqcFlag: 1
    			   sampleTime: '2016-09-01T00:00:07.037Z'
                        value: 3.4429
               sensor: 'Conductivity'
           sensorName: 'Conductivity'
        unitOfMeasure: 'S/m'
 
 
serviceMetadata: [1×1 struct]
              metadata: 'Full'
          nextDateFrom: '2016-09-01T00:00:17.041Z'
          outputFormat: 'Object'
              rowLimit: 100000
               sensors: []
    totalActualSamples: 3
                 begin: '2016-09-01T00:00:00.000Z'
                   end: '2016-09-01T00:00:10.000Z'
    deviceCategoryCode: 'TSG'
          locationCode: 'TWDP'
Property
Type
Description
metadatastructA structure of metadata, if metadata='Full' input parameter is used
metadata.bboxstructA Bounding Box structure
metadata.bbox.maxDepthfloatThe maximum depth of the instruments contributing to the results
metadata.bbox.maxLatfloatThe maximum Latitude of bounding box of the instruments contributing to the results
metadata.bbox.maxLonfloatThe maximum Longitude of the bounding box of the instruments contributing to the results
metadata.bbox.minDepthfloatThe minimum Depth of the instruments contributing to the results
metadata.bbox.minLatfloatThe minimum Latitude of bounding box of the instruments contributing to the results
metadata.bbox.minLonfloatThe minimum Longitude of the bounding box of the instruments contributing to the results
sensorDatalistA list of sensor dictionaries
sensorData[].acutalSamplesintThe number of samples in the data list
sensorData[].datalistA list of observation structures
sensorData[].data[].qaqcFlagintThe QAQC Flag for the observation
sensorData[].data[].sampleTimedatetime string

The recorded time of the observation

sensorData[].data[].valuefloatThe value of the observation
sensorData[].sensorstringThe code of the sensor that is being observed
sensorData[].sensorNamestringThe name of the sensor that is being observed
sensorData[].unitOfMeasurestringThe unit of measure of the observation
serviceMetadatastructA dictionary of the metadata for the service request
serviceMetadata.metadatastringThe requested metadata filter
serviceMetadata.nextDataFromdatetime stringThe begin date for next reading in yyyy-MM-ddTHH:mm:ss.SSSZ format
serviceMetadata.outputFormatstringThe requested JSON output format
serviceMetadata.rowLimitintThe requested row limit per sensor
serviceMetadata.sensorsstringA comma separated list of the requested sensors
serviceMetadata.totalActualSamplesintThe number of observations expected for all of the sensors
serviceMetadata.begindatetime stringThe begin date of the request in yyyy-MM-ddTHH:mm:ss.SSSZ format
serviceMetadata.enddatetime stringThe end date of the request in yyyy-MM-ddTHH:mm:ss.SSSZ format
serviceMetadata.deviceCategoryCodestringThe Device Category Code of the request. See Available Device Categories for further details
serviceMetadata.locationCodestringThe Location Code of the request. See Available Locations for further details.
Example - Print the last Thermosalinograph reading from Tswwassen - Duke Point Ferry
o = onc('YOUR_TOKEN_HERE');
 
result = o.getDirectScalar({'locationCode','TWDP','deviceCategoryCode','TSG'});

disp(result);
Example - Print 1 hour of Thermosalinograph readings from Tswwassen - Duke Point Ferry
o = onc('YOUR_TOKEN_HERE');
 
result = o.getDirectScalar(struct('locationCode','TWDP','deviceCategoryCode','TSG', 'begin','2016-09-01T00:00:00.000Z', 'end','2016-09-01T01:00:00.000Z'));

disp(result);
Example - Print 1 hour of Thermosalinograph readings from Tswwassen - Duke Point Ferry as an Array
o = onc('YOUR_TOKEN_HERE');
 
result = o.getDirectScalar(struct('locationCode':'TWDP','deviceCategoryCode':'TSG', 'begin':'2016-09-01T00:00:00.000Z', 'end':'2016-09-01T01:00:00.000Z'),'Array');

disp(result);

 

getDirectRaw()

getDirectRaw(filters, rowLimit)

 

Returns raw data from an instrument in the payload, in JSON format that meet the criteria defined by the input filters.

 

Parameter
Type
Description
Example
Required   
filter

struct

or

cell array

A comma separated list of filter criteria parameters, used to request data and metadata from the rawdata endpoint.

struct('locationCode','TWDP', ...
'deviceCategoryCode','TSG', ...
'begin','2016-09-01T00:00:00.000Z', ...
'end','2016-09-01T08:00:00.000Z')

or

{'locationCode','TWDP', ...
'deviceCategoryCode','TSG', ...
'begin','2016-09-01T00:00:00.000Z', ...
'end','2016-09-01T08:00:00.000Z'}

Option   
rowLimitinteger

The maximum number of rows of raw data that will be returned

  • Maximum is 100,000
  • If excluded, the default 100,000 is used
"rowLimit",5000

 

Returns

A structure that contains sensor data that matches the filter criteria defined in the input parameters, metadata and service metadata.

        data: [1×1 struct]
       		   rawData: '20160901-00:00:07.037, 12.9162,  3.44294,  28.9096'
    		sampleTime: '2016-09-01T00:00:07.037Z'

    metadata: [1×1 struct]
	     dataMetadata: [1×1 struct]
		        firstTimestamp: '2016-09-01T00:00:07.037Z'
		    hasMoreDataInRange: 0
		         lastTimestamp: '2016-09-01T00:00:07.037Z'
		          numberOfData: 1
		          locationName: 'Tsawwassen - Duke Point'

	    queryMetadata: [1×1 struct]
		              rowLimit: 100000
		                 begin: '2016-09-01T00:00:00.000Z'
		                   end: '2016-09-01T00:00:10.000Z'
		    deviceCategoryCode: 'TSG'
		          locationCode: 'TWDP'
Property
Type
Description
datastruct 
data[].rawDatastringA raw data string representing a sample at a specific time
data[].sampleTimedatetime string

The recorded time of the observation

metadatastructA structure containing metadata
metadata.dataMetadatastructA structure containing metadata about the data
metadata.dataMetadata.firstTimestampdatetime stringThe date time of the first record in yyyy-MM-ddTHH:mm:ss.SSSZ format
metadata.dataMetadata.hasMoreDataInRangeint

If the there is more data in the range of the request. For example if 1,000 records are returned (because the rowLimit=1000) but there should be 200,000 rows as defined by the filter criteria.

  • 0 = False
  • 1 = True
metadata.dataMetadata.lastTimestampdatetime stringThe date time of the last record in yyyy-MM-ddTHH:mm:ss.SSSZ format
metadata.dataMetadata.numberOfDataintThe number of samples in the data list
metadata.dataMetadata.locationNamestringThe full name of the location
metadata.queryMetadatastructA structure containing metadata about the data request (query)
metadata.queryMetadata.rowLimitintThe requested row limit per sensor
metadata.queryMetadata.begindatetime stringThe begin date of the request in yyyy-MM-ddTHH:mm:ss.SSSZ format
metadata.queryMetadata.enddatetime stringThe end date of the request in yyyy-MM-ddTHH:mm:ss.SSSZ format
metadata.queryMetadata.deviceCategoryCodestringThe Device Category Code of the request. See Available Device Categories for further details
metadata.queryMetadata.locationCodestringThe Location Code of the request. See Available Locations for further details.
Example - Print the last raw instrument reading from the CTD at Barkley Canyon Axis
o = onc('YOUR_TOKEN_HERE');
 
payload = o.getDirectRaw(struct('locationCode','BACAX','deviceCategoryCode','CTD'),1);

disp(payload);
Example - Print the last raw instrument reading from AIS Reviever at the Underwater Listening Station at IONA
o = onc('YOUR_TOKEN_HERE');
 
payload = o.getDirectRaw(struct('locationCode','IONA','deviceCategoryCode','AISRECEIVER'),1);

disp(payload);
Example - Print 1 hour of raw CTD readings from Barkley Canyon Axis
o = onc('YOUR_TOKEN_HERE');
 
payload = o.getDirectRaw(struct('locationCode','BACAX','deviceCategoryCode','CTD','begin','2017-05-23T00:00:00.000Z','end','2017-05-23T01:00:00.000Z'));

disp(payload);
  • No labels