The ONC MatLab client library contains a number of classes and functions that make access to the ONC data, through the Oceans 2.0 API, quick and easy to use.
Requires
Installation
The ONC client library is a Add-On Toolbox that can be installed in MatLab by performing the following
History
version | Date | Description | Toolbox |
---|---|---|---|
1.0 | 6/28/2017 | Beta version |
Classes
class | description |
---|---|
onc | A wrapper class for access to Oceans 2.0 API web services |
ONC Class
Description
This class provides a wrapper for some for the most common Oceans 2.0 API requests, including the discovery services (locations, devices, deviceCategories, properties) and delivery (dataProductDelivery, scalardata and rawdata).
constructor
onc(token, production, showInfo, outPath, timeout)
Parameter | Type | Description | Example |
---|---|---|---|
Required | |||
token | string | All Web Services require a token. This can be generated at http://dmas.uvic.ca/Profile. Click on the "Web Services" tab and click "Generate Token" | token=5ba514e8-1203-428c-8835-879c8173e387 |
Optional | |||
production | boolean | Indicates if the ONC Production server URL is used for service requests.
| true or false |
showInfo | boolean | Indicates if verbose script messages are displayed, such as request url and processing time information.
| true or false |
outPath | string | The path that downloaded files are saved to.
| 'c:/ONC/Download' |
timeout | int | The web service request timeout in seconds
| 30 |
Usage
o = onc('YOUR_TOKEN_HERE')
Or
isProduction = true showInfo = false outPath = 'c:/ONC/Data' timeout = 30 o = onc('YOUR_TOKEN_HERE',isProduction,showInfo,outPath,timeout)
methods
Discovery methods
getLocations(parameters)
Returns a list of locations, as a list of struct objects.
Parameter | Type | Description | Example |
---|---|---|---|
Optional | |||
parameters | list | A comma separated list of filter parameters, used to filter data from the ONC devices endpoint.
| {'locationCode','BACAX'} |
getDevices(parameters)
Returns a list of devices, as a list of struct objects.
Parameter | Type | Description | Example |
---|---|---|---|
Optional | |||
parameters | list | A comma separated list of filter parameters, used to filter data from the ONC devices endpoint.
| {'deviceCode','NORTEKADCP9917'} |
getDeviceCategories(parameters)
Returns a list of deviceCategories, as a list of struct objects.
Parameter | Type | Description | Example |
---|---|---|---|
Optional | |||
parameters | list | A comma separated list of filter parameters, used to filter data from the ONC deviceCategories endpoint.
| {'deviceCategoryCode','HYDROPHONE'} |
getProperties(parameters)
Returns a list of properties, as a list of struct objects.
Parameter | Type | Description | Example |
---|---|---|---|
Optional | |||
parameters | list | A comma separated list of filter parameters, used to filter data from the ONC properties endpoint.
| {'propertyCode','seawatertemperature'} |
getDataProducts(parameters)
Returns a list of data products as a list of struct objects.
Parameter | Type | Description | Example |
---|---|---|---|
Optional | |||
parameters | dictionary | A dictionary of parameters, used to filter data from the ONC dataProducts endpoint.
| {'dataProductCode','TSSD', ... 'extension','json'} |
Data Product Download methods
orderDataProduct(parameters, maxRetries, downloadResultsOnly, includeMetadataFile)
Requests a data product and downloads the generated files to the class's outPath.
This method performs the complete data product download workflow by calling additional methods to perform the following steps:
1. Request a data product, which includes estimates for download size and time.
2. Run a requested data product, which kicks off a process on the task machine to generate the data product.
3. Download the data product to disk. The process will continue to poll the web service to determine if the product is ready to download. If it is not ready, status messages will be provided. Once it is ready, it will download the data product to disk
Parameter | Type | Description | Example |
---|---|---|---|
Required | |||
parameters | list | A comma separated list of filter parameters, used to request data as a data product from the ONC dataProductDelivery endpoint.
| {'locationCode','BACAX', ... |
Option | |||
maxRetries | int | The number of times to retry the service before the function aborts.
| 10 |
downloadResultsOnly | boolean | Indicates if the files will be downloaded or if only the url to the file will be returned
| true |
includeMetadataFile | boolean | Indicates if the metadata file associated with the data product request will be downloaded
| true |
Returns
A dictionary of download results and run time statistics
url: "http://qaweb2.neptune.uvic.ca/api/dataProductDelivery?method=download&token=b6ede000-1865-4ac3-94ad-e87d8bdfd307&dpRunId=4658962&index=1" messages: {1×2 cell} status: 'complete' size: 3060227 file: '\\142.104.222.56\c$\ONC\data\BarkleyCanyon_Axis_ADCP2MHz_20160727T000005Z_20160731T235958Z-clean.csv' index: 1 downloaded: 1 url: "http://qaweb2.neptune.uvic.ca/api/dataProductDelivery?method=download&token=b6ede000-1865-4ac3-94ad-e87d8bdfd307&dpRunId=4658962&index=meta" messages: {} status: 'complete' size: 27504 file: '\\142.104.222.56\c$\ONC\data\BarkleyCanyon_Axis_ADCP2MHz_20160727T000005Z_20160731T235958Z-clean_CSV_META.pdf' index: 'meta' downloaded: 1
Property | Type | Description |
---|---|---|
downloadResults | list | A cell array of download results struct objects. |
downloadResults{}.url | string | The URL used to make the download request. |
downloadResults{}.message | list | A cell array of the messages that were returned from the dataProductDelivery download method when it was polled. |
downloadResults{}.status | string | The status of the download process at the request. Possible values are:
|
downloadResults{}.size | float | The actual size of the file in bytes. |
downloadResults{}.file | string | The full path of the file that was downloaded. |
downloadResults{}.index | string | The index of the file downloaded. The index can either be a number 1-* or "meta" for the data product metadata file. |
downloadResults{}.downloaded | boolean | Indicates if the file was downloaded. If the filter downloadResultsOnly=True is used, the data product is not downloaded and the value will be False |
requestDataProduct(parameters)
Requests a data product using the dataProductDelivery service and provides request information so that the calling function can decide if should run the data product.
The struct of information it returns includes a requestID, which is used to run the data product, and estimates of the expected download times and file sizes.
Parameter | Type | Description | Example |
---|---|---|---|
Required | |||
parameters | dictionary | A comma separated list of filter parameters, used to request data as a data product from the ONC dataProductDelivery endpoint.
| {'locationCode','BACAX', ... |
Returns
returns a data product request struct
for archived files (AD):
dpRequestId: 2046404 compressedFileSize: 12563408 fileSize: 70766230 numFiles: 4 downloadTimes: {1x3 cell} 10Mbps: 7.076623 50Mbps: 1.4153247 150Mbps: 0.47177488
for generated files:
dpRequestId: 2046404 estimatedFileSize: 8348187 estimatedProcessingTime: 16
Property | Type | Description | Example |
---|---|---|---|
Always contains | |||
dpRequestId | int | A unique id for a data product request |
|
May contain | |||
compressedFileSize | int | The compressed size of the known file(s) in bytes. | compressedFileSize : 12563408 |
fileSize | int | The size of known file(s) in bytes. | fileSize : 70766230 |
numFiles | int | The number of files. | numFiles : 4 |
downloadTimes | cell list | A dictionary of estimated download times | downloadTimes: {1x3 cell} 10Mbps: 7.076623 50Mbps: 1.4153247 150Mbps: 0.47177488 |
estimatedFileSize | int | The estimated file size of generated data product | estimatedFileSize: 8348187 |
estimatedProcessingTime | int | The estimated time, in seconds, that it will take to run the data product request. | estimatedProcessingTime: 16 |
runDataProduct(requestId)
Runs a data product request using the dataProductDelivery service with a Request Id and returns a list of run ids
Parameter | Type | Description | Example |
---|---|---|---|
Required | |||
requestId | int | A dataProductDelivery dpRequestId, which is returned by the dataProductDelivery service request method, or as the 'dpRequestId' property in the dpRequest returned by the requestDataProduct() method. | 2046404 |
Returns
a list of runIds
dpRunId: 4659102 status: 'data product running'
downloadDataProduct(runId, fileCount, estimatedProcessingTime, maxRetries, downloadResultsOnly, includeMetadataFile)
Polls the dataProductDelivery service until data product generation task is complete and downloads all of the files and/or retrieves the file information (url, file and download status).
Parameter | Type | Description | Example |
---|---|---|---|
Required | |||
runId | int | A dataProductDelivery dpRunId, which is returned by the dataProductDelivery service run method, or as the dpRunId property in the list returned by the runDataProduct() method. | 4629218 |
Option | |||
fileCount | int | The actual or estimated file count, which is returned from the dataProductDelivery request method
| 1 |
estimatedProcessingTime | int | The estimated time in seconds, that it should take to run the request on the task machine.
| 2 |
maxRetries | int | The number of times to retry the service before the function aborts.
| 10 |
downloadResultsOnly | boolean | Determines if the files will be downloaded or if only the url to the file will be returned
| true |
includeMetadataFile | boolean | Indicates if the metadata file associated with the data product request will be downloaded
| true |
Returns
returns a list of download results struct objects
url: "http://qaweb2.neptune.uvic.ca/api/dataProductDelivery?method=download&token=b6ede000-1865-4ac3-94ad-e87d8bdfd307&dpRunId=4658962&index=1" messages: {1×2 cell} status: 'complete' size: 3060227 file: '\\142.104.222.56\c$\ONC\data\BarkleyCanyon_Axis_ADCP2MHz_20160727T000005Z_20160731T235958Z-clean.csv' index: 1 downloaded: 1 url: "http://qaweb2.neptune.uvic.ca/api/dataProductDelivery?method=download&token=b6ede000-1865-4ac3-94ad-e87d8bdfd307&dpRunId=4658962&index=meta" messages: {} status: 'complete' size: 27504 file: '\\142.104.222.56\c$\ONC\data\BarkleyCanyon_Axis_ADCP2MHz_20160727T000005Z_20160731T235958Z-clean_CSV_META.pdf' index: 'meta' downloaded: 1
Property | Type | Description |
---|---|---|
url | string | The URL used to make the download request. |
message | list | A list of the messages that were returned from the dataProductDelivery download method when it was polled. |
status | string | The status of the download process at the request. Possible values are:
|
size | float | The actual size of the file in bytes. |
file | string | The full path of the file that was downloaded. |
index | string | The index of the file downloaded. The index can either be a number 1-* or "meta" for the data product metadata file. |
downloaded | boolean | Indicates if the file was downloaded. If the filter downloadResultsOnly=True is used, the data product is not downloaded and the value will be False |