The ONC Python client library contains a number of classes and functions that make access to the ONC data, through the Oceans 2.0 API, quick and easy to use.

Installation

The ONC Python client library can be installed using the pip command found in the Scripts folder of your python install.

$ pip install onc

History

version	Date	Description	Package
1.0	6/9/2017	Beta version	onc-1.0.tar.gz
1.1	6/12/2016	Rename onc.dap.DAP class to onc.dap.ERDDAP	onc-1.1.tar.gz
1.11	6/13/2016	Remove unused numpy dependency. Added python-dateutil dependency to package	onc-1.11.tar.gz

Classes

class	import	description
ONC	from onc.onc import ONC	A wrapper class for access to Oceans 2.0 API web services
SOS	from onc.sos import SOS	A wrapper class for access to ONC's implementation of Sensor Observation Service (SOS)
ERDDAP	from onc.dap import ERDDAP	A wrapper class for access to datasets published on ONC's ERDDAP (OPeNDAP) server http://qadap.onc.uvic.ca/erddap/index.html
NERC	from onc.nerc import NERC	A wrapper class for access to controlled vocabularies stored on the NERC vocabulary server http://vocab.nerc.ac.uk/

ONC Class

Description

This class provides a wrapper for some for the most common Oceans 2.0 API requests, including the discovery services (locations, devices, deviceCategories, properties) and delivery (dataProductDelivery, scalardata and rawdata).

constructor

ONC(token, production=True, showInfo=False, outPath='c:/temp')

Parameter	Type	Description	Example
Required
token	string	All Web Services require a token. This can be generated at http://dmas.uvic.ca/Profile. Click on the "Web Services" tab and click "Generate Token"	token=5ba514e8-1203-428c-8835-879c8173e387
Optional
production	boolean	Indicates if the ONC Production server URL is used for service requests. Default is True. False is intended for internal ONC testing only. All external users should use False or leave blank.	True or False
showInfo	boolean	Indicates if verbose script messages are displayed, such as request url and processing time information. Default is False. True is intended for script debugging.	True or False
outPath	string	The path that downloaded files are saved to. Default is 'c:/Temp'. Full path will be created if it does not already exist.	'c:/ONC/Download'

Usage

from onc.onc import ONC
onc = ONC('YOUR_TOKEN_HERE')

Or

from onc.onc import ONC

isProduction = True
showInfo = False
outPath = 'c:/ONC/Data'
onc = ONC('YOUR_TOKEN_HERE',isProduction,showInfo,outPath)

methods

getLocations(parameters={})

Returns a list of locations, as a list of dictionary objects.

Parameter

Type

Description

Example

Optional

parameters

dictionary

A dictionary of parameters, used to filter data from the ONC devices endpoint.

If excluded, all locations are returned.
See locations Discovery Service for available parameters (filters).

{'locationCode':'BACAX'}

Example - Print all locations in North East Pacific with Hydrophones deployed

from onc.onc import ONC
onc = ONC('YOUR_TOKEN_HERE')

locations = onc.getLocations({'locationCode':'NEP', 'deviceCategoryCode':'HYDROPHONE'}) #NEP - North East Pacific
print('Locations')

for location in locations:
    locationCode = location['locationCode']
    locationName = location['locationName']
    hasDeviceData = location['hasDeviceData']
    hasPropertyData = location['hasPropertyData']
    if (len(locationCode.split('.')) == 2):
        parentName = onc.getLocationName(locationCode.split('.')[0])
        if (parentName):
            locationName = '{} / {}'.format(parentName,locationName)
    print('  {0} - {1}'.format(locationCode,locationName))
    print('     Request data product using Device Category (Site Device): {}'.format(hasDeviceData))
    print('     Request data product using Property (Primary Sensor): {}'.format(hasPropertyData))
print('{} locations'.format(len(locations)))

getDevices(parameters={})

Returns a list of devices, as a list of dictionary objects.

Parameter

Type

Description

Example

Optional

parameters

dictionary

A dictionary of parameters, used to filter data from the ONC devices endpoint.

If excluded, all devices are returned.
See devices Discovery Service for available parameters (filters).

{'deviceCode':'NORTEKADCP9917'}

Example - Print all devices deployed at Barkley Canyon - Axis, between June 1, 2016 and May 31, 2017

from onc.onc import ONC
onc = ONC('YOUR_TOKEN_HERE')

devices = onc.getDevices({'locationCode':'BACAX','begin':'2016-06-01T00:00:00.000Z','end':'2017-05-31T23:59:59.999Z'})  #BACAX - Barkley Canyon / Axis

print('Devices:')

for device in devices:
    deviceCode = device['deviceCode']
    deviceName = device['deviceName']
    print('  {} - {}'.format(deviceCode,deviceName))
print('{} devices'.format(len(devices)))

getDeviceCategories(parameters={})

Returns a list of deviceCategories, as a list of dictionary objects.

Parameter

Type

Description

Example

Optional

parameters

dictionary

A dictionary of parameters, used to filter data from the ONC deviceCategories endpoint.

If excluded, all device categories are returned.
See deviceCategories Discovery Service for available parameters (filters).

{'deviceCategoryCode':'HYDROPHONE'}

Example - Print all device categories available at Barkley Canyon - Upper Slope

from onc.onc import ONC
onc = ONC('YOUR_TOKEN_HERE')

deviceCategories = onc.getDeviceCategories({'locationCode':'NCBC'})    #NCBC - Barkely Canyon / Upper Slope

print('Device Categories:')

for deviceCategory in deviceCategories:
    deviceCategoryCode = deviceCategory['deviceCategoryCode']
    deviceCategoryName = deviceCategory['deviceCategoryName']
    print('  {} - {}'.format(deviceCategoryCode,deviceCategoryName))
print('{} device categories'.format(len(deviceCategories)))

getProperties(parameters={})

Returns a list of properties, as a list of dictionary objects.

Parameter

Type

Description

Example

Optional

parameters

dictionary

A dictionary of parameters, used to filter data from the ONC properties endpoint.

If excluded, all properties are returned.
See properties Discovery Service for available parameters (filters).

{'propertyCode':'seawatertemperature'}

Example - Print all properties that are available at Barkley Canyon - Axis

from onc.onc import ONC
onc = ONC('YOUR_TOKEN_HERE')

properties = onc.getProperties({'locationCode':'BACAX'})  #BACAX - Barkley Canyon / Axis

print('Properties:')

for prprty in properties:
    propertyCode = prprty['propertyCode']
    propertyName = prprty['propertyName']
    description = prprty['description']
    print('  {} - {} ({})'.format(propertyCode,propertyName,description))
print('{} properties'.format(len(properties)))

getDataProducts(parameters={})

Returns a list of data products as a list of dictionary objects.

Parameter

Type

Description

Example

Optional

parameters

dictionary

A dictionary of parameters, used to filter data from the ONC dataProducts endpoint.

If excluded, all data products are returned.
See dataProducts Discovery Service for available parameters (filters).

{'dataProductCode':'TSSD','extension':'json'}

Example - Print all MatLab data product

from onc.onc import ONC
onc = ONC('YOUR_TOKEN_HERE')

dataProducts = onc.getDataProducts({'extension':'mat'}) #mat - MatLab

print('Data Products:')

for c in sorted(set([str(dp['dataProductCode']) for dp in dataProducts])):
    name = [dp['dataProductName'] for dp in dataProducts if dp['dataProductCode'] == c][0]
    print ('{} - {}'.format(c,name))
    for e in sorted(set([dp['extension'] for dp in dataProducts if dp['dataProductCode'] == c])):
        print('  {}'.format(e))
print('{} data product extensions'.format(len(dataProducts)))

orderDataProduct(parameters={}, maxRetries=100, downloadResultsOnly=False)

Requests a data product and downloads the generated files to the class's outPath.

This method performs the complete data product download workflow by calling additional methods to perform the following steps:

1. Request a data product, which includes estimates for download size and time.

2. Run a requested data product, which kicks off a process on the task machine to generate the data product.

3. Download the data product to disk. The process will continue to poll the web service to determine if the product is ready to download. If it is not ready, status messages will be provided. Once it is ready, it will download the data product to disk

Parameter	Type	Description	Example
Required
parameters	dictionary	A dictionary of parameters, used to request data as a data product from the ONC dataProductDelivery endpoint. See locations Discovery Service, devices Discovery Service, deviceCategories Discovery Service, properties Discovery Service and dataProducts Discovery Service for available parameters (filters).	{'locationCode':'BACAX', 'deviceCategoryCode':'ADCP2MHZ', 'dataProductCode':'TSSD', 'extension':'csv', 'begin':'2016-07-27T00:00:00.000Z', 'end':'2016-08-01T00:00:00.000Z', 'dpo_qualityControl':1, 'dpo_resample':'none', 'dpo_dataGaps':0}
Option
maxRetries	int	The number of times to retry the service before the function aborts. If excluded, the default of 100 is used	10
downloadResultsOnly	boolean	Indicates if the files will be downloaded or if only the url to the file will be returned False - Files are downloaded to the ONC class outPath property location True - Files will not be downloaded. The download URLs are available from the url property for each downloadResult in downloadResults in the returned dictionary If excluded, the default of False is used	True

Returns

Example - Download Time Series Scalar Data Product in CSV format for ADCP 2 MHZ at Barkley Canyon - Axis

from onc.onc import ONC
onc = ONC('YOUR_TOKEN_HERE')

results = onc.orderDataProduct({'locationCode':'BACAX',
		                        'deviceCategoryCode':'ADCP2MHZ',
        		                'dataProductCode':'TSSD',
                		        'extension':'csv',
                        		'begin':'2016-07-27T00:00:00.000Z',
                        		'end':'2016-08-01T00:00:00.000Z',
                        		'dpo_qualityControl':1,
                        		'dpo_resample':'none',
                        		'dpo_dataGaps':0})
print(results)

requestDataProduct(parameters={}, returnError=False)

Requests a data product using the dataProductDelivery service and provides request information so that the calling function can decide if should run the data product.
The dictionary of information it returns includes a requestID, which is used to run the data product, and estimates of the expected download times and file sizes.

Parameter	Type	Description	Example
Required
parameters	dictionary	A dictionary of parameters, used to request data as a data product from the ONC dataProductDelivery endpoint. See locations Discovery Service, devices Discovery Service, deviceCategories Discovery Service, properties Discovery Service and dataProducts Discovery Service for available parameters (filters).	{'locationCode':'BACAX', 'deviceCategoryCode':'ADCP2MHZ', 'dataProductCode':'TSSD', 'extension':'csv', 'begin':'2016-07-27T00:00:00.000Z', 'end':'2016-08-01T00:00:00.000Z', 'dpo_qualityControl':1, 'dpo_resample':'none', 'dpo_dataGaps':0}
Option
returnError	boolean	Indicates if error messages from the dataProductDelivery service should be returned. True - error messages from the service call payload will be returned. False - error messages will not be returned, but printed to console. If excluded, the default value of False is used.	True

from onc.onc import ONC
onc = ONC('YOUR_TOKEN_HERE')

 
requestInfo = onc.requestDataProduct({'locationCode':'BACAX',
			 						  'deviceCategoryCode':'ADCP2MHZ',
			 						  'dataProductCode':'TSSD',
			 						  'extension':'csv',
			 						  'begin':'2016-07-27T00:00:00.000Z',
			 						  'end':'2016-08-01T00:00:00.000Z',
			 						  'dpo_qualityControl':1,
			 						  'dpo_resample':'none',
			 						  'dpo_dataGaps':0})
print(requestInfo)

runDataProduct(requestId)

Runs a data product request using the dataProductDelivery service with a Request Id and returns a list of run ids

Parameter	Type	Description	Example
Required
requestId	int	A dataProductDelivery dpRequestId, which is returned by the dataProductDelivery service request method, or as the 'dpRequestId' property in the dpRequest returned by the requestDataProduct() method.	2046404

Example - Run a requested data product

from onc.onc import ONC
onc = ONC('YOUR_TOKEN_HERE')

ids = onc.runDataProduct(YOUR_REQUEST_ID_HERE)
 
print(ids)

downloadDataProduct(runId, fileCount=0, estimatedProcessingTime=1, maxRetries=100, downloadResultsOnly=False, includeMetadataFile=False, multiThreadMessages=False)

Polls the dataProductDelivery service until data product generation task is complete and downloads all of the files and/or retrieves the file information (url, file and download status).

Parameter	Type	Description	Example
Required
runId	int	A dataProductDelivery dpRunId, which is returned by the dataProductDelivery service run method, or as the dpRunId property in the list returned by the runDataProduct() method.	4629218
Option
fileCount	int	The actual or estimated file count, which is returned from the dataProductDelivery request method If there is no estimated file count, use 0. If excluded, the default of 0 is used.	1
estimatedProcessingTime	int	The estimated time in seconds, that it should take to run the request on the task machine. Used to determine how often the dataProductDelivery service is called to see if the task processing is complete. Estimated processing time is provided by the dataProductDelivery request method	2
maxRetries	int	The number of times to retry the service before the function aborts. If excluded, the default of 100 is used	10
downloadResultsOnly	boolean	Determines if the files will be downloaded or if only the url to the file will be returned False - Files are downloaded to the ONC class outPath property location True - Files will not be downloaded. The download URLs are available from the url property for each downloadResult in downloadResults in the returned dictionary If excluded, the default of False is used	True
includeMetadataFile	boolean	Indicates if the metadata file associated with the data product request will be downloaded False - Metadata file is not downloaded True - Metadata file is downloaded If excluded, the default of False is used.	True
multiThreadMessages	boolean	Determines how the method and called method should print messages to the console. If downloading data products in a multi-threaded pattern, messages written to the console can overlap and progress dots can be written out of context If excluded, the default of False is used.	True

Returns

Example - Download the results of requested data product run

from onc.onc import ONC
onc = ONC('YOUR_TOKEN_HERE')

results = onc.downloadDataProduct(YOUR_RUN_ID_HERE, 1)
 
print(result)

getDataProductUrls(parameters, maxRetries=100)

Orders a data product and return only the urls, for download at a later time. Urls can be used with the downloadFile() or getJsonFromUrl() methods

Parameter	Type	Description	Example
Required
parameters	dictionary	A dictionary of parameters, used to request data as a data product from the ONC dataProductDelivery endpoint. See locations Discovery Service, devices Discovery Service, deviceCategories Discovery Service, properties Discovery Service and dataProducts Discovery Service for available parameters (filters).	{'locationCode':'BACAX', 'deviceCategoryCode':'ADCP2MHZ', 'dataProductCode':'TSSD', 'extension':'json', 'begin':'2016-07-27T00:00:00.000Z', 'end':'2016-08-01T00:00:00.000Z', 'dpo_qualityControl':1, 'dpo_resample':'none', 'dpo_dataGaps':0, 'dpo_jsonOutputEncoding':'OM'}
Option
maxRetries	int	The number of times to retry the service before the function aborts. If excluded, the default of 100 is used	10

Example - Order a data product and get a list of the download urls

from onc.onc import ONC
onc = ONC('YOUR_TOKEN_HERE')
 
urls = onc.getDataProductUrls({'locationCode':'BACAX',
							   'deviceCategoryCode':'ADCP2MHZ',
                               'dataProductCode':'TSSD',
							   'extension':'json',
							   'begin':'2016-07-27T00:00:00.000Z',
							   'end':'2016-08-01T00:00:00.000Z',
							   'dpo_qualityControl':1,
							   'dpo_resample':'none',
							   'dpo_dataGaps':0,
							   'dpo_jsonOutputEncoding':'OM'},100)
 
print(urls)

downloadFile(url, file, multiThreadMessages=False)

Downloads a file from a URL, write it to a file and return download results information (url, file, message and download status).

Parameter	Type	Description	Example
Required
url	string	The url to be downloaded	http://dmas.uvic.ca/api/dataProductDelivery?method=download&token=<YOUR_TOKEN_HERE> &dpRunId=<YOUR_RUN_ID_HERE>&index=<YOUR_INDEX_HERE>
file	string	The fulll path of the file to be downloaded to.	c:/temp/myDownload.csv
Option
multiThreadMessages	boolean	Determines how the method and called method should print messages to the console. If downloading data products in a multi-threaded pattern, messages written to the console can overlap and progress dots can be written out of context If excluded, the default of False is used.	True

Example - Download a file from a url

from onc.onc import ONC
onc = ONC('YOUR_TOKEN_HERE')
 
url = 'http://dmas.uvic.ca/api/dataProductDelivery?method=download&token=<YOUR_TOKEN_HERE>&dpRunId=<YOUR_RUN_ID_HERE>&index=<YOUR_INDEX_HERE>'
file = 'c:/temp/myDownload.csv'

downloadResult = onc.downloadFile(url, file)
 
print(downloadResult)

getJsonFromUrl(url)

Returns a dictionary from the JSON returned from a URL

Parameter	Type	Description	Example
Required
url	string	The url to a JSON data product delivery result	http://dmas.uvic.ca/api/dataProductDelivery?method=download&token=<YOUR_TOKEN_HERE>&dpRunId=<YOUR_RUN_ID_HERE>&index=<YOUR_INDEX_HERE>

Example - Get dictionary of results from json data product request

from onc.onc import ONC
onc = ONC('YOUR_TOKEN_HERE')

 
from onc.onc import ONC
onc = ONC('YOUR_TOKEN_HERE')
 
urls = onc.getDataProductUrls({'locationCode':'BACAX',
							   'deviceCategoryCode':'ADCP2MHZ',
                               'dataProductCode':'TSSD',
							   'extension':'json',
							   'begin':'2016-07-27T00:00:00.000Z',
							   'end':'2016-08-01T00:00:00.000Z',
							   'dpo_qualityControl':1,
							   'dpo_resample':'none',
							   'dpo_dataGaps':0,
							   'dpo_jsonOutputEncoding':'OM'},100)
 
for url in urls:
	result = onc.getJsonFromUrl(url)
 	print(result)

Page tree

Installation

History

Classes

ONC Class

Description

constructor

Usage

methods

SOS Class

ERDDAP Class

NERC Class

Page tree

Python Client Library

Installation

History

Classes

ONC Class

Description

constructor

Usage

methods

SOS Class

ERDDAP Class

NERC Class