The ONC Python client library contains a number of classes and functions that make access to the ONC data, through the Oceans 2.0 API, quick and easy to use.

Requires

2.7 or Higher

3.6 Recommended

Installation

The ONC Python client library can be installed using the pip command found in the Scripts folder of your python install.

$ pip install onc

If you have a previous version of the ONC Python Client Library installed, and need to get the latest version use the following:

$ pip install onc --upgrade

History

version	Date	Description	Package	Source
1.0	6/9/2017	Beta version	onc-1.0.tar.gz
1.1	6/12/2016	Rename onc.dap.DAP class to onc.dap.ERDDAP	onc-1.1.tar.gz
1.11	6/13/2016	Removed unused numpy dependency. Added python-dateutil dependency to package.	onc-1.11.tar.gz
1.12	6/23/2016	Refactored scalar and raw data access methods. Removed print methods for testing services. Renamed private functions. Added numpy references	onc-1.12.tar.gz	source-1.12.zip

Classes

class	import	description
ONC	from onc.onc import ONC	A wrapper class for access to Oceans 2.0 API web services
SOS	from onc.sos import SOS	A wrapper class for access to ONC's implementation of Sensor Observation Service (SOS)
ERDDAP	from onc.dap import ERDDAP	A wrapper class for access to datasets published on ONC's ERDDAP (OPeNDAP) server http://qadap.onc.uvic.ca/erddap/index.html
NERC	from onc.nerc import NERC	A wrapper class for access to controlled vocabularies stored on the NERC vocabulary server http://vocab.nerc.ac.uk/

ONC Class

Description

This class provides a wrapper for some for the most common Oceans 2.0 API requests, including the discovery services (locations, devices, deviceCategories, properties) and delivery (dataProductDelivery, scalardata and rawdata).

Constructor

ONC()

ONC(token, production=True, showInfo=False, outPath='c:/temp')

Parameter	Type	Description	Example
Required
token	string	All Web Services require a token. This can be generated at http://dmas.uvic.ca/Profile. Click on the "Web Services" tab and click "Generate Token"	token=5ba514e8-1203-428c-8835-879c8173e387
Optional
production	boolean	Indicates if the ONC Production server URL is used for service requests. Default is True. False is intended for internal ONC testing only. All external users should use False or leave blank.	True or False
showInfo	boolean	Indicates if verbose script messages are displayed, such as request url and processing time information. Default is False. True is intended for script debugging.	True or False
outPath	string	The path that downloaded files are saved to. Default is 'c:/Temp'. Full path will be created if it does not already exist.	'c:/ONC/Download'

Usage

from onc.onc import ONC
onc = ONC('YOUR_TOKEN_HERE')

Or

from onc.onc import ONC

isProduction = True
showInfo = False
outPath = 'c:/ONC/Data'
onc = ONC('YOUR_TOKEN_HERE',isProduction,showInfo,outPath)

Discovery methods

getLocations()

getLocations(parameters={})

Returns a list of locations, as a list of dictionary objects.

Parameter

Type

Description

Example

Optional

parameters

dictionary

A dictionary of parameters, used to filter data from the ONC devices endpoint.

If excluded, all locations are returned.
See locations Discovery Service for available parameters (filters).

{'locationCode':'BACAX'}

Example - Print all locations in North East Pacific with Hydrophones deployed

from onc.onc import ONC
onc = ONC('YOUR_TOKEN_HERE')

locations = onc.getLocations({'locationCode':'NEP', 'deviceCategoryCode':'HYDROPHONE'}) #NEP - North East Pacific
print('Locations')

for location in locations:
    locationCode = location['locationCode']
    locationName = location['locationName']
    hasDeviceData = location['hasDeviceData']
    hasPropertyData = location['hasPropertyData']
    if (len(locationCode.split('.')) == 2):
        parentName = onc.getLocationName(locationCode.split('.')[0])
        if (parentName):
            locationName = '{} / {}'.format(parentName,locationName)
    print('  {0} - {1}'.format(locationCode,locationName))
    print('     Request data product using Device Category (Site Device): {}'.format(hasDeviceData))
    print('     Request data product using Property (Primary Sensor): {}'.format(hasPropertyData))
print('{} locations'.format(len(locations)))

getDevices()

getDevices(parameters={})

Returns a list of devices, as a list of dictionary objects.

Parameter

Type

Description

Example

Optional

parameters

dictionary

A dictionary of parameters, used to filter data from the ONC devices endpoint.

If excluded, all devices are returned.
See devices Discovery Service for available parameters (filters).

{'deviceCode':'NORTEKADCP9917'}

Example - Print all devices deployed at Barkley Canyon - Axis, between June 1, 2016 and May 31, 2017

from onc.onc import ONC
onc = ONC('YOUR_TOKEN_HERE')

devices = onc.getDevices({'locationCode':'BACAX','deploymentBegin':'2016-06-01T00:00:00.000Z','deploymentEnd':'2017-05-31T23:59:59.999Z'})  #BACAX - Barkley Canyon / Axis

print('Devices:')

for device in devices:
    deviceCode = device['deviceCode']
    deviceName = device['deviceName']
    print('  {} - {}'.format(deviceCode,deviceName))
print('{} devices'.format(len(devices)))

getDeviceCategories()

getDeviceCategories(parameters={})

Returns a list of deviceCategories, as a list of dictionary objects.

Parameter

Type

Description

Example

Optional

parameters

dictionary

A dictionary of parameters, used to filter data from the ONC deviceCategories endpoint.

If excluded, all device categories are returned.
See deviceCategories Discovery Service for available parameters (filters).

{'deviceCategoryCode':'HYDROPHONE'}

Example - Print all device categories available at Barkley Canyon - Upper Slope

from onc.onc import ONC
onc = ONC('YOUR_TOKEN_HERE')

deviceCategories = onc.getDeviceCategories({'locationCode':'NCBC'})    #NCBC - Barkely Canyon / Upper Slope

print('Device Categories:')

for deviceCategory in deviceCategories:
    deviceCategoryCode = deviceCategory['deviceCategoryCode']
    deviceCategoryName = deviceCategory['deviceCategoryName']
    print('  {} - {}'.format(deviceCategoryCode,deviceCategoryName))
print('{} device categories'.format(len(deviceCategories)))

getProperties()

getProperties(parameters={})

Returns a list of properties, as a list of dictionary objects.

Parameter

Type

Description

Example

Optional

parameters

dictionary

A dictionary of parameters, used to filter data from the ONC properties endpoint.

If excluded, all properties are returned.
See properties Discovery Service for available parameters (filters).

{'propertyCode':'seawatertemperature'}

Example - Print all properties that are available at Barkley Canyon - Axis

from onc.onc import ONC
onc = ONC('YOUR_TOKEN_HERE')

properties = onc.getProperties({'locationCode':'BACAX'})  #BACAX - Barkley Canyon / Axis

print('Properties:')

for prprty in properties:
    propertyCode = prprty['propertyCode']
    propertyName = prprty['propertyName']
    description = prprty['description']
    print('  {} - {} ({})'.format(propertyCode,propertyName,description))
print('{} properties'.format(len(properties)))

getDataProducts()

getDataProducts(parameters={})

Returns a list of data products as a list of dictionary objects.

Parameter

Type

Description

Example

Optional

parameters

dictionary

A dictionary of parameters, used to filter data from the ONC dataProducts endpoint.

If excluded, all data products are returned.
See dataProducts Discovery Service for available parameters (filters).

{'dataProductCode':'TSSD','extension':'json'}

Example - Print all MatLab data product

from onc.onc import ONC
onc = ONC('YOUR_TOKEN_HERE')

dataProducts = onc.getDataProducts({'extension':'mat'}) #mat - MatLab

print('Data Products:')

for c in sorted(set([str(dp['dataProductCode']) for dp in dataProducts])):
    name = [dp['dataProductName'] for dp in dataProducts if dp['dataProductCode'] == c][0]
    print ('{} - {}'.format(c,name))
    for e in sorted(set([dp['extension'] for dp in dataProducts if dp['dataProductCode'] == c])):
        print('  {}'.format(e))
print('{} data product extensions'.format(len(dataProducts)))

Data Product Download methods

orderDataProduct()

orderDataProduct(parameters={}, maxRetries=100, downloadResultsOnly=False, includeMetadataFile=True)

Requests a data product and downloads the generated files to the class's outPath.

This method performs the complete data product download workflow by calling additional methods to perform the following steps:

1. Request a data product, which includes estimates for download size and time.

2. Run a requested data product, which kicks off a process on the task machine to generate the data product.

3. Download the data product to disk. The process will continue to poll the web service to determine if the product is ready to download. If it is not ready, status messages will be provided. Once it is ready, it will download the data product to disk

Parameter	Type	Description	Example
Required
parameters	dictionary	A dictionary of parameters, used to request data as a data product from the ONC dataProductDelivery endpoint. See locations Discovery Service, devices Discovery Service, deviceCategories Discovery Service, properties Discovery Service and dataProducts Discovery Service for available parameters (filters).	{'locationCode':'BACAX', 'deviceCategoryCode':'ADCP2MHZ', 'dataProductCode':'TSSD', 'extension':'csv', 'begin':'2016-07-27T00:00:00.000Z', 'end':'2016-08-01T00:00:00.000Z', 'dpo_qualityControl':1, 'dpo_resample':'none', 'dpo_dataGaps':0}
Option
maxRetries	int	The number of times to retry the service before the function aborts. If excluded, the default of 100 is used	10
downloadResultsOnly	boolean	Indicates if the files will be downloaded or if only the url to the file will be returned False - Files are downloaded to the ONC class outPath property location True - Files will not be downloaded. The download URLs are available from the url property for each downloadResult in downloadResults in the returned dictionary If excluded, the default of False is used	True
includeMetadataFile	boolean	Indicates if the metadata file associated with the data product request will be downloaded False - Metadata file is not downloaded True - Metadata file is downloaded If excluded, the default of False is used.	True

Returns

A dictionary of download results and run time statistics

{'downloadResults': [
    {
        'url': 'http://dmas.uvic.ca/api/dataProductDelivery?method=download&token=<token>&dpRunId=<runid>&index=1', 
        'message': [
            'Running', 
            'Transferring (BarkleyCanyon_Axis_ADCP2MHz_20160727T000005Z_20160731T235958Z-clean.csv) to the FTP server'
            ], 
        'queuingTime': 0, 
        'runningTime': 1.567234, 
        'transferringTime': 59.616635, 
        'downloadingTime': 0.295575, 
        'requestCount': 40, 
        'status': 'complete', 
        'size': 3060227.0, 
        'file': 'c:/ONC/data/BarkleyCanyon_Axis_ADCP2MHz_20160727T000005Z_20160731T235958Z-clean.csv', 
        'index': 1,
        'downloaded': True}, 
    {
        'url': 'http://dmas.uvic.ca/api/dataProductDelivery?method=download&token=<token>&dpRunId=<runid>&index=meta', 
        'message': [], 
        'queuingTime': 0, 
        'runningTime': 0, 
        'transferringTime': 0, 
        'downloadingTime': 0.132616, 
        'requestCount': 1, 
        'status': 'complete', 
        'size': 27502.0, 
        'file': 'c:/temp/doctest/BarkleyCanyon_Axis_ADCP2MHz_20160727T000005Z_20160731T235958Z-clean_CSV_META.pdf', 
        'index': 1, 
        'downloaded': True
    }
], 
'stats': {
    'requestRequestTime': 0.328399, 
    'runRequestTime': 1.060785, 
    'queuingTime': 0, 
    'runningTime': 1.567234, 
    'transferringTime': 59.616635, 
    'downloadingTime': 0.428191
    }
}

Property	Type	Description
downloadResults	list	A list of download results dictionaries.
downloadResults[].url	string	The URL used to make the download request.
downloadResults[].message	list	A list of the messages that were returned from the dataProductDelivery download method when it was polled.
downloadResults[].queuingTime	float	The time duration, in seconds, between the first and last service request, that indicated that the task was queued.
downloadResults[].runningTime	float	The time duration, in seconds, between the first request that returned a status message that it was running and the time that the request returned that it was transferring or complete.
downloadResults[].transferringTime	float	The time duration, in seconds, between the first request that returned a status message that it was transferring.
downloadResults[].downloadingTime	float	The time duration, in seconds, between the start and completion of the file download.
downloadResults[].requestCount	integer	The number of web service request that were made before the download was complete. This includes all of the requests that returned run task status messages.
downloadResults[].status	string	The status of the download process at the request. Possible values are: "running" - The data product request is running on the Task server. "complete" - The data product request Task has completed and file has been downloaded or url has been returned. "error" - An error, preventing the task execution or download from completing successfully, has occurred.
downloadResults[].size	float	The actual size of the file in bytes.
downloadResults[].file	string	The full path of the file that was downloaded.
downloadResults[].index	string	The index of the file downloaded. The index can either be a number 1-* or "meta" for the data product metadata file.
downloadResults[].downloaded	boolean	Indicates if the file was downloaded. If the filter downloadResultsOnly=True is used, the data product is not downloaded and the value will be False
stats	list	A summary of individual result statistics.
stats.requestRequestTime	float	The time to complete the request request for the requested data product.
stats.runRequestTime	float	The time to complete the run request for the requested data product.
stats.queuingTime	float	A sum of the queueTime times from all of the run requests for the requested data product.
stats.runningTime	float	A sum of the runningTime times from all of the run requests for the requested data product.
stats.transferringTime	float	A sum of the transferringTime times from all of the run requests for the requested data product.
stats.downloadingTime	float	A sum of the downloadTime times from all of the run requests for the requested data product.

Example - Download Time Series Scalar Data Product in CSV format for ADCP 2 MHZ at Barkley Canyon - Axis

from onc.onc import ONC
onc = ONC('YOUR_TOKEN_HERE')

results = onc.orderDataProduct({'locationCode':'BACAX',
		                        'deviceCategoryCode':'ADCP2MHZ',
        		                'dataProductCode':'TSSD',
                		        'extension':'csv',
                        		'begin':'2016-07-27T00:00:00.000Z',
                        		'end':'2016-08-01T00:00:00.000Z',
                        		'dpo_qualityControl':1,
                        		'dpo_resample':'none',
                        		'dpo_dataGaps':0})
print(results)

requestDataProduct()

requestDataProduct(parameters={}, returnError=False)

Requests a data product using the dataProductDelivery service and provides request information so that the calling function can decide if should run the data product.
The dictionary of information it returns includes a requestID, which is used to run the data product, and estimates of the expected download times and file sizes.

Parameter	Type	Description	Example
Required
parameters	dictionary	A dictionary of parameters, used to request data as a data product from the ONC dataProductDelivery endpoint. See locations Discovery Service, devices Discovery Service, deviceCategories Discovery Service, properties Discovery Service and dataProducts Discovery Service for available parameters (filters).	{'locationCode':'BACAX', 'deviceCategoryCode':'ADCP2MHZ', 'dataProductCode':'TSSD', 'extension':'csv', 'begin':'2016-07-27T00:00:00.000Z', 'end':'2016-08-01T00:00:00.000Z', 'dpo_qualityControl':1, 'dpo_resample':'none', 'dpo_dataGaps':0}
Option
returnError	boolean	Indicates if error messages from the dataProductDelivery service should be returned. True - error messages from the service call payload will be returned. False - error messages will not be returned, but printed to console. If excluded, the default value of False is used.	True

Returns

returns a data product request dictionary

for archived files (AD):

{
    "dpRequestId":2046404,
    "compressedFileSize":12563408,
    "fileSize":70766230,
    "numFiles":4,
    "downloadTimes":{"10Mbps":7.076623,"50Mbps":1.4153247,"150Mbps":0.47177488}
}

for generated files:

{
    "dpRequestId":2046404,
    "estimatedFileSize":8348187,
    "estimatedProcessingTime":16
}

Property	Type	Description	Example
Always contains
dpRequestId	int	A unique id for a data product request	`"dpRequestId":`2046404
May contain
compressedFileSize	int	The compressed size of the known file(s) in bytes.	`"compressedFileSize":12563408`
fileSize	int	The size of known file(s) in bytes.	`"fileSize":70766230`
numFiles	int	The number of files.	`"numFiles":4`
downloadTimes	dict	A dictionary of estimated download times	`"downloadTimes":{"10Mbps":7.076623,"50Mbps":1.4153247,"150Mbps":0.47177488}`
estimatedFileSize	int	The estimated file size of generated data product	"estimatedFileSize":8348187
estimatedProcessingTime	int	The estimated time, in seconds, that it will take to run the data product request.	"estimatedProcessingTime":16

from onc.onc import ONC
onc = ONC('YOUR_TOKEN_HERE')

 
requestInfo = onc.requestDataProduct({'locationCode':'BACAX',
			 						  'deviceCategoryCode':'ADCP2MHZ',
			 						  'dataProductCode':'TSSD',
			 						  'extension':'csv',
			 						  'begin':'2016-07-27T00:00:00.000Z',
			 						  'end':'2016-08-01T00:00:00.000Z',
			 						  'dpo_qualityControl':1,
			 						  'dpo_resample':'none',
			 						  'dpo_dataGaps':0})
print(requestInfo)

runDataProduct()

runDataProduct(requestId)

Runs a data product request using the dataProductDelivery service with a Request Id and returns a list of run ids

Parameter	Type	Description	Example
Required
requestId	int	A dataProductDelivery dpRequestId, which is returned by the dataProductDelivery service request method, or as the 'dpRequestId' property in the dpRequest returned by the requestDataProduct() method.	2046404

Returns

a list of runIds

Example - Run a requested data product

from onc.onc import ONC
onc = ONC('YOUR_TOKEN_HERE')

ids = onc.runDataProduct(YOUR_REQUEST_ID_HERE)
 
print(ids)

downloadDataProduct()

downloadDataProduct(runId, fileCount=0, estimatedProcessingTime=1, maxRetries=100, downloadResultsOnly=False, includeMetadataFile=False, multiThreadMessages=False)

Polls the dataProductDelivery service until data product generation task is complete and downloads all of the files and/or retrieves the file information (url, file and download status).

Parameter	Type	Description	Example
Required
runId	int	A dataProductDelivery dpRunId, which is returned by the dataProductDelivery service run method, or as the dpRunId property in the list returned by the runDataProduct() method.	4629218
Option
fileCount	int	The actual or estimated file count, which is returned from the dataProductDelivery request method If there is no estimated file count, use 0. If excluded, the default of 0 is used.	1
estimatedProcessingTime	int	The estimated time in seconds, that it should take to run the request on the task machine. Used to determine how often the dataProductDelivery service is called to see if the task processing is complete. Estimated processing time is provided by the dataProductDelivery request method	2
maxRetries	int	The number of times to retry the service before the function aborts. If excluded, the default of 100 is used	10
downloadResultsOnly	boolean	Determines if the files will be downloaded or if only the url to the file will be returned False - Files are downloaded to the ONC class outPath property location True - Files will not be downloaded. The download URLs are available from the url property for each downloadResult in downloadResults in the returned dictionary If excluded, the default of False is used	True
includeMetadataFile	boolean	Indicates if the metadata file associated with the data product request will be downloaded False - Metadata file is not downloaded True - Metadata file is downloaded If excluded, the default of False is used.	True
multiThreadMessages	boolean	Determines how the method and called method should print messages to the console. If downloading data products in a multi-threaded pattern, messages written to the console can overlap and progress dots can be written out of context If excluded, the default of False is used.	True

Returns

returns a list of download results dictionaries

[
    {
        'url': 'http://dmas.uvic.ca/api/dataProductDelivery?method=download&token=<token>&dpRunId=<runid>&index=1', 
        'message': [
            'Running', 
            'Transferring (BarkleyCanyon_Axis_ADCP2MHz_20160727T000005Z_20160731T235958Z-clean.csv) to the FTP server'
            ], 
        'queuingTime': 0, 
        'runningTime': 1.567234, 
        'transferringTime': 59.616635, 
        'downloadingTime': 0.295575, 
        'requestCount': 40, 
        'status': 'complete', 
        'size': 3060227.0, 
        'file': 'c:/ONC/data/BarkleyCanyon_Axis_ADCP2MHz_20160727T000005Z_20160731T235958Z-clean.csv', 
        'index': 1,
        'downloaded': True}, 
    {
        'url': 'http://dmas.uvic.ca/api/dataProductDelivery?method=download&token=<token>&dpRunId=<runid>&index=meta', 
        'message': [], 
        'queuingTime': 0, 
        'runningTime': 0, 
        'transferringTime': 0, 
        'downloadingTime': 0.132616, 
        'requestCount': 1, 
        'status': 'complete', 
        'size': 27502.0, 
        'file': 'c:/temp/doctest/BarkleyCanyon_Axis_ADCP2MHz_20160727T000005Z_20160731T235958Z-clean_CSV_META.pdf', 
        'index': 'meta', 
        'downloaded': True
    }
]

Property	Type	Description
url	string	The URL used to make the download request.
message	list	A list of the messages that were returned from the dataProductDelivery download method when it was polled.
queuingTime	float	The time duration, in seconds, between the first and last service request, that indicated that the task was queued.
runningTime	float	The time duration, in seconds, between the first request that returned a status message that it was running and the time that the request returned that it was transferring or complete.
transferringTime	float	The time duration, in seconds, between the first request that returned a status message that it was transferring.
downloadingTime	float	The time duration, in seconds, between the start and completion of the file download.
requestCount	integer	The number of web service request that were made before the download was complete. This includes all of the requests that returned run task status messages.
status	string	The status of the download process at the request. Possible values are: "running" - The data product request is running on the Task server. "complete" - The data product request Task has completed and file has been downloaded or url has been returned. "error" - An error, preventing the task execution or download from completing successfully, has occurred.
size	float	The actual size of the file in bytes.
file	string	The full path of the file that was downloaded.
index	string	The index of the file downloaded. The index can either be a number 1-* or "meta" for the data product metadata file.
downloaded	boolean	Indicates if the file was downloaded. If the filter downloadResultsOnly=True is used, the data product is not downloaded and the value will be False

Example - Download the results of requested data product run

from onc.onc import ONC
onc = ONC('YOUR_TOKEN_HERE')

results = onc.downloadDataProduct(YOUR_RUN_ID_HERE, 1)
 
print(result)

getDataProdutUrls()

getDataProductUrls(parameters, maxRetries=100)

Orders a data product and return only the urls, for download at a later time. Urls can be used with the downloadFile() or getJsonFromUrl() methods

Example - Order a data product and get a list of the download urls

from onc.onc import ONC
onc = ONC('YOUR_TOKEN_HERE')
 
urls = onc.getDataProductUrls({'locationCode':'BACAX',
							   'deviceCategoryCode':'ADCP2MHZ',
                               'dataProductCode':'TSSD',
							   'extension':'json',
							   'begin':'2016-07-27T00:00:00.000Z',
							   'end':'2016-08-01T00:00:00.000Z',
							   'dpo_qualityControl':1,
							   'dpo_resample':'none',
							   'dpo_dataGaps':0,
							   'dpo_jsonOutputEncoding':'OM'},100)
 
print(urls)

downloadFile()

downloadFile(url, file, multiThreadMessages=False)

Downloads a file from a URL, write it to a file and return download results information (url, file, message and download status).

Parameter	Type	Description	Example
Required
url	string	The url to be downloaded	http://dmas.uvic.ca/api/dataProductDelivery?method=download&token=<YOUR_TOKEN_HERE> &dpRunId=<YOUR_RUN_ID_HERE>&index=<YOUR_INDEX_HERE>
file	string	The fulll path of the file to be downloaded to.	c:/temp/myDownload.csv
Option
multiThreadMessages	boolean	Determines how the method and called method should print messages to the console. If downloading data products in a multi-threaded pattern, messages written to the console can overlap and progress dots can be written out of context If excluded, the default of False is used.	True

Example - Download a file from a url

from onc.onc import ONC
onc = ONC('YOUR_TOKEN_HERE')
 
url = 'http://dmas.uvic.ca/api/dataProductDelivery?method=download&token=<YOUR_TOKEN_HERE>&dpRunId=<YOUR_RUN_ID_HERE>&index=<YOUR_INDEX_HERE>'
file = 'c:/temp/myDownload.csv'

downloadResult = onc.downloadFile(url, file)
 
print(downloadResult)

getJsonFromUrl()

getJsonFromUrl(url)

Returns a dictionary from the JSON returned from a URL

Parameter	Type	Description	Example
Required
url	string	The url to a JSON data product delivery result	http://dmas.uvic.ca/api/dataProductDelivery?method=download&token=<YOUR_TOKEN_HERE>&dpRunId=<YOUR_RUN_ID_HERE>&index=<YOUR_INDEX_HERE>

Example - Get dictionary of results from json data product request

from onc.onc import ONC
onc = ONC('YOUR_TOKEN_HERE')

 
from onc.onc import ONC
onc = ONC('YOUR_TOKEN_HERE')
 
urls = onc.getDataProductUrls({'locationCode':'BACAX',
							   'deviceCategoryCode':'ADCP2MHZ',
                               'dataProductCode':'TSSD',
							   'extension':'json',
							   'begin':'2016-07-27T00:00:00.000Z',
							   'end':'2016-08-01T00:00:00.000Z',
							   'dpo_qualityControl':1,
							   'dpo_resample':'none',
							   'dpo_dataGaps':0,
							   'dpo_jsonOutputEncoding':'OM'},100)
 
for url in urls:
	result = onc.getJsonFromUrl(url)
 	print(result)

Near Real-Time data product methods

getDirectScalar()

getDirectScalar(parameters={}, outputFormat='Object', metadata='Full', rowLimit=None)

Returns scalar data, in JSON Object or Array format, in the response payload.

Parameter	Type	Description	Example
Required
parameters	dictionary	A dictionary of parameters, used to request data as a data product from the ONC dataProductDelivery endpoint. See locations Discovery Service, devices Discovery Service, deviceCategories Discovery Service, properties Discovery Service and dataProducts Discovery Service for available parameters (filters).	{ 'locationCode':'TWDP', 'deviceCategoryCode':'TSG', 'begin':'2016-09-01T00:00:00.000Z', 'end':'2016-09-01T08:00:00.000Z' }
Option
outputFormat	string	The encoding of the response output. Object - Data is a list of dictionaries with sampleTime, value and qaqcFlag for a given sensor. Array - Data has 3 arrays for a given sensor. Array of sampleTime, array of values and array of qaqcFlags If excluded, the default 'Object' is used.	"outputFormat":"Array"
metadata	string	The amount of metadata detail that is included in the response output. Minimum - Provides only basic property information. Response includes serviceMetadata dictionary only. Full - Provides all property information. Response includes both metadata and serviceMetadata dictionaries. If excluded, the default 'Minimum' is used.	"metadata":"Full"
rowLimit	integer	The maximum number of rows of scalar data that will be returned for each sensor Maximum is 100,000 If excluded, the default 100,000 is used	"rowLimit":5000

Returns

A dictionary that contains sensor data that matches the filter criteria defined in the input parameters, metadata and service metadata.

{
    'metadata': {
        'bbox': {
            'maxDepth': 3.0, 
            'maxLat': 49.080909946666665, 
            'maxLon': -123.37824890666667, 
            'minDepth': 3.0, 
            'minLat': 49.080909946666665, 
            'minLon': -123.37824890666667
        }, 
        'depth': 3.0, 
        'deviceCategoryName': 'TSG', 
        'latitude': 49.080909946666665, 
        'longitude': -123.37824890666667, 
        'begin': '2016-09-01T00:00:00.000Z', 
        'end': '2016-09-01T00:00:30.000Z', 
        'locationCode': 'TWDP'
    }, 
    'sensorData': [
        {
            'actualSamples': 1, 
            'data': [
                {
                    'qaqcFlag': 1, 
                    'sampleTime': '2016-09-01T00:00:07.037Z', 
                    'value': 3.44294
                }
             ], 
             'sensor': 'Conductivity', 
             'sensorName': 'Conductivity', 
             'unitOfMeasure': 'S/m'
        }, 
        {
            'actualSamples': 1, 
            'data': [
                {
                    'qaqcFlag': 1, 
                    'sampleTime': '2016-09-01T00:00:07.037Z', 
                    'value': 28.9096
                }
            ], 
            'sensor': 'salinity', 
            'sensorName': 'Practical Salinity', 
            'unitOfMeasure': 'psu'
        },
        {...}
    ], 
    'serviceMetadata': {
        'metadata': 'Full', 
        'nextDateFrom': '2016-09-01T00:00:17.041Z', 
        'outputFormat': 'Object', 
        'rowLimit': None, 
        'sensors': None, 
        'totalActualSamples': 3, 
        'begin': '2016-09-01T00:00:00.000Z', 
        'end': '2016-09-01T00:00:10.000Z', 
        'deviceCategoryCode': 'TSG', 
        'locationCode': 'TWDP'
    }
}

Property	Type	Description
metadata	dictionary	A dictionary of metadata, if metadata='Full' input parameter is used
metadata.bbox	dictionary	A Bounding Box dictionary
metadata.bbox.maxDepth	float	The maximum depth of the instruments contributing to the results
metadata.bbox.maxLat	float	The maximum Latitude of bounding box of the instruments contributing to the results
metadata.bbox.maxLon	float	The maximum Longitude of the bounding box of the instruments contributing to the results
metadata.bbox.minDepth	float	The minimum Depth of the instruments contributing to the results
metadata.bbox.minLat	float	The minimum Latitude of bounding box of the instruments contributing to the results
metadata.bbox.minLon	float	The minimum Longitude of the bounding box of the instruments contributing to the results
sensorData	list	A list of sensor dictionaries
sensorData[].acutalSamples	int	The number of samples in the data list
sensorData[].data	list	A list of observation dictionaries
sensorData[].data[].qaqcFlag	int	The QAQC Flag for the observation
sensorData[].data[].sampleTime	datetime string	The recorded time of the observation Date Time format: yyyy-MM-dd'T'HH:mm:ss.SSS'Z' datetime is represented in Coordinated Universal Time (UTC)
sensorData[].data[].value	float	The value of the observation
sensorData[].sensor	string	The code of the sensor that is being observed
sensorData[].sensorName	string	The name of the sensor that is being observed
sensorData[].unitOfMeasure	string	The unit of measure of the observation
serviceMetadata	dictionary	A dictionary of the metadata for the service request
serviceMetadata.metadata	string	The requested metadata filter
serviceMetadata.nextDataFrom	datetime string	The begin date for next reading in yyyy-MM-ddTHH:mm:ss.SSSZ format
serviceMetadata.outputFormat	string	The requested JSON output format
serviceMetadata.rowLimit	int	The requested row limit per sensor
serviceMetadata.sensors	string	A comma separated list of the requested sensors
serviceMetadata.totalActualSamples	int	The number of observations expected for all of the sensors
serviceMetadata.begin	datetime string	The begin date of the request in yyyy-MM-ddTHH:mm:ss.SSSZ format
serviceMetadata.end	datetime string	The end date of the request in yyyy-MM-ddTHH:mm:ss.SSSZ format
serviceMetadata.deviceCategoryCode	string	The Device Category Code of the request. See Available Device Categories for further details
serviceMetadata.locationCode	string	The Location Code of the request. See Available Locations for further details.

Example - Print the last Thermosalinograph reading from Tswwassen - Duke Point Ferry

o = onc('YOUR_TOKEN_HERE')
 
result = o.getDirectScalar({'locationCode','TWDP','deviceCategoryCode','TSG'});

disp(result)

Example - Print 1 hour of Thermosalinograph readings from Tswwassen - Duke Point Ferry

from onc.onc import ONC
onc = ONC('YOUR_TOKEN_HERE')
 
result = onc.getDirectScalar({'locationCode':'TWDP','deviceCategoryCode':'TSG', 'begin':'2016-09-01T00:00:00.000Z', 'begin':'2016-09-01T01:00:00.000Z'})

sensorData = result['sensorData']
if (sensorData):
	print('Properties (variables)')
	for data in sensorData:
		print("  {}".format(data['sensorName']))
		for sample in data['data']:
			print ("    Value = {} ; Sample Time = {}".format(sample['value'],sample['sampleTime']))
 
serviceMetadata = result['serviceMetadata']
if(serviceMetadata):
    print('{} samples'.format(serviceMetadata['totalActualSamples']))

Example - Print 1 hour of Thermosalinograph readings from Tswwassen - Duke Point Ferry as an Array

from onc.onc import ONC
onc = ONC('YOUR_TOKEN_HERE')
 
result = onc.getDirectScalar({'locationCode':'TWDP','deviceCategoryCode':'TSG', 'begin':'2016-09-01T00:00:00.000Z', 'begin':'2016-09-01T01:00:00.000Z'},'Array')

if (result):
    sensorData = result['sensorData']
    if (sensorData):
        print('Properties (variables)')
        for data in sensorData:
            print("  {}".format(data['sensorName']))
            samples = data['data']
            for i in range(0,data['actualSamples'] - 1):
                print ("    Value = {} ; Sample Time = {}".format(samples['values'][i],samples['sampleTimes'][i]))
 
serviceMetadata = result['serviceMetadata']
if(serviceMetadata):
    print('{} samples'.format(serviceMetadata['totalActualSamples']))

getDirectRaw()

getDirectRaw(parameters={}, rowLimit=None)

Returns raw data from an instrument in the payload, in JSON format.

Parameter	Type	Description	Example
Required
parameters	dictionary	A dictionary of parameters, used to request data as a data product from the ONC dataProductDelivery endpoint. See locations Discovery Service, devices Discovery Service, deviceCategories Discovery Service, properties Discovery Service and dataProducts Discovery Service for available parameters (filters).	{ 'locationCode':'TWDP', 'deviceCategoryCode':'TSG', 'begin':'2016-09-01T00:00:00.000Z', 'end':'2016-09-01T08:00:00.000Z' }
Option
rowLimit	integer	The maximum number of rows of raw data that will be returned Maximum is 100,000 If excluded, the default 100,000 is used	"rowLimit":5000

Returns

A dictionary that contains sensor data that matches the filter criteria defined in the input parameters, metadata and service metadata.

{
    'data': [
        {
            'rawData': '!AIVDM,1,1,,A,14eI18gP00G<JMPL<IaCNOv62D1C,0*43', 
            'sampleTime': '2017-06-23T17:18:04.645Z'
        }, 
        {
            'rawData': '!AIVDM,1,1,,A,34eGo=P009G;qPJL:kiv3I6800li,0*5C', 
            'sampleTime': '2017-06-23T17:18:04.698Z'
        }
    ], 
    'metadata': {
        'dataMetadata': {
            'firstTimestamp': '2017-06-23T17:18:04.645Z', 
            'hasMoreDataInRange': True, 
            'lastTimestamp': '2017-06-23T17:18:04.698Z', 
            'numberOfData': 2, 
            'locationCode': 'Iona Shore Station'
        }, 
        'queryMetadata': {
            'rowLimit': 2, 
            'stationCode': 'IONA', 
            'begin': None, 
            'end': None, 
            'deviceCategoryCode': 'AISRECEIVER'
        }
    }
}

Property	Type	Description
metadata	dictionary	A dictionary of metadata, if metadata='Full' input parameter is used
metadata.bbox	dictionary	A Bounding Box dictionary
metadata.bbox.maxDepth	float	The maximum depth of the instruments contributing to the results
metadata.bbox.maxLat	float	The maximum Latitude of bounding box of the instruments contributing to the results
metadata.bbox.maxLon	float	The maximum Longitude of the bounding box of the instruments contributing to the results
metadata.bbox.minDepth	float	The minimum Depth of the instruments contributing to the results
metadata.bbox.minLat	float	The minimum Latitude of bounding box of the instruments contributing to the results
metadata.bbox.minLon	float	The minimum Longitude of the bounding box of the instruments contributing to the results
sensorData	list	A list of sensor dictionaries
sensorData[].acutalSamples	int	The number of samples in the data list
sensorData[].data	list	A list of observation dictionaries
sensorData[].data[].qaqcFlag	int	The QAQC Flag for the observation
sensorData[].data[].sampleTime	datetime string	The recorded time of the observation Date Time format: yyyy-MM-dd'T'HH:mm:ss.SSS'Z' datetime is represented in Coordinated Universal Time (UTC)
sensorData[].data[].value	float	The value of the observation
sensorData[].sensor	string	The code of the sensor that is being observed
sensorData[].sensorName	string	The name of the sensor that is being observed
sensorData[].unitOfMeasure	string	The unit of measure of the observation
serviceMetadata	dictionary	A dictionary of the metadata for the service request
serviceMetadata.metadata	string	The requested metadata filter
serviceMetadata.nextDataFrom	datetime string	The begin date for next reading in yyyy-MM-ddTHH:mm:ss.SSSZ format
serviceMetadata.outputFormat	string	The requested JSON output format
serviceMetadata.rowLimit	int	The requested row limit per sensor
serviceMetadata.sensors	string	A comma separated list of the requested sensors
serviceMetadata.totalActualSamples	int	The number of observations expected for all of the sensors
serviceMetadata.begin	datetime string	The begin date of the request in yyyy-MM-ddTHH:mm:ss.SSSZ format
serviceMetadata.end	datetime string	The end date of the request in yyyy-MM-ddTHH:mm:ss.SSSZ format
serviceMetadata.deviceCategoryCode	string	The Device Category Code of the request. See Available Device Categories for further details
serviceMetadata.locationCode	string	The Location Code of the request. See Available Locations for further details.

Example - Print the last raw instrument reading from AIS Reviever at the Underwater Listening Station at IONA

from onc.onc import ONC
onc = ONC('YOUR_TOKEN_HERE')
 
payload = onc.getDirectRaw({'locationCode':'IONA','deviceCategoryCode':'AISRECEIVER'},1)


data = payload['data']
if (data):
    for d in data:
        print ("  Raw Data = '{}' ; Sample Time = {}".format(d['rawData'],d['sampleTime']))


metadata = payload['metadata']
if(metadata):
    dataMetadata=metadata['dataMetadata']
    if(dataMetadata):
        print('{} samples'.format(dataMetadata['numberOfData']))

Example - Print 1 hour of raw CTD readings from Barkley Canyon Axis

from onc.onc import ONC
onc = ONC('YOUR_TOKEN_HERE')
 
payload = onc.getDirectRaw({'locationCode':'BACAX','deviceCategoryCode':'CTD','begin':'2017-05-23T00:00:00.000Z','end':'2017-05-23T01:00:00.000Z'})


data = payload['data']
if (data):
    for d in data:
        print ("  Raw Data = '{}' ; Sample Time = {}".format(d['rawData'],d['sampleTime']))


metadata = payload['metadata']
if(metadata):
    dataMetadata=metadata['dataMetadata']
    if(dataMetadata):
        print('{} samples'.format(dataMetadata['numberOfData']))

Page tree

Python Client Library

Requires

Installation

History

Classes

ONC Class

Description

Constructor

ONC()

Usage

Discovery methods

getLocations()

getDevices()

getDeviceCategories()

getProperties()

getDataProducts()

Data Product Download methods

orderDataProduct()

requestDataProduct()

runDataProduct()

downloadDataProduct()

getDataProdutUrls()

downloadFile()

getJsonFromUrl()

Near Real-Time data product methods

getDirectScalar()

getDirectRaw()