Near real-time (as fast as they get into our database) data access methods allow the extraction of sensor data as time-series, either as processed scalar data with Quality Assurance and Control flags (QAQC) or directly as raw data obtained from the device in its specific output format. In contrast to the Data product download methods, this data can be downloaded directly without waiting for any kind of generation process.
Common use cases include:
- Plotting time-series from properties in a specific timeframe or in "near real-time"
- Quickly obtaining the latest reading from a particular sensor
- Obtaining raw unprocessed data from our instruments (data might require processing to be readable)
The methods getDirectByLocation() and getDirectRawByLocation() obtain data readings from a location without knowing which device it came from (hence the need to specify a "device category code" instead of a single "device code"). You might want to obtain data by location instead of by device, as individual devices are often replaced/repositioned. Each request to our API can return a maximum of 100,000 samples; larger data requests must be downloaded as a sequence of "pages". Use the "allPages" parameter to automatically download all pages required for your requested timeframe. |
Summary
Method | Description |
|---|
getDirectByLocation (filters, allPages)
| Obtain scalar data readings from a device category in a location | link |
getDirectByDevice (filters, allPages) | Obtain scalar data readings from a device | link |
getDirectRawByLocation (filters, allPages)
| Obtain raw data readings from a device category in a location | link |
getDirectRawByDevice (filters, allPages)
| Obtain raw data readings from a device | link |
getDirectByLocation
(Previously called getDirectScalar)
Downloads time-series of property readings from a device category in a location. The returned data contains a value, time and QAQC flag for every measurement in the time-series.
Parameter | Type | Description | Example |
|---|
| filters | dictionary | A dictionary of filters used to configure the data product request, to be provided to the getByLocation method from the API scalardata service. The following filters are required and will set the data origin: The following filters are optional: Visit the getByLocation method documentation for information on filter usage. | {
'locationCode':'TWDP',
'deviceCategoryCode':'TSG',
'dateFrom':'2016-09-01T00:00:00.000Z',
'dateTo':'2016-09-01T08:00:00.000Z'
}
|
allPages | boolean | Whether this method should automatically download all pages when the data requested exceeds the row limit and requires to be downloaded in multiple pages. - False: Only download the first page of data
- True: If required, keep downloading all pages of data until all the requested data is obtained.
Default value: False | True |
(parameters with an underline are required)
from onc.onc import ONC
onc = ONC('YOUR_TOKEN_HERE')
filters = {
'locationCode': 'TWDP', # Tswwassen DP Ferry
'deviceCategoryCode':'TSG', # Thermosalinograph
'dateFrom':'2016-09-01T00:00:00.000Z',
'dateTo':'2016-09-01T00:01:00.000Z'
}
result = onc.getDirectByLocation(filters, allPages=True)
onc.print(result) |
|
Returns
A dictionary with separate lists of sensor readings for each property, as returned by the scalardata getByLocation method. By default, each property in the response includes a "data" dictionary with matching lists for "values", "sampleTimes" and "qaqcFlags". {
"queryUrl": "https://data.oceannetworks.ca/api/scalardata?locationCode=TWDP&(...)",
"next": null,
"sensorData": [
{
"sensorCategoryCode": "conductivity",
"sensorCode": "Conductivity",
"sensorName": "Conductivity",
"unitOfMeasure": "S/m",
"data": {
"values": [3.442, 3.448, 3.448, 3.447],
"sampleTimes": [
"2016-09-01T00:00:07.037Z",
"2016-09-01T00:00:17.041Z",
"2016-09-01T00:00:27.046Z",
"2016-09-01T00:00:37.040Z"
],
"qaqcFlags": [1,1,1,1]
}
}
]
} |
If the request filters include the "outputFormat" parameter set to "object", the data in "sensorData" for each sensor will be represented as a single list of self-contained objects as in the following example (Warning: This might noticeably increase the download size in long time-series): {
"queryUrl": "https://data.oceannetworks.ca/api/scalardata?locationCode=TWDP&(...)",
"next": null,
"sensorData": [
{
"data": [
{
"qaqcFlag": 1,
"sampleTime": "2016-09-01T00:00:07.037Z",
"value": 28.9096
},
{
"qaqcFlag": 1,
"sampleTime": "2016-09-01T00:00:17.041Z",
"value": 28.9114
},
{
"qaqcFlag": 1,
"sampleTime": "2016-09-01T00:00:27.046Z",
"value": 28.8941
},
# (...)
]
}
]
} |
If the request filters include the "metadata" parameter set to "full", the response will also include a "metadata" dictionary as in the following example: {
"queryUrl": "https://data.oceannetworks.ca/api/scalardata?locationCode=TWDP&(...)",
"next": null,
"sensorData": # (...),
"metadata": {
"boundingBox": {
"maxDepth": 3.0,
"maxLat": 49.004607,
"maxLon": -123.185037,
"minDepth": 3.0,
"minLat": 49.004607,
"minLon": -123.185037
},
"depth": 3.0,
"deviceCategoryCode": "TSG",
"lat": 49.004607,
"locationName": "Tsawwassen - Duke Point",
"lon": -123.185037
}
} |
If there are additional data pages to download, the "next" property will contain a dictionary with information to obtain the next page of data as in the following example: {
"queryUrl": "https://data.oceannetworks.ca/api/scalardata?locationCode=TWDP&(...)",
"next": {
"locationCode": "TWDP",
"deviceCategoryCode": "TSG",
"dateFrom": "2016-09-01T00:00:37.040Z",
"dateTo": "2016-09-01T00:01:00.000Z",
"token": "YOUR_TOKEN",
"method": "getByLocation"
},
"sensorData": # (...)
} |
|
The response is a dictionary with the following keys: | Key | Type | Description |
|---|
| queryUrl | string | The URL that obtained the results | | sensorData | list | A list of dictionaries, one for each property obtained, with the data readings and sensor information | | metaData | dictionary | A dictionary with general information from the instruments that contribute to the data readings returned. Only included when the "metadata" request parameter has been set to "full" | | next | dictionary | If the data requested exceeds the row limit, this dictionary contains the parameters required to produce the next page of data. If this is the last page of data, "next" will be null. |
Each "property" in the sensorData list is a dictionary with the following keys: | Key | Type | Description |
|---|
| sensorCategoryCode | float | The property code for this property | | sensorCode | string | The code of the sensor that produced the property | | sensorName | string | The name of the sensor that produced the property | | unitOfMeasure | string | The unit of measure for the property | | data | dictionary (or list) | By default, a dictionary with the readings for the property. The data for each sample is distributed into 3 lists of matching indexes: "values", "sampleTimes" and "qaqcFlags". If the request filters include the "outputFormat" parameter set to "object", the "data" key will instead hold a list of self-contained dictionaries (one per sample), each with its own "value", "sampleTime" and "qaqcFlag" as explained in the scalardata getByLocation method documentation. | data.values | list | List of values (float) | data.sampleTimes | list | List with the sampling time for each corresponding element in the "values" list | data.qaqcFlags | list | List with the QAQC flag for each corresponding element in the "values" list |
When the "next" dictionary is not null (more data pages available), it will include the following keys: | Key | Type | Description |
|---|
| url | string | The URL to obtain the next data page | | parameters | dictionary | A complete dictionary of the filters used to build the "url" to the next data page. Will usually contain exactly the same parameters provided in the original request, with the following exceptions: - "
dateFrom" will be modified to match the start of the next page of data - If "
dateTo" was provided as a relative date, it will be translated to an absolute date - In addition, includes the user "
token" and the API "method" consumed ("getByLocation")
| | next | dictionary | If the data requested exceeds the row limit, this dictionary contains the parameters required to produce the next page of data. If this is the last page of data, "next" will be null. |
|
getDirectByDevice
Downloads time-series of property readings from a particular device. The returned data contains a value, time and QAQC flag for every measurement in the time-series.
Parameter | Type | Description | Example |
|---|
| filters | dictionary | A dictionary of filters used to configure the data product request, to be provided to the getByDevice method from the API scalardata service. The deviceCode filter is required and will set the data origin. The following filters are optional: Visit the getByDevice method documentation for information on filter usage. | {
'deviceCode': 'AMLMETRECX50348',
'dateFrom':'2019-06-01T00:00:00.000Z',
'dateTo':'2019-06-01T00:00:10.000Z'
}
|
allPages | boolean | Whether this method should automatically download all pages when the data requested exceeds the row limit and requires to be downloaded in multiple pages. - False: Only download the first page of data
- True: If required, keep downloading all pages of data until all the requested data is obtained.
Default value: False | True |
(parameters with an underline are required)
from onc.onc import ONC
onc = ONC('YOUR_TOKEN_HERE')
filters = {
'deviceCode': 'AMLMETRECX50348',
'dateFrom':'2019-06-01T00:00:00.000Z',
'dateTo':'2019-06-01T00:00:10.000Z'
}
result = onc.getDirectByDevice(filters)
onc.print(result) |
|
Returns
A dictionary with separate lists of sensor readings for each property, as returned by the scalardata getByDevice method. By default, each property in the response includes a "data" dictionary with matching lists for "values", "sampleTimes" and "qaqcFlags". {
"queryUrl": "https://data.oceannetworks.ca/api/scalardata?method=getByDevice&(...)",
"next": null,
"sensorData": [
{
"sensorCategoryCode": "conductivity",
"sensorCode": "Conductivity",
"sensorName": "Conductivity",
"unitOfMeasure": "S/m",
"data": {
"values": [3.442, 3.448, 3.448, 3.447],
"sampleTimes": [
"2016-09-01T00:00:07.037Z",
"2016-09-01T00:00:17.041Z",
"2016-09-01T00:00:27.046Z",
"2016-09-01T00:00:37.040Z"
],
"qaqcFlags": [1,1,1,1]
}
}
]
} |
If the request filters include the "outputFormat" parameter set to "object", the data in "sensorData" for each sensor will be represented as a single list of self-contained objects in the same way as explained for the getDirectByLocation method response (Warning: This might noticeably increase the download size in long time-series). If the request filters include the "metadata" parameter set to "full", the response will also include a "metadata" dictionary in the same way as explained for the getDirectByLocation method response. If there are additional data pages to download, the "next" property will contain a dictionary with information to obtain the next page of data in the same way as explained for the getDirectByLocation method response. |
The response is a dictionary with the following keys: | Key | Type | Description |
|---|
| queryUrl | string | The URL that obtained the results | | sensorData | list | A list of dictionaries, one for each property obtained, with the data readings and sensor information | | metaData | dictionary | A dictionary with general information from the instruments that contribute to the data readings returned. Only included when the "metadata" request parameter has been set to "full" | | next | dictionary | If the data requested exceeds the row limit, this dictionary contains the parameters required to produce the next page of data. If this is the last page of data, "next" will be null. |
Each "property" in the sensorData list is a dictionary with the following keys: | Key | Type | Description |
|---|
| sensorCategoryCode | float | The property code for this property | | sensorCode | string | The code of the sensor that produced the property | | sensorName | string | The name of the sensor that produced the property | | unitOfMeasure | string | The unit of measure for the property | | data | dictionary (or list) | By default, a dictionary with the readings for the property. The data for each sample is distributed into 3 lists of matching indexes: "values", "sampleTimes" and "qaqcFlags". If the request filters include the "outputFormat" parameter set to "object", the "data" key will instead hold a list of self-contained dictionaries (one per sample), each with its own "value", "sampleTime" and "qaqcFlag" as explained in the scalardata getByLocation method documentation. | data.values | list | List of values (float) | data.sampleTimes | list | List with the sampling time for each corresponding element in the "values" list | data.qaqcFlags | list | List with the QAQC flag for each corresponding element in the "values" list |
When the "next" dictionary is not null (more data pages available), it will include the following keys: | Key | Type | Description |
|---|
| url | string | The URL to obtain the next data page | | parameters | dictionary | A complete dictionary of the filters used to build the "url" to the next data page. Will usually contain exactly the same parameters provided in the original request, with the following exceptions: - "
dateFrom" will be modified to match the start of the next page of data - If "
dateTo" was provided as a relative date, it will be translated to an absolute date - In addition, includes the user "
token" and the API "method" consumed ("getByLocation")
| | next | dictionary | If the data requested exceeds the row limit, this dictionary contains the parameters required to produce the next page of data. If this is the last page of data, "next" will be null. |
|
getDirectRawByLocation
Downloads time-series of raw sensor readings from a device category in a location. The response will contain readings for all properties available in the devices providing the data.
Depending on the output format of the source devices, raw data might not be in a readable format without additional processing.
Parameter | Type | Description | Example |
|---|
| filters | dictionary | A dictionary of filters used to configure the data product request, to be provided to the rawdata getByLocation API method. The following filters are required and will set the data origin: The following filters are optional: Visit the getByLocation method documentation for information on filter usage. | {
'locationCode':'TWDP',
'deviceCategoryCode':'TSG',
'dateFrom':'2016-09-01T00:00:00.000Z',
'dateTo':'2016-09-01T08:00:00.000Z'
}
|
allPages | boolean | Whether this method should automatically download all pages when the data requested exceeds the row limit and requires to be downloaded in multiple pages. - False: Only download the first page of data
- True: If required, keep downloading all pages of data until all the requested data is obtained.
Default value: False | True |
(parameters with an underline are required)
from onc.onc import ONC
onc = ONC('YOUR_TOKEN_HERE')
filters = {
'locationCode': 'TWDP', # Tswwassen DP Ferry
'deviceCategoryCode':'TSG', # Thermosalinograph
'dateFrom':'2016-09-01T00:00:00.000Z',
'dateTo':'2016-09-01T00:01:00.000Z'
}
result = onc.getDirectRawByLocation(filters, allPages=True)
onc.print(result) |
|
Returns
A dictionary with separate lists of raw device output. Each data sample in the result is represented in matching lists for "readings", "times" and "lineTypes", as returned by the rawdata getByLocation method. By default, raw readings will be returned in a structure as presented in the following example: {
"data": {
"readings": [
"20160901-00:00:07.037, 12.9162, 3.44294, 28.9096",
"20160901-00:00:17.041, 12.9798, 3.44834, 28.9114",
"20160901-00:00:27.046, 13.0016, 3.44827, 28.8941",
"20160901-00:00:37.040, 13.0094, 3.44771, 28.8829",
"20160901-00:00:47.045, 13.0038, 3.44889, 28.8981"
],
"times": [
"2016-09-01T00:00:07.037Z",
"2016-09-01T00:00:17.041Z",
"2016-09-01T00:00:27.046Z",
"2016-09-01T00:00:37.040Z",
"2016-09-01T00:00:47.045Z"
],
"lineTypes": [" "," "," "," "]
},
"metadata": {
"locationName": "Tsawwassen - Duke Point"
},
"next": null,
"queryUrl": "https://data.oceannetworks.ca/api/rawdata?locationCode=TWDP&(...)"
} |
If the request filters include the "outputFormat" parameter set to "object", the data returned will be represented as a single list of self-contained objects as in the following example (Warning: This might noticeably increase the download size in long time-series): {
"data": [
{
"lineType": " ",
"rawData": "20160901-00:00:07.037, 12.9162, 3.44294, 28.9096",
"sampleTime": "2016-09-01T00:00:07.037Z"
},
{
"lineType": " ",
"rawData": "20160901-00:00:17.041, 12.9798, 3.44834, 28.9114",
"sampleTime": "2016-09-01T00:00:17.041Z"
},
{
"lineType": " ",
"rawData": "20160901-00:00:27.046, 13.0016, 3.44827, 28.8941",
"sampleTime": "2016-09-01T00:00:27.046Z"
},
(...)
],
"metadata": {
"locationName": "Tsawwassen - Duke Point"
},
"next": null,
"queryUrl": "https://data.oceannetworks.ca/api/rawdata?locationCode=TWDP&(...)"
} |
If there are additional data pages to download, the "next" property will contain a dictionary with information to obtain the next page of data, just as explained for the getDirectScalar method. |
The response is a dictionary with the following keys: | Key | Type | Description |
|---|
| queryUrl | string | The URL that obtained the results | | data | dictionary (or list) | By default, a dictionary with the raw readings obtained. The data for each reading is distributed into 3 lists of matching indexes: "readings", "times" and "lineTypes". If the request filters include the "outputFormat" parameter set to "object", the "data" key will instead hold a list of self-contained dictionaries (one per sample), each with its own "reading", "time" and "lineType" as explained in the rawdata getByLocation documentation. | data.readings | list | List of raw readings. The readings might be strings or integers depending on the raw data type. | data.times | list | List with the sampling time for each corresponding element in the "readings" list | data.lineTypes | list | List of (string) with the type of raw reading line for each corresponding element in the "readings" list. The line type " " matches data readings and ">" and "<" are used for command lines. | | metadata | dictionary | A dictionary with information on the data origin | metadata.locationName | string | The name of the location the data originates from | | next | dictionary | If the data requested exceeds the row limit, this dictionary contains the parameters required to produce the next page of data, as explained for the getDirectScalar method. If this is the last page of data, "next" will be null. |
|
getDirectRawByDevice
Downloads time-series of raw sensor readings from a unique device. The response will contain readings for all properties available in the device.
Depending on the output format of the device, raw data might not be in a readable format without additional processing (for more information, consult the device documentation in the Data Search tool).
Parameter | Type | Description | Example |
|---|
| filters | dictionary | A dictionary of filters used to configure the data product request, to be provided to the rawdata getByDevice API method. The filter deviceCode is required and will set the data origin. The following filters are optional: - dateFrom
- dateTo
- rowLimit
- sizeLimit
- convertHexToDecimal
- outputFormat
- getLatest
Visit the getByDevice method documentation for information on filter usage. | {
'deviceCode':'AMLMETRECX50348',
'dateFrom':'2019-06-01T00:00:00.000Z',
'dateTo':'2019-06-01T00:00:05.000Z'
}
|
allPages | boolean | Whether this method should automatically download all pages when the data requested exceeds the row limit and requires to be downloaded in multiple pages. - False: Only download the first page of data
- True: If required, keep downloading all pages of data until all the requested data is obtained.
Default value: False | True |
(parameters with an underline are required)
from onc.onc import ONC
onc = ONC('YOUR_TOKEN_HERE')
filters = {
'deviceCode':'AMLMETRECX50348', # A CTD in Burrard Inlet
'dateFrom':'2019-06-01T00:00:00.000Z',
'dateTo':'2019-06-01T00:00:05.000Z'
}
result = onc.getDirectRawByDevice(filters, allPages=False)
onc.print(result) |
|
Returns
A dictionary with separate lists of raw device output. Each data sample in the result is represented in matching lists for "readings", "times" and "lineTypes", as returned by the rawdata getByDevice method. By default, raw readings will be returned in a structure as presented in the following example: {
"data": {
"readings": [
"20160901-00:00:07.037, 12.9162, 3.44294, 28.9096",
"20160901-00:00:17.041, 12.9798, 3.44834, 28.9114",
"20160901-00:00:27.046, 13.0016, 3.44827, 28.8941",
"20160901-00:00:37.040, 13.0094, 3.44771, 28.8829",
"20160901-00:00:47.045, 13.0038, 3.44889, 28.8981"
],
"times": [
"2016-09-01T00:00:07.037Z",
"2016-09-01T00:00:17.041Z",
"2016-09-01T00:00:27.046Z",
"2016-09-01T00:00:37.040Z",
"2016-09-01T00:00:47.045Z"
],
"lineTypes": [" "," "," "," "]
},
"next": null,
"queryUrl": "https://data.oceannetworks.ca/api/rawdata?deviceCode=AMLMETRECX5034&(...)"
} |
If the request filters include the "outputFormat" parameter set to "object", the data returned will be represented as a single list of self-contained objects as in the following example (Warning: This might noticeably increase the download size in long time-series): {
"data": [
{
"lineType": " ",
"rawData": "20160901-00:00:07.037, 12.9162, 3.44294, 28.9096",
"sampleTime": "2016-09-01T00:00:07.037Z"
},
{
"lineType": " ",
"rawData": "20160901-00:00:17.041, 12.9798, 3.44834, 28.9114",
"sampleTime": "2016-09-01T00:00:17.041Z"
},
{
"lineType": " ",
"rawData": "20160901-00:00:27.046, 13.0016, 3.44827, 28.8941",
"sampleTime": "2016-09-01T00:00:27.046Z"
},
(...)
],
"next": null,
"queryUrl": "https://data.oceannetworks.ca/api/rawdata?deviceCode=AMLMETRECX5034&(...)"
} |
If there are additional data pages to download, the "next" property will contain a dictionary with information to obtain the next page of data, just as explained for the getDirectScalar method. |
The response is a dictionary with the following keys: | Key | Type | Description |
|---|
| queryUrl | string | The URL that obtained the results | | data | dictionary (or list) | By default, a dictionary with the raw readings obtained. The data for each reading is distributed into 3 lists of matching indexes: "readings", "times" and "lineTypes". If the request filters include the "outputFormat" parameter set to "object", the "data" key will instead hold a list of self-contained dictionaries (one per sample), each with its own "reading", "time" and "lineType" as explained in the rawdata getByLocation documentation. | data.readings | list | List of raw readings. The readings might be strings or integers depending on the raw data type. | data.times | list | List with the sampling time for each corresponding element in the "readings" list | data.lineTypes | list | List of (string) with the type of raw reading line for each corresponding element in the "readings" list. The line type " " matches data readings and ">" and "<" are used for command lines. | | next | dictionary | If the data requested exceeds the row limit, this dictionary contains the parameters required to produce the next page of data, just as explained for the getDirectScalar method. If this is the last page of data, "next" will be null. |
|