<script src="https://wiki.oceannetworks.ca/download/attachments/75170291/docs-client-libraries.js"></script>

These methods allow users to directly download previously generated data product files from our archive.

ONC systems auto-generate and archive files of different types at set time intervals; These archived files can be downloaded without waiting for a generation process to finish (potentially faster than Data product download methods).

Archived files have a unique filename (i.e. "NAXYS_HYD_007_20091231T235919.476Z-spect.png") that includes the device code ("NAXYS_HYD_007") and the UTC date-time when the data in the file started being measured ("20091231T235919.476Z"). The filename might contain additional information.


Due to security regulations, some very recent files (i.e. hydrophone .wav files in the last hour) might not be made immediately available

Summary

Method

Description

Archive file download methods (filters, allPages)

Get a list of archived files for a device category in a location

Archive file download methods (filters, allPages)

Get a list of archived files for a device

Archive file download methods (filename, overwrite)

Download a file with the given filename
Archive file download methods (filters, overwrite, allPages)Download a list of archived files that match the filters provided


getListByLocation 

Obtains a ^list of archived files for a device category in a location.

The filenames obtained can be used to download files using the getFile() method.

Parameter

Type

Description

Example

filters^filters

A group of filters used to specify the data origin, to be provided to the getListByLocation method from the API archivefiles service.

The following filters are mandatory:

The following filters are optional:

  • dateFrom
  • dateTo
  • dateArchivedFrom
  • dateArchivedTo
  • returnOptions
  • rowLimit
  • page
  • getLatest
  • extension

The "extension" filter allows you to limit the results to filenames that have the given file extension (i.e. "mp3", "wav", "mat", etc.). This filter is only supported by the client library.

Visit the getListByLocation method for more information on filter usage.


{
'locationCode': 'BACAX',
'deviceCategoryCode': 'ADCP2MHZ',
'dateFrom': '2016-07-27T00:00:00.000Z',
'dateTo': '2016-08-01T00:00:00.000Z'
}
{
'locationCode', 'BACAX',
'deviceCategoryCode', 'ADCP2MHZ',
'dateFrom', '2016-07-27T00:00:00.000Z',
'dateTo', '2016-08-01T00:00:00.000Z'
}
list(
"locationCode" = "BACAX",
"deviceCategoryCode" = "ADCP2MHZ",
"dateFrom" = "2016-07-27T00:00:00.000Z",
"dateTo" = "2016-08-01T00:00:00.000Z"
)


allPages

boolean

Whether this method should automatically download all pages when the data requested exceeds the row limit.

  • ^False: Only download the first page of data
  • ^True: If required, keep downloading all pages of data until all the requested data is obtained.

Default value: ^False

^False 

(parameters with an underline are required)


Example: Get a list of archived files for a device category in a location



from onc.onc import ONC
onc = ONC('YOUR_TOKEN_HERE')

filters = {
    'locationCode':'BACAX',
    'deviceCategoryCode':'ADCP2MHZ',
    'dateFrom':'2016-07-27T00:00:00.000Z',
    'dateTo':'2016-08-01T00:00:00.000Z',
}
result = onc.getListByLocation(filters, allPages=True)

onc.print(result)




onc = Onc(YOUR_TOKEN_HERE);

filters = {
    'locationCode',       'BACAX',
    'deviceCategoryCode', 'ADCP2MHZ',
    'dateFrom',           '2016-07-27T00:00:00.000Z',
    'dateTo',             '2016-08-01T00:00:00.000Z'
};
result = onc.getListByLocation(filters, 'allPages', true);

onc.print(result);




library(onc)
onc <- Onc(YOUR_TOKEN_HERE)

filters = list(
    "locationCode"       = "BACAX",
    "deviceCategoryCode" = "ADCP2MHZ",
    "dateFrom"           = "2016-07-27T00:00:00.000Z",
    "dateTo"             = "2016-08-01T00:00:00.000Z"
);
result = onc$getListByLocation(filters, "allPages"=TRUE);

onc$print(result)




Returns

A ^dictionary that includes a list of the filenames of the archived files found, as in the following example:


{
    "next": null,
    "queryUrl": "https://data.oceannetworks.ca/api/archivefiles?(...)",
    "files": [
        "BC_POD1_AD2M_20160727T000000.000Z.txt",
        (...)
    ]
}



struct(
    next: []
    queryUrl: 'https://data.oceannetworks.ca/api/archivefiles?(...)'
    files: {
        'BC_POD1_AD2M_20160727T000000.000Z.txt',
        ()
    }
)



[
  next: NULL,
  queryUrl: "https://data.oceannetworks.ca/api/archivefiles?(...)",
  files: [
    "BC_POD1_AD2M_20160727T000000.000Z.txt",
    (...)
  ]
]


 

By default, only the filename is returned for each file. Setting the "returnOptions" filter allows you to increase the information returned for each file in the "files" list, as shown in the examples below (warning: this might noticeably increase the request size for long lists):



"files": [
    {
        "filename": "MYDEV_20160727T(...).txt",
        "dateFrom": "2016-07-27T00:00:00.000Z",
        "archiveLocation": "/ONC_Archive",
        "path": "15/32/19"
    },
    (...)
]



files: [
    struct(
        filename: 'MYDEV_20160727T(...).txt'
        dateFrom: '2016-07-27T00:00:00.000Z'
        archiveLocation: '/ONC_Archive'
        path: '15/32/19'
    ),
    (...)
]



files: [
  [
    filename: "MYDEV_20160727T(...).txt",
    dateFrom: "2016-07-27T00:00:00.000Z",
    archiveLocation: "/ONC_Archive",
    path: "15/32/19"
  ],
  (...)
]





"files": [
    {
        "filename": "MYDEV_20160727T000000.000Z.txt",
        "dateFrom": "2016-07-27T00:00:00.000Z",
        "dateTo": "2016-07-28T00:00:00.000Z",
        "dataProductCode": "LF",
        "deviceCode": "BC_POD1_AD2M",
        "archivedDate": "2016-08-24T21:51:42.000Z",
        "modifyDate": "2016-08-24T21:51:48.000Z",
        "fileSize": 6269663,
        "uncompressedFileSize": 33359040,
        "compression": "gz"
    },
    (...)
]



files: [
    struct(
        filename: 'BC_POD1_AD2M_20160727T(...).txt'
        dateFrom: '2016-07-27T00:00:00.000Z'
        dateTo: '2016-07-28T00:00:00.000Z'
        dataProductCode: 'LF'
        deviceCode: 'BC_POD1_AD2M'
        archivedDate: '2016-08-24T21:51:42.000Z'
        modifyDate: '2016-08-24T21:51:48.000Z'
        fileSize: 6269663
        uncompressedFileSize: 33359040
        compression: 'gz'
    ),
    (...)
]



files: [
  [
    filename: "BC_POD1_AD2M_20160727T(...).txt",
    dateFrom: "2016-07-27T00:00:00.000Z",
    dateTo: "2016-07-28T00:00:00.000Z",
    dataProductCode: "LF",
    deviceCode: "BC_POD1_AD2M",
    archivedDate: "2016-08-24T21:51:42.000Z",
    modifyDate: "2016-08-24T21:51:48.000Z",
    fileSize: 6269663,
    uncompressedFileSize: 33359040,
    compression: "gz"
  ],
  (...)
]




If there are additional data pages to download, the "next" property will contain a ^dictionary with information to obtain the next page of data (and the corresponding URL) as illustrated in the following example:


{
    "next": {
        "parameters": {
            "method": "getListByLocation",
            "locationCode": "BACAX",
            "deviceCategoryCode": "ADCP2MHZ",
            "dateFrom": "2016-07-27T00:00:00.000Z",
            "dateTo": "2016-08-01T00:00:00.000Z",
            "token": "YOUR_TOKEN"
            "page": "2",
        },
        "url": "https://data.oceannetworks.ca/api/archivefiles?(...)"
    },
    (...)
}



struct(
    next: struct(
        parameters: struct(
            method: 'getListByLocation'
            locationCode: 'BACAX'
            deviceCategoryCode: 'ADCP2MHZ'
            dateFrom: '2016-07-27T00:00:00.000Z'
            dateTo: '2016-08-01T00:00:00.000Z'
            token: 'YOUR_TOKEN'
            page: '2'
        )
        url: 'https://data.oceannetworks.ca/api/archivefiles?(...)'
    )
    (...)
)



[
  next: [
    parameters: [
      method: "getListByLocation",
      locationCode: "BACAX",
      deviceCategoryCode: "ADCP2MHZ",
      dateFrom: "2016-07-27T00:00:00.000Z",
      dateTo: "2016-08-01T00:00:00.000Z",
      token: "YOUR_TOKEN",
      page: "2"
    ],
    url: "https://data.oceannetworks.ca/api/archivefiles?(...)"
  ],
  (...)
]




Returns a ^dictionary with the structure described below:

PropertyTypeDescription
queryUrlstringThe URL that obtained the results
files^list

A ^list of archived files found.

By default, this will be a list of strings with the filenames. However, if the filter "returnOptions" is set as "archiveLocation" or "all", this ^list will contain ^dictionaries instead.

next^dictionary

If the data requested exceeds the row limit, this element contains the parameters required to produce the next page of data and the corresponding URL (see below).

If this is the last / only page of data, "next" will be ^null | an empty list: [ ] | NULL.


If "returnOptions" is set to "archiveLocation", "files" will contain ^dictionaries with the following structure:

Property

Type

Description

Example

filenamestringThe name of the file"BC_POD2_JB_20090908T000000.000Z.txt"
dateFromstringThe beginning date & time for the data included in the file "2010-04-21T00:00:00.000Z"
archiveLocationstringRoot location of the archived file"/ONC_Archive"
pathstringPath for this file in the ONC archive"14/38/22"

Additionally, if "returnOptions" is set to "all", "files" will contain ^dictionaries with the following structure:

Property

Type

Description

Example

filenamestringThe name of the file"BC_POD2_JB_20090908T000000.000Z.txt"
dataProductCodestringThe data product code that generates this file"LF"
deviceCodestringDevice code for the origin of this file"BC_P0D1_AD2M"
dateFromstringThe beginning of the data time range included in this file"2010-04-21T00:00:00.000Z"
dateTostringThe end of the data time range included in this file"2010-04-22T00:00:00.000Z"
archivedDatestringThe UTC timestamp when the file was archived"2010-04-21T00:00:00.000Z"
modifyDatestringThe last timestamp of any modification related to the file"2010-04-22T01:55:00.000Z"
fileSizeintFile size (bytes) in the archive after compression6349499
uncompressedSizeintOriginal file size (bytes) before compression156545
compressionstringThe file extension if the file is compressed in the archive"gz"



getListByDevice

Obtains a list of archived files from a specific device.

The filenames obtained can be used to download files using the getFile() method.

Parameter

Type

Description

Example

filters^filters

A group of filters used to configure the data origin, to be provided to the getListByDevice method from the API archivefiles service.

The filter deviceCode is mandatory and sets the data source.

The following filters are optional:

  • dateFrom
  • dateTo
  • dateArchivedFrom
  • dateArchivedTo
  • returnOptions
  • rowLimit
  • page
  • getLatest
  • extension

The "extension" filter allows you to limit the results to filenames that have the given file extension (i.e. "mp3", "wav", "mat", etc.). This filter is provided by the client library (not the API).

Visit the getListByDevice method for more information on filter usage.


{
'deviceCode': 'NAXYS_HYD_007',
'dateFrom': '2010-01-01T00:00:00.000Z',
'dateTo': '2010-01-01T00:00:10.000Z'
}
{
'deviceCode', 'NAXYS_HYD_007',
'dateFrom', '2010-01-01T00:00:00.000Z',
'dateTo', '2010-01-01T00:00:10.000Z'
}
list(
"deviceCode" = "NAXYS_HYD_007",
"dateFrom" = "2010-01-01T00:00:00.000Z",
"dateTo" = "2010-01-01T00:00:10.000Z"
)


allPages

boolean

Whether this method should automatically download all pages when the data requested exceeds the row limit.

  • ^False: Only download the first page of data
  • ^True: If required, keep downloading all pages of data until all the requested data is obtained.

Default value: ^False

^False 

(parameters with an underline are required)


Example: Get a list of archived files in a device



from onc.onc import ONC
onc = ONC('YOUR_TOKEN_HERE')

filters = {
    'deviceCode': 'NAXYS_HYD_007',
    'dateFrom'  : '2010-01-01T00:00:00.000Z',
	'dateTo'    : '2010-01-01T00:00:10.000Z'
}
result = onc.getListByDevice(filters, allPages=True)

onc.print(result)




onc = Onc(YOUR_TOKEN_HERE);

filters = {
    'deviceCode', 'NAXYS_HYD_007',
    'dateFrom',   '2010-01-01T00:00:00.000Z',
	'dateTo',     '2010-01-01T00:00:10.000Z'
};
result = onc.getListByDevice(filters, 'allPages', true);

onc.print(result);




library(onc)
onc <- Onc(YOUR_TOKEN_HERE)

filters = list(
    "deviceCode" = "NAXYS_HYD_007",
    "dateFrom"   = "2010-01-01T00:00:00.000Z",
	"dateTo"     = "2010-01-01T00:00:10.000Z"
)
result = onc$getListByDevice(filters, "allPages" = TRUE)

onc$print(result)




Returns

A ^dictionary with a list of the filenames of the archived files found for the filters provided.


{
    "next": null,
    "queryUrl": "https://data.oceannetworks.ca/api/archivefiles?(...)",
    "files": [
        "NAXYS_HYD_007_20091231T235919.476Z.wav",
        (...)
    ]
}



struct(
    next: []
    queryUrl: 'https://data.oceannetworks.ca/api/archivefiles?(...)'
    files: {
        'NAXYS_HYD_007_20091231T235919.476Z.wav',
        (...)
    }
)



[
  next: NULL,
  queryUrl: "https://data.oceannetworks.ca/api/archivefiles?(...)",
  files: [
    "NAXYS_HYD_007_20091231T235919.476Z.wav",
    (...)
  ]
]



By default, only a string filename is returned for each file. Setting the "returnOptions" filter allows you to increase the information returned for each file in the "files" list, in the same way as explained for the getListByLocation method.


The method returns a ^dictionary with the structure described below.

PropertyTypeDescription
queryUrlstringThe URL that obtained the results
files^list

A ^list of archived files found.

By default, this will be a list of strings with the filenames. However, if the filter "returnOptions" is set as "archiveLocation" or "all", this ^list will contain ^dictionaries instead.

next^dictionary

If the data requested exceeds the row limit, this element contains the parameters required to produce the next page of data and the corresponding URL (see below).

If this is the last / only page of data, "next" will be ^null | an empty list: [ ] | NULL.


If "returnOptions" is set to "archiveLocation", "files" will contain ^dictionaries with the following structure:

Property

Type

Description

Example

filenamestringThe name of the file"BC_POD2_JB_20090908T000000.000Z.txt"
dateFromstringThe beginning date & time for the data included in the file "2010-04-21T00:00:00.000Z"
archiveLocationstringRoot location of the archived file"/ONC_Archive"
pathstringPath for this file in the ONC archive"14/38/22"

Additionally, if "returnOptions" is set to "all", "files" will contain ^dictionaries with the following structure:

Property

Type

Description

Example

filenamestringThe name of the file"BC_POD2_JB_20090908T000000.000Z.txt"
dataProductCodestringThe data product code that generates this file"LF"
deviceCodestringDevice code for the origin of this file"BC_P0D1_AD2M"
dateFromstringThe beginning of the data time range included in this file"2010-04-21T00:00:00.000Z"
dateTostringThe end of the data time range included in this file"2010-04-22T00:00:00.000Z"
archivedDatestringThe UTC timestamp when the file was archived"2010-04-21T00:00:00.000Z"
modifyDatestringThe last timestamp of any modification related to the file"2010-04-22T01:55:00.000Z"
fileSizeintFile size (bytes) in the archive after compression6349499
uncompressedSizeintOriginal file size (bytes) before compression156545
compressionstringThe file extension if the file is compressed in the archive"gz"



getFile

Downloads a file from the archive.

Parameter

Type

Description

Example

filenamestring

The filename for the file requested.

All archived files have a unique filename.

"NAXYS_HYD_007_20091231T235919.476Z.wav"

overwrite

boolean

Whether new files downloaded will overwrite previous files with the same filename found in the output directory.

  • ^True: Overwrite files with the same filename
  • ^False: Do not overwrite files (downloaded files with the same filename will be skipped)

Default: ^False

^False 

(parameters with an underline are required)


Example: Download a file from the archive



from onc.onc import ONC
onc = ONC('YOUR_TOKEN_HERE')

result = onc.getFile('NAXYS_HYD_007_20091231T235919.476Z.wav')

onc.print(result)




onc = Onc(YOUR_TOKEN_HERE);

result = onc.getFile('NAXYS_HYD_007_20091231T235919.476Z.wav');

onc.print(result);




library(onc)
onc <- Onc(YOUR_TOKEN_HERE)

result = onc$getFile("NAXYS_HYD_007_20091231T235919.476Z.wav")

onc$print(result)




Returns

A ^dictionary with information on the file download result.


{
    "file": "NAXYS_HYD_007_20091231T235919.476Z.wav",
    "status": "completed",
    "size": 10240044,
    "downloadTime": 1.535,
    "url": "https://data.oceannetworks.ca/api/archivefiles?(...)"
}



struct(
    file: 'NAXYS_HYD_007_20091231T235919.476Z.wav'
    status: 'completed'
    size: 10240044
    downloadTime: 1.595
    url: 'https://data.oceannetworks.ca/api/archivefiles?(...)'
)



[
  file: "NAXYS_HYD_007_20091231T235919.476Z.wav",
  status: "completed",
  size: 10240044,
  downloadTime: 1.34,
  url: "https://data.oceannetworks.ca/api/archivefiles?(...)"
]




The response is a ^dictionary with the following structure:

Property

Type

Description

urlstringThe URL used to download the file
statusstring

The status of the download process for this file. Possible values are:

  • complete: The file was downloaded
  • error: The file was not downloaded due to an error
  • skipped: The file was not downloaded because a file with the same name already exists in the output path and "overwrite" is set to ^False
filestringThe filename of the file if it was downloaded (otherwise, an empty string)
sizeintThe size of the downloaded file in bytes (0 if it was not downloaded)
downloadTimefloatThe time duration (in seconds) spend downloading this file (0 if it was not downloaded)



getDirectFiles

Downloads all archived files that match the filters specified. Will also print progress messages to the console.

Internally, this function invokes the method Archive file download methods() or Archive file download methods() to obtain a list of files, and the method Archive file download methods() to download all files found.

Parameter

Type

Description

Example

filters^filters

A group of filters used to configure the request.

It is mandatory to provide either of the following filters to identify the data origin:

The following filters are optional:

  • dateFrom
  • dateTo
  • dateArchivedFrom
  • dateArchivedTo
  • returnOptions
  • rowLimit
  • page
  • getLatest
  • extension

The "extension" filter allows you to limit the results to filenames that have the given file extension (i.e. "mp3", "wav", "mat", etc.). This filter is provided by the client library (not in the API).

Visit getListByLocation and getListByDevice for more information on filter usage.


{
'deviceCode': 'NAXYS_HYD_007',
'dateFrom': '2010-01-01T00:00:00.000Z',
'dateTo': '2010-01-01T00:00:10.000Z'
}
{
'deviceCode', 'NAXYS_HYD_007',
'dateFrom', '2010-01-01T00:00:00.000Z',
'dateTo', '2010-01-01T00:00:10.000Z'
}
list(
"deviceCode" = "NAXYS_HYD_007",
"dateFrom" = "2010-01-01T00:00:00.000Z",
"dateTo" = "2010-01-01T00:00:10.000Z"
)


overwrite

boolean

Whether new files downloaded will overwrite previous files with the same filename found in the output directory.

  • ^True: Overwrite files with the same filename
  • ^False: Do not overwrite files (downloaded files with the same filename will be skipped)

Default: ^False

^False 

allPages

boolean

Whether this method should automatically download all pages when the data requested exceeds the row limit.

  • ^False: Only download the first page of data
  • ^True: If required, keep downloading all pages of data until all the requested data is obtained.

Default value: ^False

^False 

(parameters with an underline are required)


Example: Download all archived files from a device in a time range



from onc.onc import ONC
onc = ONC('YOUR_TOKEN_HERE')

filters = {
    'deviceCode':'NAXYS_HYD_007',
    'dateFrom': '2010-01-01T00:00:00.000Z',
	'dateTo'  : '2010-01-01T00:00:10.000Z'
}
result = onc.getDirectFiles(filters, allPages=True)

onc.print(result)




onc = Onc(YOUR_TOKEN_HERE);

filters = {
    'deviceCode', 'NAXYS_HYD_007',
    'dateFrom',   '2010-01-01T00:00:00.000Z',
	'dateTo',     '2010-01-01T00:00:10.000Z'
};
result = onc.getDirectFiles(filters, 'allPages', true);

onc.print(result);




library(onc)
onc <- Onc(YOUR_TOKEN_HERE)

filters = list(
    "deviceCode" = "NAXYS_HYD_007",
    "dateFrom"   = "2010-01-01T00:00:00.000Z",
	"dateTo"     = "2010-01-01T00:00:10.000Z"
)
result = onc$getDirectFiles(filters, "allPages" = TRUE)

onc$print(result)




Returns

A ^dictionary with information on the file download results and aggregated statistics on the full download process, as in the following example:


{
    "downloadResults": [
        {
            "file": "NAXYS_HYD_007_20091231T235919.476Z.wav"
            "status": "completed",
            "size": 10240044,
            "downloadTime": 1.223,
            "url": "https://data.oceannetworks.ca/api/archivefiles?(...)",
        },
        (...)
    ],
    "stats": {
        "totalSize": 10821184,
        "downloadTime": 1.743,
        "fileCount": 6
    }
}



struct(
    downloadResults: [
        struct(
            file: 'NAXYS_HYD_007_20091231T235919.476Z.wav'
            status: 'completed'
            size: 10821184
            downloadTime: 1.403
            url: 'https://data.oceannetworks.ca/api/archivefiles?(...)'
        ),
        (...)
    ]
    stats: struct(
        totalSize: 581140
        downloadTime: 1.298
        fileCount: 6
    )
)



[
  downloadResults: [
    [
      file: "NAXYS_HYD_007_20091231T235919.476Z.wav",
      status: "completed",
      size: 10240044,
      downloadTime: 1.193,
      url: "https://data.oceannetworks.ca/api/archivefiles?(...)"
    ],
    (...)
  ],
  stats: [
    totalSize: 581140,
    downloadTime: 1.456,
    fileCount: 6
  ]
]


Downloaded files with a "status" of "completed" should appear in the ONC object's "outPath"  directory ("output" by default).


The response is a ^dictionary with the following structure:

Properties

Type

Description

downloadResults^listA ^list of ^dictionaries, with the download results for each file (see below)
stats^dictionaryContains aggregated metrics of the complete download process

stats.totalSize

stringThe total size of all files downloaded (in bytes)

stats.downloadTime

intThe time duration (in seconds) spend downloading all files

stats.fileCount

floatThe number of files downloaded

The "downloadResults" ^list contains ^dictionaries with the download results for each file, with the following structure:

Properties

Type

Description

urlstringThe URL used to download the file
statusstring

The status of the download process for this file. Possible values are:

  • complete: The file was downloaded
  • error: The file was not downloaded due to an error
  • skipped: The file was not downloaded because a file with the same name already exists in the output path and "overwrite" is ^False
filestringThe filename of the file if it was downloaded (otherwise, an empty string)
sizeintThe size of the downloaded file in bytes (0 if it was not downloaded)
downloadTimefloatThe time duration (in seconds) spend downloading this file (0 if it was not downloaded)