Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Migrated to Confluence 5.3

Data Search Help

Finding Data

Our Data Search lets you browse our large and ever-expanding data archive, choose data products, generate them and download them.

Quick Instructions

  1. Log in - You need to log in to search for data. If you haven't registered, it's easy to create a new account.
  2. Open ONC Data Search
  3. Choose a Data Source from the Data Source Selection panel.
    1. You can browse instruments by type, location, network topology, or project.
    2. Click an instrument to select it (a red marker will appear on the map)
  4. Select a Data Product
    1. Use the calendar icons to choose start and end times for the data you want to download.
    2. Choose data products for your instrument by selecting check-boxes in the product selection panel.
  5. Add to Cart
  6. Click Checkout; your data request will be processed for you to view and download.
Tips
  • Once you have clicked Process Data Products you can continue with new searches.
  • You can download multiple time periods for the same instrument.
  • Data collected during commissioning phase (in step 2 click Annotations link) is not guaranteed to meet rigorous quality standards, as data continuity and instrument configurations fluctuated during this time.

Table of Content

Table of Contents

Detailed Instructions

Step 1: Choose Data Source

When you first open Data Search, you see a map and a data source selection panel:

  1. Choose an instrument from the selection panel, and its location is displayed on the map.
  2. Hover your mouse over an instrument pin on the map to see latitude, longitude and depth.
  3. Click the node icons for details on the node location.

Instrument Selection Panel

The upper-left panel lets you browse the full list of instruments with data in our archive. Use the dropdown selector to display instruments by type, location, network topology or research project.

Refine the list of instruments using the Filter dropdown selector. This lets you choose water properties and other measured quantities of interest.

When you find the instrument you're looking for, select it by clicking on it.

Step 2: Selecting Data Products

Once you have selected an instrument, click the Step 2 button to choose:

  1. date and time periods
  2. data product types
  3. data product options (not shown in the illustration below)

Instrument Info

When you select an instrument, its current and past locations are marked on the map. Find details and documentation for the instrument via the links on Step 2.
 

Defining a Time Period


There are several options for defining the time period for your data request. You can click the calendar icons to choose start/end dates and enter hours and minutes in the time boxes. Or you can click the Custom... selector and choose time periods covering the last week, day, 2 hours or 10 minutes. The graph to the right shows data availability. Green shading indicates time periods for which data are available from your chosen instrument.


You can also zoom in on the data availability graph by clicking and dragging anywhere within the graph. In the above illustration, the initial data availability graph covers a period of time from 2009 into 2011. By clicking an dragging on a small segment of that graph, the availability graph was zoomed to cover only the time period between November 2009 - January 2010.

Unzoom to the full extent by double-clicking on the graph.

Selecting Data Product Formats and Options


Various data product formats, such as MatLab 7, CSV,PNG and PDF plot, and Raw log file are available for your chosen instrument and its sensors. Further details on individual products can be obtained here. If you would like to see additional data products, please let us know.

Click the check-box to choose the data product format you want.

On selection of a data product format, the data product options form will appear for many, but not all, data products. As indicated by the red arrow in the above illustration, the green highlighted check-box corresponds to the options form. Modify the options as you see fit. Please refer to the data product options home page for more information. In general, there are options for the normal scalar data products, such as the CSV, but are no options for complex or binary data, such as the log file. Note that the options form has no memory; it will reset to its default on every click of a data product format check-box. 

To add more data products, click on additional check-boxes, filling out the options form every time. Please be aware that one cannot go back to a previous check-box to review the selected options. This behavior may be improved in future versions. New options will also be added to satisfy user requests.

When you have finished, click Add to Cart. Note that additional clicks on Add to Cart will not add duplicate entries.

Step 3: Viewing the Cart and Downloading Results


(The subsampling column shown above will be replaced with the data product options in future versions).

Searches are started by clicking the Check Out All button. The searches are then are added to a processing queue until a server picks up the job (there's not usually a delay). Status updates will appear regularly until the processing is complete. Processing time may be as little as 2 seconds, up to a week (processing a day of Nortek ADCP data can take 20 minutes).

As processing completes for each data product, links appear allowing you to download the files.
To download your results:

  • click the links to download individual results. Depending on your computer, PNG plots will likely open into a photo viewer, CSV files will open into your spreadsheet editor, etc. If multiple files were produced, they will be bundled into zip files.
  • To download data from multiple searches at once, first select the ones you want (by clicking the check-boxes), then click the Download button. 
    • Multiple files will be placed in a zip file for you.
    • Zip files must be opened using Keka in Mac OSX or 7-Zip in Windows XP.
  • Individual searches and search results may be deleted from your cart with the Delete button.

Metadata

Metadata will automatically be generated with most data products. They contain important information regarding the actual data and data sources, the observatory, the instrument/sensor, and other background information. Once generated, they are shown as links to the file and can be downloaded for every data product.

In particular for long time series from several sensors that are combined in a single data file, Metadata list all contributing sensors and link to their calibration details and annotated data issues. Therefore, metadata are considered vital to interpret the scientific data and should be downloaded whenever possible.

Questions & Discussion

You can also send us a help request by clicking the question mark (question) icons in the requested data product list (in Step 3):

If you have comments, questions or requests related to the data search tool, please post them below by clicking Add Comment below or by adding a comment in our Help Discussion forum.