API

The CNRA Open Data Platform (ODP) doesn't just catalog data, it also provides an API (Application Programming Interface) to connect to and explore data programmatically. All the metadata within ODP can be accessed through the API as well as some datasets that are stored on ODP. Please be aware that some datasets are only cataloged on ODP and the CKAN API only contains metadata about these datasets. This metadata includes URLs and descriptions of datasets, but it does not include the actual data within each dataset.

 

CNRA ODP CKAN API

The CNRA ODP catalog is powered by CKAN, a powerful open source data platform that includes a robust API. All metadata is available through the API, but to access the actual data, only datasets with resources stored on ODP can be accessed and those resources stored in a data table format will be accessible through the API. A good way to know if a dataset resource is available through the API is if you can preview the data resource on ODP as a table. You will also see a "Data API" button appear on the same dataset resource preview page. The most common data type that is accessible this way is from a CSV (Comma Separated Value) file. Please note that some datasets listed on ODP are purely cataloged and the actual data is stored on their related locations outside of ODP (check the metadata to understand the location of the data).

The base URL for the CNRA ODP CKAN API is:

The CKAN API returns results in a JSON (JavaScript Object Notation) format.

 

TERMINOLOGY

Even though we call data in the ODP a dataset, the CKAN API uses the older terminology of a "package".  A dataset can have one or more resources.

Complete API documentation is available from CKAN: https://docs.ckan.org/en/2.9/api/

 

LIMITATIONS

Data on ODP is not guaranteed to always be available on a continuous 24/7/365 basis. Sometimes the system will be under maintenance or have some unforeseen downtime. Links may change without notice for a variety of reasons. It is highly discouraged to build any applications and/or systems that rely on API calls to ODP data to be always available and at the same location. An example of this is if a dataset has to change its name and is republished. Downloading data is recommended.
 

Due to the high volume of requests for data, there are some limitations on using the CKAN API when accessing data. Requests for dataset metadata is limited to 1000 items per request. This does not mean you cannot get the metadata for all the datasets in ODP, but instead it means you must make multiple requests with a limit of 1000 items per request. For the Datastore Search SQL API Call, it has a special limit of 32000 records returned.

 

Assorted API Syntax and Examples

Basic API Syntax:

Get a list of all the datasets:
https://data.cnra.ca.gov/api/3/action/package_list

Get a list of all the topics:
https://data.cnra.ca.gov/api/3/action/group_list

Get a list of all the tags:
https://data.cnra.ca.gov/api/3/action/tag_list

Full metadata information:

https://data.cnra.ca.gov/api/3/action/package_show?id=<replace with the dataset id>

https://data.cnra.ca.gov/api/3/action/resource_show?id=<replace with the resource id>

https://data.cnra.ca.gov/api/3/action/group_show?id=<replace with the topic id>

https://data.cnra.ca.gov/api/3/action/tag_show?id=<replace with the tag id>

Search for packages or resources matching a query:

https://data.cnra.ca.gov/api/3/action/package_search?q=spending

https://data.cnra.ca.gov/api/3/action/resource_search?query=name:District%20Names

Full metadata information for a list of datasets along with their resources. This request allows for up to 1000 datasets returned per request. The "offset" parameter is used to page through the resources. To better understand this request, it is recommended to read the CKAN API documentation: https://docs.ckan.org/en/2.9/api/#ckan.logic.action.get.current_package_list_with_resources

https://data.cnra.ca.gov/api/3/action/current_package_list_with_resources?limit=1000&offset=0

Related to the DataStore:

Query (note: default limit of items returned if not given is 100):
https://data.cnra.ca.gov/api/3/action/datastore_search

Query (via SQL) (note: upper limit of returned items 32000):
https://data.cnra.ca.gov/api/3/action/datastore_search_sql

Some examples using the "Periodic Groundwater Level Measurements" dataset "Stations" resource:

Dataset:
https://data.cnra.ca.gov/dataset/periodic-groundwater-level-measurements

Resource:
https://data.cnra.ca.gov/dataset/periodic-groundwater-level-measurements/resource/af157380-fb42-4abf-b72a-6f9f98868077

Metadata example of dataset (includes all the resources):
https://data.cnra.ca.gov/api/3/action/package_show?id=periodic-groundwater-level-measurements

Metadata example of resource:
https://data.cnra.ca.gov/api/3/action/resource_show?id=af157380-fb42-4abf-b72a-6f9f98868077

Query example (first 5 results)
https://data.cnra.ca.gov/api/3/action/datastore_search?resource_id=af157380-fb42-4abf-b72a-6f9f98868077&limit=5

Query example (results containing 'Surprise Spring')
https://data.cnra.ca.gov/api/3/action/datastore_search?resource_id=af157380-fb42-4abf-b72a-6f9f98868077&q="Surprise Spring"

Query example (via SQL statement)
https://data.cnra.ca.gov/api/3/action/datastore_search_sql?sql=SELECT * from "af157380-fb42-4abf-b72a-6f9f98868077" WHERE "basin_name" LIKE '%Spring%'

Example: Python 3.x

(Note: For a more robust API call, it is recommended to use the Requests library and requires Python 3.7+. This simple example uses URLLib)

import json
import urllib.request

 

url = 'https://data.cnra.ca.gov/api/3/action/datastore_search?resource_id=af157380-fb42-4abf-b72a-6f9f98868077&limit=5&q="Surprise Spring"'  
fileobj = urllib.request.urlopen(url)
response_dict = json.loads(fileobj.read())

 

print(response_dict)