Accessing discrete water quality and sample data on WDFN

The USGS just released new ways to access water quality and sample data in a recently expanded WQX data format.

Introduction

We are excited to share with you some updates to how USGS discrete (occurring as separate and distinct events) water quality and field sample (a.k.a. ‘samples’) data can be viewed and downloaded from the Water Data for the Nation (WDFN) website. This is the third blog in a series describing the updates to public samples data delivery. Our first installment briefly introduces the changes, and the second installment provides background on the new data delivery standard, called the Water Quality Exchange (WQX) standard. In summary, this post:

• Reviews the legacy web tools available for accessing samples data and why they are being retired
• Introduces three new web resources available on WDFN for viewing and accessing samples data
• Provides a glimpse into future resources, including APIs and a map-based download form, that are being built to give users more options for accessing USGS water data

Modernizing an aging system

The USGS National Water Information System (NWIS) has been undergoing a major, multi-year modernization that will enable us to more securely, reliably, and effectively deliver high-quality water data at a national scale. Historically, we delivered samples data using two main features:

• Web tables designed for viewing on a computer screen
• Data, either printed to a computer screen or as a download, in relational database (.rdb) format

The legacy samples websites have become difficult to update because they rely on aging technology. Therefore, we are modernizing delivery of samples data.

Modernized features for delivering samples data

With modernization, we are delivering samples data in new ways. Here’s an overview of the features available for accessing different information.

View a summary of samples data

The Summary of Sample Data Table located on the Monitoring Location Pages allows users to view a summary of all available water quality and field sample data available at a monitoring location over the entire period of record. Users can get an idea about what data are available and decide whether to download the data.

Key features

• View a list of the available data for each observed property, the time period in which data are available and the number of result values or field visits (e.g. sampling activities).
• Page through the entire list of available samples data, or easily sort and filter the table based on the observed property or groups of observed properties (characteristic group).
• Easily initiate a data download by clicking on an observed property, which redirects to the new Download Discrete Sample Data page with your site and observed property pre-populated to make it easy to retrieve data of interest.

WDFN download discrete samples

The Download Discrete Sample Data page retrieves samples data for monitoring locations of interest.

Key features

• Users can select data from one or many locations by providing a single identifier or list of identifiers. Queries can also be performed without a location filter to get data from all possible locations as long as a result filter is specified.

• Users can also select locations within a geographic region such as US State or Hydrologic Unit code (HUC), or users can create a Bounding Box from a set of coordinates. Locations can be further filtered to specific Site Types (Stream, Well, etc.).

• Users can further hone their data queries to retrieve results for samples by Sample media (water, biological tissue, soil, etc.), Observed properties or Characteristic group, and by date.

• The returned data will follow the Water Quality Exchange (WQX) data standard that we describe in an earlier blog post . In brief, the WQX standard includes comprehensive metadata about a measurement or observation. Users will be able to choose a subset of the comprehensive list of metadata fields to download based on their needs and use cases. The options that are currently available are:

  • Monitoring locations – includes monitoring location information, including geospatial information, site type and aquifer data.

  • Sample results (full physical chemical) – includes results data and complete metadata for physical and chemical measurements and related metadata, as well as information about the project, location, and sampling activity.

  • Sample results (narrow) – includes results data for physical, chemical, and biological measurements and related metadata that are broadly useful for interpreting the results, as well as information about the project, location, and sampling activity.

  • Sampling activity – contains detailed information about water quality field collection events such as collection method and measurement depth.

• Data can be downloaded in the comma separated (csv) file format for easily opening in different software such as MS Excel, R or Python.

Search parameters in the Code Dictionary

As we transition to serving data in a new format, we want to make it easy for users to know how the parameter codes (pcodes) will map to the new WQX metadata fields. The Code Dictionary provides a cross-reference between a USGS parameter code and the new USGS term called observed property (wondering what is an observed property? Check out this blog post).

Key features

• Provides answers to basic questions about observed properties and parameter codes and how they relate to each other.

• Enter a parameter code and have it translated into an observed property.

• Enter an observed property and have it translated into the related parameter code(s)*.

• View all available observed properties, with options to view a list of names in text or JavaScript array format, and to obtain the full list of observed properties with their parameter code relationships in JavaScript Object Notation (JSON) format.

• Users can view the version date currently in use by the Code Dictionary, which is updated periodically.

*Note: Observed properties are not identical to parameter codes in the type of information they include, specifically they do not contain measurement units. A single observed property can be related to more than one parameter code.

Public API services

New APIs for samples data are in development. Stay tuned for updates.

As part of system modernization, existing API services for USGS discrete water quality and sample data from Period of Record queries using the Site service will change. On March 11, 2024, the Site Service no longer retrieved new or updated water quality and field samples data from queries using the Water quality samples/results filters (hasDataTypeCd=qw). No new data will be available using this request. Service responses will only return data available prior to this date.

Alternative services are being developed to enable users to obtain updated site-level water quality samples and results data. To stay up to date on services-related updates, please check out the waterservices documentation page at https://waterservices.usgs.gov/docs/ .

What’s next for samples data?

Explore and download USGS data

We are already working on a new map-based search feature to find locations that have data of interest. Users can search for data of various types including streamflow data and samples data all in the same map interface.

Once you’ve found the data of interest, you can then download the data in a single download package: no more downloading streamflow data and samples data separately!

Let us know how we’re doing

The features introduced in this blog will be in beta release. We’ll collect information on how they are working for users to better serve users’ needs. Please email us at wdfn@usgs.gov to let us know what you think.