New service interface for River Forecasting Center derived quantitative precipitation estimates

For more than a decade, the National Weather Service (NWS) River Forecast Centers (RFCs) have been estimating spatially distributed rainfall by applying quality-control procedures to radar-indicated rainfall estimates in the eastern United States and other best practices in the western United States to produce a national Quantitative Precipitation Estimate (QPE) (National Weather Service, 2013). The availability of archives of QPE information for analytical purposes has been limited to manual requests for access to raw binary file formats that are difficult for scientists who are not in the climatic sciences to work with. An archive had been available from the NWS National Precipitation Verification Unit (NPVU), an organization responsible for comparing quantitative precipitation forecasts to QPE (McDonald, 2000). This archive was a complete set of gridded binary files distributed by the RFCs via the Internet Data Distribution (IDD) system. Various other partial archives are known (Lin and Mitchell, 2005; NOAA Hydrologic Data Systems Group, 2008), but the NPVU’s archive represents the most complete record of this data. Unfortunately, operation of the NPVU was suspended as of February 2012 and the archive is no longer being maintained.

Data from the NPVU archive and the Unidata Internet Data Distribution system (Dominico, 2003) real-time feed from RFCs to the NPVU have been found to be useful for modeling real-time beach-water quality (Juckem and others, 2013). It is expected that, given easy access to the data, other scientists will find this dataset useful. The NWS provided the NPVU’s QPE archives to the U.S. Geological Survey (USGS), and the contents of the real-time feed from the RFCs are being saved by the USGS for incorporation into the archives.

RFC QPE data are distributed on a grid that uses the Hydrologic Rainfall Analysis Project (HRAP) projection. The HRAP grid uses a polar stereographic projection and a spherical datum (Fulton, 1998). When radar station locations were originally referenced to the HRAP grid, their latitude-longitude locations were read from maps using a projection based on an ellipsoidal datum. These latitude-longitude locations were then projected to the HRAP polar stereographic plane by using equations meant for projection of spherical-Earth latitude-longitude coordinates. This process resulted in a dataset that must be unprojected assuming one (spherical) datum, and then related to other datasets assuming a second (ellipsoidal) datum (Reed and Maidment, 1999). This use of two datums effectively breaks any automated workflow meant to relate datasets referenced to different coordinate reference systems. The USGS has implemented a solution to this issue as presented below. As described, the most appropriate way to relate HRAP data to other geospatial data is to unproject the grid by using spherical Earth datum parameters, and then assume the resulting coordinates are from a modern ellipsoidal datum. This process results in a gridded dataset with uneven grid spacing. In geographic information systems, gridded data is normally treated as a raster (with an origin and even grid spacing). Because the HRAP data must be unprojected assuming one datum then assigned a new one, there is no way to represent the native pixels using a normal raster data model. The data must be resampled or represented using unique latitude and longitude values for each grid cell. That is, the original grid values and geometry must be altered or the grid’s latitude-longitude coordinates must be represented using a two-dimensional grid of points with unique locations.

The Network Common Data Format Climate and Forecasting (NetCDF-CF) metadata conventions allow data to be represented by using two-dimensional latitude longitude “coordinate variable” (Eaton and others, 2011). Data using a two-dimensional coordinate variable is interpreted by software that implements NetCDF-CF conventions as a grid that can be accessed using normal grid indexing notation as is used for normal raster grids. The geolocation of each grid cell center is determined by accessing latitude and longitude coordinate variables that hold a unique value for every grid cell.

The NetCDF-Java library (Caron, 2012) implements the NetCDF-CF metadata conventions as well as a convenient method to modify metadata and data using Extensible Markup Language (XML) that does not require modification of source data. By use of this NetCDF Markup Language (NCML) (Caron, 2010), appropriately unprojected two-dimensional latitude-longitude coordinate variables can be added to a dataset, the corresponding datum metadata can be added, and the original HRAP-projected coordinate reference system information can be removed without the need to rewrite or alter the source data. NCML also allows construction of time-series aggregations that create a single logical time-series dataset out of many related files. The USGS has applied this time-series aggregation technique in combination with the addition of two-dimensional latitude-longitude coordinate variables to publish the RFC QPE data provided by the National Weather Service.

NetCDF-Java forms the functional core of the Thematic Realtime Environmental Data Distribution System (THREDDS) (Unidata, 2012). THREDDS allows creation of standard Web service interfaces for datasets that are compatible with NetCDF-Java. Publishing the RFC QPE data via THREDDS allows users to access the data via the Opensource Project for a Network Data Access Protocol (OPeNDAP) (OPeNDAP, 2010), Open Geospatial Consortium Web Map Service protocol (Open Geospatial Consortium, 2006b) and Open Geospatial Consortium Web Coverage Service (Open Geospatial Consortium, 2006a) protocols, among others. These services provide users with direct (index-based) data access for OPeNDAP-capable NetCDF software libraries, rendered visualizations of the data for Web and desktop geospatial software, and resampled raster representations of the source data in common geographic information formats. These services are made available by the USGS at a new interface at http://cida.usgs.gov/thredds/rfc-qpe/.

References Cited

Caron J., 2010, NetCDF Markup Language: accessed March 10, 2013, at http://www.unidata.ucar.edu/software/netcdf/ncml/.

Caron J., 2012, NetCDF-Java Library: accessed March 10, 2013, at http://www.unidata.ucar.edu/software/netcdf-java/documentation.htm.

Dominico, B., 2003, Unidata Internet Data Distribution: An overview of the IDD: accessed March 10, 2013, at http://www.unidata.ucar.edu/software/idd/overview/idd.html.

Eaton, Brian, Gregory, Jonathan, Drach, Bob, Taylor, Karl, and Hankin, Steve, 2011, NetCDF Climate and Forecast Metadata Conventions, ver. 1.6: accessed March 10, 2013, at http://cf-pcmdi.llnl.gov/documents/cf-conventions/latest-cf-conventions-document-1.

Fulton, R.A., 1998, WSR–88D polar-to-HRAP mapping: Silver Spring, Md., National Weather Service, Office of Hydrology, Hydrologic Research Laboratory Technical Memorandum, 33 p.

Juckem, P.F., Corsi, S.R., McDermott, Colleen, Kleinheinz, Gregory, Fogarty, L.R., Haack, S.K., and Johnson, H.E., 2013, Evaluation of potential sources and transport mechanisms of fecal indicator bacteria to beach water, Murphy Park Beach, Door County, Wisconsin: U.S. Geological Survey Scientific Investigations Report 2012–5190, 29 p.

Lin, Y., and Mitchell, K.E., 2005, The NCEP Stage II/IV hourly precipitation analyses: Development and applications, in Preprints, Conference on Hydrology, 19th, American Meteorological Society, San Diego, California, January, 9–13, 2005, Paper 1.2.

McDonald, B. E., Graziano, T.M., and Kluepfel C.K. 2000, The NWS National QPF verification program, in Conference on Hydrology, 15th , American Meteorological Society , accessed March 10, 2013, at https://ams.confex.com/ams/annual2000/techprogram/paper_6441.htm.

National Weather Service, Advanced Hydrologic Prediction Service, 2013, About the Precipitation Analysis Page: accessed March 10, 2013, at http://water.weather.gov/precip/about.php.

NOAA Hydrologic Data System Group, 2008: accessed March 10, 2013, at http://dipper.nws.noaa.gov/hdsb/data/nexrad/nexrad.html.

Open Geospatial Consortium Inc., 2006a, Web Coverage Service implementation standard: accessed March 10, 2013, at http://www.opengeospatial.org/standards/wcs.

Open Geospatial Consortium, Inc., 2006b, OpenGIS Web Map Service implementation specification: accessed March 10, 2013, at http://www.opengeospatial.org/standards/wms.

Opensource Project for a Network Data Access Protocol, Inc., 2010, OPeNDAP Web site: accessed March 10, 2013, at http://opendap.org/.

Reed, S., and Maidment, D., 1999, Coordinate transformations for using NEXRAD data in GIS-based hydrologic modeling: Journal of Hydrologic Engineering, v. 4, no. 2, p. 174–182, http://dx.doi.org/10.1061/(ASCE)1084-0699(1999)4:2(174).

Unidata, 2012, THREDDS Data Server: accessed March 10, 2013, http://www.unidata.ucar.edu/projects/THREDDS/tech/TDS.html.

For more information, contact:

David Blodgett

Center for Integrated Data Analytics 8505 Research Way Middleton Wisconsin 53562 Phone: 608-281-3899 email: dblodgett@usgs.gov

Where this page came from

This page was imported from U.S. Geological Survey. Published by the U.S. Geological Survey and, as a work of the United States government, in the public domain.

Nobody has written it yet — it is the source material at a new address, which is why search engines are asked to skip it and why no one earns from it. It is up for grabs: take it on, and it is yours to rewrite and to earn from.