WESTERN REGION TECHNICAL ATTACHMENT
NO. 03-01
MARCH 24, 2003

THE WESTERN REGION GRIDDED VERIFICATION SYSTEM

Chris Jordan and Kirby Cook
Western Region-SSD, Salt Lake City, UT

Introduction

As the National Weather Service moves fully into IFPS operations, forecaster feedback on gridded forecast product accuracy becomes critical to the production of quality forecast products. While verification of gridded products against point observations is useful, it does not provide enough useful feedback to forecasters. Comparisons must also be carried out against quality gridded analyses. A gridded verification system has been developed by Jeff Davis (WFO Tucson) that utilizes the Graphical Forecast Editor (GFE) to compare IFPS forecast grids against gridded observational analyses (Davis, 2002). Forecast error grids can be used as a means to provide feedback to the NWS forecaster on the accuracy of IFPS forecasts. This system has been modified at Western Region Headquarters, Scientific Services Division, in order to simplify its installation and its use with other GFE based systems such as the MatchObsAll analysis system developed at the Boise forecast office (Foisy, 2003). The program can easily be configured and modified to meet the needs of each forecast office. This technical attachment is intended to be an overview of the program and to describe its components in order to facilitate its use in forecast operations at the WFO level.

Overview

Currently, this system is configured to produce difference grids (forecast - observed) for the MaxT, MinT, QPF and Wind forecasts. The default configuration for each of these grids is assumed, local configuration other than the default may require some changes to the procedures and smartTools that make up this verification system. MaxT is defined as a 12-hour grid beginning at 1400 UTC while MinT is defined as a 13-hour grid beginning at 0200 UTC. Wind grids are assumed to be 3-hour grids. To simplify the computation and interpretation of QPF comparisons against the gridded analysis, the 6-hour default QPF grids are summed into a 24-hour grid (QPF24) valid from 0000-2400 UTC. These 24-hour QPF grids are compared against 24-hour total precipitation grids (TP24) created from the Stage IV Precipitation Analysis (ST4PPT) generated by the National Centers for Environmental Prediction (NCEP). The ST4PPT analysis is an hourly, 4 km, multi-sensor precipitation analysis that uses both gage and radar-estimate data. This data is processed by Western Region, Scientific Services Division and distributed to Western Region forecast offices via the Western Region wide area network. In order for this dataset to available for use in this verification system, local offices must first down load and install the ST4PPT localization and processing scripts for AWIPS/D2D from the WR development server (165.92.200.12). Questions about this installation should be directed to Kirby Cook (WR/SSD, 801-524-5131). The GFE configuration and the smartInit script for the ST4PPT are included as part of the installation distribution of this verification system.

Fundamentally the core of this system is the three mutable or changeable databases in which the forecast, observed and error grids are stored and manipulated. These mutable databases are modified by using special gfeConfig files that are specific to each database (ObsArchiveConfig, FcstArchiveConfig and DiffArchiveConfig). The ObsArchive, FcstArchive, and DiffArchive mutable databases store the necessary fields from the gridded analysis, Fcst database and the difference grids generated from the system. Each of these databases is capable of storing data over the latest 7 day period (168 hours). Three GFE procedures (Save_Obs, Save_Fcst and Calc_Diff) are used to transfer these datasets to the mutable databases, and compare forecasts against the gridded analyses. Each of these procedures is run automatically from the cron, which allows the procedure to be executed using the appropriate gfeConfig file for each of the mutable databases. Below is a description of each procedure.

Save_Obs

The Save_Obs procedure copies the MaxT, MinT, TP24 and Wind grids from the appropriate analysis to the separate observed database called the ObsArchive database. Because this procedure operates on a separate mutable database it must be run from the cron using the runProcedure command and a gfeConfig file (ObsArchiveConfig) specific to the ObsArchive database. Save_obs is executed from the cron at 0400 UTC daily (Fig. 1) to ensure that the necessary observed data is available for that day (e.g. the MaxT, MinT, TP24 and Wind grids). Save_Obs can be configured to use any available gridded analysis such as ADAS02, LAPS, MSAS or MatchObsAll (Foisy, 2003).

Save_Fcst

The Save_Fcst procedure copies the current forecast grids from the Fcst database to the FcstArchive database. In addition, it is here that the QPF grids are summed into a 24 hour grid. Like the Save_Obs procedure, Save_Fcst is run from the cron using the runProcedure command and a gfeConfig file (FcstArchiveConfig) specific to the FcstArchive database. However, Save_Fcst is executed from the cron at 1100 and 2300 UTC (Fig. 2). This is done to fill in the forecast database such that more than one wind forecast is available at each comparison time (i.e. there is a 12-hour, 24-hour 36-hour forecast valid at both 0000 and 1200 UTC) and has little affect on the MaxT, MinT and QPF24 grids.

Save_Fcst archives the forecast grids renaming each element to include the forecast lead-time in the element name. This is done in order to account for forecast lead-time later when computing difference grids. Thus each element in the FcstArchive database is named:

• FcstMaxT01, FcstMinT01, and FcstQPF2401 = 1 day MaxT, MinT and QPF24 forecast
• FcstMaxT02, FcstMinT02, and FcstQPF2402 = 2 day MaxT, MinT and QPF24 forecast
• FcstMaxT03, FcstMinT03, and FcstQPF2403 = 3 day MaxT, MinT and QPF24 forecast

• FcstWind12 = 12 hour Wind forecast
• FcstWind24 = 24 hour Wind forecast
• FcstWind36 = 36 hour Wind forecast
• FcstWind48 = 48 hour Wind forecast

To simplify the process and to minimize the number of Wind variables (i.e. there will be a new wind variable for each forecast hour saved) the wind forecasts are limited to the 12, 24, 36 and 48 -hour lead-times.

Calc_Diff

The Calc_Diff procedure compares forecast grids stored in the FcstArchive database against gridded analyses stored in the ObsArchive database for a given valid time each day. As with the other procedures, the Calc_Diff procedure must be run from the cron using the runProcedure command and a gfeConfig file (DiffArchiveConfig) specific to the DiffArchive database. The Calc_Diff procedure is executed from the cron at 0430 UTC, after the Save_Obs procedure, in order to ensure that all of the observed analysis grids are available for use in the comparison. The comparison of forecast grids against gridded analyses can be described with the equation:

where N is the number of gridpoints. Because only one MaxT, MinT and QPF24 grid is valid per day, only one comparison per forecast (day 01, 02, and 03) is carried out for these grids (Fig. 3). The resulting difference grid is labeled with the element name and forecast and saved to the Diff_Archive database. This is done in order to account for forecast lead-time in the difference grids. Wind comparisons are done for 12, 24, 36 and 48 hour forecasts valid at 0000 and 1200 UTC (Fig. 3). Wind comparisons are carried out for both wind speed (kts) and wind direction (absolute value of the difference between forecast direction and observed direction at each grid point), returning scalar grids for both values. Thus each element in the DiffArchive database is named:

• MaxT01, MinT01, and QPF2401 = 1 day MaxT, MinT and QPF24 error
• MaxT02, MinT02, and QPF2402 = 2 day MaxT, MinT and QPF24 error
• MaxT03, MinT03, and QPF2403 = 3 day MaxT, MinT and QPF24 error

• Winddir12 =12 hour Wind direction error
• Winddir24 = 24 hour Wind direction error
• Winddir36 = 36 hour Wind direction error
• Winddir48 = 48 hour Wind direction error

• Windspd12 =12 hour Wind speed error
• Windspd24 = 24 hour Wind speed error
• Windspd36 = 36 hour Wind speed error
• Windspd48 = 48 hour Wind speed error

Finally, the DiffArchive database is saved to a netCDF file using the ifpnetCDF program.

SumQPF24

The SumQPF24 smartTool sums the QPF grids over the period 0000 to 2400 UTC creating the FcstQPF2401, FcstQPF2402 and FcstQPF2403 grids in the FcstArchive database. SumQPF24 is called by the Save_Fcst procedure.

Installation and Configuration

It is recommended that you install this software on an RPP system, and not the operational IFPS computer used by your office. This system can be installed on the same computer running the MatchObsAll analysis system (Foisy, 2003) developed at the Boise Forecast office. If installed on an RPP computer, you will need to transfer your official database from your operational IFPS system using iscMosaic. The default configuration (time constraints, etc.) for MaxT, MinT, QPF and Wind is assumed, local configuration other than the default may require some changes to the procedures and smartTools that make up this verification system.

1. Download the ifpsVerify.tar file and ifpsVerify.README files from the Western Region Development Server (165.92.200.12). To do this you will need to be on your ds1. To obtain the user and password information for this server please contact Kirby Cook at WR/SSD (801-524-5131).

2. Untar the ifpsVerify.tar file in the $GFEHOME directory (for RPP systems this will be the release/ directory). This will create the ifpsVerify directory, cd to this directory.

tar -xvf ifpsVerify.tar
cd ifpsVerify

3. You will need to make some changes to the install_ifpsVerify.csh script. Vi the file and make the appropriate changes to the $user and $GFEHOME variables.

4. Before running the install_ifpsVerify.csh script you will need to make edits to many of the files in this directory.

First, edit the following files to change the site information to your local site id (i.e. SLC, BOI, VEF, etc.):

• DiffArchive_FPMaxT.BUNDLE
• DiffArchive_FPMinT.BUNDLE
• DiffArchive_FPQPF24.BUNDLE
• DiffArchive_FPWind.BUNDLE
• ObsArchive.BUNDLE
• Calc_Diff.pcd

Second, if you intend on using an analysis (such as MatchObsAll) other than the 2km ADAS (which is the default) you will need to make the appropriate changes to the elementList entry in the Save_Obs.pcd file. Changes to the analysis used in the verification can be made at any later time after the system has been installed by editing the Save_Fcst procedure through the Define Procedures pull-down menu.

5. Run the install_ifpsVerify.csh script.

./install_ifpsVerify.csh

6. To make the above installation and configuration take effect you will need to make two edits to your localConfig.py file:

First, add at the top of the file the line:

from localVerify import *

Second, you will need to add the following entries for the new databases to the dbs variable list (do not remove any entries already included in the list):

dbs = [(ST4PPT,parmsST4PPT), (ObsArchive, ObsArchiveParms), (FcstArchive,FcstArchiveParms), (DiffArchive,DiffArchiveParms),

7. Stop and Start your IFPServer.

./stopIFPServer
./runIFPServer

8. Create/Edit a new cron entry for each of the procedures (Save_Obs, Save_Fcst and Calc_Diff) using the sample crontab file included in the distribution as a template. Any changes in the times which each procedure is run may require changes to the procedure itself.

Acknowledgements

Thanks to Jeff Davis for providing technical support and permission to use his procedures, smartTools and mutable databases as a framework for this system.

References

Davis, J.T., 2003: GFE GRID BASED VERIFICATION PROGRAM. WR-Technical Attachment.
Foisy, T., 2003: THE MATCHOBSALL ANALYSIS SYSTSM. WR-Technical Attachment.