WESTERN REGION TECHNICAL ATTACHMENT
NO. 02-06
MARCH 12, 2002

A GFE-BASED TEMPERATURE VERIFICATION AT WFO BOISE

Les Colin, Weather Forecast Office, Boise, Idaho

Introduction

The FSL's GFE Smart-Tool methodology has made it possible to advance from traditional point-based verification to a real, grid-based verification, in which forecast grids are compared to analysis grids for accuracy. This article shows one approach as applied to a maximum temperature forecast.

Methodology

The GFE provides a framework to create grids of many weather elements. The GFE can ingest model forecast data and adapt it to its own finer grid mesh and higher resolution topography. Grid edit tools and Smart Tools enable the forecaster to make further refinements to those grids.

Eventually, "verifying" grids become available. These may consist of zero-hour model grids, or grids from an independent source (e.g., ADAS). Boise WFO has taken a step in this direction by creating max/min temperature analysis grids from about 100 observations reported in our Regional Temperature and Precipitation (RTP) summary (a table). The RTP data are point values, but they can be contoured spatially to form a grid of their own. A mathematical function well suited to this purpose is the serpentine function (CRC, 1964), which is the two-point inverse-square distance weighting function. This function can be generalized to accommodate any number of points, and modified so that max and min values need not occur on the observed values (which would otherwise happen). Another modification compensates for data clustering in the RTP (several sites near each other) which often occurs in more populated areas. Figure 1a shows a serpentine fit to two points, Figure 1b, to three points. Figure 1c fits the same three points as Figure 1b, but whose influence also depends on elevation, while Figure 1d adjusts Figure 1c's fit to the high-resolution topography in the GFE.

Figure 2 (below) shows a 3-d serpentine RTP analysis of all Max temps on WFO Boise's CWA.

Verification is the difference between the forecast grid and the verifying grid (f-v), and can be displayed graphically (Fig. 3).

Errors at every point on the grid can also be collected and grouped to form a frequency distribution. For example, BOI's CWA contains about 7500 grid points. Collecting errors in whole degrees (F) might show 1000 of them between -2 and -1, 800 between -1 and 0, 620 between 0 and +1, etc. Figure 4 shows the distribution at intervals of 0.1 degree, ranging from 10.0 degrees too cold on the left end to 10.0 degrees too warm on the right. The number of points with errors worse than +/- 10 is also printed on the graph. The vertical axis shows the number of points (of the 7500) that fall into each error interval on the horizontal axis. A perfect forecast would have all 7500 points at zero error. Any other distribution indicates variance. The horizontal color bar rates the forecasts "good" or "bad" according to average MAE of all 7500 points (including the outliers). The left (bad) end corresponds to MAE of about 8 degrees.

The original RTP values (i.e., in the table) can also be merged into the forecast, as follows: Each RTP site is positioned on the GFE domain, and its value is compared with the forecast there. The errors are then analyzed (using the serpentine function) to a "work" grid, which is then subtracted from the forecast grid. The resulting grid now is an "improved" forecast in the sense that it fits all the RTP values exactly. This improved forecast can be used as a starting grid for the next forecast.

It's interesting to verify the original forecast to the improved forecast, using the latter as "truth." As shown in Figure 5, the scores are higher. This is to be expected since part of the original forecast is used to verify the original forecast, i.e., the two grids are not independent. There is also a local spike in the frequency distribution, probably corresponding to those GFE points far away from any RTP site, and least influenced by them.

References

CRC, 1964: Standard Mathematical Tables. The Chemical Rubber Co., Cleveland, OH, 561pp.

NOAA Forecast Systems Laboratory, 2000: GFE Training Guide, Boulder, CO.