WESTERN REGION TECHNICAL ATTACHMENT
NO. 99-10
May 11, 1999

A COMPARISON OF THE ETA MODEL-DERIVED SURFACE DATA
WITH ASOS TEMPERATURE AND WINDS

Beth L. B. McNulty* and Mary Cairns - NWSFO Reno, NV
*Current affilitation: NWSO Glasgow, MT

Introduction

This study was undertaken to determine whether model-derived surface parameters provided by the Automated Weather Interactive Processing System (AWIPS) for public products would be reasonable in the automated generation of forecast products. The National Weather Service (NWS), as part of its modernization efforts using AWIPS, is developing the Interactive Forecast Preparation (IFP) System. The IFP uses, as one method of initialization, the AWIPS Forecast Preparation System (AFPS), a system dependent on forecast grid fields produced from numerical models. The question was, if the model output remained within the error margin currently allowed for human produced values, would any adjustments to the model output be needed? However, if the derived values varied substantially more than the error margin allowed for human forecasts, then the model output would need modification before use in various public products.
The local forecaster needs to know whether she/he can depend on an automated forecast. If the automated product was accurate, then she/he can concentrate on watch and warning duties. If the automated product was inaccurate, the focus must be put on public products, e.g., the zone temperature and winds, possibly taking time away from, and/or requiring a second person to monitor watch and warning duties. This will become even more important as public products change to include more detailed information out to longer forecast periods.

The National Center for Environmental Prediction (NCEP) continues to work on more refined mesoscale models. The Eta model is an example of efforts to provide higher-resolution mesoscale models to the field forecasters (Black 1994; Staudenmaier 1996a; Black et al. 1997). In 1997, NCEP made available to the western NWS forecast offices an experimental 10 km version of the Meso Eta. This model output was available to the field for a six month evaluation period (Staudenmaier 1997). The 29-km version of the Meso Eta was already operational and was being used in day-to-day forecasts.

The impetus of this study was to provide the forecasters with an objective view on how well the higher resolution model-generated fields performed in the complex terrain of western Nevada. Model-derived surface temperatures and winds from the 29-km and 10-km Meso Eta were compared to the Reno, Nevada Automated Surface Observing Systems (ASOS) site for the period of early April through mid-August 1997. It was hoped that the information gleamed from this project would benefit both the NCEP modelers and the IFP developers in addition to field forecasters.

Data and Methodology

Data was collected from early April to mid-August 1997. The 10 km Meso Eta evaluation ran from January through April 1997, but due to the availability of computer time the model continued to be available through the summer. The weather patterns which occurred during this time consisted mainly of spring fronts, and summer convection. No wintertime model data was available or evaluated.

A procedure on the Science and Operations Officers' Application Computer collected and stored the output after every model run period. Model data was obtained through a GEMPAK program which extracted the data by averaging the nearest four grid points to a specific location, and wrote the data to a file. Forecasts of model-derived 2 m temperature, dewpoint temperature, and wind direction and speed were extracted for the 0- through 33-h forecast periods. Model data from the 10 km Meso Eta (run at 0300 UTC), and the 29 km Meso Eta (available from the 0300 and 1500 UTC runs) were saved. ASOS data from Reno were collected for the matching period. Data were then entered into a spreadsheet to be used for evaluation purposes. Data were visually inspected for partial data (indicating an incomplete model run), and questionable data were flagged for elimination during the analysis process.

Two main statistical methods were used to analyze the data, the bias and mean absolute error. These are easily defined using Fi as the forecasts, Oi as the observations, and n as the number of forecast/observation pairs. The bias, or mean (algebraic) error (ME), indicates the average direction of the deviation from the observed values. The bias is defined as

A positive bias indicates that the forecast exceeds the observed value on the average (overforecasting), and a negative bias corresponds to a forecast below the observed value on the average (underforecasting). For example, a negative arithmetic temperature error means that the model was too cold. The bias range is from -infinity to +infinity , and a value of zero is desired.

The mean absolute error (MAE) is a linear score that calculates the average magnitude of the error. The MAE is defined as

The MAE range is from 0 to infinity, and a MAE of zero is desired.

Results

For purposes of the following discussion, the 29-km Meso Eta will be referred to as the Eta-29, and the 10-km Meso Eta as the Eta-10.

Temperature

A definite cold bias was apparent in the temperature analysis (Fig. 1). This cold bias occurred in the Eta-10 forecasts and both model runs of the Eta-29. This result was somewhat expected. Staudenmaier (1996b) discussed the details in which the Eta model was initialized. Due to the differences between the model surface and the true station elevation during initialization procedures, surface data typically was not used as is the case with surface data from Reno. 2 m temperatures in the Eta models are generated from a computer algorithm which interpolates the temperature using the mid-point of the lowest model surface. Because model levels are relatively thick (deep) over the Great Basin as compared to the more shallow surface layers in the eastern United States, this leads to a cold bias. That is, more extrapolation has to occur to get to the real surface in the west. This extrapolation also doesn't account for many of the boundary layer processes which occur at the surface-air interface (e.g., superadiabatic layers). What is noteworthy is that the Eta-10 bias is less than the 0300 UTC Eta-29 model run. This would indicate that the increased resolution improved the temperature forecasts as a whole.

In addition, these results are similar to those reported by Nutter and Manobianco (1999). Their Eta-29 evaluation revealed a cold bias of as much as 5.4-10.8 F at Edwards Air Force Base in California. The data they used covered the warm season (May - August) of 1996, and during the same period in 1997 they reported that model physics changes to components of the soil, cloud, and radiation packages in the 29-km Meso Eta increased this bias.

It is interesting to note the cyclic nature of the biases. The Eta-10 and the 0300 UTC run of the Eta-29 are similar, which one would expect since they are run at similar initial times. The 1500 UTC Eta-29 run biases were somewhat colder than the Eta-10, but not as cold as the 0300 UTC Eta-29 run. The largest errors with the Eta-10 and Eta-29 0300 UTC run occurred around the 0- and 24-h forecasts. This would occur near the time of maximum heating. Recalling the previous discussion, one would expect that the model would not be able to derive an accurate 2 m temperature close to the maximum due to the elevation versus model topography differences. The 1500 UTC Eta-29 run mean errors are not as easily explained. There is some evidence of the problem at maximum heating at the 9- to 15-h forecast period where the greatest error exists. However, all model runs have the greatest error at around 0000 UTC, again mostly likely due to the model topography differences.

The method used for data extraction from the gridded data and the representation of the station may have also affected the results. Reno lies in a valley just west of a strong topographical gradient to the west, with about a 1890 m (6200 ft) elevation change within 8 km (5 mi). A cause of the bias errors produced by both the Eta-10 and Eta-29 models may again be the result of the model topography versus actual topography (Staudenmaier 1997). The Eta model family uses step functions to simulate mountain terrain (Staudenmaier 1996a). The grid scale of the model determined how accurately the step function depicted the mountain and valley complex of western Nevada. The step function determined the elevation of the model surface above sea level. Reno is located near a mountain crest, in a valley surrounded by more, lower mountains. The valley was smaller than the Eta-29 grid size and about one and one-half times the Eta-10 grid size. The models could not distinguish the valley from the surrounding mountains clearly. Therefore the cold bias could be caused by the model terrain being about 1524 m (5000 feet) above the valley floor. The other possibility is that the grid points selected to represent Reno from the GEMPAK program were not very exact. It should be pointed out theat this data was transformed from the native model grids to an intermediate grid, which was then transformed into a GEMPAK grid. This introduces a few levels of smoothing and potential interpolation to the data before it was extracted. This problem is even more significant when dealing with the Eta-29 which gets remapped to a 40 km grid. This all could result in biases not related to model physics.

Figure 2 represents the MAE for the Eta temperature forecasts. Again, the Eta-10 errors are less than both the Eta-29 model runs. But, the MAE is indicating that model errors with magnitudes of 10-15 F are possible. These errors show the largest error to be associated with the time of maximum heating, similar to the bias. Again, these are mostly likely the result of the method of deriving the 2 m temperature grids and the vertical resolution differences of the model.

Dewpoint Temperature

The bias for the dewpoint temperature forecasts showed a somewhat reversed pattern to that of the temperature bias (Fig. 3). In general, the Eta-10 and 0300 UTC Eta-29 run were very similar with an overall cold (dry) bias. The exception was the bias between the 15- and 24-h forecast periods where the bias was nearly zero. Locally this is in the afternoon close to maximum heating. One would typically expect the model dry bias to be worse in the afternoon when low-level moisture has been mixed out. It is suggested that this is opposite of the temperature bias due to the fact that the derivation of the 2 m dewpoint temperature fields are most likely very dry to begin with. In Reno during late spring and summer it is very common to have extremely dry boundary layers, sometimes up to 500 hPa. Again, the problems with initialization and derivation routines probably lead to on-the-average dry surface dewpoints which were not much different than the low afternoon dewpoints. Less error is indicated in the 1500 UTC Eta-29 run, though a cold (dry) bias is still evident. Nutter and Manobianco (1999) also found a general warm (moist) bias in dewpoint temperature, with warm season biases of up to 9 F at Edwards and 0-4 F at two other sites in Florida during 1996. After the model physics changes in February 1997, their results during 1997 showed a drier bias of the dewpoint temperature. It is suggested that these difficulties could relate to problems with PBL mixing and/or incorrect specification of soil moisture processes. In addition, errors could be exacerbated by elevation errors (again the model elevation differences) while translating mixing ratios into 2-m dewpoint temperatures.

Figure 4 presents the MAE for the Eta dewpoint temperatures. Errors ranged from around 2-8 F. Eta-10 and the 0300 UTC Eta-29 run errors are nearly the same, except with larger Eta-10 errors after 24 hours. All three model runs had nearly the same errors through the 15- to 21-h forecast periods.

Wind Speed

The wind speed biases for the Eta model forecasts displayed characteristics similar to each other. However there were some significant differences. Again, the Eta-10 and 0300 UTC model runs displayed a negative bias around the 24-h forecast, indicating that the model forecasts were not as strong as the observations during the climatological strongest time of data for winds. The derivation of the 10 m winds assumes some reduction of the lowest model layer u- and v-component variables down to the actual surface. The result is most likely an underestimate of the surface wind speed in the model. In addition, in the summer a strong west wind occurs in the afternoon driven by the movement of the surface thermal trough which is not forecast well in model data.

The Eta-29 bias for wind speed also showed a tendency to under forecast wind speed in the afternoon hours and over forecast wind speed at night. This tendency was apparent regardless whether the data was from the 0300 UTC runs or the 1500 UTC run. The wind pattern observed from ASOS showed that winds typically decreased in the evening and became very light at night through the morning hours. Winds would then increase in the afternoon. This would account for the cyclic pattern shown in the analysis. Eta-29 warm season biases found by Nutter and Manobianco were as bad as 9 kts, but in general ranged from 0 to 4 kts, similar to our findings.

The final analysis of the model output data compared to the ASOS observations was the MAE of the wind speed forecasts. The only noteworthy point are the increased errors from the Eta-10 run between the 18- and 24-h forecast periods. This again could be related to the afternoon underforecasts of wind speed by the Eta-10 model, enhanced by the topography problem. No other explanation is offered.

Conclusions

Meso Eta forecasts for Reno, Nevada of temperature, dewpoint temperature, and wind speed were compared to ASOS observations for the period of early April through mid-August 1997. The 10 km Meso Eta was found to have a noticeable negative bias in the forecasts. This is attributed mainly to the initialization process, a combination of model topography differences between the model surface and real station elevation, and the method used to derive the 2-m grids and the points verified. In addition, the 10 km Meso Eta was found to have less error when compared to the 29 km Meso Eta, suggesting that increased resolution improves temperatures forecasts and to some extent the dewpoint temperature and winds. But, model MAE still indicate that there are substantial errors to be addressed. It is suggested that emphasis be put on improving methods of using more data in the model initialization, as well as an attempt to improve the methodology to extract the 2 m and 10 m surface fields from the Eta grids, most likely by increasing the resolution of the lowest layer of the models. Another suggestion for forecasters is to use the hourly model data that has been extracted for a specified location straight from the model grids, thus preventing any smoothing or interpolation sources of error or biases. Improvements in these areas will lead to better model grids which the field forecasters will use in the future for forecast product generation and make the process of IFP much less forecaster intensive, leaving more time towards watch and warning processes .

References

Black, T.L., 1994: The new NMC mesoscale Eta Model: Description and forecast examples. Wea. Forecasting, 9, 265-278.

Black, T.L., and Coauthors, 1997: Changes to the Eta forecast systems. NWS Tech. Proc. Bull. 441. National Oceanic and Atmospheric Administration/National Weather Service, 16 pp.

Nutter, P.A., and J. Manobianco, 1999: Evaluation of the 29-km Eta Model. Part I: Objective verification at three selected stations. Wea. Forecasting, 14, 5-17.

Staudenmaier, M., 1996a: A description of the Meso Eta model. Western Region Technical Attachment, WR96-06, 7 pp.

Staudenmaier, M., 1996b: The initialization procedure in the Meso Eta model. Western Region Technical Attachment, WR96-30, 11 pp.

Staudenmaier, M., 1997: Why is high resolution important in the West? An early look at the Eta-10 model output. Western Region Technical Attachment WR97-03, 18 pp.

Staudenmaier, M., and J. Mittelstadt, 1997: Results of the Western Region evaluation of the Eta-10 model. Western Region Technical Attachment WR97-18, 12 pp.