Introduction

Once each year, the National Centers for Environmental Prediction (NCEP) hosts a review of their model production suite. During the meeting, each national modeling center presents accomplishments of the past year and plans for the coming year. The NCEP forecast centers, NWS regions and representatives from forecast offices, as the primary users of the models, present feedback and requirements. The ensuing discussion helps to define the important issues currently facing developers and users of NCEP models. This Technical Attachment (TA) describes four of the "most significant" issues facing NCEP modeling, based on the 1997 Annual Review held December 3-4, 1997.

The Debate over NGM Model Output Statistics (MOS)

Ron McPherson (Director of NCEP) stated that NCEP does not plan to make the NGM model Y2K (Year 2000) compliant or make the changes necessary to run the NGM on the next generation of supercomputers. All scientific and technical development work stopped on the NGM several years ago. Making the NGM Y2K compliant and able to run on the new supercomputers would require considerable work and divert NCEP development resources away from the newer models. However, discontinuing the NGM is not without impact.

The primary arguments in favor of running the NGM beyond the year 2000 include: (1) The TDL MOS guidance for max/min temperatures, PoPs, etc., is derived primarily from the NGM; (2) NWS verification depends on NGM MOS; (3) Forecasters have a lot of experience using the NGM and NGM-BASED MOS.

Since the primary argument for continuing to run the NGM is MOS, there are several activities underway: (1) TDL is developing Eta and AVN MOS to replace the NGM MOS (Fig. 1). The Techniques Development Laboratory (TDL) plans to make these new MOS available by April 1998, well before the scheduled demise of the NGM. (2) A technique using seven-day running averages of Eta-29 biases to forecast max/min temperatures has been tested by Baldwin and Hrebenach (1998). These max/min temperature forecasts compared favorably to NGM and AVN MOS, and testing of other variables is underway. NCEP is also testing a technique that uses a Kalman Filter to remove model biases. These latter two techniques, if successful, would eliminate the need for a long record of model statistics from a frozen model.

Due to the resolution and physics improvements of the Eta and AVN models over the last five years, cases where NGM forecasts are superior to other NCEP models are very rare. The diminishing NGM performance has been documented many times, for example by the NCEP QPF Statistics (Staudenmaier, 1996) and by a recent Master's Thesis (White, 1997). White showed that for 24-hour forecasts, the NGM scored poorest in 10 of 12 categories (Fig. 2). Model improvements are the primary reason for the NWS gain in skill over the previous 10 years in the 1 to 3 day range, and there is a need to phase out poorer performing models.

The general consensus expressed by the audience at the NCEP Annual Review was TDL should stick to their deadline of April 1998 for developing MOS for the Eta and AVN models. The April 1998 date would provide the forecast offices time to evaluate and become familiar with the MOS before the NGM MOS goes away. Therefore, NCEP should not use resources to make the NGM Y2K compliant nor to port it to the new supercomputer.

Modeling the Planetary Boundary Layer (PBL)

The WR evaluation of the Eta-10 revealed weaknesses in the simulation of the planetary boundary layer, including:

(1) low-level winds are too light,

(2) a 2-meter temperature cold bias,

(3) downslope mountain waves do not seem to form, and

(4) winds do not respond to strong surface temperature gradients, for example, over the Great Salt Lake.

The challenge posed by the PBL demands a paradigm shift for NCEP modelers who have been able to largely ignore the PBL because coarser resolution models either quickly adjust the PBL to their own climate or were unable to adequately simulate small scale interactions. For example, surface winds are not assimilated into the AVN/MRF model.

The first step in modeling the PBL is improving the initialization of wind, temperature, and moisture. Staudenmaier (1996) showed that over WR, less than 15 percent of surface and PBL RAOB data are used in the Eta Data Assimilation System (EDAS). A new EDAS uses a more sophisticated assimilation scheme to better capture more of this low-level data. The next step is to improve the performance of the model itself. The mesoscale (ETA) branch at NCEP recognized the importance of the PBL in their list of problems (Fig. 3) and by creating a small Eta-10 domain over Utah and Idaho for testing the PBL problems identified by WR.

Forecaster Training

Models are the most important tool for forecasting beyond 6 to 12 hours. However, with today's rapid rate of development, it is difficult for forecasters to stay abreast the strengths, weaknesses, biases, etc. of models and their assimilation systems. New tools and techniques can be misused; for example, a forecaster might assume the BUFR hourly model soundings represent a single point location, rather than a model grid box. Because of the need to rapidly respond to model developments, WR has proposed a dedicated effort to improve training on NCEP models. This issue, which is primarily a FTE issue, is currently under review by NCEP and NWSHQ.

Mesoscale Verification

A standard set of statistical measures for scoring synoptic forecasts, such as Root-Mean-Square Errors and Anomaly Correlations, is more or less agreed upon. However, forecasts on the mesoscale are much harder to verify, as stated by Drogemeier (1997):

Consider, for example, the situation in which a model produces a supercell thunderstorm of precisely the correct type, morphology, and timing, but with a position error of 20km. Traditional error scores would judge this forecast to be a failure, while it clearly has some value (based in part on the manner in which the information is to be used).

For example, a standard set of measures needs to be identified so that a mesoscale model running at NWSO Pendleton, Oregon can be compared to another running in Tallahassee, Florida. The National Science Foundation (NSF) recently provided funds and tasked CAPS to research a standard method of mesoscale verification. Plans were made at the NCEP review to send NWS field representatives to an upcoming meeting with CAPS on this issue.

Conclusion

Given today's rapid change of NCEP initialization schemes and numerical models, it is more important than ever to have a two-way exchange of information between forecast offices and NCEP developers. The NCEP review provides a valuable forum for this exchange to occur. The author welcomes questions and concerns about these or any NWP issues.

References

Baldwin, M. E. and Hrebenach, S.D., 1998: Experiments with Bias-Corrected Temperature Guidance Using NCEP's Mesoscale Eta Model; Preprints, AMS 16th Conference on Weather Analysis and Forecasting, pp 388.

Drogemeier, K.K., 1997: The numerical prediction of thunderstorms, challenges, potential benefits and results from real-time operational tests. World Meteorological Organization Bulletin, October 1997.

Staudenmaier, M.J., 1996: Precipitation Verification Statistics from the NCEP Operational Model Suite. WR-Technical Attachment 96-28.

White, B.G., 1997: Short-term Forecast Validation Of Six Models For Winter 1996. Masters Thesis, University of Utah, 99 pp.