WESTERN REGION TECHNICAL ATTACHMENT
NO. 03-02
MARCH 24, 2003
THE MATCHOBSALL ANALYSIS SYSTEM
Todd Foisy, Western Region-SSD, Salt Lake City, Ut
Introduction
MatchObsAll is a collection of GFESuite SmartTools and procedures developed by Tim Barker (WFO Boise) that generate an hourly gridded analysis of temperature, dewpoint, and wind as well as derived grids of relative humidity, max/min RH, and max/min temperature. The SmartTools and procedures are written in Numerical Python, based on an analysis scheme developed by Les Colin, WFO Boise (Barker, 2003a). This analysis system is powerful in that it may be run locally and is highly customizable. In addition, it runs fairly quickly, and produces an analysis on the same grid as the local IFPS forecast grid. Because the analysis is produced at the same resolution as the IFPS forecast grid, verification can be done for all parts of the forecast zones. The MatchObsAll software distribution is available for download from the National Weather Service Smart Tool Repository (http://www.mdl.nws.noaa.gov/~applications/STR/).
Gridded Analyses
The uses of gridded analyses in operational
and research meteorology are wide and varied, ranging from the initialization
of numerical models to the accurate representation of the atmosphere. The utility
of gridded analyses is dependent on the dominant scales of motion, data spacing,
available computational resources and most importantly, the intended application
of the analysis. Analyses generated to initialize numerical models will have
different attributes and should be judged differently than those intended solely
to best represent the atmosphere. Analyses used in the data assimilation of
numerical models are tuned to produce the best forecast within that system,
and may not produce the best analysis or representation of the atmosphere (UCAR,
2003). The focus of this technical attachment is on the use of analyses to best
represent the atmosphere with application in operational meteorology (nowcast)
and verification of gridded NWS forecasts (IFPS).
The quality of gridded analyses intended as accurate representations of the
atmosphere is dependent upon both the background field and the observations
used. The background field is typically supplied by a model forecast, and provides
information and detail where there are few observations. Thus, a quality analysis
requires a quality background field, especially in data sparse regions. In addition,
a good background field avoids extrapolation far from observations and provides
dynamical consistency to the analysis. Observation correction modifies the background
field in order to provide the best possible representation of the atmospheric
state. Observation correction is dependent upon the weighted differences between
the observations and background values at observation locations. These weights
typically depend on the distance of an observation from the analysis grid point
and the expected error of the observations and background field. Finally, as
with the background field, the quality of the analysis is dependent upon the
number and quality of observations used in the correction. Few observations,
or observations that are invalid or contain large errors, will decrease the
quality and accuracy of the analysis. In an operational setting, data latency
can limit the quality of a gridded analysis. Observational and forecast data
sets are rarely available in a timely manner, which may preclude their use in
producing current gridded analyses.
Quality gridded analyses in complex terrain face additional hurdles. Complex analysis systems can in fact produce degraded representations of the atmosphere over larger areas. Data assimilation in complex terrain must be able to handle a wide variety of scale interactions, ranging from strongly forced to weakly forced. One possible solution to this problem is running the analysis scheme at higher resolution. This adds value only if high resolution, quality observations are used and or the background field is at a high resolution. High-resolution analyses based on coarse resolution background fields and sparse data is simply downscaling to a specified resolution's terrain.
The MatchObsAll Analysis System and the SERP Algorithm
The MatchObsAll analysis system was designed to mitigate several deficiencies typical to analyses that are available at the local forecast offices and used in both general operations and IFPS. The scheme is simple enough that it can be run efficiently on a local system at the forecast office level, allowing for local configuration and control. This includes the ability of re-running the analysis at subsequent time steps to take advantage of updated data and mitigate data latency in the observations. Because the system runs as a part of the GFESuite system, the analysis is common to the grid and resolution of the gridded forecast products produced by the forecast office as a function of IFPS. This is not only optimal for verification, but allows for the use of the WFO's gridded forecasts as the analyses' background field. A more detailed description of the components of the system as well as the installation instructions can be found in the distribution notes (Barker, 2003b).
The MatchObsAll system is configured to produce two analyses at each hourly time step of: temperature, dewpoint, wind, and derived grids of relative humidity, max/min RH, and max/min temperature. Each analysis utilizes different background fields: one generated using the previous hour's analysis and a second using model. These analyses are averaged, producing a single analysis minimizing the impact of bad observational data. Because the background field is taken from the GFE, it takes advantage of the GFE smart initialization, which downscales model data to much higher resolutions, accommodating the influence of topography.
Observation correction is carried out using METAR, mesonet and marine observations available via the local WFO's AWIPS system. Observations at or closest to the top of the hour are used, while less frequent data is time-interpolated. Because the analysis is re-run at each hour over and up to the last 6 hours, the amount of observations used in the analysis is maximized. There is no quality control performed on any of the observational data used in the observation correction. Future improvements to the system may include the ability to remove bad observations from the analysis manually.
The MatchObsAll system utilizes the SERP SmartTool (Barker, 2003a), which fits the observation corrections with a serpentine curve (Fig. 1). The SERP SmartTool forces the analysis to draw to the all the data points used in the correction, thus the analysis will match perfectly any observation used. In fitting the analysis to this data, the observation corrections are distance weighted, but there is no radius of influence. This allows for observation corrections to have larger weights at grid points near the observation location and at similar locations. However, because there is no radius of influence, the effect of the observation correction is not limited to those nearby grid points. Thus, the influence of an observation correction is spread throughout the analysis.
This observation correction scheme is quite simplistic and runs fairly quickly on the local GFE, generating analyses for a 257x257 grid, or over 66000 grid points, in about 10 minutes (Barker, 2003b). Typical system requirements will be determined by the size and resolution of the analysis domain (i.e. total number of grid points). The test cases below were generated on a Pentium 4, 2 GHz processor with 1.0 GB of RAM.
Examples
Figs. 2-4 show the observed temperatures and analyses from the MatchObsAll and 2km ADAS for a situation with light synoptic forcing during the late morning around the Salt Lake Valley of Utah. In this case, the MatchObsAll analysis (Fig. 3) and the 2km ADAS (Fig. 4) show similar temperatures in the Salt Lake Valley, as can be seen at the sample points. However, the two analyses differ greatly in the Wasatch and Oquirrh Mountains. The MatchObsAll analysis better represents the cooler temperatures at the higher elevations of the Wasatch and Oquirrh Mountains. Note the cold temperatures (6° F and 15° F respectively) recorded at the Hidden Peak and Farnsworth Peak MesoWest sites (Fig. 2). While the MatchObsAll captured these cooler temperatures with values of 6° F and 15° F at these peaks, the 2km ADAS analysis was 12-14° F too warm. In addition, the MatchObsAll analysis seems to be resolving the warmer temperatures (27° F ) in the Wasatch Mountain valleys in nearby Park City, while the 2km ADAS is considerably cooler (17° F) .
In this case, the MatchObsAll analysis does a better job than the 2km ADAS over this area because of the high density of MesoWest observations in this domain which are used in the observation correction. The analysis is forced to draw to the observations exactly by the SERP algorithm, while the 2km ADAS must follow other constraints.
Figs. 5-7
show wind velocity for a situation with little or no synoptic forcing during
the late morning near Dugway Proving Grounds in the West Desert of Utah. Winds
are light and up-valley (Northwest 5-10 kts) at this time (Fig.
5). The MatchObsAll analysis picks up on the light NW
breeze throughout the Dugway valley (Fig. 6) as a result
of the high density of MesoWest observations in this area. The 2km ADAS (Fig.
7), however, does not show these up-valley winds. The winds are similar
in both analyses over the Stansbury Mountains (east side of the image), where
no observations exist to correct the analysis, giving the background field the
predominant weight.
Summary
In summary, the utility of gridded analyses is determined by the intended application of the analysis. Analyses generated to initialize numerical models will have different attributes and should be judged differently than those intended solely to best represent the atmosphere. The quality of gridded analyses intended as accurate representations of the atmosphere is dependent upon both the background field and the observations used. A good analysis requires a quality background field, especially in data sparse regions, as well as numerous and accurate observations.
The MatchObsAll analysis system was designed to mitigate several deficiencies typical to analyses that are available at the local forecast offices and used in both general operations and IFPS. The scheme is simple enough that it can be run efficiently on a local system, allows for local configuration and control, minimizes data latency in the observations and has a common grid domain and resolution to the IFPS grids produced by the local forecast office. The common grid allows for easy use in forecast verification, and for the use of the IFPS gridded forecasts as the analyses' background field. The MatchObsAll analysis algorithm includes distance and elevation weighting without a radius of influence in the observation correction. This allows single observations to affect changes on the background field not just as nearby gridpoints but at potentially all gridpoints. Strong weights in the observation correction force the background field to match the observations exactly.
Acknowledgments
Special thanks to Tim Barker and Les Colin of WFO Boise for providing documentation and expertise on both the SERP smart tool and algorithm as well as the MatchObsAll analysis.
References
Barker, T. / National Weather Service, Boise, ID, cited 2003a: SERP User Documentation. [Available online at http://www.mdl.nws.noaa.gov/~applications/STR/.]
Barker, T. / National Weather Service, Boise, ID, cited 2003b: MatchObsAll User Documentation. [Available online at http://www.mdl.nws.noaa.gov/~applications/STR/.]
UCAR / COMET, cited 2003: Numerical Weather Prediction Professional Development Series: Understanding Data Assimilation. [Available online at http://meted.comet.ucar.edu/nwp/pcu1/ic6/index.htm.]