Assessing the Meteorological Skill of the Navy Weather Model (COAMPS)

Assessing the Meteorological Skill of the Navy Weather Model (COAMPS)

Authors: Rachel Kennedy, Sandra Yuter, Matthew Miller

North Carolina State University, Raleigh, NC

Abstract: The Coupled Ocean/Atmosphere Mesoscale Prediction System (COAMPS) is an operational weather forecasting model developed and used by the United States Navy. This study aims to analyze model biases under different meteorological conditions, including the amount of observed cloud cover, time of day, location (coastal vs inland), season, and wind speed, to see which conditions have the strongest influences on model biases. We compare COAMPS and NOAA GFS model output to surface observations at more than 300 weather stations in the US, Asia, and Europe. Identifying conditions where a model has a higher and lower forecast skill helps to constrain where refinements in different aspects of model physics will have a larger impact and helps model users to account for typical biases.

We use several types of evaluation metrics to assess overall model biases. Matched time comparisons of model output - observations are a standard metric but apparent biases can result from small storm timing and location errors. We use bulk distributions of a given variable for a time of day, region, and season to determine if biases detected in matched-time comparisons are consistent when the timing criterion is relaxed. We have also developed several metrics to quantify timing errors in key variables the models forecast. We use pressure tendency to determine the timing of low pressure center passages and assess to what degree the model forecasts low pressure center passages too early or too late. In addition, we examine forecast and observed start and stop times of precipitation events. There is regional geographic spatial coherence to the timing errors in low pressure center passages and in the timing errors of precipitation events.

The model physics are identical and the initializations are very similar between two COAMPS regional model runs that cover California with different grid spacing. We examined temperature errors at several lead times for COAMPS NEPAC (15.5 km grid) and COAMPS CENCOOS (3.65 km grid) for the summer season from May - Sep 2022. Most California locations have warm biases (positive temperature error of model - observation) with CENCOOS typically having stronger warm biases than NEPAC. Warm biases were present both in the matched time comparisons and in the seasonal temperature distributions for forecast versus observed temperatures. Examination of temperature biases conditioned on the amount of observed cloud cover indicated that these errors in COAMPS are not apparently a function of cloud cover.