Cloud Regime Verification using Enhanced Model Evaluation Tools

The enhanced Model Evaluation Tools (METplus) verification system offers a wide range of applications with different temporal and spatial scales. The METplus framework and its many use cases have been transitioned to some operational centers such as NOAA and NRL. This study focuses on the usage of Method for Object-Based Diagnostic Evaluation (MODE) to verify cloud products derived from the NRL’s regional model – COAMPS and GOES-16 data.

The MODE within the METplus package provides a quantitative feature-based spatial verification between an observed field and a forecast. MODE provides statistics beyond the traditional verification metrics such as the Probability of Detection, Root Mean Squared Error, and Equitable Threat Score. Feature based verification treats forecast and observed cloud regimes as discrete features with identifying attributes (shape, location, coverage, etc.), therefore can offer more in-depth insights on spatial forecast quality, which can feed back to model improvement, or can be helpful for making decisions.

MODE can be run with a configuration file with a wide range of flexibility to define, process, match and merge objects. However, there is little evidence on how various parameters in the configuration file affect the MODE output statistics. Therefore, the fidelity of these statistics is in question. This study will demonstrate how configurable parameters in MODE affect various cloud regime verification statistics. A brief summary of the regional cloud forecasts at different regimes will be reported as well.