Outstanding problems in the assessment of convective forecast skill are: a) a measure of convection that matches the convective forecast criteria, and b) the use of appropriate assessment methodologies. In previous CCFP skill studies, National Convective Weather Detection (NCWD) data has been used as the observed convection in different ways. In this study, CCFP skill is assessed using Convective SIGMETs (CSIG) as the observed convection. Although the criteria used for generating CCFP and CSIG areas are different, they have enough similarities to find value in comparing them in this manner.

The purpose of this study is to introduce three new methodologies for assessing CCFP skill using CSIG which are: a) diurnal verification from evaluation of each grid point, b) attempting to assess “forecaster work” in cases where CCFP and CSIG areas did not occur (“null cases”) using the Rapid Update Cycle (RUC) model K-Index analyses, and c) jet airway segment (JAS) impacts defined as an area 20 nm either side of a portion of a jet airway into a major airport having a “yes” or “no” value for the occurrence of a CCFP and/or CSIG area on the entire JAS.

The most significant findings of each of these methods are: a) diurnal CCFP skill scores have high missed forecast rates at mid-day, high PODs and CSIs in the late afternoon, and high FARs and BIASs in the evening, b) “forecaster work” in “null cases” during peak convective periods was deemed to be potentially significant approximately half the time over high traffic areas in the eastern CONUS, and c) resultant skill scores using the JAS method were generally in agreement with other methods for assessing CCFP skill.