Sea Ice Model Skill Assessment

A challenge for sea ice modelling, particularly for the few days to weeks time scales, is how to verify model skill.  The challenge is aggravated by the fact that familiar skill measures, such as might be used for precipitation or temperature, grant very high scores even to simple persistence when applied to sea ice concentration.  The skill scores for persistence forecasts verifying on 23 May 2016 are given in the figure, for leads of 1 to 16 days.  Bias and RMS errors in concentration for the Northern Hemisphere persistence forecasts are in red (+) and green (x).  False Alarm Rate (blue, *) is where sea ice is forecast to be present, but is not.  False Confidence Rate (fcr, purple, boxes) is where the forecast says that there will not be sea ice in a grid cell, but the analysis shows that there is.  Late May is a time when the Arctic ice pack is starting to decline rapidly.  As such, it is the time when persistence should be at its least skilled.  Yet these skill levels are also comparable or superior to those for some complex sea ice models for the same valid date. 

We might attempt to follow 500 hPa skill methods and examine an anomaly correlation.  For sea ice, however, anomalies from a climatology will be dominated by zeroes and ones.  Most area will have either a zero anomaly -- whether that the places which had ice before still do, or those that didn't still don't -- or a +/- 1, that sea ice has advanced (or retreated) from an area.  The correlation will thence be quite high. There remains little scope for even anomaly correlation to inform about the skill of the model.

Essential to a proper operational objective skill measure for sea ice models is that:

• Little skill should be credited to persistence forecasts
• It may be computed with operational analyses and observations
• It has correspondence with subjective assessments from operational sea ice forecasters

Work continues, with such measures as above and modifications thereto.  The latest results will be presented at the conference, both what might satisfy these goals, and discussion of what doesn't and why not.

In the case of sea ice drift, the picture is much more straightforward.  Of 13 different measures, 5 are found to be useful. The index of agreement, for instance, between forecast and observed drift distance is not useful as it fails to find any skill difference between 1 and 16 day forecasts.  RMS error radius is not useful in the sense that it is not independant.  It is very highly correlated to the mean error radius.  The remaining 5 are mean error radius, vector correlation, linear correlation between forecast and observed drift distance, the slope of the regression line between forecast and observed drift distance, and the rms direction error.