VALUE Validation Indices and Performance Measures
The following list presents different classes of validation indices. For each meteorological variable, the specific values used to describe the indices (e.g. thresholds) are chosen based on expert knowledge (e.g., impact researchers, STARDEX report, etc.). This list will be updated on a regular basis.
mpe: mean percentage error
mse: mean squared error
|
Index |
Performance Measure |
|
|
|
Distributionwise Validation |
Pairwise Validation |
|
Marginal Distribution |
||
|
Mean |
bias/mpe |
|
|
Variance |
mpe |
|
|
Skewness |
bias |
|
|
Temporal Dependence |
||
|
time series |
|
mse |
|
acf lag 1,2,3 |
just indices |
|
|
quantiles/return values |
bias |
|
|
threshold exceedance |
|
Brier score/mse of 0/1 series |
|
number of threshold exceedances |
bias |
|
|
amount above threshold |
bias |
|
|
shape parameter of GEV |
bias |
|
|
time of maximum/minimum of annual cycle |
bias |
|
|
amplitude of annual cycle |
mpe |
|
|
spell length dist quants |
bias |
|
|
transition probabilities |
just indices |
|
|
proportion of variance in low frequency band |
just indices |
|
|
sign of the low pass filtered series |
|
Brier score/mse of 0/1 series |
|
Spatial Dependence |
||
|
distribution of daily relative areas of threshold exceedances |
see above |
mse |
|
range of smoothed variogram |
mpe |
|
|
madogram |
mpe |
|
|
Multivariate Dependence |
||
|
corr(detrended variables) |
just indices |
|
|
variable conditional on (no) exceedance |
see above |
|
|
joint exceedances |
see above |
|
|
Spatial Climatological Summaries |
||
|
all indices |
mean absolute error, mean bias |
|
|
absolute errors of spatial means |
|
|
|
mpe of spatial means |
|
|
|
means of Brier scores |
|
|
|
Taylor diagram |
|
|