Forecast Validation Statistics

Forecast Validation 2009image002

To assess how useful the wave packet diagnostics based on a single deterministic forecast are, the forecast hovmoller diagram is validated against the analysis. Here, the forecasts from January 13th through January 31st, a total of 38 forecasts, are validated. We will update the validation statistics as we build up more archives.

The correlation between the forecast and analyzed hovmoller plot (based on averaging 300 hPa v between 30-60N) as a function of longitude and time, is plotted on the LHS. The RHS plot gives the RMS difference between the forecast and analyzed hovmoller plot (normalized by the analyzed standard deviation at each longitude and time).

image003If we take a correlation of 0.6 as being useful, then the LHS plot suggests that the deterministic forecast should be useful out to nearly 10 days over east Asia and western Pacific, and out to 6-8 days elsewhere. The RHS plot also shows that the normalized RMS difference is also smallest over east Asia and western Pacific. This may be related to the findings of Chang and Yu (1999) that wave packets propagate most coherently across Asia.

Forecasts of wave packet envelopes have also been validated. The correlation between forecast and analyzed hovmoller diagrams for both v (shown in black) and envelope (red), averaged over 0-360E and 120-180E, are shown in the figure below. We can see that out to day 10, the envelope is forecast a bit better than v itself. Both v and the envelope are forecast better over the western Pacific.

 

Forecast Validation (2010)image001

To assess how useful the wave packet diagnostics based on a single deterministic forecast are, the forecast hovmoller diagram is validated against the analysis. Here, the forecasts from January 13th through February 5th, 2010, a total of 48 forecasts, are validated.

The correlation between the forecast and analyzed hovmoller plot (based on averaging 300 hPa v between 30-60N) as a function of longitude and time, is plotted on the LHS. The RHS plot gives the RMS difference between the forecast and analyzed hovmoller plot (normalized by the analyzed standard deviation at each longitude and time).

image002If we take a correlation of 0.6 as being useful, then the LHS plot suggests that the deterministic forecast should be useful out to 6-8 days over all longitudes. Compared to 2009, the correlation across the Western Pacific appeared to have lessened. This may be related to the observation that wave packets over the Pacific during January-February 2010 appeared to be less coherent than their counterpart in 2009.

Forecasts of wave packet envelopes have also been validated. The correlation between forecast and analyzed hovmoller diagrams for both v (shown in black) and envelope (red), averaged over 0-360E and 120-180E, are shown in the figure below. Unlike 2009, v is forecast betterthan envelope, again possibly related to less coherent wave packets in 2010.