Hi Everybody,
As I work towards a new release, I’m also updating some of the help and reference stuff, and I’ve re-done my analysis of mean absolute error. I’m using Fahrenheit degrees, so for Celsius, divide these temperatures by 1.8. For over two years now, I’ve been running five parallel versions of WXSIM (forecasting for Atlanta), with WXSIM-Lite mixtures of 0, 40, 60, 80, and 100%. At times, one or another of the instances has failed, so I’ve meticulously weeded out all forecast times that don’t have all 5 run. Also, last spring I messed up and left the 0% one off for nearly three months! Anyway, I still got 2306 forecast times (a total of 11,530 forecasts!).
Here’s what I’ve found:
- Pure WXSIM-Lite always beat pure WXSIM (though they were almost tied for summer)
- An appropriate mix of the two is always better than either one alone (most dramatically in spring and summer).
- There has been a small but significant improving trend during this study, possibly due to the addition of year (or more) old data in autolearn and WXSIM-Lite analysis runs.
- The optimal WXSIM-Lite mix appears to be about 60% in spring and summer, about 65% in fall, and about 75% in winter. If you want to leave it constant, the best values are anywhere from 60-70% (recommending 65).
- Minima are forecast more accurately than maxima (this may vary a lot by site).
- MAE for both maxima and minima increase by about a third of a degree per day (going later into the forecast).
I’m also posting some graphics here.
Interesting stuff!
Tom