Updated WXSIM-Lite optimal mix results

Hi Everybody,

As I work towards a new release, I’m also updating some of the help and reference stuff, and I’ve re-done my analysis of mean absolute error. I’m using Fahrenheit degrees, so for Celsius, divide these temperatures by 1.8. For over two years now, I’ve been running five parallel versions of WXSIM (forecasting for Atlanta), with WXSIM-Lite mixtures of 0, 40, 60, 80, and 100%. At times, one or another of the instances has failed, so I’ve meticulously weeded out all forecast times that don’t have all 5 run. Also, last spring I messed up and left the 0% one off for nearly three months! Anyway, I still got 2306 forecast times (a total of 11,530 forecasts!).

Here’s what I’ve found:

  1. Pure WXSIM-Lite always beat pure WXSIM (though they were almost tied for summer)
  2. An appropriate mix of the two is always better than either one alone (most dramatically in spring and summer).
  3. There has been a small but significant improving trend during this study, possibly due to the addition of year (or more) old data in autolearn and WXSIM-Lite analysis runs.
  4. The optimal WXSIM-Lite mix appears to be about 60% in spring and summer, about 65% in fall, and about 75% in winter. If you want to leave it constant, the best values are anywhere from 60-70% (recommending 65).
  5. Minima are forecast more accurately than maxima (this may vary a lot by site).
  6. MAE for both maxima and minima increase by about a third of a degree per day (going later into the forecast).

I’m also posting some graphics here.

Interesting stuff! :slight_smile:

Tom


errvshour.JPG

Hi Tom, where do you set the “65%” mix i havent touched it in over a year and forgot just about anything i did know
It just runs in the background, i know wxsim lite runs on schedule, as to whether it actually does anything at the time
i dont know, i see the box appear but nothing ever happens, that i notice…

Ok scrub the latter, it must be doing sommat, i just clicked on the 3 “show…” buttons lol still not sure where, oh let me
check wxsim itself…

Maybe not lol ok, i give up, where is it please?

PS, is auto learn meant to be in demo mode :0


Durinal.png

In WXSIM, go to Start/Auto Run and Other Settings, and then click on the big “Mix with …” button in the lower-middle part of that form. You’ll have the option to check usage of WXSIM-Lite and also set the percentage.

In looking at the graphs you posted, WXSIM-Lite is clearly coming up with significant corrections to the raw GFS data, but the fit your getting to actual temperatures (the last graph) is not as good as I’d like. I think it’s an artifact of it being winter, with lots of clouds and low sun angle making a really small diurnal range. That kind of makes me think you should keep the WXSIM-Lite mix down to about 50% until you see better agreement, which may happen as the sun gets up higher in the sky with spring approaching.

Tom