As quite a few people are now using autolearn, this “break-in” period has let me know about some of the potential issues that can come up, as I’ve worked with users to resolve some problems. Most of these have been very simple things, related to proper use of the program (for which my directions may not be as clear as they could be at this point). Here, in no particular order, are some things to know:
(1) wret.exe (which is activated and used by autolearn) is designed to “handle” only 9999 forecasts. I never anticipated people would have more than that, but some do! People running it every hour can hit that limit in a bit over a year. I don’t think running every hour adds much to the forecast accuracy, though it’s OK to do it. Maybe every two hours, but less often in the middle of the night, would be a good compromise. Anyway, if you have a huge number of forecasts, I suggest moving all but the last three months to another folder, with a name like “c:\wxsim\oldforecasts” (or whatever you like). Just don’t make wret.exe have to sort through thousands of forecasts! 
(2) While I’ve tried to make wret.exe able to “deal with” various kinds of “bad” forecasts, there may still be some types of problems I haven’t “prepared it for” yet. If you get some kind of crash of wret.exe during the process, please take a look at details.txt, which wret.exe writes as it runs. This may clarify what file it had trouble with.
(3) Related to the above, I did just the other day discover a problem which I am correcting. Forecasts that were somehow aborted before finishing 4 days cause some confusion in the analysis. I had a couple of these (they were manual forecasts for experimental purposes) which messed up the dew point analysis in particular. In my source code, I’ve got this fixed, but haven’t tested enough to release yet. Meanwhile, if you know of any “bad” forecasts like that, it’s best to delete them.
(4) It’s easy to forget to tell WXSIM to actually use the results! Be sure to go to Preferences/Settings, and to the 4th tab and check ‘Use learned bias corrections’ if you intend these results to be incorporated in the forecasts.
(5) The “View correc.txt” button seems to cause problems on some computers. I don’t know exactly why, but what this button does is to make a little VB Script to launch Notepad with correc.txt in it. If this doesn’t work, or if an error happens after trying it, just use Notepad (or whatever you like) if you want to view correc.txt. I few people have found it necessary, it seems, to delete correc.txt after such an error. It’s better not to do that if you don’t have to, though. Correc.txt is not actually used in the process, but it’s a very handy record of how things are going. The goal is for the “MAE” to get smaller than the “raw”, and hopefully almost as small as the “pred”. All these will vary over time. I would be very happy if the “pred” actually shrinks, too, but I’m not sure if it will.
(6) The starting day and number of days in wret.exe should be left alone. One would almost always want “1” as the starting day. The number of days is just how many days you want shown on graphs or text, or written into lastret.txt. This has nothing to do with Auto Select or autolearn (though I can understand the confusion). Autolearn temporarily sets the number of days to 4, for its own purposes, but then sets it right back so you don’t need to worry about it.
(7) The type of weather station software and the path to the folder containing the log files must be set in wret.exe before using autolearn. Do this by selecting a forecast in wret.exe, and then click Comparison to Actuals. You will be prompted for such information. After entering it, just click Close on that Auto Compare form and try again. Exit wret.exe properly (NOT using the little red Windows “X”).
(8) The “Name” of the site in autolearn’s upper left text box is your actual site name (like “Atlanta”), NOT a METAR code (like “ATL+72219”).
(9) wret.exe can easily look frozen, when it’s not. It just gets very busy and sometimes “forgets” to refresh the Auto Compare form, usually leaving it blank. I can look into ways to fix this, but meanwhile, just be patient. The details.txt files includes the run time for the process, so you can get a good idea of how long to wait.
(10) The graph of error versus departure from normal has a sort of “artifact” which produces streaks of dots slanted from lower left to upper right. I’ve thought about this carefully, and it’s actually correct. It results from cases where many forecasts were made in the same day. There is obviously a connection between the forecast’s departure from normal and the error, because, when compared to what actually happened, the warmer forecasts with more positive errors are naturally the ones which are further above normal! I actually have a sort of 80:20 mix of whether it’s the forecast or the actuals which are being compared to normal. This choice determines to slope of the sets of dots. The only problem this may cause is if you have a lot of forecasts per day and not many days analyzed, you may get a big (out of allowed range) slope correction. For this reason, I think you should have no more than perhaps 10 forecasts a day and should use at least 30 days in the analysis.
(11) If you want to run autolearn on an external scheduler, be sure to check the ‘Close after execution …’ box.
(12) An appropriate “debugging” procedure is to first use wret.exe to see if you can do a Compare to Actuals display on a few random forecasts from 5 to 45 days ago. Then you might try the Auto Select routine in wret.exe, where you first do a Comparison to Actuals, then click a forecast and click ‘Set as Earliest Date’, then click on a somewhat later (but at least 5 days ago) forecasts and click ‘Auto Select’. If that works, then close wret.exe and open autolearn.exe, make sure your settings are right there, and try a ‘Run Immediately’. If some problem occurs during this sequence of experiments, please let me know how far you got.
Hope that helps!
Tom
