cloudy cloudy

Author Topic: Possible autolearn/wret bug, and hopeful fix!  (Read 583 times)

0 Members and 1 Guest are viewing this topic.

Offline Tom Ehrensperger

  • Posts: 2,562
  • OS/Browser:
  • Win NT 10.0
  • Firefox 54.0
Possible autolearn/wret bug, and hopeful fix!
« on: August 07, 2017, 02:01:40 PM »
I have been trying to figure out a problem, reported by two or three users, in which ticking the box in autolearn for using previous years' data hurt forecast accuracy, when it should have been helping.  I'd been sent some files (mainly correc.txt) with puzzling effects of ticking the box, the most consistent being that the diurnal range bias correction value ("rgcor") would hit its ceiling value of 1.25 when that box was ticked.  The effect is mainly to makes the daytime highs a bit too warm and/or nighttime lows too cold.

Thanks to a recent big batch of compare and contrast files Henrik sent to me, I *think* I've figured it out (and am awaiting his test results), but it's an important enough issue so I'm going public with it now.  I realized that in a little temporary file which communicates between autolearn and wret, I had put values on one line separated by commas.  However, in countries where commas are used as decimal separators, these get all strung out in one line, so that when wret reads it, it chops it up wrong and gets a mixture of whole numbers and zeros, instead of separate decimal values.  I've long known about this issue, and have addressed it throughout my programs, but somehow this one slipped through!

Anyway, if I'm right, this is good news, because if you've been suffering from this bug, your forecasts may be about to get better than ever!  If you think you might be affected (or even if you're not), here are the new versions to try.  They need to be done together:

www.wxsim.com/autolearn.exe
www.wxsim.com/wret.exe

Use these to replace your existing ones.  Autolearn is now Version 3.5, and wret is still Build 1.2, but with a "b" after it.  If this fixes the problem, you can tell because in correc.txt, a column of 1.25's may suddenly turn into something closer to 1.

One piece of good news: I do not thing there are many "after-effects" of this which would preclude immediate or continued use of the use of older data in the analysis, because wret "knows" what bias corrections were used in the forecast - even if they were bad ones - and calculates knew ones with that taken into account.  It still may take it a while to settle into optimal performance, but it should start to get better right away.

Let me know what you find!  And sorry about the bug!!

Tom

Offline Mearns

  • Posts: 165
  • Glenbervie, N.E. Scotland
  • OS/Browser:
  • Win NT 10.0
  • Firefox 54.0
    • Glenbervie Weather
Re: Possible autolearn/wret bug, and hopeful fix!
« Reply #1 on: August 07, 2017, 03:22:54 PM »
Thanks Tom - I had quite a few rgcor 1.25 records.  I've downloaded both files but Autolearn is still showing as version 3.3 .......
Tony
Wireless Davis VP2 USB
WD 10.37S Build 55 on Windows 10 64bit
www.glenbervie-weather.org

Offline tornadochaser

  • Posts: 22
  • Ober÷wisheim - Germany
  • OS/Browser:
  • Win NT 10.0
  • Firefox 54.0
    • Kraichtal-Wetter
Re: Possible autolearn/wret bug, and hopeful fix!
« Reply #2 on: August 07, 2017, 03:35:03 PM »
Thanks Tom - I had quite a few rgcor 1.25 records.  I've downloaded both files but Autolearn is still showing as version 3.3 .......

Hi Tom,

Still 3.3 when I switched the files too and I also got this two messages when I tried to run autolearn


Offline Tom Ehrensperger

  • Posts: 2,562
  • OS/Browser:
  • Win NT 10.0
  • Firefox 54.0
Re: Possible autolearn/wret bug, and hopeful fix!
« Reply #3 on: August 07, 2017, 04:12:16 PM »
I think I had a bug in my bug fix!  Try this again:

www.wxsim.com/autolearn.exe

It still says version 3.5.  When you download it, besure to direct it into c:\wxsim and not a subfolder.  You shoud be asked (by Windows) if it's OK to overwrite the existing file.  Say yes.

I'll get this right soon! :)

Tom

Offline Tom Ehrensperger

  • Posts: 2,562
  • OS/Browser:
  • Win NT 10.0
  • Firefox 54.0
Re: Possible autolearn/wret bug, and hopeful fix!
« Reply #4 on: August 07, 2017, 07:07:54 PM »
I *think* this problem was limited to users in countries where the comma is the decimal separator, but I'd certainly like to check results in general just in case there's more to it.  The files I'd like to see to check on the function of this are correc.txt, cortest.txt, and wretlog.txt.  I actually just uploaded an even newer wret.exe:

www.wxsim.com/wret.exe

It says 2017 Build 1.2c this time.  Its only change since the one earlier today is that it writes some data to wretlog.txt (which is just a diagnostic for me to look at) in a less ambiguous way.

Thanks!

Tom

Offline Tom Ehrensperger

  • Posts: 2,562
  • OS/Browser:
  • Win NT 10.0
  • Firefox 54.0
Re: Possible autolearn/wret bug, and hopeful fix!
« Reply #5 on: August 07, 2017, 09:48:23 PM »
Henrik's problem doesn't seem fixed yet, though I do think the changes I made already were necessary.  I've just posted wret Version 2017 Build 1.2d:

www.wxsim.com/wret.exe

and be sure you also got

www.wxsim.com/autolearn.exe

It has an additional diagnostic file: alrn2.txt, which I'd like to see along with wretlog.txt, cortest.txt, and correc.txt.  I'm interested in verifying normal, successful operation (as most users are seeing, I think), as well as diagnosing and fixing this problem.  Again, it's if you are using autolearn with the box checked about using last two years' data.

Thanks!

Tom

Offline Tom Ehrensperger

  • Posts: 2,562
  • OS/Browser:
  • Win NT 10.0
  • Firefox 54.0
Re: Possible autolearn/wret bug, and hopeful fix!
« Reply #6 on: August 08, 2017, 06:50:12 PM »
I found yet another problem with the reading of those comma-decimal numbers, and I think I've fixed that.  If you'd like to take a chance that I've got it this time, here are autolearn 3.6 and wret 2017 Build 1.2e.  They need to be used together:

www.wxsim.com/autolearn.exe

www.wxsim.com/wret.exe

Let me know if this works!

By the way, the problems this hopefully solves involve two conditions:

(1) you live in a place where commas are used instead of periods for decimal separators and

(2) you were using the 'use last two years' check box in autolearn

The 'symptom' you would have been getting would be, in most cases, too large of a difference between mins and maxes.  In some cases, the resulting errors would be slight, and in others it would be fairly noticeable, but not completely terrible.  Now it should be a lot better, though!

Tom


Offline Tom Ehrensperger

  • Posts: 2,562
  • OS/Browser:
  • Win NT 10.0
  • Firefox 54.0
Re: Possible autolearn/wret bug, and hopeful fix!
« Reply #7 on: August 08, 2017, 08:54:30 PM »
Tentative good news!  Henrik has tested this latest version (see last post) and it seems to have fixed the problem.  If you were having this problem, your forecasts may be about to get better than ever! :)

I would like to see some posted samples, though, of cortest.txt and correc.txt.

Tom

Offline runarbg

  • Posts: 610
  • Drammen, Norway
  • OS/Browser:
  • Win NT 10.0
  • Chrome 60.0.3112.90
    • Konnerud weather
Re: Possible autolearn/wret bug, and hopeful fix!
« Reply #8 on: August 10, 2017, 06:55:45 AM »
Rgcor has been 1.2500 every day since April 2016 for me. After installing the latest version it shows 1.1478 today  :D

Offline tornadochaser

  • Posts: 22
  • Ober÷wisheim - Germany
  • OS/Browser:
  • Win NT 10.0
  • Firefox 54.0
    • Kraichtal-Wetter
Re: Possible autolearn/wret bug, and hopeful fix!
« Reply #9 on: August 10, 2017, 07:50:37 AM »
Seems to be fixed here as well, thanks! :)

 

cumulus