Autolearn/wret bug discovered

I tried to post a few minutes ago, and something went wrong. It might appear eventually, but just in case it doesn’t, here’s the short version:

I found that if you started using autolearn in the first month or so after I released it in August, 2013 (or wret manually around then), AND live in a country where you use commas as decimal separators, AND have checked the box in autolearn to use previous years’ data, there’s a problem. It’s easy to spot. Look in correc.txt. If you see many lines of data, showing perhaps just one item on each line instead of lines with all the data (probably followed by “good behavior” starting by October, 2013 or so), you should make some changes, as it’s likely skewed your bias correction factors. My last post didn’t work becaus of my attachments. Let me see if this goes through …

Tom

OK, trying to post examples here. Here is a file demonstrating the problem, up until September 24, 2013.


correc-12.txt (213 KB)

Here’s a partial fix, by deleting the bad data early in the file …


correc2.txt (208 KB)

But there’s still a problem, because the user began using the ‘Use previous years’ data’ check bo in autolearn on February 28, 2016, so that can still contaminate future analyses. So, here’s the full fix …


correc3.txt (130 KB)

Please let me know if the above issue affected you. I’ll be glad to help clean up your files if you need it! :slight_smile:

Tom

Hi Tom

Thanks for the corrected files. I have been running AutoLearn with both your correc2.txt and correc3.txt and have sent you both results in .zip files for you.

I hope you can see if that has change to the better?

Best regards,
Henrik

I think I spoke too soon! I now believe there’s no bug there (though there could be some other problem with jwwd’s situation … I’m still investigating), because autolearn was looking back no further than two years, and by the time this feature was introduced, the badly formatted data was too old to be looked at anyway.

I’ll post more later as I figure this out. It’s probably OK, but I want to make sure!

Tom