2016 Beta versions ready to test!

I’m late on parade as usual :oops: and have only upgraded today but everything here is working normally with beta4. Thanks Tom!

Hi Tom,

As per earlier post all was running fine until overnight last night.

Background: installed 2016beat4 prior to my 2016022706z run, that ran without error.
Overnight that night wxsimlite did analysis at 01:20 (no error) Autolearn ran at 02:40 (no error) all following forecasts (wxsim/wxsimate/wxsim-lite) all ran without error (2016022712z/18z/2016022800z/06z)

Then last night wxsim-lite failed while doing the overnight analysis at 01:20 this morning (with attached error): Though seems as though Autolearn ran at 02:40 this morning without error ?
This morning wxsim wxsim-lite auto forecast 2016022812z failed due to above and gfs data 2016022812z had not downloaded prior to 12z wxsim forecast as wxsin-lite was still in error during that time (prior to me waking up).

Though the wxsim forecast did look ok at 2016022812z (which ran at 05:00 while wxsim-lite was still in error) ?

just an observation or rather question, in my earlier post I mentioned as per documentation wxsim-lite states that DST(Start/End)(Day/Month) entries to be adjusted yearly (so I did that the other day. Just curious how wxsim-lite handles the previous years DST situation ? In my case this year both start and end days changed by 2 days versus 2015.


cortest.txt (13.8 KB)

correc.txt (131 KB)

I had an Error 62 at my 2200 auto run in WXSIM-Lite - no idea why as all data looks good. :-k I then scheduled an auto run at 2300 and everything was fine.

Thanks for all this (mostly!) good news!

To answer some specific questions:

Christian and Henrik: That Langesoe data suggests there was a max temperature of 44.2 C one day!!! It appears this was on February 1. Of course, that’s not real, but wret.exe did read it out of the log file, so you might want to go back and look. It might be a single “typo” that you could manually fix, and then wret could learn from that data after all. To change to weighting recent data more, check the box in wret.exe about that.

Tony, it looks like there was a crash on some old data, from April, 2015. Maybe it’s a bad GFS file (???). You could remove it. One change I may need to make is that it really shouldn’t have made a fdata.txt file from that. I’ll look into that.

Thanks again! :slight_smile:

Tom

Just a follow-up: I’ve confirmed that WXSIM-Lite is making and exporting fdata.txt files all during its analysis run. That’s not a problem, until it either gets hung up on bad old data or the current forecast fails to run. I can change that pretty easily (haven’t yet - want to be careful!). I think that will pretty much cure the “old” fdata problem. The only drawback is that it could mask symptoms when there are problems like bad downloads. Almost all those are brief, though, and fresh forecasts would be restored well within the 24 hour window. I do think I will make the change and post a new version in the next couple of days. However, if you are seeing this message (about old fdata.txt) I would be interested in posted screen shots. I think most of them are 4-day old data, from some failure to run a fresh forecast after the analysis (could be from bad scheduling or an unexpectedly long analysis run). If it’s a lot older than 4 days, it’s probably old bad GFS data, which could happen now that it can look back a year. There could be old bad GFS files which haven’t been analyzed in many months and which could now show up like that. That also means they can be weeded out.

Thanks for all your patience and help with this. I think we are on the verge of a real improvement across the board! :slight_smile:

Tom

Tom, thanks for the explanation concerning the data included in “cortest.txt”. I must have started autolearn later or deleted 2014 autolearn data as I only have 2015 history #-o. It will catch up in 2017. I have another question. Looking at the very bottom of my “cortest.txt” file I see these statements.

Current date and midpoint, and average values to be sent to wret.exe for inclusion
2/27/2016 5861 -0.01679 0.9627 0.0028 0.02429 67.22088 n = 91

Average corrected values for analyses done with WXSIM-Lite before February 20, 2016
-0.01679 0.9627 0.0028 0.02429 67.22088 n = 91

The data is the same. Is this correct? :?

Cheers,

Jeff

Hi Tom,

I looked at all the gfs files around that time 22/04/2015 and all seem perfect, not only that but wxsim all previous forecasts had been error free.

When I look at the detail of that message I just thought it must have been due to the changes I made to the DST Days/Month for 2016 as it’s saying (rounding up hour) = 2016-02-28-19z (so the 19z throws me) as there would only be 00/06/12/18z files (the 19z I thought was being derived from my local time of forecast being 2016-02-29-05:00 back the current +11hrs for DST would be 2016-02-28-18z (which would be correct if that’s what it was doing) though 2016-02-28-19z if we had been in ST being +10hrs. Don’t mind me just a play on numbers I suppose. I did check the fdata.file after wxsim had run the forecast with wxsim-lite and wxsim error messages still up and all it contained was 0

Once I accepted both error dialogs wxsim-lite automatically went into a complete analysis run and completed successfully. Then I checked fdata again and it was as normal full of what seemed correct data.

No no change to GFS Files (ie. I have deleted none since your suggestion).

On 2016-03-02 04:00 (AEDT) I awoke before the wxsim forecast (for at 2016-02-29-12z) and found that Autolearn had run without error, wxsim-lite had run an analysis without error but when I checked fdata.txt it just had:
0
0
in it. so I just let wxsimate and the forecast run and it all went without error.

Same again this morning 2016-03-02 04:00 (AEDT) Autolearn had complated no errors, but wxsim-lite had been in error as it run prior to Autolearn.
wxsim-lite had error52 (sorry did not have time to screen dump it), accepted that error and lite continued with it’s analysis, then had 2 more error52’s, then once it was completed, checked fdata.txt and it contained only:
0
0
again, did a Run Immediately on wxsim-lite (Analysis) this time it ran start to finish no errors, once complete I checked and found fdata.txt still only contained:
0
0
again, did a Run Immediately on wxsim-lite (forecast) which downloaded the latest gfs data and completed correctly, then checked fdata.txt and found it contained the correct data.

Subsequent wxsim forecast ran in auto an hour later without error.

Tony


Hi Tom,
this night Wxsimlite had run with fdata error:

…with GFS old data window:

Regards,

Alessandro

Hi Tom

At Christian from Langesoe beta4 is now running without any errors…BUT…in his detailes there are 490 forecasts (yesterday it was 1301 and after that we corrected a bad logfiles from WD) that were skipped due to bad correction factors - can we in any way correct that so his forecast can be used?

Note: Values below are weighted towards more recent forecasts, unless otherwise indicated.
Forecasts for Langesoe. Using Celsius.
Time to complete run = 59 min, 2 sec
Number of forecasts used = 1285 (72,4% of total checked)
Number of forecasts rejected (C) due to bad bias correction factors = 490 (27,6% of remaining candidates)

See attached details.txt

Best regards,

Christian and Henrik


details.txt (209 KB)

Tom - had the attached message this morning. The 0700 forecast completed okay and I can’t find anything wrong. The gfsdat0dt=2016-03-01-00 downloaded fine with no errors so I don’t know what the error message refers to…

Edit - same thing happened at the 1200 forecast but it completed okay.

Edit…and again at the 1800 and 2300 forecasts…


wxsim13.JPG

Same issues today at 0700 as for yesterday above plus error 62 and the attached message…

Edit…no 2016-03-02-00 gfs file downloaded (McMahon) but just had a good 1200 forecast with no errors at all. :-k


wxsimlite3.JPG

(1) wret.exe has additional information, about the last file opened, in that line 1939 message. Also, I added information to wretlog.txt, an error-diagnosing log file created with each run, and which I almost forgot I had in there! Between that error message and the log file, I think I can quickly identify where that line 1939 error is coming from (the 1939 doesn't tell me as much as it would seem, as it's only the location of that error handling section, and the error generally originated elsewhere in the program).

I did a try with a clean install with only 30 days of data with wxsimlite and had this error about wret line 1939. So this is basic.
Back to previous version and all went well.
Maybe this will help for investigation,Tom.

OK, first, a new beta to test! :slight_smile:

www.wxsim.com/2016beta5.exe

The changes here are:

(1) In WXSIM, it should no longer have dates like February 30 listed as the fdata.txt file time in that message about fdata.txt being old.

(2) In WXSIM-Lite, the message about max and min residuals being out of range (this is rarely seen anyway) will now NOT stop the program. It just shows as a clearable “non-critical message” at the bottom of the form.

(3) The big one (I think!): I believe I now have sufficient protection against old (for example four day old) fdata.txt files being “accidentally” generated during the analysis run. I’ve had this reported quite a bit, and have seen it myself at least a few times, too. I haven’t quite determined all the ways it can happen. I believe it can result from some kind of problem during the analysis (maybe related to the error 62 reported sometimes) or to issues with scheduling or the time taken to complete the analysis run. In any of these cases, though, at least a “bogus” fdata.txt will not be created, and the last successful one (probably much less than 24 hours old) will still be waiting for you. While this could “mask” some sorts of problems, those problems are likely temporary and usually result only in using a 6-hour-too-old fdata.txt for a few isolated runs.

Please let me know how this goes!

Regarding errors at line 1939 in wret.exe, I’d like to see the contents of the message box and the wretlog.txt file.

Thanks so much for your patience and support!

Tom

No (line) errors here !

:smiley:

Hi Tom,

Thanks again for trudging your way through these issues, greatly appreciated.

Installed latest 2016Beta5 and first run (2016-03-03-12z) this morning wxsim-lite downloaded 2016-03-03-12z (though said it was bad data.

wxsim forecast run resulted in : msglog.txt below:

Franklin
04/03/16
5:10:00 AM
No READY or GFS model data found
*
04/03/16
5:10:34 AM

Also attached :

wretlog.txt
wdata.txt
fdata.txt
gfsdat0dt=2016-03-03-12.txt (downloaded but wxsim-lite reported bad) though can’t see why.

Fcast_hr, tmp2m, dp2m, wspd, wdir, tcdc, prmsl, hgtsfc, tmp850, rh700, hgt1000, hgt500, apcp 2016-03-03-12 +0,12.3,11.1,2.5,9.5,50.0,1021.7,276.6,10.1,22.8,180.1,5781.7,0.00, +3,12.3,10.9,1.9,300.2,89.6,1020.5,276.6,10.0,19.8,169.3,5773.8,0.00, +6,11.7,10.5,3.4,272.9,82.0,1019.9,276.6,10.0,17.8,165.2,5772.8,0.00, +9,13.3,11.2,2.9,272.7,56.5,1019.7,276.6,9.8,15.8,163.4,5775.2,0.00, +12,20.2,12.1,0.9,289.9,63.2,1018.8,276.6,8.8,11.2,155.2,5776.1,0.00, +15,22.5,11.9,2.1,186.7,57.6,1017.4,276.6,8.9,11.7,141.6,5772.1,0.00, +18,21.0,12.1,3.4,144.0,63.3,1015.7,276.6,9.6,14.5,129.1,5761.6,0.00, +21,14.1,12.8,3.1,98.6,59.1,1016.0,276.6,9.6,23.0,133.7,5762.5,0.00, +24,12.4,11.4,1.4,293.8,64.6,1016.8,276.6,9.9,24.9,140.5,5765.2,0.00, +27,12.4,11.2,4.6,278.2,78.9,1016.5,276.6,10.1,18.6,138.0,5758.1,0.00, +30,10.7,9.5,5.5,290.7,31.2,1016.7,276.6,9.4,13.5,142.6,5754.9,0.00, +33,12.2,10.3,5.7,300.0,7.2,1018.0,276.6,9.4,18.2,152.8,5764.1,0.00, +36,17.9,11.0,7.3,275.6,16.7,1019.0,276.6,8.5,24.3,159.7,5778.3,0.00, +39,20.3,11.3,7.7,263.6,24.3,1018.8,276.6,8.3,12.4,156.9,5782.6,0.03, +42,20.4,11.3,6.5,260.0,9.6,1018.7,276.6,9.6,15.5,153.7,5787.1,0.00, +45,12.4,10.8,1.4,233.8,2.4,1019.7,276.6,10.1,21.4,163.2,5797.7,0.00, +48,11.8,10.7,2.4,344.3,7.6,1021.1,276.6,8.8,18.5,175.5,5812.5,-0.00, +51,11.9,11.0,3.6,287.4,3.2,1020.2,276.6,9.8,14.9,168.2,5804.6,0.00, +54,11.2,10.4,4.3,271.9,2.4,1019.2,276.6,10.4,19.5,160.9,5798.0,0.00, +57,12.2,10.9,4.8,269.4,13.5,1021.1,276.6,10.4,28.4,178.6,5805.5,0.00, +60,17.4,9.9,4.1,271.8,29.7,1021.7,276.6,10.7,31.1,181.5,5806.6,0.00, +63,21.9,9.5,3.4,214.1,23.6,1021.6,276.6,8.1,25.2,175.8,5798.9,0.00, +66,17.9,10.0,3.9,175.5,99.5,1020.9,276.6,8.4,17.5,171.6,5785.0,0.00, +69,13.6,10.7,2.3,115.2,48.8,1022.4,276.6,7.6,18.5,184.7,5798.7,0.01, +72,12.6,10.8,0.9,190.0,69.5,1024.0,276.6,7.1,26.1,200.0,5806.2,0.07, +75,11.2,10.1,1.9,252.5,94.9,1022.8,276.6,7.6,21.8,191.5,5801.0,0.02, +78,10.1,9.1,3.3,262.3,82.9,1022.6,276.6,8.0,17.1,190.1,5800.3,0.01, +81,11.5,9.7,4.0,250.0,82.7,1024.7,276.6,7.0,14.2,207.9,5812.5,0.00, +84,18.1,9.8,3.2,231.2,76.2,1025.7,276.6,6.9,16.3,214.0,5822.4,0.00, +87,19.9,9.6,4.5,182.1,64.7,1025.1,276.6,6.5,13.3,207.3,5822.0,0.00, +90,17.3,10.4,6.9,148.0,77.8,1024.5,276.6,7.3,11.2,203.8,5817.4,0.02, +93,12.8,10.7,4.6,127.5,85.2,1025.9,276.6,6.6,11.0,218.1,5822.1,0.00, +96,12.2,10.4,3.1,88.3,99.6,1026.4,276.6,6.9,10.8,221.3,5824.5,0.07, +99,11.5,10.4,3.3,54.6,80.0,1024.8,276.6,8.0,12.3,207.7,5814.5,0.00, +102,10.8,10.0,2.9,2.5,71.8,1023.5,276.6,8.5,17.0,195.8,5807.9,0.07, +105,12.7,11.4,2.7,324.3,56.1,1023.7,276.6,9.0,32.5,197.4,5815.1,0.00, +108,19.7,13.0,1.5,322.9,43.3,1022.8,276.6,8.8,73.3,190.1,5814.0,0.00, +111,17.9,13.3,1.9,93.4,92.0,1019.8,276.6,10.3,98.2,168.6,5802.8,0.15, +114,17.0,14.6,3.1,149.0,91.9,1017.9,276.6,10.2,99.2,153.2,5793.7,1.06, +117,14.7,14.1,3.5,186.7,97.6,1018.1,276.6,11.4,97.2,153.7,5794.2,1.21, +120,14.4,13.8,2.6,217.9,96.4,1017.3,276.6,11.2,95.6,144.3,5786.8,0.44, +123,14.5,13.9,3.4,228.1,98.5,1016.0,276.6,11.1,91.0,132.2,5765.9,1.97, +126,14.0,13.3,4.9,223.3,99.5,1016.5,276.6,9.9,98.6,134.9,5746.0,0.89, +129,13.3,12.4,5.9,225.9,96.5,1018.3,276.6,8.3,78.2,154.2,5754.3,0.40, +132,13.2,10.6,7.8,214.9,90.3,1020.4,276.6,4.2,23.7,176.5,5765.0,0.03, +135,15.0,8.6,9.5,183.9,92.6,1021.9,276.6,3.5,10.9,188.4,5775.3,0.11, +138,13.8,6.6,8.4,147.8,63.8,1023.5,276.6,3.5,0.9,204.8,5786.4,0.09, +141,9.2,5.8,5.3,121.7,46.8,1026.2,276.6,5.4,0.6,224.7,5800.6,0.00, +144,7.4,5.5,4.6,86.1,2.2,1027.8,276.6,6.1,1.0,237.4,5814.2,0.00, +147,7.3,5.3,5.1,50.5,16.2,1027.6,276.6,6.5,4.1,234.1,5815.4,0.00, +150,8.1,5.7,6.1,21.9,21.6,1027.5,276.6,7.0,11.1,231.3,5814.4,0.00, +153,9.3,6.3,6.2,11.3,35.8,1028.0,276.6,7.1,18.4,236.0,5822.5,0.00, +156,15.3,8.0,8.1,31.9,22.6,1026.9,276.6,7.8,19.8,226.2,5824.5,0.00, +159,19.2,9.5,8.9,51.8,29.5,1024.8,276.6,7.8,25.0,206.2,5823.3,0.00, +162,18.9,10.9,9.8,61.7,29.6,1022.8,276.6,9.6,47.0,190.1,5825.4,0.00, +165,13.1,11.2,7.0,53.5,41.7,1022.6,276.6,12.2,48.0,191.9,5835.3,0.00, +168,12.0,10.7,7.1,27.3,27.3,1022.2,276.6,14.5,67.0,185.2,5842.8,0.00, +171,12.0,10.4,7.1,15.8,0.3,1020.2,276.6,16.0,67.2,169.6,5837.4,0.00, +174,12.7,10.4,6.9,3.4,56.6,1018.9,276.6,17.7,63.2,153.5,5833.1,0.00, +177,14.3,11.3,6.3,9.8,93.9,1018.5,276.6,17.5,68.4,152.1,5835.1,0.15, +180,19.3,14.8,6.6,9.2,98.3,1016.8,276.6,18.1,78.1,137.0,5829.7,0.32,
regards,

wxsim.zip (338 KB)

No problems so far with 2016beta5 - both my 1200 and 1800 forecasts ran through okay.

So far so good with 2016Beta5. My runs with GFS 2016-03-03-06 and 2016-03-03-12 both ran cleanly.

Steve

My 2016-03-03-18Z just ran without error.

regards.

Hi Tom,
beta5 worked fine :smiley: :smiley:

Hi

I am getting error 9 message in wxsimlite - could anybody help please

regards

Dave


wxsilite error subscript.jpg

32016lg.txt (372 KB)