Since yesterday,when I run WXSIMATE, I get an error during the download of the MOS data stating:
“System.Net.WebException: An exception occurred during a WebClient request. System.ArgumentOutOfRangeException: Non-negative number required.” and so on.
What would be causing this error?
I have had the same thing and haven’t figured out where to point the finger. I’m set up to download all 3 MOS types and have experienced the error at different times on each of the three. Often right after a failure I can re-start and get a full download. It would be nice to figure this one out.
I’ve seen this one, too. It’s really strange, because - like Jim - I’ve seen it work perfectly on a second try only moments after a failure, and have seen it affect different ones of the three MOS products at different times. If I can catch it in the act while I’m in the development environment, I might be able to figure it out.
Tom
I’ve found a way to catch the web access errors for MOS in WXSIMATE. It turns out I had such error trapping already in there, but it was basically taking two looks - the first to see how big the file is, and the second to actually download it. The problem is that my trapping was on only the first step. I figured if a file had a size, it must exist!!! I’m not sure why, but it seems like the file was becoming inaccessible in the brief time between the request for file size and the request for the actual file.
I’ve now put the trapping at both stages, but this creates a dilemma: should I suppress the error message boxes in auto mode? On one hand, I should, so that auto runs won’t be interrupted. On the other hand, if too much data is missing, the forecast will be bad (and conceivably there might be some unanticipated crash in WXSIM as a result).
If WXSIMATE can’t finish due to such missing data, and then it’s closed by System Scheduler, a forecast will probably be produced using older data anyway. Maybe I should just create an error log system in WXSIMATE. Any thoughts on this?
Thanks!
Tom
I’ve been fighting this intermittently for at least 3 days now. I think I even sent Tom a screen grab of the error box. I even installed Wxsimate on a second computer to see if the main one was at fault (it wasn’t, obviously).
At least I know now that it’s not just me, and that there is some sort of resolution.
I would think the error message should probably stay if there’s inadequate data for a forecast … most folks are using system scheduler to shut it down when unattended anyway. We don’t want it to continue on and collect current local data since that’s where the time of the forecast comes from, and that’s a big clue that something went awry. A log entry to show what the problem was would be nice too.
Since we’ve seen it be successful on subsequent attempts, would it be possible to add a couple retries before quitting?
Thanks. - Jim
I just had two failures with NAM MOS in rapid succession, then success for all 3 on the third run. I think that just a little more “patience” during the download portion and this will be ok. Tom will know through his testing though whether this is the answer.
This is probably an NOAA server issue since nothing has changed on our ends. Maybe retrying the download a few times before erroring out and then a flag set for incomplete data that WXSIM could read and not run the forecast? I leave WXSIM and WXSIMATE running all the time, so now the error box just sits there until I notice it or notice that the forecast hasn’t updated.
Hi Everybody,
I’m working on this. I’m pretty sure it’s a server issue. I’ve tentatively compiled a version of WXSIMATE which gives a choice of whether or not to display the error messages. With the MOS data, it seems to get two out of three on most tries. I haven’t decided yet whether or not to have it make multiple attempts. I’m hoping the server problem gets resolved some other way. Another source would be nice, but so far I haven’t seen these large MOS files elsewhere.
Suggestions welcome!
Tom
Maybe the ftp links would work better than the http ones?:dontknow: I noticed them (in blue), at the top of the Current MOS Forecast Products page, at: http://www.nws.noaa.gov/mdl/synop/products.shtml
Hi Kevin,
Thanks for that FTP suggestion. I’d seen that, too, but somehow it didn’t cross my mind that the FTP links might work when the http ones didn’t. I’m experimenting with it now. I still don’t understand the hit and miss nature of the http links. It’s also odd that I haven’t had a failure why trying to access that data manually. Perhaps it’s somehting about the way VB.Net handles the data request.
The FTP approach isn’t quite as straightforward, but I’ve tentatively gotten it to work (by creating DOS batch files, of all things!) and it seems to be faster than the other way, so maybe this will turn out to be a good thing! 
Tom
I’ve got a tentative new version of WXSIMATE you can try. It addresses the web client problem with MOS data in two ways.
First, if you do access that site in auto mode, WXSIMATE should now skip the error messages so that the run can complete. On average it seems it should get about two out of the three MOS products. I still have no idea what caused this problem to appear, other than that it must be something on the server end.
Second, there’s a new option: FTP (thanks for the suggestion, Kevin!). It turned out I had to do this in a rather roundabout way, as VB.Net does not make it easy to use FTP prorocol (it seems I would have to download some kind of plug-in, that would probably only work in Net Framework 2.0, and then learn how to use it). What I did was to have WXSIMATE create DOS batch and script files and run them to get the MOS files via FTP, through Internet Explorer (I think). This works quite well on my computer, and may actually be faster than the first option. A slightly odd aspect is that you see the command window’s icon appear up to three times for a couple of seconds each down on the bottom task bar. Also, an hourglass may appear for a few seconds, preventing updates of WXSIMATE’s status bar, so that it skips right from “Searching for GFS MOS…” to “Files appended”, without telling you about the culling process or advancing the blue progress bar. That’s OK, though, because it does do everything, and pretty quickly!
I’ve posted the program on my web space, but not accessible from the web page itself. Instead, just go to www.wxsim.com/wxsimate.exe and choose “Save”, and save it to c:\wxsim (or whatever your WXSIM folder is called).
You should notice two things different when you run WXSIMATE: there’s a new box for the UTC minute value (to the right of month, day, hour). This helps WXSIMATE know what MOS files should be available for download. Also, on the MOS list box, there should now appear arrows for drop-down on the right. The http source will still be in the box (at first), but you can now scroll down to the ftp choice right below it, and click on that to select this new option.
You might want to try the old option first, in auto mode, just to make sure it really doesn’t stall the rest of the process. Again, this should largely work, omitting an average of one of the three sources. I still want it as an option.
I’m interested in the behavior of the new option, of course. First, confirm that it works. Second, take note of whether the hourglass appears, and whether the progress bar and status bar update (no big deal if they don’t, but I’d like to know).
By the way, I believe that, at least with the new option (and possibly with the older one as well), all checked MOS products will be included in the final data file, even if they weren’t downloaded on this try (it will patch in old data from the latest successful run). I think this is a good thing (better old data than none - up to a point), but will think about it some more.
I also am updating WXSIM itself. The main item on the agenda this time is the Auto Haze option. I’ve rarely used it, but have refined it considerably and will be recommending its use in general in the future.
Let me know how it goes!
Tom
Tom,
Downloaded the updated WXSIMATE, did a forced ‘auto’ run with the http and all went well. Changed to the ftp for MOS and did another forced ‘auto’ run. I did forecasts with both downloads.
Everything worked as it should. I did see the hourglass appear but didn’t see the command window but did notice that the WXSIMATE window did ‘blink’ several times. The progress and status bar did not update during the download or cull/append.
The forecasts were slightly different. The high temps for Sunday and Monday were 1 degree higher all else was the same. The download was definitely faster. Works for me!!!
–Dave
Hi Dave,
Glad it worked!
The difference in the forecast could be from one or more MOS files missing in the http download, or conceivably from the ftp version failing to get the very latest files. I’ve been monitoring the release time of these MOS files and am allowing a margin of 5 or 10 minutes before WXSIMATE goes looking for a supposedly just-released file.
I’ve tried all kinds of things to get the status and progress bars to continue, but it seems with the batch file running and the file copying that must be done, the computer just gets too busy for a few seconds to “bother” with updating them. At least the job gets done, and quickly! (I didn’t actually time it, but I’ve occasionally had delays using the http route).
If this stands up to a bit more scrutiny, I’ll release it in the next couple days or so, along with a new version of WXSIM.
Thanks!
Tom
Everything works on mine as well. One thing for others to note: After you select the MOS ftp site from the dropdown box, be sure to actually click on that URL in the box, until you see it appear in WXSIMATE’s “Target URL” box at the top of the downloads section. I forgot to do that the first time, and WXSIMATE did the http MOS download instead. #-o
I did the same but wasn’t going to tell. :oops:
And I am out of town while you guys are having all the fun.
I’ve (well WXSIM and WXSIMATE) made 8 different runs since yesterday. All is working well and the ftp download is much faster that the http. This is good for me as I have a satellite internet connection which is slow compared to other broadband. Thanks Tom.
–Dave
My setup ran straight through yesterday evening and last night without stopping or erroring. I am using the FTP method for MOS. The only improvement I could suggest is a way to tell if all the files downloaded correctly. That way if I noticed the failure, I could rerun manually but if I wasn’t around it would keep on going by itself.
Thanks Tom for all the hard work and time you’re putting into this.
Hi Dan,
There is actually a way to tell if all the files downloaded correctly, though it’s not very obvious. This new version creates a log file called wmatelog.txt, which should list any download failures. You can find it in the folder with all the other WXSIM files (usually c:\wxsim).
Hope that helps!
Tom