Forcast

Hi Everybody,

I’m excited to see all this discussion of WXSIM! I’ll try to briefly answer a few questions that have been raised, and offer a few comments.

First, WXSIMATE already does much of what Robert was suggesting. With a single click it will go and download all the internet data WXSIM needs (EXCEPT the READY data - more on that in a minute), cull through it to pick out the parts that are relevant to your station, and then append (previously saved) READY model data to it. It saves this data under a file name you can define in advance - for later use in WXSIM.

Regarding the READY data, it would be nice if it could be downloaded automatically, but it’s informative to read the note on the site. They specifically want data to be accessed manually (using the password, for a couple of reasons. First, too many automated programs grabbing data from it would crash their site. Second, they’ve got a great site and they want you to actually look at it. I must add that they are performing a wonderful service, using a few US taxpayer dollars, to the whole world. In fact, for most of the world, this GFS model data is the only free data of its type. In contrast, output from the EMCWF (the big European model) is highly restricted and you have to pay to get it. I’m really grateful to the NOAA ARL that such data is available at all!

As for WD data imported by WXSIMATE - it gets various types of data, going back as far as 30 days, and does a variety of averging processes to “feed” WXSIM useful data. I actually think that the internet data retrieval is more vital to running WXSIM than is the WD data - though both help.

Responding to Chris’s desire for automatically exported files - they already exist, including the .csv file (which I implemented a few versions ago). Just be sure to get the latest version of WXSIM (it’s 10.6 right now). Every forecast is saved, though by default with the name 'latest (.txt, .csv, and .wxf for use in the Retrieval module).

Thoughts on automation…

I must say, that would be fun! I’d like to walk up to the computer and see a forecast already made, and then see current data superimposed on it to see how it’s doing so far! Three issues are relevant here:

One is technical - that is how I would actually implement that. A brief effort at that several months ago didn’t get very far before I ran into difficulties - though I don’t think they’re insurmountable.

Another is that the forecasts would simply not be as good. Most feedback I get on forecast accuracy is very positive, but there are exceptions. Usually these result from the user not understanding (which is understandable!) what all to do, though occasionally it’s just a tough place to forecast for, or I had insufficient data to do a good customization.

The third item is that, actually, there’s quite a bit of value in being “forced” to work through the process of running WXSIM. I’ve had comments from users to the effect that they had learned more in a few weeks of using WXSIM than in all their years of their weather hobby. I think carefully running it (and just playing around with it, making fake forecasts under various ‘pretend’ conditions) helps develop forecasting intuition very rapidly. Also, it’s good to learn about all those data types, though they can be confusing at first.

That said, it would still be great to have at least the option of a quick, automated run! I see the interest is there, and I will definitely revisit the possibility!

Thanks!

Tom

I’ve just had another look and upgraded to 10.6, but I can’t seen an option to automatically export a file at the end of a forecast run. I’m probably missing something obvious! I know about 'latest, but because I’m still trying to learn WxSim, I try running the same data with different options to see what happens to the output data. To allow comparison between two runs, I really want to make sure I capture the output from each run. I quite often forget to copy 'latest to a stored run file and by the time I realise I’ve re-run the forecast and wiped out the previous 'latest file.

I must say, that would be fun! I'd like to walk up to the computer and see a forecast already made, and then see current data superimposed on it to see how it's doing so far!

I agree. That would be excellent :smiley:

Another is that the forecasts would simply not be as good. Most feedback I get on forecast accuracy is very positive, but there are exceptions. Usually these result from the user not understanding (which is understandable!) what all to do, though occasionally it's just a tough place to forecast for, or I had insufficient data to do a good customization.

I think this is the most difficult issue. I’m not experienced enough in running WxSim, but your experience of years of running it would be interesting. How many runs do you do where you effectively choose a set of default button clicks to produce a forecast. For example, I always click the ‘Use READY data’ button. Maybe 9 out of 10 runs you never adjust any parameters during a run because they don’t need adjusting for the current conditions. If so, it’s possible that 9 out 10 (the number may be much lower!) runs might produce reasonable forecasts using default answers/settings, and you only need extra input under difficult/unusual conditions? If you can get reasonable forecasts more often than not using the defaults, it would be good to be able to do that.

The third item is that, actually, there's quite a bit of value in being "forced" to work through the process of running WXSIM.

I totally agree. If there was a way of running WxSim automatically using the default values for days when I’m busy, I would still want to drive it manually to learn more about how this thing we call ‘the weather’ works! I would never suggest that there was only an automated version…the manual version would still need to be available.

Finally, just another passing thought. There are three ‘periods’ of GFS READY data. I don’t know if this is sensible, but if we could download three files, containing the 0-84, 0-180 and 192-384hr data, would it be possible for WxSim to combine the three files to get data for 0-384 hours? I know that 384 hours is pushing the WxSim methods way too far, but grabbing a set of READY file on, say, a Monday that still had model data on a Tuesday or Wednesday might allow more runs to be automated. I know the data gets old, but old model data could (!) be better than no model data.

Hi Tom,

This is great news. Please keep us updated on your progress as I am very interested. Thanks.

Thanks again for all the discussion!

A couple of quick notes:

Chris, the .csv file is being created automatically, whether you tell it to or not. What you can do, though, is click ‘Save’ when you’re done (or also under ‘File/Save’ on the Entry form if you have already returned to it) and give that ‘latest’ file (all three of them, actually) whatever name you want. The File/Retrieve option pulls up the .wxf version, with tons of display options.

You can use old READY data, understanding it gets less accurate with the passage of time. The 192-384 hour data is only a little better than chance, but the 0-180 isn’t bad, for a rough 1 week outlook. I almost always keep it to just the 0-84 hour range myself. If there ever is an automated option (which I will look into), you could probably get the READY data once every day (or even two, perhaps) and the program could still use it.

As for differences in how I run it from time to time, by far the biggest issue involves use of advection data. In the US, you can import MOS data after wind shifts, for advection, and there are choices to make about the flow curvature, the shape of smoothing fit to be applied, etc. In England (or anywhere outside the US actually), the MOS data isn’t available and you have to resort to the somewhat less reliable ‘Neutral’ or ‘Default (frontal codes)’ advection options. I suppose a standard choice could be defined, though, and the output would at least be reasonable.

As long as users clearly understand the trade-offs involved, i agree that the automated runs would be really nice. I’d get a kick out of it myself! (I would just take such forecasts with ‘a grain of salt’).

Well, thanks. I’ve got to go do a customization for Iowa!

Tom

Sounds like what I do with the local TV Weatherman and his forecasts #-o

Hello Everyone,

I haven’t bragged about WXSim’s software for a while now…so its time for me to start bragging again…:slight_smile:

Before I do though, my ONLY suggest that I want to throw at Tom is to WORK WITH BRIAN about adding custom tags to WD with your forecast data. Thats the ONLY thing I would really love to see automated…If we get that then it would save me a lot of copy and pasting and all Id have to do is take the 5-10 minutes to actually run the forecast on a given day and WD could do the rest for me. Maybe make seperate files for each day (day1.txt, day2.txt,etc) and have the forecast post in there and then have Brian add some kind of custom code that WD would read whats in that file and use that text or whatnot. I’m not sure how you both program but I’m just throwing my only idea to you guys right now :slight_smile:

Anyways now to brag about the software…LOL I ran a forcecast last night with WXSim and check out the predictions (The full forecast is at http://www.hobartweather.net/forecast.html) Forecast Feb 15-17.

Feb 16 ---------------------------------------------------------------------------------------
Thursday
TIME WEATHER AIR DEW WET WCF HT.I W.DIR W.SP %RH INT TOT SN.C THK
12:00a DNS.OVCS 32.6 30.7 31.8 25.4 32.6 NE 8 93 0.00 0.06 0.0 5423
12:30a DNS.OVCS 32.2 30.4 31.5 25.0 32.2 NE 8 93 0.00 0.06 0.0 5426
1:00a DNS.OVCS 31.9 30.1 31.2 24.6 31.9 NE 8 93 0.00 0.06 0.0 5430
1:30a CLOUDY SLT/FRZ.R 31.6 29.9 30.9 24.2 31.6 NE 8 93 0.03 0.07 0.0 5433
2:00a DNS.OVCS 31.2 29.7 30.6 23.7 31.2 NE 8 94 0.00 0.08 0.0 5436
2:30a DNS.OVCS 30.9 29.4 30.3 23.4 30.9 NE 8 94 0.00 0.08 0.0 5439
3:00a DNS.OVCS 30.6 29.1 30.0 23.0 30.6 NE 8 94 0.00 0.08 0.0 5442
3:30a DNS.OVCS 30.3 28.7 29.7 22.7 30.3 NE 8 94 0.00 0.08 0.0 5445
4:00a DNS.OVCS 30.0 28.4 29.4 22.3 30.0 NE 8 94 0.00 0.08 0.0 5447

Below is parts of the Daily login times from WD.

Time Temperature Humidity Dewpoint
00:20 34.2 83 29.5
00:30 35.1 94 33.5
00:40 34.9 94 33.3
00:50 34.9 94 33.3
01:00 34.7 94 33.2
01:10 34.7 94 33.2
01:19 34.5 95 33.2
01:29 34.5 95 33.2
01:38 34.5 95 33.2
01:48 34.5 95 33.2
01:58 34.2 95 32.9
02:08 34.2 95 32.9
02:17 34.2 95 32.9
02:25 34.2 95 32.9
02:34 34.2 95 32.9
02:43 34.2 95 32.9
02:52 34.2 95 32.9
03:03 34.2 95 32.9
03:10 34.2 95 32.9
03:19 34.2 95 32.9
03:28 34.2 95 32.9
03:38 34.2 95 32.9
03:48 34.2 95 32.9
03:57 34.2 95 32.9
04:07 34.2 95 32.9

WXSim was able to predict the temp, humidity, and dew point within -/+ 5 degrees or percentage for this morning (on the 16). In fact I was at work yesterday (on the 15th) and my coworker said “Its Raining outside” , i said “I know, the forecast predicted it…” so I checked the actual temp from my website and the actual “Condition” said “7:30p DNS.OVCS SNOW/RAIN 37.9” The actual temp was like 35.6 or so (estimate from my log file). He hit me and said “See your in the wrong field, go be a weather man.” LOL!

Anyways…just giving my props to Tom and to throw the suggestion at him (and Brian). I’m sure Tom loves hearing when forecasts come out good! heh heh

-Jason

If the forecast is in a text file or files you could use the “include a file” custom tags to get it on a webpage. See http://discourse.weather-watch.com/t/14143

Having looked at the WXSim site and a few of the messages posted here, it is obvious that what I call a forecast and what WXSim produces are two different things. All those codes and numbers…yuck.

I would prefer a simple icon/text forecast similar to what I get now from the NWS as seen here

http://www.wrh.noaa.gov/total_forecast/printable_forecast.php?wfo=sto&zone=caz016&fire=caz216&county=cac115&dgtl=&lat=39.49000&lon=-121.5500

I just wanted to post a message that a couple months ago I suggested to Tom that he adds a feature to where it will produce a sentence forecast version such as what NWS does. (ie: Monday: Rain with the high in the upper 60. Monday Night: Lows in the lower 30’s with west winds, etc)

He said he had others request this and he will consider implimenting it, just no eta at this time. I don’t want to speak for Tom since I don’t know if he’s really doing this or not but thats what he told me in his reply to my suggestion.

I personally would love to use a sentence base forecast then the hourly one. The hourly one would be perfect on a seperate website (ie: file by itself). As far as the sentence base i’d love to use this on my flyer and get rid of the NWS forecast and put WXSim’s forecast up vs the NWS. I’d make WXSim’s forecast the main forecast that users see when they click forecast but I’d add a “Click here for NWS Forecast” in case users want to view that.

Sorry if im not making sense, Im just waking up and about to leave for work. yay…So im a litte out of out it and running low on surgar :slight_smile:

Wouldn’t it be rather trivial for a code junkie to parse the data to produce such a thing? Find the high value and time, find the low value and time… and look for any precipitation and amounts… then dump that into a tag or some other file format for displaying… Could WD perhaps do this with the .csv forecast file WXSIM produces? Brian is amazing at parsing, perhaps he could create a custom page that looks for a current WXSIM forecast file and converts it into html…

Not being able to do this is about the only thing preventing me from buying WxSim. It would be fun to play with but all of the cutting and pasting would take a lot of the “fun” out of it.

I may be confused at what you are saying. If you do get wxsimm then also get wxmate. It downloads every thing you need except,ready. IT does get the info from WD and uses in in your forcast.
Chuck

Agreed. The best solution I have read so far involves having WD read an file output from WxSim and creating a simple icon based forecast from that for the next 3, 5, or 7 days. If it could be done and made totally automated I would be on board as well. Hopefully the guys are working something out…

I think we have convinced Tom to work on an automated version. He is such a fanatic about getting it right I understand his reluctance to automate. I have wxsim, and with the companion wxsimate it really does only take about 10 minutes max to get a good forecast. Due to the availability and reliability of data I’ve not seen very precise forecasts past 3 days, but I can tell you he is VERY GOOD at 3 days and under… down to the minute good in some cases! WXSIM put rainfall yesterday at about 1:30pm… rain began at 1:22pm… how is that for close?

But to be honest I do almost the exact same thing each time I run WXSIM with WXSIMATE, so automating it shouldn’t be a problem for short range forecasts. Getting READY and MAPS/RUC-2 data has to be done by hand, but we can add that in optionally once or twice a day to tweak the forecasts. I’d like to see it so that WD can call wxsim to update the forecast every hour or two automatically… it can look for the existance of any READY/MAPS/RUC-2 data and use it if it is there, and skip it if not…

It may happen… it may happen faster if we can convince Tom that he’d sell more copies of WXSIM if it did get done! It’s a great product and deserves to be used!

Hopefully we have convinced him already :wink:

I’d be very interested in a more automated version.

–Dave

Anyone using this in the southern hemisphere, in particular Australia successfully ?

It sure sounds interesting. :slight_smile:

Chuck,

What I am looking for is a way to have WxSim generate a simple 3 day forecast that is automated and hands free. Some simple text, ie. saturday partly cloudy in the morning then becoming mostly cloudy. a chance of light rain. highs around 53. southeast winds up to 10 mph increasing to 10 to 15 mph in the afternoon and then, have WD include the icon with the forecast so I can post it to my website. From looking at the WxSim site examples it appears to be very complicated. I have some free time today and I am going to download WxSim and WxMate and have another go at it and see what I come up with for data.

I would even settle for something like:

Today:
7 am - partly sunny 60 deg. wind SW 7mph
noon - sunny 70 deg. wind S 15mph
6 pm - sunny 75 deg wind S 10mph
midnight - clear 55 deg. wind calm
low - 55 deg at midnight
high - 77 deg at 5pm

(or whatever times we choose to display)

I know that is not as wordy as Dans suggestion but I could live with the above for now if that would be easier to do while you work on getting a discriptive forcast working properly. Heck, if these items were broken down and WD tags created for each item within a time then we could construct the discriptive part of the text.

Hi All,

I’m glad to see all this discussion of WXSIM! :slight_smile:

To back up what a couple of others have reported (or at least expressed a wish for), I am indeed trying to create a much more automated option. Making it an option (with some kind of disclaimer attached) would satisfy my concerns about reduced accuracy when run in that mode. I think on most days, it makes little difference, but there are other situations that call for some good human decision making with regard to what options to use.

Another thing I’d like to point out is the learning value of actually taking the time to go through the whole (few minutes’ worth) process of running WXSIM ‘manually’. I think carefully looking at the data, and how WXSIM uses it (and by running different scenarios with slight variations - or just playing around with ‘pretend’ weather conditions in other seasons, even some of the other sample locations), can make the user a better forecaster.

Also, I’ve given considerable thought to the ‘plain English’ forecast output. I think I can do it. The challenges are mainly related to wording of changing conditions, like, under what circustances to say “increasing cloudiness tonight” instead of just “partly cloudy tonight”.

I’ve done actual work on the automating part, and have it partially working. I need to make improvements (like maybe even a timer to have the program start a forecast at prescribed times). I’m very interested in doing this, but quickly get swamped. In between teaching high school full time, doing WXSIM customizations, and of course spending time with my wife and two kids, there’s not as much left for program development as I’d like. I am plugging away, though as time allows.

Thanks!

Tom