Forcast

Hello,

For the people who didn’t know, I registered WXSim’s to get my own customization, had it about a month maybe now and wanted to show you my 3 day forcast. The program is simply amazing (just like WD) and once you start you can’t stop. lol

Below is the link to my website that has the 3- day forcast for January 10 - 13. What I’ll do is monitor the high’s / lows of what it predicts to what actually happends and show how close they are.

As I’m writing this topic message I wanted to show you how close it is. The temperature is suppose to drop tonight and get higher by mid morning (around 8:30) and looking at the Dew point and Humidity they are really close right now.

http://mysite.verizon.net/hobartweather/dailylogout.html
Time Temperature Humidity Dewpoint
03:15 26.6 75 19.7
03:26 25.3 78 19.4

http://mysite.verizon.net/hobartweather/forcast.html
TIME WEATHER AIR DEW WET WCF HT.I W.DIR W.SP %RH INT TOT SN.C THK
3:00a CLEAR 29.1 21.9 26.8 29.1 29.1 SE 1 74 0.00 0.00 0.0 5385
3:30a CLEAR 28.0 21.3 25.8 27.6 28.0 SE 2 76 0.00 0.00 0.0 5387
4:00a CLEAR 26.9 20.8 25.0 25.3 26.9 SE 2 77 0.00 0.00 0.0 5389

Does anyone else here have WXSim’s? Have you had good luck with the software? Let me know…need more wxsims buddy’s…That way i can stop bugging Tom (LOL) and bug you for questions and help. JK :slight_smile:

Talk to you later, time for bed now! :slight_smile:

I do indeed have it, though to be honest, can’t figure it out! I’ve read through much of the manuals and just can’t seem to figure out what to feed it to get the accuracy that Tom got for me when he first customized it! I truly wish Tom would customize the ‘download’ for you as well and give you links to EXACTLY what he ‘suggests’ to add to the automated download to get a good forecast… then we could tweak his ‘baseline’. I realize that some of the data can’t be automated, but much of it can… I thought about buying his optional package for downloading… but without a hand holding it wouldn’t help me as I’d still not know what to ask it to download…

I had hopes that eventually I could either automate WXSim to produce forecasts and then have WD use that data to provide a forecast.html type page, or have it used such a way that WD could automatically call it, pass what it needs to generate the forecast and then use the resulting forecast and data to create a forecast page… well… I’d be in Heaven, and I imagine Tom would sell a LOT more copies of WXSIM!

WXSIM would need to be able to support command line type inputs… like c:\wxsim.exe -source C:\mysources.txt -pathtoWD -output c:\forecast.txt

Where the mysources.txt would be a text file with links to the internet based data WXSim needs to use to create it’s forecast, and the -pathtowd would be the path to archived recent weather from our personal Weather Stations. The -output would create a text file with the resulting forecast that WD could then parse or use to make a forecast page in HTML…

Man… would that be awesome or what???

Until it can be better automated, and until I can figure out what exactly WXSim really needs to best create a forecast for me, it’s pretty much a tool that I can’t use with any consistency or regularity… with the automation and possible integration with WD it would be something I would use constantly… and I’d bet, so would others!

Just my thoughts!

Hi Jason and Robert,

First - Jason, thanks for the good word! :slight_smile:

Robert - thanks for the suggestions you listed. Automating it further would surely be attractive. The nature of the program is such that too much automation would start to rob it of accuracy, because there are decisions the user really needs to make to make it a good forecast.

Please let me know what questions you have about what to import. Basically, it should be the same sources as in the sample data file I sent with the customization. In terms of priorities, you almost always want the READY data, at least if any upcoming changes in the weather pattern are likely. If there’s any significant wind, you also want advection data (METAR). The sample I sent I some of everything you might ever need or want to use, but certainly more than is required in most cases.

WXSIMATE automatically gets just the items you want (READY being a notable exception), the same way every time, so it essentially perorms the input end of the automation you were talking about. The output still requires manually running the program (doing the appropriate mouse clicks, etc.).

Thanks for posting, and I’d be glad to hear from either of you (or anybody else who reads this, for that matter).

Tom

I’ve sometimes wondered whether it would be worth having an automated ‘run with defaults’ option. I find that I tend to end up pressing the same buttons each time I run a simulation. A lot of that is that I’m still learning how to drive it and I think the default options are safest until I understand the impact of all the options (which could take years!) However, I have wondered whether running with defaults might give a fairly decent simulation of conditions over the next 24 hours, accepting that on occasions the output could be very wrong or the temperature, etc results might not be as good as they could be. Maybe the default run parameters could be user definable, e.g. always use all stations for advection if the program uses another default.

You’ve identified a significant problem with automation though, i.e. that you do really need the READY data and that can’t be downloaded automatically. I can also see why you might not want to add a ‘run with defaults’ option…if it makes people lazy and they just use that option all the time then they won’t get good results and you might get some negative comments about how well to software works.

One specific automation item that I would find useful is the ability to automatically output the results to a file (including CSV output) at the end of the run. The output file would probably need to have a timestamp included to make it unique. I’ve lost count of the number of times that I’ve re-run a simulation to compare the output of two runs only to find that I forgot to save the output from the first run!

Hi Tom! I’ve looked at the WXSIMATE application and agree it will do most of what I asked, but even that program isn’t all that intuitive to me! I’d love to have the sample data file you provided with the customization be a ‘template’ for wxsimate, or perhaps if I bought wxsimate you could provide a template for wxsimate using the information sources you selected for my customization… does that make sense? So in wxsimate I could just load the ‘Valle Vista’ template and wxsimate would use all the ‘standard’ downloadable sources to produce a file that I could then import into wxsim. One button push… I would then have to go get the model data, but there seems to be no way around that yet (why is that? why don’t they make that data available in such a way that we could automate it?).

I’d also like to see wxsimate be able to pull data files from Weather Display via http… currently I have to pull those files off my webserver and then load them locally (I don’t run wxsimate or wxsim on the webserver itself)…

As for Weather Display, I’d love for Brian to have an option to “Import Forecast from WXSim” so that we could post forecast data using your apps output as an option…

So for me… modify WXSIMATE to be capable of being a ‘One Button’ app… using a customized template that contains all the data optimized for our individual sites… allow it to pull the appropriate WD data via http… I don’t think I’ve ever been able to create a data file even close to the one that you provided for me when I got my customization, in part due to the fact that I’m not sure where you got all your information nor am I clear as to exactly what you got (Especially with the READY data… there are SO MANY options and I’m not entirely clear which you used, which you suggested and which I should use)… to that end I’d love to not only have the sample data, but to have a template file that I can use with wxsimate to recreate the data (minus the READY) to the same level as what you provided… and as for the READY, perhaps better documentation in the sample data file to explain which of the many, many, options you used to create the sample… that way I could more easily jump into the site and grab the exact 6 or more fields I should look at and get back out pretty quickly.

If you were willing to really modify wxsim for WD users, a “Get current conditions from WD” would also be a time saver… I’d guess if we supplied the URL to our clientraw.txt file it wouldn’t be very difficult to get that data… I’d bet we could even add data to provide some of your ‘optional’ input… past temps, past rains… so a typical morning, which is generally a rushed, chaotic type of thing would involve…

  1. Open WXSIMATE and do a standard download (1 button push) and export it to a single file.
  2. Get the appropriate READY Model data and append this to the file WXSIMATE produced.
  3. Open WXSIM and import the file just produced, hit a button to ‘get current conditions from station’, tweak a few minor things and then run the forecast resulting in a timestamped forecast file.
  4. Using WD import the forecast file and have a htm page generated using that data.

Sound easy doesn’t it? (Wink)

Anyhow, I’d love to use the apps more, but find the curve is realistically too tough and too time consuming for widespread usage. If there is any way to make it easier to come up with acceptable forecasts then I believe it would be much more widely used… and the more it is used the more we learn it and the more we learn it the better it’s use! The first step though is to get the unskilled hobbiest to use it, even if much of it is automated at first…

WxSimate does this. It reads the WD log file to get as much relevant info as possible. I suspect that it uses more info than is available in the clientraw files, or perhaps put another way, more info processed from the minute by minute records over a longer period than clientraw contains. The import of WD data in WxSimate is probably the easiest bit to do. It’s the other combinations of RAOB/METAR/SYNOP/etc and working out reliable download sites for your area that I find more difficult.

I never got this to work, so I stand corrected… I had assumed that WXSIM would use the WD data, but didnt’ know it would set the initial settings to match the most recent readings… temp, dew point, barometer and such… it’s always a bit tedious to screen toggle back and forth to see what my current temp is, current dew point is, current winds are, current barometer…

But you are right, the RAOB/METAR/SYNOP stuff kills me… takes forever and I’m not even certain which sources and which options Tom used to get that sample data he sent me. Automating that in a single template would make it MUCH easier to import!

The forecast ability is the one thing I would love to have with WD.

Yeah, me too… which is why I’d love to see WXSIM automate a bit better… I realize it can’t automate it all… but it does a VERY good job with just the customization Tom gave us… I imagine Brian wouldn’t have too hard a time grabbing any type of forecast output from WXSIM and spitting out a html page with it, or creating WXSIM Tags from it…

If Tom could find a way to better simplify getting the basic forecast and minimize the amount of work needed to get that forecast (again I’m talking about a “Good” forecast, not a “Perfect” forecast) and WD could then put out a reasonable 3 day forecast… well that would be AWESOME!

WD would have the capacity then to create the most accurate 3 day forecasts of any commercial software, and more users would be interested in Tom’s amazing product, which would help him no doubt as well!

Even though I don’t use it much, and don’t have the time to learn it to the extent I’d like, I am happy to have payed for it to further Tom’s hobby and contribution to the Weather Forecasting world!!! But man, I’d love to integrate it better and be able to use it effeciently and quickly!

HI,
As stated by Chris, wxsimate does all of the downloading except ready that you need. As for getting data from …wd it does that very well. It takes the last 24 hours and averages it . If you get wxsimm I recommend wxsimate also.
Chuck

Hi Everybody,

I’m excited to see all this discussion of WXSIM! I’ll try to briefly answer a few questions that have been raised, and offer a few comments.

First, WXSIMATE already does much of what Robert was suggesting. With a single click it will go and download all the internet data WXSIM needs (EXCEPT the READY data - more on that in a minute), cull through it to pick out the parts that are relevant to your station, and then append (previously saved) READY model data to it. It saves this data under a file name you can define in advance - for later use in WXSIM.

Regarding the READY data, it would be nice if it could be downloaded automatically, but it’s informative to read the note on the site. They specifically want data to be accessed manually (using the password, for a couple of reasons. First, too many automated programs grabbing data from it would crash their site. Second, they’ve got a great site and they want you to actually look at it. I must add that they are performing a wonderful service, using a few US taxpayer dollars, to the whole world. In fact, for most of the world, this GFS model data is the only free data of its type. In contrast, output from the EMCWF (the big European model) is highly restricted and you have to pay to get it. I’m really grateful to the NOAA ARL that such data is available at all!

As for WD data imported by WXSIMATE - it gets various types of data, going back as far as 30 days, and does a variety of averging processes to “feed” WXSIM useful data. I actually think that the internet data retrieval is more vital to running WXSIM than is the WD data - though both help.

Responding to Chris’s desire for automatically exported files - they already exist, including the .csv file (which I implemented a few versions ago). Just be sure to get the latest version of WXSIM (it’s 10.6 right now). Every forecast is saved, though by default with the name 'latest (.txt, .csv, and .wxf for use in the Retrieval module).

Thoughts on automation…

I must say, that would be fun! I’d like to walk up to the computer and see a forecast already made, and then see current data superimposed on it to see how it’s doing so far! Three issues are relevant here:

One is technical - that is how I would actually implement that. A brief effort at that several months ago didn’t get very far before I ran into difficulties - though I don’t think they’re insurmountable.

Another is that the forecasts would simply not be as good. Most feedback I get on forecast accuracy is very positive, but there are exceptions. Usually these result from the user not understanding (which is understandable!) what all to do, though occasionally it’s just a tough place to forecast for, or I had insufficient data to do a good customization.

The third item is that, actually, there’s quite a bit of value in being “forced” to work through the process of running WXSIM. I’ve had comments from users to the effect that they had learned more in a few weeks of using WXSIM than in all their years of their weather hobby. I think carefully running it (and just playing around with it, making fake forecasts under various ‘pretend’ conditions) helps develop forecasting intuition very rapidly. Also, it’s good to learn about all those data types, though they can be confusing at first.

That said, it would still be great to have at least the option of a quick, automated run! I see the interest is there, and I will definitely revisit the possibility!

Thanks!

Tom

I’ve just had another look and upgraded to 10.6, but I can’t seen an option to automatically export a file at the end of a forecast run. I’m probably missing something obvious! I know about 'latest, but because I’m still trying to learn WxSim, I try running the same data with different options to see what happens to the output data. To allow comparison between two runs, I really want to make sure I capture the output from each run. I quite often forget to copy 'latest to a stored run file and by the time I realise I’ve re-run the forecast and wiped out the previous 'latest file.

I must say, that would be fun! I'd like to walk up to the computer and see a forecast already made, and then see current data superimposed on it to see how it's doing so far!

I agree. That would be excellent :smiley:

Another is that the forecasts would simply not be as good. Most feedback I get on forecast accuracy is very positive, but there are exceptions. Usually these result from the user not understanding (which is understandable!) what all to do, though occasionally it's just a tough place to forecast for, or I had insufficient data to do a good customization.

I think this is the most difficult issue. I’m not experienced enough in running WxSim, but your experience of years of running it would be interesting. How many runs do you do where you effectively choose a set of default button clicks to produce a forecast. For example, I always click the ‘Use READY data’ button. Maybe 9 out of 10 runs you never adjust any parameters during a run because they don’t need adjusting for the current conditions. If so, it’s possible that 9 out 10 (the number may be much lower!) runs might produce reasonable forecasts using default answers/settings, and you only need extra input under difficult/unusual conditions? If you can get reasonable forecasts more often than not using the defaults, it would be good to be able to do that.

The third item is that, actually, there's quite a bit of value in being "forced" to work through the process of running WXSIM.

I totally agree. If there was a way of running WxSim automatically using the default values for days when I’m busy, I would still want to drive it manually to learn more about how this thing we call ‘the weather’ works! I would never suggest that there was only an automated version…the manual version would still need to be available.

Finally, just another passing thought. There are three ‘periods’ of GFS READY data. I don’t know if this is sensible, but if we could download three files, containing the 0-84, 0-180 and 192-384hr data, would it be possible for WxSim to combine the three files to get data for 0-384 hours? I know that 384 hours is pushing the WxSim methods way too far, but grabbing a set of READY file on, say, a Monday that still had model data on a Tuesday or Wednesday might allow more runs to be automated. I know the data gets old, but old model data could (!) be better than no model data.

Hi Tom,

This is great news. Please keep us updated on your progress as I am very interested. Thanks.

Thanks again for all the discussion!

A couple of quick notes:

Chris, the .csv file is being created automatically, whether you tell it to or not. What you can do, though, is click ‘Save’ when you’re done (or also under ‘File/Save’ on the Entry form if you have already returned to it) and give that ‘latest’ file (all three of them, actually) whatever name you want. The File/Retrieve option pulls up the .wxf version, with tons of display options.

You can use old READY data, understanding it gets less accurate with the passage of time. The 192-384 hour data is only a little better than chance, but the 0-180 isn’t bad, for a rough 1 week outlook. I almost always keep it to just the 0-84 hour range myself. If there ever is an automated option (which I will look into), you could probably get the READY data once every day (or even two, perhaps) and the program could still use it.

As for differences in how I run it from time to time, by far the biggest issue involves use of advection data. In the US, you can import MOS data after wind shifts, for advection, and there are choices to make about the flow curvature, the shape of smoothing fit to be applied, etc. In England (or anywhere outside the US actually), the MOS data isn’t available and you have to resort to the somewhat less reliable ‘Neutral’ or ‘Default (frontal codes)’ advection options. I suppose a standard choice could be defined, though, and the output would at least be reasonable.

As long as users clearly understand the trade-offs involved, i agree that the automated runs would be really nice. I’d get a kick out of it myself! (I would just take such forecasts with ‘a grain of salt’).

Well, thanks. I’ve got to go do a customization for Iowa!

Tom

Sounds like what I do with the local TV Weatherman and his forecasts #-o

Hello Everyone,

I haven’t bragged about WXSim’s software for a while now…so its time for me to start bragging again…:slight_smile:

Before I do though, my ONLY suggest that I want to throw at Tom is to WORK WITH BRIAN about adding custom tags to WD with your forecast data. Thats the ONLY thing I would really love to see automated…If we get that then it would save me a lot of copy and pasting and all Id have to do is take the 5-10 minutes to actually run the forecast on a given day and WD could do the rest for me. Maybe make seperate files for each day (day1.txt, day2.txt,etc) and have the forecast post in there and then have Brian add some kind of custom code that WD would read whats in that file and use that text or whatnot. I’m not sure how you both program but I’m just throwing my only idea to you guys right now :slight_smile:

Anyways now to brag about the software…LOL I ran a forcecast last night with WXSim and check out the predictions (The full forecast is at http://www.hobartweather.net/forecast.html) Forecast Feb 15-17.

Feb 16 ---------------------------------------------------------------------------------------
Thursday
TIME WEATHER AIR DEW WET WCF HT.I W.DIR W.SP %RH INT TOT SN.C THK
12:00a DNS.OVCS 32.6 30.7 31.8 25.4 32.6 NE 8 93 0.00 0.06 0.0 5423
12:30a DNS.OVCS 32.2 30.4 31.5 25.0 32.2 NE 8 93 0.00 0.06 0.0 5426
1:00a DNS.OVCS 31.9 30.1 31.2 24.6 31.9 NE 8 93 0.00 0.06 0.0 5430
1:30a CLOUDY SLT/FRZ.R 31.6 29.9 30.9 24.2 31.6 NE 8 93 0.03 0.07 0.0 5433
2:00a DNS.OVCS 31.2 29.7 30.6 23.7 31.2 NE 8 94 0.00 0.08 0.0 5436
2:30a DNS.OVCS 30.9 29.4 30.3 23.4 30.9 NE 8 94 0.00 0.08 0.0 5439
3:00a DNS.OVCS 30.6 29.1 30.0 23.0 30.6 NE 8 94 0.00 0.08 0.0 5442
3:30a DNS.OVCS 30.3 28.7 29.7 22.7 30.3 NE 8 94 0.00 0.08 0.0 5445
4:00a DNS.OVCS 30.0 28.4 29.4 22.3 30.0 NE 8 94 0.00 0.08 0.0 5447

Below is parts of the Daily login times from WD.

Time Temperature Humidity Dewpoint
00:20 34.2 83 29.5
00:30 35.1 94 33.5
00:40 34.9 94 33.3
00:50 34.9 94 33.3
01:00 34.7 94 33.2
01:10 34.7 94 33.2
01:19 34.5 95 33.2
01:29 34.5 95 33.2
01:38 34.5 95 33.2
01:48 34.5 95 33.2
01:58 34.2 95 32.9
02:08 34.2 95 32.9
02:17 34.2 95 32.9
02:25 34.2 95 32.9
02:34 34.2 95 32.9
02:43 34.2 95 32.9
02:52 34.2 95 32.9
03:03 34.2 95 32.9
03:10 34.2 95 32.9
03:19 34.2 95 32.9
03:28 34.2 95 32.9
03:38 34.2 95 32.9
03:48 34.2 95 32.9
03:57 34.2 95 32.9
04:07 34.2 95 32.9

WXSim was able to predict the temp, humidity, and dew point within -/+ 5 degrees or percentage for this morning (on the 16). In fact I was at work yesterday (on the 15th) and my coworker said “Its Raining outside” , i said “I know, the forecast predicted it…” so I checked the actual temp from my website and the actual “Condition” said “7:30p DNS.OVCS SNOW/RAIN 37.9” The actual temp was like 35.6 or so (estimate from my log file). He hit me and said “See your in the wrong field, go be a weather man.” LOL!

Anyways…just giving my props to Tom and to throw the suggestion at him (and Brian). I’m sure Tom loves hearing when forecasts come out good! heh heh

-Jason

If the forecast is in a text file or files you could use the “include a file” custom tags to get it on a webpage. See http://discourse.weather-watch.com/t/14143

Having looked at the WXSim site and a few of the messages posted here, it is obvious that what I call a forecast and what WXSim produces are two different things. All those codes and numbers…yuck.

I would prefer a simple icon/text forecast similar to what I get now from the NWS as seen here

http://www.wrh.noaa.gov/total_forecast/printable_forecast.php?wfo=sto&zone=caz016&fire=caz216&county=cac115&dgtl=&lat=39.49000&lon=-121.5500

I just wanted to post a message that a couple months ago I suggested to Tom that he adds a feature to where it will produce a sentence forecast version such as what NWS does. (ie: Monday: Rain with the high in the upper 60. Monday Night: Lows in the lower 30’s with west winds, etc)

He said he had others request this and he will consider implimenting it, just no eta at this time. I don’t want to speak for Tom since I don’t know if he’s really doing this or not but thats what he told me in his reply to my suggestion.

I personally would love to use a sentence base forecast then the hourly one. The hourly one would be perfect on a seperate website (ie: file by itself). As far as the sentence base i’d love to use this on my flyer and get rid of the NWS forecast and put WXSim’s forecast up vs the NWS. I’d make WXSim’s forecast the main forecast that users see when they click forecast but I’d add a “Click here for NWS Forecast” in case users want to view that.

Sorry if im not making sense, Im just waking up and about to leave for work. yay…So im a litte out of out it and running low on surgar :slight_smile:

Wouldn’t it be rather trivial for a code junkie to parse the data to produce such a thing? Find the high value and time, find the low value and time… and look for any precipitation and amounts… then dump that into a tag or some other file format for displaying… Could WD perhaps do this with the .csv forecast file WXSIM produces? Brian is amazing at parsing, perhaps he could create a custom page that looks for a current WXSIM forecast file and converts it into html…