Sharing XML (WDFULLDATA.XML)

Not sure if there is a need or even a topic for this. But I was wondering, should we start a topic on sharing the wdfulldata.xml data so you can put different weather station data on your webpage. Might be cool for weatherstation that are in the same county or state?

Just a tought.

Anyway I am working on a wdfulldata.xml file that allows all the tags as they are in the site made by carterlake http://discourse.weather-watch.com/t/14006 I been using his layout on my sites. Why? because they pretty damn clever that is why :slight_smile:

My wdfulldata.xml from my weatherstation can be found on http://www.dayboro.info/wdfulldata.xml
Some of the templates in action http://dayboro.theweatherzone.com

I will post them for download if there is a need, but my real question was about the need for sharing wdfulldata.xml stuff.

Have fun
H.

What’s needed is a standard (formal or de facto) XML schema (format) for weather data. If we can agree one of those the sharing of data is possible. I’ve been thinking about providing data from the MML server in XML format and had started to look at possible schemas, but haven’t made much progress yet.

The schema in wdfulldata.xml would that not do for a starter? Maybee setup a dif tree structure for different weather stations. At the moment it is only:

-wxdata
–param

What if we can do something like:
–wddata (data for weather display)
—vpst (data for vp weather station)
—dalas (dalas weather station
–mmf
—etc.
—etc.

Setting up the xml and schema should be the easiest part would it not? Anyway lets see where otherpeople come back with.

H.

looks good in IE, but its not working in FF, well not for me anyway…hummmm
i see AJAX is making more news…Julian got that to work with WD on his site :slight_smile:

This is intended as constructive comment…NOT criticism…

The example wdfulldata.xml is an XML record that’s appropriate to your station, but it’s a very big record and structured the way it is simply because it’s the set of tags that WD happens to output for your station/station type. That’s probably not a good starting point for the design of an XML schema. If you’re designing something for general use, probably by users of many station and software types, you need to make sure it’s as station/software agnostic as possible, with station/software specific data defined as optional.

There needs to be discussion/agreement about a number of topics. We can’t be sure what people will use the data for, so we need to define the fields/values/etc very well to ensure people don’t use the data incorrectly, especially if trying to output it from some other software than WD or more so if they’re using the data as a feed into some forcast modelling package! Some things that I think need discussion/agreement (and this isn’t an exhaustive list) are:

  • should it be one huge record, or should some normalisation of the structure be done, e.g. why not have a ‘current conditions’ schema and a ‘historical conditions’ schema or lite, standard and full schemas? Some people won’t have any interest in historic data, so making them download what will end up (when fully defined) as at least a 30-40k data file to get the few current conditions details they want is probably unrealistic…both in terms of their own and your bandwidth usage. It might not be a problem to everyone, but if I started to use the schema as defined to publish data from MML in wdfulldata.xml format my bandwidth usage would multiply by a factor of at least 6-8 times!

  • should fields be named after the WD tag names, or given a more appropriate name, e.g ‘hum’ v ‘Rel_Humidity_%’

  • date/time formats (12 v 24 hour, dd/mm/yy v mm/dd/yy v yyyy-mm-dd v dd-mmm-yyyy, etc).

  • are times in UTC or local time, with or without DST? Is a DST indicator needed?

  • whether fields are output with embedded units or with separate units fields or whether units are just assumed/part of the schema definition, e.g. what units apply to ???

  • whether fields are output in all possible units (pressure in mbar/hPa/in/???)

  • numbers of significant digits/number of decimal places. How to handle very big/small numbers 3.142*10^6?

  • how should missing data be denoted?

  • whether to include all possible fields just because there are tags for them, or whether they’re not really weather related so should be in another schema or even calculated locally, e.g. Easter dates.

  • how do you define a month/day/other period? Is it the last 31 days, last calendar month, calendar month to date, etc? Does it start at midnight UTC, 0900 local time, or can it vary?

  • what language to adopt, e.g. for descriptive fields…local or english or french or spanish or Esperanto, or include all possible languages

  • should composite fields, e.g. 22.7 C on: 28 Dec 2005 be broken down into constituent parts, e.g. temp, temp units and date field?

  • what to do about WD tags (or other data) added after the schema is defined and what to do if a WD tag changes it’s data/structure subsequent to a schema being agreed? wdFULLdata.xml is really only wdFULLdataon31December2005.xml because new tags seem to arrive every week or three.

Assuming people don’t just adopt the WD tags version of wdfulldata.xml, then it will probably need Brian to add some new tags to support the new schema, so there’s probably work required in WD to enable all this once it’s been decided what data is needed.

Dude, you must be a keyboard wizard. Yep spot on. I am just a “kiddo” looking around the corner on this stuff (like windy pointed out, my site works in IE not in FF so i need to fix that).

I like using the XML file to reduce the traffic, that was its purpose. If we can reduce the traffic etc to update web pages would be great. I can only see what I can see and that is the output from wd. The tags sort of make sense to me but if we can come up with a common naming convention that would be great but “why fix what ain’t need fixing :-)”. This is why I also start stripping the non VP tags.

If there is the ability to have multiple XML files for different purposes would be good, but my initial thought was not to upset the WD code to much but that is my oppinion.

Maybee something like:
general tags XML file
station specific tags XML file (or have all station writing to the same tags but that reduces the ability to have different stations hooked up, not sure if you actually can)
history specific tags XML file (more a search and retrieve from somewhere eg using the client*.txt files)

But good to get the discussion going. The question remains are you guys ready for something like this. I am just a bystander and actually do not do anything :-). But here is a commercial thing, if we can have multiple mods for different purpose it might be actually an add on $$ (Oops, shouldn’t give Brian any ideas)

PS, criticism??? Puff I always assume other people know more than I do so I see it as a learning thing. I do not get easy offended.

Couple of thoughts here:

Some of your schema questions could be answered by simply following WMO standards. Like weather observations are typically reported in UTC. Wind speeds are reported in KTS (I know everyone thinks that because the U.S has not converted but really it’s the WMO standard). Temps in Celsius. So on and so forth.

Secondly, (and by applying this statement I’m not proporting favoritism to the U.S NWS) why not apply a schema similar to the U.S NWS XML “standard”. Or better yet, determine the national schemas and utilize them. I’ve utilized the NWS so that when my personal weather station is off line, I can easily use the closest NWS site because the schema doesn’t change. Also, if all of the local personal weather stations did similiarly a “standard” XSL could be applied to all obs utilized by a website for use in ‘plotting’ obs.

Just my thoughts.
Paul

Forgot to mention it, in a couple years this may all be for not anyway as WMO working out that all alpha-numeric data will be formatted into the digital format of BUFR. We’ll have to wait and see what an observation will look like when that comes out!?!

Paul

I’m sure there are standards, e.g. WMO, that could be adopted, but you’ve got to get through the multi-national discussion first. For example, US users really won’t like using Celcius and having to convert to get to the real temperature :wink: Perhaps a starting point is propose WMO units, allow a 4 week discussion/argument period then enforce the WMO units :wink:

NWS XML schema…is it published. Last time I looked all I could find was an example snippet. From memory the definition of temperature seemed to run into about 400 lines. Either I didn’t look hard enough, or perhaps they’ve published the scheme since I went looking (about 4 months ago I think).

Sorry for dipping in and out of this discussion. I’m a little busy with various things at the moment, but I am really interested in the definition or adoption of a reasonable XML weather schema.

WMO is World Meteorological Organization. That is the multinational body of the UN. Thus I’m not proposing a US standard. And US standard temp is celsius in a METAR observation because we use the WMO standard. Any posting on the web of an ‘official’ US observation is already a conversion. Last time I looked at it, only the Eastern Bloc nations reported outside the WMO standard and I believe the only difference there was that they used MPS rather than KTS. Not sure why, but not like in this computer age we can’t convert easily! Besides I’m 'merican and like Celsius! It’s origins are more meteorologically sound and it is extremely easy to convert to F (notice I can’t even spell our unit of measure) if need be.

F= ((C x 2) - 10%) + 32

Just my thoughts!
Paul

oC:= (oF - 32) / 1 . 8 :wink:

Brian,
You are absolutely correct. I forgot to mention with my formula it was a close approximation that you can easily run in your head. It was a formula given to me back in my observing days by a senior officer.

Paul

Sorry…I know who WMO are and how they fit into the equation. My comment about US units issues is that the intended purpose for sharing XML files was to allow data sharing between web sites. When the subject of units comes up there is often a comment from US users that they don’t want Celcius because no-one understands it. WD is littered with tags showing degF equivalents of the base degC data (and other units) mainly to suit the US market.

I have no problem with degC…although KTS is a bit alien to me. I tend to read KTS as being more or less MPH to avoid confusion!

Have you found the NWS XML schema definition?

Looks like they are still trying to sort it out.

http://www.nws.noaa.gov/forecasts/xml/xml_change_notices.php

http://xml.coverpages.org/ni2004-12-10-a.html

H.

Sorry all, I should have realized that my NWS comments would stir up US nonstandardization comments. But what I’m trying to get at is what do your Met services use. I’ll break from any more comments here until I can contact my friend at UKMO and see what their XML standards are. Sorry for intruding on your parade.

Paul

Seems like the NWS isn’t as xenophobic as some think, all temps appear to be in both F and C in their XML/RSS spec.

http://products.weather.gov/PDD/NWS_Current_Observations_RSS_XML.pdf

eg

91 F (33 C) 91 33

Paul, that is where this forum is for. If we (actually Brian and Co) can get it right we all will be winners. Different opinions only create stronger solutions. :lol:

H.

It’s only a discussion, so please carry on contributing.

I was just trying to inject a note of realism into the discussion process. I know there are many US WD users who are quite happy working in non-local units (like Celcius). However, the US public at large don’t seem to be quite as up to speed with Celcius, mBar/hPa, mm, etc. US WD users understandably tend to build their sites using units they expect their users to be familiar with.

So if a XML schema includes data mainly in ‘metric’ units, I’m sure there will be calls for other equivalent tags containing the same data in non-metric format. We know that the data can easily be converted between units, but too many people expect the information to be handed to them on a plate without them having to do any work for themselves. And with XML some people might want to use it in ways that don’t have easy methods of scripting the conversions. If you use XML from PHP or Perl (or any number of other languages) then it’s not difficult to convert the values. But stick the XML into a XML viewer/formatter and you might not find it as easy to view the data in the units you are happiest with.