NHC RSS Feed WERE Useless - But Ken fixed it.

[1080iAddict]

Dear National Hurricane Center:

Your RSS Feed is useless.  Instead, change it to the RSS feed methodology for the rest of the country's alerting system so we can read the advisories and discussions under the "$Description" array in our scripts.  I don't need a scrolling hyperlink.  Also, while your at it, RSS me up some images for the 3 and 5 day tracking plots.

Thanks.  Have it done before Depression TWO forms please.   

Yours truly,

Anonymous weather buff.

[saratogaWX]

I agree, the content of the feed (other than the Outlook) is devoid of info in the <description> parts of the feed.  I'd created an rss-tropical.php to watch it, but it only shows links to the real pages, not the contents in the descriptions.  Cayman-wx and I are still working on it, but you can try the source yourself at http://saratoga-weather.org/scripts.php#NHCRSS.

Atlantic: http://saratoga-weather.org/rss-tropical-test.php?zone=A&summary=Y
Pacific:  http://saratoga-weather.org/rss-tropical-test.php?zone=P&summary=Y

I hope they'll change the feed soon to have more content 

Edit: Now the script has a second version which pulls the text from the links and returns it on the page.  Please see link above for source and docs.

Best regards,
Ken

[administrator]

Dear Anonymous Weather Buff

I saw a comment somewhere else (not this forum) a week or so ago bemoaning the fact that site 'X' didn't publish some relatively time-critical data available from their site in a form that was easily usable on other people's web sites.

It set me thinking...why would the person asking for the data want to re-publish it? The data (like the NHC data) is freely available to anyone who wants to view it on site 'X'. The person asking the question didn't appear to want to do anything with the data other than re-publishing it on their own web site. No reason was given for the re-publication...other than "We should be able to do this and demand the right to do it".

Then I thought a little further and came to the conclusion that even if the data was re-published on site 'Y' (and sites 'Z' thru 'ZZ9 Plural Z Alpha') would I go to site X or the other sites to view the data. The data is presented in a perfectly usable format on site 'X' so that's not a problem. Also if I view the data on site 'X' then I can be pretty sure the data is up to date and if it's not, then there's a reasonable chance that the same data at other sites will be similarly out of date. Also how can I be sure that if I view the data on site 'Y', etc, that those sites are actually working and aren't displaying a cached version of the data grabbed from the real data source some minutes, hours, days, weeks or months earlier?

Then I thought about the kind of sites that I find useful and those that I find less useful. Useful sites tend to have a lot of original content and aren't simply regurgitating content from elsewhere. Some of the most annoying sites are those that grab a UseNet feed and make it available via a web page. There's nothing more irritating that doing a search for a fairly uncommon subject on Google and finding 100 pages of results. You start looking at the results, only to find that 99 of them are UseNet re-publications of one original article

Not having to worry about hurricanes, means that I have no interest in the NHC data, and I'll be honest and say that I don't know what data is contained in their feeds. However, I have a feeling that if I was worried about hurricanes then I'd probably want to go to the horse's mouth to hear the information first hand rather than look at it on a site which may be somewhat out of date (and in many cases sites that re-publish data rarely give an indication of information currency).

I know there's a lot of weather data re-publication going on and of course your intention may be to do more than just re-publish the data. Perhaps you intend re-processing the data and embellishing it to make it far more useful than the raw data. If so, I think it's a perfectly reasonable and commendable activity to attempt that.

All that is of course IMHO I doubt that my HO will count for much, but I thought it was worth spending 10 minutes passing comment

[saratogaWX]

Thanks for your well-reasoned comments, Chris.  Living in California, where hurricanes are very rare, makes this more of a programming exercise for me -- the local folks in Saratoga have only a passing interest in East Pacific and Atlantic hurricane info.

I think the intent of having the information (especially for Gulf/East Coast weather stations) is to have a small alert on the website say what is up, and provide links to 'the horses mouth' (www.nhc.noaa.gov) for definitive information.

NOAA does have a nice set of RSS feeds for advisories/watches/warnings from www.weather.gov/alerts/ and many of us have scripts that read/format that for display on our home page.

The primary difference is that NOAA does include the actual text of the watch/warning/advisory in the <description> area in the RSS feed, while the NHC only includes 'Updated <date> <time>' and a link to the www.nhc.noaa.gov site.  i1080iAddict's 'lament' is that NHC is not consistent with the rest of NOAA in use of RSS, and we'd really like them to be.

I sent a polite request to nhcwebmaster at noaa.gov asking for their consideration for improving the usefulness of their RSS feeds.

Best regards,
Ken

[carterlake]

It set me thinking...why would the person asking for the data want to re-publish it? The data (like the NHC data) is freely available to anyone who wants to view it on site 'X'. The person asking the question didn't appear to want to do anything with the data other than re-publishing it on their own web site. No reason was given for the re-publication...other than "We should be able to do this and demand the right to do it".

I'm guessing the difference is that part of the NWS mission (which US taxpayers pay for) is to make the data freely available and they do actually have an RSS feed to make that possible.

After looking at the RSS, I can see why he's a bit frustrated. It needs a LOT more info to be useful. It would be really cool to have lon and lat designations, then we could use PHP to map the storms (and add value to the data!).

[administrator]

I can fully understand the challenge of programming things...I've been rising to that challenge for 29 years now and many of my programmes have been conceptual to allow \me to learn new things rather than doing anything that's really useful.

I'm guessing the difference is that part of the NWS mission (which US taxpayers pay for) is to make the data freely available and they do actually have an RSS feed to make that possible.

There are (at least!) two ways of using RSS feeds...

1) Using them in a feed reader running on your own PC. In this case you're using the unprocessed NWS data for your own purposes and using the feed reader as a quick way of spotting changing info. You also (probably) know if your PC and/or feed reader and/or Internet connection are working, so you know whether the data is likely to be valid.

2) Re-publishing the data on your web site for your own consumption and possibly for many other people to view. In this case the owner of the site might know that the feed is working properly, but other viewers probably won't if the data is valid but as with all web sites, we all know that they always tell the truth all of the time

Maybe NHC/NWS/NOAA publish RSS feeds primarily for (1) rather than (2)?

My thoughts in this case were also that hurricanes are serious things that people need to know accurate and timely information about. NHC/NWS/NOAA are responsible for getting that data to people as quickly as possible. If you republish the data, then you've got one of three conditions...

a) Feed working well. People get accurate/timely info from your web site

b) Feed broken and displaying data about a hurricane which nearly hit your area 2 weeks ago. A quick look at your site then panics people into thinking there's another storm on the way (War of the Worlds type scare!)

c) Feed broken and not displaying data about a hurricane which might hit your area in the next 12 hours. A quick look at your site makes people complacent, thinking there's nothing to worry about. I hate to think of the consequences in this case

Perhaps the programming challenge needs to ensure that conditions (b) and (c) are addressed?

[saratogaWX]

Chris:

The feed-reader program doesn't cache the feed, just fetches it and formats it so it's as fresh as the RSS feed provided by NHC.  Before the season started, the RSS feed had really old content left over from last-season's info (instead of a more polite 'Season starts May 15 (East Pacific) or June 1st (Atlantic)' -- we were all amused by the stale content, but it was NHC that was providing it.

The rest of NOAA (NWS) and USGS have also adopted a new CAP (Common Alerting Protocol) as an alternative feed .. this uses the FIPS Zone (US only) and various law-enforcement agencies can also issue CAP statements.  Unfortunately, the CAP feeds are agency-specific, and no aggregation is available.   NHC doesn't provide a CAP feed yet.

We don't want folks to get 'stale data', so no caching is done -- we'd really like NHC to address the 'freshness' and content-usefulness of the feed.  Even the www.nhc.noaa.gov homepage is a somewhat bewildering array of text-links, small maps, all organized by storm event by basin (Atlantic, Pacific).  I've found that a reaggregator BoatUS has a much more useful set of displays with bigger maps (etc).  http://www.boatus.com/hurricanes/tracking.asp but they haven't started tracking TD1 (and may not engage until it becomes a named storm).

Best regards,
Ken

[cayman wx]

Have you ever tried to get on to the NHC site when an even like Kartina is occurring? The numbers of hits is staggering and the resulting delays are sufficiently long that people give up. I was at a confernce at the NHC in Miami at which they actually encouraged people to republish the data to try to take the load off their system.

Many site operators don't want to loose a viewer to the "X" close so choose to republish the data with their page with full credit given to the source, including logos and errors but also a working "back" button.

By the way I receive an email from NHC for each of their advisories, graphics etc. There is no secret that you click a button on the NHC site and they send you an email. Would it not be more efficient to disseminate the same information by RSS feed?

Any information handled irresponsibly is useless which is why WD exists.



Roger

[Indy]

The NHC should look at doing what software download sites do and establish mirrors.  There are alot of people looking for unique content for their sites and have alot of unused resources.  The NHC could make good use of them and likely do it at no cost to taxpayers.

[administrator]

Seems like the NHC has a number of lessons to learn about the reliable dissemination and management of their data

[saratogaWX]

The NHC site is already two distinct IP addresses (which may or may not be backed up by multiple servers using load balancing).  The RSS feed is far less intensive on the servers than the main page/subordinate pages with it's high graphics content (39KB page, about 100KB graphics versus <3KB for the RSS XML data).  I would think that they'd appreciate folks grabbing the small RSS with one HTTP rather than the full page with many HTTP pulls for the graphic elements (even though some of them are locally cached on disk).  Their main page is set to auto refresh every 30 minutes.

If they'd only include the text of the items in the description fields, it would be a far more efficient way to pull and display the data.

Ken

[administrator]

The NHC should look at doing what software download sites do and establish mirrors.  There are alot of people looking for unique content for their sites and have alot of unused resources.  The NHC could make good use of them and likely do it at no cost to taxpayers.

Mirrors can work well, e.g. the SourceForge download mirrors, but unless managed, maintained and monitored properly they can also cause problems. How many times have you tried to download something from a mirror site that no-longer exists and probably hasn't existed for some time.

I'm also a little puzzled about how you can have unique content that's provided on a lot of mirror sites

[Indy]

Maybe I should call it nearly unique

I mean how many sites do you know of that offer NHC RSS Feeds?  But I definately see your point on the mirrors.  Some sites are really bad about keeping the list up to date.  Maybe if people like us started to mirror the feed on our own and advertised that fact to our visitors we can in a way help relieve stress from the NHC servers during extremely busy times.

[1080iAddict]

Chris and others,

My frustration is purely selfish.  I am not publishing the data for anyone other than Interface Viewers in my own house on my own home intranet.  As a home automation buff, I have touchscreens located throughout the house running a product called CQC ( http://www.charmedquark.com ).  You can check out the Forum at http://208.101.20.242/vb_forum/index.php .  

On these interactive touchscreens, I have complete control of everything digital in my house including control and display of digital media to all projector, plasma and other display devices throughout the house, and I can also control whole house audio and video feeds, live internet information, controlling lighting, controling media selections such as DVD servers amd cable channel programming, etc.  I can create custom user interfaces including "web widgets" as small windows on the Interface Viewer real estate on the touchscreen.  I can also create command buttons for automation actions and change display overlays to show different sequences of interface pages.  It is very cool.

So, my participation here is based on my personal desire to serve up web widgets for the information I want - information which is dynamic and changing all the time.  So I installed a webserver on my intranet with PHP and thankfully the good folks here upload scripts.  I download them and hack them - barely knowing what I am doing - to get what I want.  During next the week I will post some images of my weather templates on the CQC platform either in this forum or over on the other forum with a link here.  You'll get a better idea of what I am doing.  It's a labor of love.

So I have all the alerting done for my home and surrounding counties using a variety of the scripts I found here and the end users, i.e., me, wife, kids, guests, are the end users.  These alerts are displayed in the manner I like on my interfaces.  Some are static messages and some are scrolling alerts in the event the interface does not have much screen real estate.  Aside from radar feeds, etc.,alot is RSS based.

But I cant make a web scroller for the NHC RSS because I think they got it wrong.  They should dish out the info the same way they dish out the other alerts.  Sure, I can go to the NHC site but when I am watching TV changing channels on my interactive portal while in bed I want to glance over when I see the new message pop up that a new NHC discussion is out and watch the text scroll by with the messages.  If it warrants Ill actually get my lazy arse up and go to the PC to see the real NHC site.

So.... that is really why I am frustrated.  I think it needs to be fixed and Ill send an e-mail too.      


      

[JaxWeather]

I read this post when originally posted and considered posting then but held back, but after reading the additional posts I will offer my two cents...

While I certainly would not be adverse to the NHC modifying their RSS feeds I absolutely would not call what they have now as being useless...  I use them to pop up alerts on my site, which provides an alert that something is going on and link to the NHC for detailed information. 

Even if they alter them i would still only use them in the manner currently as I for one believe in getting the news from the source whenever possible...

Also let us not forget that here in the US we are spoiled with our unfettered access to Weather Data compared to the rest of the world (assuming Accu Weather and others of their ilk don't get their Politicians for hire to change that) and we should be grateful for the access...

Making suggestions to the NHC, or NOAA in general, is something that I would believe welcomed by those agencies and each year they invite feedback over possible changes in how data is presented, such as this past off season the method of presenting hurricane forecast tracking, however again i must decry the use of the word "Useless" as the data I am getting from the NHC is far from useless to me and my site...

Like I said my two cents (adjusted for inflation = about zero)

-Bob