Globe

[aardvark]

One of the major sources of weather data for education is at GLOBE.  http://www.globe.gov/globe_flash.html

There are also lesson plans and research projects.   What is unique to Weather Display is that it will automatically send atmospheric data and if you have the sensors, soil data  to Globe.  I send six hours of data every 6 hours to them with no effort on my part.  Hence they have a continious stream of information.  That information along with that of others can provide information for many research projects.   This is an international effort so it makes the information interesting. 

A project could be comparing rainfall trends in your area of the world with that in another part and then discussing all sorts of things.

There is one thing you have to do is to be GLOBE certified and most areas have a local  coordinator.  THis certification is to make sure that the data being sent is not bogus data, which would corrupt the database.

The database automatically screens what is sent to it and rejects that , which has errors. it can be manually corrected and then resent. 

I have rarely had any data rejected.  Weatherdisplay does an excellent job.  I asked Brian to do that for me and he has.  I know of no other software that does it automatically.

[Rachel]

Globe is a great program.  I really enjoy the fact that it is free!  The training was a very nominal fee and it included a lot of materials.  I really appreciate that it is a government program.  A competitor of Globe's is quite costly for school districts and districts are stuck paying a large renewal fee every year.  Globe's internet site technology is a little lacking, but hopefully they'll get up to speed. 

On a related note, WD does not report all of the weather data accurately to Globe.  I have reported it to Brian a couple of times but he has not had an opportunity to fix the bugs.  A simple comparison of the raw data (in globe format) reported by WeatherLink and WD will illustrate the problems with WD's data.  Globe has specific reporting formats.  The problem is not with WD's data in general, but how WD converts the raw data to Globe's required format.  So, in the meantime, it is best to use WeatherLink and manually upload the data monthly.  It involves several manual steps of exporting the raw data to a text file and cutting/paste that data into an email to globe. But, I'll be really excited when WD formats globe properly.  WD is a great program, and my preferred program for other areas. 

[aardvark]

I don't seem to  have any errors. 

I send soil and atmospheric data.   I have 7 hrs of data and I send them every six hours and it seems to work fine for me.

Sometimes, and very rarely will something be rejected such as an odd rain value or something that didn't come through, but it works for me.

I have been over my data back and forth and it is fine.   

Soil:    DAVSD USIAXABV SMS-03 200801141714 -10.2 1.1 1.7 1.7 3.3 12 18 20 10
Atmosphere :  DAVAD USIAXABV ATM-01 200801141714 -10.2 66 2.6 NW 5.7 NW 90.0 0.3 0.3 1029.6

[Weather Display]

strange its working for one person and not the other
can you summarise the problem you have again  ramandala?

[Tim Hanko]

So how does one contribute his data?

[aardvark]

well, you have to start by contacting them or your state coordinator, which they will let you know.  You go to a short training session that will give you information about stuff you already know, but oh well.  Then they give you an ID number and this manual..   You have to set up your ID and information on your data you are going to send them according to the manual.  Once that happens they activate your account and you tinker with your settings until you are happy and then let WD  do the rest.  They are a good group to work with

http://www.globe.gov/

[Rachel]

Brian,

My WD stopped sending Globe data one month ago when I moved the computer.  I am trying to figure out why or what I need to do to get it to send the data gain.  When I do (any suggestions on how to repair my glove uploads?) I'll send you a side by side comparison (WeatherLink & WD data) for you. 

In the meantime, if anyone has a Globe account, WeatherLink and WD, can you try to create a txt file for the Globe data for a 24 hour period of time in each program and upload it so we can compare the two programs handling of the data?  WeatherLink was certified by Globe, as is Ambient and WeatherView32.  Brian, it might be nice if you could obtain certification for the software.  That way, if there are any errors, it will probably be discovered in the process.  Thanks Brian for including Globe in WD.

In the meantime, can you try to make the following change for your next version of WD?  Globe requires that the data be collected in 15 minute increments.  Mine is sending it with 1 minute increments.  Can you change the Globe output to be fixed at fifteen minutes so that other elements in WD don't affect Globe's fifteen minute interval?

Also, can you change the "m" to an "M" for reporting of missing/suspect data such as wind=0?

Each field number corresponds to the 14 fields displayed in a measurement entry.
DAVAD USCAF4DT ATM-01 200702150337 11.9 63 0.5 N 0.0 N 73.4 0.3 0.3 1023.5
Field   Description
1   DAVAD - code for the Davis Weather Station Protocol
2   School ID code
3   Site identifier or number
4   UTC time using this format: YYYYMMDDHHmm
5   Air temperature (degrees Celsius)1
6   Relative Humidity (%)1
7   Average Wind Speed (meters/second)1
8   Average Wind Direction (up to 3 level compass direction e.g. N or NNE or NE)1
9   Maximum Wind Speed (meters/second)1
10   Maximum Wind Direction (compass direction)1
11   Wind Run (kilometers)1
12   Rainfall (millimeters)1
13   Rainfall Rate (millimeters per hour)1
14   Sea Level Pressure (mbar)1
1 "M" indicates missing or suspect data

Thanks!

Rachel

[Rachel]

Tim,
I don't recall from my training class but Globe may be restricted to use by schools, as it is a program to foster math/science exploration within a global partnership/awareness.
Please do inquire.  It is a good program and I don't want to mislead you.
Rachel

[aardvark]

I have globe  and here is a portion of my data.  I send it every 7 hours, with 1 min intervals and it does work.

atmospheric:

//AA
DAVAD USIAXABV ATM-01 200801281114 5.2 63 1.5 S 1.5 S 50.3 0.0 0.0 1004.4
DAVAD USIAXABV ATM-01 200801281115 5.2 64 1.0 S 2.1 S 50.3 0.0 0.0 1004.4
DAVAD USIAXABV ATM-01 200801281116 5.2 64 0.5 S 1.0 S 50.3 0.0 0.0 1004.3
DAVAD USIAXABV ATM-01 200801281117 5.2 64 0.5 S 1.5 S 50.3 0.0 0.0 1004.3
DAVAD USIAXABV ATM-01 200801281118 5.2 64 0.5 SW 1.5 SW 50.3 0.0 0.0 1004.3
//ZZ

Soil:
//AA
DAVSD USIAXABV SMS-03 200801281114 5.2 0.6 1.1 1.1 2.8 43 24 22 12
DAVSD USIAXABV SMS-03 200801281115 5.2 0.6 1.1 1.1 2.8 43 24 22 12
DAVSD USIAXABV SMS-03 200801281116 5.2 0.6 1.1 1.1 2.8 43 24 22 12
DAVSD USIAXABV SMS-03 200801281117 5.2 0.6 1.1 1.1 2.8 43 24 22 12
DAVSD USIAXABV SMS-03 200801281118 5.2 0.6 1.1 1.1 2.8 43 24 22 12
DAVSD USIAXABV SMS-03 200801281119 5.2 0.6 1.1 1.1 2.8 43 24 22 12
DAVSD USIAXABV SMS-03 200801281120 5.2 0.6 1.1 1.1 2.8 43 24 22 12
DAVSD USIAXABV SMS-03 200801281121 5.2 0.6 1.1 1.1 2.8 43 24 22 12
//ZZ

A couple of things you want to check.  Go to the Globe set up in WD and have it produce the txt file.  go to the webfiles and see what the date and time that it was produced. It should be a few moments before you looked for it.

Then go back into the globe setup and set the time to produce and upload the data then look if the wdftp fires about the time you set it for.

Then send an email off to help@globe.gov  to have them check that they are receiving that data.  I had a period that I was getting errors or that the data was going to globe hell.  They had to do something at their end so that the data would be sent then the report sent back to me

I really would hate for Brian to change the settings as my data would be bounced back to me as in error.  There is a very specific format for that data to be accepted by Globe and Brian has it right.  I would be livid to the point of being torked if I had a problem with my data getting there.

You can manually set the time at 15 min intervals  or hourly or how ever you want them to be sent.

for the other information:
GLOBE Automatic E-Mail Data Entry Processor ( data@globe.gov )

Complete instructions for Email Data Entry can found at
http://www.globe.gov/hq/templ.cgi?emaildatanew&lang=en&nav=1

----------------------------------------------------
Thank you for entering your GLOBE data!  Please review the
data you entered below.  Any records followed by an
error message were NOT accepted by the system and need to be
corrected and resubmitted before the data can be entered in
the GLOBE database.  Lines that are not followed by an error
message were accepted and stored in our database.
If you have further questions, please contact your Country
Coordinator, or the GLOBE Help Desk at help@globe.gov

[Rachel]

Aardvark, I understand you have been a globe participant for several years.  So have I.  I completed training in April 2004, right after I started www.glendoraweather.com.  I also understand that you were instrumental in having Brian include Globe formatting in WD (thank you!).  My comment was addressed to Brian.  As a globe participant, I'm sure you recall that Globe was created for a very specific purpose: for students and scientists to study global weather patterns.  Because it is a scientific endeavor, it is not up to us to create our own data standards.  There is a very specific reason why for instance that Globe uses UTC, in order for all of the  data to be synchronized/accurately assimilated into data streams.  For that reason, Globe wants the data to be generated at 15 minutes intervals, on the hour, based on the UTC.  If you refer to Globe's site, you'll see that the only three software programs recognized by globe are: Ambient, Davis & WeatherLink.  That is because Globe's scientists certified their data formating.  Did you ever seek to have globe certify the data produced by WD?  You stated that their is a very specific format and that Brian has it right.  Yet, the software has clearly produced errors for globe data in the past.  1) the windrun should not be a static number (I reported this to Brian in March 2007).  I discovered the problem myself.  Globe's quality checks did not catch this.  2)  I also reported to Brian that Globe was rejecting some data because the average wind was exceeding the max wind speed some of the time.  So, why don't we approach globe and ask if they can test WD and give their blessings to it or tell us what needs to be corrected?  One of the reasons Globe may refuse to certify WD, is that in the past Globe did not want automated data generation.  That is why the other software packages do not transmit globe data automatically.  Globe lacks the sophisticated quality control data analysis used by MesoWest.  As such, they are concerned about automated data transmissions.  If we aren't actively watching our data for skewed temps, pressure, etc. a lot of poor data could find its way into globe's data-stream.  In fact, if we are actively circumventing their protocols, there could be consequences for us.

As I stated in my previous entry today, has anyone performed a line by line comparison of data generated by WeatherLink and WD over a period of at least one week?  Short of doing so, all data should be considered suspect.  Also, do you know if there are any settings that could be changed within WD that would cause different values for the data generated by WD?  Or, is the data calculated directly from raw data and as such cannot be affected by user configuration?  If it is the former, then there's another problem.

http://www.weather-watch.com/smf/index.php/topic,23355

Best wishes,

Rachel

[aardvark]

look up dsmweather....

I have done that comparison and it is fine.    Weatherdisplay is new to them. I have pushed it and get medium response.  If I use weather link, I have to cut and paste it.   I haven't lobbyed like I need to .  However,  the data meets their screening and I periodically compare the two .. Wlink and wd.. it is fine.

[Rachel]

Aardvark,
Thanks for doing the comparison.  I'm glad to hear the data matches.  Yes, I've been cutin' & pastin' for several years and it does suck!  I am grateful that you and Brian worked on this.  I just wanted to be sure that the data reporting errors were corrected.  And thank you for your tips on how I can try to fix my Globe output.  ...still isn't working.     So, where do I set Globe's data interval if i wanted to change it from 1 minute to 15 minutes?  I also have a daily weather stat report e-mailed to me by WD and that works fine.  It was working up until I moved the station and computer.  I must have inadvertently changed something.  At least you have some real weather to report on your nice site.  I have a nice site and no weather.  Wait, I take that back...it rained! 
Regards,
Rachel

[aardvark]

so if you do the cut and paste, you get back from globe one of those  data reply emails?  if you do then the data is being received, processed and then the confirmation report comes to you.

So with WD.   the only problem I had was getting the utc off set  right.  I am normally  use the -5 or -6 offset.  I had to set -7, then have the data upload every six hours to get it right.

Check two things.  Is the globedata.txt  being produced in your web files?  If you have soil then you will also have globedatasoil.txt

if not  then we might have a problem within your ini file and Brian would need to look at it .

Below is a picture of my screen that has my setup.

[Rachel]

Thanks for the info.  My setup is correct (I compared it to your screen-shot).   I also have the FTP Setup (for e-mail) configured to send a daily/weekly weather report.  Which I receive so I know the e-mail is configured correctly.  I contacted Gobe; we'll see.

UPDATE:  I learned from Globe that one of their data validators is that the data must be sent from the e-mail address on file with Globe.  I changed my e-mail address when Globe stopped working.  I requested an e-mail address update and I am waiting to see if that fixes the problem.

Rachel

[Rachel]

Nope- still not fixed.  I am not receiving a confirmation email on the cut/paste version either.  I filled out a trouble ticket for Globe.