Weather-Watch.com

Weather and the Internet => Weather Web Sites => Topic started by: mihec on September 15, 2017, 12:47:49 PM

Title: Post data to wunderground from terminal
Post by: mihec on September 15, 2017, 12:47:49 PM
I'm collection air quality data with my Raspberry Pi and would like to do uploads directly from it. I went through instructions in the WU but it's not clear to me how could I send data to WU. Does anyone has an example for it? Ideally I'd do it with a bash script.
Thanks.
Title: Re: Post data to wunderground from terminal
Post by: mihec on September 15, 2017, 01:41:45 PM
Found a solution, it's pretty easy. Here is a sample code if anyone finds it useful:

Code: [Select]
curl "https://weatherstation.wunderground.com/weatherstation/updateweatherstation.php?ID=<station_id>&PASSWORD=<password>&dateutc=now&AqOZONE=11&AqPM2.5=8&action=updateraw"
Check WU's wiki for other fields.
Title: Re: Post data to wunderground from terminal
Post by: Weather Display on September 15, 2017, 06:30:41 PM
now that WD supports the purpleair sensor, I can send that data (the PM2.5 data)
does it then show up on your wunderground data?
Title: Re: Post data to wunderground from terminal
Post by: mihec on September 15, 2017, 07:03:00 PM
I have my own sensors (PM2.5 and O3) not the purpleair. I'd be interested if WD could read my textfile with air quality data and post it for me :-) (Same for the lightning)
I only did 2 uploads so far to test it. When I have more time, I'll write a script. I just don't have a measure to say what is 20 ppb or 10 ug/m3 . I want to let it run for a while to see if the sensors read reliably.
Title: Re: Post data to wunderground from terminal
Post by: Weather Display on September 15, 2017, 07:21:08 PM
does your PM2.5 data actually show on your WU data ?
Title: Re: Post data to wunderground from terminal
Post by: mihec on September 15, 2017, 07:27:54 PM
No, not yet. I only tested for 2 uploads in the afternoon. When I set up a script to do 5 min uploads, I'll tell you.
Title: Re: Post data to wunderground from terminal
Post by: mihec on September 15, 2017, 08:07:25 PM
OK, the script is posting the data now. It calculates the average of last 300 seconds and pushes the PM2.5 and ozone data to my WU. It is not calibrated but it should show a trend. We're going into Autumn/Winter so the PM should increase gradually.
You can check my WU ID if you want but I don't know when they accept my source. I'm ITOLMINT6
Thanks.
Title: Re: Post data to wunderground from terminal
Post by: Weather Display on September 15, 2017, 09:37:54 PM
https://www.wunderground.com/personal-weather-station/dashboard?ID=ITOLMINT6

its not obvious where they will show that PM2.5 data?
Title: Re: Post data to wunderground from terminal
Post by: mihec on September 16, 2017, 05:12:06 AM
I think (?) it shows only when selecting a city, not exact ID (correct me if I'm wrong). There is a field "Air Quality"

https://www.wunderground.com/weather/si/tolmin/ITOLMINT6?cm_ven=pwsdash_cityforecast

Does anyone send the air quality data to WU already (with purpleair)? What does it look like on the WU?
Title: Re: Post data to wunderground from terminal
Post by: Weather Display on September 16, 2017, 06:27:04 AM
I have added it to this update of WD
http://www.weather-display.com/downloadfiles/weatherdisplaytest.zip
Title: Re: Post data to wunderground from terminal
Post by: mihec on September 16, 2017, 08:04:31 AM
I've noticed my WU data is blanked out because it's being uploaded from 2 different sources. WD uploads weather values, a bash script uploads OZONE and PM2.5 fields only.
Hmm ... could WD read my OZONE and PM2.5 text files? I can adapt the content to any format required.
Title: Re: Post data to wunderground from terminal
Post by: Weather Display on September 16, 2017, 08:37:44 AM
WD does currently read data from a purple air sensor
via a text file
if you have the PM2.5 data in that file in the same format that WD will read it in
Title: Re: Post data to wunderground from terminal
Post by: mihec on September 16, 2017, 09:32:34 AM
Downloaded and installed. I see the PurpleAir tab. Could you please advise where should the file be placed to? What is the file format and name? Could I upload the O3 data as well?
Title: Re: Post data to wunderground from terminal
Post by: Weather Display on September 16, 2017, 06:55:46 PM
there is a thread about the purpleair sensor in WD
there are 2 files
airquality.txt and airquality2.txt
that thread has examples of what is in those files  :wink:
Title: Re: Post data to wunderground from terminal
Post by: Weather Display on September 16, 2017, 08:49:55 PM
if you can create a file called airquality2.txt, where WD is installed, and have in it format like
"PM2_5Value":"2.5"
then WD will pick up that value

I could also make WD look for a ozone value in that file
e.g use format
"OZONEValue":"2.5"
Title: Re: Post data to wunderground from terminal
Post by: mihec on September 17, 2017, 07:06:23 AM
You are genious. Thanks! I suppose it uploads to WU as well?
Title: Re: Post data to wunderground from terminal
Post by: Weather Display on September 17, 2017, 07:36:04 AM
it does
make sure you have ticked that you have the purple air sensor under setup, advanced/misc
and make sure you have the latest update
Title: Re: Post data to wunderground from terminal
Post by: mihec on September 27, 2017, 08:24:08 PM
I've been sending the PM2.5 data to WU for a week but I can't see it in their records. WD (under WU setup) shows that it does send the AqPM2.5 tag so I suppose this should be OK?
Title: Re: Post data to wunderground from terminal
Post by: Weather Display on September 27, 2017, 08:36:43 PM
yeah, same problem here
so what was all this about wunderground in partnership with purple air etc but they do not actually show the air quality data you send them as per their own api?
Title: Re: Post data to wunderground from terminal
Post by: mihec on September 27, 2017, 08:47:35 PM
Agree. If I find time, I'll set up another station ID tomorrow and send only PM2.5 and Ozone to it. Let's see then.
Title: Re: Post data to wunderground from terminal
Post by: mihec on October 12, 2017, 08:13:17 PM
if you can create a file called airquality2.txt, where WD is installed, and have in it format like
"PM2_5Value":"2.5"
then WD will pick up that value

I could also make WD look for a ozone value in that file
e.g use format
"OZONEValue":"2.5"

Just to follow up on this; I've looked at the data log today and I see that WD is not taking the file content reliably. I get lots of zero (0) readings even if file content is different (it is a floating point value, e.g. 5.6789). I compared WD's day log with raw plot (with gnuplot) to confirm my raw data is OK.
It's not urgent but I thought to ask you here. I've checked for file permissions and should be OK.
Title: Re: Post data to wunderground from terminal
Post by: mihec on October 16, 2017, 04:18:20 PM
Any opinion to the thing I'm observing?
Title: Re: Post data to wunderground from terminal
Post by: Weather Display on October 16, 2017, 06:28:48 PM
the latest update (.zip) WD will not not use a value if picked up as zero (hopefully)
Title: Re: Post data to wunderground from terminal
Post by: mihec on October 20, 2017, 05:36:29 AM
Hmm... I did use the latest (Oct.9) build but there are still 0 readings even if the actual airquality2.txt contains a value greater than 0. I've been monitoring for a couple of days.
With Purple Air, it is probably more reliable? I'm trying to avoid the cost of the Purple Air and just use this cheap dust sensor :-)
Title: Re: Post data to wunderground from terminal
Post by: Weather Display on October 20, 2017, 05:52:49 AM
if you could email me the file used when WD picks up a zero reading?
Title: Re: Post data to wunderground from terminal
Post by: mihec on November 02, 2017, 09:15:33 PM
I think I have resolved the issue. Arduino was sending the 20-second average data to a RPi. Here, the data was further averaged by 3 samples (=1 min in total) and then written to an airquality2.txt file. It was a floating point number with no restriction in the decimal places. When there were only 2 ( I think), WD picked the number OK and calculated the AQI. When there were more than 2 decimal places, it took 0. I changed my script to limit the decimal places to 2 and now it works OK.
Title: Re: Post data to wunderground from terminal
Post by: Weather Display on November 03, 2017, 06:05:44 PM
that will be it
Title: Re: Post data to wunderground from terminal
Post by: mihec on January 02, 2018, 06:24:10 PM
Coming back to this. Is there a limitation in the WD for the maximum AQI? My sensor is very sensitive to fog and it currently reads very high values (AQI >170). In WD I see values drop to 0. When it comes back to <160, it is OK.

BTW: WU is showing my data now.
Title: Re: Post data to wunderground from terminal
Post by: Weather Display on January 02, 2018, 06:32:23 PM
if above 500 PM2_5 then that will happen
I can fix that (and so just set a ceiling)
Title: Re: Post data to wunderground from terminal
Post by: mihec on January 02, 2018, 06:39:38 PM
Don't do that just for me. It could also be my sensor. I just wanted to verify if there is a such limitation (by accident) that would limit the AQI calculation to 170. I'll keep an eye what is going on. Right now we have conditions for such readings (fog + smoke from heating).