cloudy

Author Topic: Data quality checking  (Read 10946 times)

0 Members and 1 Guest are viewing this topic.

Offline carterlake

  • Tom Chaplin
  • Posts: 2,273
  • Carter Lake, Iowa USA
    • Carter Lake, Iowa Weather
Data quality checking
« on: August 28, 2004, 01:10:21 PM »
The daily data quality checking in CWOP is simply the coolest feature!

CWOP Data Quality Check

The only issue I have is that it's entirely based on neighboring stations without considering THEIR errors in data.

(Yes, I've been checking my humidity versus the nearby METAR {1.2 miles} and it does seem a bit off... but then that station is 1.2 miles closer to the Missouri River so mine should have slightly lower humidity. And perhaps that is why mine is off from other area data... I am closer to the river as well.)

WD; Davis VP2 6153; Quickcam for Notebooks Pro; Boltek w/ Nexstorm; GRLevel3; Live NOAA radio

Offline carterlake

  • Tom Chaplin
  • Posts: 2,273
  • Carter Lake, Iowa USA
    • Carter Lake, Iowa Weather
Data quality checking
« Reply #1 on: August 28, 2004, 08:06:42 PM »
Well, I decided to do my own looking at my data versus the local METAR by using the last 7 days of METAR data.

Temperature is very close. If anything I would say the METAR needs some better solar shielding. There are times mine is reading 70F when the METAR is at 80F. Or perhaps they have it too close to a runway.

In fact, 10 pm to 6 am, the difference between our stations is usually 1-2F. If I look at 7 am to 9 pm, the difference can be as high as 13F (mine being the lower of the two). I would say it's my station but I have another thermometer about 1 foot from the station and it reads within .2F

Barometeric pressure is spot on between the two. In fact, the average difference between the two stations over the past 7 days has been .04 inches.

Humidity is where we have the big descripancy with a high of 30% and an average of 4.5% difference. Interestingly, the two graph very much in line with one another. It's just the peaks and valleys from the METAR are more pronounced.

Anyway, I think I've answered to myself how well this WS-2310 is operating. It's pretty darn good for being one of the most inexpensive weather stations on the market.

WD; Davis VP2 6153; Quickcam for Notebooks Pro; Boltek w/ Nexstorm; GRLevel3; Live NOAA radio

Offline carterlake

  • Tom Chaplin
  • Posts: 2,273
  • Carter Lake, Iowa USA
    • Carter Lake, Iowa Weather
Data quality checking
« Reply #2 on: August 28, 2004, 11:42:25 PM »
Temp (METAR in Black)


Baro (METAR in Black)


Humidity (METAR in Black)

WD; Davis VP2 6153; Quickcam for Notebooks Pro; Boltek w/ Nexstorm; GRLevel3; Live NOAA radio

Offline Weather Display

  • Posts: 65,579
Data quality checking
« Reply #3 on: August 30, 2004, 05:50:58 AM »
wow, amazing work in analysis ! :)

Offline carterlake

  • Tom Chaplin
  • Posts: 2,273
  • Carter Lake, Iowa USA
    • Carter Lake, Iowa Weather
Data quality checking
« Reply #4 on: August 30, 2004, 11:31:37 PM »
Quote from: windy
wow, amazing work in analysis ! :)


Thanks.

Anybody can do this by the way.... the METAR data is online for the taking and you've provided the log data is very easily sorted CSV export format.

Excel handles all the rest.

I guess CWOP does raise an interesting question. So you're collecting weather data. How accurate is it?

I know my temp and barometeric pressure are spot on. But I wonder about my hydrometer. Unfortunately, I'm not currently wanting to take it down for the 6-8 hours it would take to test (using either the wet salt or wet towel calibration methods). After all, I don't know anything is wrong since the METAR is so much nearer to water.

WD; Davis VP2 6153; Quickcam for Notebooks Pro; Boltek w/ Nexstorm; GRLevel3; Live NOAA radio

Offline Weather Display

  • Posts: 65,579
Data quality checking
« Reply #5 on: August 31, 2004, 12:09:29 AM »
alot of the "cheaper" electronic humidity sensor too seem lacking at go much above 95% humidity.,....i.e the responsiveness gets less and less....also, they have the same issue once down to 15% or less humidity....i.e there responisiveness is not linear.....and they have not been calibrated very well i guess....

Offline carterlake

  • Tom Chaplin
  • Posts: 2,273
  • Carter Lake, Iowa USA
    • Carter Lake, Iowa Weather
Data quality checking
« Reply #6 on: August 31, 2004, 12:22:52 AM »
Quote from: windy
alot of the "cheaper" electronic humidity sensor too seem lacking at go much above 95% humidity.,....i.e the responsiveness gets less and less....also, they have the same issue once down to 15% or less humidity....i.e there responisiveness is not linear.....and they have not been calibrated very well i guess....


Well, I've also learned after doing some digging that the humdity sensor is +/- 7% accuracy (LaCrosse apparently doesn't advertise this). That seems pretty loose to me but I guess that's fairly standard for <$300 weather stations. If I graph the range versus the METAR, the curve I'm seeing makes sense. The sensor just may not be sensitive enough to pick up the extremes.

WD; Davis VP2 6153; Quickcam for Notebooks Pro; Boltek w/ Nexstorm; GRLevel3; Live NOAA radio

Offline niko

  • syzygy
  • Global Moderator
  • Posts: 22,157
  • Crystal Ball broken! Please post the URL.
  • Northern California, U.S.A.
    • http://www.realweatherstation.com
Data quality checking
« Reply #7 on: August 31, 2004, 02:05:14 PM »
According to the spec: Measuring Range Rel. Humidity : 20% to 95% (if the relative humidity is less than 20% or greater than 95%, it will display 19% or 96%)

Offline CaptDilli

  • Posts: 986
  • Little Switzerland NC USA
    • Little Switzerland Weather - NC - Butterfly Crossing
Data quality checking
« Reply #8 on: September 01, 2004, 10:39:40 AM »
Quote from: carterlake
......
I know my temp and barometeric pressure are spot on. But I wonder about my hydrometer. Unfortunately, I'm not currently wanting to take it down for the 6-8 hours it would take to test (using either the wet salt or wet towel calibration methods). After all, I don't know anything is wrong since the METAR is so much nearer to water.
Have you considered using a Sling Psychrometer to compare temperature and humidity readings with your WS-2310 readings? That would give you in idea of what error the WS-2310 has if any. You may find you do not have a significant measurable error in the humidity readings.
A difference of a few tenths of a degree between wet and dry bulb values can make a large deviation in humidity values.
Unless my humidity sensor is adjacent to the Weather Service sensor I would not be concerned with a variation in the humidity readings.
BC
System Info:
XP Home, SP 3, Firefox 3.5.7, IE-7, Pentium 4 2.26GH, 1024MB RAM, Desktop, ADSL, No screen saver, http://littleswitzerlandweather.info/bfcrealtime.htm
http://littleswitzerlandweather.info/live/index.html

Offline carterlake

  • Tom Chaplin
  • Posts: 2,273
  • Carter Lake, Iowa USA
    • Carter Lake, Iowa Weather
Data quality checking
« Reply #9 on: September 01, 2004, 10:58:32 AM »
Quote from: CaptDilli
Quote from: carterlake
......
Have you considered using a Sling Psychrometer to compare temperature and humidity readings with your WS-2310 readings? That would give you in idea of what error the WS-2310 has if any. You may find you do not have a significant measurable error in the humidity readings.
A difference of a few tenths of a degree between wet and dry bulb values can make a large deviation in humidity values.
Unless my humidity sensor is adjacent to the Weather Service sensor I would not be concerned with a variation in the humidity readings.


The couple of Sling Psychrometer's I just checked out in Google cost 1/2 as much as my weather station!

And like you say, there may be nothing wrong with my humidity sensor.

WD; Davis VP2 6153; Quickcam for Notebooks Pro; Boltek w/ Nexstorm; GRLevel3; Live NOAA radio

Offline CaptDilli

  • Posts: 986
  • Little Switzerland NC USA
    • Little Switzerland Weather - NC - Butterfly Crossing
Data quality checking
« Reply #10 on: September 01, 2004, 12:47:38 PM »
Did Google list
http://www.sciencekit.com/search.asp?t=ss&ss=WW621-3701&c=0&x=17&y=8

Taylor Sling Psychrometer     Taylor Sling Psychrometer - (6213701)

$89.00 Each   :wink:
BC
System Info:
XP Home, SP 3, Firefox 3.5.7, IE-7, Pentium 4 2.26GH, 1024MB RAM, Desktop, ADSL, No screen saver, http://littleswitzerlandweather.info/bfcrealtime.htm
http://littleswitzerlandweather.info/live/index.html

Offline niko

  • syzygy
  • Global Moderator
  • Posts: 22,157
  • Crystal Ball broken! Please post the URL.
  • Northern California, U.S.A.
    • http://www.realweatherstation.com
Data quality checking
« Reply #11 on: September 01, 2004, 08:38:00 PM »
You can usually find a selection on ebay too, but caveat emptor as always.

Offline carterlake

  • Tom Chaplin
  • Posts: 2,273
  • Carter Lake, Iowa USA
    • Carter Lake, Iowa Weather
Data quality checking
« Reply #12 on: September 01, 2004, 09:26:56 PM »
Quote from: nikoshepherd
You can usually find a selection on ebay too, but caveat emptor as always.


$11 is looking better.

Also, I updated the graphs with more days...

Notice the METAR is peaking higher temps on hot days and we're about even on the cooler days? The peak days are sunny and the even days are cloudy.

WD; Davis VP2 6153; Quickcam for Notebooks Pro; Boltek w/ Nexstorm; GRLevel3; Live NOAA radio

Offline carterlake

  • Tom Chaplin
  • Posts: 2,273
  • Carter Lake, Iowa USA
    • Carter Lake, Iowa Weather
Data quality checking
« Reply #13 on: September 20, 2004, 10:36:58 PM »
Just a followup note....

I found a AWS weather station less than a mile away and it compares very favorably to my station.

Example from today:

AWS ---------------------My Station
Temp low/high:
65.4 at 7:05a------------66.7 °F at 07:00 AM----(Metar: 66 at 06:52am)
84.6 at 4:18p------------84.7 °F at 3:23 PM------(Metar: 87 at 03:52pm)

Humdity low/high:
28.2 at 4:26p------------32% at 3:24 PM--------(Metar: 26 at 02:52pm)
65.1 at 6:15a------------63% at 05:51 AM------(Metar: 66 at 04:52am)

Pressure low/high:
29.89 at 5:01p-----------29.88 in. at 4:46 PM----(Metar: 29.85 at 04:52pm)
30.02 at 5:43a-----------30.02 in. at 05:32 AM--(Metar: 30.00 at 06:52am)

That really looks pretty darn good and well within the variance of sensitivity and slight location differences.

http://www.aws.com/aws_2001/asp/obsForecast.asp?id=OMAHA&obs=full

and

http://www.wrh.noaa.gov/cgi-bin/Missoula/msoobs?site=KOMA&type=1&src=lcl

WD; Davis VP2 6153; Quickcam for Notebooks Pro; Boltek w/ Nexstorm; GRLevel3; Live NOAA radio

Offline carterlake

  • Tom Chaplin
  • Posts: 2,273
  • Carter Lake, Iowa USA
    • Carter Lake, Iowa Weather
Re: Data quality checking
« Reply #14 on: October 24, 2004, 04:03:38 PM »
Just another update. I've been looking at the data for quite some time now.

This fall raised new concerns about how good my thermometer is functioning.

I added a fan to the thermo/hydro housing a few weeks ago and I think that has really paid off in quality of data.

Rarely am I getting emails from CWOP these days on temp/dew point variances.

Here is today and yesterday

I've come to accept that my temps are just a bit higher than the closest surrounding area sensors as they're very near the river.

The humidity is still lower than others during a rain but I now believe that is entirely due to the sensor's location in a sheltered spot. If it were out in the open where the ground was wet, I'm sure it would peg out at 96%.

Again, everything goes back to careful planning of the sensors. I highly recommend adding a fan even if you're getting good readings. It makes a big difference on days with no wind. I took a 12v computer fan and wired it up, but you can just as easily add a solar-powered fan for a few dollars.

WD; Davis VP2 6153; Quickcam for Notebooks Pro; Boltek w/ Nexstorm; GRLevel3; Live NOAA radio