[SOLVED] Clientrawextra invalid info

This morning looking at my charts I see we had a major event that I must have slept through… :slight_smile:

We had 1.16 Inches of Rain along with a temp spike of 140

I’m seeing the same thing as before tonight.

#21 of the data is wrong…

`[b]566 Temp21 C 57

Based on the way it’s parsed out in your post, it looks to me like the whole array is shifted, rain21 is a temp value, baro21 is a rain value, temp21 is something else, rather than individual values being incorrect. Sorry if that’s what you meant but it didn’t come over that way.

Not sure what the pattern is, but it appears to get worse.
The clientrawextra data being put into the extra values of 21-24 is off.

It appears to be interrmittent, but happens several times during the day.

562 Windspeed 21 K kts = 0 mph = 0 km/h
563 Windspeed 22 K 0 kts = 0 mph = 0 km/h
564 Windspeed 23 K kts = 0 mph = 0 km/h
565 Windspeed 24 K 0 kts = 0 mph = 0 km/h

No major cool down taking place …

[b]566 Temp21 C


This really shows up in clientraw driven stuff like charts.

I tried restarting WD, thinking that may clear it up, but after the clientrawextra uploads, it still has invalid data.

From the raw logs, you can see those numbers simply are not there…

6 9 2006 8 4 84.7 51 64.6 30.006 1 2 116 0.000 0.000 0.531 6.807 86.3
6 9 2006 8 5 84.5 52 64.9 30.006 0 1 112 0.000 0.000 0.531 6.807 86.3
6 9 2006 8 6 84.4 51 64.3 30.009 0 2 109 0.000 0.000 0.531 6.807 85.9
6 9 2006 8 7 84.4 52 64.8 30.008 0 1 109 0.000 0.000 0.531 6.807 86.0
6 9 2006 8 8 84.5 53 65.5 30.005 0 1 109 0.000 0.000 0.531 6.807 86.6
6 9 2006 8 9 84.8 54 66.3 30.006 2 3 99 0.000 0.000 0.531 6.807 87.2
6 9 2006 8 10 84.8 53 65.8 30.007 2 3 92 0.000 0.000 0.531 6.807 86.9
6 9 2006 8 11 84.7 53 65.7 30.007 1 2 99 0.000 0.000 0.531 6.807 86.8
6 9 2006 8 12 84.7 52 65.1 30.007 3 3 138 0.000 0.000 0.531 6.807 86.5
6 9 2006 8 13 84.8 51 64.7 30.007 1 3 146 0.000 0.000 0.531 6.807 86.4
6 9 2006 8 14 84.9 52 65.3 30.006 2 3 148 0.000 0.000 0.531 6.807 86.7
6 9 2006 8 15 85.0 52 65.4 30.006 1 2 162 0.000 0.000 0.531 6.807 87.0
6 9 2006 8 16 85.2 51 65.0 30.008 2 3 158 0.000 0.000 0.531 6.807 87.0
6 9 2006 8 17 85.2 51 65.1 30.009 2 3 146 0.000 0.000 0.531 6.807 87.0
6 9 2006 8 18 85.3 51 65.1 30.009 2 3 164 0.000 0.000 0.531 6.807 87.2
6 9 2006 8 19 85.4 50 64.7 30.010 3 7 160 0.000 0.000 0.531 6.807 87.0
6 9 2006 8 20 85.5 50 64.7 30.010 2 3 103 0.000 0.000 0.531 6.807 87.2
6 9 2006 8 21 85.4 50 64.6 30.009 2 5 116 0.000 0.000 0.531 6.807 87.0
6 9 2006 8 22 85.4 50 64.6 30.010 1 3 103 0.000 0.000 0.531 6.807 87.0
6 9 2006 8 23 85.4 50 64.6 30.009 3 5 137 0.000 0.000 0.531 6.807 87.0
6 9 2006 8 24 85.4 50 64.6 30.009 3 6 159 0.000 0.000 0.531 6.807 87.0
6 9 2006 8 25 85.5 50 64.7 30.009 1 3 140 0.000 0.000 0.531 6.807 87.2
6 9 2006 8 26 85.6 50 64.8 30.009 0 2 144 0.000 0.000 0.531 6.807 87.3


I loaded the newest version of WD 10.35W[actually it is X] Sept 6 8am version.

And after waiting for the clientrawextra to load it…

Same thing.

I’m not the only one having this issue.

I looked around at others that have done recent updates and see the same issues. Most of them may not realize it yet, but if they run WDL and look at their charts for Temp, Baro, Rain you will see the same spikes.

The last site I checked was: http://quickskys.com/wdl/

Yup, spikes and dips are there. I don’t look at the graphs that often. Now to see if my all time records are showing the same. Thanks for pointing it out.

–Dave

This only seems to be effecting the output of clientrawextra and does not appear to be anywhere else in the system.

I have a slightly earlier 10.35x and mine (I think) is OK so it must be something recent which caused it.

Stuart

It appears to be time related. I’ve seen times like last night, where it was okay.

I am running a 10.35x from about 10am this morning (a download from Brian) and cannot see any spikes in the WDL graphs at this time. Will monitor.

Let me try again. I don’t think this is a case of bad data being stuffed into data elements, I do think the problem is one or more missing or additional elements (spaces?) in the string and that is causing wrong values to be read when the string is parsed. It’s way too coincidental that 566 would be a good value for 565, 570 for 569 etc.

Will have to check that the next time it happesn. Right now, it looks okay.

nikoshepherd is on to something…

Found a site that is still doing it. and Found it…

there is an extra space being output by WD in the clientrawextra.txt file

It is hard to see since the parser doesn’t show the extra white space… but when I did a dump from a site that had the issue and looked at it with a binary editor, it showed up with a double space when only one space should be there.

Example:

The raw data in the parser looks like:

98 13 2 13 46

but in reality, it is:

98 13 2 13 46

An extra space between the 13 and the 2.

My parser and apparently the one used WDL see a second space as an end of value = 0

Which results in the skipped field with a 0 in it:

560 Moon Phase P 98 % 561 Moon Age L 13 [b]562 Windspeed 21 K kts = 0 mph = 0 km/h[/b] 563 Windspeed 22 K 2 kts = 2.3 mph = 3.7 km/h 564 Windspeed 23 K 13 kts = 14.95 mph = 24.08 km/h 565 Windspeed 24 K 46 kts = 52.9 mph = 85.19 km/h

This then shifts all the rest of the values down by 1 causing weird results.

There must be something in WD that on occassions adds an extra space.

I’ve not seen enough of them yet to know if it does it in the same place each time though.

Just thinking out loud but it is very common to see extra spaces in many of the text logs created by WD as well. This does not seem to cause a problem in WD’s use of these logs, in fact in my log checker script I specifically edit out double spaces because it happened so often and did not seem to cause any problems with WD’s use of the files. Perhaps this is related in some way to some general problem where the white space is not controlled properly?

Stuart

According to the specification, a space is the delimiter. Two spaces indicate null data and is most likely an error since a null value would most likely never be desired.

No spaces are allowed in the data which is why fields like the forecast and station names have _ underscores between the words.

MOSTLY_SUNNY_IN_THE_MORNING...BECOMING_PARTLY_SUNNY._HIGHS_96_TO_101._SOUTHEAST_WIND_5_TO_15_MPH_IN_THE_ MORNING..._BECOMING_SOUTH_IN_THE_AFTERNOON._

WDL treats the clientraw files the same whay which is why now, when it happens, its charts have the same exact errors in them.

I could easily strip extra spaces from the clientraw data before using, but WDL doesn’t do that and none of the clientraw generated web pages do that so that would not really fix the issue.

This appears to be fairly new and is somewhat random (comes and goes) but the fix would be in WD when it generates the files and then all existing utilites would be fine.

I’ve only seen the issue in clientrawextra so far.

in fact in my log checker script I specifically edit out double spaces because it happened so often and did not seem to cause any problems with WD's use of the files.

Note that these are not log files, but data files which are parsed with the assumption that a space is a delimiter. I think log files are treated differently.

ok, i have changed the way the data is written out, and now should not have any extra spaces
use this update
http://www.weather-display.com/downloadfiles/WeatherD.zip

also, with this new 31 day data file, i will be able to use that for the 31 day data in WDL, and so avoid problems with that, i.e with trying to marry together 2 month stamped data files :wink:

Cool,

I did the upgrade and so far it looks fine.

Not sure I understand what you mean by the “new 31 day data file” though.

i have been working on a new , always last 31 day data file :wink:
(now in use in the latest 10.35x update)