cloudy cloudy

Author Topic: Software to use with WXSIM  (Read 1023 times)

0 Members and 1 Guest are viewing this topic.

Offline Asperitas

  • Posts: 255
  • OS/Browser:
  • Android 7.0
  • Chrome 71.0.3578.99
Software to use with WXSIM
« on: January 12, 2019, 07:45:21 PM »
Hi,

Is it possible to use WXSIM with Loggernet from Campbell Scientific?
https://www.campbellsci.com/loggernet

Im using WD but its missing weatherdata during the minutechange,  sometimes even 6 seconds, while Loggernet is running without missing weatherdata.
On a daily basis thats average more then an hour worth of data,  missing...

So i would like to know when i choose to buy the WXSIM Professional, of its running along with Loggernet.?
The datasource is in .dat format.
I really dont want to miss weatherdata and spend 249,00 while i know the datasource (from WD) isnt accurate.

With kindest regards. :D
« Last Edit: January 12, 2019, 08:00:16 PM by Asperitas »

Offline Tom Ehrensperger

  • Posts: 2,682
  • OS/Browser:
  • Win NT 10.0
  • Firefox 64.0
Re: Software to use with WXSIM
« Reply #1 on: January 13, 2019, 02:49:18 AM »
Thanks for your question!  I don't know why WD is missing data, but maybe Brian or someone here on the forum can address that (?).  WXSIM (through its WXSIMATE module) is set to read data from Weather Display, Cumulus, Virtual Weather Station, and WeatherLink.  I haven't seen loggernet before, but will take a look.  I don't know whether or not it's OK for you to have both WD and loggernet running, but some missing minutes of data will probably not affect WXSIM, as it can use averages over 10 minutes, for example.  I'm sure there will be some solution, but you're right to figure that out before buying! :)

Tom Ehrensperger (WXSIM developer)

Offline Asperitas

  • Posts: 255
  • OS/Browser:
  • Android 7.0
  • Chrome 71.0.3578.99
Re: Software to use with WXSIM
« Reply #2 on: January 13, 2019, 10:06:19 AM »
Hi Tom,

Thanks for your reply.
Since i dont get an answer on my questions for more then a month now,  i doubt this issue will be solved.
Ive always been polite and friendly in asking, i even tried some suggestions to try to solve,  but no response unfortunately....
Im missing many hours worth of valuable weatherdata over long periods this way, which isnt necessary and not what i want.

It really isnt rocketsience i think. But this way im forced to use Loggernet instead of WD. Which is ok for me.
So it really depends if WXSIM is the way to go for me if its working with Loggernet instead of WD.


With kind regards.
« Last Edit: January 13, 2019, 10:08:32 AM by Asperitas »

Offline oreitsma

  • Posts: 558
  • Assen, Netherlands
  • OS/Browser:
  • Win NT 10.0
  • Chrome 71.0.3578.98
    • Weerstation Assen
Re: Software to use with WXSIM
« Reply #3 on: January 13, 2019, 10:32:15 AM »
Hi Tom,

Thanks for your reply.
Since i dont get an answer on my questions for more then a month now,  i doubt this issue will be solved.
Ive always been polite and friendly in asking, i even tried some suggestions to try to solve,  but no response unfortunately....
Im missing many hours worth of valuable weatherdata over long periods this way, which isnt necessary and not what i want.

It really isnt rocketsience i think. But this way im forced to use Loggernet instead of WD. Which is ok for me.
So it really depends if WXSIM is the way to go for me if its working with Loggernet instead of WD.


With kind regards.

Hi Asperitas,

It is true that after minutechange WD stops counting the seconds for 0 to 10 seconds. It looks like time stops but it is not.
But when you look at the green LED indicating that data is being received during the minute change, it continues. So you do not miss any data at all. You can not see the time from 0 to about 10 seconds but it just continues. You also see during the moment that the time counter stops after a minute change that the data in WD just refreshes. You can also see in your logfile that data is changing right after minutechange.
So, in my opinion no problem at all.

Brgds,
Oebel

Offline Asperitas

  • Posts: 255
  • OS/Browser:
  • Win NT 10.0
  • Chrome 71.0.3578.98
Re: Software to use with WXSIM
« Reply #4 on: January 13, 2019, 11:34:18 AM »
Hi Oebel,

Thanks for your reply. :D
But youre wrong unfortunately :(
And the green led indicating that its getting data during that time isnt true, it stops during that period...
The data received count stops also and starts counting again where it stopped before the minutechange, so thats an indication its stops collecting data.
When the data received count stopped at example 1310, it starts counting after some seconds after the minutechange, at 1311...
I know its missing data because my rain totals in WD differ from the actual source (im running loggernet, the source, 24/7, updating every second flawless without delay and making graphs like in WD without the hassle im seeing in WD).
Ive watched it for some time and saw it actual missed raintips during the minutechange, which sometimes takes almost 6 seconds.
So i assume its missing more weatherdata during the minutechange besides the raintotals.
Having seen that, the actual 10 minute averages arent correct, because in a day worth of data its missing more then an hour of data, thats alot on daily/monthly/yearly basis.
Thats approx 360-400 hours of missing data 8O 8O 8O

Brian has made a check some time ago (when WD was running without Loggernet, but with cronpakbusnew.exe running) so during the minute change WD looked back a minute to check for missing data (60 excel files in WD folder).
He aslo mentioned WD is probably taking to long during that period. So theres something wrong here.
But now im running WD along with Loggernet so cronpakbusnew.exe isnt needed. ive asked if he could implement this again when cronpakbusnwe.exe isnt running, but i dont got an answer for more then a month now...
Or just resolve the minutechange delay by collecting and writing the data (polling like in Loggernet...about 100msec or faster...) at much faster interval.
This way the rainrate isnt affected either.

I know i have a weatherstation that isnt regular in use over here, probably thats why its no priority for Brian unfortunately :(

So all in all i dont want to use WD in combination with WXSIM as long as this isnt solved because of the above.
Again, this isnt rocketsience is it. :D

With the kindest regards.
« Last Edit: January 13, 2019, 11:58:34 AM by Asperitas »

Offline Tom Ehrensperger

  • Posts: 2,682
  • OS/Browser:
  • Win NT 10.0
  • Firefox 64.0
Re: Software to use with WXSIM
« Reply #5 on: January 13, 2019, 06:03:15 PM »
Does loggernet produce any plain text files of the data, and if so, can you send a sample?  Have you ever tried Cumulus?  Does it work with your station, and if so, does it have that missing data problem?  I'm pretty sure the missing data would not affect WXSIM.  Things like wind, temperature, and humidity are not going to be detectably different in any meaningful way if a few seconds of data were missing from each minute.  In fact, it wouldn't matter if half of the minutes themselves were missing.  It DOES sound like what you're saying would affect rainfall totals (I don't know that much about, and don't use WD, so I can't vouch for anything directly related to this data logging issue ... just to what does or doesn't affect WXSIM).  Past rainfall amounts being off by 10% would have only a vanishingly small (very much lost in the noise) effect on WXSIM's output, and then only if you have certain options checked.

Again, please send samples of any text (i.e. human-readable) files loggernet might produce, and let me know of any questions you might have.  The bottom line is that I can definitely understand your concerns about the missing data, for actual data archiving purposes (especially if that's actually throwing off rainfall amounts), but I'm pretty sure that would not affect WXSIM's forecasts in any detectable way.

Thanks again for your interest in this! :)

Tom

Offline Weather Display

  • Posts: 87,022
  • Davis VP2
  • New Zealand
  • OS/Browser:
  • Win NT 10.0
  • Chrome 71.0.3578.98
    • Weather Display
Re: Software to use with WXSIM
« Reply #6 on: January 13, 2019, 06:39:14 PM »
@Asperitas
I was not able to duplicate the period of no data here
using the settings you sent me and your input data file
so its not something that easy to fix when you can not duplicate it
(a more powerfull system might be hour solution)

Offline Asperitas

  • Posts: 255
  • OS/Browser:
  • Android 7.0
  • Chrome 71.0.3578.99
Re: Software to use with WXSIM
« Reply #7 on: January 13, 2019, 07:14:56 PM »
Hi Brian,

Thanks for your reply.
Ive mentioned earlier it has nothing to do with my pc.
You mentioned "maybe WD is taking too long" in a previous post...
Its an i5 processor with 8gb memory, running WD and Loggernet only.
CPU is 25%, memory used by WD:75 mb. Loggernet even less.
It really is something in WD which is causing the lag.
Its a bit short-sighted to say its working ok for you with my settingsfile etc.
Ive given you numerous suggestions on how to help you and me get this solved.
Im a bit dissappointed in WD at the moment to be honest, sorry :(
I really liked WD.

With the kindest regards.

Offline NorCal Dan

  • Dan - KJ6RGX
  • Posts: 10,532
  • Davis Vue/Win10
  • Ocean Park, WA.
  • OS/Browser:
  • Mac OS X 10.10.5
  • Chrome 71.0.3578.98
    • Traveling RV Weather
Re: Software to use with WXSIM
« Reply #8 on: January 13, 2019, 09:01:43 PM »
Sitting here watching my clock display in WD and the only time I see any lag is at the even minute change.  It appears that WD is updating graphs or log files every other minute and I do not see up to 3 seconds before it starts counting again.  Also, at the hour change over I see the clock lag again for about 7 seconds, I assume WD has more housekeeping that needs to be done.  Accessing the harddrive and writing data is the slow spot in the process and has little to do with computer horsepower.  Other items can also effect WD processing, ie. webcam, tide calculation, etc.

How often is your system writing to the log file?  My log file has an entry for each minute and I do not know if WD is capable of writing more than one entry per minute.  I'm trying to understand how your system is missing data.  Can you post a clip of your log file that shows where/when the data is missing?

Offline Asperitas

  • Posts: 255
  • OS/Browser:
  • Android 7.0
  • Chrome 71.0.3578.99
Re: Software to use with WXSIM
« Reply #9 on: January 13, 2019, 09:27:05 PM »
Hi Norcal Dan,

Thanks for your reply.  :D

Ive made a video just before the minutechange, as you can see it takes a while :(
No webcam, no tide calculation, no animated gifs etc.

https://youtu.be/ljmdikk7_pI

Offline Asperitas

  • Posts: 255
  • OS/Browser:
  • Android 7.0
  • Chrome 71.0.3578.99
Re: Software to use with WXSIM
« Reply #10 on: January 13, 2019, 09:56:04 PM »
...ill post further in the topic about it...

Offline Asperitas

  • Posts: 255
  • OS/Browser:
  • Android 7.0
  • Chrome 71.0.3578.99
Re: Software to use with WXSIM
« Reply #11 on: January 24, 2019, 06:50:02 AM »
Hi Tom,

Sorry for replying late...
Is it possible you show me an example in CSV or txt of what is needed to run WXSIM with Loggernet?
So the exact order of data and the file type to use.
Loggernet is far more customizable and i can change things without the need to ask the author.

With kind regards.
« Last Edit: January 24, 2019, 06:54:44 AM by Asperitas »

Offline Tom Ehrensperger

  • Posts: 2,682
  • OS/Browser:
  • Win NT 10.0
  • Firefox 64.0
Re: Software to use with WXSIM
« Reply #12 on: January 27, 2019, 07:24:11 PM »
WXSIM (actually the WXSIMATE module of it) can read either Weather Display's or Cumulus' text files.  For WD, they're the ones like 12019lg.txt and 12019vantagelog.txt (the second one may not be necessary unless you have solar data).  For Cumulus, they are the ones like Jan2019log.txt.  If the files look like those, WXSIMATRE can read them.

Hope that helps!

Tom

 

cumulus