Possible problem with VVP and WD on startup

Steve could you please review this thread and let us know if you feel VVP could be implicated in this problem, or if there is anyway to trace this from a VVP perspective to aid diagnosis.

Thanks Stuart

Steve please review this thread as I am still having this problem. I’d appreciate your advice on anything I can do to determine if this is a VVP or a WD problem.

Stuart

I have done some work on this issue. I see the same behavior in my own setup. But I haven’t found the cause. There’s a very good possibility that VVP is causing the problem, but I don’t know if that’s the case. WD downloads the entire archive, but WeatherLink doesn’t. That points to WD. But WD doesn’t download the entire archive when connected directly to the console. That points to VVP. I’ll look at it more today and hopefully I’ll find something.

Steve

Steve thanks for looking at this, glad I’m not the only one seeing it.

I’ve been thinking about this and I know that if WD talks directly to the logger getting the archive the logger does not record any more records while WD is doing this. However I thought that using VVP WD talked to VVP which read the data from the archive it maintains, therefore I assumed the logger would still record data and VVP would still be adding data to the archive. So I had expected WD to 'see’archive records created during the time it was reading. Now when this problem happened I still saw a gap of a few minutes in the data recorded by WD which means that VVP did not pass any of this extra data to WD, to explain a little better

WD starts reading archive at 13:15 looking for data from 13:13 & 13:14
WD takes a few minutes to scan the archive and eventually gets it 2 records
WD then carries on creating its 1 minute data records from 13:23

so I have a gap with no data from 13:15 to 13:22 - 7 minutes missed.

I guess what I’m saying is what should WD see from VVP in these circumstances?

Stuart

The console tells you how many records you will get. You read in a loop for that count. So regardless of whether VVP or the console is sending the data, you’d still have that gap. The gap isn’t because the console stops writing to the archive data. The gap is because of the way the get archive data command works.

Steve thanks for that explanation. However I think this means we do need to get to the cause of this. If there is anything I can do to assist in getting diagnostics please let me know.

Stuart

I think he is saying there is no “fix”.

I think what he is saying is that is how Davis built the command structure. He can’t change how that works.

VirtualVP has to work like the console does otherwise it won’t interface with other program properly because they expect it to work the way it does.

I can’t do anything about the gap, but I might be able to do something about WD downloading the entire archive when running with VVP.

Steve

Yes I realise that Kevin. Maybe I was not clear enough in my reply. AS Steve says we need to find out why WD is downloading the whole archive when it only needs the last few minutes and fix that!

Stuart

Just thinking out loud some more about this problem. I think it might be happening when I reboot and everything starts up form scratch. On my old WD pc I used WDStartdelay to give VVP 30 secs to come up. Now I use Startwatch on my new PC and start WD when the ports on the local copy of VVP are ready so I dont think its a problem with WD starting too soon. This problem happened on my old PC (new one been going less than 5 days) so I dont think the start config is the problem but I am wondering if it might be something to do with VVP startup after a reboot.

Stuart

That’s a good idea of something to check out. I can test it easily by shutting down WD for a little while but keeping VVP running. Then start WD back up. This would rule out something having to do with the state of VVP’s archive cache right after VVP starts up.

After doing some tests, I did note something. If the console has an archive interval >1, WD still sends the archive request command with a precision of 1 minute. This means that unless you get lucky and WD just happens to send the request with a time that jives with the console’s interval, then the console will return a result that the requested time couldn’t be found in the archive. The next step after that is downloading the entire archive since a point within the archive wasn’t found. I did a test with my VP1 test console, which has its archive interval set to 10 minutes. Upon restarting, WD would download the entire archive, whether it was hooked up to VVP or directly to the console.

I’m now changing the archive interval on my test console to 1 minute. After some archive data has built up, I will do more tests.

VirtualVP was written to duplicate the behavior of the console. It would have been nice if the archive command (DMPAFT) had been written to return the position of the archive record that was >= the requested time instead of only = to the request time (and an error code if that exact time wasn’t found).

While I was at it I did the same test with WeatherLink. With an archive interval of 10 minutes, WL always sent the DMPAFT command with times on 10 minute boundaries. Also, interesting, it requested a time about 5 records prior to the last request it had made. In other words, it got data starting a little older than it needed to, presumably as an extra measure to prevent gaps in its data (WL’s archive file - the .wlk file - is built directly from the downloaded archive data).

I’m curious if the people seeing the WD full archive download issue have their consoles set to other than 1 minute intervals, or if those with 1 minute intervals are seeing it too.

Steve

you can set the interval in wd in the data logger setup
if 5 minutes, then wd rounds off the time request to the nearest 5 minutes

Having made lots of other changes, I forgot to change WD’s data logger settings. I’ll do that and re-run the 10 minute archive interval test.

Steve

Steve I have my console (and WD) set to 1 minute intervals and always have had and since I was the one starting this thread off yes I do see the problem :wink:

Stuart

Edit: I just had to restart WD (without a reboot) for a non-wd related problem and it started just fine picking up the couple of minutes it needed. I’m fairly sure it only happens after a reboot. So could it be something to do with the very first communication with VVP after it starts?

Thanks. That’s good information that will help me zero in on this.

After more testing I’ve narrowed things down sufficiently to be able to say pretty conclusively that it is a bug in VirtualVP. Although I haven’t found the exact line yet that is the culprit, the problem is in the code that initializes VVP’s archive cache when VVP starts. It appears that VVP is reading one page of archive shy of what it should, so the cache is starting out missing from 1 to 5 minutes of data (assuming a 1 minute interval). There are 5 archive records per page. Archive data is retrieved from the console by page. This missing data would get pulled in within a minute, when the console writes its next archive record, causing VVP to pull in the current archive page. Broadstairs, you could do a little test by delaying the startup of WD for 2 minutes after VVP is ready to serve data.

It shouldn’t take much longer to find and fix the problem, but I have to tidy up some other things I was tinkering with before I can release it. I’ll try to get it out this week.

Steve
SoftWx

Steve thanks for all your work on this. I’ll do as you suggest and delay the start of WD while you sort it out.

Stuart

Steve I was wondering if you have got anywhere yet with this bug?

I have delayed WD’s start as suggested and so far have not seen the problem of reading the whole logger, so it very much looks like your theory is correct.

It would be nice to be able to remove the extra delay however.

Stuart

I’m still working on it, but I’m very close to having it ready. I will email you with a beta version for you to try before I release it.

Good to hear that the delay worked. That makes me confident I’ve found the problem.

Steve