The more I think about this the more I feel that in order to get somewhere we need some way to log the raw data coming in, not just as currently is available where only the latest is captured but we need it running to a file which is appended to, ideally not as currently formatted but as one line per record from the logger and time stamped. That way it would be clear whether or not the raw data was bad.
Stuart