meteohub.de

Hi,

I'm not sure when this started to happen, but I suspect recently as I found it while looking for the reason behind something I noticed in WD, but it seems that the "most recent sensor" call of the logging protocol is now serving up the same sensor over and over again for something like a minute at a time even though many more sensors are reporting in.

I can reproduce the behaviour with both my Meteohubs (both on 4.5d (Build 313)) using just a basic browser call of:

http://meheoub-ip/meteolog.cgi?mode=data

This is causing problems in WD because some data is only "seen" by WD in the "most recent sensor" call. My barometer wasn't "seen" for almost 3 hours at one point this morning. More worrying is that rainfall is only "seen" from the "most recent sensor" call.

Is this reproducible elsewhere?

skyewright wrote:More worrying is that rainfall is only "seen" from the "most recent sensor" call.

We've had 5.25mm of rain since midnight according to my WMR928 and Meteohub. So far (and it's over 2 hours since the ran stopped)WD has only 'seen' 1.75mm - and all of that in one 'lump'.

I may try reverting to an earlier version of Meteohub to see if that helps...

It would be helpful if someone using v4.5d could try the

http://meheoub-ip/meteolog.cgi?mode=data

call with their browser to see if the effect (i.e. repeated reporting of the same sensor) is local or elsewhere too.

The only change here aside from update of Meteohub has been the introduction of a VP2 - that does issue a lot more sensor readings but System Load on the Meteohub's is nothing dramatic.

I'm on 4.5e and wind0 has been the only sensor for ~5 hours now...

rune wrote:I'm on 4.5d and wind0 has been the only sensor for ~5 hours now...

I tried updating to 4.5e. That was just the same. :(


I've now reproduced the problem on a totally fresh and independent system too.

I downloaded and installed the VMware Meteohub, then ran that with just USB connection to my WMR200 console. This is a very lightly loaded system (Syetem Load typically < 0.1).

It's showing exactly the same behaviour, i.e. the "most recent sensor"/"not time range" call keeps reporting the same reading long after other sensors readings have been recieved.

For anyone using the logging protocol with WD this is a serious issue, and not just for recording of rain if the Meteohub's time happens to be even just a little behind time on the WD PC.

I made changes to the logging processes to reduce write operations to flash memory. Meteohub now buffers readings internally in RAM and just writes every couple of minutes all the logged data to the "raw" file. Meteohub http data logging protocol makes use of the data written into the raw file. So the behavior you describe is in sync what I would expect.

When changing the implementation that way I had not in mind that WD's kind of using the http-data logger might have a problem with this. As Brian needed quite some iterations to get a working connection to Meteohub's http data logging as is, I doubt that he will be able to make his logging mechanism compatible to this any time soon.

What I could do, is to make the data logging interval user selectable. Setting it to a low value (10 secs i.e.) will make every data logged immediately available in the raw file, but will also put a lot of stress to the flash storage with increased probability of failure. By making this a choice of the user, everyone can decide what fits best.

Any other ideas are also welcome.

A bit off topic and perhaps not. Have you considered the possibility of implementing data storage "off" the Meteohub? If it was selectable to put all logs, graphs etc. on an external server, then storage wear on the flash wouldn't be an issue for those having external webservers etc.

admin wrote:Any other ideas are also welcome.

It just so happens that I did have an idea. You have mail...

admin wrote:...buffers readings internally in RAM and just writes every couple of minutes all the logged data to the "raw" file. Meteohub http data logging protocol makes use of the data written into the raw file. So the behavior you describe is in sync what I would expect.

I've just realised that that also makes sense of something else I'd seen in the WD 'debug' view over the last couple of days, i.e. sometimes the "all sensors in last minute" was returning blank. Presumably that was because at that point none of the readings from that minute had been written to raw?

Sorry I didn't spot this earlier. I usually keep my fallback/dev Meteohub right up to date (it's unusual I hadn't picked up 4.5e straight away), but that one doesn't regularly feed WD. I tend not to update the 'live' Meteohub, which feeds WD 24/7, so often. Late last week I updated it from a release that was I think just before you optimised the writes (4.4d or e I think?).