Weather Network Backfill

[Rainman32]

A feature request for the ability to "backfill" missing data from Weather Network uploads. Say Internet was down for a day but everything was logged on the Meteohub just fine.. a manual selection to upload all data from mm:dd:hh-mm:dd:hh. Specifically for Weather Underground as the functionality provided in the WeatherLink add-on.

Thanks!

[DACscouter]

I would like to second this request. I administer two weather stations at remote scout camps. One is at 9,000 feet in the Rocky Mountains and during snow events the internet connection (satellite-based) is interrupted. The weather station console (Davis Vantage Pro2) and the meteohub are on a 48-hour UPS so the local network stays up. If the hub could detect HTTP GET failures and cache the data (perhaps as a fully formed HTTP GET?) until a HTTP GET success (then flush the cached data) that would fantastic.

We use Weather Underground and WeatherBug.

As a short term alternative, is there a way to extract the data and send it properly to these services?

Thank you!

Jim R
Denver Area Council, Boy Scouts of America

[bsparks]

I don't know if he's still around on the forums, but at one point Chris had come up with something that did this for Weather Underground. I successfully tested it during a hurricane (my station is in Louisiana).

An update to Meteohub kept this from working, though. This, paired with the new (and much-requested) ability to read from the Vantage's internal datalogger, really would make Meteohub phenomenal!

A huge thank-you is in order for Boris getting Meteohub to pull from the Davis datalogger. I just saw that today when I looked at my station's logfiles!

[crm0922]

Man, I guess my ears were ringing and that prompted me to log in to this forum after probably 2 years. I think my last post said something about buying a house and fixing it up...lol well, 2 years later...almost done.

I haven't touched my weather station in these 2 years either, although it has been working. I never did fix my script although I had it all worked out how it was going to handle concurrency/locking issues and be able to update the web piece-by-piece until the local cache was expired.

The problem is my weather station is 3000 miles away on a satellite link, so the best time to work on it is when I'm there in person, which is somewhat rare these days.

Can someone (Branson) inform me as to what new features meteohub has that are related to this "wish"...something about meteohub being able to pull old data from the internal data logger now? How is this handled? Is it a manual fetching process or does it try to fill in the blanks automatically or something slick like that?

Please forgive my ignorance of recent updates, the last time I logged into the shell of my NSLU2 was literally more than 2 years ago! lol...

Chris

PS - does meteohub run on x86 now? Does that mean I can run it in a VM and write a script locally? Hey that would be neat...where would I get my weather data from...I think Boris can help me redirect it from my NSLU2 over the interwebs...hmmm...

[bsparks]

Hi there, Chris!

You and Boris need to collaborate on this! I'm pretty sure that y'all could make this come to fruition.

I'm still running on NSLU2, but I know that Meteohub can run on x86. I think you could do a VM.

Uncertain how exactly it pulls data from the internal logger, but I know it does. You can initialize the logger to save data every 1 minute, 5 minutes, 10 minutes, and on up. I have mine set for 10 minutes. Obviously, the more frequent the logging, the less time that can be logged. The logging frequency/logging time table is in some of the documentation of the Pro2 and I can get it if you'd like to see it. The process is pretty slick. Here's a snapshot from my log, showing a time when the logger IS polled and when it ISN'T polled (both in red). Apparently ~10 minutes is the magic number (even if your logging interval is something higher or lower):

logger (22.06.2011 20:52:19): data logger (version 4.7j, build 1132) started.
logger (22.06.2011 20:52:23): connect station 0 (Vantage via USB serial).
logger (22.06.2011 20:52:23): station 0 (Vantage), wakeup sent.
logger (22.06.2011 20:52:23): station 0 (Vantage), init sequence sent.
logger (22.06.2011 20:52:23): station 0 (Vantage), internal logger activated.
logger (22.06.2011 20:52:23): station 0 (Vantage), syncing.
logger (22.06.2011 20:52:23): station 0 (Vantage), gettime request sent.
logger (22.06.2011 20:53:35): station 0 (Vantage), gettime request sent.
logger (22.06.2011 20:53:45): station 0 (Vantage), gettime request sent.
logger (22.06.2011 20:53:45): station 0 (Vantage), time offset of Vantage: -2 minutes
logger (22.06.2011 20:53:45): station 0 (Vantage), eeprom request sent.
logger (22.06.2011 20:53:45): station 0 (Vantage), rain collector size 0.01 inch per tick.
logger (22.06.2011 20:53:45): station 0 (Vantage), transmitter 1 active as station type (#0) "ISS".
logger (22.06.2011 20:53:45): station 0 (Vantage), no reading of Vantage's internal logger as data gap is less than 10 minutes: 283 secs
wmr928fulleval(23.06.2011 23:57:01): recomputation of weather data finished: 25625092 records processed in 98060 seconds (261 records per second)
logger (26.06.2011 07:48:58): parent process received termination signal (15).
logger (26.06.2011 07:48:58): child process received termination signal (15).
logger (26.06.2011 08:41:41): data logger (version 4.7j, build 1132) started.
logger (26.06.2011 08:41:44): connect station 0 (Vantage via USB serial).
logger (26.06.2011 08:41:44): station 0 (Vantage), wakeup sent.
logger (26.06.2011 08:41:44): station 0 (Vantage), internal logger activated.
logger (26.06.2011 08:41:44): station 0 (Vantage), syncing.
logger (26.06.2011 08:41:44): station 0 (Vantage), gettime request sent.
logger (26.06.2011 08:41:45): station 0 (Vantage), time offset of Vantage: -2 minutes
logger (26.06.2011 08:41:45): station 0 (Vantage), eeprom request sent.
logger (26.06.2011 08:41:45): station 0 (Vantage), rain collector size 0.01 inch per tick.
logger (26.06.2011 08:41:45): station 0 (Vantage), transmitter 1 active as station type (#0) "ISS".
logger (26.06.2011 08:41:50): station 0 (Vantage), data gap of 53 minutes, start reading Vantage's internal logger after 2011-06-26 07:45 (Vantage's local time).
logger (26.06.2011 08:41:52): station 0 (Vantage), internal logger provides 513 pages of data.
logger (26.06.2011 08:43:10): station 0 (Vantage), finished reading internal logger, 5 records inserted.
wmr928fulleval(26.06.2011 08:45:01): recomputation of weather data started.
wmr928fulleval(27.06.2011 07:50:52): recomputation of weather data finished: 25689896 records processed in 82975 seconds (309 records per second)

Isn't the stability of Meteohub and the NSLU2 amazing? 2 years. Wow!

Now, as soon as I say that, my NSLU2 is going to hit the skids. Just cloned and swapped the flash drive.

Good to see you again, Chris!

--Branson