Data loss after power failure **solved**

This section covers the Meteobridge PRO, PRO2, NANO SD, Raspberry Pi and VM platforms exclusively

Moderator: Mattk

Post Reply
skuske
Fresh Boarder
Fresh Boarder
Posts: 7
Joined: Mon Jan 13, 2020 3:49 pm

Data loss after power failure **solved**

Post by skuske »

I had to disconnect the Meteobridge PRO from power, and after powering it on again I noticed that data of 5 hours are lost. See attached screenshot:

dataloss.png
dataloss.png (73.97 KiB) Viewed 2551 times

Disconnecting from power happened at 12:07pm, but data between 7:07 am and 12:07 pm was lost, although in station settings, the time interval to save historical data is set to 30 minutes.

This happens whenever power is lost. Can somebody explain this and offer a solution? Many thanks in advance!
skuske
Fresh Boarder
Fresh Boarder
Posts: 7
Joined: Mon Jan 13, 2020 3:49 pm

Re: Data loss after power failure

Post by skuske »

I tested this and removed power from my MB Pro, and this time data of about one hour got lost. Before I removed power, I created a backup of the database, but restoring the database actually did not restore the data that got lost. That means to me that the data that was lost actually never existed in the database, or it did not exist in a proper format, or whatever. I have no clue.

So, whenever the MB looses power, more or less data is lost. This totally unacceptable and I would like to know what I can do to fix that (if there is anything I can do). Maybe this is even a bug within the system. In any case, I can reproduce this problem in a targeted manner: with every power cut, the MB loses data, either for several hours or for a shorter period of time.

I also found that it is obviously not just me seeing this: viewtopic.php?f=61&t=13421

Thanks for any help.
dataloss-2.png
dataloss-2.png (90.99 KiB) Viewed 2542 times
skuske
Fresh Boarder
Fresh Boarder
Posts: 7
Joined: Mon Jan 13, 2020 3:49 pm

Re: Data loss after power failure

Post by skuske »

I must mention that each of these power offs were for a maximum of 10 seconds only!
skuske
Fresh Boarder
Fresh Boarder
Posts: 7
Joined: Mon Jan 13, 2020 3:49 pm

Re: Data loss after power failure

Post by skuske »

I did some more research and have now found out that on the 'Historical data' tab, data from the missing time span seems to be there:
hist.png
hist.png (131.43 KiB) Viewed 2529 times

But when I create a chart for the same day, data is not there. There's something fundamentally wrong here, either with the database, or with the chart generator.

graph.png
graph.png (187.03 KiB) Viewed 2529 times
User avatar
admin
Platinum Boarder
Platinum Boarder
Posts: 7854
Joined: Mon Oct 01, 2007 10:51 pm

Re: Data loss after power failure

Post by admin »

Per minute data is only kept in a RAM database. When a proper shutdown happens RAM database is stored permanently and restored from that on reboot. But if you have a hard shutdown (like power fail) this gap will appear.

So no bug, but to be expected behavior.
skuske
Fresh Boarder
Fresh Boarder
Posts: 7
Joined: Mon Jan 13, 2020 3:49 pm

Re: Data loss after power failure **solved**

Post by skuske »

Why is the RAM database not saved every 15 minutes or on a freely definable basis so that data loss of several hours can be prevented? If a data loss of several hours is 'expected behavior' here, then I wonder why the database is not saved more frequently to avoid situations like this...

I also tried to restore the data from Meteoplug to Meteobridge, but that does not help either: Meteobridge says it restored (no errors), but although the missing data is on Meteoplug and within the export file, Meteobridge does not restore the missing data from the Meteoplug export.

Many thanks.
Mattk
Platinum Boarder
Platinum Boarder
Posts: 1387
Joined: Mon Sep 22, 2014 3:24 am

Re: Data loss after power failure **solved**

Post by Mattk »

If you are serious then you would have UPS and backup power capable of supporting the worst case scenario. There is no point running a station from a power supply that when it stops the station basically stops as well, in a no power situation then no data anyway.
User avatar
admin
Platinum Boarder
Platinum Boarder
Posts: 7854
Joined: Mon Oct 01, 2007 10:51 pm

Re: Data loss after power failure **solved**

Post by admin »

skuske wrote: Sun Jan 19, 2020 8:06 am also tried to restore the data from Meteoplug to Meteobridge, but that does not help either: Meteobridge says it restored (no errors), but although the missing data is on Meteoplug and within the export file, Meteobridge does not restore the missing data from the Meteoplug export.
Meteobridge imports data into its persistent database which operates on hourly/daily/monthly/yearly level, not into the non-persistent per minute RAM database. Therefore, you don't see any resurrection of the lost per minute data.

When you solve the hardware problem, this issue will vanish as well. As written, I am fine to repair the flumsy power connector when you send it to me. using the other micro USB connector for powering the system can be used as a work around until then.
Post Reply