MB Pro - cannot restore database from backup - not enough space *solved w/ build 10-Jul-2021*

This section covers the Meteobridge PRO, PRO2, NANO SD, Raspberry Pi and VM platforms exclusively

Moderator: Mattk

Post Reply
User avatar
Gyvate
Platinum Boarder
Platinum Boarder
Posts: 453
Joined: Thu May 14, 2020 4:36 pm
Location: Saarbrücken, Germany

MB Pro - cannot restore database from backup - not enough space *solved w/ build 10-Jul-2021*

Post by Gyvate »

I had run into two issues lately where MB complained of not having enough space available
1. rebuilding the database (forced rebuild)
2. restoring the database from a backup

1. has been solved by Boris with the update 14426 for MB Pro from 28-Jun-2021 - Thanks for the fast action !! :D
2. is still an open issue (which will earlier or later hit all MB Pro users - see below)

Background
My MB database is bigger than 500 MB meanwhile and the internal memory, where the database "sits" is 960 MB only in a MB Pro.
The rebuild script used the internal storage only and during the rebuild, the existing database is exported to a compressed file dbase.sql.gz and and new database dbase.new is built. Of course 2 * 500+ MB plus about 200 MB compressed export won't fit into 0.9 GB storage.
Boris corrected the script and offered the possibility to choose an external USB for the rebuild process in the system/administration page.
I tested it - and it worked (took 7 hours to export the 600 MB database and 6 hours to rebuild it, but it worked).

Now, I tested the restore and ran into the same issue - not enough storage available.
Obviously the database backup is copied into the internal storage where already (still) the old database resides. 600 + 600 > 960. :twisted:
The only way to make this work for me was to SSH into the MB, delete the dbase* files from the /tmp/mnt directory where they reside, reboot and then the backup would run.

I think a new approach is necessary as not every user wants to/can/knows how to work with SSH and possibly mess with the file system.
At least an option for databases whose size * 2 is bigger than the available internal storage.


A possibility I could think of would be an option to use the USB stick not only to copy the backup from but also for the whole process as a temporary or permanent swap file
e.g. copy the old database files (dbase*) onto the USB stick, then delete the dbase* files in the internal storage, copy the backup into the internal storage and do the necessary work (rename, restart, re-create the RAM-DB ....)
The old database could be kept on the USB drive/stick for those (experts) who want to do some research with them.

When the size of the backup file plus the old dbase files surpasses the available internal storage and a restore is initiated, an error message should be posted (log file) telling the user e.g. "not enough internal memory for database restore, please select USB swap option [to be created] and make sure your external USB storage has double the size of your backup file free storage available" - something like this

Any other working solution is of course also welcome.
Last edited by Gyvate on Sat Jul 10, 2021 2:02 pm, edited 2 times in total.
WH4000SE 1.6.6/1 x DP1500/4 x GW1000 1.7.7/GW1100 2.3.0/HP1000SE Pro 1.9.3//2 x WH2650 1.7.7/GW2000 3.1.0
2xMeteobridge Pro [B+R] 15161, 2xRPi4B-2GB/16/32 3139,VM128 1704
Weather Landing page: https://meshka.eu
Ecowitt WiKi: https://meshka.eu/Ecowitt/dokuwiki
User avatar
admin
Platinum Boarder
Platinum Boarder
Posts: 7854
Joined: Mon Oct 01, 2007 10:51 pm

Re: MB Pro - cannot restore database from backup - not enough space

Post by admin »

For rebuilds the situation has been solved:
Most current version of Meteobridge does allow for Meteobridge PRO and RPI to use an external USB stick as temporary storage when rebuilding the database. There is a new option for that on "System - Administration" tab in the rebuild selection drop-down.

On DB restores Meteobridge pays save and copies restored database first before deleting the old one. I will provide an option to have the old one deleted first.
User avatar
admin
Platinum Boarder
Platinum Boarder
Posts: 7854
Joined: Mon Oct 01, 2007 10:51 pm

Re: MB Pro - cannot restore database from backup - not enough space

Post by admin »

I just released an update that allows to delete the database first before replacing by one from a backup. This is controlled via a checkbox. Please see details here: https://meteobridge.com/wiki/index.php/Forum
Post Reply