Page 1 of 1

Options when running out of storage space

Posted: Wed Jul 04, 2012 2:21 am
by roller
Hi all,

I have been running the excellent meteohub software since Nov 2007.

i currently have the alix 1d and my 4gb CF card is 89% full.

What are my best options for more space - delete the older data to make room or get a new bigger (expensive) CF card. How easy is it to expand the data partition of the 4gb software image and replace the CF card - are there also licensing issues?

Perhaps as this is going to be a common problem for a lot of users going forward it might be helpful to either produce an 8gb (or larger) partition image or create an archive function as part of the software maintenance options?

Regards


Richard


http://WWW.elyweather.org.uk

Re: Options when running out of storage space

Posted: Wed Jul 04, 2012 9:15 am
by YJB
Hi,

The easiest is delete old data, but it will imply that you will need to run a recomputation after the delete, and you will loose all your "statistics" pertaining to the deleted data.

Running an 8GB flash card is an option, my setup is based on 8GB in an Alix1D. It's basically doing a basic setup on a new 8GB card, grow the /data filesystem to the max and then restore old your data from your 4GB card. This implies that you will need a temporary "landing space" for the data on your 4GB card.

This topic describes the steps to grow your filesystem. I added the instructions for someone who needed this for VMware, but this was based on my effort to grow the filesystem on my Alix.

viewtopic.php?f=8&t=8172

Hope this helps, otherwise let us know.

Ysbrand

Re: Options when running out of storage space

Posted: Wed Jul 04, 2012 9:49 pm
by wfpost
for all the folks who do not want to play around with dozens of linux commands

If you have a Linux box running - GParted is your friend.
If not:
You can download a GParted Live CD and extend the storage space with a simple GUI

http://gparted.sourceforge.net/livecd.php

Image

Image

Image

Re: Options when running out of storage space

Posted: Thu Jul 05, 2012 7:44 pm
by skyewright
roller wrote:i currently have the alix 1d and my 4gb CF card is 89% full.

What are my best options for more space
My 4GB Alix-1d gets that full with not much more than a year's worth of data (I have quite a lot of sensors)!
My solution is to prune the data once a year, having first made copies. If I want to report on longer periods I use the Meteoplug based version of the data.

Horses for courses. What suits me may not be right for you. :)

Re: Options when running out of storage space

Posted: Tue Aug 07, 2012 9:57 am
by oliverb
skyewright wrote: My 4GB Alix-1d gets that full with not much more than a year's worth of data (I have quite a lot of sensors)!
My solution is to prune the data once a year, having first made copies. If I want to report on longer periods I use the Meteoplug based version of the data.

Horses for courses. What suits me may not be right for you. :)
My 4BG card is now 96% full and I too have data held on Meteoplug.
How easy is it to prune data on my Alix-1d?

I can see the data on my Windows network, is it just a case of deleted the months I no longer require and then doing a "Weather Data Recomputation" or is it more complex than that?
Thanks.

Re: Options when running out of storage space

Posted: Tue Aug 07, 2012 10:41 am
by skyewright
oliverb wrote:How easy is it to prune data on my Alix-1d?

I can see the data on my Windows network, is it just a case of deleted the months I no longer require and then doing a "Weather Data Recomputation"
That's what I do (after having first made an extra copy of those months elsewhere[1]).


[1] Maybe someday I'll have Meteohub on a big fast system that can handle many years worth of data with comfort, in which case I think all I'd need to do would be to copy back all those archived months and recompute...

Re: Options when running out of storage space

Posted: Tue Aug 07, 2012 11:44 am
by oliverb
Thanks skyewright.

I backup Meteohub every night using cwrsync. I also run Syncback every now and then from my Windows PC to backup to my Stora Network drive.

Just to check before I delete some of my data.

See attachment from Windows 7.
Can I just right click on the folder and select delete?
Then goto Meteohub and under "Maintenance" and select "Recompute" or do I have to Stop Data logging first?

In the future if I get a larger card I presume I can just copy the folder back in from my backup and do another recompute?

Re: Options when running out of storage space

Posted: Tue Aug 07, 2012 12:35 pm
by skyewright
oliverb wrote:I backup Meteohub every night using cwrsync. I also run Syncback every now and then from my Windows PC to backup to my Stora Network drive.
I like to keep an extra "raw" (i.e. uncompessed & readable) archive too. "Belt & Braces" (I hope). :)
oliverb wrote:See attachment from Windows 7.
Can I just right click on the folder and select delete?
Then goto Meteohub and under "Maintenance" and select "Recompute" or do I have to Stop Data logging first?
I think that in theory stopping Data Logging might be "A Good Idea", but I figure that the files that are being deleted are not actively being added to so for several years I haven't bother to do that while doing my annual prune, and I've not seen any problem with that approach.
oliverb wrote:In the future if I get a larger card I presume I can just copy the folder back in from my backup and do another recompute?
That's the general idea I have in mind.
However if you were adding a larger card to the alix-1d I think you'd also need some work (repartitioning?) to make the extra space accesssible to Meteohub first?

Re: Options when running out of storage space

Posted: Tue Aug 07, 2012 4:44 pm
by oliverb
Thanks again for your help.
I have deleted the two months from 2007. This should buy me some time while I decide if I am going to get a larger flash card or just run with a few years data on Meteohub and use Meteoplug for my main data source.

Re: Options when running out of storage space

Posted: Fri Aug 10, 2012 6:31 pm
by HWTest
I'm running a Sheeva plug, so the files are on a SD card, but it should be the same for Alix on a CF card.
I was running out of space on the 4GB card, 3 years of data, lots of sensors. I have bought a 8GB card and a card reader.
Booted Ubuntu, using Partimage I have created backups of both partitions (system, data).
On the new card I created 3 new partitions, the first for the system I created a little bigger, than it was originally, the second (swap) the same as it was and the third for data on rest of the free space. I don't know if it is necessary, but I also aligned the new partitions to 4k sectors.
Then I restored the backups using Partimage, for safety did a fsck, put the card back in the Sheeva.
Booted, checked all is running fine. I had to change the licence, because the system id changed with the new card and now I have plenty of room :-)
Until you change your licence, you can allways go back to your old card if something goes wrong.