How big is your monthly raw data file?

Discussion of the Meteohub software package

Moderator: Mattk

Post Reply
cbhiii
Gold Boarder
Gold Boarder
Posts: 306
Joined: Fri Feb 15, 2008 2:02 am
Location: Michigan, USA
Contact:

How big is your monthly raw data file?

Post by cbhiii »

Now that I've got a few months of data collected I'm wondering just how much data others have collected for each month of the past few months.

Here is my data from my Vantage Pro 2:
/data/weather/200803/raw 79,504,228 bytes
/data/weather/200804/raw 13,932,528 bytes
/data/weather/200805/raw 25,460,302 bytes

The information above poses some interesting questions. First I understand that the 79.5MB of data for March was due to the software storing more data than it needed, but I'm curious about the dip in April to 13.9MB and then back up to 25.4MB for May. What could account for the nearly 10MB increase is raw data in May?

How do you raw files looks? How will they look by the end of the year and when you reboot in December, how long will it take Meteohub to recompute that data?
Station: Davis Vantage Pro2 Plus
Hardware: Raspberry Pi 2 (Meteohub status)
Chill
Expert Boarder
Expert Boarder
Posts: 151
Joined: Fri Feb 15, 2008 6:38 pm

Re:How big is your monthly raw data file?

Post by Chill »

Hi,

My raw files size is more regular:
20809991 February
21168483 March
19853784 April
20529416 May

But I do suppose it must have a relation with how many sensors you have. I'm using a regular Oregon WMR-200 (Rain, Temp Int-Out, Barometer, Wind)

And all time I've had same sensors ;)

Chill
skyewright
Platinum Boarder
Platinum Boarder
Posts: 873
Joined: Fri Jan 25, 2008 6:27 pm
Location: Isle of Skye, Scotland

Re:How big is your monthly raw data file?

Post by skyewright »

March 16.42 MB (part month)
April 22.43 MB
May 28.03 MB (2 extra temp sensors, plus some other sensors getting picked up better)

Full sensor count is now:

2 rain
2 wind
1 THB
5 TH
2 T
cbhiii
Gold Boarder
Gold Boarder
Posts: 306
Joined: Fri Feb 15, 2008 2:02 am
Location: Michigan, USA
Contact:

Re:How big is your monthly raw data file?

Post by cbhiii »

2 rain, 2 wind, 5 TH? Wow!

What is your setup like? How do you use those sensors? Are they off in different parts of your property? Do you see a lot of variations in them?
Station: Davis Vantage Pro2 Plus
Hardware: Raspberry Pi 2 (Meteohub status)
skyewright
Platinum Boarder
Platinum Boarder
Posts: 873
Joined: Fri Jan 25, 2008 6:27 pm
Location: Isle of Skye, Scotland

Re:How big is your monthly raw data file?

Post by skyewright »

cbhiii wrote:What is your setup like? How do you use those sensors? Are they off in different parts of your property? Do you see a lot of variations in them?
I have an OS WMR200 and an OS WMR928 plus extra sensors for the WMR928. All the sensors are received by an RFXCOM which is connected to Meteohub. Meteohub then feeds data to WD.

It wasn't my original intention to have 2 weather stations I got the WMR928 at least partly as a result of frustrations with the WMR200!

The 2 rain & 2 wind are in similar positions at present.

One of the rain gauges has a funnel to increase sensitivity, so the other is a useful comparison if rainfall were ever heavy enough to swamp the sensitive one. The sensitive one is used for the feed to WD.

The WMR200 wind seems a bit more responsive than the WMR928, and certainly has a lower start up speed, but is less reliable (i.e. it seems to lock up occasionally and need a reset). When I get a taller mast it is very likely to be the WMR928 that goes at the top!
At that point I may move the WMR200 gauge to a different location too to see how wind varies.

Because the anemometers are in a similar positon at present I intercept the wind data between Meteohub and WD and combine both anemometers - so WD thinks there is only one. That way I double the sampling rate (only every 15 seconds with OS anemometers), catch the 'best' of the gusts and still get data even if one sensor fails (wind is the value of most interest to me).

The various temp sensors are spread about for different purposes either domestic or gardening, e.g. attic, conservatory, cold frame, soil temp. One of the temp sensors is placed in a small sealed jar in a sunny position. WD then does analysis on the difference in temp between that sensor and the standard outdoor temp to provide a form of solar sensor (known as 'temp in a jar').

That's a brief overview.
cbhiii
Gold Boarder
Gold Boarder
Posts: 306
Joined: Fri Feb 15, 2008 2:02 am
Location: Michigan, USA
Contact:

Re:How big is your monthly raw data file?

Post by cbhiii »

Neat info. Thanks.
Station: Davis Vantage Pro2 Plus
Hardware: Raspberry Pi 2 (Meteohub status)
bsparks
Senior Boarder
Senior Boarder
Posts: 69
Joined: Sat Mar 29, 2008 3:01 pm
Location: Alexandria, LA

Re:How big is your monthly raw data file?

Post by bsparks »

That is really neat. Thanks for sharing!
Post Reply