How fast is recomputation? Version 3.2d reports numbers!

[admin]

Meteohub just takes a look at some system details to guess what kind of platform it is. As 3d2 and 3c2 are nearly identically Meteohub does not recognize the difference. Good to hear that 3c2 works.

[admin]

ebox-3300

• Meteohub v4.0-rc8
• 15765073 records processed in 18108 seconds (870 records per second)
• Transcend 4GB 266x

[AenG]

Alix.1d
- meteohub v4.1a
- 1792349 records processed in 850 seconds (2108 records per second)
- Kingston ultimate 4GB 266 cf-card

[P3R]

ALIX.1D with SanDisk Ultra II 4 GB CF.
With version 3.3: 1178419 records processed in 611 seconds (1928 records per second).
With version 4.1c: 1191975 records processed in 636 seconds (1874 records per second).

[AAMFlyer]

ALIX 3D2
Transcend 4GB Industrial (Ultra)
3944816 records processed in 1805 seconds (2185 records per second)

Next Day:
3967637 records processed in 1774 seconds (2236 records per second)

[wfpost]

ebox-4300
cf card kingston 266x
meteohub 4.1d


(01.03.2009 20:25:03): recomputation of weather data started.
(01.03.2009 21:22:54): recomputation of weather data finished: 6816272 records processed in 3464 seconds (1967 records per second)

(25.02.2009 00:20:01): recomputation of weather data started.
(25.02.2009 01:09:02): recomputation of weather data finished: 6679563 records processed in 2936 seconds (2275 records per second)

[HLucas01]

Alix 3D2
Transcend 4Gb 266Mbs CF-card
Meteohub 4.1d

Yesterday I transferred my software from NSLU2 to Alix 3D2. Without any graphs, webpages and weather network uploading scheduled I got the following speed:

recomputation of weather data finished: 574497 records processed in 200 seconds (2872 records per second)

Turning on graphs, webpages and weather network uploading I got the following speed:

recomputation of weather data finished: 586205 records processed in 455 seconds (1288 records per second)

My opinion: the speedinfo is very much dependent on the "load" other processes cause and as such cannot be compared from one user to another....

[cbhiii]

[*]eBox-3300[*]Transcend 4GB x133[*]v4.1[*]5370219 records processed in 6479 seconds
(828 records per second)
[*]5370304 records processed in 6536 seconds
(821 records per second)


I received two 'finished' messages for some reason. I'm not sure why or if they were running at the same time.

Update:

After v4.1e upgrade

recomputation of weather data finished:
5384065 records processed in 5596 seconds
(962 records per second)

[skyewright]

I received two 'finished' messages for some reason. I'm not sure why or if they were running at the same time.

That can easily happen, especially while setting up, as it seems that each change can potentially invoke a further recomputation, but ones already in progress don't get terminated.

Early on when I was experimenting with v4 and setting up and tweaking multiple weather stations, I believe there were something like 12 recomputations going on at one point, i.e. 12(?) "finished" after the last "start"! :blink:

[P3R]

Turning on graphs, webpages and weather network uploading I got the following speed:
/.../
My opinion: the speedinfo is very much dependent on the "load" other processes cause and as such cannot be compared from one user to another....

Of course the numbers will be very much affected by load.

Since this isn't a competition I don't think our minor configuration differences will matter that much. When looking at different results on roughly the same platform (memory performance will have an effect too) you will get a feel for what to expect, but don't take it too serious.

Naturally we should avoid doing other things at the same time so having Meteohub doing an actual upload of data concurrently with the recaclulation isn't smart if you want fair numbers. Just having it configured, but not active, I can't imagine will have much of an impact.

What you mean by turning on graphs and webpages? How do you turn them off and why?

[HLucas01]

I fully agree that there is and should not be any competition, but what surprised me was the fact that a system, on which no generation and upload of graphs and webpages and data for weathernetworks is scheduled, is almost twice as fast with the recomputation of data compared to a system on which they are.

By the way: what I meant with "turning on" graphs et cetera is putting the graphs and webpages into the cron schedule by means of the menu option "Schedule FTP Uploads of Graphs and Data" and making uploads to weathernetworks active by means of the tickbox "Upload data for Weather Networks via FTP" in the menu option "Upload of local weather data to Weather Networks".

Normally one will never turn these options off, but I was able to compare the performance when I was migrating my Metehub environment to a new platform and -for a short while- had an environment with no uploading active.

[P3R]

I agree that the huge difference you notice is remarkable.

Since I don't use any wheather network I've never had it enabled. This was what I noticed when having the tickbox for FTP-Upload activated first checked and then unchecked.
1352155 records processed in 691 seconds (1956 records per second)
1352798 records processed in 679 seconds (1992 records per second)

This minor variation I find reasonable but those results are better than my previously reported result of 1874 records per second with the same software (4.1c).

On the other hand when browsing back even further in the log I notice that I once had a number as low as 1652 records per second. I guess I was doing something else on the Meteohub at the same time.

Maybe it's just a fact that sometimes the numbers will show big variations, on the same hardware and even with the same configuration?

My advice for someone interested in finding out performance would be to not focus on a single reported value here but calculate a mean value from several different users (and maybe disregard extreme high or low results) with the same platform. The more results reported here, the more trustworthy the mean values will get.

My hardware: ALIX.1D with SanDisk Ultra II 4 GB CF.

[skyewright]

I agree that the huge difference you notice is remarkable.

Like you I don't upload to weather networks at present, but I understand that some people upload to several of them, so that's another factor that is going to vary.

I imagine that Weather Underground "Rapid Fire" might especially put a noticeable extra load on the system?

[HLucas01]

FYI: I have Weather Underground "Rapid Fire" active....

[P3R]

FYI: I have Weather Underground "Rapid Fire" active....

I got the impression that you only had uploads scheduled, not that it was actually sending stuff during recalculation ...

Now that you told us that I'm not surprised at what you see any more.

As I said earlier, just having stuff scheduled shoudn't affect recalculation much but uploading data at the same time will definately require CPU and slow other things down.