meteohub.de

As far as I understand the fastest my webserver can get data from Meteohub is once per minute.
That is because
1. the upload times using FTP is set to that minimum
2. FTP is the only way to "upload" data to ones own website.

Using one of the network uploads is even less interesting as that is once / 5 minutes, but there is one instance Meteohub can upload very fast, as it uploads to WeatherFlow / WeatherUnderground at once every 5 seconds in rapid fire mode but that data goes to WF/WU not to our own site.

At the moment I am developing release 3 of the Leuven Template were I need
1. one normal tag-file every 5 minutes or slower with all weatherdata that does not update fast => FTP is adequate
2. a realtime file to supply data to all realtime parts of a modern website, be it gauges / ajax and so on.

As there is only 1 realtime file to be used by all realtime parts of the users website it should be at least as fast as other weather-programs such as Meteobridge.
Meteobridge and most other weather-programs are far more capable, a few examples:
Meteobridge can drive the gauges and ajax data at once every 5 seconds (which is to fast to my liking).
WD and consoleWD supply clientraw.txt faster then once / minute.
VirtualWeatherStation supplies wflash data every so many seconds.
WVIEW is limited by the "only 1" ftp upload possibillity but can easily upload every 10 seconds using FTP

I amd supporting 12 weather-programs with the template and only Weatherlink.com is far slower then Meteohub.

Meteobridge and VWS are extremely faster as they support HTTP "upload".

@Boris
Can you please consider either allowing the WeatherUnderground Rapidfire to upload to our own sites, if you supply the tags I can filter the needed data
OR
to make available one or more http uploads.
OR
to make the upload frequency of the clientraw.txt file (only 1, not 4) faster, although that is more a mess than a solution for all involved parties.

If you check the list of the 200+ users of my Leuven-Template (http://leuven-template.eu/userlist.php?lang=en) there are 25 Meteobridge users and 34 Meteohub users and the numbers are still growing. So it would be pleasant to have the same user experience for Meteohub as for Meteobridge users.

Wim

Problem is that Meteohub does make use of crontab to schedule uploads. This limits repeating intervals.
So it will be a huge rework of basic Meteohub functions and I am reluctant do start such a big operation
right now.

With Meteobridge I redesigned that from scratch to get this limitation out of the way.

admin wrote:Problem is that Meteohub does make use of crontab to schedule uploads. This limits repeating intervals.
So it will be a huge rework of basic Meteohub functions and I am reluctant do start such a big operation
right now.

With Meteobridge I redesigned that from scratch to get this limitation out of the way.

I try to understan: FTP uploads use crontoab, so they can not be made faster.

But the WU/WF "uploads" go once every 5 seconds?

So I assume they do not use crontab, but HTTP "upload" such as used in MB ?
Can I put my personal website HTTP answer address in for such an upload?

Alternative solutions:
I tested using "http://www.meteoplug.com/cgi-bin/livedata.pl?" as used for the Meteobridge/Meteoplug dashboard but that would mean getting an extra account for Meteohub users. And it is not clear to me to get smaller requests for more data. For now the request is a hundred times bigger than the returned 12 datafields.

Using the "http://stationdata.wunderground.com/cgi ... p?station=" fetch of fresh data is a solution if MH users are using Rapid Fire already.
But for those who are not, it would be far easier to use the upload directly to the user-webserver directly instead of going meteohub => WU => user-webserver.

Wim

Hi,

Even with crontab, one can still run at a higher pace when using a script:
upload;sleep 5;upload; sleep 5.

So if you call a script that runs the upload 11 times (Depends on how fast the upload is) after being called, one would experience a higher frequency. Could also consider not to used a fixed sleep, but more dynamic based on the actual execution time of the upload.

Whether it's possible of not will also depend on the platform MH is running on, not all of them are that capable of running a lot of stuff on a high frequency, especially when the "larger" aggregates are taking place.

YJB wrote:Hi,

Even with crontab, one can still run at a higher pace when using a script:
upload;sleep 5;upload; sleep 5.

So if you call a script that runs the upload 11 times (Depends on how fast the upload is) after being called, one would experience a higher frequency. Could also consider not to used a fixed sleep, but more dynamic based on the actual execution time of the upload.

Whether it's possible of not will also depend on the platform MH is running on, not all of them are that capable of running a lot of stuff on a high frequency, especially when the "larger" aggregates are taking place.

I agree, also a psossibillity.

The problem is that FTP is really VERY inefficient, providers do not like it when using FTP to do "rapid-fire' type uploads. Multiple users were already banned when doing 10 sec's uploads using FTP. But a webserver can easily handle HTTP uploads every few seconds. No problem at all.

So I think the way to go is HTTP uploads for realtime data as that kind of data is not very big also. That is why I do not like clientraw.txt for use as realtime-data. Far to big file, far to complex to generate and almost all data is not needed for realtime display. Recent wind, gust, temp some date/time fields and so on is all that is needed.

And FTP 5 minutes in-between or for even longer periods were one should upload also all the other stuff, like high-lows today yesterday all time, soil temp/moist. All those data fields which do not change very fast. I now generate the gauges data from standard FTP upload and 1 realtime upload. Only a handfull of data fields are needed from the realtime uploads.

Wim

Bump

Now also WeatherDisplay is going the http route as discussed on this post.
http://www.weather-watch.com/smf/index. ... #msg502805

It is becoming more and more problamatic to use ftp for frequent "realtime' data. If possible in one way or another add some kind of HTTP upload also to Metehub.

Wim