Is there a supported or recommended method of uploading data to a custom web server at a higher frequency than 1 minute. I would like to "feed" the gauges on my web site with as near to real-time wind data as possible. The 1 minute FTP upload frequency is not sufficient for this. I would like to stay away from opening my meteohub to the internet for security reasons and because of it's limited capacity to handle frequent http requests.
Thanks.
Baobab
High frequency uploads to custom web server
Moderator: Mattk
-
ZebraWetter
- Junior Boarder
- Posts: 22
- Joined: Sun Aug 14, 2011 11:44 am
- Location: Duisburg/northrhine-westfalia/germany/earth/sol system
Re: High frequency uploads to custom web server
I think the best way to do this is to use the HTTP data logger protocol which is provided by Meteohub.
With this protocol you can retrieve all values you need with a very small overhead and - and that is the best thing in my opinion - you can request only the values you really need.
Andy
With this protocol you can retrieve all values you need with a very small overhead and - and that is the best thing in my opinion - you can request only the values you really need.
Andy
Re: High frequency uploads to custom web server
Thanks for the suggestion Andy. I have spent the last couple of days pulling my hair out trying to get the http logger data into php variables but have been unsuccessful. There seems to be very little available in terms of documentation on how to implement many of the great features provided by meteohub.
So, the following php script works flawlessly to pull data from Google's weather API:
<?php
$xml = simplexml_load_file('http://www.google.com/ig/api?weather=brooks+alberta');
$current = $xml->xpath("/xml_api_reply/weather/current_conditions");
$currentwx = $current[0]->condition['data'];
?>
I have been trying in vain to use the above script to access the logger data on my metohub (with the proper url of course). The port forwarding on my network (port 7777) is properly configured as I can pull logger data using a web browser from a computer outside of my network. However I cannot get the meteohub to accept a connection using the simple_load_file php function that works for the google API. I've also tried a couple of other php functions without success. Any suggestions?
Also, I see the returned XML is not exactly in "legal" XML format so I suspect the xpath function will not work either. Any suggestions on how to parse the response once I get the meteohub to return a request?
Thanks for any insight that anyone can share.
Phil
So, the following php script works flawlessly to pull data from Google's weather API:
<?php
$xml = simplexml_load_file('http://www.google.com/ig/api?weather=brooks+alberta');
$current = $xml->xpath("/xml_api_reply/weather/current_conditions");
$currentwx = $current[0]->condition['data'];
?>
I have been trying in vain to use the above script to access the logger data on my metohub (with the proper url of course). The port forwarding on my network (port 7777) is properly configured as I can pull logger data using a web browser from a computer outside of my network. However I cannot get the meteohub to accept a connection using the simple_load_file php function that works for the google API. I've also tried a couple of other php functions without success. Any suggestions?
Also, I see the returned XML is not exactly in "legal" XML format so I suspect the xpath function will not work either. Any suggestions on how to parse the response once I get the meteohub to return a request?
Thanks for any insight that anyone can share.
Phil
Re: High frequency uploads to custom web server
I use this script to put Meteohub data to MySql. HTTP datalogger request in plain text, no XML. You have to modify it depending to your sensors. I use 'solar in a jar' if you don't you can skip it. This script is called every minute by cronjob on my webserver.
Code: Select all
<?PHP
echo '<html><head><meta http-equiv="Author" content="Jos van Ruitenbeek 2012 www.hazajon.eu" /></head>';
//******* PLEASE LEAVE MY CREDITS *********
/*
* Create a MySql table and fields of your sensors
* Example output of http://$Meteohub_adress:$Meteohub_port/meteolog.cgi?mode=data
*
* 20120503075257 wind0 0 1.3 1.3 23.1
* 20120503075259 th2 40.4 48 27.2
* 20120503075259 sol0 562
* 20120503075245 rain0 0.0 22.1 0.0 *! We only use the first value: rainrate. Once an hour we get rainfall by RAINFALL LAST HOUR
* 20120503075115 th1 22.1 54 12.4
* 20120503075256 th0 23.1 48 11.5
* 20120503075257 thb0 22.5 51 11.9 992.0 1008.6 2
* 20120503075232 uv0 2.0
*
* You should create these tables:
*
* Table: wind0
* fields: localtime(TIMESTAMP, PRIMARY), meteohub_utc(BIGINT, UNSIGNED, INDEX), dir(SMALLINT, UNSIGNED, INDEX)), gust(DECIMAL 4,1, INDEX)),
speed(DECIMAL 4,1, INDEX)), chill(DECIMAL 3,1, INDEX))
* Table: th2
* fields: localtime, meteohub_utc, temp, hum, dew
* etc.
* rain0 we don't use, instaed we use rainrate and rainlasthour.
* More info about sensor value format: http://www.meteohub.de/files/HTTP-Data-Logging-Protocol-v1.2.pdf
*/
//SETUP
//Meteohub and MySql Variables
$Meteohub_adress = "http://yourip";
$Meteohub_port = "yourport";
$MySql_name = "name";
$MySql_user = "user";
$MySql_pass = "pass";
$MySql_host = "localhost";
$time_zone = 1; // hours plus or minus utc
$daylightsave_flag = "yes"; // correct time with European daylightsaving time
$timeStamp = date('Y-m-d H:i:s');
$lat = "46.15";
$long = "18.33";
$zenith = "90.5";
//calculate daylightsaving time
if ($daylightsave_flag == "yes")
{
//Calculate begin of wintertime, last sunday of october
$date =time () ;
$year = date('Y', $date);
$last_day_of_month = mktime(0,0,0,10,31, $year) ;
$day_of_week = date('D', $last_day_of_month); //What day is the 31th
//this is the base to calculate the date
switch($day_of_week)
{
case "Sun": $countdown = 0; break;
case "Mon": $countdown = 1; break;
case "Tue": $countdown = 2; break;
case "Wed": $countdown = 3; break;
case "Thu": $countdown = 4; break;
case "Fri": $countdown = 4; break;
case "Sat": $countdown = 6; break;
}
$dday = 31 - $countdown;
$wintertime = mktime (3,0,0,10,$dday,$year); //03.00h localtime last sunday of october
//Calculate begin of sumertime, last sunday of march
$date =time () ;
$year = date('Y', $date);
$last_day_of_month = mktime(0,0,0,3,31, $year) ;
$day_of_week = date('D', $last_day_of_month); //What day is the 31th
//this is the base to calculate the date
switch($day_of_week)
{
case "Sun": $countdown = 0; break;
case "Mon": $countdown = 1; break;
case "Tue": $countdown = 2; break;
case "Wed": $countdown = 3; break;
case "Thu": $countdown = 4; break;
case "Fri": $countdown = 4; break;
case "Sat": $countdown = 6; break;
}
$dday = 31 - $countdown;
$sumertime = mktime (2,0,0,3,$dday,$year); //02.00h localtime last sunday of march
//add one hour daylightsaving time or not
if(time() >= $sumertime and time() <= $wintertime) $add_daylightsaving = 1; else $add_daylightsaving = 0;
}
//calculate dayflag
$dayflag = "no";
$sunrise = date_sunrise(time(), SUNFUNCS_RET_TIMESTAMP, $lat, $long, $zenith, ($time_zone + $add_daylightsaving));
$sunset = date_sunset(time(), SUNFUNCS_RET_TIMESTAMP, $lat, $long, $zenith, ($time_zone + $add_daylightsaving));
if (time() > $sunrise and time() < $sunset) $dayflag = "yes";
//Open MySql databas
$DBconnect = mysql_connect($MySql_host,$MySql_user,$MySql_pass);
if (!$DBconnect)
{
die('Could not connect: ' . mysql_error());
}
mysql_select_db($MySql_name, $DBconnect);
// Retrieve data from meteohub by http request.
$allSensors=file_get_contents("$Meteohub_adress:$Meteohub_port/meteolog.cgi?mode=data"); // No start or end results in last data
// Split data in seperate lines
list($line[1], $line[2], $line[3], $line[4], $line[5], $line[6], $line[7], $line[8]) = explode(PHP_EOL, $allSensors);
//Split lines in seperate values and search for sensors.
for ($i = 1; $i <= 8; $i++)
{
list($meteohub_utc, $sensor, $val1, $val2, $val3, $val4, $val5) = explode(" ", $line[$i]);
//transform meteohub_utc to MySql timestamp. Correct with timezone and daylightsaving time
$timeStamp = strtotime($meteohub_utc);
$timeStamp =$timeStamp+(($time_zone + $add_daylightsaving)*60)*60; $timeStamp = date('YmdHis',$timeStamp);
switch ($sensor)
{
case "wind0":
$dir = $val1; $speed = $val2; $gust = $val3; $chill = $val4;
// calculate speedkmh and gustkmh skip if you want
$speed = $speed * 3.6;
$gust = $gust * 3.6;
// schrijf naar MySql
mysql_query("INSERT INTO wind0 VALUES ('$timeStamp','$meteohub_utc', '$dir', '$speed', '$gust', '$chill')");
break;
case "thb0":
$temp = $val1; $hum = $val2; $dew = $val3; $press = $val4; $sealevel = $val5;
// schrijf naar MySql
mysql_query("INSERT INTO thb0 VALUES ('$timeStamp','$meteohub_utc', '$temp', '$hum', '$dew', '$press', '$sealevel')");
break;
case "th0":
$temp = $val1; $hum = $val2; $dew = $val3;
// schrijf naar MySql
mysql_query("INSERT INTO th0 VALUES ('$timeStamp','$meteohub_utc', '$temp', '$hum', '$dew')");
break;
case "th1":
$temp = $val1; $hum = $val2; $dew = $val3;
// schrijf naar MySql
mysql_query("INSERT INTO th1 VALUES ('$timeStamp','$meteohub_utc', '$temp', '$hum', '$dew')");
break;
case "th2":
$temp = $val1; $hum = $val2; $dew = $val3;
// schrijf naar MySql
mysql_query("INSERT INTO th2 VALUES ('$timeStamp','$meteohub_utc', '$temp', '$hum', '$dew')");
break;
case "uv0":
$uvi = $val1;
// schrijf naar MySql
mysql_query("INSERT INTO uv0 VALUES ('$timeStamp','$meteohub_utc', '$uvi')");
break;
case "sol0":
$wqm = $val1;
$sunshine = 0;
//Calculate sky; sunshine or not
if ($dayflag <> "no") //it's daytime
{
include 'hemel.php';
if ($zonProcent >= 70) $sunshine = 1; else $sunshine = 0;
}
// schrijf naar MySql
mysql_query("INSERT INTO sol0 VALUES ('$timeStamp','$meteohub_utc', '$wqm', '$sunshine')");
break;
case "rain0":
//we only use rainrate, rainfall in the past hour we get in RAINFALL LAST HOUR below
$mm = $val1;
// schrijf naar MySql
mysql_query("INSERT INTO rainrate VALUES ('$timeStamp','$meteohub_utc', '$mm')");
break;
}
}
//RAINFALL LAST HOUR
// all the sensorvalues above we want every minute (set by cronjob on your webserver)
// the only value we want once per hour is the value for rain in the last hour
// to make sure meteohub calculated that value we wait until one minute past every hour
$minute = date('i');
if ($minute == 1)// one minute past every hour get the rainfall of the last hour
{
//make a request to Meteohub
$rain = file_get_contents("$Meteohub_adress:$Meteohub_port/meteograph.cgi?text=seqhour1_rain0_total_mm");
list($mm, $dummy) = explode("_",$rain); // we only need the first value (last hour)
//localtime
//because there's no meteohub timestamp make a timestamp of actual time
//minus 2 minutes so rainfall counts in the hour that the rain was falling and make a timestamp of a round hour (hh:00:00)
$timeStamp = time(); $timeStamp = $timeStamp - 120; $timeStamp = date('Y-m-d H:i', $timeStamp);
// schrijf naar MySql
mysql_query("INSERT INTO rainlasthour VALUES ('$timeStamp', '$mm')");
}
//close MySql database
mysql_close($DBconnect);
?>
Re: High frequency uploads to custom web server
So after more hair pulling I just found out that my web host is blocking port 7777 (the port that is exposed by meteohub's data logger). Thus, using file_get_contents("http://xxx.yyy:7777/Meteohub_port/meteo ... ?mode=data") or a similar php function cannot work. I don't suppose anyone knows if port 7777 can be changed to something else that is not blocked?
Phil
Phil
Re: High frequency uploads to custom web server
Change in menu 'network' in Meteohub the Meteohub port (eg. 81), forward this port in your router to your internal Meteoheb IP.
Re: High frequency uploads to custom web server
yes, but this will change the regular http port of meteohub, so no operation on port 80 will be possible anymore. mostly not a problem, but I wanted to mention that. other idea might be to leave meteohub settings as is and to make a port forwarding of a port of your liking (81, 8080, etc) at your router to meteohubs port 80 (or 7777). that should normally work and you dont have to deal with complications that meteohub is no longer avail at standard http port in your lan.
Re: High frequency uploads to custom web server
Thanks everyone. I got it to work. I changed port 80 on the MH to one of the ports not blocked by my web host and forwarded that port on my router to the internal MH ip. As far as web admin for the MH port 7777 still works and 7777 is easy to remember.