Page 1 of 1

Weather34 historic data

Posted: Fri Mar 19, 2021 9:12 pm
by mcwx1359
Is there method of exporting historic data from nano sd to weather34 csv file in the format for the charts for the previous months.
I have just started using the template but have data on for previous months for 2021 / 2020
Regards
Mike

Re: Weather34 historic data

Posted: Mon Mar 22, 2021 11:49 am
by Gyvate
You create a Meteobridge customized export template of the values in the sequence of the w34 month and year file (they are the same for both).
Depending on the template (i.e. W34 8.34, 9.34, Aurora ...) you are using, the record definition (sequence of fields in the CSV) is different - the newer versions have more fields added at the end.
For older versions it is no harm having the bigger records, the smaller records with the newer versions will create empty graphs where the values are missing.

which expressions to use -dxxxx of which selector e.g. thb0seapress to be formed to e.g. [thb0seapress-dmax:1] for max daily pressure or [thb0seapress-dmin:1] for the minimum daily pressure, you find in the wiki, section template
https://www.meteobridge.com/wiki/index.php/Templates

below the 8.34 and 9.34
----------------------------version 8.34 and 9.34 Aurora ------------------------------
1: Day Month - the first entry/record is a copy of the last entry of the previous month
2: T out high
3: T out low
4: dew point high
5: dew point low
6: rain total
7: max gust m/s
8: max average wind m/s
9: average temp out
10: sea level pressure high
11: sea level pressure low
---------------------------------- only since Aurora/9.34 ----------------------------
12:uv index max
13:solar radiation max (W/m2)
14: ?? contains always "--" (without "") - Purple Air
15: ?? airquality Davis ?? - haven't seen yet where it's used in the monthly/yearly charts
16: wind bearing average

a typical day record in the month and year file would then look like:
31 Mar,10.8,-0.5,-1.1,-6.0,0.0,1.6,7.1,4.8,1025.0,1023.0,7.4,792.5,--,66.0,171.0

I have a general multipurpose export definition which you can save under e.g. all-sensors-iso-new.exp in the Meteobridge templates folder
\\METEOBRIDGE\data\templates
you can remove (or add) sensors/variables, change the export sequence, modify the selectors
below between the ------------------------
from this you should be able to easily create your export
----------------------------------
# modified Standard CSV template with data in metric units
#
$# date, time, temperature[C], humidity[%], dew point[C], sealevel pressure[hPa], avg wind speed[m/s], gust speed[m/s], winddir, rainfall[mm], Solar [W/m2], PM25(1) [ug/m3], PM25(2), extra-temp1 [°C], extra-hum1 [%], extra-temp2 [°C], extra-hum2 [%], extra-temp3 [°C], extra-hum3 [%], extra-temp4 [°C], extra-hum4 [%],soil moisture (1) [%], soil moisture (2) [%], soil moisture (3) [%], soil moisture (4) [%], soil moisture (5) [%], lightning total, lightning distance, lightning energy
[YYYY]-[MM]-[DD],[hh]:[mm],[th0temp-avg:],[th0hum-avg.0:],[th0dew-avg.1:],[thb0seapress-avg.1:],[wind0avgwind-avg.1:],[wind0wind-max.1:],[wind0dir-avg.0:],[rain0total-sum.1:], [sol0rad-avg.2:], [air1!0pm-avg.1:], [air1pm-avg.1:], [th1temp-avg.1:], [th1hum-avg.1:], [th2temp-avg.1:], [th2hum-avg.1:], [th3temp-avg.1:], [th3hum-avg.1:], [th4temp-avg.1:], [th4hum-avg.1:],[th20hum-avg.0:], [th21hum-avg.0:], [th22hum-avg.0:], [th23hum-avg.0:], [th24hum-avg.0:], [lgt0total-avg.0:], [lgt0dist-avg.0:], [lgt0energy-avg.0:]
--------------------------------

Re: Weather34 historic data

Posted: Tue Mar 23, 2021 4:01 pm
by ToTo
Hi,

interesting, I am trying to use the export to complete my 2019.csv for the Weather34 template.
I can only export the current year, am I missing something?
exp.jpg
exp.jpg (35.83 KiB) Viewed 3648 times

@Gyvate: Are you sure this is correct?
9: average temp out

I do not get the same value from my
2021.csv:
1 Mar,6.6,1.6,2.7,0.4,0.0,3.1,0.4,1.7,1037.4,1035.3
weather34 export output:
1 03,6.6,1.6,2.7,0.4,0.0,3.1,0.4,3.3,1037.4,1035.3

Also the month variable is not working for me, maybe a firmware bug? From https://www.meteobridge.com/wiki
(MMM: month as three character English abbreviation ("Jan", Feb", ...).
I use [MM]) MMM should work, but not for me, because [MMM] is output 1:1 like this MMM.

I make this weather34.exp:

# Weather34 CSV Export with data in metric units
#
$# Day Month,T out high,T out low,dew point high,dew point low,rain total,max gust m/s,max average wind m/s,average temp out,sea level pressure high,sea level pressure low
[D] [MM],[th0temp-max:],[th0temp-min:],[th0dew-max:],[th0dew-min:],[rain0total-sum.1],[wind0wind-max:],[wind0wind-avg:],[th0temp-avg:],[thb0seapress-max:],[thb0seapress-min:]

Re: Weather34 historic data

Posted: Tue Mar 23, 2021 6:08 pm
by Gyvate
that position #9 has been repurposed from what I have understood, but for the years 2019/2020 it's still ok.
Anyhow - it's afaik not used in the past year charts - no nothing to worry about I'd say.

Of course, for the W34 date, you will have to change/modify the script date definition 8)
you get things on a silver tray from me, but the gold you have to earn yourself :wink:

I also created my 2020 and 2019.CSV files, but from that general script and some copy&paste in Excel.
Plus some historical data were before I had Meteobridge. I had 5 minute intervals and would create the average T out by a formula in Excel.
Afaik it is not used in the year charts anyway. They only use Tmin and Tmax.
The 8 year charts are predefined and cannot be changed by customizing, only by modifying the scripts.
So look up what is shown in these charts - only these values are needed and need to be accurate - of course in the file structure something has to be there (and be it empty space between the commas.

Before 2021 the W34 data definition/record definition was different - probably that's why it's not implemented for direct export.
For earlier dates you may need to create your own export script - just modify the one I gave you.

And you cannot run it in the export dialogue. That's limited to 2021 for some reason.

The workaround I propose:
You need to create the script, test it in the export dialogue.
Then you create a service event, export, with that script. Then you can go back farther than 2021.

Re: Weather34 historic data

Posted: Tue Mar 23, 2021 7:17 pm
by ToTo
Thank you, yes of course I can still edit the export manually, that's not a problem.

The export for 2019 also does not go via a service event, only current year and last year.
I will try "@YYYYMMDDhh" specifier to select year to export (https://www.meteobridge.com/wiki/index.php/Templates).

Re: Weather34 historic data

Posted: Wed Mar 24, 2021 11:08 pm
by mcwx1359
Thank you again for the information , now updated 2021 .
Next issue is the forecast page , I don't get the forecast data populated only the section at the bottom. Checked WU API, all seems good there .

Any ideas ?
Regards
Mike

Re: Weather34 historic data

Posted: Thu Mar 25, 2021 7:30 pm
by ToTo
I have the same problem, no idea why that is.

But I think you're more likely to get an answer if you ask the question in the right thread.
viewtopic.php?t=15243&start=600

Re: Weather34 historic data

Posted: Thu Mar 25, 2021 8:44 pm
by weatherist34
ToTo wrote: Thu Mar 25, 2021 7:30 pm I have the same problem, no idea why that is.

But I think you're more likely to get an answer if you ask the question in the right thread.
viewtopic.php?t=15243&start=600
Sorry its a short reply but thanks it was missing a jQuery string in recent master download only !

please run the simple update just uploaded.Should fix the problem.

literally unavailable as of tonight for a few weeks or more so I hope it resolves that issue.

Download Update 25th March 2021
* Fix jquery in chart forecast Ref mcwx1359/Toto (21:38).

Re: Weather34 historic data

Posted: Thu Mar 25, 2021 9:14 pm
by ToTo
Thanks Brian, for the quick fix!

Re: Weather34 historic data

Posted: Thu Mar 25, 2021 10:24 pm
by mcwx1359
Brian,
Very much appreciated, all working . I have been running the wordpress weather plugin but always wanted to run the weather34, had a few technical issues last year in getting it working (my error) so now a happy chappy.

Best Regards
Mike

Re: Weather34 historic data

Posted: Sun Mar 28, 2021 3:14 pm
by ToTo
Since I myself had some gaps of CSV data from 2019 to 2021, I made myself an export template for 2019-2020 and 2021. With Weather34 Aurora MKII Version the CSV format was changed a bit, so you need two different templates for the export. As information source which data are written into the CSV files, I took the *chart-end-of-day.php.

The easiest way is to call the export.cgi in the browser, because the Meteobridge interface can export max. 2 years backwards.

To create an entry for each day, the call for 2019 looks like this:
http://meteobridge-IP/cgi-bin/export.cg ... 9_2020.exp

Please check against your own CSV data, if it fits! The values can differ minimally, depending on when chart-end-of-day.php was originally executed. Why in Weather34 Aurora MKII Version is double writing in the 7th and 8th place (Windspeed-AVG) can probably only be answered by developer Brian. I have taken this from weather34-chart-end-of-day.php.

The files must be placed in the meteobridge-IP\data\templates folder.

Re: Weather34 historic data

Posted: Sun Mar 28, 2021 5:03 pm
by weatherist34
position 8 is not used in the template and none of the month/year charts look for position 8 reserved for future value perhaps max rain rate as that is frequently requested ,the current duplicate value just fills a void and doesn't interfere with anything.hope that clears your curiosity ..

#######################################################
# CREATED FOR Aurora MKII version
# https://weather34.com/homeweatherstation/index.html
# Creates and adds month and year csv data
# Updated Release: Revised January 2021
#
# https://www.weather34.com
#######################################################
// 0 (date-month-0)),
// 1 MAX(yesterday Temp-1-$meteobridgeapi[82])
// 2 MIN(yesterday Temp-2-$meteobridgeapi[84])
// 3 MAX(yesterday Dewpoint-3-$meteobridgeapi[52])
// 4 MIN(yesterday Dewpoint-4-$meteobridgeapi[120])
// 5 TOTAL(yesterday Rain total-5-$meteobridgeapi[100])
// 6 MAX(yesterday Windspeed-6-$meteobridgeapi[94])
// 7 AVG(yesterday WindSpeed-7-$meteobridgeapi[123])
// 8 not currently used $meteobridgeapi[--] * reserved
// 9 MAX(yesterday Barometer-9-$meteobridgeapi[135])
// 10 MIN(yesterday Barometer-10-$meteobridgeapi[137])
// 11 Max( yesterday UV-11-$meteobridgeapi[114])
// 12 Max( yesterday SOLAR-12-$meteobridgeapi[107])
// 13 Purple Air(13-$aqiweather["aqi24h"]) *if yes
// 14 AIRQ 24H(14-$meteobridgeapi[184]) * Davis Airlink
// 15 yesterday Wind DIRECTION average (15-$meteobridgeapi[171])

Re: Weather34 historic data

Posted: Sun Mar 28, 2021 5:25 pm
by ToTo
Thanks for the good explanation, I was just surprised that the same value is written 2x.