Page 1 of 1

Upload from Domoticz to Meteobridge

Posted: Mon Jan 29, 2024 9:14 am
by Tln7559
Plenty of types of weatherstations have uploadfunctions readily integrated in Meteobridge.
Obviously still quite a few outside that catalogue.
Having such ‘misfit’ setup, but lucky to be able to read the subject weatherstation (and a series of DIY-sensors) with Domoticz.
Looking for a possibility to upload their data from Domoticz to Meteobridge, searched the website and this forum without direct result.

Thinking in the direction of userdefined weatherstation (= for Meteobridge the bottom selection in the column of available options).
What would be a compatible output from Domoticz, with what setting in Meteobridge?
Any examples for learning?

Re: Upload from Domoticz to Meteobridge

Posted: Mon Jan 29, 2024 11:29 am
by Mattk
From all accounts a light weight home automation system is not what meteobridge is about.

Re: Upload from Domoticz to Meteobridge

Posted: Tue Jan 30, 2024 11:11 am
by Tln7559
@Mattk

Completely agreed that a home automation system is 'not the best PWS',
However, in my view Domoticz for Meteobridge potentially is a useful gateway/portal.
Because at least that home automation system for me has the technical & functional interfaces to read that 'misfit' PWS and a bunch of additional, non-standard sensors.
In that perspective considering Domoticz as a practical intermediary to get the data from that PWS and sensors towards Meteobridge,
now looking how to make technical interface between Domoticz and Meteobridge.

Why investigating the path from Domoticz to Meteobridge?
The 'misfit' PWS in fact is a legacy LaCrosse_WS7000 (aka WS2500 or ELV/Conrad_WS2000), most reliably operating since 20 years!
With some of the T&H-sensors upgraded with SHT31 as sensor and expanded with a cloned WS7000/19 (aka WS2500/19) it is Qualitywise worth keeping as long as it lives.
The present interface from the LaCrosse_WS7000 to the outside world is by a PC, which is aging (already far exceeding predicted end-of-life) and 24/7 PC-operation takes more energy than a Raspberry+Meteobridge, even if replacing the present PC by some NUC.
Non-standard sensors have non-standard interfaces, for which Domoticz flexibly might translate into format Meteobridge-compatible.

Why Domoticz as Gateway?
Meteobridge has a large collection of interfaces for PWSes, but Domoticz has peripherals like RFXCom and RFLinkGateway which have an abundance of wireless readout-functions for many more PWSes. Superior datatransfer if that RFXCom and/or RFLinkGateway fitted with 'real' antenna's.
Domoticz also most flexible to interface with all kinds of stand-alone sensors, over Wifi, or cabled also with interface types like RS485.
Domoticz has a very active supporting community.

Re: Upload from Domoticz to Meteobridge

Posted: Wed Jan 31, 2024 10:45 am
by Tln7559
In Domoticz the simplest readout is making call to the desired Virtual Device.
For a T&H-sensor that results in a JSON-file like attached in zip-file Domoticz_RIDreadout.zip.
A related plugin for Meteobridge might rip out the data for T&H for that specific device:
is straightforward solution for 1 single device, to be repeated for any other device,
and (relative to Meteobridge) therefore not a suitable solution if you want to get data for a series of devices.
Domoticz_RIDreadout.zip
Sample readout from Domoticz
(816 Bytes) Downloaded 221 times
Searching further in this forum, combining with hints in other forums, have the impression that a pragmatic, stepwise solution might be to combine the readings for several Virtual Devices from Domoticz into one, small file:
1. Take the script from this thread as basic approach to read the data from Domoticz to Meteobridge as JSON-file.
For a first trial setup clone & tune a Luftdaten-datafile to e.g. the Temperature and Humidity from a single T&H-sensor in Domoticz and test against a tuned script.
2. Expand with a few more parameters, for testing against a dynamic datafile generated by Domoticz.
Obvious to take as candidates a set of sensor data presently missing in the datasets uploaded to my Meteobridge.
3. For 'grand' setup of the JSON-file from Domoticz and for related read-out in Meteobridge aim at the structure and dataset of Meteobridge's userdefined CampbellSci-plugin
However, the structure of the input-file for that CS-plugin is not JSON-layout.
Choice to be made:
either give file from Domoticz a layout&contents fitting the CS-plugin and suitable filename,
or rework the file to get 'real' JSON-layout with a matching plugin-script developed during steps 1. and 2.
File with 'real' JSON-layout might be alike the file coming out of this exercise,
quite convenient as source to have a dynamic input-file for testing this plugin-script (and comparable applications) with 'realistic' data:
such file-layout included in the test-script as example for 'extended version'.

ToDo:
=> Develop user-defined plugin-script for Meteobridge to read from Domoticz a T&H-dataset as JSON-file
=> Make in Domoticz a dynamic JSON-file matching this plugin.
=> Before exposure to Meteobridge, test this combination with script at LAN-client and and datafile at webserver.
Status:
Step 1. completed [attached zip-file contains both trial-plugin and test-json-file]
Step 2. in progress.
.
Domotiz_to_Meteobridge_Step1a.zip
Package of trial-plugins and related testfiles.
(4.19 KiB) Downloaded 237 times
.
Observations:
In the sourcescript (and therefore in the test-script) a readout and multiplication is present as

Code: Select all

awk -F '"' '{printf "%.1f\n", ($8*10.0)}'
For checking, in the testscript printlines have been inserted.
These printlines show that the read value is rounded and is multiplied by 10.
IMHO not correct to round the read values, because unnecessary loss of information.
Assume that 10*multiplication is an entry condition for Meteobridge's calculations.

Adaptation Feb03, 2024:
Added status and bottomline.

Adaptation Feb05, 2024:
Package with files corrected (=> version 1a).

Adaptation Feb06, 2024:
Added sample readout from Domoticz with related expansion of 1sr section of this message, and adapted bottomlines.

Re: Upload from Domoticz to Meteobridge

Posted: Tue Feb 06, 2024 7:07 pm
by Tln7559
In parallel trying to clone&tune the CampbellSci-plugin for transfer of a complete PWS-dataset.
Question popping up:
if the PWS-dataset is not complete, what is the correct code in the string to signal to Meteobridge the absence of a component?
Just leave the 'bin' empty which is designated to the subject sensor?

Re: Upload from Domoticz to Meteobridge

Posted: Thu Feb 08, 2024 12:26 am
by Tln7559
Having Domoticz generate a datafile meeting the format of the CampbellSci-plugin is not a major effort.
Reading that datafile using the reading structure applied in the Luftdaten-plugin proved handier than application of nc.
In the sh-script below you see a mixture of both plugins mentioned above.
Important to note that the datafile from Domoticz is 100% SI-oriented => the original CampbellSci-plugin cannot be used!!
For pretesting running on a Raspberry with no error reports.
Also loading & running on Meteobridge, but (this being my first ever plugin for Meteobridge) most appreciated if developers of Meteobridge-software have a look whether best practises for plugins have been obeyed.
The url for access to the datafile is active at this moment to allow practise-testing: the data-source is an active PWS type LaCrosse_WS7000.
The url will be de-activated as soon as sufficient experience with the plugin-script for use with Meteobridge.

Code: Select all

#!/bin/sh
#
# Adaptation of Campbell Scientific data logger for SI/metric data-input from Domoticz
# Reading of the datafile has been borrowed from the Luftdaten-plugin,
# to allow reading from any location either at LAN or in webspace at internet.
# Delay in the read-loop has been inserted, because refresh of the datafile is per 5 minutes.
# This setup covers an extra T&H-set to include the 'grass'-sensor of the PWS.
# Line 008 = Layout of dataset
# Data from Domoticz compiled in datafile with 1 textstring:
# 1  average wind speed (m/s)
# 2  gust wind speed in last X minutes (m/s)
# 3  current wind direction (degrees north)
# 4  outdoor temperature (Celsius) # sensor at standard level
# 5  relative outdoor humidity (%) # sensor at standard level
# 6  windchill (Celsius)
# 7  current rain rate (mm/h)
# 8  cummulated raintotal of today, or this/last year, or since start (mm)
# 9  sealevel pressure (hPa)
# 10 uv index (uvi)
# 11 solar radiation (w/qm)
# 12 indoor temperature (Celsius)
# 13 relative indoor humidity (%)
# 14 outdoor temperature (Celsius)  # sensor at grass level
# 15 relative outdoor humidity (%)  # sensor at grass level 
#
# Line 026 = Start of script for reading datafile of Virtual_PWS as compiled by Domoticz.
while true;
do
#  Line 029 = LINE=$(wget --timeout=2 -qO- http://<source-ip>/<file-name>)
#  <source-ip> and <file-name> have to be tuned for your configuration in the next line!
  LINE=$(wget --timeout=2 -qO- http://vannwnhzn.nl/Meteobridge/Test/MB_WS7000PWS.txt)
  echo $LINE
# Line 033 = Check data
  status=$?
  if [ $status -ne 0 ] || [ -z "$LINE" ]; then
    >&2 echo "Error retrieving LINE -> sleeping"
    sleep 60
  else
# Line 039 = Read data and recompile & rescale
     CNT=1
     for i in $(echo "$LINE" | tr "," "\n")
     do
       eval value$CNT=\"$i\"
       CNT=$((CNT+1))
     done
     if [ -n $value1 ] && [ -n $value2 ] && [ -n $value3 ]
     then
       if [ -n $value6 ]
       then
         echo "$value1 $value2 $value3 $value6" | awk '{ print "wind0", int($3), int($2*10), int($1*10), int($4*10)}'
       else
         echo "$value1 $value2 $value3" | awk '{ print "wind0", int($3), int($2*10), int($1*10)}'
       fi
     fi
     if [ -n $value4 ] && [ -n $value5 ]
     then
       echo "$value4 $value5" | awk '{ print "th0", int($1*10), int($2)}'
     fi
     if [ -n $value7 ] && [ -n $value8 ]
     then
       echo "$value7 $value8" | awk '{ print "rain0", int($1*10), int($2*10)}'
     fi
     if [ -n $value9 ] && [ -n $value12 ] && [ -n $value13 ]
     then
       echo "$value9 $value12 $value13" | awk '{ print "thb0", int($2*10), int($3), int($1*10)}'
     fi
     if [ -n $value10 ]
     then
       echo "$value10" | awk '{ print "uv0", int($1*10)}'
     fi
     if [ -n $value11 ]
     then
       echo "$value11" | awk '{ print "sol0", int($1)}'
     fi
     if [ -n $value14 ] && [ -n $value15 ]
     then
       echo "$value14 $value15" | awk '{ print "th1", int($1*10), int($2)}'
     fi
# Delay next read-cycle
  >&2 echo "Delay reading next LINE -> sleeping"
  sleep 10
  fi
done
As remarked in earlier message, it looks as if also this plugin-script rounds some values out of the datafile before the multiplication *10.
In the print a resulting loss of accuracy, which is confusing: is that correct impression, or misunderstanding of the operation of awk?
Looking in 'LiveData' at Meteobridge the end-result within Meteobridge seems in line with the datafile: any explanation?
Some experimentation with the delay in the read-loop, because (if the delay is set too high) Meteobridge apparently not always reading the datafile, although a datafile always present.

Re: Upload from Domoticz to Meteobridge

Posted: Sun Feb 11, 2024 9:46 pm
by Tln7559
Now running the plugin a few days.
All seems well, besides today (= Feb11) following report in the Logging

Code: Select all

logger (11.02.2024 20:40:40): station 3 (User-defined), unknown sensor type in plugin data "0.0,0.0,34.0,8.5,94.0,18.4,0.0,0.5,993.0,0.0,0.0,19.7,55.0,8.3,95.0"
logger (11.02.2024 20:40:40): station 3: Bad thb0 sensor data, "pressure x10" (993) out of range [4000:13000] 
Observation:
Reading the error report, incoming Pressure must be multiplied by 10 in the plugin-script.
In that perspective, what's wrong in the 'multiplying' segment of the scriptline for thb0 (in which $value9 is pressure)?

Code: Select all

echo "$value9 $value12 $value13" | awk '{ print "thb0", int($2*10), int($3), int($1)}'
Has been cloned from CampbellSci-plugin, except for translation from 'Imperial values'
No error report and correct LiveData if changed into

Code: Select all

echo "$value9 $value12 $value13" | awk '{ print "thb0", int($2*10), int($3), int($1*10)}'
but that would mean that formula in the CampbelSci-plugin might be wrong.

Adaptation Feb13,2024:
Changed the question into observation, and aligned the script in previous message

Re: Upload from Domoticz to Meteobridge

Posted: Tue Feb 13, 2024 6:57 pm
by Tln7559
Making progress, but not yet free from error reports = too early to cry victory.
Datafile is

Code: Select all

2.5,0.0,225.0,8.7,74.0,9.36,0.0,0.0,1015.0,0.2,0.3,19.8,42.0,8.3,79.0
See all these values nicely popping up under Live Data, with all fields also reported under Mapping.
However under Logging this report

Code: Select all

logger (13.02.2024 17:50:17): station 3 (User-defined), unknown sensor type in plugin data "2.5,0.0,225.0,8.7,74.0,9.36,0.0,0.0,1015.0,0.2,0.3,19.8,42.0,8.3,79.0"
logger (13.02.2024 17:54:49): message above repeats 9 times.
What could be reason?
Functionally the error is just 'cosmetics' (because the datatransfer works OK),
but have nagging feeling that it has relation with the 'error' in the previous message.