Integrating AirLink - no sensor data

All about the standard Meteobridge devices based on mobile routers from TP-Link, D-Link, ASUS

Moderator: Mattk

thunderstorm
Senior Boarder
Senior Boarder
Posts: 75
Joined: Sun Mar 31, 2019 11:00 am

Re: Integrating AirLink - no sensor data

Post by thunderstorm »

My impression was also that my Airlink is also under-reading, but it looks like it isn't that consistent. For lower levels it seems more noticable.
My Airlink PM10 data isn't that different from my Nova SDS011, but for PM2.5 there's quite some difference. And the Airlink values for PM10 and PM2.5 are much closer together than for the SDS011. Please note that the sensors aren't at the exact same position, and the SDS011 is reading the data every 2min.

pm2.5.png
pm2.5.png (81.04 KiB) Viewed 3750 times
pm10.png
pm10.png (98.53 KiB) Viewed 3750 times
pm10-w.png
pm10-w.png (63.71 KiB) Viewed 3750 times

Unfortunately there's no good calibrated sensor nearby, so I can't tell which one is more accurate.

Bernd
User avatar
jasonmfarrow
Gold Boarder
Gold Boarder
Posts: 249
Joined: Mon May 25, 2020 4:21 pm
Contact:

Re: Integrating AirLink - no sensor data

Post by jasonmfarrow »

Screenshot 2020-10-03 at 3 October 2020 17.15.43 pm.png
Screenshot 2020-10-03 at 3 October 2020 17.15.43 pm.png (165.94 KiB) Viewed 3746 times
They are mostly following each other but the Airlink values are consistently lower than the Nova SDS011. PM10 much worse than the PM2.5.

Even applying a factor of 10 against the Airlink sensors is not making much difference (if it's having any affect at all:
Screenshot 2020-10-03 at 3 October 2020 17.17.09 pm.png
Screenshot 2020-10-03 at 3 October 2020 17.17.09 pm.png (37.29 KiB) Viewed 3746 times
Regards
Jason
https://jmfweather.uk | @jasonmfarrow
ToTo
Gold Boarder
Gold Boarder
Posts: 243
Joined: Wed Apr 03, 2019 9:24 am
Location: Magdeburg, Germany
Contact:

Re: Integrating AirLink - no sensor data

Post by ToTo »

Is it possible to upload the Airlink measured values to the Luftdaten.info network?
I found no entry in Weather Networks Tab.

https://github.com/opendata-stuttgart/meta/wiki/APIs
woolfg
Expert Boarder
Expert Boarder
Posts: 157
Joined: Wed Mar 20, 2019 4:01 pm
Contact:

Re: Integrating AirLink - no sensor data

Post by woolfg »

Hi Thunderstorm

Can you tell me how you got your SD011 data into meteobridge

I have this sensor attached to a raspberry pi but have no idea how to get my meteobridge pro to see it

How is yours set up and what did you do to get the data - did you use a plug -in if so would you mind sharing the script

Kind Regards

Graham
xray77
Gold Boarder
Gold Boarder
Posts: 182
Joined: Sat Mar 07, 2020 11:54 pm
Location: Buedelsdorf, SH Germany
Contact:

Re: Integrating AirLink - no sensor data

Post by xray77 »

Graham,
for integrate the NOVA SDS011 to MeteoBridge look at this Thread:
viewtopic.php?f=61&t=15158
Don't forget to create a user defined Station after saved the script into MB-Scripts Folder.
best regards
Mike
woolfg
Expert Boarder
Expert Boarder
Posts: 157
Joined: Wed Mar 20, 2019 4:01 pm
Contact:

Re: Integrating AirLink - no sensor data

Post by woolfg »

Hi

I had seen that but I dont know how to get the data from my raspberry pi (which has the sensor connected to a USB port ) into the meteobridge

How does the meteobridge know where to look

Regards

Graham
woolfg
Expert Boarder
Expert Boarder
Posts: 157
Joined: Wed Mar 20, 2019 4:01 pm
Contact:

Re: Integrating AirLink - no sensor data

Post by woolfg »

Hi Mike

I have now created the script and I think its running

I have created a Station 2 using the plugin but I think the json data file is not the same as the plugin script is expecting

I have attached the two files and I wondered if you would have a look when you have time to see if the plugin and the data are compatable

Regards

Graham
Plugin -------------------------------------------------------------------------------
#!/bin/sh
#
while true;
do
data=$(wget --timeout=2 -qO- http://192.168.0.***/aqi.json)
status=$?
if [ $status -ne 0 ] || [ -z "$data" ]; then
>&2 echo "Error retrieving data -> sleeping"
sleep 60
else
age=$(echo $data | tr '{' '\n' | grep age | awk -F '"' '{print $8}')
if [ $age -gt 120 ]; then
>&2 echo "Data too old ($age secs.) -> sleeping"
sleep 30
else
aqi100=$(echo $data | tr '{' '\n' | grep SDS_P1 | awk -F '"' '{printf "%.1f\n", ($8*10.0)}')
if [ ! -z "$aqi100" ]; then
echo "air0 $aqi100"
else
>&2 echo "PM10 not available"
fi
aqi25=$(echo $data | tr '{' '\n' | grep SDS_P2 | awk -F '"' '{printf "%.1f\n", ($8*10.0)}')
if [ ! -z "$aqi25" ]; then
echo "air1 $aqi25"
else
>&2 echo "PM2.5 not available"
fi
# sensor sleeps for 120 secs, 30+ secs for new measurement
sleeptime=$((152-age))
sleep $sleeptime
fi
fi
done
----------------------------------------------------------------------------------------------------
Sample Data
------------------------------------------------------------------------------------------------------
{"pm25": 2.1, "pm10": 4.1, "time": "10.10.2020 10:56:10"}, {"pm10": 4.3, "pm25": 1.9, "time": "10.10.2020 10:57:42"}, {"pm25": 2.0, "pm10": 4.6, "time": "10.10.2020 10:59:14"}, {"pm10": 4.3, "pm25": 2.0, "time": "10.10.2020 11:00:46"}, {"pm25": 2.1, "pm10": 4.6, "time": "10.10.2020 11:02:19"}, {"pm10": 5.0, "pm25": 2.3, "time": "10.10.2020 11:03:51"}, {"pm25": 2.2, "pm10": 4.0, "time": "10.10.2020 11:05:23"}, {"pm10": 5.4, "pm25": 2.3, "time": "10.10.2020 11:06:55"}, {"pm25": 2.3, "pm10": 5.1, "time": "10.10.2020 11:08:27"}, {"pm10": 4.3, "pm25": 1.9, "time": "10.10.2020 11:09:59"}, {"pm25": 2.1, "pm10": 4.9, "time": "10.10.2020 11:11:31"}, {"pm10": 7.0, "pm25": 2.1, "time": "10.10.2020 11:13:03"}, {"pm25": 1.8, "pm10": 5.0, "time": "10.10.2020 11:14:36"}, {"pm10": 4.0, "pm25": 1.8, "time": "10.10.2020 11:16:08"}, {"pm25": 2.1, "pm10": 6.1, "time": "10.10.2020 11:17:40"}, {"pm10": 5.8, "pm25": 2.1, "time": "10.10.2020 11:19:12"}, {"pm25": 2.2, "pm10": 5.6, "time": "10.10.2020 11:20:44"}, {"pm10": 7.1, "pm25": 2.1, "time": "10.10.2020 11:22:16"}, {"pm25": 1.9, "pm10": 4.4, "time": "10.10.2020 11:23:48"}, {"pm10": 7.9, "pm25": 2.1, "time": "10.10.2020 11:25:
User avatar
jasonmfarrow
Gold Boarder
Gold Boarder
Posts: 249
Joined: Mon May 25, 2020 4:21 pm
Contact:

Re: Integrating AirLink - no sensor data

Post by jasonmfarrow »

woolfg wrote: Sat Oct 10, 2020 10:30 pm I had seen that but I dont know how to get the data from my raspberry pi (which has the sensor connected to a USB port ) into the meteobridge
How does the meteobridge know where to look
I have two scripts; one on the raspberry pi collecting the data and saving it in discrete files the other on the meteobridge nano in the /scripts folder.
There are three steps to make all this work:
1) Ensure that your Raspberry Pi is running a html service to provide a basic website. I used lighttpd (https://pimylifeup.com/raspberry-pi-lighttpd/)
2) Use whatever NOVA SDS011 script and add a few lines to output to the website some files accessible over http(s)
3) Use the user-defined station on the meteobridge with the second script to read the files created in #2 and use the data.

Here are the relevant parts of the two scripts:

1) The bits you add to your Nova SDS011 reading script. The PM2.5 and PM10 values in my script are stored in an array called values[]. Before I use them I ensure that they are multiplied by 10 and converted to strings. This is because I spotted that when Meteobridge reads them they are divided by 10.

Code: Select all

pm25d = str(int(values[0]*10))
pm10d = str(int(values[1]*10))
with open("/var/www/html/aqi25.data","w") as aqifile:
       aqifile.write(pm25d)
       aqifile.close()
with open("/var/www/html/aqi10.data","w") as aqifile:
       aqifile.write(pm10d)
       aqifile.close()
2) The content of the aqijmf.plugin that you put on your meteobridge in the /scripts folder. Call yours whatever you want. My Raspberry PI can be accessed on the same LAN as the Meteobridge. Hence the call to 10.10.10.202. You could easily use a domain name depending on how you've setup your RPi

Code: Select all

#!/bin/sh
#
while true;
do
  aqi25=$(wget -qO- http://10.10.10.202/aqi25.data)
  echo "air1 $aqi25"
  aqi10=$(wget -qO- http://10.10.10.202/aqi10.data)
  echo "air2 $aqi10"
  sleep 300
done
Screenshot 2020-10-11 at 11 October 2020 13.32.11 pm.png
Screenshot 2020-10-11 at 11 October 2020 13.32.11 pm.png (60.25 KiB) Viewed 3677 times
My live data looks like this:
Screenshot 2020-10-11 at 11 October 2020 13.27.58 pm.png
Screenshot 2020-10-11 at 11 October 2020 13.27.58 pm.png (15.54 KiB) Viewed 3677 times
That's it. You now see your values in the live data.

If you want to use these values as your main AQI values, because say, you don't have a Davis Airlink or PurpleAir or any other device, then map these to Air0PM for PM10 and Air1PM for PM 2.5. These will then work using the AirQuality Davis modules that Brian has provided in his Weather34 template. I have both sensors running in comparison and the Airlink, which is additional sensor2 uses the mapping to Air0PM, Air1PM (and AirPM2) as you can see below:
Screenshot 2020-10-11 at 11 October 2020 13.46.53 pm.png
Screenshot 2020-10-11 at 11 October 2020 13.46.53 pm.png (178.91 KiB) Viewed 3673 times
Regards
Jason
https://jmfweather.uk | @jasonmfarrow
woolfg
Expert Boarder
Expert Boarder
Posts: 157
Joined: Wed Mar 20, 2019 4:01 pm
Contact:

Re: Integrating AirLink - no sensor data

Post by woolfg »

Hi Jason

I think I follow this

The only bit im not sure about is where to add the text to create the files - I dont know what my NOVA SDS011 script would be

I am also using lighttpd and I have three files style.css, index.html and aqi.js plus the json file

In which of the files do I append your first bit of code to create the.data files or do I create a new file

Kind Regards

Graham
woolfg
Expert Boarder
Expert Boarder
Posts: 157
Joined: Wed Mar 20, 2019 4:01 pm
Contact:

Re: Integrating AirLink - no sensor data

Post by woolfg »

Hi Jason

All sorted now - I figured it out

Thanks for your help

Regards

Graham
User avatar
jasonmfarrow
Gold Boarder
Gold Boarder
Posts: 249
Joined: Mon May 25, 2020 4:21 pm
Contact:

Re: Integrating AirLink - no sensor data

Post by jasonmfarrow »

Graham,

How are you collecting data today from your SDS011? Yes, it's connected to your RPi via USB, as is mine, but you must be running some kind of script (Python, Bash, etc.) to run the sensor and regularly capture the readings.

It is to this script that you add my sample code, not any of the lighttpd files. My full script (aqi.py) to run the sensor is below:

Code: Select all

#!/usr/bin/python -u
# coding=utf-8
# "DATASHEET": http://cl.ly/ekot
# https://gist.github.com/kadamski/92653913a53baf9dd1a8
from __future__ import print_function
import serial, struct, sys, time, json, subprocess, syslog, logging, logging.handlers
import requests

DEBUG = 0
CMD_MODE = 2
CMD_QUERY_DATA = 4
CMD_DEVICE_ID = 5
CMD_SLEEP = 6
CMD_FIRMWARE = 7
CMD_WORKING_PERIOD = 8
MODE_ACTIVE = 0
MODE_QUERY = 1
PERIOD_CONTINUOUS = 0

JSON_FILE = '/var/www/html/aqi.json'
AQI_FILE = '/home/pi/aqi.data'

MQTT_HOST = ''
MQTT_TOPIC = '/weather/particulatematter'

ser = serial.Serial()
ser.port = "/dev/ttyUSB0"
ser.baudrate = 9600

ser.open()
ser.flushInput()

byte, data = 0, ""

def dump(d, prefix=''):
    print(prefix + ' '.join(x.encode('hex') for x in d))

def construct_command(cmd, data=[]):
    assert len(data) <= 12
    data += [0,]*(12-len(data))
    checksum = (sum(data)+cmd-2)%256
    ret = "\xaa\xb4" + chr(cmd)
    ret += ''.join(chr(x) for x in data)
    ret += "\xff\xff" + chr(checksum) + "\xab"

    if DEBUG:
        dump(ret, '> ')
    return ret

def process_data(d):
    r = struct.unpack('<HHxxBB', d[2:])
    pm25 = r[0]/10.0
    pm10 = r[1]/10.0
    checksum = sum(ord(v) for v in d[2:8])%256
    return [pm25, pm10]
    #print("PM 2.5: {} μg/m^3  PM 10: {} μg/m^3 CRC={}".format(pm25, pm10, "OK" if (checksum==r[2] and r[3]==0xab) else "NOK"))

def process_version(d):
    r = struct.unpack('<BBBHBB', d[3:])
    checksum = sum(ord(v) for v in d[2:8])%256
    print("Y: {}, M: {}, D: {}, ID: {}, CRC={}".format(r[0], r[1], r[2], hex(r[3]), "OK" if (checksum==r[4] and r[5]==0xab) else "NOK"))

def read_response():
    byte = 0
    while byte != "\xaa":
        byte = ser.read(size=1)

    d = ser.read(size=9)

    if DEBUG:
        dump(d, '< ')
    return byte + d

def cmd_set_mode(mode=MODE_QUERY):
    ser.write(construct_command(CMD_MODE, [0x1, mode]))
    read_response()

def cmd_query_data():
    ser.write(construct_command(CMD_QUERY_DATA))
    d = read_response()
    values = []
    if d[1] == "\xc0":
        values = process_data(d)
    return values

def cmd_set_sleep(sleep):
    mode = 0 if sleep else 1
    ser.write(construct_command(CMD_SLEEP, [0x1, mode]))
    read_response()

def cmd_set_working_period(period):
    ser.write(construct_command(CMD_WORKING_PERIOD, [0x1, period]))
    read_response()

def cmd_firmware_ver():
    ser.write(construct_command(CMD_FIRMWARE))
    d = read_response()
    process_version(d)

def cmd_set_id(id):
    id_h = (id>>8) % 256
    id_l = id % 256
    ser.write(construct_command(CMD_DEVICE_ID, [0]*10+[id_l, id_h]))
    read_response()

def pub_mqtt(jsonrow):
    cmd = ['mosquitto_pub', '-h', MQTT_HOST, '-t', MQTT_TOPIC, '-s']
    print('Publishing using:', cmd)
    with subprocess.Popen(cmd, shell=False, bufsize=0, stdin=subprocess.PIPE).stdin as f:
        json.dump(jsonrow, f)

# all the subroutines are done. Now the main programme.

if __name__ == "__main__":
    cmd_set_sleep(0)
    time.sleep(1)
    cmd_firmware_ver()
    cmd_set_working_period(PERIOD_CONTINUOUS)
    time.sleep(1)
    cmd_set_mode(MODE_QUERY);
    while True:
        cmd_set_sleep(0)
        for t in range(10):
            values = cmd_query_data();
            if values is not None and len(values) == 2:
                print("PM2.5: ", values[0], ", PM10: ", values[1])
                time.sleep(3)

        # open stored data
        try:
            with open(JSON_FILE) as json_data:
                data = json.load(json_data)
        except IOError as e:
            data = []

        # check if length is more than 100 and delete first element
        if len(data) > 100:
            data.pop(0)

        # append new values
        jsonrow = {'pm25': values[0], 'pm10': values[1], 'time': time.strftime("%Y.%m.%d %H:%M:%S")}
        data.append(jsonrow)
        # syslog.syslog(syslog.LOG_INFO, data)

        # save it
        with open(JSON_FILE, 'w') as outfile:
            json.dump(data, outfile)

        if MQTT_HOST != '':
            pub_mqtt(jsonrow)
        
        cmd_set_sleep(1)  

        pm25d = str(int(values[0]*10))
        pm10d = str(int(values[1]*10))
        with open("/var/www/html/aqi25.data","w") as aqifile:
             aqifile.write(pm25d)
             aqifile.close()
        with open("/var/www/html/aqi10.data","w") as aqifile:
             aqifile.write(pm10d)
             aqifile.close()
Similar to the above is what's written here (http://www.geothread.net/particulates-s ... he-sds011/) along with other stuff.

I have this running every 30 minutes to capture current data. The SDS011 can be run continuously but it will shorten the life of the sensor.
Regards
Jason
https://jmfweather.uk | @jasonmfarrow
woolfg
Expert Boarder
Expert Boarder
Posts: 157
Joined: Wed Mar 20, 2019 4:01 pm
Contact:

Re: Integrating AirLink - no sensor data

Post by woolfg »

Hi Jason

Thats exactly what I have done

Thanks for your help its now working

Kind Regards

Graham
woolfg
Expert Boarder
Expert Boarder
Posts: 157
Joined: Wed Mar 20, 2019 4:01 pm
Contact:

Re: Integrating AirLink - no sensor data

Post by woolfg »

Hi Jason

Which script do you run every 30 minutes - is it aqi.py or the script on the meteobridge

I have tried setting up a cron job to run the python script every 30 minutes but it seems to run every minute

Regards

Graham
User avatar
jasonmfarrow
Gold Boarder
Gold Boarder
Posts: 249
Joined: Mon May 25, 2020 4:21 pm
Contact:

Re: Integrating AirLink - no sensor data

Post by jasonmfarrow »

I run my AQI.PY script every 30-mins. My crontab entry for this is below:

Code: Select all

*/30 * * * * python /home/pi/aqi.py > /home/pi/aqi.log 2>&1
This does xx:00 and xx:30 minutes of every hour, day, week, month. It also outputs any messages and errors to a log file.

The meteobridge aqijmf.plugin runs continuously. It is the sleep(300) in the plugin code that waits 5-minutes (300 seconds) before taking the next reading. The plugin does not align to any specific time slot. I could wait much longer but if I do then I might end up with a much bigger delay after the RPi updates the data.
Regards
Jason
https://jmfweather.uk | @jasonmfarrow
woolfg
Expert Boarder
Expert Boarder
Posts: 157
Joined: Wed Mar 20, 2019 4:01 pm
Contact:

Re: Integrating AirLink - no sensor data

Post by woolfg »

Hi Jason

Thanks - Ill give that a go

Thanks for the help

By the way does the fan on your SDS011 run continuously - mine seems to

Regards

Graham
Post Reply