Virtual Sensor using awk; help please

Moderator: Mattk

Post Reply
mikeo
Fresh Boarder
Fresh Boarder
Posts: 10
Joined: Thu Jan 07, 2016 11:03 pm

Virtual Sensor using awk; help please

Post by mikeo » Wed Jan 13, 2016 9:46 am

There is a reason behind this, and if you care about the reason please see viewtopic.php?f=16&t=10651

I don't know gawk/awk at all, and I've stumbled my was through the online documentation and help that's out there and I'm still running in to a problem. Here's what I'm dealing with:

I have a file: /var/local/heading that contains the heading of my wind vane.
I am trying to create a virtual sensor, wind1, using the data from wind0 and the value in /var/local/heading

I'm using this awk expression:
awk ' { printf ''%d %d %d %d'', $1, $5, $6, $7 }' /var/local/heading

since I can observe that the value in /var/local/heading is appearing as $1. I seem to have "lost" the stdin data from wind0 since $5, $6, and $7 show as "0 0 0000"
I incremented the string variable by 1 since the data from /var/local/heading appears to be read as $1. This is almost a blatant copy of the example code in the documentation on page #32-33.

Could someone please tell me how to fix the awk statement, or suggest another way to do this (calling a script?)

Here's my wind0 & wind1 sensor output:
20160113072510 wind0 0 0 0 0134
20160113072510 wind1 123 0 0 0000

123 is the only text in the file /var/local/heading

mikeo
Fresh Boarder
Fresh Boarder
Posts: 10
Joined: Thu Jan 07, 2016 11:03 pm

Re: Virtual Sensor using awk; help please

Post by mikeo » Thu Jan 14, 2016 10:30 am

OK, so I still suck at awk but I did discover that using "-" tells awk to read in from stdin. Now I've got the following statement that I'm playing with at the command line:

echo "20160114071407 wind0 1 2 3 0124" | awk ' { printf "%d %d %d %d", $3, $4, $5, $6 }' - /var/local/heading

and it returns:
1 2 3 1240 0 0 0

file file at /var/local/heading contains:
123

I also tried:
echo "20160114071407 wind0 1 2 3 0124" | awk ' { printf "%d %d %d %d", $7, $4, $5, $6 }' - /var/local/heading

and it returns:
0 2 3 1240 0 0 0

I'm banging my head against the wall. Anybody have any ideas on awk?

User avatar
YJB
Platinum Boarder
Platinum Boarder
Posts: 385
Joined: Thu Feb 19, 2009 5:53 pm
Location: Venhuizen, Netherlands
Contact:

Re: Virtual Sensor using awk; help please

Post by YJB » Thu Jan 14, 2016 11:28 am

Hmm, I've never seen an input variable trailing the awk command, so I'm not sure if awk is capable to use it as $7.

I'm not sure what you are trying to do, but I would probably use something like this:

Code: Select all

 echo "20160114071407 wind0 1 2 3 0124 `cat /var/local/heading`" | awk {'printf "%d %d %d %d", $7, $4, $5, $6 '}
Of course this is just generic awk syntax, not sure how that fits in your script.

Ysbrand

mikeo
Fresh Boarder
Fresh Boarder
Posts: 10
Joined: Thu Jan 07, 2016 11:03 pm

Re: Virtual Sensor using awk; help please

Post by mikeo » Thu Jan 14, 2016 7:06 pm

Ysbrand, thank you. That makes it work on the command line, but there is something weird going on with the syntax/formatting of $6. I'll have to figure out the syntax later, I want to get stdout to return the correct data first. Here's a bit more detail of what I'm doing here:

The

Code: Select all

echo "20160114071407 wind0 1 2 3 0124"
is to replicate stdin passing to a "conversion" in Virtual Sensors so I can test via the command line
The

Code: Select all

awk ' { printf "%d %d %d %d", $3, $4, $5, $6 }'
is the code I have in the "conversion" in Virtual Sensors

There are some (minor) differences called out by Boris in how to pass syntax via http, but I don't think that's relevant right now.

I now need to figure out how to start a conversion expression reading /var/local/heading . I was trying to avoid UUOC, so I was able to accomplish the same thing with:

Code: Select all

echo 20160114071407 wind0 1 2 3 0124 `< /var/local/heading` | awk {'printf "%d %d %d %d", $7, $4, $5, $6 '}
It returns:
123 2 3 124

I don't know the http method that is being used in meteohub, so it appears that

Code: Select all

< /var/local/heading
can't be used to start an expression for evaluation, and it appears to overwrite stdin

Here's the whole expression I attempted to use in Conversion in the Virtual Sensors section

Code: Select all

`< /var/local/heading` | awk {'printf ''%d %d %d %d'', $7, $4, $5, $6 '}

User avatar
YJB
Platinum Boarder
Platinum Boarder
Posts: 385
Joined: Thu Feb 19, 2009 5:53 pm
Location: Venhuizen, Netherlands
Contact:

Re: Virtual Sensor using awk; help please

Post by YJB » Thu Jan 14, 2016 8:03 pm

I probably miss the point, but why don't you use "cat" to read the file and pipe it to awk?

mikeo
Fresh Boarder
Fresh Boarder
Posts: 10
Joined: Thu Jan 07, 2016 11:03 pm

Re: Virtual Sensor using awk; help please

Post by mikeo » Thu Jan 14, 2016 10:13 pm

UUOC and I can't get cat to work via the "conversion" input. I'm trying to figure out if the / is being escaped correctly in the http get/put method.

mikeo
Fresh Boarder
Fresh Boarder
Posts: 10
Joined: Thu Jan 07, 2016 11:03 pm

Re: Virtual Sensor using awk; help please

Post by mikeo » Fri Jan 15, 2016 3:18 am

OK, so I made some more progress, but the problem still seems to lie in my lack of knowledge of awk. I used this in the "conversion" field:

Code: Select all

< /var/local/heading awk {'printf ''%d %d %d %d'', $1, $5, $6, $7 '}
and was able to get the following output as wind1 in the Inspect Data view:
20160115005726 wind0 1 2 3 0134
20160115005726 wind1 123 0 0 0000

So, it appears that I can get /var/local/heading in to wind1, so now I just need to get the rest of the data formatted correctly. Trying to simulate this action at the command line as:

Code: Select all

echo "20160115005726 wind0 0 0 0 0134" | < /var/local/heading awk {'printf "%d %d %d %d", $1, $4, $5, $6 '}
returns:
123 0 0 0

when I expect:
123 2 3 0134

mikeo
Fresh Boarder
Fresh Boarder
Posts: 10
Joined: Thu Jan 07, 2016 11:03 pm

Re: Virtual Sensor using awk; help please

Post by mikeo » Thu Jun 08, 2017 8:00 pm

After umpteen months of on and off trying to hack this together I finally had someone help me and now have a working virtual sensor that pull data from a file and from an existing sensor. The magic line of code that made it work is:

Code: Select all

awk 'NR==FNR{b=$1;next}{print b, $4,$5,$6}' /var/local/heading -
The file /var/local/heading is updated by cron once a minute as it reads from the attached magnetometer/compass; I used pre-existing python code with the RaspberryPi to read the heading data. The "-" character at the end is very important as it tells awk to read from stdin and not just use the input file specified as the only input.

If anyone cares, the station is reporting publicly here:
https://www.wunderground.com/personal-w ... =KCADOBBI2

Post Reply