meteohub.de

I'm having an issue with 2 of my graphs, wind and rain. I get graphs like this:


(hope you can see it, the graph is transparant )

more problems over here: http://hetweerinbleiswijk.nl/jaaroverzi ... zicht-2014 and here http://hetweerinbleiswijk.nl/jaaroverzi ... zicht-2015
(in the 'neerslag' aka rain section)

some values are waaay to large. now when i check the raw file of that month (2014-10)

root@meteohub:/data/weather/201410$ tail -n 20 raw | grep rain0
20141031235839 rain0 0 0 49599
20141031235926 rain0 0 0 49599
root@meteohub:/data/weather/201410$ head -n 20 raw | grep rain0
20141001000014 rain0 0 0 48768
20141001000101 rain0 0 0 48768

the total of the rain0 looks to me like it should be 49599 - 48768 = 831mm. now when I check the monthly total:

root@meteohub:/data/weather/201410$ cat ../rain0-month1 | grep 2014
...
20140801000000 0.8 0.0 90.9 132.90 22
20140901000000 0.0 0.0 1.8 12.40 3
20141001000000 0.5 0.0 588.8 4976.40 7
20141101000000 9.9 0.0 180.8 9867.10 4
20141201000000 0.1 0.0 128.8 114.50 11
root@meteohub:/data/weather/201410$

the value is almonst 5000mm? where does that value come from? how does meteohub do the total recomputation? hope you can shed some light on this. if you want to, i can send you the raw file for inspection

btw: been using your program on a NSLU2 since 2008. working ever since! well, i do have to change the usb stick once in a while, but ey, it 'just works'! good job!

no idea how to fix this?

Hi,

I can't tell you where these values are coming from. What I can tell you is that you will need to:
- Remove the offending values from the raw data files
- Followed by a recomputation, as this will clear the incorrect aggregates

You can edit the raw data by going to the "Inspect data" menu, or edit the raw files directly. I would chose the second option only if you understand what you are doing. In that case a backup upfront might be a good idea.

Ysbrand

thanks for your reply! the problem is that i don't know where the values come from either :/ there aren't any wrong or ridiculous high values in any of the raw files, only in the cummulation files. changing them won't do anything since they will be overwritten when a recalculation starts again

Yes, I agree it looks funny, but I would not be surprised if there is an odd (read: corrupt) value somewhere in the middle of the month.

Currently you are listing the start and end of the month values, but I would recommend to look at the entire month. For analysis pruposes, I would grep for rain0 redirect it to a file, import the file in xls and sort on the total. Once you've indentified an issue you can then go back to the original file and fix it.

YJB wrote:For analysis pruposes, I would grep for rain0 redirect it to a file, import the file in xls and sort on the total. Once you've indentified an issue you can then go back to the original file and fix it.

ah! that's the idea i was looking for! i can do an easy sort in excel. will give that a shot today. perfect

hmm. when importing and sorting the file from 10-2014 in excel, i do get some high values as shown in the attachment.

if only i knew how to map these values onto the values of the bold line below

20140901000000 0.0 0.0 1.8 12.40 3
20141001000000 0.5 0.0 588.8 4976.40 7
20141101000000 9.9 0.0 180.8 9867.10 4

20141001000000 0.5 0.0 588.8 4976.40 7 is referring to the entire month.

Keep in mind that the file rain0-month1 will just contain a single line for each and every month. So this is your statiscal information that has bene collected for that month.

Most likely are the 0 values and 20808 values confusing the aggregate functionality. It appears to me that those entries are not correct and should be removed from the raw file.

YJB wrote:Most likely are the 0 values and 20808 values confusing the aggregate functionality. It appears to me that those entries are not correct and should be removed from the raw file.

I analysed all raw files in excel and removed all the lines containing ridiculous values. Have to wait a few days for the results though (the system is running on a NSLU2 with weather data from 2008 up until now. recomputation takes around 5 days )