meteohub.de

This post is mainly to document the issue for Boris and back the observations up with some data
SW Version: Meteobridge 4.3 (May 28 2020, build 13497), FW 1.4

I'm running an hourly export of the minute sensor values. Until upgrade to FW 1.5.9 everything was fine. After the upgrade line gaps (lines with missing minute data sensor values) appeared in the export. However the corresponding data were sent and stored to Ecowitt.net. Most of the gaps are 5 minutes long, some up to 8 minutes and one very large gap of more than an hour also shows in the MB Pro database ("archive").
The big one is shown below,

From discussions in the wxnet forum where I first published the issue I hear that there are more people who have similar observations after updating the GW1000 firmware to 1.5.9.

In order not to lose data I have switched to my GW1000 backup device which still runs on FW 1.5.6 and which now provides all minute data without gaps including soil moisture #1-#3, extra temp/humidity #1, and since MB Pro update from 28-May also my PM2.5 #1 and #2.

The fact that missing data in the MB Pro DB still show on Ecowitt.net suggests that we have here an issue with how MB handles the updated Ecowitt API which came with FW 1.5.9. Still not to be excluded that the isuue is on the Ecowitt end (or with both).

export data below - the two blank gaps inside a line come from the air0pm and air1pm sensors. Since 28-May upgrade they are filled with sensor data.
# date, time, temperature[C], humidity[%], dew point[C], sealevel pressure[hPa], avg wind speed[m/s], gust speed[m/s], winddir, rainfall00, Solar [W/m2], PM25(1) [ug/m3], PM25(2), extra-temp1 [°C], extra-hum [%], soil moisture (1) [%], SM2, SM3
2020-05-16,16:00,23.5,29,4.4,1018.7,0.5,1.0,190,0.0, 686.00, , , 25.2, 27, 47, 51, 41
2020-05-16,16:01,23.4,29,4.2,1018.9,0.5,1.5,184,0.0, 683.33, , , 25.2, 28, 47, 51, 41
2020-05-16,16:02,23.0,29,4.1,1018.6,0.0,0.5,143,0.0, 680.00, , , 25.3, 27, 46, 51, 41
2020-05-16,16:03,22.7,30,4.0,1018.8,0.2,0.5,146,0.0, 678.00, , , 25.3, 28, 46, 50, 41
2020-05-16,16:04,22.6,29,3.7,1018.8,0.0,0.0,152,0.0, 676.50, , , 25.4, 27, 47, 51, 41
2020-05-16,16:05,22.6,29,3.5,1018.8,0.0,0.5,137,0.0, 673.50, , , 25.4, 28, 47, 51, 41
2020-05-16,16:06,22.8,28,3.3,1018.8,0.1,0.5,129,0.0, 669.50, , , 25.4, 29, 47, 50, 41
2020-05-16,16:07,23.2,28,3.7,1018.8,0.0,0.0,129,0.0, 665.67, , , 25.4, 28, 47, 51, 41
2020-05-16,16:08,23.6,28,4.0,1018.7,0.1,0.5,140,0.0, 662.67, , , 25.4, 27, 46, 51, 41
2020-05-16,16:09,23.7,29,4.6,1018.6,0.2,1.0,154,0.0, 659.67, , , 25.3, 28, 46, 50, 41
2020-05-16,16:10,23.7,29,4.6,1018.6,0.2,1.0,154,0.0, 659.67, , , 25.3, 28, 46, 50, 41
2020-05-16,16:11,23.7,29,4.6,1018.6,0.2,1.0,154,0.0, 659.67, , , 25.3, 28, 46, 50, 41
2020-05-16,16:12,23.7,29,4.6,1018.6,0.2,1.0,154,0.0, 659.67, , , 25.3, 28, 46, 50, 41
2020-05-16,16:13,23.7,29,4.6,1018.6,0.2,1.0,154,0.0, 659.67, , , 25.3, 28, 46, 50, 41
2020-05-16,16:14,23.7,29,4.6,1018.6,0.2,1.0,154,0.0, 659.67, , , 25.3, 28, , 50, 41
2020-05-16,16:15,,,,,,,,, , , , , , , ,
2020-05-16,16:16,,,,,,,,, , , , , , , ,
2020-05-16,16:17,,,,,,,,, , , , , , , ,
2020-05-16,16:18,,,,,,,,, , , , , , , ,
2020-05-16,16:19,,,,,,,,, , , , , , , ,
2020-05-16,16:20,,,,,,,,, , , , , , , ,
2020-05-16,16:21,,,,,,,,, , , , , , , ,
2020-05-16,16:22,,,,,,,,, , , , , , , ,
2020-05-16,16:23,,,,,,,,, , , , , , , ,
2020-05-16,16:24,,,,,,,,, , , , , , , ,
2020-05-16,16:25,,,,,,,,, , , , , , , ,
2020-05-16,16:26,,,,,,,,, , , , , , , ,
2020-05-16,16:27,,,,,,,,, , , , , , , ,
2020-05-16,16:28,,,,,,,,, , , , , , , ,
2020-05-16,16:29,,,,,,,,, , , , , , , ,
2020-05-16,16:30,,,,,,,,, , , , , , , ,
2020-05-16,16:31,,,,,,,,, , , , , , , ,
2020-05-16,16:32,,,,,,,,, , , , , , , ,
2020-05-16,16:33,,,,,,,,, , , , , , , ,
2020-05-16,16:34,,,,,,,,, , , , , , , ,
2020-05-16,16:35,,,,,,,,, , , , , , , ,
2020-05-16,16:36,,,,,,,,, , , , , , , ,
2020-05-16,16:37,,,,,,,,, , , , , , , ,
2020-05-16,16:38,,,,,,,,, , , , , , , ,
2020-05-16,16:39,,,,,,,,, , , , , , , ,
2020-05-16,16:40,,,,,,,,, , , , , , , ,
2020-05-16,16:41,,,,,,,,, , , , , , , ,
2020-05-16,16:42,,,,,,,,, , , , , , , ,
2020-05-16,16:43,,,,,,,,, , , , , , , ,
2020-05-16,16:44,,,,,,,,, , , , , , , ,
2020-05-16,16:45,,,,,,,,, , , , , , , ,
2020-05-16,16:46,,,,,,,,, , , , , , , ,
2020-05-16,16:47,,,,,,,,, , , , , , , ,
2020-05-16,16:48,,,,,,,,, , , , , , , ,
2020-05-16,16:49,,,,,,,,, , , , , , , ,
2020-05-16,16:50,,,,,,,,, , , , , , , ,
2020-05-16,16:51,,,,,,,,, , , , , , , ,
2020-05-16,16:52,,,,,,,,, , , , , , , ,
2020-05-16,16:53,,,,,,,,, , , , , , , ,
2020-05-16,16:54,,,,,,,,, , , , , , , ,
2020-05-16,16:55,,,,,,,,, , , , , , , ,
2020-05-16,16:56,,,,,,,,, , , , , , , ,
2020-05-16,16:57,,,,,,,,, , , , , , , ,
2020-05-16,16:58,,,,,,,,, , , , , , , ,
2020-05-16,16:59,,,,,,,,, , , , , , , ,
2020-05-16,17:00,,,,,,,,, , , , , , , ,
2020-05-16,17:01,,,,,,,,, , , , , , , ,
2020-05-16,17:02,,,,,,,,, , , , , , , ,
2020-05-16,17:03,,,,,,,,, , , , , , , ,
2020-05-16,17:04,,,,,,,,, , , , , , , ,
2020-05-16,17:05,,,,,,,,, , , , , , , ,
2020-05-16,17:06,,,,,,,,, , , , , , , ,
2020-05-16,17:07,,,,,,,,, , , , , , , ,
2020-05-16,17:08,,,,,,,,, , , , , , , ,
2020-05-16,17:09,,,,,,,,, , , , , , , ,
2020-05-16,17:10,,,,,,,,, , , , , , , ,
2020-05-16,17:11,,,,,,,,, , , , , , , ,
2020-05-16,17:12,,,,,,,,, , , , , , , ,
2020-05-16,17:13,,,,,,,,, , , , , , , ,
2020-05-16,17:14,,,,,,,,, , , , , , , ,
2020-05-16,17:15,,,,,,,,, , , , , , , ,
2020-05-16,17:16,,,,,,,,, , , , , , , ,
2020-05-16,17:17,,,,,,,,, , , , , , , ,
2020-05-16,17:18,,,,,,,,, , , , , , , ,
2020-05-16,17:19,,,,,,,,, , , , , , , ,
2020-05-16,17:20,,,,,,,,, , , , , , , ,
2020-05-16,17:21,,,,,,,,, , , , , , , ,
2020-05-16,17:22,,,,,,,,, , , , , , , ,
2020-05-16,17:23,22.1,29,3.2,1018.4,0.6,1.5,169,0.0, 210.00, , , 23.5, 26, 46, 50, 38
2020-05-16,17:24,21.9,29,3.2,1018.3,0.5,1.0,180,0.0, 182.00, , , 23.5, 27, 46, 50, 38
2020-05-16,17:25,21.3,30,2.9,1018.4,0.5,1.0,156,0.0, 251.50, , , 23.5, 28, 46, 51, 37
2020-05-16,17:26,21.0,30,2.5,1018.3,0.1,1.0,159,0.0, 351.75, , , 23.5, 27, 46, 51, 38
2020-05-16,17:27,20.9,30,2.7,1018.5,0.7,1.5,189,0.0, 231.33, , , 23.4, 26, 46, 51, 38

sensor data uploaded to ecowitt.net (sequence inverse)
gaps in the MB Pro database for 5 sample sensors at from 16:00-17:30 CEST

I'm experiencing the same issue with Meteobridge Standard and Ecowitt GW1000, so this issue is not limited to Pro.

Well the issue is a bit different. I'm not looking to export but rather just upload to other services and I notice missing gaps of 5 or 8 minutes randomly, like 1 or 2 per hour.

Perhaps these issues are unrelated but I think the stem from some compatibility issue with GW1000 firmware 1.5.9.

Who is a fault? Ecowitt or Meteobridge?

One would think that if the GW1000 works ok with FW 1.5.6 and not 1.5.9 under the same latest MB version then that points to the updated Ecowitt API.

I can only image it must get really annoying for developers when other parties make changes which crashes previously working links and then have to turn around and fix the other parties changes? I suppose that's the nature of the beast. I have seen this occur in other fields where opposing manufacturers who are supposedly following common "standards" slightly tweak a few things just to be different

There is good reason for the new Ecowitt firmware 1.5.9 to be different. Ecowitt's new firmware 1.5.9 does change the API because it supports new sensors that are not for sale yet. I've sent Boris the new GW1000 API documentation. But the fact remains that both Cumulus MX and Weather-Display that both also use the GW1000 API are not experiencing problems. At least I've not seen any reports by their users on their forums. The difference between weather software could be in the different way that they receive and parse the API response from the GW1000 or perhaps they quickly made adjustments in their code to better support firmware 1.5.9.

I've analysed the GW1000 API and its new response and compared firmware 1.5.9 to previous versions. The API data response from the GW1000 is noticeably bigger (considerably more 2 byte data sets sent) even though I'm using the same sensors as before. It just means that the GW1000 is now sending more data with the latest firmware. I also believe that it now sends a rain event that was missing the the API before (your software had to calculate it before). I've not totally looked in depth into all the hex data that it spews out in detail. Just a side by side comparison is enough to realize that with 1.5.9 the API response is much bigger. We are getting more information from the GW1000 than before even without adding sensors. This means that there is more to leverage if software takes advantage of these added capabilities. It also makes it more plausible that perhaps just the way the Meteobridge parses through this added data may need an update. There are many ways to parse received data and how to handle that incoming payload. Software developers are free to manage that as they wish. Some software for example may have predefined a way to store that payload into RAM and the code expects the payload to be of a certain defined length and then it tackles the parts (parsing) from the end or from the beginning or they pick out certain parts just by matching search strings in identification of each sensor. It is because of the flexibility that the software developer has that sometimes a particular software may or may not be impacted by API changes. In this case I really think that the ball is in Meteobridge's court to determine how to handle this new 1.5.9 version. At least I think the attempt should be made to see how to handle this new API format, and then Meteobridge's developer could perhaps be in the best position to even discover an error in the API or maybe not an error but make a suggestion for improving the API if deemed necessary.

New firmware 1.6.0 from Ecowitt addresses data loss issue bug that Boris discovered. Previous GW1000 firmware 1.5.9 was not allowing the Meteobridge to keep a constant connection with the GW1000.

Please give latest Meteobridge SW a try.

With FW 1.6.0 no more gaps occur (tested over 48 h with hourly minute-based exports).
And this with SW Version: Meteobridge 4.3 (May 28 2020, build 13497), FW 1.4.
I.e. without upgrading to the 11-14 June 2020 release.

With FW 1.5.6 no gaps - with FW 1.5.9 random gaps occur (API changed between 1.5.8 and 1.5.9) - with FW 1.6.0 no more gaps.
No change in the MB (PRO) SW version.

This shows imho that the issue was firmware based and not MB (PRO) based.
This does not mean that the issue could not have either been "fixed" from the MB end.

And - this won't keep me from upgrading as my lightning sensor is on its way from China to me.

I will close the thread/mark as solved.

....This does not mean that the issue could not have either been "fixed" from the MB end....

And I don't think every developer should be chasing the tail of manufacturers that stuff up their own firmware updates, a fix to fix a stuff up will eventually come back to be just that, a stuff up

Mattk wrote: ↑Mon Jun 15, 2020 10:26 pm

....This does not mean that the issue could not have either been "fixed" from the MB end....

And I don't think every developer should be chasing the tail of manufacturers that stuff up their own firmware updates, a fix to fix a stuff up will eventually come back to be just that, a stuff up

Mattk, its now a common practice to blame someone or something else for the mistake unfortunately.