Min/Max Historical Data: sensors of added station not shown **solved**

[Gyvate]

Meteobridge 5.4 (Jul 26 2022, build 2826), FW 1.3 (on RPi4)
I recently added a 3rd station what load-wise isn't an issue for a RPi4B-2GB.
In the Raw Sensor data tab all sensors from all stations are shown - HOWEVER in the Min/Max Historical Data tab quite of few of the sensors of the 3rd station are missing even though they have already provided data.

Apart from the fact that I never understood how this table is sorted (roughly 0 --> 1 --> 2, but some 0 and 1 (station #) appear out of alphabetical sorting sequence, sometimes only at the end of the list), the 3rd station data appear truncated - only sensors starting with "th" are shown - no air quality, no wind, no solar, no lightning, just only sensors whose name starts with "th". Station 0 and 1 sensors are complete.

So I'm wondering where this comes from - I would expect to see them all - or is there a limit for the possible line items in that list ?

Not sure if this can provide a hint - the 3rd station added is an Ecowitt GW2001 (GW2000 + WS90 with a second rain gauge connected [rain1 sensor]) running with the MB GW1000 API interface/driver.
That same station runs flawlessly on a test MB Pro as the only station. There, for station 0, all sensors are shown in the Min/Max history (build 14798).

[Gyvate]

no reaction by @admin
maybe I should add a picture showing the issue - only sensors starting with "th" in the list for station 2, station 0 and 1 complete.

MB-RPi-sensors-st2-missing.JPG (338.51 KiB) Viewed 1838 times

[admin]

I will have to check code - which I can't right now as being on travel - to answer about the limit on number of lines. However, you can check via template evaluation if the expected but not displayed sensors are providing data.
You are really pushing hard with that amount of sensors

[Gyvate]

They do provide data in the live raw sensor data. But there's also a lot of calculated values from the sensor readings.
And in a custom made template supposed to display some of the not shown entries in the Min/Max History are displayed.

So my assumption is that there is some limitation of the number of list entries.
Why they start with "th" only may be circumstantial.
I did a count now and it appears that even for station 0 not all sensors are shown in Min/Max.
As you can tell from complete shown list, there are
29 entries station 0
57 entries station 1
33 entries station 2
0 entries station 9
----
119
There should be
57 entries for station 0
57 entries for station 1
59 entries for station 2
7 entries for station 9
----
180

the raw sensor data show 25 sensors for each station (except station 2: 26 rain0 and rain1) and 7 for station 9 and all providing data.

Maybe creating a separate tab for each station (except station 9, this could be together with station 0) for the Min/Max History could help and could also create a better overview.

[Gyvate]

You are really pushing hard with that amount of sensors

Maybe - but what if someone uses all possible five (5) stations with extra sensors ..... ??

As for Ecowitt (clone) stations there is a list of maximum number of sensors per console - MB should be able to serve them - also for 3-5 stations

Ecowitt-sensor-max-number-table.JPG (142.57 KiB) Viewed 1692 times

For most consoles it's 56 sensors out of which some are 2-in-1 plus combo sensors (WH32B/indoor 3-in-1, WH45 5-in-1); => the max number of possible observations is much bigger: +/- 64 - in addition come the derived/calculated values like dew point etc. So the max number of observations for an Ecowitt station with all possible sensors will be around 80. 80 observations x 5 stations = 400 (plus station #9).

The max/min history list should be able to accomodate 400 entries !! (maybe a split list across tabs would make for a better user readability)

Any progress there ?
When even for station 0 not all sensors are shown, something must also be "fishy" at least with the sorting of that list ...

[Mattk]

Obviously with anything when push comes to shove not everything remains practical. To run 400 sensors appears to be more of an obsession, just because one can, doesn't mean one should

[Gyvate]

Obviously with anything when push comes to shove not everything remains practical. To run 400 sensors appears to be more of an obsession, just because one can, doesn't mean one should

Obviously ??? For whom ?
When you design a piece of software, you make sure it can manage the available and promised features. As simple as that.

What is practical or not is not for the forum moderator to judge but is a single case situation and decision.
I think it would rather be appropriate to keep the ball down here.

What you call an obsession is using a promised feature. And not a particularly tricky or difficult one.
All other portions/features/aspects of the software can cope with that many sensors/observations.
It's just that the Min/Max history is not (yet) properly developed/designed.

When you design software, you look for the orders of magnitude - and here, with 5 possible stations (If one follows your line of argumentation, the possibility of using up to five stations was only created for people with obsessive behaviour ), you see an upper limit of 400. Knowing that, you can make a proper design (e.g. define the size of an array and the needed RAM allocation).

It's not that the database needs to be redesigned or the like - just a list needs to contain all observations and it needs to be sorted in a logical manner (or offer different sorting for different columns). That's a standard task for every programmer. So it's not that I'm asking for something impossible or difficult. Programming sorted lists with group change was one of the first things I learnt 40+ years ago when becoming a certified programmer.

I've been producing/developing professional software for 20+ years, and that type of reasoning for not providing a properly functioning solution would never have been accepted by the customers. You want to work with many items ? You are obsessed. Here you go ...
"Thank you very much ! We've been working with you two times so far: the first and [at the same time] the last time."

Or you are telling me I should run 5 x MB on RPi to get all the sensors/observations of the five stations (in my case only 3 though) shown in the min/max history ? Or I should not use more than one station ??? Right now the list can even not yet depict all observations of two of my stations.

[admin]

Please let's stay calm. It is just about technology, nothing personal.

Meteobridge supports multiple stations and lots of sensors per station. But it is limited HW and handling of 400 sensors is nowhere promised nor reasonable request. I am fine to find out what goes on with missing sensors. The provided list on "Live Data" tab is not much sorted, it is in order how sensors show up. To invent something like multi tabbed lists for edge cases like yours is not what I would like to do because most users with a more standard setup prefer to have it all on one page.

[admin]

When I get my code right, the limit of rows is 1000 but there is also a limit of html data, which is 1MB for the table on MB PRO, NANO SD, RPI and 200k otherwise. Could you please inspect how big the html data is in your specific situation? While RAM is no issue on the RPI and even on the NANO it is tight on the PRO.

[Gyvate]

When I get my code right, the limit of rows is 1000 but there is also a limit of html data, which is 1MB for the table on MB PRO, NANO SD, RPI and 200k otherwise. Could you please inspect how big the html data is in your specific situation? While RAM is no issue on the RPI and even on the NANO it is tight on the PRO.

How and where do I do this ?

And a PRO cannot handle two of my stations, only one. Otherwise it collapses. That's why I moved stations to the RPi and have my PROs only run single Ecowitt stations (which have by far not the maximum sensors per console attached).

If RAM is an issue, why not have a separate solution (=allowing for more RAM) for NANO and RPi - as already done e.g. with the Weather dashboards

[Gyvate]

Could you please inspect how big the html data is in your specific situation? While RAM is no issue on the RPI and even on the NANO it is tight on the PRO.

If the number of bytes of the html code of the min/max history page are meant, they are:
307,301 bytes

[admin]

Newest release does detect when sensors are omitted because of space restrictions and tells so in the last row. Please give it a try and tell how many are displayed as missing. May be I can adjust limits a bit.

[admin]

I just released another update that gives 5 times the headroom when using a NANO or RPI. If the problem was related to this limit (which is so far just a hypothesis) then it should be solved by this.

[Gyvate]

nothing to be seen with 2844
the picture is the same as before - or did I look in the wrong placce ?

2844-min-max-last_row.JPG (50.5 KiB) Viewed 1642 times

[Gyvate]

tried with 2845 - same picture (meaning as before in the thread - still not all station 0 observations, all station 1 observations, only station 2 observations starting with "th" and no station 9 values)
can't see any line telling which sensors are missing ...
so maybe it's not space restrictions ...
if you want access to the system, let me know