Midnight Rollover values Today/Yesterday

[Mattk]

Following is the daily summaries from a VP2 as of around 3pm for the 23rd
Of main interest is the rainfall of 38.0mm/21st, 88.0mm/22nd & 1.4mm/23rd

19 18.6 26.6 12:40 10.6 4:50 2.0 2.3 0.0 1.3 14.5 17:30 SE
20 20.2 26.7 10:40 13.1 6:20 1.1 3.0 0.0 1.0 16.1 13:40 SE
21 19.5 23.9 11:40 15.4 23:00 0.4 1.6 38.0 3.4 32.2 19:10 SSE
22 17.3 19.3 20:30 15.2 2:40 1.0 0.1 88.0 4.0 27.4 15:00 SSE
23 17.8 19.1 7:20 16.9 0:10 0.2 0.0 1.4 2.7 16.1 1:30 SSE

Ok Meteobridge email from a "on day change" based on the following and all this basically for some testing so I can visually see what we get back, obviously nothing all that complicated
HV Rain [hh]:[mm]#Total [rain0total-act.1]mm\nToday [rain0total-daysum.1]mm\nLast Hour [rain0total-sum60.1]mm\nRate [rain0rate-act.1]mm/h\nYesterday [rain0total-ydaysum.1]mm

HV Rain 00:00
Total 1448.2mm
Today [rain0total-daysum.1]mm
Last Hour 0.4mm
Rate 0.0mm/h
Yesterday 37.8mm

The way I interpret this is
Day change time is ok 00:00 (midnight)
Total rain is spot on
Today rain is telling me there has been no rain today meaning @ midnight 00:00 the date is the 23rd (midnight on the 22nd)
Last hour of rain ok
Rate ok
Yesterday is pulling the rain total for the 21st. So at midnight on the 23rd is yesterday the 21st or the 22nd? IF today's rain is essentially -- (none) one would have thought Yesterdays rain would be 88mm?

Run again at 8:29am all all is good as expected

HV Rain 08:29
Total 1449.6mm
Today 1.4mm
Last Hour 0.0mm
Rate 0.0mm/h
Yesterday 88.2mm

What should be expected "on day change" through a midnight period such as this? Should a -24h switch be used instead of the daysum etc?

[admin]

According to the wiki (templates section), you can use "sum24h" for raintotal of last 24h.

That "daysum" returns 0 or an uninitialized value (which can be defaulted to 0 by adding ":0" to the variable) when new day starts at 0:00 should be obvious. When you want to avoid this clipping at 0:00 you should go with "sum24h". Then it does not matter if you check at 23:59 or 0:00.

[Mattk]

sum24h? Well that would work if the only time of interest was midnight but if doing a continuous/regular update for "Today" values then sum24h won't achieve the intent.

Some other daily selectors (dmin, dmax etc) also have a similar issue but have no work around at midnight which is the main result required for a daily summary. Simpledash.htm template has this issue.

[admin]

May be I got wrong what your "problem" is. When your posting is about some of the variables not being evaluated to numbers on 0:00, that is because there have not been any data recorded for the selected period. You can make the variables to return default number with the ":" option for cases like that. Again I am not sure what you are really looking for.

One more thought... Meteobridge does recomputing of values each time a new values is delivered. So at 0:00 there might not have been any rain data delivery for the new day. That make the rain0total variable undefined for the current day (might be defaulted to 0) and yesterdays rain might not have been assigned to that. When you get that one minute later (after having received the first rain data of today) everything should be fine.

[Mattk]

Yes I understand the ability to include 00, -- etc if/when there is no data but let me see if I can explain what I am observing.



Based on the above timeline/dates above MB follows the blue sequence and right on the dot of midnight on the 25th MB sees the time as 00:00 which to MB immediately says "Today" is now Friday and for that instant "Today" has no data as there has been no observations so any "day" selector (daysum/sumday, dmin, dmax, dmintime, dmaxtime etc) returns no values, MB still sees "Yesterday" as the 24th so what happens to the 25th, well it's sort of lost when in fact the 25th for all intent and purposes as of midnight on the 25th should be "Today" and return the values for the 25th but midnight returning 00:00 in typical time language is accepted as the next day. The blue sequence is currently the case if the email/FTP option of "on day change" is selected. Either Yesterday (eg 24) or Today (eg 26) is therefore out of sequence as what happens to the 25th?

Looking at the magenta sequence if midnight was deemed to be 24:00 and at that precice instant Today would still be the 25th, Yesterday would be the 24 and all would be good as in the next instant midnight effectively transitions from 24:00 (Today 25th) to 00:00 (Today 26th) but in affect the next day after the the 25th.

Using the MB "/public_html/dashboard.html" and "on day change" currently results in the following with no data available basically due to midnight 00:00 being seen as a new day without any data. Essentially Today (min) & Today (max) should be based on the full 24 hours of data leading up to midnight on that day, not the single instant at 00:00 the next day.

Sensor Current Today (min) Today (max)
Outdoor Temperature 17.9° C --° C --° C
Outdoor Humidity 95 % -- % -- %
Outdoor Dew Point 17.1° C --° C --° C
Wind Chill Temperature 17.9° C --° C --° C
Air Pressure 1015.8 hPa -- hPa -- hPa
Wind Direction 56°
Wind Speed (averaged) 0.0 m/s -- m/s -- m/s
Gust Speed 0.0 m/s -- m/s -- m/s
Rain rate: 0.0 mm/h sum -- mm
Solar Radiation 0 W/qm -- W/qm -- W/qm
UV Index -- uvi -- uvi -- uvi

Hopefully that explains where I am coming from on this one?

[glennpm]

I have been following this thread with interest since I noticed the same issue, first noticed with an incorrect rain total for "yesterday" a couple of weeks back.

[admin]

From my understanding of time there is no 24:00.
23:59:59.999 is a days last microsecond, followed by 0:00:00.000 which is first microsecond of the next day.

The problem you are facing is, that moving values from today to yesterday does not happen with a
time trigger at midnight, but is done on the first data event of the new day. Each time data comes in,
Meteobridge checks if a new day has approached and moves then todays values for this type of data
into the yesterday section.

I can think of introducing an external event that does a swapping of all data stores from today to yesterday at 0:00.
When I get it right this "problem" is just related to the very first few seconds of a new day. Having your
weather station continuously delivering data, everything will be fine at 0:01. I am still not convinced that
this small glimpse does really matter, but as said, I will think about a resolution to avoid hair splitting.

Thanks for your extensive explanation.

[Mattk]

This small glimpse does matter as what one is trying to achieve "on day change" is a historical snap shot of the day's data (24 hours) prior to midnight but this is not occuring.

Yes understand the general thinking behind 24:00 but in programming terms it is sometimes convenient to use/include 24:00 to distinguish the day before 00:00 as opposed to 00:00 also the beginning of the new day. Without 24:00 it is difficult to trigger actions precisely at midnight based on 23:59:00 -> 23:59:59.999 -> 00:00:00 -> 00:01 -> etc.

For the purposes here I don't believe there is a need to trigger exactly/absolutely at midnight (24:00/00:00) but "on day change" the correct day does need to be captured. Triggering "on day change" at 23:59:59 would at least achieve the expected result for Today & Yesterday with relevant selectors.

Each time data comes in, Meteobridge checks if a new day has approached and moves then todays values for this type of data into the yesterday section.

Just to confirm what I have observed with this. At midnight (00:00) "Today" is essentially the beginning of the following day and "Todays" values (those being the 24 hours prior to midnight) at this point have not been moved to Yesterday, hence there is a point where there is a full 24 hours between what MB is using as Today and Yesterday. An example of this gap between Today & Yesterday is outlined in the first post.

[admin]

This 24:00 discussion is going to nowhere for me, but I think I got your basic request.

[Mattk]

A little more feedback on this one. Have not fully tested all options but have confirmed apart from the "on day change" rollover any daysum, sumday type totals or using day type selectors (dmin, dmax etc) falling on midnight (periodical every full #minutes or full #hours which all will fall on midnight) display the same midnight issue.

[admin]

Yes, this is obvious and in sync with my explanation.

As a work around of this minor issue, please just avoid asking for today or yesterday values during the first couple of seconds after 0:00.

[Mattk]

Well I beg to differ that this is a minor issue.

Probably the closest "work around" (for some output only) that I can quickly see is a specific time option at 23:45. Are there others?

[admin]

At 0:15 (and all time slots after that) you will have yesterdays data in perfect shape (unless your sensors do not send less often then every 15 minutes, which would be rather unlikely).

If you don't mind, I would like stop discussing that for now (as all has been said imho) and doing a SW update next week instead, which you are invited to test. Thanks.

[glennpm]

At 0:15 (and all time slots after that) you will have yesterdays data in perfect shape (unless your sensors do not send less often then every 15 minutes, which would be rather unlikely).

If you don't mind, I would like stop discussing that for now (as all has been said imho) and doing a SW update next week instead, which you are invited to test. Thanks.

The update works good for me. Thanks!