issues with Nano SD/homeweather

[pwilman]

My Nano SD is doing some funny things at the moment. The SD charts are being upload correctly (www.paulwilman.com/graphs.php), but my homeweather template (www.paulwilman.com) is not working. It went offline about 3am.... MBrealitimeupload.php doesnt seem to be uploaded, and if it is an empty file is produced of 0KB.

A reboot of the Nano gets it all working again. It has done this a few times over the last week....

From the system log of the Nano It looks like the Nano is continually doing something....I dont like the sound of a "child process termination signal.

logger (23.11.2019 05:14:42): databases connected. Update interval 3600 seconds.
logger (23.11.2019 05:14:42): data logger (version 5.1b, build 2668) started.
logger (23.11.2019 05:14:42): error (22) while doing bind (port 22222): Invalid argument
logger (23.11.2019 05:14:42): killed process "2112/sh" listening to port 5551.
logger (23.11.2019 05:14:42): killed process "2113/meteosys" listening to port 5552.
logger (23.11.2019 05:14:44): data logger stopped.
logger (23.11.2019 05:14:44): databases disconnected.
logger (23.11.2019 05:15:12): databases connected. Update interval 3600 seconds.
logger (23.11.2019 05:15:12): data logger (version 5.1b, build 2668) started.
logger (23.11.2019 05:15:13): connect station 0 (Vantage via USB serial).
logger (23.11.2019 05:15:13): connect station 9 (System Data via Plug-in).
logger (23.11.2019 05:15:13): wakeup sent.
logger (23.11.2019 05:15:16): waiting for sensors (wind0, rain0, th0, thb0) to provide data.
logger (23.11.2019 05:15:16): services paused.
logger (23.11.2019 05:15:25): execution of setup script failed (console is not responding).
logger (23.11.2019 05:15:25): internal logger activated.
logger (23.11.2019 05:15:25): syncing.
logger (23.11.2019 05:15:25): gettime request sent.
logger (23.11.2019 05:15:25): communication with Davis console/envoy seems to be stalled. Please power cycle the console/envoy by (1) disconnect from AC, (2) remove batteries and (3) disconnect USB. Wait a minute and power on again. Weak batteries need to be replaced.
logger (23.11.2019 05:15:35): gettime request sent.
logger (23.11.2019 05:15:35): no Vantage time offset.
logger (23.11.2019 05:15:35): eeprom request sent.
logger (23.11.2019 05:15:35): rain collector size 0.2 mm per tick.
logger (23.11.2019 05:15:35): transmitter 1 active as station type #0 "ISS".
logger (23.11.2019 05:15:35): firmware version request sent.
logger (23.11.2019 05:15:35): firmware version 1.90 (supports LOOP2)
logger (23.11.2019 05:15:35): no reading of Vantage's internal logger.
logger (23.11.2019 05:15:40): sensors are now providing data.
logger (23.11.2019 05:15:40): services started.
logger (23.11.2019 05:35:18): child process received termination signal (11) in functions: termination_handler < inetupload_getline < inetupload_getline < inetupload_getline < inetuploa...
logger (23.11.2019 05:35:18): disconnect station 0 (Vantage): exit
logger (23.11.2019 05:35:18): disconnect station 9 (System Data): exit
logger (23.11.2019 05:35:19): databases connected. Update interval 3600 seconds.
logger (23.11.2019 05:35:19): data logger (version 5.1b, build 2668) started.
logger (23.11.2019 05:35:20): connect station 0 (Vantage via USB serial).
logger (23.11.2019 05:35:20): connect station 9 (System Data via Plug-in).
logger (23.11.2019 05:35:20): wakeup sent.
logger (23.11.2019 05:35:23): waiting for sensors (wind0, rain0, th0, thb0) to provide data.
logger (23.11.2019 05:35:23): services paused.
logger (23.11.2019 05:35:23): no need to update console date/time.
logger (23.11.2019 05:35:24): setup script executed.
logger (23.11.2019 05:35:24): internal logger activated.
logger (23.11.2019 05:35:24): syncing.
logger (23.11.2019 05:35:34): gettime request sent.
logger (23.11.2019 05:35:34): no Vantage time offset.
logger (23.11.2019 05:35:34): eeprom request sent.
logger (23.11.2019 05:35:34): rain collector size 0.2 mm per tick.
logger (23.11.2019 05:35:34): transmitter 1 active as station type #0 "ISS".
logger (23.11.2019 05:35:34): firmware version request sent.
logger (23.11.2019 05:35:34): firmware version 1.90 (supports LOOP2)
logger (23.11.2019 05:35:34): no reading of Vantage's internal logger.
logger (23.11.2019 05:35:40): sensors are now providing data.
logger (23.11.2019 05:35:40): services started.
logger (23.11.2019 05:55:25): child process received termination signal (11) in functions: termination_handler < inetupload_getline < inetupload_getline < inetupload_getline < inetuploa...
logger (23.11.2019 05:55:25): disconnect station 0 (Vantage): exit
logger (23.11.2019 05:55:25): disconnect station 9 (System Data): exit
logger (23.11.2019 05:55:26): databases connected. Update interval 3600 seconds.
logger (23.11.2019 05:55:26): data logger (version 5.1b, build 2668) started.
logger (23.11.2019 05:55:27): connect station 0 (Vantage via USB serial).
logger (23.11.2019 05:55:27): connect station 9 (System Data via Plug-in).
logger (23.11.2019 05:55:27): wakeup sent.
logger (23.11.2019 05:55:30): waiting for sensors (wind0, rain0, th0, thb0) to provide data.
logger (23.11.2019 05:55:30): services paused.
logger (23.11.2019 05:55:30): no need to update console date/time.
logger (23.11.2019 05:55:31): setup script executed.
logger (23.11.2019 05:55:31): internal logger activated.
logger (23.11.2019 05:55:31): syncing.
logger (23.11.2019 05:55:41): gettime request sent.
logger (23.11.2019 05:55:41): no Vantage time offset.
logger (23.11.2019 05:55:41): eeprom request sent.
logger (23.11.2019 05:55:41): rain collector size 0.2 mm per tick.
logger (23.11.2019 05:55:41): transmitter 1 active as station type #0 "ISS".
logger (23.11.2019 05:55:41): firmware version request sent.
logger (23.11.2019 05:55:41): firmware version 1.90 (supports LOOP2)
logger (23.11.2019 05:55:41): no reading of Vantage's internal logger.
logger (23.11.2019 05:55:46): sensors are now providing data.
logger (23.11.2019 05:55:46): services started.
logger (23.11.2019 06:15:32): databases connected. Update interval 3600 seconds.
logger (23.11.2019 06:15:32): data logger (version 5.1b, build 2668) started.
logger (23.11.2019 06:15:32): error (22) while doing bind (port 22222): Invalid argument
logger (23.11.2019 06:15:32): killed process "28957/sh" listening to port 5551.
logger (23.11.2019 06:15:32): killed process "28958/meteosys" listening to port 5552.
logger (23.11.2019 06:15:35): data logger stopped.
logger (23.11.2019 06:15:35): databases disconnected.
logger (23.11.2019 06:16:02): databases connected. Update interval 3600 seconds.
logger (23.11.2019 06:16:02): data logger (version 5.1b, build 2668) started.
logger (23.11.2019 06:16:03): connect station 0 (Vantage via USB serial).
logger (23.11.2019 06:16:03): connect station 9 (System Data via Plug-in).
logger (23.11.2019 06:16:04): wakeup sent.
logger (23.11.2019 06:16:07): waiting for sensors (wind0, rain0, th0, thb0) to provide data.
logger (23.11.2019 06:16:07): services paused.
logger (23.11.2019 06:16:07): no need to update console date/time.
logger (23.11.2019 06:16:07): setup script executed.
logger (23.11.2019 06:16:08): internal logger activated.
logger (23.11.2019 06:16:08): syncing.
logger (23.11.2019 06:16:18): gettime request sent.
logger (23.11.2019 06:16:28): message above repeats 1 time.
logger (23.11.2019 06:16:28): no Vantage time offset.
logger (23.11.2019 06:16:28): eeprom request sent.
logger (23.11.2019 06:16:28): rain collector size 0.2 mm per tick.
logger (23.11.2019 06:16:28): transmitter 1 active as station type #0 "ISS".
logger (23.11.2019 06:16:28): firmware version request sent.
logger (23.11.2019 06:16:28): firmware version 1.90 (supports LOOP2)
logger (23.11.2019 06:16:28): no reading of Vantage's internal logger.
logger (23.11.2019 06:16:34): sensors are now providing data.
logger (23.11.2019 06:16:34): services started.
logger (23.11.2019 06:36:09): child process received termination signal (11) in functions: termination_handler < inetupload_getline < inetupload_getline < inetupload_getline < inetuploa...
logger (23.11.2019 06:36:09): disconnect station 0 (Vantage): exit
logger (23.11.2019 06:36:09): disconnect station 9 (System Data): exit
logger (23.11.2019 06:36:10): databases connected. Update interval 3600 seconds.
logger (23.11.2019 06:36:10): data logger (version 5.1b, build 2668) started.
logger (23.11.2019 06:36:11): connect station 0 (Vantage via USB serial).
logger (23.11.2019 06:36:11): connect station 9 (System Data via Plug-in).
logger (23.11.2019 06:36:11): wakeup sent.
logger (23.11.2019 06:36:14): waiting for sensors (wind0, rain0, th0, thb0) to provide data.
logger (23.11.2019 06:36:14): services paused.
logger (23.11.2019 06:36:14): no need to update console date/time.
logger (23.11.2019 06:36:15): setup script executed.
logger (23.11.2019 06:36:15): internal logger activated.
logger (23.11.2019 06:36:15): syncing.
logger (23.11.2019 06:36:25): gettime request sent.
logger (23.11.2019 06:36:25): no Vantage time offset.
logger (23.11.2019 06:36:25): eeprom request sent.
logger (23.11.2019 06:36:25): rain collector size 0.2 mm per tick.
logger (23.11.2019 06:36:25): transmitter 1 active as station type #0 "ISS".
logger (23.11.2019 06:36:25): firmware version request sent.
logger (23.11.2019 06:36:25): firmware version 1.90 (supports LOOP2)
logger (23.11.2019 06:36:25): no reading of Vantage's internal logger.
logger (23.11.2019 06:36:30): sensors are now providing data.
logger (23.11.2019 06:36:30): services started.
logger (23.11.2019 06:56:16): child process received termination signal (11) in functions: termination_handler < inetupload_getline < inetupload_getline < inetupload_getline < inetuploa...
logger (23.11.2019 06:56:16): disconnect station 0 (Vantage): exit
logger (23.11.2019 06:56:16): disconnect station 9 (System Data): exit
logger (23.11.2019 06:56:17): databases connected. Update interval 3600 seconds.
logger (23.11.2019 06:56:17): data logger (version 5.1b, build 2668) started.
logger (23.11.2019 06:56:18): connect station 0 (Vantage via USB serial).
logger (23.11.2019 06:56:18): connect station 9 (System Data via Plug-in).
logger (23.11.2019 06:56:18): wakeup sent.
logger (23.11.2019 06:56:21): waiting for sensors (wind0, rain0, th0, thb0) to provide data.
logger (23.11.2019 06:56:21): services paused.
logger (23.11.2019 06:56:21): no need to update console date/time.
logger (23.11.2019 06:56:22): setup script executed.
logger (23.11.2019 06:56:22): internal logger activated.
logger (23.11.2019 06:56:22): syncing.
logger (23.11.2019 06:56:32): gettime request sent.
logger (23.11.2019 06:56:32): no Vantage time offset.
logger (23.11.2019 06:56:32): eeprom request sent.
logger (23.11.2019 06:56:32): rain collector size 0.2 mm per tick.
logger (23.11.2019 06:56:32): transmitter 1 active as station type #0 "ISS".
logger (23.11.2019 06:56:32): firmware version request sent.
logger (23.11.2019 06:56:32): firmware version 1.90 (supports LOOP2)
logger (23.11.2019 06:56:32): no reading of Vantage's internal logger.
logger (23.11.2019 06:56:38): sensors are now providing data.
logger (23.11.2019 06:56:38): services started.
logger (23.11.2019 07:16:23): child process received termination signal (11) in functions: termination_handler < inetupload_getline < inetupload_getline < inetupload_getline < inetuploa...
logger (23.11.2019 07:16:23): disconnect station 0 (Vantage): exit
logger (23.11.2019 07:16:23): disconnect station 9 (System Data): exit
logger (23.11.2019 07:16:24): databases connected. Update interval 3600 seconds.
logger (23.11.2019 07:16:24): data logger (version 5.1b, build 2668) started.
logger (23.11.2019 07:16:25): connect station 0 (Vantage via USB serial).
logger (23.11.2019 07:16:25): connect station 9 (System Data via Plug-in).
logger (23.11.2019 07:16:25): wakeup sent.
logger (23.11.2019 07:16:28): waiting for sensors (wind0, rain0, th0, thb0) to provide data.
logger (23.11.2019 07:16:28): services paused.
logger (23.11.2019 07:16:28): no need to update console date/time.
logger (23.11.2019 07:16:29): setup script executed.
logger (23.11.2019 07:16:29): internal logger activated.
logger (23.11.2019 07:16:29): syncing.
logger (23.11.2019 07:16:39): gettime request sent.
logger (23.11.2019 07:16:39): no Vantage time offset.
logger (23.11.2019 07:16:39): eeprom request sent.
logger (23.11.2019 07:16:39): rain collector size 0.2 mm per tick.
logger (23.11.2019 07:16:39): transmitter 1 active as station type #0 "ISS".
logger (23.11.2019 07:16:39): firmware version request sent.
logger (23.11.2019 07:16:39): firmware version 1.90 (supports LOOP2)
logger (23.11.2019 07:16:39): no reading of Vantage's internal logger.
logger (23.11.2019 07:16:44): sensors are now providing data.
logger (23.11.2019 07:16:44): services started.

[pwilman]

Well after 2 hours of operating the issue has come back again. This time:

- nano charts are still being updated
- my local home weather dashboard (the one built into the nano)
- nothing recorded in the system log for the nano
- home weather template not updating at www.paulwilman.com, and the api file MBrealtimeupload.txt is empty (0kb in size)
- in the live data tab it’s looks like the data is current, but it has not been sent to any of my weather networks (eg WOW, home weather.

Paul

[admin]

Upload to the Meteoplug cloud service does run into a crash of the logging process. Logger is restarted automatically but this has the side effects you experience.
When you give me remote access I can try to find out, why this process crashes.

[pwilman]

Boris,

Thanks for looking into this. Just sent my access details to you at info(at)meteobridge.com

I do usually upload to meteoplug cloud services, but have that switched off at the moment

Paul

[pwilman]

Boris,

Based on the fact that you thought my issue was related to meteoplug cloud servies I have turned off the upload to meteoplug. It lasted for 7.5 hours before the issue came back. Conditions are:

- all internet weather network upload services switched off other than homeweather and weather underground
- homeweather template stops updating; MBrealtimeupload.txt stops being uploaded
- charts continue to be uploaded
- no errors show up in system log
- livedata tab shows upto date data

A reboot of the rig seems to return things to normal. I have set a script to run every few hours to do a reboot to try and keep my site running.

Fingers crossed you can find out what is wrong.

Paul

[pwilman]

Hello,

I think I have found the cause of my issue. The nano lasted for about 1.5 days this time before all the services stopped. This time I noticed that the buffer was at 100% with 1000 items. Whilst it was working it had 1 or 2 items in it.

I am not sure what the buffer is, but it looks like it suddenly fills up which stops my nano doing http uploads, weather underground uploads and home weather uploads etc.

Paul

[pwilman]

I had these issues every few hours for about 5 days. My SD has now been up and working beautifully ( as ever) for over 12 days now.

I am still however none the wiser as to what was causing my buffer to fill up and stop all my services working.

I guess time will tell if it returns?

Paul