DC01 NEW box,disc,latest release: NO FTP and more errors

[wvdkuil]

Today i did a fresh install, new download, new stick, new and formatted sdd, install release and upgrade to latest downloaded release.
Tried Vantage Pro via IP-direct, ip-meteohub:5500 and usb. Makes no difference. The sensors are seen and correctly processed in all three cases

Still the same FTP/HTTP errors.

No FTP upload: tried different ftp accounts. Saved first the credentials (works) and than hit save&test. White page after 30 seconds, as described in other thread.
Same errors in log
2012.04.25 17:30:02 meteohub: upload: FTP Upload failed: cp /var/run/meteohub/all-sensors0.txt /data/myweb/uploads/allsensors.php
2012.04.25 17:30:13 meteohub: upload: FTP Upload failed: mv /var/run/meteohub/all-sensors0.txt /var/run/meteohub/uploads/allsensors.php
Tried on two other meteohubs (VMware and Alix): no problems

No wdlive/html: error 403 The requested URL '/live/wdlive.html' resolves to a file outside the permitted web server directory tree.

Strange dates in public/myweb/uploads. Two hours later 19:xx and not 17:xx as here in belgium.

System data The "weatherstation" for systemdata can not be seen when using ip-meteohub:5500. switching to normal ip or an usb connected device shows systemdata sensors.

I want to use the DC01 because of the more reliable SDD instead of the SD-card in the Alex as my stations are in other networks so backup via internet works but still is a security risk.

i am not able to let it work correctly.

Has anyone the same problems and / or any ideas to solve these issues?
I am willing to ship the complete unit to an experienced DC01/Linux user for testing.

Wim van der Kuil

[admin]

May be there are still some quirks in the DC01 branch, impossible to test all combination of features.
Can you please give me access on ports 7777 and 22? Please send WAN IP or dyndns account to "info(at)meteohub.de".

[admin]

Inspection showed that inital boot sequence located in /etc/rc.boot did not run through correctly caused by backup and .swp versions of original file being also there. Next update will clean that automatically.

[wvdkuil]

Good morning,
I am carefully comparing the DC01 logs with the Alix logs
Found a new error today which showed up for the first time (at least i noticed these error) after last update 4.9g.
When starting the logger:
>> logger (28.04.2012 09:19:01): error reading from wget process (status): error, cannot read /var/run/meteohub/inetupload.log
then continuous:
>> logger (28.04.2012 09:19:05): error opening stream /var/run/meteohub/inetupload in append mode: No such file or directory

Using SSH-FTP (Cyberduck for the mac) i saw that there is indeed no meteohub/ folder inside /var/run/.
So there is no /var/run/uploads either in the DC01 file structure. These folders are there in VMWARE and Alix version and contain the files to be uploaded.
Are these 'missing' folders maybe the reason for the fail of the FTP uploads or has the DC01 just a different file-structure?

The /data/uploads/ shortcut is present and funtional in VMWARE and Alix.
It is a "invalid link" in DC01.

Hope this will somehow help to solve the NO-FTP issue.

[admin]

Looks like a good finding. Will check with the DC01 distribution of Meteohub. Folder setup should be identical with all the Meteohubs, but when building a new platform things can be omitted by accident.

[admin]

A bug-fix that gets rid of the ftp upload problem (caused by a faulty version of ncftpput) and the errors resulting from not setting up the /var/run subdirectories on tempfs is on its way.

Sorry for the inconveniances, but as written, new platforms always have their surprises

[wvdkuil]

All problems as mentioned here are solved with this update.

Thanks a lot.

[wvdkuil]

Good evening,
Although the aforementioned problems are gone two new issues showed up:

Meteohub Log For me just a strange issue, while i do not know the meaning for these errors. The last lines in the log are:
loggereval (02.05.2012 20:43:14): illegal data record in line 88972 while reading from file /data/weather/201204/raw: 1335639799, wind0 251 13 9 0144
loggereval (02.05.2012 20:43:14): illegal data record in line 88973 while reading from file /data/weather/201204/raw: 1335639801, wind0 252 9 9 0144
large numbers of the same type of errors
loggereval (02.05.2012 20:44:15): illegal data record in line 142019 while reading from file /data/weather/201204/raw: 1335807489, wind0 105 9 9 0206
loggereval (02.05.2012 20:44:15): illegal data record in line 142020 while reading from file /data/weather/201204/raw: 1335807497, wind0 108 9 9 0206
loggereval (02.05.2012 20:44:15): illegal data record in line 142021 while reading from file /data/weather/201204/raw: 1335807503, sol0 25 242
loggereval (02.05.2012 20:44:15): illegal data record in line 142022 while reading from file /data/weather/201204/raw: 1335807503, wind0 108 4 9 0206
loggereval (02.05.2012 20:44:15): illegal data record in line 142023 while reading from file /data/weather/201204/raw: 1335807507, wind0 108 0 9 0206
loggereval (02.05.2012 20:44:15): illegal data record in line 142024 while reading from file /data/weather/201204/raw:

the last line even stops exactly as shown without an illegal record.

Schedule FTP Uploads of Graphs and Data for " individual schedule 59 213 * * * " uploads also on 20:45. I use that kind of upload to get yesterday values for my website. Last night the upload went OK at midnight, but today at 20:45 the same html template was processed and was uploaded resulting in yesterday and today values being equal. I resolved that by uploading a backup file by hand.

Because those log entries and the weird ftp upload are happening at the same time i wonder if there is a relation?
I wil check more frequent and inform if there are other uploads occurring at unscheduled moments.

[admin]

please delete the ill-formatted lines from the raw file. May be this is something related to the troubles in the past.

[wvdkuil]

Good morning, some good and other news.
Log errors
I deleted the ill-formatted lines. They where on different places in the april-file. So definitely from the past and the log stays clear now.

Time scheduled FTP
BUT the FTP at " 59 23 * * * " which is used for mine and saratoga's template to get yesterdays values does not work.
Just to make sure i included an extra file to be uploaded at midnight, but no result. More than 24 hours passed, all other uploads work fine. But not the time scheduled ones.
I included a few other scheduled uploads for meteohub supplied files (29 08 * * * , 59 08 * * *). they did not upload either.

Uploads folder
An annoyance. The upload folder on DC01 is not a real folder but a shortcut to another folder. This shortcut does not work when using smb and results in an error message on Mac and PC. With Alix and VMware one can click the uploads folder just to check if everything is being uploaded. With DC01 when files do not arrive at the web host i have to use ssh/ftp to check if they are left in the uploads folder.

[admin]

Regarding the upload schedule problems. Did you try looking directly at the crontab via "crontab -l"? Anything mysterious there?
What puzzles me is, that other uploads work fine, but the ones you scheduled with crontab mimic don't do. Or do all scheduled uploads fail?

[wvdkuil]

Good evening,
Only the uploads with a specified time (like: 59 23 * * *) fail.
Normal uploads like every 5 minutes are working correct.

If you want to look inside the machine, you still can log in yourself as you did a few days ago.

I have never used crontab so i will study that tomorrow. This is a small part of the output for relevant jobs:
#- "wsYdayTags *" "59 23 * * *" "wsYdayTags.php" 1
59 23 * * * sleep 70; /srv/www/meteohtml.cgi "/data/graphs/wsYdayTags.html" >"/var/run/meteohub/wsYdayTags14.html"; /srv/www/cgi-bin/stamp.cgi cp "/var/run/meteohub/wsYdayTags14.html" "/data/myweb/uploads/wsYdayTags.php"; /srv/www/cgi-bin/stamp.cgi mv "/var/run/meteohub/wsYdayTags14.html" "/var/run/meteohub/uploads/wsYdayTags.php"
#- "Meteoware-live" "40 15 * * *" "mwlive.xml" 0
40 15 * * * sleep 75; /srv/www/meteohtml.cgi /home/meteohub/mwlive.conf >/var/run/meteohub/mwlive.out; cp /var/run/meteohub/mwlive.out /data/myweb/uploads/mwlive.xml; cp /var/run/meteohub/mwlive.out /var/run/meteohub/perm/mwlive.xml; mv /var/run/meteohub/mwlive.out "/var/run/meteohub/uploads/mwlive.xml"
#- "Meteoware-live" "42 15 * * *" "mwlive2.xml" 0
42 15 * * * sleep 80; /srv/www/meteohtml.cgi /home/meteohub/mwlive.conf >/var/run/meteohub/mwlive.out; cp /var/run/meteohub/mwlive.out /data/myweb/uploads/mwlive.xml; cp /var/run/meteohub/mwlive.out /var/run/meteohub/perm/mwlive.xml; mv /var/run/meteohub/mwlive.out "/var/run/meteohub/uploads/mwlive2.xml"

[admin]

59 23 * * * sleep 70; /srv/www/meteohtml.cgi ...

Please have a look at the sleep commands that do a specific delay to each of the schedules commands. So your job is started 10 seconds after midnight with more or less empty data. You can change the start time or can change the order of your command invocations on "graph upload" page.

[wvdkuil]

Good morning,

Good tip about " sleep 70 ".
Off course I will change this by putting the hour/minute scheduled updates first in the list or by using " 55 23 * * * ".

But as you can see in the CRONTAB list I scheduled also as a test standard meteohub supplied uploads (like #- "Meteoware-live" "42 15 * * *) at different times during the day.

NONE of those ever showed up in /data/myweb/uploads/ or in var/run/meteohub/uploads/
Al other uploads (scheduled with once per so many minutes) appear shortly in var/run/meteohub/uploads/ and arrive and stay in /data/myweb/uploads/ and at the website.

So NORMAL FTP uploads like " */5 * * * * " or " 6 */1 * * * ": OK
but minute-hour uploads like "58 23 * * *" FAIL.

I hope you can get a clue what is happening.
Please feel free to investigate on the DC01 box itself. I opened the ports as requested last week.

To all readers of this post:
I know that not that many people use a DC01 especially in combination with "indivuduell" uploads like "59 23 * * *" .
But if its working by anyone reading this post, please let me know. My website template uses this "59 23 * * *" upload to get yesterday values and with Meteohub on VMWARE or on the Alix box it works, just the DC01 which i want to use for remote weatherstations (in our children houses) fail.

[wvdkuil]

Some more erros: Maybe it gives a clue:

Other strange errors (for a non Linux user), these on from system log:

-04-25T12:10:50.686137+02:00 meteohub rsyslogd: Warning: backward compatibility layer added to following directive to rsyslog.conf: ModLoad imuxsock
cat: /data/weather/20*/raw: No such file or directory
expr: division by zero
expr: division by zero
expr: division by zero
2012-04-25T19:08:42.389047+02:00 meteohub rsyslogd: WARNING: rsyslogd is running in compatibility mode. Automatically generated config directives may interfer with your rsyslog.conf settings. We suggest upgrading your config and adding -c4 as the first rsyslogd option.

==== last error goes on until today (division by zero only in april with old meteohub release)

2012-05-04T15:33:38.189214+02:00 meteohub rsyslogd: WARNING: rsyslogd is running in compatibility mode. Automatically generated config directives may interfer with your rsyslog.conf settings. We suggest upgrading your config and adding -c4 as the first rsyslogd option.
2012-05-04T15:33:38.198609+02:00 meteohub rsyslogd: Warning: backward compatibility layer added to following directive to rsyslog.conf: ModLoad immark
2012-05-04T15:33:38.206698+02:00 meteohub rsyslogd: Warning: backward compatibility layer added to following directive to rsyslog.conf: MarkMessagePeriod 1200
2012-05-04T15:33:38.206778+02:00 meteohub rsyslogd: Warning: backward compatibility layer added to following directive to rsyslog.conf: ModLoad imuxsock