Meteobridge - Import Weewx DB failed **solved**

[HighWay]

Hello

Recently switched from MB TP-Link to MB /Rpi/. Used TP-Link during the last 3 years, and now I would like to transfer my Weewx DB to my new MB. I also have weewx running in order to save my history data as backup.


My weewx DB is stored in MySQL , and I exported the data using phpmyadmin , as CSV . MySQL have about 220 MB of data collected during the years.
Unfortunately I am not able to import that file file to MB. I did lot of tests , but both failed, without error message:
ex:
import (25.04.2024 07:32:26): import of file "test.csv" (35209 kb) started: type 5, mode 1
import (25.04.2024 07:32:26): completed 241025 lines, performing 0 database updates.
---
My CSV file contain many rows, with sensors I don't actually have, and they appear as NULL in the file.
I did another tests removing all of these rows, using external CSV editor , but it does not help me so much :
ex:
import (25.04.2024 09:35:49): import of file "weewx.csv" (48007 kb) started: type 5, mode 1
import (25.04.2024 09:35:59): completed 339477 lines, performing 292 database updates.
---
Seems there is something wrong with that DB file, and I would have your opinion how to sort it out.

Many thanks in advance !

Regards,

Victor

[admin]

Did you check with the wiki if the data you are trying to upload is in the supported format? There are even example snippets.
https://meteobridge.com/wiki/index.php/ ... rial#Weewx

A sql dump will for sure not work, which is obvious when have a 1 second look at it. I don't know weewx and how to export data in a CSV like style from there, but others in the forum might be able to help.

[HighWay]

Did you check with the wiki if the data you are trying to upload is in the supported format? There are even example snippets.
https://meteobridge.com/wiki/index.php/ ... rial#Weewx

A sql dump will for sure not work, which is obvious when have a 1 second look at it. I don't know weewx and how to export data in a CSV like style from there, but others in the forum might be able to help.

Hi Admin,

Yes, I looked at this page , and found out more about the database structure.
At first I didn't pay enough attention to the structure.
I completely edited the CSV file by copying all the columns one by one. But it turned out that it didn't work again. But finally I found where the problem is. It turned out to be in the "usUnits" column.
Database import only works with usUnits=1.
According weewx manual :
---
Name Encoded value Note
US 0x01 U.S. Customary
METRICWX 0x11 Metric, with rain related measurements in mm and speeds in m/s
METRIC 0x10 Metric, with rain related measurements in cm and speeds in km/hr
---
So, import only works with U.S metric system, but my database is METRICWX where usUnits=17 decimal. I checked also usUnits=16 too , but it again not working.

With usUnits=1 I am able to import DB successfully , but I have a mess in my imported values .
Not sure is it possible to convert my DB to US , from weewx side , and then to export it, or I have to find another workaround , if only possibilities is to use usUnits=1.

Thanks that you bring my attention to the link above and your help.

Regards,

Victor

[HighWay]

Hi

After 3 days of "fight" I was able to sort it out.

For those users of the forum who are yet to migrate, I would give some guidelines.

1. Import weewx database from SQL to CSV . In my case i used phpmyadmin.
2. Edit CSV , "search and replace" and clean all NULL cells it could appear. Good CSV editor I found is "Modern CSV" . This could take over 10 minutes and the tool "freezing" but just be patient.
3. Rearrange the columns exactly at the same order shown on the manual - https://meteobridge.com/wiki/index.php/ ... rial#Weewx
4. Remove /delete/ the rest columns which are not as per wiki tutorial.
After the test /one month CSV sample/ migration I found that I missed rain data in the chart , but I guess it can be solved migrating the sensor data.

CSV exported file may contain some additional data , as summarized and accumulated sensor data , but I skip it when I migrated it using "ignore bad data" option.
At the moment it working only when CSV export column usUnits=1 and unable to get it working with 16 or 17. I guess my DB is not so healthy.

The result:

import (28.04.2024 16:22:48): import of file "weewx-one-month-only-edited!.csv" (1501 kb) started: type 5, mode 1
import (28.04.2024 16:26:49): import of file "weewx-one-month-only-edited!.csv" reached 79% (up to line 7435).
import (28.04.2024 16:31:04): completed 11184 lines, performing 119738 database updates.



Regards,

Victor

[HighWay]

Hi

Just small update:
Unfortunately sensor data migration not working for me.
Found similar post : viewtopic.php?p=47720&hilit=sensor+migration#p47720

I have nothing in the log after I initiated migration and no changes in the sensor data.

BR

Victor

[admin]

Thanks for the heads up. Should be solved with just released update.

[HighWay]

Thanks admin

Just tested - working fine !

BR

Victor