odo 在 csv 和 mysql 之间转换数据
odo converting data between csv and mysql
当使用 python odo
模块将 .csv's
之一转换为 pd.DataFrame
时,我收到 TypeError
TypeError: Cannot cast array from dtype('float64') to dtype('int64')
according to the rule 'safe'
适用于其他人的代码csv's
# csv table file name
csvNm = 'table.csv'
# convert mysql table to csv
odo_csv = odo(tstConn.connect_string + '::' + tbl , csvNm)
# convert csv to pandas
odo_df = odo(odo_csv , pd.DataFrame)
以下是我迄今为止尝试过但无济于事的方法:
import pandas as pd
from odo import odo, resource, discover, convert
odo_csv=odo(tstConn.connect_string + '::' + tbl , csvNm)
csv=resource(csvNm)
ds=discover(csv)
# Convert csv to pandas
odo_df = odo(odo_csv , pd.DataFrame, dshape=ds)
还有这个:
odo_df = odo(odo_csv , pd.DataFrame, casting='unsafe')
更新 1
看来我忽略了这个错误中最明显的提示
pandas\parser.pyx in pandas.parser.TextReader._convert_tokens (pandas\parser.c:11816)()
导致 Windows SO 中的编码问题。
但这都不是:
odo_df = odo(odo_csv , pd.DataFrame, encoding=odo_csv.encoding)
或这项工作
odo_df = odo(odo_csv , pd.DataFrame, encoding='cp1252')
这种不优雅的方式(对于我的用例)取自 pandas-reading-csv-files(与上面相同的 link)
# Python3
with open('/tmp/test.csv', 'r', encoding='cp1252') as f:
df = pd.read_csv(f)
print(df)
不确定接下来要尝试什么,如有任何帮助,我们将不胜感激。
有效的解决方案是:
import pandas as pd
from odo import odo, resource, discover, convert
# convert mysql to csv
odo_csv=odo(raw_dbConn.connect_string + '::' + tblName , csvNm, header=True)
# Get odo resource aka sqlalchemy.Table instance
resc=resource(raw_dbConn.connect_string + '::' + tblName )
# Discover the resc
ds=discover(resc)
# Convert csv to dataframe
odo_df = odo(odo_csv , pd.DataFrame, dshape=ds ,encoding=odo_csv.encoding)
当使用 python odo
模块将 .csv's
之一转换为 pd.DataFrame
时,我收到 TypeError
TypeError: Cannot cast array from dtype('float64') to dtype('int64')
according to the rule 'safe'
适用于其他人的代码csv's
# csv table file name
csvNm = 'table.csv'
# convert mysql table to csv
odo_csv = odo(tstConn.connect_string + '::' + tbl , csvNm)
# convert csv to pandas
odo_df = odo(odo_csv , pd.DataFrame)
以下是我迄今为止尝试过但无济于事的方法:
import pandas as pd
from odo import odo, resource, discover, convert
odo_csv=odo(tstConn.connect_string + '::' + tbl , csvNm)
csv=resource(csvNm)
ds=discover(csv)
# Convert csv to pandas
odo_df = odo(odo_csv , pd.DataFrame, dshape=ds)
还有这个:
odo_df = odo(odo_csv , pd.DataFrame, casting='unsafe')
更新 1 看来我忽略了这个错误中最明显的提示
pandas\parser.pyx in pandas.parser.TextReader._convert_tokens (pandas\parser.c:11816)()
导致 Windows SO 中的编码问题。 但这都不是:
odo_df = odo(odo_csv , pd.DataFrame, encoding=odo_csv.encoding)
或这项工作
odo_df = odo(odo_csv , pd.DataFrame, encoding='cp1252')
这种不优雅的方式(对于我的用例)取自 pandas-reading-csv-files(与上面相同的 link)
# Python3
with open('/tmp/test.csv', 'r', encoding='cp1252') as f:
df = pd.read_csv(f)
print(df)
不确定接下来要尝试什么,如有任何帮助,我们将不胜感激。
有效的解决方案是:
import pandas as pd
from odo import odo, resource, discover, convert
# convert mysql to csv
odo_csv=odo(raw_dbConn.connect_string + '::' + tblName , csvNm, header=True)
# Get odo resource aka sqlalchemy.Table instance
resc=resource(raw_dbConn.connect_string + '::' + tblName )
# Discover the resc
ds=discover(resc)
# Convert csv to dataframe
odo_df = odo(odo_csv , pd.DataFrame, dshape=ds ,encoding=odo_csv.encoding)