Pandas DF 往返 MySQL

Question

import config
import pandas as pd
import pymysql

username = config.username
dbpassword = config.dbpassword
dbhost = config.dburl
engine =  pymysql.connect(host=dbhost, port=3306,user=username,password=dbpassword,db='db',autocommit=True) 

tableBuilder1='''SELECT b.`IssueId` AS `Id`, b.`ShortId` AS `ShortId`, b.`Path` AS `Path`, b.`Data` AS `Data`,  b.`Actual Create Date` AS `Actual Create Date` FROM `SIM_FE_Audit_Data` b WHERE b.`Data` IN ( 'Open', 'Comment', 'Pending Others', 'Work in Progress', 'Resolved') AND NOT b.`IssueId` IN (SELECT c.`IssueId` FROM `SIM_FE_Audit_Data` c WHERE b.`Actual Create Date` = c.`Actual Create Date` AND b.`Data` = 'Comment' AND c.`Data` = 'Open') ORDER BY b.`IssueId`, b.`Actual Create Date`'''

df = pd.read_sql(tableBuilder1, con=engine)
df.to_sql('SIM_FE_Audit_Durations_No_First_Comment', con=engine, if_exists='replace',index=False)

开发上述代码是为了替换视图，因为视图需要 15 分钟以上的时间来呈现并导致 tableau 仪表板出现故障。第一部分是制作系列 3 中的第一个 table。但是，我目前在 df.to_sql 上收到错误 DatabaseError: Execution failed on sql 'SELECT name FROM sqlite_master WHERE type='table' AND name=?;': not all arguments converted during string formatting，我无法弄清楚为什么会收到该错误。我做了一个 print(df) 来验证它正在读取 sql 并且确实如此。将其写回新的 table 会产生错误，我不明白为什么。

Answer 1

直接切换到 sqlalchemy 似乎已经解决了这个问题：

import config
import pandas as pd
from sqlalchemy import create_engine
import time

username = config.username
dbpassword = config.dbpassword
dbhost = config.dburl
engine =  create_engine('mysql://%s:%s@%s/db?charset=utf8' %(username, dbpassword, dbhost), encoding="utf-8") 

tableBuilder1='''SELECT b.`IssueId` AS `Id`, b.`ShortId` AS `ShortId`, b.`Path` AS `Path`, b.`Data` AS `Data`,  b.`Actual Create Date` AS `Actual Create Date` FROM `SIM_FE_Audit_Data` b WHERE b.`Data` IN ( 'Open', 'Comment', 'Pending Others', 'Work in Progress', 'Resolved') AND NOT b.`IssueId` IN (SELECT c.`IssueId` FROM `SIM_FE_Audit_Data` c WHERE b.`Actual Create Date` = c.`Actual Create Date` AND b.`Data` = 'Comment' AND c.`Data` = 'Open') ORDER BY b.`IssueId`, b.`Actual Create Date`'''

df = pd.read_sql(tableBuilder1, con=engine)
df.to_sql('SIM_FE_Audit_Durations_No_First_Comment', con=engine, if_exists='replace')

Pandas DF 往返 MySQL

Pandas DF From and Back to MySQL

python

pandas

dataframe

pymysql