Python, Oracle_cx, 使用查询结果作为参数列表进入循环插入语句
Python, Oracle_cx, using query result as parameter list into looping insert statement
过去我写了一个 pl/sql 脚本,它将 table 名称和列名称(表示来源)作为参数,然后分析 table 中的所有列给出有用的计数。
我目前正在自学 python 并且正在以一种可以针对其他 sql 数据库执行的方式重写该 pl/sql 脚本,而不仅仅是 oracle。所以我是 Python 的新手。我正在通过 Automate the Boring Stuff on Udemy。目前我不关心 sql 注入,因为我只是在学习 Python 语言。我省略了 create table 语句以减少我粘贴的代码量。
脚本在循环的第一遍中插入了正确的记录,但是它没有启动第二个循环。这是 IDLE 输出,然后是代码。
================================================ RESTART: C:\Users\nathan\Documents\_work\_data_profiling_script\profiling_python_tester.py ================================================
('ETL_INS_DTM',)
insert into PROFILING_NWS6_PRT
select 'PROFILING_NWS6', 'ETL_INS_DTM', SRCRECNO, count(*), null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null
from PROFILING_NWS6
group by SRCRECNO
order by 1,2,3
executed
committed
**Traceback (most recent call last):
File "C:\Users\nathan\Documents\_work\_data_profiling_script\profiling_python_tester.py", line 39, in <module>
for row in cursor:
cx_Oracle.InterfaceError: not a query**
import cx_Oracle
conn = cx_Oracle.connect("system", "XXXX", "localhost/xe")
cursor = conn.cursor()
## parameter declaration
##########################################################################
# These 2 parameters populated by user
v_st = 'PROFILING_NWS6' # Source Table - table in which we are profiling the data
v_srcno = 'SRCRECNO' # Source Number - numeric column in v_st that identifies the source system
# These 3 parameters automatically populated
v_prt = v_st + '_PRT' # Profile Report Table - table name we want our report created as
v_log = v_st + '_LOG' # Log Table - script logging goes here, used for monitoring and debugging
v_top = v_st + '_TOP' # Top Table - temporary table to hold top 5 counts
# write script that populates Profile Report Table with rows for each source/column combination from source table
# these are required to join to when updating analysis fields
##########################################################################
sql = "Select column_name from user_tab_columns where table_name = '"+ v_st + "' and column_name <> '" + v_srcno + "'"
cursor.execute(sql)
for row in cursor:
print(row)
sql = """insert into {x_prt}
select '{x_st}', '{x_row}', {x_srcno}, count(*), null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null
from {x_st}
group by {x_srcno}
order by 1,2,3""".format(x_prt = v_prt, x_srcno = v_srcno, x_st = v_st, x_row = row[0])
print(sql)
cursor.execute(sql)
print('executed')
cursor.execute('commit')
print('committed')
#close connections
##########################################################################
cursor.close()
conn.close()
for row in cursor:
中的光标仍在使用中,直到循环完成。当您在循环内执行 cursor.execute(sql)
时,它会更改 cursor
对象。所以在第二个循环中,被迭代的 cursor
项是循环内 commit
的游标。解决方案是在循环内创建或使用不同的游标对象。
cursor = conn.cursor() # original cursor, as above
insert_cursor = conn.cursor() # new one for insert
sql = "Select column_name from user_tab_columns where table_name " # etc
for row in cursor.execute(sql):
print(row)
sql = """second sql""".format(...)
print(sql)
insert_cursor.execute(sql)
print('executed')
insert_cursor.execute('commit')
print('committed')
cursor.close()
insert_cursor.close()
conn.close()
此外,for row in cursor:
应该是 for row in cursor.fetchall():
。要不就
for row in cur.execute(sql):
.
这种编码架构将在 Python 和数据库之间做很多 'round-trips',因此它远非最佳。小的改进包括使用 connection.autocommit
instead of a full SQL commit (or connection.commit() call). Then you could look at using executemany()
而不是多次 execute()
调用。总的来说,对于 Oracle,只需使用 PL/SQL 调用,因为这只需要一个往返。
过去我写了一个 pl/sql 脚本,它将 table 名称和列名称(表示来源)作为参数,然后分析 table 中的所有列给出有用的计数。
我目前正在自学 python 并且正在以一种可以针对其他 sql 数据库执行的方式重写该 pl/sql 脚本,而不仅仅是 oracle。所以我是 Python 的新手。我正在通过 Automate the Boring Stuff on Udemy。目前我不关心 sql 注入,因为我只是在学习 Python 语言。我省略了 create table 语句以减少我粘贴的代码量。
脚本在循环的第一遍中插入了正确的记录,但是它没有启动第二个循环。这是 IDLE 输出,然后是代码。
================================================ RESTART: C:\Users\nathan\Documents\_work\_data_profiling_script\profiling_python_tester.py ================================================
('ETL_INS_DTM',)
insert into PROFILING_NWS6_PRT
select 'PROFILING_NWS6', 'ETL_INS_DTM', SRCRECNO, count(*), null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null
from PROFILING_NWS6
group by SRCRECNO
order by 1,2,3
executed
committed
**Traceback (most recent call last):
File "C:\Users\nathan\Documents\_work\_data_profiling_script\profiling_python_tester.py", line 39, in <module>
for row in cursor:
cx_Oracle.InterfaceError: not a query**
import cx_Oracle
conn = cx_Oracle.connect("system", "XXXX", "localhost/xe")
cursor = conn.cursor()
## parameter declaration
##########################################################################
# These 2 parameters populated by user
v_st = 'PROFILING_NWS6' # Source Table - table in which we are profiling the data
v_srcno = 'SRCRECNO' # Source Number - numeric column in v_st that identifies the source system
# These 3 parameters automatically populated
v_prt = v_st + '_PRT' # Profile Report Table - table name we want our report created as
v_log = v_st + '_LOG' # Log Table - script logging goes here, used for monitoring and debugging
v_top = v_st + '_TOP' # Top Table - temporary table to hold top 5 counts
# write script that populates Profile Report Table with rows for each source/column combination from source table
# these are required to join to when updating analysis fields
##########################################################################
sql = "Select column_name from user_tab_columns where table_name = '"+ v_st + "' and column_name <> '" + v_srcno + "'"
cursor.execute(sql)
for row in cursor:
print(row)
sql = """insert into {x_prt}
select '{x_st}', '{x_row}', {x_srcno}, count(*), null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null
from {x_st}
group by {x_srcno}
order by 1,2,3""".format(x_prt = v_prt, x_srcno = v_srcno, x_st = v_st, x_row = row[0])
print(sql)
cursor.execute(sql)
print('executed')
cursor.execute('commit')
print('committed')
#close connections
##########################################################################
cursor.close()
conn.close()
for row in cursor:
中的光标仍在使用中,直到循环完成。当您在循环内执行 cursor.execute(sql)
时,它会更改 cursor
对象。所以在第二个循环中,被迭代的 cursor
项是循环内 commit
的游标。解决方案是在循环内创建或使用不同的游标对象。
cursor = conn.cursor() # original cursor, as above
insert_cursor = conn.cursor() # new one for insert
sql = "Select column_name from user_tab_columns where table_name " # etc
for row in cursor.execute(sql):
print(row)
sql = """second sql""".format(...)
print(sql)
insert_cursor.execute(sql)
print('executed')
insert_cursor.execute('commit')
print('committed')
cursor.close()
insert_cursor.close()
conn.close()
此外,for row in cursor:
应该是 for row in cursor.fetchall():
。要不就
for row in cur.execute(sql):
.
这种编码架构将在 Python 和数据库之间做很多 'round-trips',因此它远非最佳。小的改进包括使用 connection.autocommit
instead of a full SQL commit (or connection.commit() call). Then you could look at using executemany()
而不是多次 execute()
调用。总的来说,对于 Oracle,只需使用 PL/SQL 调用,因为这只需要一个往返。