为什么内存中的 sqlite 数据库 table 创建失败？
Why is table creation failing for an in-memory sqlite database?
我正在尝试使用 twisted.enterprise.adbapi.ConnectionPool 创建内存中的 sqlite 数据库。
问题描述：
以下代码按预期工作：
#! /usr/bin/env python
from twisted.internet.task import react
from twisted.internet import defer
from twisted.enterprise.adbapi import ConnectionPool
sql_init = """
CREATE TABLE ajxp_changes ( seq INTEGER PRIMARY KEY AUTOINCREMENT, node_id NUMERIC, type TEXT, source TEXT, target TEXT, deleted_md5 TEXT );
CREATE TABLE ajxp_index ( node_id INTEGER PRIMARY KEY AUTOINCREMENT, node_path TEXT, bytesize NUMERIC, md5 TEXT, mtime NUMERIC, stat_result BLOB);
CREATE TRIGGER LOG_INSERT AFTER INSERT ON ajxp_index BEGIN INSERT INTO ajxp_changes (node_id,source,target,type) VALUES (new.node_id, "NULL", new.node_path, "create"); END;
"""
sql_insert = "INSERT INTO ajxp_index (node_path,bytesize,md5,mtime,stat_result) VALUES (?,?,?,?,?);"
sql_file_path = "/tmp/test.sqlite"
@react
@defer.inlineCallbacks
def main(reactor):
cp = ConnectionPool("sqlite3", sql_file_path, check_same_thread=False)
yield cp.runInteraction(lambda c, s: c.executescript(s), sql_init)
params = (
"/tmp/test.txt",
"64",
"5d41402abc4b2a76b9719d911017c592",
2832345,
"xxxxxx"
)
yield cp.runOperation(sql_insert, params)
但是，将 sql_file_path="/tmp/test.sqlite 替换为 sql_file_path=":memory:" 突然导致脚本失败，回溯如下：
$ python test.py
main function encountered error
Traceback (most recent call last):
File "/Users/lthibault/.pyenv/versions/3.5.3/lib/python3.5/site-packages/twisted/internet/defer.py", line 500, in errback
self._startRunCallbacks(fail)
File "/Users/lthibault/.pyenv/versions/3.5.3/lib/python3.5/site-packages/twisted/internet/defer.py", line 567, in _startRunCallbacks
self._runCallbacks()
File "/Users/lthibault/.pyenv/versions/3.5.3/lib/python3.5/site-packages/twisted/internet/defer.py", line 653, in _runCallbacks
current.result = callback(current.result, *args, **kw)
File "/Users/lthibault/.pyenv/versions/3.5.3/lib/python3.5/site-packages/twisted/internet/defer.py", line 1357, in gotResult
_inlineCallbacks(r, g, deferred)
--- <exception caught here> ---
File "/Users/lthibault/.pyenv/versions/3.5.3/lib/python3.5/site-packages/twisted/internet/defer.py", line 1299, in _inlineCallbacks
result = result.throwExceptionIntoGenerator(g)
File "/Users/lthibault/.pyenv/versions/3.5.3/lib/python3.5/site-packages/twisted/python/failure.py", line 393, in throwExceptionIntoGenerator
return g.throw(self.type, self.value, self.tb)
File "test.py", line 35, in main
yield cp.runOperation(sql_insert, params)
File "/Users/lthibault/.pyenv/versions/3.5.3/lib/python3.5/site-packages/twisted/python/threadpool.py", line 250, in inContext
result = inContext.theWork()
File "/Users/lthibault/.pyenv/versions/3.5.3/lib/python3.5/site-packages/twisted/python/threadpool.py", line 266, in <lambda>
inContext.theWork = lambda: context.call(ctx, func, *args, **kw)
File "/Users/lthibault/.pyenv/versions/3.5.3/lib/python3.5/site-packages/twisted/python/context.py", line 122, in callWithContext
return self.currentContext().callWithContext(ctx, func, *args, **kw)
File "/Users/lthibault/.pyenv/versions/3.5.3/lib/python3.5/site-packages/twisted/python/context.py", line 85, in callWithContext
return func(*args,**kw)
File "/Users/lthibault/.pyenv/versions/3.5.3/lib/python3.5/site-packages/twisted/enterprise/adbapi.py", line 477, in _runInteraction
compat.reraise(excValue, excTraceback)
File "/Users/lthibault/.pyenv/versions/3.5.3/lib/python3.5/site-packages/twisted/python/compat.py", line 467, in reraise
raise exception.with_traceback(traceback)
File "/Users/lthibault/.pyenv/versions/3.5.3/lib/python3.5/site-packages/twisted/enterprise/adbapi.py", line 467, in _runInteraction
result = interaction(trans, *args, **kw)
File "/Users/lthibault/.pyenv/versions/3.5.3/lib/python3.5/site-packages/twisted/enterprise/adbapi.py", line 486, in _runOperation
trans.execute(*args, **kw)
sqlite3.OperationalError: no such table: ajxp_index
我试过的
1。在标准库中复制
我首先试图确定问题是否与sqlite有关，或者是否与twisted有关。为此，我 运行 以下脚本， 其行为符合预期。
#! /usr/bin/env python
import sqlite3
conn = sqlite3.connect(":memory:")
conn.executescript(sql_init)
conn.execute(
sql_insert,
("/tmp/test.txt", "64", "5d41402abc4b2a76b9719d911017c592", 2832345, "xxxxxx"),
)
结论：问题源于twisted.enterprise.adbapi.ConnectionPool
2。尝试使用不同的 ConnectionPool 方法来 运行 INSERT 语句。
诚然，此时我正在抓住救命稻草，但我认为问题可能源于我对 runOperation 的使用。我决定使用 runInteraction 和 runQuery.
复制原始示例
yield cp.runOperation(sql_insert, params) 的以下替换也失败并出现相同的错误。
yield cp.runInteraction(lambda c, s, p: c.execute(s), sql_insert, params)yield cp.runQuery(sql_insert, params)
重要的是，将 sqlite 数据库路径从 :memory: 更改为持久存储上的某个路径，runInteraction 和 runQuery 都按预期工作。
结论：问题与在 Twisted 中使用内存中的 sqlite 数据库有关。
有什么想法吗？
好的，事实证明，在幕后，ConnectionPool 每次调用查询方法时都试图连接到 :memory:，因此每次都重新创建数据库。
解决方案似乎是编写一个 DB-API v.20 模块来包装 sqlite3 并在调用其连接函数时始终返回相同的 :memory: 连接。
我正在尝试使用 twisted.enterprise.adbapi.ConnectionPool 创建内存中的 sqlite 数据库。
问题描述：
以下代码按预期工作：
#! /usr/bin/env python
from twisted.internet.task import react
from twisted.internet import defer
from twisted.enterprise.adbapi import ConnectionPool
sql_init = """
CREATE TABLE ajxp_changes ( seq INTEGER PRIMARY KEY AUTOINCREMENT, node_id NUMERIC, type TEXT, source TEXT, target TEXT, deleted_md5 TEXT );
CREATE TABLE ajxp_index ( node_id INTEGER PRIMARY KEY AUTOINCREMENT, node_path TEXT, bytesize NUMERIC, md5 TEXT, mtime NUMERIC, stat_result BLOB);
CREATE TRIGGER LOG_INSERT AFTER INSERT ON ajxp_index BEGIN INSERT INTO ajxp_changes (node_id,source,target,type) VALUES (new.node_id, "NULL", new.node_path, "create"); END;
"""
sql_insert = "INSERT INTO ajxp_index (node_path,bytesize,md5,mtime,stat_result) VALUES (?,?,?,?,?);"
sql_file_path = "/tmp/test.sqlite"
@react
@defer.inlineCallbacks
def main(reactor):
cp = ConnectionPool("sqlite3", sql_file_path, check_same_thread=False)
yield cp.runInteraction(lambda c, s: c.executescript(s), sql_init)
params = (
"/tmp/test.txt",
"64",
"5d41402abc4b2a76b9719d911017c592",
2832345,
"xxxxxx"
)
yield cp.runOperation(sql_insert, params)
但是，将 sql_file_path="/tmp/test.sqlite 替换为 sql_file_path=":memory:" 突然导致脚本失败，回溯如下：
$ python test.py
main function encountered error
Traceback (most recent call last):
File "/Users/lthibault/.pyenv/versions/3.5.3/lib/python3.5/site-packages/twisted/internet/defer.py", line 500, in errback
self._startRunCallbacks(fail)
File "/Users/lthibault/.pyenv/versions/3.5.3/lib/python3.5/site-packages/twisted/internet/defer.py", line 567, in _startRunCallbacks
self._runCallbacks()
File "/Users/lthibault/.pyenv/versions/3.5.3/lib/python3.5/site-packages/twisted/internet/defer.py", line 653, in _runCallbacks
current.result = callback(current.result, *args, **kw)
File "/Users/lthibault/.pyenv/versions/3.5.3/lib/python3.5/site-packages/twisted/internet/defer.py", line 1357, in gotResult
_inlineCallbacks(r, g, deferred)
--- <exception caught here> ---
File "/Users/lthibault/.pyenv/versions/3.5.3/lib/python3.5/site-packages/twisted/internet/defer.py", line 1299, in _inlineCallbacks
result = result.throwExceptionIntoGenerator(g)
File "/Users/lthibault/.pyenv/versions/3.5.3/lib/python3.5/site-packages/twisted/python/failure.py", line 393, in throwExceptionIntoGenerator
return g.throw(self.type, self.value, self.tb)
File "test.py", line 35, in main
yield cp.runOperation(sql_insert, params)
File "/Users/lthibault/.pyenv/versions/3.5.3/lib/python3.5/site-packages/twisted/python/threadpool.py", line 250, in inContext
result = inContext.theWork()
File "/Users/lthibault/.pyenv/versions/3.5.3/lib/python3.5/site-packages/twisted/python/threadpool.py", line 266, in <lambda>
inContext.theWork = lambda: context.call(ctx, func, *args, **kw)
File "/Users/lthibault/.pyenv/versions/3.5.3/lib/python3.5/site-packages/twisted/python/context.py", line 122, in callWithContext
return self.currentContext().callWithContext(ctx, func, *args, **kw)
File "/Users/lthibault/.pyenv/versions/3.5.3/lib/python3.5/site-packages/twisted/python/context.py", line 85, in callWithContext
return func(*args,**kw)
File "/Users/lthibault/.pyenv/versions/3.5.3/lib/python3.5/site-packages/twisted/enterprise/adbapi.py", line 477, in _runInteraction
compat.reraise(excValue, excTraceback)
File "/Users/lthibault/.pyenv/versions/3.5.3/lib/python3.5/site-packages/twisted/python/compat.py", line 467, in reraise
raise exception.with_traceback(traceback)
File "/Users/lthibault/.pyenv/versions/3.5.3/lib/python3.5/site-packages/twisted/enterprise/adbapi.py", line 467, in _runInteraction
result = interaction(trans, *args, **kw)
File "/Users/lthibault/.pyenv/versions/3.5.3/lib/python3.5/site-packages/twisted/enterprise/adbapi.py", line 486, in _runOperation
trans.execute(*args, **kw)
sqlite3.OperationalError: no such table: ajxp_index
我试过的
1。在标准库中复制
我首先试图确定问题是否与sqlite有关，或者是否与twisted有关。为此，我 运行 以下脚本， 其行为符合预期。
#! /usr/bin/env python
import sqlite3
conn = sqlite3.connect(":memory:")
conn.executescript(sql_init)
conn.execute(
sql_insert,
("/tmp/test.txt", "64", "5d41402abc4b2a76b9719d911017c592", 2832345, "xxxxxx"),
)
结论：问题源于twisted.enterprise.adbapi.ConnectionPool
2。尝试使用不同的 ConnectionPool 方法来 运行 INSERT 语句。
诚然，此时我正在抓住救命稻草，但我认为问题可能源于我对 runOperation 的使用。我决定使用 runInteraction 和 runQuery.
yield cp.runOperation(sql_insert, params) 的以下替换也失败并出现相同的错误。
yield cp.runInteraction(lambda c, s, p: c.execute(s), sql_insert, params)yield cp.runQuery(sql_insert, params)
重要的是，将 sqlite 数据库路径从 :memory: 更改为持久存储上的某个路径，runInteraction 和 runQuery 都按预期工作。
结论：问题与在 Twisted 中使用内存中的 sqlite 数据库有关。
有什么想法吗？
好的，事实证明，在幕后，ConnectionPool 每次调用查询方法时都试图连接到 :memory:，因此每次都重新创建数据库。
解决方案似乎是编写一个 DB-API v.20 模块来包装 sqlite3 并在调用其连接函数时始终返回相同的 :memory: 连接。