使用 cassandra-python-driver 记录所有查询
Logging all queries with cassandra-python-driver
我正在尝试找到一种方法来记录从 python 代码在 Cassandra 上完成的所有查询。使用 BatchStatement
完成执行时专门记录
有没有我可以用来记录这个的挂钩或回调?
您是否考虑过为您的 execute
或等效项(例如 execute_concurrent
)创建装饰器来记录用于您的语句或准备语句的 CQL 查询?
您可以采用仅在查询成功执行后才记录 CQL 查询的方式编写。
add_request_init_listener(fn, *args, **kwargs)
Adds a callback with arguments to be called when any request is created.
It will be invoked as fn(response_future, *args, **kwargs) after each client request is created, and before the request is sent*
使用回调,您可以轻松记录该会话进行的所有查询。
示例:
from cassandra.cluster import Cluster
from cassandra.auth import PlainTextAuthProvider
class RequestHandler:
def on_request(self, rf):
# This callback is invoked each time a request is created, on the thread creating the request.
# We can use this to count events, or add callbacks
print(rf.query)
auth_provider = PlainTextAuthProvider(
username='cassandra',
password='cassandra'
)
cluster = Cluster(['192.168.65.199'],auth_provider=auth_provider)
session = cluster.connect('test')
handler = RequestHandler()
# each instance will be registered with a session, and receive a callback for each request generated
session.add_request_init_listener(handler.on_request)
from time import sleep
for count in range(1, 10):
print(count)
for row in session.execute("select * from kv WHERE key = %s", ["ed1e49e0-266f-11e7-9d76-fd55504093c1"]):
print row
sleep(1)
2 个选项:
坚持session.add_request_init_listener
来自源代码:
a) BoundStatement
https://github.com/datastax/python-driver/blob/3.11.0/cassandra/query.py#L560
传递的值存储在raw_values
中,你可以尝试提取它
b) BatchStatement
https://github.com/datastax/python-driver/blob/3.11.0/cassandra/query.py#L676
它存储了在_statements_and_parameters
中用于构造该对象的所有语句和参数。
虽然不是 public 属性
,但似乎可以获取
c) 只调用了这个hook,没找到其他hook
https://github.com/datastax/python-driver/blob/master/cassandra/cluster.py#L2097
但它与查询的实际执行无关 - 它只是一种检查已构建何种查询并可能添加额外的方法 callbacks/errbacks
从不同的角度接近它并使用痕迹
https://datastax.github.io/python-driver/faq.html#how-do-i-trace-a-request
https://datastax.github.io/python-driver/api/cassandra/cluster.html#cassandra.cluster.ResponseFuture.get_all_query_traces
Request tracing can be turned on for any request by setting trace=True in Session.execute_async(). View the results by waiting on the future, then ResponseFuture.get_query_trace()
这是使用选项 2 的 BatchStatement
跟踪示例:
bs = BatchStatement()
bs.add_all(['insert into test.test(test_type, test_desc) values (%s, %s)',
'insert into test.test(test_type, test_desc) values (%s, %s)',
'delete from test.test where test_type=%s',
'update test.test set test_desc=%s where test_type=%s'],
[['hello1', 'hello1'],
['hello2', 'hello2'],
['hello2'],
['hello100', 'hello1']])
res = session.execute(bs, trace=True)
trace = res.get_query_trace()
for event in trace.events:
if event.description.startswith('Parsing'):
print event.description
它产生以下输出:
Parsing insert into test.test(test_type, test_desc) values ('hello1', 'hello1')
Parsing insert into test.test(test_type, test_desc) values ('hello2', 'hello2')
Parsing delete from test.test where test_type='hello2'
Parsing update test.test set test_desc='hello100' where test_type='hello1'
我正在尝试找到一种方法来记录从 python 代码在 Cassandra 上完成的所有查询。使用 BatchStatement
有没有我可以用来记录这个的挂钩或回调?
您是否考虑过为您的 execute
或等效项(例如 execute_concurrent
)创建装饰器来记录用于您的语句或准备语句的 CQL 查询?
您可以采用仅在查询成功执行后才记录 CQL 查询的方式编写。
add_request_init_listener(fn, *args, **kwargs)
Adds a callback with arguments to be called when any request is created.
It will be invoked as fn(response_future, *args, **kwargs) after each client request is created, and before the request is sent*
使用回调,您可以轻松记录该会话进行的所有查询。
示例:
from cassandra.cluster import Cluster
from cassandra.auth import PlainTextAuthProvider
class RequestHandler:
def on_request(self, rf):
# This callback is invoked each time a request is created, on the thread creating the request.
# We can use this to count events, or add callbacks
print(rf.query)
auth_provider = PlainTextAuthProvider(
username='cassandra',
password='cassandra'
)
cluster = Cluster(['192.168.65.199'],auth_provider=auth_provider)
session = cluster.connect('test')
handler = RequestHandler()
# each instance will be registered with a session, and receive a callback for each request generated
session.add_request_init_listener(handler.on_request)
from time import sleep
for count in range(1, 10):
print(count)
for row in session.execute("select * from kv WHERE key = %s", ["ed1e49e0-266f-11e7-9d76-fd55504093c1"]):
print row
sleep(1)
2 个选项:
坚持
session.add_request_init_listener
来自源代码:
a)
BoundStatement
https://github.com/datastax/python-driver/blob/3.11.0/cassandra/query.py#L560
传递的值存储在
raw_values
中,你可以尝试提取它b)
BatchStatement
https://github.com/datastax/python-driver/blob/3.11.0/cassandra/query.py#L676
它存储了在
,但似乎可以获取_statements_and_parameters
中用于构造该对象的所有语句和参数。 虽然不是 public 属性c) 只调用了这个hook,没找到其他hook https://github.com/datastax/python-driver/blob/master/cassandra/cluster.py#L2097
但它与查询的实际执行无关 - 它只是一种检查已构建何种查询并可能添加额外的方法 callbacks/errbacks
从不同的角度接近它并使用痕迹
https://datastax.github.io/python-driver/faq.html#how-do-i-trace-a-request https://datastax.github.io/python-driver/api/cassandra/cluster.html#cassandra.cluster.ResponseFuture.get_all_query_traces
Request tracing can be turned on for any request by setting trace=True in Session.execute_async(). View the results by waiting on the future, then ResponseFuture.get_query_trace()
这是使用选项 2 的 BatchStatement
跟踪示例:
bs = BatchStatement()
bs.add_all(['insert into test.test(test_type, test_desc) values (%s, %s)',
'insert into test.test(test_type, test_desc) values (%s, %s)',
'delete from test.test where test_type=%s',
'update test.test set test_desc=%s where test_type=%s'],
[['hello1', 'hello1'],
['hello2', 'hello2'],
['hello2'],
['hello100', 'hello1']])
res = session.execute(bs, trace=True)
trace = res.get_query_trace()
for event in trace.events:
if event.description.startswith('Parsing'):
print event.description
它产生以下输出:
Parsing insert into test.test(test_type, test_desc) values ('hello1', 'hello1')
Parsing insert into test.test(test_type, test_desc) values ('hello2', 'hello2')
Parsing delete from test.test where test_type='hello2'
Parsing update test.test set test_desc='hello100' where test_type='hello1'