如何通过 Stackdriver 登录将错误记录到 Stackdriver 错误报告 Python
How to log a error to Stackdriver Error Reporting via Stackdriver Logging in Python
我已经使用 Stackdriver Logging 很长时间了,现在我也想利用错误报告。如果可能(出于各种原因),我宁愿使用 Python 的日志记录机制并从日志文件中删除异常,而不是使用 error_reporting 库。话虽如此,文档非常混乱。例如文档说:
https://cloud.google.com/error-reporting/docs/setup/compute-engine#log_exceptions
First, install the fluent-logger-python library:
sudo pip install google-cloud-error-reporting --upgrade
这让我相信 google-cloud-error-reporting 是 fluent-logger-python 的分支或与之相关,但是,当我初始化 google-cloud 时-error-reporting 它直接调用 GCE 元数据服务器而不是连接到本地 fluentd。这两个不相关的包还是文档错误或误导?如果我将 JSON 格式的异常发送到 fluentd 或由 fluentd 监控的日志文件,错误报告会理解它们吗?
感谢任何澄清
文档有误。
TL;DR 你必须输出看起来像 https://cloud.google.com/error-reporting/docs/formatting-error-messages
的东西
这是我的解决方案:
#Parse raw log entries to expose severity field so that
#StackDriver log viewer can properly categorize (and so we can filter)
<source>
@type tail
path /var/log/conductor
pos_file /var/log/td-agent/conductor.pos
format multiline
format_firstline /\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}/
format1 /^(?<message>(?<time>\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}),\d*\s*[a-zA-Z_]*\s*(?<severity>[A-Z]*).*)/
read_from_head true
multiline_flush_interval 3s
tag conductor.app
</source>
#Add hostname field
<filter conductor.app>
@type record_transformer
<record>
hostname ${hostname}
</record>
</filter>
#Filter and tag log entries of severity ERROR or CRITICAL
<match conductor.app>
@type rewrite_tag_filter
rewriterule1 severity ERROR|CRITICAL conductor.err
rewriterule2 severity .+ conductor.info
</match>
#Process entries with tracebacks differently than those without
<match conductor.err>
@type rewrite_tag_filter
rewriterule1 message .*Traceback conductor.err.traceback
rewriterule2 message .+ conductor.err.message
</match>
#Parse out the traceback
<match conductor.err.traceback>
@type parser
key_name message
format multiline
format1 /^(?<message>(?<time>\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}),\d*\s*(?<log>[a-zA-Z_]*)\s*(?<severity>[A-Z]*).*(?
<traceback>Traceback .*))/
tag conductor.err.traceback.report
</match>
#Format traceback reports
<filter conductor.err.traceback.report>
@type record_transformer
<record>
serviceContext {
"service": "${record[\"log\"]}"
}
message ${record["traceback"]}
</record>
remove_keys traceback
</filter>
#Process errors that don't have tracebacks
<match conductor.err.message>
@type parser
key_name message
format multiline
format1 /^(?<message>(?<time>\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}),\d*\s*(?<log>[a-zA-Z_]*)\s*(?<severity>[A-Z]*):\s*(?<report>.*))/
tag conductor.err.message.report
</match>
#For errors without tracebacks we have to stub out some fields that
#error reporting requires, but we don't have
<filter conductor.err.message.report>
@type record_transformer
<record>
serviceContext {
"service": "${record[\"log\"]}"
}
message ${record["report"]}
reportLocation {
"filePath": "None",
"lineNumber": 0,
"functionName": "None"
}
</record>
</filter>
#Send to StackDriver logging!
<match conductor.**>
@type google_cloud
buffer_chunk_limit 2M
flush_interval 5s
max_retry_wait 300
disable_retry_limit
num_threads 8
</match>
您可以按照此处的教程进行操作:https://cloud.google.com/run/docs/logging#writing_structured_logs
您可以使用 python 创建一个字典并将其记录到 Stackdriver - 它将与跟踪相关联。它应该看起来像这样(来自教程):
# Build structured log messages as an object.
global_log_fields = {}
# Add log correlation to nest all log messages
# beneath request log in Log Viewer.
trace_header = request.headers.get('X-Cloud-Trace-Context')
if trace_header and PROJECT:
trace = trace_header.split('/')
global_log_fields['logging.googleapis.com/trace'] = (
f"projects/{PROJECT}/traces/{trace[0]}")
# Complete a structured log entry.
entry = dict(severity='NOTICE',
message='This is the default display field.',
# Log viewer accesses 'component' as jsonPayload.component'.
component='arbitrary-property',
**global_log_fields)
print(json.dumps(entry))
有关错误报告的更多信息:https://cloud.google.com/run/docs/error-reporting
我已经使用 Stackdriver Logging 很长时间了,现在我也想利用错误报告。如果可能(出于各种原因),我宁愿使用 Python 的日志记录机制并从日志文件中删除异常,而不是使用 error_reporting 库。话虽如此,文档非常混乱。例如文档说: https://cloud.google.com/error-reporting/docs/setup/compute-engine#log_exceptions
First, install the fluent-logger-python library:
sudo pip install google-cloud-error-reporting --upgrade
这让我相信 google-cloud-error-reporting 是 fluent-logger-python 的分支或与之相关,但是,当我初始化 google-cloud 时-error-reporting 它直接调用 GCE 元数据服务器而不是连接到本地 fluentd。这两个不相关的包还是文档错误或误导?如果我将 JSON 格式的异常发送到 fluentd 或由 fluentd 监控的日志文件,错误报告会理解它们吗?
感谢任何澄清
文档有误。
TL;DR 你必须输出看起来像 https://cloud.google.com/error-reporting/docs/formatting-error-messages
这是我的解决方案:
#Parse raw log entries to expose severity field so that
#StackDriver log viewer can properly categorize (and so we can filter)
<source>
@type tail
path /var/log/conductor
pos_file /var/log/td-agent/conductor.pos
format multiline
format_firstline /\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}/
format1 /^(?<message>(?<time>\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}),\d*\s*[a-zA-Z_]*\s*(?<severity>[A-Z]*).*)/
read_from_head true
multiline_flush_interval 3s
tag conductor.app
</source>
#Add hostname field
<filter conductor.app>
@type record_transformer
<record>
hostname ${hostname}
</record>
</filter>
#Filter and tag log entries of severity ERROR or CRITICAL
<match conductor.app>
@type rewrite_tag_filter
rewriterule1 severity ERROR|CRITICAL conductor.err
rewriterule2 severity .+ conductor.info
</match>
#Process entries with tracebacks differently than those without
<match conductor.err>
@type rewrite_tag_filter
rewriterule1 message .*Traceback conductor.err.traceback
rewriterule2 message .+ conductor.err.message
</match>
#Parse out the traceback
<match conductor.err.traceback>
@type parser
key_name message
format multiline
format1 /^(?<message>(?<time>\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}),\d*\s*(?<log>[a-zA-Z_]*)\s*(?<severity>[A-Z]*).*(?
<traceback>Traceback .*))/
tag conductor.err.traceback.report
</match>
#Format traceback reports
<filter conductor.err.traceback.report>
@type record_transformer
<record>
serviceContext {
"service": "${record[\"log\"]}"
}
message ${record["traceback"]}
</record>
remove_keys traceback
</filter>
#Process errors that don't have tracebacks
<match conductor.err.message>
@type parser
key_name message
format multiline
format1 /^(?<message>(?<time>\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}),\d*\s*(?<log>[a-zA-Z_]*)\s*(?<severity>[A-Z]*):\s*(?<report>.*))/
tag conductor.err.message.report
</match>
#For errors without tracebacks we have to stub out some fields that
#error reporting requires, but we don't have
<filter conductor.err.message.report>
@type record_transformer
<record>
serviceContext {
"service": "${record[\"log\"]}"
}
message ${record["report"]}
reportLocation {
"filePath": "None",
"lineNumber": 0,
"functionName": "None"
}
</record>
</filter>
#Send to StackDriver logging!
<match conductor.**>
@type google_cloud
buffer_chunk_limit 2M
flush_interval 5s
max_retry_wait 300
disable_retry_limit
num_threads 8
</match>
您可以按照此处的教程进行操作:https://cloud.google.com/run/docs/logging#writing_structured_logs
您可以使用 python 创建一个字典并将其记录到 Stackdriver - 它将与跟踪相关联。它应该看起来像这样(来自教程):
# Build structured log messages as an object.
global_log_fields = {}
# Add log correlation to nest all log messages
# beneath request log in Log Viewer.
trace_header = request.headers.get('X-Cloud-Trace-Context')
if trace_header and PROJECT:
trace = trace_header.split('/')
global_log_fields['logging.googleapis.com/trace'] = (
f"projects/{PROJECT}/traces/{trace[0]}")
# Complete a structured log entry.
entry = dict(severity='NOTICE',
message='This is the default display field.',
# Log viewer accesses 'component' as jsonPayload.component'.
component='arbitrary-property',
**global_log_fields)
print(json.dumps(entry))
有关错误报告的更多信息:https://cloud.google.com/run/docs/error-reporting