如何使用 python 正则表达式从日志文件中查找所有回溯？

Question

有一个来自 jenkins 的日志文件。我想从日志文件中提取所有回溯。（许多不同类型的回溯可能会出现在日志中）。如何编写 python 正则表达式？我试过下面的方法，但效果不佳。

trace_regex = r'^Traceback[\s\S]*Error.*|^Traceback[\s\S]*timeout.*|^Traceback[\s\S]*HunterDeviceOffline.*|Traceback[\s\S]*Exception.*|Traceback[\s\S]*BadStatusLine.*|Traceback[\s\S]*NoSuchDevice.*|Traceback[\s\S]*AuthenticationsFail.*'
trace = re.findall(trace_regex, log, re.M | re.I)

日志文件内容如吹：

Started by upstream project "run_script" build number 435
originally caused by:
 Started by user test
[Pipeline] {
[Pipeline] stage (Svnup)
Entering stage Svnup
+ python -m test.cli install 479CI9SOZ android://0.0.0.0:10000/479CI9SOZ /root/workspace/group/apk/com.data.app.apk - - false false
[03:21:33][DEBUG]<android> /root/env/common/test/test/core/android/adb/linux/adb -H 0.0.0.0 -P 10000 -s 479CI9SOZ wait-for-device
Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/root/env/common/test/test/cli/parser.py", line 55, in main
    run_script(args)
  File "/root/env/common/test/test/core/helper.py", line 224, in wrapper
    return f(pictarget, *args[1:], **opargs)
  File "/root/env/common/test/test/core/main.py", line 408, in assert_exists
    raise AssertionTimeout("%s does not exist in screen" % v)
AssertionTimeout: MoaPic(group_login.owl/tpl1487844291185.png) does not exist in screen
EndOfStream
[03:23:18][DEBUG]<android> /root/env/common/test/test/core/android/adb/linux/adb -H 0.0.0.0 -P 10000 -s 479CI9SOZ forward --remove tcp:12671
+ python -m test.cli run 479CI9SOZ android://0.0.0.0:10000/479CI9SOZ /root/workspace/group/util/scripts/group_login.owl 
rpc_uri: http://0.0.0.0:12708/jsonrpc/0
rpc_uri: http://0.0.0.0:12708/jsonrpc/0
Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/root/env/common/test/test/__main__.py", line 5, in <module>
    main()
  File "/root/env/common/test/test/cli/parser.py", line 55, in main
    run_script(args)
  File "/root/env/common/test/test/cli/runner.py", line 124, in run_script
    exec_script(args.script, scope=globals(), root=True)
  File "/root/env/common/test/test/cli/runner.py", line 186, in exec_script
    exec(compile(code, scriptpath, 'exec')) in scope
  File "/root/env/common/mator/mator/mator.py", line 520, in start
    raise IOError("RPC server not started!")
IOError: RPC server not started!
[03:24:40][DEBUG]<android> /root/env/common/test/test/core/android/adb/linux/adb -H 0.0.0.0 -P 10000 -s 479CI9SOZ forward --remove tcp:12708
[03:26:41][DEBUG]<main>  ->sift result: None
[03:26:41][DEBUG]<main> match result: None
Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/root/env/common/test/test/core/main.py", line 195, in touch
    pos = loop_find(v, timeout=timeout)
  File "/root/env/common/test/test/core/utils/logwraper.py", line 70, in wrapper
    res = f(*args, **kwargs)
  File "/root/env/common/test/test/core/cv.py", line 83, in loop_find
    raise NotFoundException('Picture %s not found in screen' % query)
test.core.error.NotFoundException: 'Picture MoaPic(group_login.owl/tpl1482311722698.png) not found in screen'
+ python -m test.cli stop_screen_record android://0.0.0.0:10000/479CI9SOZ 479CI9SOZ.mp4
ERROR: script returned exit code 1
Finished: FAILURE

有什么想法吗？谢谢大家

Answer 1

你可能会和

相处得很好

^Traceback  # look for Traceback right at the start of a line
[\s\S]+?    # everything lazily
(?=^\[|\Z)  # pos. lookahead, either [ at the start of a line...
            # or the very end of the string

在MULTILINE和VERBOSE模式下，见a demo on regex101.com。

在 Python 中，这将是：

import re

rx = re.compile(r'''
    ^Traceback
    [\s\S]+?
    (?=^\[|\Z)
    ''', re.M | re.X)

for match in rx.finditer(your_string_here):
    print match.group(0)

或者，也可以使用 DOTALL 修饰符（而不是 [\s\S] 构造，即）：

rx = re.compile(r'''
    ^Traceback
    .+?
    (?=^\[|\Z)
    ''', re.M | re.X | re.S)

如何使用 python 正则表达式从日志文件中查找所有回溯？

How to find all traceback from log file by using python regex?

python

regex

traceback