地址清理 Boost.Python 个模块

Address sanitizing Boost.Python modules

我的项目包括一个大型 C++ 库和 Python 绑定(通过 Boost.Python)。测试套件主要是在 Python 绑定之上编写的,我想 运行 它与消毒剂一起使用,从 ASAN 开始。

我正在 运行ning macOS(10.13.1 FWIW,但我也遇到了以前版本的问题),我似乎找不到 运行 ASAN 的方法Python 模块(我很怀疑这与Boost.Python有关,我想其他技术也是一样)。

这是一个简单的 Python 模块:

// hello_ext.cc
#include <boost/python.hpp>

char const* greet()
{
  auto* res = new char[100];
  std::strcpy(res, "Hello, world!");
  delete [] res;
  return res;
}

BOOST_PYTHON_MODULE(hello_ext)
{
  using namespace boost::python;
  def("greet", greet);
}

这是我使用的 Makefile,是为 MacPorts 制作的:

// Makefile
CXX = clang++-mp-4.0
CXXFLAGS = -g -std=c++14 -fsanitize=address -fno-omit-frame-pointer
CPPFLAGS = -isystem/opt/local/include $$($(PYTHON_CONFIG) --includes)
LDFLAGS = -L/opt/local/lib
PYTHON = python3.5
PYTHON_CONFIG = python3.5-config
LIBS = -lboost_python3-mt $$($(PYTHON_CONFIG) --ldflags)

all: hello_ext.so

hello_ext.so: hello_ext.cc
        $(CXX) $(CPPFLAGS) $(CXXFLAGS) $(LDFLAGS) -shared -o $@ $< $(LIBS)

check: all
        $(ENV) $(PYTHON) -c 'import hello_ext; print(hello_ext.greet())'

clean:
        -rm -f hello_ext.so

没有 asan,一切都很好(好吧,实际上太好了......)。但是对于 ASAN,我遇到了 LD_PRELOAD 个问题:

$ make check
python -c 'import hello_ext; print(hello_ext.greet())'
==19013==ERROR: Interceptors are not working. This may be because AddressSanitizer is loaded too late (e.g. via dlopen). Please launch the executable with:
DYLD_INSERT_LIBRARIES=/opt/local/libexec/llvm-4.0/lib/clang/4.0.1/lib/darwin/libclang_rt.asan_osx_dynamic.dylib
"interceptors not installed" && 0make: *** [check] Abort trap: 6

好的,让我们这样做:定义 DYLD_INSERT_LIBRARIES

$ DYLD_INSERT_LIBRARIES=/opt/local/libexec/llvm-4.0/lib/clang/4.0.1/lib/darwin/libclang_rt.asan_osx_dynamic.dylib \
  python -c 'import hello_ext; print(hello_ext.greet())'
==19023==ERROR: Interceptors are not working. This may be because AddressSanitizer is loaded too late (e.g. via dlopen). Please launch the executable with:
DYLD_INSERT_LIBRARIES=/opt/local/libexec/llvm-4.0/lib/clang/4.0.1/lib/darwin/libclang_rt.asan_osx_dynamic.dylib
"interceptors not installed" && 0zsh: abort      DYLD_INSERT_LIBRARIES= python -c 'import hello_ext; print(hello_ext.greet())'

让我们怀疑 SIP,所以我在这里禁用了 SIP,让我们解析符号链接:

$ DYLD_INSERT_LIBRARIES=/opt/local/libexec/llvm-4.0/lib/clang/4.0.1/lib/darwin/libclang_rt.asan_osx_dynamic.dylib \
  /opt/local/Library/Frameworks/Python.framework/Versions/3.5/bin/python3.5 -c 'import hello_ext; print(hello_ext.greet())'
==19026==ERROR: Interceptors are not working. This may be because AddressSanitizer is loaded too late (e.g. via dlopen). Please launch the executable with:
DYLD_INSERT_LIBRARIES=/opt/local/libexec/llvm-4.0/lib/clang/4.0.1/lib/darwin/libclang_rt.asan_osx_dynamic.dylib
"interceptors not installed" && 0zsh: abort      DYLD_INSERT_LIBRARIES=  -c 'import hello_ext; print(hello_ext.greet())'

正确的做法是什么?我还尝试使用 ctypes.PyDLL 加载 libasan,即使使用 sys.setdlopenflags(os.RTLD_NOW | os.RTLD_GLOBAL) 我也无法正常工作。

这可能不是一个理想的答案,但它应该能满足您的需求。

我建议你创建一个 Xcode 项目来构建所有内容(是的,我知道你想使用 make,稍后会出现)

假设您已经知道如何获得一个 Xcode 构建所有内容的项目,我将跳到启用 Address Sanitizer:

Product -> Scheme -> Edit Scheme 将打开此 window:

选中 Address Sanitizer 复选框。您只需要为您的主应用程序执行此操作。根据我的经验,所有的依赖关系都会被适当地构建。

下一步构建主要应用程序目标。我们需要检查编译器和链接器调用,以便我们可以将任何必要的 flags/steps 复制到 make 文件。

我圈出了 2 个相关按钮以获取输出。最右边的按钮将展开用于构建的编译器调用。

这张照片显示了一个保存按钮(在另一个程序中可能更容易检查)和一些 用于构建的相关标志。您将必须检查 所有 目标(依赖项)的输出以获得清晰的画面。

希望这对您有所帮助。 FWIW 我示例中的项目有一个用 C++ 编写的自定义 Python 模块(直接使用 libPython,而不是 Boost)所以我知道你可以在那些甚至使用系统提供的 Python 上使用 Asan框架。

所以,我终于设法让它工作了:

$ libasan=/opt/local/libexec/llvm-4.0/lib/clang/4.0.1/lib/darwin/libclang_rt.asan_osx_dynamic.dylib
$ python=/opt/local/Library/Frameworks/Python.framework/Versions/3.5/Resources/Python.app/Contents/MacOS/Python
$ DYLD_INSERT_LIBRARIES=$libasan $python -c 'import hello_ext; print(hello_ext.greet())'
=================================================================
==70859==ERROR: AddressSanitizer: heap-use-after-free on address 0x60b000002770 at pc 0x000108c2ef60 bp 0x7ffee6fe8c20 sp 0x7ffee6fe83c8
READ of size 2 at 0x60b000002770 thread T0
    #0 0x108c2ef5f in wrap_strlen (libclang_rt.asan_osx_dynamic.dylib:x86_64h+0x14f5f)
    #1 0x109a8d939 in PyUnicode_FromString (Python:x86_64+0x58939)
[...]

发生了什么变化?编译链中没有任何内容,只有调用。

PYDIR=/opt/local/Library/Frameworks/Python.framework/Versions/3.5:之前我调用$PYDIR/bin/python3.5(因为/opt/local/bin/python3.5是它的符号链接),现在我调用$PYDIR/Resources/Python.app/Contents/MacOS/Python.

为了了解发生了什么,我 运行 DYLD_INSERT_LIBRARIES=$libasan python3.5 并查找其打开的文件

$ ps
  PID TTY           TIME CMD
  900 ttys000    0:07.96 -zsh
70897 ttys000    0:00.11 /opt/local/Library/Frameworks/Python.framework/Versions/3.5/Resources/Python.app/Contents/MacOS/Python
53528 ttys001    0:05.14 -zsh
  920 ttys002    0:10.28 -zsh
$ lsof -p 70897
COMMAND   PID USER   FD   TYPE DEVICE   SIZE/OFF       NODE NAME
Python  70897 akim  cwd    DIR    1,4        480 8605949500 /Users/akim/src/lrde/vcsn/experiment/sanitizer
Python  70897 akim  txt    REG    1,4      12988 8591019542 /opt/local/Library/Frameworks/Python.framework/Versions/3.5/Resources/Python.app/Contents/MacOS/Python
Python  70897 akim  txt    REG    1,4    2643240 8591012758 /opt/local/Library/Frameworks/Python.framework/Versions/3.5/Python
Python  70897 akim  txt    REG    1,4     107524 8590943656 /opt/local/lib/libintl.8.dylib
Python  70897 akim  txt    REG    1,4    2097528 8590888556 /opt/local/lib/libiconv.2.dylib
Python  70897 akim  txt    REG    1,4      20224 8591016920 /opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/lib-dynload/_heapq.cpython-35m-darwin.so
Python  70897 akim  txt    REG    1,4     326996 8591375651 /opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/readline/gnureadline.cpython-35m-darwin.so
Python  70897 akim  txt    REG    1,4     603008 8605907803 /opt/local/lib/libncurses.6.dylib
Python  70897 akim  txt    REG    1,4     837248 8606849556 /usr/lib/dyld
Python  70897 akim  txt    REG    1,4 1155837952 8606860187 /private/var/db/dyld/dyld_shared_cache_x86_64h
Python  70897 akim    0u   CHR   16,0  0t2756038        667 /dev/ttys000
Python  70897 akim    1u   CHR   16,0  0t2756038        667 /dev/ttys000
Python  70897 akim    2u   CHR   16,0  0t2756038        667 /dev/ttys000

显然 libasan 不在这里,这就是全部问题。但是,我还注意到 ps 指的是另一个 Python 而不是我 运行 的那个(当然,它是打开文件的一部分)。

原来这个目录中有几个Python可执行文件:$PYDIR/bin/python3.5$PYDIR/Resources/Python.app/Contents/MacOS/Python,第一个以某种方式跳转到第二个。如果我 运行 第二个 DYLD_INSERT_LIBRARIES=$libasan

$ lsof -p 71114
COMMAND   PID USER   FD   TYPE DEVICE  SIZE/OFF       NODE NAME
Python  71114 akim  cwd    DIR    1,4       480 8605949500 /Users/akim/src/lrde/vcsn/experiment/sanitizer
Python  71114 akim  txt    REG    1,4     12988 8591019542 /opt/local/Library/Frameworks/Python.framework/Versions/3.5/Resources/Python.app/Contents/MacOS/Python
Python  71114 akim  txt    REG    1,4   3013168 8604479549 /opt/local/libexec/llvm-4.0/lib/clang/4.0.1/lib/darwin/libclang_rt.asan_osx_dynamic.dylib
Python  71114 akim  txt    REG    1,4   2643240 8591012758 /opt/local/Library/Frameworks/Python.framework/Versions/3.5/Python
Python  71114 akim  txt    REG    1,4    107524 8590943656 /opt/local/lib/libintl.8.dylib
Python  71114 akim  txt    REG    1,4   2097528 8590888556 /opt/local/lib/libiconv.2.dylib
Python  71114 akim  txt    REG    1,4     20224 8591016920 /opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/lib-dynload/_heapq.cpython-35m-darwin.so
Python  71114 akim  txt    REG    1,4    326996 8591375651 /opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/readline/gnureadline.cpython-35m-darwin.so
Python  71114 akim  txt    REG    1,4    603008 8605907803 /opt/local/lib/libncurses.6.dylib
Python  71114 akim  txt    REG    1,4    837248 8606849556 /usr/lib/dyld
Python  71114 akim    0u   CHR   16,0 0t2781894        667 /dev/ttys000
Python  71114 akim    1u   CHR   16,0 0t2781894        667 /dev/ttys000
Python  71114 akim    2u   CHR   16,0 0t2781894        667 /dev/ttys000

\o/ libasan 就在那里!很明显,第一个 Python 调用了第二个,而 DYLD_INSERT_LIBRARIES 没有被转发。

我目前不知道为什么有两个 Python。它似乎并不特定于 MacPorts,因为苹果的 Python.

也是如此
$ cd /System/Library/Frameworks/Python.framework/Versions/2.7
$ ls -l bin/python*
lrwxr-xr-x  1 root  wheel      7 10 déc 08:17 bin/python -> python2
lrwxr-xr-x  1 root  wheel     14 10 déc 08:17 bin/python-config -> python2-config
lrwxr-xr-x  1 root  wheel      9 10 déc 08:17 bin/python2 -> python2.7
lrwxr-xr-x  1 root  wheel     16 10 déc 08:17 bin/python2-config -> python2.7-config
-rwxr-xr-x  1 root  wheel  43104  1 déc 21:42 bin/python2.7
-rwxr-xr-x  1 root  wheel   1818 16 jul 02:20 bin/python2.7-config
lrwxr-xr-x  1 root  wheel      8 10 déc 08:17 bin/pythonw -> pythonw2
lrwxr-xr-x  1 root  wheel     10 10 déc 08:17 bin/pythonw2 -> pythonw2.7
-rwxr-xr-x  1 root  wheel  43104  1 déc 21:42 bin/pythonw2.7
$ ls -l Resources/Python.app/Contents/MacOS/Python
-rwxr-xr-x  1 root  wheel  51744  1 déc 21:48 Resources/Python.app/Contents/MacOS/Python