将 FreeTDS ODBC 与 pyodbc 结合使用时,glibc 检测到损坏的双链表

glibc detected corrupted double-linked list when using FreeTDS ODBC with pyodbc

我们的生产环境中 运行 pyodbc 4.0.1 突然开始面临内存错误。我不确定为什么会开始发生这种情况,因为在那之前一切正常...是否有任何已知的服务器配置问题的根本原因?

我们已将 DEV 和 UAT 中的 Pyodbc 版本从 4.0.1 升级到 4.0.16。不幸的是,问题仍然存在并且可以重现(见下面的跟踪)。

[...]
*** glibc detected *** python: corrupted double-linked list: 0x0000000002644650 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x75f4e)[0x7f751c639f4e]
/lib64/libc.so.6(+0x79066)[0x7f751c63d066]
/usr/lib64/libtdsodbc.so.0(+0x1e2fd)[0x7f74fd2aa2fd]
/usr/lib64/libtdsodbc.so.0(+0x2095e)[0x7f74fd2ac95e]
/usr/lib64/libtdsodbc.so.0(+0x27de5)[0x7f74fd2b3de5]
/usr/lib64/libtdsodbc.so.0(+0x3729f)[0x7f74fd2c329f]
/usr/lib64/libtdsodbc.so.0(+0x11e15)[0x7f74fd29de15]
/usr/lib64/libtdsodbc.so.0(SQLGetTypeInfo+0x187)[0x7f74fd2a0547]
/opt/cloudera/parcels/Anaconda3-4.2.0-python-3.5/envs/ed/lib/libodbc.so.2(SQLGetTypeInfo+0x1e3)[0x7f75033c8c93]
/opt/cloudera/parcels/Anaconda3-4.2.0-python-3.5/envs/ed/lib/python3.5/site-packages/pyodbc.cpython-35m-x86_64-linux-gnu.so(_Z17GetConnectionInfoP7_objectP10Connection+0x42f)[0x7f75036297af]
/opt/cloudera/parcels/Anaconda3-4.2.0-python-3.5/envs/ed/lib/python3.5/site-packages/pyodbc.cpython-35m-x86_64-linux-gnu.so(_Z14Connection_NewP7_objectbblbS0_R6Object+0x6a9)[0x7f7503621249]
/opt/cloudera/parcels/Anaconda3-4.2.0-python-3.5/envs/ed/lib/python3.5/site-packages/pyodbc.cpython-35m-x86_64-linux-gnu.so(+0xb174)[0x7f7503622174]
/opt/cloudera/parcels/Anaconda3-4.2.0-python-3.5/envs/ed/bin/../lib/libpython3.5m.so.1.0(PyCFunction_Call+0x131)[0x7f751d4c4621]
/opt/cloudera/parcels/Anaconda3-4.2.0-python-3.5/envs/ed/bin/../lib/libpython3.5m.so.1.0(PyEval_EvalFrameEx+0x9017)[0x7f751d54bc37]
/opt/cloudera/parcels/Anaconda3-4.2.0-python-3.5/envs/ed/bin/../lib/libpython3.5m.so.1.0(PyEval_EvalFrameEx+0x9546)[0x7f751d54c166]
/opt/cloudera/parcels/Anaconda3-4.2.0-python-3.5/envs/ed/bin/../lib/libpython3.5m.so.1.0(PyEval_EvalFrameEx+0x9546)[0x7f751d54c166]
/opt/cloudera/parcels/Anaconda3-4.2.0-python-3.5/envs/ed/bin/../lib/libpython3.5m.so.1.0(PyEval_EvalFrameEx+0x9546)[0x7f751d54c166]
/opt/cloudera/parcels/Anaconda3-4.2.0-python-3.5/envs/ed/bin/../lib/libpython3.5m.so.1.0(+0x144b49)[0x7f751d54cb49]
/opt/cloudera/parcels/Anaconda3-4.2.0-python-3.5/envs/ed/bin/../lib/libpython3.5m.so.1.0(PyEval_EvalCodeEx+0x48)[0x7f751d54ccd8]
/opt/cloudera/parcels/Anaconda3-4.2.0-python-3.5/envs/ed/bin/../lib/libpython3.5m.so.1.0(PyEval_EvalCode+0x3b)[0x7f751d54cd1b]
/opt/cloudera/parcels/Anaconda3-4.2.0-python-3.5/envs/ed/bin/../lib/libpython3.5m.so.1.0(PyRun_FileExFlags+0x130)[0x7f751d572020]
/opt/cloudera/parcels/Anaconda3-4.2.0-python-3.5/envs/ed/bin/../lib/libpython3.5m.so.1.0(PyRun_SimpleFileExFlags+0x173)[0x7f751d573623]
/opt/cloudera/parcels/Anaconda3-4.2.0-python-3.5/envs/ed/bin/../lib/libpython3.5m.so.1.0(Py_Main+0xca7)[0x7f751d58e8c7]
python(main+0x15d)[0x400add]
/lib64/libc.so.6(__libc_start_main+0xfd)[0x7f751c5e2d5d]
python[0x4008b9]
======= Memory map: ========
00400000-00401000 r-xp 00000000 fd:05 11011012                           /opt/cloudera/parcels/Anaconda3-4.2.0-python-3.5/envs/ed/bin/python3.5
00601000-00602000 rw-p 00001000 fd:05 11011012                           /opt/cloudera/parcels/Anaconda3-4.2.0-python-3.5/envs/ed/bin/python3.5
01241000-02873000 rw-p 00000000 00:00 0                                  [heap]
7f74f8000000-7f74f8021000 rw-p 00000000 00:00 0
7f74f8021000-7f74fc000000 ---p 00000000 00:00 0
7f74fc354000-7f74fc356000 r-xp 00000000 fd:01 3837                       /usr/lib64/gconv/CP1252.so
7f74fc356000-7f74fc555000 ---p 00002000 fd:01 3837                       /usr/lib64/gconv/CP1252.so
7f74fc555000-7f74fc556000 r--p 00001000 fd:01 3837                       /usr/lib64/gconv/CP1252.so
7f74fc556000-7f74fc557000 rw-p 00002000 fd:01 3837                       /usr/lib64/gconv/CP1252.so
7f74fc557000-7f74fc558000 r-xp 00000000 fd:01 4029                       /usr/lib64/gconv/ISO8859-1.so
7f74fc558000-7f74fc758000 ---p 00001000 fd:01 4029                       /usr/lib64/gconv/ISO8859-1.so
7f74fc758000-7f74fc759000 r--p 00001000 fd:01 4029                       /usr/lib64/gconv/ISO8859-1.so
7f74fc759000-7f74fc75a000 rw-p 00002000 fd:01 4029                       /usr/lib64/gconv/ISO8859-1.so
7f74fc75a000-7f74fc75f000 r-xp 00000000 fd:01 3803                       /lib64/libnss_dns-2.12.so
7f74fc75f000-7f74fc95e000 ---p 00005000 fd:01 3803                       /lib64/libnss_dns-2.12.so
7f74fc95e000-7f74fc95f000 r--p 00004000 fd:01 3803                       /lib64/libnss_dns-2.12.so
7f74fc95f000-7f74fc960000 rw-p 00005000 fd:01 3803                       /lib64/libnss_dns-2.12.so
7f74fc960000-7f74fc963000 r-xp 00000000 fd:01 5388                       /lib64/libgpg-error.so.0.5.0
7f74fc963000-7f74fcb62000 ---p 00003000 fd:01 5388                       /lib64/libgpg-error.so.0.5.0
7f74fcb62000-7f74fcb63000 r--p 00002000 fd:01 5388                       /lib64/libgpg-error.so.0.5.0
7f74fcb63000-7f74fcb64000 rw-p 00003000 fd:01 5388                       /lib64/libgpg-error.so.0.5.0
7f74fcb64000-7f74fcb74000 r-xp 00000000 fd:01 6697                       /usr/lib64/libtasn1.so.3.1.6
7f74fcb74000-7f74fcd73000 ---p 00010000 fd:01 6697                       /usr/lib64/libtasn1.so.3.1.6
7f74fcd73000-7f74fcd74000 rw-p 0000f000 fd:01 6697                       /usr/lib64/libtasn1.so.3.1.6
7f74fcd74000-7f74fcde6000 r-xp 00000000 fd:01 5401                       /lib64/libgcrypt.so.11.5.3
7f74fcde6000-7f74fcfe5000 ---p 00072000 fd:01 5401                       /lib64/libgcrypt.so.11.5.3
7f74fcfe5000-7f74fcfe6000 r--p 00071000 fd:01 5401                       /lib64/libgcrypt.so.11.5.3
7f74fcfe6000-7f74fcfe9000 rw-p 00072000 fd:01 5401                       /lib64/libgcrypt.so.11.5.3
7f74fcfe9000-7f74fd086000 r-xp 00000000 fd:01 13301                      /usr/lib64/libgnutls.so.26.14.12
7f74fd086000-7f74fd285000 ---p 0009d000 fd:01 13301                      /usr/lib64/libgnutls.so.26.14.12
7f74fd285000-7f74fd28c000 rw-p 0009c000 fd:01 13301                      /usr/lib64/libgnutls.so.26.14.12
7f74fd28c000-7f74fd2ee000 r-xp 00000000 fd:01 52715                      /usr/lib64/libtdsodbc.so.0.0.0
7f74fd2ee000-7f74fd4ed000 ---p 00062000 fd:01 52715                      /usr/lib64/libtdsodbc.so.0.0.0
7f74fd4ed000-7f74fd4f1000 rw-p 00061000 fd:01 52715                      /usr/lib64/libtdsodbc.so.0.0.0
7f74fd4f1000-7f74fd4f5000 r-xp 00000000 fd:05 7997029                    /opt/cloudera/parcels/Anaconda3-4.2.0-python-3.5/pkgs/python-3.5.2-0/lib/python3.5/lib-dynload/termios.so
7f74fd4f5000-7f74fd6f5000 ---p 00004000 fd:05 7997029                    /opt/cloudera/parcels/Anaconda3-4.2.0-python-3.5/pkgs/python-3.5.2-0/lib/python3.5/lib-dynload/termios.so
7f74fd6f5000-7f74fd6f7000 rw-p 00004000 fd:05 7997029                    /opt/cloudera/parcels/Anaconda3-4.2.0-python-3.5/pkgs/python-3.5.2-0/lib/python3.5/lib-dynload/termios.so
7f74fd6f7000-7f74fd6fa000 r-xp 00000000 fd:05 7997070                    /opt/cloudera/parcels/Anaconda3-4.2.0-python-3.5/pkgs/python-3.5.2-0/lib/python3.5/lib-dynload/fcntl.so
7f74fd6fa000-7f74fd8f9000 ---p 00003000 fd:05 7997070                    /opt/cloudera/parcels/Anaconda3-4.2.0-python-3.5/pkgs/python-3.5.2-0/lib/python3.5/lib-dynload/fcntl.so
7f74fd8f9000-7f74fd8fb000 rw-p 00002000 fd:05 7997070                    /opt/cloudera/parcels/Anaconda3-4.2.0-python-3.5/pkgs/python-3.5.2-0/lib/python3.5/lib-dynload/fcntl.so
7f74fd8fb000-7f74fd8fe000 r-xp 00000000 fd:05 10749483                   /opt/cloudera/parcels/Anaconda3-4.2.0-python-3.5/pkgs/sqlalchemy-1.0.13-py35_0/lib/python3.5/site-packages/sqlalchemy/cresultproxy.so
7f74fd8fe000-7f74fdafe000 ---p 00003000 fd:05 10749483                   /opt/cloudera/parcels/Anaconda3-4.2.0-python-3.5/pkgs/sqlalchemy-1.0.13-py35_0/lib/python3.5/site-packages/sqlalchemy/cresultproxy.so
7f74fdafe000-7f74fdaff000 rw-p 00003000 fd:05 10749483                   /opt/cloudera/parcels/Anaconda3-4.2.0-python-3.5/pkgs/sqlalchemy-1.0.13-py35_0/lib/python3.5/site-packages/sqlalchemy/cresultproxy.so
7f74fdaff000-7f74fdb00000 r-xp 00000000 fd:05 10749167                   /opt/cloudera/parcels/Anaconda3-4.2.0-python-3.5/pkgs/sqlalchemy-1.0.13-py35_0/lib/python3.5/site-packages/sqlalchemy/cutils.so
7f74fdb00000-7f74fdd00000 ---p 00001000 fd:05 10749167                   /opt/cloudera/parcels/Anaconda3-4.2.0-python-3.5/pkgs/sqlalchemy-1.0.13-py35_0/lib/python3.5/site-packages/sqlalchemy/cutils.so
7f74fdd00000-7f74fdd01000 rw-p 00001000 fd:05 10749167                   /opt/cloudera/parcels/Anaconda3-4.2.0-python-3.5/pkgs/sqlalchemy-1.0.13-py35_0/lib/python3.5/site-packages/sqlalchemy/cutils.so
7f74fdd01000-7f74fdd41000 rw-p 00000000 00:00 0
7f74fdec1000-7f74fdf01000 rw-p 00000000 00:00 0
7f74fdf41000-7f74fdf44000 r-xp 00000000 fd:05 10749471                   /opt/cloudera/parcels/Anaconda3-4.2.0-python-3.5/pkgs/sqlalchemy-1.0.13-py35_0/lib/python3.5/site-packages/sqlalchemy/cprocessors.so
7f74fdf44000-7f74fe143000 ---p 00003000 fd:05 10749471                   /opt/cloudera/parcels/Anaconda3-4.2.0-python-3.5/pkgs/sqlalchemy-1.0.13-py35_0/lib/python3.5/site-packages/sqlalchemy/cprocessors.so
7f74fe143000-7f74fe144000 rw-p 00002000 fd:05 10749471                   /opt/cloudera/parcels/Anaconda3-4.2.0-python-3.5/pkgs/sqlalchemy-1.0.13-py35_0/lib/python3.5/site-packages/sqlalchemy/cprocessors.soAborted

在对我们进行更多测试后,问题似乎是在连接到 Microsoft SQL 服务器数据库时出现的。我们使用以下连接字符串连接 dsn-less。话虽如此,我不确定内存问题何时发生(连接之前/期间/之后)...问题似乎或多或少是随机触发的。

[...]
pyodbc.pooling=False
oConnexion = pyodbc.connect(Driver="{FreeTDS};Server=xxx;Port=1433;TDS_Version=7.0;UID=xxx;PWD=xxx;", autocommit=True)
oConnexion.setdecoding(pyodbc.SQL_CHAR, encoding='utf-8')
oConnexion.setdecoding(pyodbc.SQL_WCHAR, encoding='utf-8')
oConnexion.setdecoding(pyodbc.SQL_WMETADATA, encoding='utf-8')
oConnexion.setencoding(encoding='utf-8')
[...]

配置:
- Python 3.5.2
- 蟒蛇 4.2.0(64 位)
- linux
上的 Red Hat 4.4.7-1 - Pyodbc 4.0.16

I'm not sure why it started to happen since everything was working fine until then...

一切正常,直到你改变了一些东西。你需要知道你改变了什么,然后把它改回来。

Is there any known root cause for the issue in terms of server configuration?

此问题与服务器配置无关。问题是您正在加载到 python 中的内容存在错误并导致堆损坏。

您可以通过 Valgrind 下的 运行 您的 python 找到堆损坏发生的位置。

ODBC 驱动程序的先前配置是 FreeTDS v0.91。 将 ODBC 驱动程序升级到 FreeTDS 1.0rc5 解决了这个问题。