在 M1 mac 上安装 lxml 时出现问题

Problems installing lxml on M1 mac

所以,我在安装 lxml 时遇到了经典问题。

最初我只是安装 pip,但是当我尝试使用 Element.clear() 释放内存时,出现以下错误:

Python(58695,0x1001b4580) malloc: *** error for object 0x600000bc3f60: pointer being freed was not allocated

我认为这一定是因为 lxml 使用的可能是过时的系统 libxml2。

所以我使用 homebrew 安装了 libxml2 和 libxlt,并强制链接了它们。

然后我尝试使用以下命令进行安装:

❯ STATIC_DEPS=true pip install lxml --no-cache-dir                                                13:01:46
Collecting lxml
  Downloading lxml-4.8.0.tar.gz (3.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.2/3.2 MB 5.4 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... done
Building wheels for collected packages: lxml
  Building wheel for lxml (setup.py) ... done
  Created wheel for lxml: filename=lxml-4.8.0-cp310-cp310-macosx_12_0_arm64.whl size=1683935 sha256=47912c1ba66d274c3ad7b2a2db00243f96d334a3fd5e439725f5005a7a72a602
  Stored in directory: /private/var/folders/g9/lqph46sj36n9kkvjt1pzdxhm0000gn/T/pip-ephem-wheel-cache-4_v4ov7s/wheels/e4/52/34/64064e2e2f1ce84d212a6dde6676f3227846210a7996fc2530
Successfully built lxml
Installing collected packages: lxml
Successfully installed lxml-4.8.0

..但是当我尝试导入 etree 时,我会得到这个错误:

Traceback (most recent call last):
  File "/Users/human/Code/ia_book_images/viewer/book_image_downloader.py", line 4, in <module>
    from lxml import etree as ET
ImportError: dlopen(/Users/human/.virtualenvs/ia_book_images/lib/python3.10/site-packages/lxml/etree.cpython-310-darwin.so, 0x0002): symbol not found in flat namespace '___htmlDefaultSAXHandler'

然后我想让我们 100% 确定它使用 CFLAGS 的正确版本的 libxml2 并得到以下结果:

❯ CFLAGS="-I/opt/homebrew/opt/libxslt/include -I/opt/homebrew/opt/libxml2/include" STATIC_DEPS=true pip install lxml --no-cache-dir
Collecting lxml
  Downloading lxml-4.8.0.tar.gz (3.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.2/3.2 MB 4.4 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [199 lines of output]
      Checking for gcc...
      Checking for shared library support...
      Building shared library libz.1.2.12.dylib with gcc.
      Checking for size_t... Yes.
      Checking for off64_t... No.
      Checking for fseeko... Yes.
      Checking for strerror... Yes.
      Checking for unistd.h... Yes.
      Checking for stdarg.h... Yes.
      Checking whether to use vs[n]printf() or s[n]printf()... using vs[n]printf().
      Checking for vsnprintf() in stdio.h... Yes.
      Checking for return value of vsnprintf()... Yes.
      Checking for attribute(visibility) support... Yes.
      gcc -I/opt/homebrew/opt/libxslt/include -I/opt/homebrew/opt/libxml2/include -DHAVE_HIDDEN -I. -c -o example.o test/example.c
      gcc -I/opt/homebrew/opt/libxslt/include -I/opt/homebrew/opt/libxml2/include -DHAVE_HIDDEN  -c -o adler32.o adler32.c
      gcc -I/opt/homebrew/opt/libxslt/include -I/opt/homebrew/opt/libxml2/include -DHAVE_HIDDEN  -c -o crc32.o crc32.c
      gcc -I/opt/homebrew/opt/libxslt/include -I/opt/homebrew/opt/libxml2/include -DHAVE_HIDDEN  -c -o deflate.o deflate.c
      gcc -I/opt/homebrew/opt/libxslt/include -I/opt/homebrew/opt/libxml2/include -DHAVE_HIDDEN  -c -o infback.o infback.c
      gcc -I/opt/homebrew/opt/libxslt/include -I/opt/homebrew/opt/libxml2/include -DHAVE_HIDDEN  -c -o inffast.o inffast.c
      gcc -I/opt/homebrew/opt/libxslt/include -I/opt/homebrew/opt/libxml2/include -DHAVE_HIDDEN  -c -o inflate.o inflate.c
      gcc -I/opt/homebrew/opt/libxslt/include -I/opt/homebrew/opt/libxml2/include -DHAVE_HIDDEN  -c -o inftrees.o inftrees.c
      gcc -I/opt/homebrew/opt/libxslt/include -I/opt/homebrew/opt/libxml2/include -DHAVE_HIDDEN  -c -o trees.o trees.c
      gcc -I/opt/homebrew/opt/libxslt/include -I/opt/homebrew/opt/libxml2/include -DHAVE_HIDDEN  -c -o zutil.o zutil.c
      gcc -I/opt/homebrew/opt/libxslt/include -I/opt/homebrew/opt/libxml2/include -DHAVE_HIDDEN  -c -o compress.o compress.c
      gcc -I/opt/homebrew/opt/libxslt/include -I/opt/homebrew/opt/libxml2/include -DHAVE_HIDDEN  -c -o uncompr.o uncompr.c
      gcc -I/opt/homebrew/opt/libxslt/include -I/opt/homebrew/opt/libxml2/include -DHAVE_HIDDEN  -c -o gzclose.o gzclose.c
      gcc -I/opt/homebrew/opt/libxslt/include -I/opt/homebrew/opt/libxml2/include -DHAVE_HIDDEN  -c -o gzlib.o gzlib.c
      gcc -I/opt/homebrew/opt/libxslt/include -I/opt/homebrew/opt/libxml2/include -DHAVE_HIDDEN  -c -o gzread.o gzread.c
      gcc -I/opt/homebrew/opt/libxslt/include -I/opt/homebrew/opt/libxml2/include -DHAVE_HIDDEN  -c -o gzwrite.o gzwrite.c
      gcc -I/opt/homebrew/opt/libxslt/include -I/opt/homebrew/opt/libxml2/include -DHAVE_HIDDEN -I. -c -o minigzip.o test/minigzip.c
      gcc -I/opt/homebrew/opt/libxslt/include -I/opt/homebrew/opt/libxml2/include -fPIC -DHAVE_HIDDEN  -DPIC -c -o objs/adler32.o adler32.c
      gcc -I/opt/homebrew/opt/libxslt/include -I/opt/homebrew/opt/libxml2/include -fPIC -DHAVE_HIDDEN  -DPIC -c -o objs/crc32.o crc32.c
      gcc -I/opt/homebrew/opt/libxslt/include -I/opt/homebrew/opt/libxml2/include -fPIC -DHAVE_HIDDEN  -DPIC -c -o objs/deflate.o deflate.c
      gcc -I/opt/homebrew/opt/libxslt/include -I/opt/homebrew/opt/libxml2/include -fPIC -DHAVE_HIDDEN  -DPIC -c -o objs/infback.o infback.c
      gcc -I/opt/homebrew/opt/libxslt/include -I/opt/homebrew/opt/libxml2/include -fPIC -DHAVE_HIDDEN  -DPIC -c -o objs/inflate.o inflate.c
      gcc -I/opt/homebrew/opt/libxslt/include -I/opt/homebrew/opt/libxml2/include -fPIC -DHAVE_HIDDEN  -DPIC -c -o objs/inffast.o inffast.c
      gcc -I/opt/homebrew/opt/libxslt/include -I/opt/homebrew/opt/libxml2/include -fPIC -DHAVE_HIDDEN  -DPIC -c -o objs/inftrees.o inftrees.c
      gcc -I/opt/homebrew/opt/libxslt/include -I/opt/homebrew/opt/libxml2/include -fPIC -DHAVE_HIDDEN  -DPIC -c -o objs/trees.o trees.c
      gcc -I/opt/homebrew/opt/libxslt/include -I/opt/homebrew/opt/libxml2/include -fPIC -DHAVE_HIDDEN  -DPIC -c -o objs/zutil.o zutil.c
      gcc -I/opt/homebrew/opt/libxslt/include -I/opt/homebrew/opt/libxml2/include -fPIC -DHAVE_HIDDEN  -DPIC -c -o objs/gzclose.o gzclose.c
      gcc -I/opt/homebrew/opt/libxslt/include -I/opt/homebrew/opt/libxml2/include -fPIC -DHAVE_HIDDEN  -DPIC -c -o objs/uncompr.o uncompr.c
      gcc -I/opt/homebrew/opt/libxslt/include -I/opt/homebrew/opt/libxml2/include -fPIC -DHAVE_HIDDEN  -DPIC -c -o objs/compress.o compress.c
      gcc -I/opt/homebrew/opt/libxslt/include -I/opt/homebrew/opt/libxml2/include -fPIC -DHAVE_HIDDEN  -DPIC -c -o objs/gzlib.o gzlib.c
      libtool -o libz.a adler32.o crc32.o deflate.o infback.o inffast.o inflate.o inftrees.o trees.o zutil.o compress.o uncompr.o gzclose.o gzlib.o gzread.o gzwrite.o
      gcc -I/opt/homebrew/opt/libxslt/include -I/opt/homebrew/opt/libxml2/include -fPIC -DHAVE_HIDDEN  -DPIC -c -o objs/gzread.o gzread.c
      gcc -I/opt/homebrew/opt/libxslt/include -I/opt/homebrew/opt/libxml2/include -fPIC -DHAVE_HIDDEN  -DPIC -c -o objs/gzwrite.o gzwrite.c
      gcc -I/opt/homebrew/opt/libxslt/include -I/opt/homebrew/opt/libxml2/include -DHAVE_HIDDEN -o example example.o -L. libz.a
      gcc -I/opt/homebrew/opt/libxslt/include -I/opt/homebrew/opt/libxml2/include -DHAVE_HIDDEN -o minigzip minigzip.o -L. libz.a
      gcc -dynamiclib -install_name /private/var/folders/g9/lqph46sj36n9kkvjt1pzdxhm0000gn/T/pip-install-kl4hmrrk/lxml_4ecb3c255ad049e39a89a66ee0a50e76/build/tmp/libxml2/lib/libz.1.dylib -compatibility_version 1 -current_version 1.2.12 -I/opt/homebrew/opt/libxslt/include -I/opt/homebrew/opt/libxml2/include -fPIC -DHAVE_HIDDEN -o libz.1.2.12.dylib adler32.lo crc32.lo deflate.lo infback.lo inffast.lo inflate.lo inftrees.lo trees.lo zutil.lo compress.lo uncompr.lo gzclose.lo gzlib.lo gzread.lo gzwrite.lo  -lc -arch x86_64
      ld: warning: ignoring file crc32.lo, building for macOS-x86_64 but attempting to link with file built for unknown-arm64
      ld: warning: ignoring file adler32.lo, building for macOS-x86_64 but attempting to link with file built for unknown-arm64
      ld: warning: ignoring file deflate.lo, building for macOS-x86_64 but attempting to link with file built for unknown-arm64
      ld: warning: ignoring file infback.lo, building for macOS-x86_64 but attempting to link with file built for unknown-arm64
      ld: warning: ignoring file inffast.lo, building for macOS-x86_64 but attempting to link with file built for unknown-arm64
      ld: warning: ignoring file inflate.lo, building for macOS-x86_64 but attempting to link with file built for unknown-arm64
      ld: warning: ignoring file inftrees.lo, building for macOS-x86_64 but attempting to link with file built for unknown-arm64
      ld: warning: ignoring file trees.lo, building for macOS-x86_64 but attempting to link with file built for unknown-arm64
      ld: warning: ignoring file compress.lo, building for macOS-x86_64 but attempting to link with file built for unknown-arm64
      ld: warning: ignoring file zutil.lo, building for macOS-x86_64 but attempting to link with file built for unknown-arm64
      ld: warning: ignoring file uncompr.lo, building for macOS-x86_64 but attempting to link with file built for unknown-arm64
      ld: warning: ignoring file gzread.lo, building for macOS-x86_64 but attempting to link with file built for unknown-arm64
      ld: warning: ignoring file gzlib.lo, building for macOS-x86_64 but attempting to link with file built for unknown-arm64
      ld: warning: ignoring file gzclose.lo, building for macOS-x86_64 but attempting to link with file built for unknown-arm64
      ld: warning: ignoring file gzwrite.lo, building for macOS-x86_64 but attempting to link with file built for unknown-arm64
      rm -f libz.dylib libz.1.dylib
      ln -s libz.1.2.12.dylib libz.dylib
      ln -s libz.1.2.12.dylib libz.1.dylib
      gcc -I/opt/homebrew/opt/libxslt/include -I/opt/homebrew/opt/libxml2/include -DHAVE_HIDDEN -o examplesh example.o -L. libz.1.2.12.dylib
      gcc -I/opt/homebrew/opt/libxslt/include -I/opt/homebrew/opt/libxml2/include -DHAVE_HIDDEN -o minigzipsh minigzip.o -L. libz.1.2.12.dylib
      ld: warning: ignoring file libz.1.2.12.dylib, building for macOS-arm64 but attempting to link with file built for macOS-x86_64
      ld: warning: ignoring file libz.1.2.12.dylib, building for macOS-arm64 but attempting to link with file built for macOS-x86_64
      Undefined symbols for architecture arm64:
        "_gzclose", referenced from:
            _gz_compress in minigzip.o
            _gz_uncompress in minigzip.o
        "_gzdopen", referenced from:
            _main in minigzip.o
        "_gzerror", referenced from:
            _gz_compress in minigzip.o
            _gz_uncompress in minigzip.o
        "_gzopen", referenced from:
            _file_compress in minigzip.o
            _file_uncompress in minigzip.o
            _main in minigzip.o
        "_gzread", referenced from:
            _gz_uncompress in minigzip.o
        "_gzwrite", referenced from:
            _gz_compress in minigzip.o
      ld: symbol(s) not found for architecture arm64
      Undefined symbols for architecture arm64:
        "_compress", referenced from:
            _test_compress in example.o
           (maybe you meant: _test_compress)
        "_deflate", referenced from:
            _test_deflate in example.o
            _test_large_deflate in example.o
            _test_flush in example.o
            _test_dict_deflate in example.o
           (maybe you meant: _test_large_deflate, _test_deflate , _test_dict_deflate )
        "_deflateEnd", referenced from:
            _test_deflate in example.o
            _test_large_deflate in example.o
            _test_flush in example.o
            _test_dict_deflate in example.o
        "_deflateInit_", referenced from:
            _test_deflate in example.o
            _test_large_deflate in example.o
            _test_flush in example.o
            _test_dict_deflate in example.o
        "_deflateParams", referenced from:
            _test_large_deflate in example.o
        "_deflateSetDictionary", referenced from:
            _test_dict_deflate in example.o
        "_gzclose", referenced from:
            _test_gzio in example.o
        "_gzerror", referenced from:
            _test_gzio in example.o
        "_gzgetc", referenced from:
            _test_gzio in example.o
        "_gzgets", referenced from:
            _test_gzio in example.o
        "_gzopen", referenced from:
            _test_gzio in example.o
        "_gzprintf", referenced from:
            _test_gzio in example.o
        "_gzputc", referenced from:
            _test_gzio in example.o
        "_gzputs", referenced from:
            _test_gzio in example.o
        "_gzread", referenced from:
            _test_gzio in example.o
        "_gzseek", referenced from:
            _test_gzio in example.o
        "_gztell", referenced from:
            _test_gzio in example.o
        "_gzungetc", referenced from:
            _test_gzio in example.o
        "_inflate", referenced from:
            _test_inflate in example.o
            _test_large_inflate in example.o
            _test_sync in example.o
            _test_dict_inflate in example.o
           (maybe you meant: _test_large_inflate, _test_inflate , _test_dict_inflate )
        "_inflateEnd", referenced from:
            _test_inflate in example.o
            _test_large_inflate in example.o
            _test_sync in example.o
            _test_dict_inflate in example.o
        "_inflateInit_", referenced from:
            _test_inflate in example.o
            _test_large_inflate in example.o
            _test_sync in example.o
            _test_dict_inflate in example.o
        "_inflateSetDictionary", referenced from:
            _test_dict_inflate in example.o
        "_inflateSync", referenced from:
            _test_sync in example.o
        "_uncompress", referenced from:
            _test_compress in example.o
        "_zlibCompileFlags", referenced from:
            _main in example.o
        "_zlibVersion", referenced from:
            _main in example.o
      clang: error: linker command failed with exit code 1 (use -v to see invocation)
      ld: symbol(s) not found for architecture arm64
      clang: error: linker command failed with exit code 1 (use -v to see invocation)
      make: *** [minigzipsh] Error 1
      make: *** Waiting for unfinished jobs....
      make: *** [examplesh] Error 1
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/private/var/folders/g9/lqph46sj36n9kkvjt1pzdxhm0000gn/T/pip-install-kl4hmrrk/lxml_4ecb3c255ad049e39a89a66ee0a50e76/setup.py", line 270, in <module>
          **setup_extra_options()
        File "/private/var/folders/g9/lqph46sj36n9kkvjt1pzdxhm0000gn/T/pip-install-kl4hmrrk/lxml_4ecb3c255ad049e39a89a66ee0a50e76/setup.py", line 162, in setup_extra_options
          ext_modules = setupinfo.ext_modules(
        File "/private/var/folders/g9/lqph46sj36n9kkvjt1pzdxhm0000gn/T/pip-install-kl4hmrrk/lxml_4ecb3c255ad049e39a89a66ee0a50e76/setupinfo.py", line 74, in ext_modules
          XML2_CONFIG, XSLT_CONFIG = build_libxml2xslt(
        File "/private/var/folders/g9/lqph46sj36n9kkvjt1pzdxhm0000gn/T/pip-install-kl4hmrrk/lxml_4ecb3c255ad049e39a89a66ee0a50e76/buildlibxml.py", line 428, in build_libxml2xslt
          cmmi(zlib_configure_cmd, zlib_dir, multicore, **call_setup)
        File "/private/var/folders/g9/lqph46sj36n9kkvjt1pzdxhm0000gn/T/pip-install-kl4hmrrk/lxml_4ecb3c255ad049e39a89a66ee0a50e76/buildlibxml.py", line 352, in cmmi
          call_subprocess(
        File "/private/var/folders/g9/lqph46sj36n9kkvjt1pzdxhm0000gn/T/pip-install-kl4hmrrk/lxml_4ecb3c255ad049e39a89a66ee0a50e76/buildlibxml.py", line 335, in call_subprocess
          raise Exception('Command "%s" returned code %s' % (cmd_desc, returncode))
      Exception: Command "make -j6" returned code 2
      Building lxml version 4.8.0.
      Latest version of zlib is 1.2.12
      Downloading zlib into libs/zlib-1.2.12.tar.gz from https://zlib.net/zlib-1.2.12.tar.gz
      Unpacking zlib-1.2.12.tar.gz into build/tmp
      Latest version of libiconv is 1.16
      Downloading libiconv into libs/libiconv-1.16.tar.gz from https://ftp.gnu.org/pub/gnu/libiconv/libiconv-1.16.tar.gz
      Unpacking libiconv-1.16.tar.gz into build/tmp
      Latest version of libxml2 is 2.9.12
      Downloading libxml2 into libs/libxml2-2.9.12.tar.gz from http://xmlsoft.org/sources/libxml2-2.9.12.tar.gz
      Unpacking libxml2-2.9.12.tar.gz into build/tmp
      Latest version of libxslt is 1.1.34
      Downloading libxslt into libs/libxslt-1.1.34.tar.gz from http://xmlsoft.org/sources/libxslt-1.1.34.tar.gz
      Unpacking libxslt-1.1.34.tar.gz into build/tmp
      Starting build in build/tmp/zlib-1.2.12
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

我是否需要做一些特殊的事情来在 m1 上构建 lxml mac?

事实证明,使用简单的 pip install 安装 lxml 工作正常。

我的 malloc 错误的原因是我试图在看到结束标记之前清除元素。事实证明这是不可能的,即使您已经知道自己对该元素不感兴趣,也需要等待结束标记。