通过 repl 导入很好，但从脚本中中断

Question

我有以下 test1.py 脚本：

import tfrecorder

如果我运行它，它失败了：

I have no name!@519b05bc3bf9:/test$ python test1.py 
2021-10-26 00:22:56.863971: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'libcudart.so.10.1'; dlerror: libcudart.so.10.1: cannot open shared object file: No such file or directory
2021-10-26 00:22:56.863989: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
Traceback (most recent call last):
  File "test1.py", line 1, in <module>
    import tfrecorder
  File "/usr/local/lib/python3.7/site-packages/tfrecorder/__init__.py", line 19, in <module>
    from tfrecorder import accessor
  File "/usr/local/lib/python3.7/site-packages/tfrecorder/accessor.py", line 28, in <module>
    from tfrecorder import converter
  File "/usr/local/lib/python3.7/site-packages/tfrecorder/converter.py", line 27, in <module>
    import apache_beam as beam
  File "/usr/local/lib/python3.7/site-packages/apache_beam/__init__.py", line 104, in <module>
    from apache_beam import coders
  File "/usr/local/lib/python3.7/site-packages/apache_beam/coders/__init__.py", line 19, in <module>
    from apache_beam.coders.coders import *
  File "/usr/local/lib/python3.7/site-packages/apache_beam/coders/coders.py", line 43, in <module>
    from future.moves import pickle
  File "/usr/local/lib/python3.7/site-packages/future/moves/__init__.py", line 8, in <module>
    import_top_level_modules()
  File "/usr/local/lib/python3.7/site-packages/future/standard_library/__init__.py", line 810, in import_top_level_modules
    with exclude_local_folder_imports(*TOP_LEVEL_MODULES):
  File "/usr/local/lib/python3.7/site-packages/future/standard_library/__init__.py", line 781, in __enter__
    module = __import__(m, level=0)
  File "/test/test.py", line 1, in <module>
    from tfrecorder import input_schema
  File "/usr/local/lib/python3.7/site-packages/tfrecorder/input_schema.py", line 26, in <module>
    from tfrecorder import types
  File "/usr/local/lib/python3.7/site-packages/tfrecorder/types.py", line 24, in <module>
    from tensorflow_transform import beam as tft_beam
  File "/usr/local/lib/python3.7/site-packages/tensorflow_transform/beam/__init__.py", line 20, in <module>
    from tensorflow_transform.beam import analyzer_cache
  File "/usr/local/lib/python3.7/site-packages/tensorflow_transform/beam/analyzer_cache.py", line 163, in <module>
    class _WriteToTFRecordGzip(beam.io.WriteToTFRecord):
AttributeError: module 'apache_beam' has no attribute 'io'

但是，在 REPL 中没问题：

I have no name!@ff017054aef4:/$ python
Python 3.7.12 (default, Oct 12 2021, 03:26:18) 
[GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import tfrecorder
2021-10-26 00:24:33.293181: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'libcudart.so.10.1'; dlerror: libcudart.so.10.1: cannot open shared object file: No such file or directory
2021-10-26 00:24:33.293201: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
>>>

这对我来说没有任何意义。有什么想法吗？

Answer 1

看来这是一个bug，在apache_beam的最新版本中已经修复了。该线程中的解释是描述性的。 Circular Import Issue in apache_beam

问题是 TOP_LEVEL_MODULES 变量定义了一个模块列表，其中包括 test。您的 test 模块名称与 apache_beam 中的内置模块冲突。您可以尝试重命名您的 test 模块，然后检查。

您在 shell 中看不到相同的问题，因为 sys.path 在 shell 环境中没有目录 /test。

通过 repl 导入很好，但从脚本中中断

import through repl is fine but breaks from script

python

apache-beam