colab中的tf-sentencepiece导入错误(导入路径很奇怪)
tf-sentencepiece import error in colab (the import path is weird)
我在 colab 上导入 tf-sentencepiece 时出错。
我使用的是 tensorflow 2.0 版,我安装了 tf-sentencepiece。
!pip install --upgrade pip
!pip install tensorflow==2.0.0-rc1
!pip3 install --quiet tensorflow-hub
!pip3 install --quiet seaborn
!pip3 install sentencepiece
!pip3 install tf_sentencepiece
但是当我尝试导入tf-sentencepice时,出现以下错误
/usr/local/lib/python3.6/dist-packages/tf_sentencepiece/sentencepiece_processor_ops.py:43: UserWarning: No so file is found for [2.0.0-rc1] from [3.6/dist-packages/tf_sentencepiece/_sentencepiece_processor_ops.so.1.11.0, 3.6/dist-packages/tf_sentencepiece/_sentencepiece_processor_ops.so.2.0.0-beta1, 3.6/dist-packages/tf_sentencepiece/_sentencepiece_processor_ops.so.1.7.0, 3.6/dist-packages/tf_sentencepiece/_sentencepiece_processor_ops.so.1.14.0, 3.6/dist-packages/tf_sentencepiece/_sentencepiece_processor_ops.so.1.9.0, 3.6/dist-packages/tf_sentencepiece/_sentencepiece_processor_ops.so.1.8.0, 3.6/dist-packages/tf_sentencepiece/_sentencepiece_processor_ops.so.1.10.0, 3.6/dist-packages/tf_sentencepiece/_sentencepiece_processor_ops.so.1.13.1]
(tf.__version__, ', '.join(versions)))
/usr/local/lib/python3.6/dist-packages/tf_sentencepiece/sentencepiece_processor_ops.py:44: UserWarning: use the latest version 3.6/dist-packages/tf_sentencepiece/_sentencepiece_processor_ops.so.2.0.0-beta1
warnings.warn('use the latest version %s' % (latest))
---------------------------------------------------------------------------
NotFoundError Traceback (most recent call last)
<ipython-input-17-cadbc83432ee> in <module>()
9 from tensorflow.python.framework.ops import disable_eager_execution
10 import sentencepiece
---> 11 import tf_sentencepiece
12
13 #disable_eager_execution()
2 frames
/usr/local/lib/python3.6/dist-packages/tf_sentencepiece/__init__.py in <module>()
3 from __future__ import print_function
4
----> 5 from tf_sentencepiece.sentencepiece_processor_ops import *
/usr/local/lib/python3.6/dist-packages/tf_sentencepiece/sentencepiece_processor_ops.py in <module>()
45 so_file = so_base + '.' + latest
46
---> 47 _gen_sentencepiece_processor_op = tf.load_op_library(so_file)
48
49
/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/framework/load_library.py in load_op_library(library_filename)
59 RuntimeError: when unable to load the library or get the python wrappers.
60 """
---> 61 lib_handle = py_tf.TF_LoadLibrary(library_filename)
62
63 op_list_str = py_tf.TF_GetOpList(lib_handle)
NotFoundError: /usr/local/lib/python3.6/dist-packages/tf_sentencepiece/_sentencepiece_processor_ops.so.3.6/dist-packages/tf_sentencepiece/_sentencepiece_processor_ops.so.2.0.0-beta1: cannot open shared object file: No such file or directory
于是手动进入目录查看sentencepiece_processor_ops.so.2.0.0-beta是否存在。奇怪的是它在文件夹中,但路径不同。 colab 尝试从 /usr/local/lib/python3.6/dist-packages/tf_sentencepiece/_sentencepiece_processor_ops.so.3.6/dist-packages/tf_sentencepiece/_sentencepiece_processor_ops.so.2.0.0-beta1 导入文件(路径很奇怪,应该早点注意到...)但文件实际上存在于 /usr/local/lib/python3.6/dist-packages/tf_sentencepiece/_sentencepiece_processor_ops.so.2.0.0-beta1
怎么在colab中设置的路径很奇怪?我应该如何解决这个问题?
正在安装 tensorflow_text
!pip3 install tensorflow_text>=2.0.0rc0
解决了问题——包中包含 SentencepieceOps。
仍然不知道为什么路径恰好很奇怪,但是...
我在 colab 上导入 tf-sentencepiece 时出错。
我使用的是 tensorflow 2.0 版,我安装了 tf-sentencepiece。
!pip install --upgrade pip
!pip install tensorflow==2.0.0-rc1
!pip3 install --quiet tensorflow-hub
!pip3 install --quiet seaborn
!pip3 install sentencepiece
!pip3 install tf_sentencepiece
但是当我尝试导入tf-sentencepice时,出现以下错误
/usr/local/lib/python3.6/dist-packages/tf_sentencepiece/sentencepiece_processor_ops.py:43: UserWarning: No so file is found for [2.0.0-rc1] from [3.6/dist-packages/tf_sentencepiece/_sentencepiece_processor_ops.so.1.11.0, 3.6/dist-packages/tf_sentencepiece/_sentencepiece_processor_ops.so.2.0.0-beta1, 3.6/dist-packages/tf_sentencepiece/_sentencepiece_processor_ops.so.1.7.0, 3.6/dist-packages/tf_sentencepiece/_sentencepiece_processor_ops.so.1.14.0, 3.6/dist-packages/tf_sentencepiece/_sentencepiece_processor_ops.so.1.9.0, 3.6/dist-packages/tf_sentencepiece/_sentencepiece_processor_ops.so.1.8.0, 3.6/dist-packages/tf_sentencepiece/_sentencepiece_processor_ops.so.1.10.0, 3.6/dist-packages/tf_sentencepiece/_sentencepiece_processor_ops.so.1.13.1]
(tf.__version__, ', '.join(versions)))
/usr/local/lib/python3.6/dist-packages/tf_sentencepiece/sentencepiece_processor_ops.py:44: UserWarning: use the latest version 3.6/dist-packages/tf_sentencepiece/_sentencepiece_processor_ops.so.2.0.0-beta1
warnings.warn('use the latest version %s' % (latest))
---------------------------------------------------------------------------
NotFoundError Traceback (most recent call last)
<ipython-input-17-cadbc83432ee> in <module>()
9 from tensorflow.python.framework.ops import disable_eager_execution
10 import sentencepiece
---> 11 import tf_sentencepiece
12
13 #disable_eager_execution()
2 frames
/usr/local/lib/python3.6/dist-packages/tf_sentencepiece/__init__.py in <module>()
3 from __future__ import print_function
4
----> 5 from tf_sentencepiece.sentencepiece_processor_ops import *
/usr/local/lib/python3.6/dist-packages/tf_sentencepiece/sentencepiece_processor_ops.py in <module>()
45 so_file = so_base + '.' + latest
46
---> 47 _gen_sentencepiece_processor_op = tf.load_op_library(so_file)
48
49
/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/framework/load_library.py in load_op_library(library_filename)
59 RuntimeError: when unable to load the library or get the python wrappers.
60 """
---> 61 lib_handle = py_tf.TF_LoadLibrary(library_filename)
62
63 op_list_str = py_tf.TF_GetOpList(lib_handle)
NotFoundError: /usr/local/lib/python3.6/dist-packages/tf_sentencepiece/_sentencepiece_processor_ops.so.3.6/dist-packages/tf_sentencepiece/_sentencepiece_processor_ops.so.2.0.0-beta1: cannot open shared object file: No such file or directory
于是手动进入目录查看sentencepiece_processor_ops.so.2.0.0-beta是否存在。奇怪的是它在文件夹中,但路径不同。 colab 尝试从 /usr/local/lib/python3.6/dist-packages/tf_sentencepiece/_sentencepiece_processor_ops.so.3.6/dist-packages/tf_sentencepiece/_sentencepiece_processor_ops.so.2.0.0-beta1 导入文件(路径很奇怪,应该早点注意到...)但文件实际上存在于 /usr/local/lib/python3.6/dist-packages/tf_sentencepiece/_sentencepiece_processor_ops.so.2.0.0-beta1
怎么在colab中设置的路径很奇怪?我应该如何解决这个问题?
正在安装 tensorflow_text
!pip3 install tensorflow_text>=2.0.0rc0
解决了问题——包中包含 SentencepieceOps。
仍然不知道为什么路径恰好很奇怪,但是...