QUEX_PATH 使用分词器时出现问题

Question

我正在尝试安装 trainable-tokenizer。我已经按照自述文件安装了所有依赖项。这是可训练分词器 https://github.com/jirkamarsik/trainable-tokenizer。我已经使用 quex.org 中的安装程序安装了 quex.deb，这是可训练分词器的依赖项。

当我尝试执行 sudo make install 时，出现以下错误

ThinkPad-T430:~/tmp/OpenFST/tokenizer/trainable-tokenizer/src$ sudo make install 
[  6%] Building FeaturesReader with quex
Environment variable QUEX_PATH has not been defined.
error: environment variable 'QUEX_PATH' is not defined.
error: your system is 'posix'.
error: if you are using bash-shell, append the following line
error: to your '~/.bashrc' file:
error:
error: export QUEX_PATH=directory-where-quex-has-been-installed
make[2]: *** [quex_files/trtok_read_features_FeaturesReader] Error 255
make[1]: *** [CMakeFiles/trtok.dir/all] Error 2
make: *** [all] Error 2

但我已经设置了 QUEX_PATH。

ThinkPad-T430:~/tmp/OpenFST/tokenizer/trainable-tokenizer/src$ echo $QUEX_PATH 
/opt/quex/quex-0.65.4/
ThinkPad-T430:~/tmp/OpenFST/tokenizer/trainable-tokenizer/src$ ll /opt/quex/quex-0.65.4/
total 68
drwxrwxr-x 5 root root  4096 May 25 17:42 ./
drwxrwxr-x 3 root root  4096 May 25 17:42 ../
-rw-r--r-- 1 root root   181 Oct 13  2013 COPYRIGHT.txt
drwxr-xr-x 5 root root  4096 May 25 17:42 demo/
-rw-r--r-- 1 root root     0 Feb 26 02:01 __init__.py
-rwxr-xr-x 1 root root 26424 Oct 13  2013 LGPL.txt*
drwxr-xr-x 2 root root  4096 May 25 17:42 manpage/
drwxr-xr-x 6 root root  4096 May 25 18:02 quex/
-rw-r--r-- 1 root root  3098 Oct 13  2013 quex.bat
-rwxrwxr-x 1 root root  2649 Feb 26 02:01 quex-exe.py*
-rw-r--r-- 1 root root  4074 Oct 13  2013 README
-rw-rw-r-- 1 root root   118 Mar 22 15:22 unit_test_results.txt

我还添加了 export QUEX_PATH=/opt/quex/quex-0.65.4/ 到 .bashrc

ThinkPad-T430:~/tmp/OpenFST/tokenizer/trainable-tokenizer/src$ vim ~/.bashrc
ThinkPad-T430:~/tmp/OpenFST/tokenizer/trainable-tokenizer/src$ echo $QUEX_PATH 
/opt/quex/quex-0.65.4/
ThinkPad-T430:~/tmp/OpenFST/tokenizer/trainable-tokenizer/src$ sudo make install
[  6%] Building FeaturesReader with quex
Environment variable QUEX_PATH has not been defined.
error: environment variable 'QUEX_PATH' is not defined.
error: your system is 'posix'.

但我仍然得到同样的错误。

Answer 1

我猜真正的问题是你的 python 是 python 3.0+ 而不是 Quex 所需的 python 2.7。

诚然，错误消息并没有直接说明这一点。

Answer 2

我使用 -I 标记解决了这个问题。我不得不使用 quex -I.

QUEX_PATH 使用分词器时出现问题

QUEX_PATH issue while using tokenizer

c++

makefile

tokenize

make-install

quex