UnicodeDecodeError 当我使用 cuda 训练数据集时

Question

我用chainer训练了一些图像但是出现了错误。

不知道是UnicodeDecodeError还是安装cupy的错误

P:\dcgans\chainer-DCGAN\chainer-DCGAN>python DCGAN.py

Traceback (most recent call last):
  File "DCGAN.py", line 279, in <module>
    train_dcgan_labeled(gen, dis)
  File "DCGAN.py", line 171, in train_dcgan_labeled
    zvis = (xp.random.uniform(-1, 1, (100, nz), dtype=np.float32))
  File "P:\Python35\lib\site-packages\cupy\random\distributions.py", line 132, in uniform
    return rs.uniform(low, high, size=size, dtype=dtype)
  File "P:\Python35\lib\site-packages\cupy\random\generator.py", line 235, in uniform
    rand = self.random_sample(size=size, dtype=dtype)
  File "P:\Python35\lib\site-packages\cupy\random\generator.py", line 153, in random_sample
    RandomState._1m_kernel(out)
  File "cupy/core/elementwise.pxi", line 552, in cupy.core.core.ElementwiseKernel.__call__ (cupy\core\core.cpp:43810)
  File "cupy/util.pyx", line 39, in cupy.util.memoize.decorator.ret (cupy\util.cpp:1480)
  File "cupy/core/elementwise.pxi", line 409, in cupy.core.core._get_elementwise_kernel (cupy\core\core.cpp:42156)
  File "cupy/core/elementwise.pxi", line 12, in cupy.core.core._get_simple_elementwise_kernel (cupy\core\core.cpp:34787)
  File "cupy/core/elementwise.pxi", line 32, in cupy.core.core._get_simple_elementwise_kernel (cupy\core\core.cpp:34609)
  File "cupy/core/carray.pxi", line 87, in cupy.core.core.compile_with_cache (cupy\core\core.cpp:34264)
  File "P:\Python35\lib\site-packages\cupy\cuda\compiler.py", line 133, in compile_with_cache
    base = _empty_file_preprocess_cache[env] = preprocess('', options)
  File "P:\Python35\lib\site-packages\cupy\cuda\compiler.py", line 99, in preprocess
    pp_src = pp_src.decode('utf-8')
UnicodeDecodeError: 'utf-8' codec can't decode bytes in position 27-28: invalid continuation byte

Answer 1

似乎 nvcc 生成了非 UTF8 输出，而 CuPy 无法对其进行解码。这是 CuPy 的一个错误（我发布了一个问题：#378）。

目前可能的解决方案是将 cupy/cuda/compiler.py 中 pp_src = pp_src.decode('utf-8') 行的 'utf-8' 替换为与您的环境相匹配的内容。例如，在日语环境中，'cp932' 应该可以，而 'cp936' 应该可以用于简体中文。

您也可以尝试 locale.getdefaultlocale()[1] 作为通用解决方案（一定要 import locale）。

更新：修复已合并。它应该在即将发布的 CuPy v1.0.3 中修复。

UnicodeDecodeError 当我使用 cuda 训练数据集时

UnicodeDecodeError When I use cuda to train dataset

python

chainer

cupy