When using Bag.to_textfiles with dask, I get the error "AttributeError: 'dict' object has no attribute 'endswith'"

Question

标题说了大部分 但有问题的 object 是：

>>> import dask.bag as db
>>> b = db.from_sequence([{'name': 'Alice',   'balance': 100},
...                       {'name': 'Bob',     'balance': 200},
...                       {'name': 'Charlie', 'balance': 300}],
...                      npartitions=2)

但是当我尝试

>>> b.to_textfiles('*.json')

我明白了

AttributeError: 'dict' object has no attribute 'endswith'

Traceback
---------
  File "/Users/jlatmann/anaconda/envs/python3/lib/python3.5/site-packages/dask/async.py", line 267, in execute_task
    result = _execute_task(task, data)
  File "/Users/jlatmann/anaconda/envs/python3/lib/python3.5/site-packages/dask/async.py", line 249, in _execute_task
    return func(*args2)
  File "/Users/jlatmann/anaconda/envs/python3/lib/python3.5/site-packages/dask/bag/core.py", line 1025, in write
    if not (firstline.endswith(os.linesep) or firstline.endswith('\n')):

dask 版本：0.9.0

sys.version

3.5.1 |Anaconda 4.0.0 (x86_64)| (default, Dec 7 2015, 11:24:55) [GCC 4.2.1 (Apple Inc. build 5577)]

感谢观看！

Answer 1

to_textfiles函数假设包的元素是字符串。我建议先在你的包上映射 str

b.map(str).to_textfiles('*.json')

或者更好的是，如果您的输出文件是 json，请将您的数据显式转储为 json 格式

b.map(json.dumps).to_textfiles('*.json')