scrapyd 异常期望属性名称用双引号引起来

Question

当我执行

curl http://localhost:6800/schedule.json -d project=default -d spider=spider1

我转到 scrapyd 控制台，我看到异常：

[-] Unhandled Error
Traceback (most recent call last):
  File "/home/scrapyV1/venv/lib/python2.7/site-packages/twisted/internet/defer.py", line 150, in maybeDeferred
    result = f(*args, **kw)
  File "/home/scrapyV1/venv/lib/python2.7/site-packages/twisted/internet/defer.py", line 1274, in unwindGenerator
    return _inlineCallbacks(None, gen, Deferred())
  File "/home/scrapyV1/venv/lib/python2.7/site-packages/twisted/internet/defer.py", line 1128, in _inlineCallbacks
    result = g.send(result)
  File "/home/scrapyV1/venv/lib/python2.7/site-packages/scrapyd/poller.py", line 23, in poll
    msg = yield maybeDeferred(q.pop)
--- <exception caught here> ---
  File "/home/scrapyV1/venv/lib/python2.7/site-packages/twisted/internet/defer.py", line 150, in maybeDeferred
    result = f(*args, **kw)
  File "/home/scrapyV1/venv/lib/python2.7/site-packages/scrapyd/spiderqueue.py", line 21, in pop
    return self.q.pop()
  File "/home/scrapyV1/venv/lib/python2.7/site-packages/scrapyd/sqlite.py", line 119, in pop
    return self.decode(msg)
  File "/home/scrapyV1/venv/lib/python2.7/site-packages/scrapyd/sqlite.py", line 170, in decode
    return json.loads(text)
  File "/usr/local/lib/python2.7/json/__init__.py", line 338, in loads
    return _default_decoder.decode(s)
  File "/usr/local/lib/python2.7/json/decoder.py", line 366, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/local/lib/python2.7/json/decoder.py", line 382, in raw_decode
    obj, end = self.scan_once(s, idx)
exceptions.ValueError: Expecting property name enclosed in double quotes: line 1 column 2 (char 1)nter code here

在那之后我不能运行下一个 Spider，我得到

{"status": "error", "message": "No JSON object could be decoded", "node_name": "spider-test2"}

如何确定该错误与什么有关？

Answer 1

问题出在我的Centos 6.5 中安装的sqlite3。决定：需要添加

self.conn.text_factory = bytes

到 SqliteDict 和 Sqlite PriorityQueue 类在 scrapyd/sqlite.py b/scrapyd/sqlite.py

中创建连接后

scrapyd 异常期望 属性 名称用双引号引起来

scrapyd Exception Expecting property name enclosed in double quotes

python

scrapy

scrapyd

scrapyd 异常期望属性名称用双引号引起来