运行 Scrapy 在 Raspberry Pi 3，python 3.4

Question

当运行在 Raspberry Pi 3 上使用 scrapy 时，我收到一个错误。

我已经成功安装了它，但是当我尝试使用之前创建的蜘蛛启动项目或爬网时，出现以下错误：

Traceback (most recent call last):
  File "/usr/local/bin/scrapy", line 7, in <module>
    from scrapy.cmdline import execute
  File "/usr/local/lib/python3.4/dist-packages/scrapy/cmdline.py", line 9, in <module>
    from scrapy.crawler import CrawlerProcess
  File "/usr/local/lib/python3.4/dist-packages/scrapy/crawler.py", line 7, in <module>
    from twisted.internet import reactor, defer
  File "/usr/local/lib/python3.4/dist-packages/twisted/internet/reactor.py", line 38, in <module>
    from twisted.internet import default
  File "/usr/local/lib/python3.4/dist-packages/twisted/internet/default.py", line 56, in <module>
    install = _getInstallFunction(platform)
  File "/usr/local/lib/python3.4/dist-packages/twisted/internet/default.py", line 44, in _getInstallFunction
    from twisted.internet.epollreactor import install
  File "/usr/local/lib/python3.4/dist-packages/twisted/internet/epollreactor.py", line 24, in <module>
    from twisted.internet import posixbase
  File "/usr/local/lib/python3.4/dist-packages/twisted/internet/posixbase.py", line 18, in <module>
    from twisted.internet import error, udp, tcp
  File "/usr/local/lib/python3.4/dist-packages/twisted/internet/tcp.py", line 28, in <module>
    from twisted.internet._newtls import (
  File "/usr/local/lib/python3.4/dist-packages/twisted/internet/_newtls.py", line 21, in <module>
    from twisted.protocols.tls import TLSMemoryBIOFactory, TLSMemoryBIOProtocol
  File "/usr/local/lib/python3.4/dist-packages/twisted/protocols/tls.py", line 65, in <module>
    from twisted.internet._sslverify import _setAcceptableProtocols
  File "/usr/local/lib/python3.4/dist-packages/twisted/internet/_sslverify.py", line 1865, in <module>
    "ECDH+AESGCM:ECDH+CHACHA20:DH+AESGCM:DH+CHACHA20:ECDH+AES256:DH+AES256:"
  File "/usr/local/lib/python3.4/dist-packages/twisted/internet/_sslverify.py", line 1845, in fromOpenSSLCipherString
    SSL.SSLv23_METHOD, SSL.OP_NO_SSLv2 | SSL.OP_NO_SSLv3)
  File "/usr/local/lib/python3.4/dist-packages/twisted/internet/_sslverify.py", line 1797, in _expandCipherString
    ctx.set_cipher_list(cipherString.encode('ascii'))
TypeError: must be str, not bytes

我不知道为什么会收到此消息或如何解决它，请帮忙？

解决方案：

感谢大家的帮助，我最终通过以下步骤让 Scrapy 在 Raspberry Pi 3 上工作：

首先安装virtualenv：

sudo pip install virtualenv

然后创建一个 virtualenv 并为 Scrapy 激活它：

virtualenv scrapyenv
source scrapyenv/bin/activate

然后我运行更新了里面的所有内容：

apt-get update
apt-get upgrade

安装所有依赖项：

apt-get install libffi-dev
apt-get install libxml2-dev
apt-get install libxslt1-dev
apt-get install libssl-dev
apt-get install python-dev

然后安装Scrapy

sudo pip install scrapy

然后我用这个更新了我的 pyOpenSSL：

pip -vvvv install --upgrade pyOpenSSL

这创建了很多日志文件并花费了一些时间，之后 scrapy 使用普通的 scrapy 命令运行良好，我也有运行一个蜘蛛 - 一切正常。

Answer 1

您使用的是什么用户？我认为你需要使用 sudo。还更新您的 ssl。看起来 ssl 连接有问题..

你运行 : pip install --verbose twisted

同时更新你的 openssl

pip -vvvv 安装 --upgrade pyOpenSSL，请在此处复制输出以检查是否已更新

此致

运行 Scrapy 在 Raspberry Pi 3，python 3.4

Running Scrapy on Raspberry Pi 3, python 3.4

python

scrapy

raspberry-pi

scrapy-spider

raspberry-pi3