无法在 ec2 中运行 scrapy

Question

我正在尝试运行 ec2 服务器上的代码。这是一个 python scrapy 项目，在我自己的电脑上运行良好。当尝试在 ec2 中运行它时，我得到这个：

Traceback (most recent call last):
File "/usr/local/bin/scrapy", line 4, in <module>
execute()
File "/usr/local/lib/python2.7/site-packages/scrapy/cmdline.py", line 122, in execute
cmds = _get_commands_dict(settings, inproject)
File "/usr/local/lib/python2.7/site-packages/scrapy/cmdline.py", line 46, in _get_commands_dict
cmds = _get_commands_from_module('scrapy.commands', inproject)
File "/usr/local/lib/python2.7/site-packages/scrapy/cmdline.py", line 29, in _get_commands_from_module
for cmd in _iter_command_classes(module):
File "/usr/local/lib/python2.7/site-packages/scrapy/cmdline.py", line 20, in _iter_command_classes
for module in walk_modules(module_name):
File "/usr/local/lib/python2.7/site-packages/scrapy/utils/misc.py", line 68, in walk_modules
submod = import_module(fullpath)
File "/usr/local/lib/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
File "/usr/local/lib/python2.7/site-packages/scrapy/commands/bench.py", line 3, in <module>
from scrapy.tests.mockserver import MockServer
File "/usr/local/lib/python2.7/site-packages/scrapy/tests/mockserver.py", line 6, in <module>
from twisted.internet import reactor, defer, ssl
File "/usr/local/lib/python2.7/site-packages/twisted/internet/ssl.py", line 59, in <module>
from OpenSSL import SSL
File "/usr/local/lib/python2.7/site-packages/OpenSSL/__init__.py", line 8, in <module>
from OpenSSL import rand, crypto, SSL
File "/usr/local/lib/python2.7/site-packages/OpenSSL/rand.py", line 11, in <module>
from OpenSSL._util import (
File "/usr/local/lib/python2.7/site-packages/OpenSSL/_util.py", line 4, in <module>
binding = Binding()
File "/usr/local/lib/python2.7/site-packages/cryptography/hazmat/bindings/openssl/binding.py", line 87, in __init__
self._ensure_ffi_initialized()
File "/usr/local/lib/python2.7/site-packages/cryptography/hazmat/bindings/openssl/binding.py", line 106, in _ensure_ffi_initialized
libraries=libraries,
File "/usr/local/lib/python2.7/site-packages/cryptography/hazmat/bindings/utils.py", line 39, in build_ffi
ffi = cffi.FFI()
File "/usr/local/lib/python2.7/site-packages/cffi/api.py", line 56, in __init__
**import _cffi_backend as backend
ImportError: libffi.so.5: cannot open shared object file: No such file or directory**

我已尝试更新并重新安装所有提到的库。

它总是说它是最新版本。

这是我的 linux 发行版，默认 python 是 2.7：

#  cat /etc/*-release
NAME="Amazon Linux AMI"
VERSION="2015.03"
ID="amzn"
ID_LIKE="rhel fedora"
VERSION_ID="2015.03"
PRETTY_NAME="Amazon Linux AMI 2015.03"
ANSI_COLOR="0;33"
CPE_NAME="cpe:/o:amazon:linux:2015.03:ga"
HOME_URL="http://aws.amazon.com/amazon-linux-ami/"
Amazon Linux AMI release 2015.03

更新：看起来像是 scrapy 依赖项中的某种问题。尝试

 pip install scrapy

给我以下错误：

creating /usr/lib/python2.6/dist-packages/cssselect
error: could not create '/usr/lib/python2.6/dist-packages/cssselect':       Permission denied

----------------------------------------
Command "/usr/bin/python2.6 -c "import setuptools,   
tokenize;__file__='/tmp/pip-build- aoghBl/cssselect/setup.py'
;exec(compile(getattr(tokenize, 'open', open)
(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))"
install --record /tmp/pip-1Qpx2a-record/install-record.txt
--single-version-externally-managed --compile" 
failed with error code 1 in /tmp/pip-build-aoghBl/cssselect

Answer 1

我在安装 scrapy 时也遇到了问题。我发现以下页面很有用：

最后，我不得不启动一个 "fresh" 亚马逊 Linux 实例，它按照这些页面的说明成功安装。我的旧实例有太多问题需要解决。

Answer 2

您更新的问题是权限错误。你可能需要运行

sudo pip install cssselect

可能需要为其他依赖项执行此操作。奇怪的是它试图在 python2.6...

下安装

编辑：鉴于它试图运行在 2.6 下，您的实例有问题。我建议终止并重新创建实例。以下命令是我运行获得 scrapy dependancies 和我的项目以及运行ning

sudo su
yum groupinstall 'Development Tools'
yum install -y libffi-devel libxslt-devel libxml2-devel openssl-devel python-devel mysql-devel

无法在 ec2 中运行 scrapy

unable to run scrapy in ec2

python

amazon-ec2

scrapy

amazon-ami

无法在 ec2 中 运行 scrapy

unable to run scrapy in ec2

python

amazon-ec2

scrapy

amazon-ami

无法在 ec2 中运行 scrapy