Python & Scrapy：Scrapy 版本问题

Question

我正在使用 Ubuntu 14.04 x64，我已经按照 Scrapy 文档使用点子:

pip install scrapy

然后我按照示例项目的初始化，尝试执行example spider:

scrapy crawl example

我得到这个错误：

2015-02-23 10:23:42+0100 [scrapy] INFO: Scrapy 0.14.4 started (bot: example)
2015-02-23 10:23:42+0100 [scrapy] DEBUG: Enabled extensions: LogStats, TelnetConsole, CloseSpider, WebService, CoreStats, MemoryUsage, SpiderState
Traceback (most recent call last):
  File "/usr/local/bin/scrapy", line 11, in <module>
    sys.exit(execute())
  File "/usr/lib/python2.7/dist-packages/scrapy/cmdline.py", line 132, in execute
    _run_print_help(parser, _run_command, cmd, args, opts)
  File "/usr/lib/python2.7/dist-packages/scrapy/cmdline.py", line 97, in _run_print_help
    func(*a, **kw)
  File "/usr/lib/python2.7/dist-packages/scrapy/cmdline.py", line 139, in _run_command
    cmd.run(args, opts)
  File "/usr/lib/python2.7/dist-packages/scrapy/commands/crawl.py", line 43, in run
    spider = self.crawler.spiders.create(spname, **opts.spargs)
  File "/usr/lib/python2.7/dist-packages/scrapy/command.py", line 34, in crawler
    self._crawler.configure()
  File "/usr/lib/python2.7/dist-packages/scrapy/crawler.py", line 36, in configure
    self.spiders = spman_cls.from_crawler(self)
  File "/usr/lib/python2.7/dist-packages/scrapy/spidermanager.py", line 37, in from_crawler
    return cls.from_settings(crawler.settings)
  File "/usr/lib/python2.7/dist-packages/scrapy/spidermanager.py", line 33, in from_settings
    return cls(settings.getlist('SPIDER_MODULES'))
  File "/usr/lib/python2.7/dist-packages/scrapy/spidermanager.py", line 23, in __init__
    for module in walk_modules(name):
  File "/usr/lib/python2.7/dist-packages/scrapy/utils/misc.py", line 65, in walk_modules
    submod = __import__(fullpath, {}, {}, [''])
  File "/home/alvaro/vagrantenvs/example/example/spiders/example_spider.py", line 3, in <module>
    class ExampleSpider(scrapy.Spider):
AttributeError: 'module' object has no attribute 'Spider'

我用 pip freeze | grep 'Scrapy' 检查了 Scrapy 的安装版本，我得到：

Scrapy==0.24.4

但正如您在回溯错误中看到的那样：

2015-02-23 10:23:42+0100 [scrapy] INFO: Scrapy 0.14.4 started (bot: example)

我尝试卸载并重新安装
我也试过更新 pip，卸载 scrapy 并重新安装
我尝试使用 apt-get

为什么如果我安装了最新版本 (0.24.4) 它似乎正在执行 0.14.4 ?
此问题是否与 Scrapy、Pip 或两者有关？

编辑：

这是文件 example_spider.py:

import scrapy

class ExampleSpider(scrapy.Spider):
    name = "example"
    allowed_domains = ["dmoz.org"]
    start_urls = [
        "http://www.dmoz.org/Computers/Programming/Languages/Python/Books/",
        "http://www.dmoz.org/Computers/Programming/Languages/Python/Resources/"
    ]

    def parse(self, response):
        filename = response.url.split("/")[-2]
        with open(filename, 'wb') as f:
            f.write(response.body)

如果我按照@aberna 的建议执行scrapy version，我得到：

Scrapy 0.14.4

Answer 1

我当时遇到了类似的问题，这是因为 Ubuntu 包与 pip 包混淆了。

当我使用 pip 安装 Scrapy 并尝试运行它时，scrapy 说这个包缺少一些依赖项然后我使用 apt-get 安装了 Scrapy 并且我搞砸了包版本。

我建议您从 Ubuntu 和 Pip 中删除两个软件包，然后仅使用 pip 重新安装：

apt-get remove --purge python-scrapy
pip uninstall scrapy

完成这些操作后，您可以 使用 pip 再次安装它，然后如果您这样做 scrapy version 您应该得到最新版本的响应，在您的情况下它应该是 0.24.4.

编辑

您可以在 Scrapy 文档中阅读：

python-scrapy is a different package provided by official debian repositories, it’s very outdated and it isn’t supported by Scrapy team.

要在 Ubuntu 中安装 Scrapy，他们说：

Import the GPG key used to sign Scrapy packages into APT keyring:

sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 627220E7

Create /etc/apt/sources.list.d/scrapy.list file using the following command:

echo 'deb http://archive.scrapy.org/ubuntu scrapy main' | sudo tee /etc/apt/sources.list.d/scrapy.list

3.Update package lists and install the scrapy-0.24 package:

sudo apt-get update && sudo apt-get install scrapy-0.24

Check Here: Scrapy on Ubuntu

Python & Scrapy：Scrapy 版本问题

Python & Scrapy: Issue with Scrapy version

python

pip

scrapy

编辑：

编辑