Scrapy 图像管道上的 IOError

Question

我正在使用 Scrapy 的图像管道，对于某些图像我收到了这个错误：

[scrapy.pipelines.files] ERROR: File (unknown-error): Error processing file from <GET https://www.example.com/folder-name/image.jpg> referred in <None>
Traceback (most recent call last):
  File "c:\users\user\anaconda2\lib\site-packages\scrapy\pipelines\files.py", line 401, in media_downloaded
    checksum = self.file_downloaded(response, request, info)
  File "c:\users\user\anaconda2\lib\site-packages\scrapy\pipelines\images.py", line 101, in file_downloaded
    return self.image_downloaded(response, request, info)
  File "c:\users\user\anaconda2\lib\site-packages\scrapy\pipelines\images.py", line 105, in image_downloaded
    for path, image, buf in self.get_images(response, request, info):
  File "c:\users\user\anaconda2\lib\site-packages\scrapy\pipelines\images.py", line 125, in get_images
    image, buf = self.convert_image(orig_image)
  File "c:\users\user\anaconda2\lib\site-packages\scrapy\pipelines\images.py", line 151, in convert_image
    image.save(buf, 'JPEG')
  File "c:\users\user\anaconda2\lib\site-packages\PIL\Image.py", line 1916, in save
    self.load()
  File "c:\users\user\anaconda2\lib\site-packages\PIL\ImageFile.py", line 254, in load
    raise_ioerror(err_code)
  File "c:\users\user\anaconda2\lib\site-packages\PIL\ImageFile.py", line 59, in raise_ioerror
    raise IOError(message + " when reading image file")
IOError: broken data stream when reading image file

这些图片在服务器上可用（没有重定向），我没有发现有效的图像和无效的图像之间有任何区别。知道我错过了什么吗？

Answer 1

这似乎是一个已知的 issue。升级 Pillow 依赖项 (pip install Pillow --upgrade) 修复了它。

Scrapy 图像管道上的 IOError

IOError on Scrapy Images Pipeline

python

scrapy

scrapy-pipeline