Scrapy Parse 函数没有将找到的值传递给 parse_page2 函数

Question

我正在尝试抓取 playstation 网上商店以从主页抓取标题、游戏link，并从第二页抓取每个游戏的价格。但是当使用parse_page2的回调函数时，所有返回的项目都包含最近项目的标题和项目['link']值。（我们最后一个重新制作）

我的代码如下：

class PsStoreSpider(scrapy.Spider):
    name = 'psstore'
    start_urls =['https://store.playstation.com/en-ie/pages/browse']

    def parse(self, response):
        item = PlaystationItem()
        products = response.css('a.psw-link')
 
        for product in products:

            item['main_url'] = response.url
            item['title'] = product.css('span.psw-t-body.psw-c-t-1.psw-t-truncate-2.psw-m-b-2::text').get()
            item['link'] = 'https://store.playstation.com' + product.css('a.psw-link.psw-content-link').attrib['href']
            link = 'https://store.playstation.com' + product.css('a.psw-link.psw-content-link').attrib['href']

            request = Request(link, callback=self.parse_page2)
            request.meta['item'] = item
            yield request

    def parse_page2(self, response):
        item = response.meta['item']
        item['price'] = response.css('span.psw-t-title-m::text').get()
        item['other_url'] = response.url
        yield item

和部分输出：

2022-05-09 19:54:16 [scrapy.core.scraper] DEBUG: Scraped from <200 https://store.playstation.com/en-ie/concept/229261> 
{'link': 'https://store.playstation.com/en-ie/concept/228638',
 'main_url': 'https://store.playstation.com/en-ie/pages/browse',
 'other_url': 'https://store.playstation.com/en-ie/concept/229261',
 'price': 'Free',
 'title': 'The Last of Us™ Remastered'}
2022-05-09 19:54:16 [scrapy.core.scraper] DEBUG: Scraped from <200 https://store.playstation.com/en-ie/concept/232847> 
{'link': 'https://store.playstation.com/en-ie/concept/228638',
 'main_url': 'https://store.playstation.com/en-ie/pages/browse',
 'other_url': 'https://store.playstation.com/en-ie/concept/232847',
 'price': '€59.99',
 'title': 'The Last of Us™ Remastered'}
2022-05-09 19:54:16 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://store.playstation.com/en-ie/concept/224802> (referer: https://store.playstation.com/en-ie/pages/browse)
2022-05-09 19:54:16 [scrapy.core.scraper] DEBUG: Scraped from <200 https://store.playstation.com/en-ie/concept/224802> 
{'link': 'https://store.playstation.com/en-ie/concept/228638',
 'main_url': 'https://store.playstation.com/en-ie/pages/browse',
 'other_url': 'https://store.playstation.com/en-ie/concept/224802',
 'price': '€29.99',
 'title': 'The Last of Us™ Remastered'}

如您所见，价格已正确返回，但标题和 link 取自上次抓取的 object。我在这里错过了什么？

谢谢

Answer 1

问题是您在解析方法的开头创建了 item，然后一遍又一遍地更新它。这也意味着您始终将相同的项目传递给 parse_page2.
如果您要在 for 循环中创建您的项目，您将在每次迭代中获得一个新项目，并且应该得到预期的结果。
像这样：

    def parse(self, response):
        products = response.css('a.psw-link')
 
        for product in products:
            item = PlaystationItem()
            item['main_url'] = response.url
            item['title'] = product.css('span.psw-t-body.psw-c-t-1.psw-t-truncate-2.psw-m-b-2::text').get()
            item['link'] = 'https://store.playstation.com' + product.css('a.psw-link.psw-content-link').attrib['href']
            link = 'https://store.playstation.com' + product.css('a.psw-link.psw-content-link').attrib['href']

            request = Request(link, callback=self.parse_page2)
            request.meta['item'] = item
            yield request

Scrapy Parse 函数没有将找到的值传递给 parse_page2 函数

Scrapy Parse function not passing found values to the parse_page2 function

python

scrapy