如何通过 Python 在新选项卡中打开网站中的每个产品以使用 Selenium 进行抓取

How to open each product within a website in a new tab for scraping using Selenium through Python

我正在使用硒“https://www.medline.com/catalog/category-products.jsp?itemId=Z05-CA02_03&N=111079+4294770643&iclp=Z05-CA02_03"

对于单个页面和单个产品,我可以通过传递产品 url 来抓取,但我正在尝试通过 selenium 来实现,即自动 select 产品页面的离子 在 select 所有产品一个接一个之后,它应该移动到下一页,在打开产品详细信息页面后,它应该刮掉由 beautiful soup 完成的这里是来自基础 url 的产品 url "https://www.medline.com/product/SensiCare-Powder-Free-Nitrile-Exam-Gloves/SensiCare/Z05-PF00342?question=&index=P1&indexCount=1"

这是我的代码:

chromeOptions = webdriver.ChromeOptions()
chromeOptions.add_experimental_option('useAutomationExtension', False)
driver = webdriver.Chrome(executable_path='C:/Users/ptiwar34/Documents/chromedriver.exe', chrome_options=chromeOptions, desired_capabilities=chromeOptions.to_capabilities())
driver.get("https://www.medline.com/catalog/category-products.jsp?itemId=Z05-CA02_03&N=111079+4294770643&iclp=Z05-CA02_03")

while True:
    try:  
        WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//div[contains(@class, 'resultGalleryViewRow')]//div[@class='medGridProdTitle']//a[contains(@href]"))).click()
        print("Clicked for next page")
    except TimeoutException:
        print("No more pages")
        break
driver.quit()

这里不报错

它不会为每个产品打开页面,我想在新标签页中打开每个产品,然后在抓取它后删除并打开新标签页的新产品

BeautifulSoup 的文档说 (https://www.crummy.com/software/BeautifulSoup/bs4/doc/#find)

If find() can’t find anything, it returns None

这意味着用 class medSkuProductListExpanded productFamilyOrderInfo 查找 div 不成功,您可能应该打印出关于此的内容并仔细检查您正在搜索的条件。

错误发生是因为后面的选择器div = soup.find('div', {'class'没有匹配到任何东西,因此返回了NoneType。搜索多个 类 时,可以使用以下语法:

div = soup.find('div', {'class': 'medSkuProductListExpanded productFamilyOrderInfo'})
        right_table = div.find('table', {'class': ['actualDataTable', 'table-striped']})
        div1 = soup.find('div', {'class': ['medTabAreaContent', 'Basic']})
        right_table1 = div1.find('table', {'class': 'table-striped'})

如果元素应该匹配所有 类 css 可以使用选择器:https://www.crummy.com/software/BeautifulSoup/bs4/doc/#css-selectors

您不需要 selenium 来抓取产品,相反,简单 requests 就足够了。此解决方案可以简单地遍历分页和 return 每个页面的所有产品列表:

from bs4 import BeautifulSoup as soup
d = soup(requests.get('https://www.medline.com/catalog/category-products.jsp?itemId=Z05-CA02_03&N=111079+4294770643&iclp=Z05-CA02_03').text, 'html.parser')
def get_p(_d):
  return [{'img':i.img['src'], 'link':i.a['href'], 'title':i.find('div', {'class':'medGridProdTitle'}).text, } for i in _d.find_all('div', {'class':'product'})]

r = [get_p(d)]+[get_p(soup(requests.get(f'https://www.medline.com{i["href"]}').text, 'html.parser')) for i in d.find('div', {'class':'pagination'}).find_all('a')]

输出:

[[{'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_08/PF00342/PF00342_PRI05.JPG?quality=high&width=180&height=180', 'link': '/product/SensiCare-Powder-Free-Nitrile-Exam-Gloves/SensiCare/Z05-PF00342;ecomsessionid=8YLYmH_8oOnAQOELjXzF4KY84kOVBXYqYWmHXXtR.OC2-b2b?question=&index=P1&indexCount=1', 'title': '\nSensiCare Powder-Free Nitrile Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_02/CA02_03_02_05/PF00350/PF00350_PRI04.JPG?quality=high&width=180&height=180', 'link': '/product/MediGuard-Vinyl-Synthetic-Exam-Gloves/MediGuard/Z05-PF00350;ecomsessionid=8YLYmH_8oOnAQOELjXzF4KY84kOVBXYqYWmHXXtR.OC2-b2b?question=&index=P2&indexCount=2', 'title': '\nMediGuard Vinyl Synthetic Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_02/CA02_03_02_03/PF00348/PF00348_PRI05.JPG?quality=high&width=180&height=180', 'link': '/product/CURAD-Stretch-Vinyl-Exam-Gloves/Curad/Z05-PF00348;ecomsessionid=8YLYmH_8oOnAQOELjXzF4KY84kOVBXYqYWmHXXtR.OC2-b2b?question=&index=P3&indexCount=3', 'title': '\nCURAD Stretch Vinyl Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_03/PF00331/PF00331_PRI03.JPG?quality=high&width=180&height=180', 'link': '/product/CURAD-Nitrile-Exam-Gloves/Non-Navigable-For-Boost/Z05-PF00331;ecomsessionid=8YLYmH_8oOnAQOELjXzF4KY84kOVBXYqYWmHXXtR.OC2-b2b?question=&index=P4&indexCount=4', 'title': '\nCURAD Nitrile Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_08/PF56141/PF56141_PRI10.JPG?quality=high&width=180&height=180', 'link': '/product/SensiCare-Ice-Blue-Powder-Free-Nitrile-Exam-Gloves/SensiCare/Z05-PF56141;ecomsessionid=8YLYmH_8oOnAQOELjXzF4KY84kOVBXYqYWmHXXtR.OC2-b2b?question=&index=P5&indexCount=5', 'title': '\nSensiCare Ice Blue Powder-Free Nitrile Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_02/CA02_03_02_05/PF19768/PF19768_PRI02.JPG?quality=high&width=180&height=180', 'link': '/product/MediGuard-Synthetic-Exam-Gloves/MediGuard/Z05-PF19768;ecomsessionid=8YLYmH_8oOnAQOELjXzF4KY84kOVBXYqYWmHXXtR.OC2-b2b?question=&index=P6&indexCount=6', 'title': '\nMediGuard Synthetic Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_02/CA02_03_02_01/PF00344/PF00344_PRI04.JPG?quality=high&width=180&height=180', 'link': '/product/Accutouch-Synthetic-Exam-Gloves/Accutouch/Z05-PF00344;ecomsessionid=8YLYmH_8oOnAQOELjXzF4KY84kOVBXYqYWmHXXtR.OC2-b2b?question=&index=P7&indexCount=7', 'title': '\nAccutouch Synthetic Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_02/PF00327/PF00327_PRI03.JPG?quality=high&width=180&height=180', 'link': '/product/Aloetouch-Ice-Powder-Free-Nitrile-Exam-Gloves/Aloetouch/Z05-PF00327;ecomsessionid=8YLYmH_8oOnAQOELjXzF4KY84kOVBXYqYWmHXXtR.OC2-b2b?question=&index=P8&indexCount=8', 'title': '\nAloetouch Ice Powder-Free Nitrile Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_02/CA02_03_02_02/PF00345/PF00345_PRI05.JPG?quality=high&width=180&height=180', 'link': '/product/Aloetouch-3G-Powder-Free-Synthetic-Exam-Gloves/Aloetouch/Z05-PF00345;ecomsessionid=8YLYmH_8oOnAQOELjXzF4KY84kOVBXYqYWmHXXtR.OC2-b2b?question=&index=P9&indexCount=9', 'title': '\nAloetouch 3G Powder-Free Synthetic Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_07/CA02_03_07_01/PF00370/PF00370_PRI02.JPG?quality=high&width=180&height=180', 'link': '/product/SensiCare-Powder-Free-Stretch-Vinyl-Sterile-Exam-Gloves/Sensicare/Z05-PF00370;ecomsessionid=8YLYmH_8oOnAQOELjXzF4KY84kOVBXYqYWmHXXtR.OC2-b2b?question=&index=P10&indexCount=10', 'title': '\nSensiCare Powder-Free Stretch Vinyl Sterile Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_03/CA02_03_03_02/PF00356/PF00356_PRI05.JPG?quality=high&width=180&height=180', 'link': '/product/CURAD-Powder-Free-Textured-Latex-Exam-Gloves/Non-Navigable-For-Boost/Z05-PF00356;ecomsessionid=8YLYmH_8oOnAQOELjXzF4KY84kOVBXYqYWmHXXtR.OC2-b2b?question=&index=P11&indexCount=11', 'title': '\nCURAD Powder-Free Textured Latex Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_01/PF00326/PF00326_PRI01.JPG?quality=high&width=180&height=180', 'link': '/product/Accutouch-Chemo-Nitrile-Exam-Gloves/Accutouch/Z05-PF00326;ecomsessionid=8YLYmH_8oOnAQOELjXzF4KY84kOVBXYqYWmHXXtR.OC2-b2b?question=&index=P12&indexCount=12', 'title': '\nAccutouch Chemo Nitrile Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_06/CA02_03_06_01/PF00367/PF00367_PRI01.JPG?quality=high&width=180&height=180', 'link': '/product/Aloetouch-12-Powder-Free-Nitrile-Exam-Gloves/Aloetouch/Z05-PF00367;ecomsessionid=8YLYmH_8oOnAQOELjXzF4KY84kOVBXYqYWmHXXtR.OC2-b2b?question=&index=P13&indexCount=13', 'title': '\nAloetouch 12" Powder-Free Nitrile Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_02/CA02_03_02_07/PF00353/PF00353_PRI03.JPG?quality=high&width=180&height=180', 'link': '/product/Ultra-Stretch-Synthetic-Exam-Gloves/Ultra/Z05-PF00353;ecomsessionid=8YLYmH_8oOnAQOELjXzF4KY84kOVBXYqYWmHXXtR.OC2-b2b?question=&index=P14&indexCount=14', 'title': '\nUltra Stretch Synthetic Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_02/CA02_03_02_04/PF00349/PF00349_PRI07.JPG?quality=high&width=180&height=180', 'link': '/product/Generation-Pink-3G-Synthetic-Exam-Gloves/Generation-Pink/Z05-PF00349;ecomsessionid=8YLYmH_8oOnAQOELjXzF4KY84kOVBXYqYWmHXXtR.OC2-b2b?question=&index=P15&indexCount=15', 'title': '\nGeneration Pink 3G Synthetic Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_08/PF00339/PF00339_PRI05.JPG?quality=high&width=180&height=180', 'link': '/product/SensiCare-Extended-Cuff-Powder-Free-Nitrile-Exam-Gloves/SensiCare/Z05-PF00339;ecomsessionid=8YLYmH_8oOnAQOELjXzF4KY84kOVBXYqYWmHXXtR.OC2-b2b?question=&index=P16&indexCount=16', 'title': '\nSensiCare Extended Cuff Powder-Free Nitrile Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_03/CA02_03_03_04/PF00358/PF00358_PRI04.JPG?quality=high&width=180&height=180', 'link': '/product/Eudermic-MP-High-Risk-Powder-Free-Latex-Exam-Gloves/Eudermic-MP/Z05-PF00358;ecomsessionid=8YLYmH_8oOnAQOELjXzF4KY84kOVBXYqYWmHXXtR.OC2-b2b?question=&index=P17&indexCount=17', 'title': '\nEudermic MP High-Risk Powder-Free Latex Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_03/CA02_03_03_01/PF00354/PF00354_PRI03.JPG?quality=high&width=180&height=180', 'link': '/product/Aloetouch-Powder-Free-Latex-Exam-Gloves/Aloetouch/Z05-PF00354;ecomsessionid=8YLYmH_8oOnAQOELjXzF4KY84kOVBXYqYWmHXXtR.OC2-b2b?question=&index=P18&indexCount=18', 'title': '\nAloetouch Powder-Free Latex Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_03/PF61953/PF61953_PRI01.JPG?quality=high&width=180&height=180', 'link': '/product/CURAD-Powder-Free-Nitrile-Exam-Gloves/Curad/Z05-PF61953;ecomsessionid=8YLYmH_8oOnAQOELjXzF4KY84kOVBXYqYWmHXXtR.OC2-b2b?question=&index=P19&indexCount=19', 'title': '\nCURAD Powder-Free Nitrile Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_08/CA02_03_08_02/PF00373/PF00373_PRI01.JPG?quality=high&width=180&height=180', 'link': '/product/Medline-Sterile-Powder-Free-Latex-Exam-Gloves/Medline/Z05-PF00373;ecomsessionid=8YLYmH_8oOnAQOELjXzF4KY84kOVBXYqYWmHXXtR.OC2-b2b?question=&index=P20&indexCount=20', 'title': '\nMedline Sterile Powder-Free Latex Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_08/PF00338/PF00338_PRI04.JPG?quality=high&width=180&height=180', 'link': '/product/SensiCare-Silk-Powder-Free-Nitrile-Exam-Gloves/SensiCare/Z05-PF00338;ecomsessionid=8YLYmH_8oOnAQOELjXzF4KY84kOVBXYqYWmHXXtR.OC2-b2b?question=&index=P21&indexCount=21', 'title': '\nSensiCare Silk Powder-Free Nitrile Exam Gloves'}, {'img': '/media/catalog/sku/MDS/MDS194034_PRI03.JPG?quality=high&width=180&height=180', 'link': '/product/Medline-Sterile-Powder-Free-Latex-Exam-Glove-Pairs/Medline/Z05-PF00372;ecomsessionid=8YLYmH_8oOnAQOELjXzF4KY84kOVBXYqYWmHXXtR.OC2-b2b?question=&index=P22&indexCount=22', 'title': '\nMedline Sterile Powder-Free Latex Exam Glove Pairs'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_06/PF21246/PF21246_PRI03.JPG?quality=high&width=180&height=180', 'link': '/product/MediGuard-20-Nitrile-Exam-Gloves/MediGuard/Z05-PF21246;ecomsessionid=8YLYmH_8oOnAQOELjXzF4KY84kOVBXYqYWmHXXtR.OC2-b2b?question=&index=P23&indexCount=23', 'title': '\nMediGuard 2.0 Nitrile Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_02/CA02_03_02_10/PF13890/PF13890_PRI03.JPG?quality=high&width=180&height=180', 'link': '/product/Designer-Boxed-Vinyl-Exam-Gloves/Medline/Z05-PF13890;ecomsessionid=8YLYmH_8oOnAQOELjXzF4KY84kOVBXYqYWmHXXtR.OC2-b2b?question=&index=P24&indexCount=24', 'title': '\nDesigner Boxed Vinyl Exam Gloves'}], [{'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_09/PF00343/PF00343_PRI03.JPG?quality=high&width=180&height=180', 'link': '/product/Venom-Nonsterile-Powder-Free-Nitrile-Exam-Gloves/Venom/Z05-PF00343;ecomsessionid=j9Vn9v-IKFBcIb1m4Mvtv0eGE-m7t9iR0CPpgwsx.OC1-b2b?question=&index=P25&indexCount=25', 'title': '\nVenom Nonsterile Powder-Free Nitrile Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_03/CA02_03_03_07/PF00361/PF00361_PRI02.JPG?quality=high&width=180&height=180', 'link': '/product/Spruce-Nonsterile-Powder-Free-Latex-Exam-Gloves/Spruce/Z05-PF00361;ecomsessionid=j9Vn9v-IKFBcIb1m4Mvtv0eGE-m7t9iR0CPpgwsx.OC1-b2b?question=&index=P26&indexCount=26', 'title': '\nSpruce Nonsterile Powder-Free Latex Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_10/PF21875/PF21875_PRI04.JPG?quality=high&width=180&height=180', 'link': '/product/Solstice-Powder-Free-Nitrile-Exam-Gloves/Medline/Z05-PF21875;ecomsessionid=j9Vn9v-IKFBcIb1m4Mvtv0eGE-m7t9iR0CPpgwsx.OC1-b2b?question=&index=P27&indexCount=27', 'title': '\nSolstice Powder-Free Nitrile Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_05/PF00333/PF00333_PRI05.JPG?quality=high&width=180&height=180', 'link': '/product/Generation-Pink-Pearl-Nitrile-Exam-Gloves/Generation-Pink/Z05-PF00333;ecomsessionid=j9Vn9v-IKFBcIb1m4Mvtv0eGE-m7t9iR0CPpgwsx.OC1-b2b?question=&index=P28&indexCount=28', 'title': '\nGeneration Pink Pearl Nitrile Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_10/PF149180/PF149180_PRI02.JPG?quality=high&width=180&height=180', 'link': '/product/FitGuard-Prime-Nitrile-Exam-Gloves/Medline/Z05-PF149180;ecomsessionid=j9Vn9v-IKFBcIb1m4Mvtv0eGE-m7t9iR0CPpgwsx.OC1-b2b?question=&index=P29&indexCount=29', 'title': '\nFitGuard Prime Nitrile Exam Gloves'}, {'img': '/media/catalog/sku/MDS/MDS198314_PRI03.JPG?quality=high&width=180&height=180', 'link': '/product/Medline-Sterile-Powder-Free-Nitrile-Exam-Glove-Pairs/Medline/Z05-PF00369;ecomsessionid=j9Vn9v-IKFBcIb1m4Mvtv0eGE-m7t9iR0CPpgwsx.OC1-b2b?question=&index=P30&indexCount=30', 'title': '\nMedline Sterile Powder-Free Nitrile Exam Glove Pairs'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_06/CA02_03_06_01/PF00368/PF00368_PRI01.JPG?quality=high&width=180&height=180', 'link': '/product/Aloetouch-9-Powder-Free-Nitrile-Single-Exam-Gloves/Aloetouch/Z05-PF00368;ecomsessionid=j9Vn9v-IKFBcIb1m4Mvtv0eGE-m7t9iR0CPpgwsx.OC1-b2b?question=&index=P31&indexCount=31', 'title': '\nAloetouch 9" Powder-Free Nitrile Single Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_10/PF64087/PF64087_PRI04.JPG?quality=high&width=180&height=180', 'link': '/product/SmartGuard-Powder-Free-Nitrile-Exam-Gloves/Non-Navigable-For-Boost/Z05-PF64087;ecomsessionid=j9Vn9v-IKFBcIb1m4Mvtv0eGE-m7t9iR0CPpgwsx.OC1-b2b?question=&index=P32&indexCount=32', 'title': '\nSmartGuard Powder-Free Nitrile Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_02/CA02_03_02_10/PF112126/PF112126_PRI07.JPG?quality=high&width=180&height=180', 'link': '/product/Glide-On-Powder-Free-Vinyl-Exam-Gloves/Medline/Z05-PF112126;ecomsessionid=j9Vn9v-IKFBcIb1m4Mvtv0eGE-m7t9iR0CPpgwsx.OC1-b2b?question=&index=P33&indexCount=33', 'title': '\nGlide-On Powder-Free Vinyl Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_02/CA02_03_02_10/PF84819/PF84819_PRI05.JPG?quality=high&width=180&height=180', 'link': '/product/FitGuard-Stretch-Vinyl-Exam-Gloves/Medline/Z05-PF84819;ecomsessionid=j9Vn9v-IKFBcIb1m4Mvtv0eGE-m7t9iR0CPpgwsx.OC1-b2b?question=&index=P34&indexCount=34', 'title': '\nFitGuard Stretch Vinyl Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_13/PF129312/PF129312_PRI04.JPG?quality=high&width=180&height=180', 'link': '/product/CS-Pro-16-Cuff-Nitrile-Exam-Gloves/Branded-Nonsterile-Nitrile-Exam-Gloves/Z05-PF129312;ecomsessionid=j9Vn9v-IKFBcIb1m4Mvtv0eGE-m7t9iR0CPpgwsx.OC1-b2b?question=&index=P35&indexCount=35', 'title': '\nCS Pro 16" Cuff Nitrile Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_10/PF84816/PF84816_PRI03.JPG?quality=high&width=180&height=180', 'link': '/product/FitGuard-Nitrile-Exam-Gloves/Medline/Z05-PF84816;ecomsessionid=j9Vn9v-IKFBcIb1m4Mvtv0eGE-m7t9iR0CPpgwsx.OC1-b2b?question=&index=P36&indexCount=36', 'title': '\nFitGuard Nitrile Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_02/PF81028/PF81028_PRI06.JPG?quality=high&width=180&height=180', 'link': '/product/Medline-Professional-Nitrile-Exam-Gloves-with-Aloe/Aloetouch/Z05-PF81028;ecomsessionid=j9Vn9v-IKFBcIb1m4Mvtv0eGE-m7t9iR0CPpgwsx.OC1-b2b?question=&index=P37&indexCount=37', 'title': '\nMedline Professional Nitrile Exam Gloves with Aloe'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_06/CA02_03_06_02/PF149184/PF149184_PRI01.JPG?quality=high&width=180&height=180', 'link': '/product/Medline-Extended-Cuff-Nitrile-Exam-Glove-Pairs/Medline/Z05-PF149184;ecomsessionid=j9Vn9v-IKFBcIb1m4Mvtv0eGE-m7t9iR0CPpgwsx.OC1-b2b?question=&index=P38&indexCount=38', 'title': '\nMedline Extended Cuff Nitrile Exam Glove Pairs'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_06/CA02_03_06_02/PF149183/PF149183_PRI01.JPG?quality=high&width=180&height=180', 'link': '/product/Medline-Extended-Cuff-Nitrile-Exam-Glove-Singles/Medline/Z05-PF149183;ecomsessionid=j9Vn9v-IKFBcIb1m4Mvtv0eGE-m7t9iR0CPpgwsx.OC1-b2b?question=&index=P39&indexCount=39', 'title': '\nMedline Extended Cuff Nitrile Exam Glove Singles'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_10/PF84818/PF84818_PRI04.JPG?quality=high&width=180&height=180', 'link': '/product/FitGuard-Ultra-Powder-Free-Nitrile-Exam-Gloves/Medline/Z05-PF84818;ecomsessionid=j9Vn9v-IKFBcIb1m4Mvtv0eGE-m7t9iR0CPpgwsx.OC1-b2b?question=&index=P40&indexCount=40', 'title': '\nFitGuard Ultra Powder-Free Nitrile Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_05/PF00334/PF00334_PRI03.JPG?quality=high&width=180&height=180', 'link': '/product/Generation-Pink-Powder-Free-Nitrile-Exam-Gloves/Generation-Pink/Z05-PF00334;ecomsessionid=j9Vn9v-IKFBcIb1m4Mvtv0eGE-m7t9iR0CPpgwsx.OC1-b2b?question=&index=P41&indexCount=41', 'title': '\nGeneration Pink Powder-Free Nitrile Exam Gloves'}, {'img': '/media/catalog/sku/MDS/MDS198214_PRI03.JPG?quality=high&width=180&height=180', 'link': '/product/Medline-Sterile-Powder-Free-Non-Latex-Nitrile-Exam-Gloves/Medline/Z05-PF65609;ecomsessionid=j9Vn9v-IKFBcIb1m4Mvtv0eGE-m7t9iR0CPpgwsx.OC1-b2b?question=&index=P42&indexCount=42', 'title': '\nMedline Sterile Powder-Free Non-Latex Nitrile Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_13/PF152511/PF152511_PRI03.JPG?quality=high&width=180&height=180', 'link': '/product/FitGuard-Touch-Nitrile-Exam-Gloves/Branded-Nonsterile-Nitrile-Exam-Gloves/Z05-PF152511;ecomsessionid=j9Vn9v-IKFBcIb1m4Mvtv0eGE-m7t9iR0CPpgwsx.OC1-b2b?question=&index=P43&indexCount=43', 'title': '\nFitGuard Touch Nitrile Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_03/PF42081/PF42081_PRI04.JPG?quality=high&width=180&height=180', 'link': '/product/CURAD-White-Nitrile-Exam-Gloves/Non-Navigable-For-Boost/Z05-PF42081;ecomsessionid=j9Vn9v-IKFBcIb1m4Mvtv0eGE-m7t9iR0CPpgwsx.OC1-b2b?question=&index=P44&indexCount=44', 'title': '\nCURAD White Nitrile Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_02/CA02_03_02_02/PF101925/PF101925_PRI01.JPG?quality=high&width=180&height=180', 'link': '/product/Aloetouch-3G-Synthetic-Exam-Gloves-CA-Only/Aloetouch/Z05-PF101925;ecomsessionid=j9Vn9v-IKFBcIb1m4Mvtv0eGE-m7t9iR0CPpgwsx.OC1-b2b?question=&index=P45&indexCount=45', 'title': '\nAloetouch 3G Synthetic Exam Gloves - CA Only'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_02/CA02_03_02_05/PF101932/PF101932_PRI01.JPG?quality=high&width=180&height=180', 'link': '/product/MediGuard-Vinyl-Synthetic-Exam-Gloves-CA-Only/MediGuard/Z05-PF101932;ecomsessionid=j9Vn9v-IKFBcIb1m4Mvtv0eGE-m7t9iR0CPpgwsx.OC1-b2b?question=&index=P46&indexCount=46', 'title': '\nMediGuard Vinyl Synthetic Exam Gloves - CA Only'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_02/CA02_03_02_03/PF101930/PF101930_PRI01.JPG?quality=high&width=180&height=180', 'link': '/product/CURAD-Stretch-Vinyl-Exam-Gloves-CA-Only/Curad/Z05-PF101930;ecomsessionid=j9Vn9v-IKFBcIb1m4Mvtv0eGE-m7t9iR0CPpgwsx.OC1-b2b?question=&index=P47&indexCount=47', 'title': '\nCURAD Stretch Vinyl Exam Gloves - CA Only'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_03/PF164126/PF164126_HRE01.JPG?quality=high&width=180&height=180', 'link': '/product/CURAD-Germshield-Nitrile-Exam-Gloves/Curad/Z05-PF164126;ecomsessionid=j9Vn9v-IKFBcIb1m4Mvtv0eGE-m7t9iR0CPpgwsx.OC1-b2b?question=&index=P48&indexCount=48', 'title': '\nCURAD Germshield Nitrile Exam Gloves'}], [{'img': '/media/catalog/CA02/CA02_03/CA02_03_02/CA02_03_02_01/PF101924/PF101924_PRI01.JPG?quality=high&width=180&height=180', 'link': '/product/Accutouch-Synthetic-Exam-Gloves-CA-Only/Accutouch/Z05-PF101924;ecomsessionid=sX4N8IskisxTruFKifHnaQM2wDnvaTBQFFnD7YP3.OC8-b2b?question=&index=P49&indexCount=49', 'title': '\nAccutouch Synthetic Exam Gloves - CA Only'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_03/CA02_03_03_01/PF84222/PF84222_PRI04.JPG?quality=high&width=180&height=180', 'link': '/product/Medline-Professional-Latex-Exam-Gloves/Aloetouch/Z05-PF84222;ecomsessionid=sX4N8IskisxTruFKifHnaQM2wDnvaTBQFFnD7YP3.OC8-b2b?question=&index=P50&indexCount=50', 'title': '\nMedline Professional Latex Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_02/CA02_03_02_10/PF101935/PF101935_PRI02.JPG?quality=high&width=180&height=180', 'link': '/product/Vinyl-Exam-Gloves-CA-Only/Medline/Z05-PF101935;ecomsessionid=sX4N8IskisxTruFKifHnaQM2wDnvaTBQFFnD7YP3.OC8-b2b?question=&index=P51&indexCount=51', 'title': '\nVinyl Exam Gloves - CA Only'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_02/CA02_03_02_05/PF101931/PF101931_PRI01.JPG?quality=high&width=180&height=180', 'link': '/product/MediGuard-Synthetic-Exam-Gloves-CA-Only/MediGuard/Z05-PF101931;ecomsessionid=sX4N8IskisxTruFKifHnaQM2wDnvaTBQFFnD7YP3.OC8-b2b?question=&index=P52&indexCount=52', 'title': '\nMediGuard Synthetic Exam Gloves - CA Only'}, {'img': '/media/catalog/sku/EQP/EQPT4121_PRI01.JPG?quality=high&width=180&height=180', 'link': '/product/EQPT-Powder-Free-Disposable-Latex-Gloves/Branded-Nonsterile-PF-Latex-Exam-Gloves/Z05-PF146919;ecomsessionid=sX4N8IskisxTruFKifHnaQM2wDnvaTBQFFnD7YP3.OC8-b2b?question=&index=P53&indexCount=53', 'title': '\nEQPT Powder-Free Disposable Latex Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_02/CA02_03_02_10/PF101934/PF101934_PRI01.JPG?quality=high&width=180&height=180', 'link': '/product/Designer-Boxed-Vinyl-Exam-Gloves-CA-Only/Medline/Z05-PF101934;ecomsessionid=sX4N8IskisxTruFKifHnaQM2wDnvaTBQFFnD7YP3.OC8-b2b?question=&index=P54&indexCount=54', 'title': '\nDesigner Boxed Vinyl Exam Gloves - CA Only'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_02/PF84142/PF84142_PRI04.JPG?quality=high&width=180&height=180', 'link': '/product/Medline-Professional-Lightly-Textured-Aloe-Exam-Gloves/Aloetouch/Z05-PF84142;ecomsessionid=sX4N8IskisxTruFKifHnaQM2wDnvaTBQFFnD7YP3.OC8-b2b?question=&index=P55&indexCount=55', 'title': '\nMedline Professional Lightly Textured Aloe Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_02/CA02_03_02_04/PF111036/PF111036_PRI02.JPG?quality=high&width=180&height=180', 'link': '/product/CA-Generation-Pink-3G-Synthetic-Exam-Gloves/Generation-Pink/Z05-PF111036;ecomsessionid=sX4N8IskisxTruFKifHnaQM2wDnvaTBQFFnD7YP3.OC8-b2b?question=&index=P56&indexCount=56', 'title': '\nCA - Generation Pink 3G Synthetic Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_02/CA02_03_02_10/PF113869/PF113869_PRI02.JPG?quality=high&width=180&height=180', 'link': '/product/Glide-On-Powder-Free-Vinyl-Exam-Gloves-CA-Only/Medline/Z05-PF113869;ecomsessionid=sX4N8IskisxTruFKifHnaQM2wDnvaTBQFFnD7YP3.OC8-b2b?question=&index=P57&indexCount=57', 'title': '\nGlide-On Powder-Free Vinyl Exam Gloves - CA Only'}, {'img': '/media/catalog/sku/546/546846_PRI01.JPG?quality=high&width=180&height=180', 'link': '/product/ProVision-Select-Powder-Free-Exam-Gloves/Medline/Z05-PF106917;ecomsessionid=sX4N8IskisxTruFKifHnaQM2wDnvaTBQFFnD7YP3.OC8-b2b?question=&index=P58&indexCount=58', 'title': '\nProVision Select Powder-Free Exam Gloves'}, {'img': '/media/catalog/sku/OAT/OAT4583_PRI02.JPG?quality=high&width=180&height=180', 'link': '/product/Restore-Sense-Powder-Free-Nitrile-Exam-Gloves-with-maxOat/SensiCare/Z05-PF188723;ecomsessionid=sX4N8IskisxTruFKifHnaQM2wDnvaTBQFFnD7YP3.OC8-b2b?question=&index=P59&indexCount=59', 'title': '\nRestore Sense Powder-Free Nitrile Exam Gloves with maxOat+'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_08/PF166969/PF166969_PRI02.JPG?quality=high&width=180&height=180', 'link': '/product/Restore-Touch-Nitrile-Exam-Gloves-with-Oatmeal/SensiCare/Z05-PF166969;ecomsessionid=sX4N8IskisxTruFKifHnaQM2wDnvaTBQFFnD7YP3.OC8-b2b?question=&index=P60&indexCount=60', 'title': '\nRestore Touch Nitrile Exam Gloves with Oatmeal'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_10/PF189775/PF189775_PRI01.JPG?quality=high&width=180&height=180', 'link': '/product/Woodland-Friends-Nitrile-Exam-Gloves/Medline/Z05-PF189775;ecomsessionid=sX4N8IskisxTruFKifHnaQM2wDnvaTBQFFnD7YP3.OC8-b2b?question=&index=P61&indexCount=61', 'title': '\nWoodland Friends Nitrile Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_08/PF149181/PF149181_PRI04.JPG?quality=high&width=180&height=180', 'link': '/product/VersaShield-Extended-Cuff-Powder-Free-Nitrile-Exam-Gloves/SensiCare/Z05-PF149181;ecomsessionid=sX4N8IskisxTruFKifHnaQM2wDnvaTBQFFnD7YP3.OC8-b2b?question=&index=P62&indexCount=62', 'title': '\nVersaShield Extended Cuff Powder-Free Nitrile Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_08/PF178153/PF178153_HRE01.JPG?quality=high&width=180&height=180', 'link': '/product/Generation-Pink-Sense-Powder-Free-Nitrile-Exam-Gloves/SensiCare/Z05-PF178153;ecomsessionid=sX4N8IskisxTruFKifHnaQM2wDnvaTBQFFnD7YP3.OC8-b2b?question=&index=P63&indexCount=63', 'title': '\nGeneration Pink Sense Powder-Free Nitrile Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_10/PF146918/PF146918_PRI01.JPG?quality=high&width=180&height=180', 'link': '/product/EQPT-Nitrile-Powder-Free-Disposable-Gloves/Medline/Z05-PF146918;ecomsessionid=sX4N8IskisxTruFKifHnaQM2wDnvaTBQFFnD7YP3.OC8-b2b?question=&index=P64&indexCount=64', 'title': '\nEQPT Nitrile Powder-Free Disposable Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_10/PF105686/PF105686_PRI09.JPG?quality=high&width=180&height=180', 'link': '/product/Critical-Response-Powder-Free-Nitrile-Exam-Gloves/Medline/Z05-PF105686;ecomsessionid=sX4N8IskisxTruFKifHnaQM2wDnvaTBQFFnD7YP3.OC8-b2b?question=&index=P65&indexCount=65', 'title': '\nCritical Response Powder-Free Nitrile Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_03/PF149182/PF149182_PRI04.JPG?quality=high&width=180&height=180', 'link': '/product/VersaShield-Powder-Free-Nitrile-Exam-Gloves/Curad/Z05-PF149182;ecomsessionid=sX4N8IskisxTruFKifHnaQM2wDnvaTBQFFnD7YP3.OC8-b2b?question=&index=P66&indexCount=66', 'title': '\nVersaShield Powder-Free Nitrile Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_10/PF84817/PF84817_PRI04.JPG?quality=high&width=180&height=180', 'link': '/product/FitGuard-Select-Powder-Free-Nitrile-Exam-Gloves/Medline/Z05-PF84817;ecomsessionid=sX4N8IskisxTruFKifHnaQM2wDnvaTBQFFnD7YP3.OC8-b2b?question=&index=P67&indexCount=67', 'title': '\nFitGuard Select Powder-Free Nitrile Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_08/PF88156/PF88156_PRI12.JPG?quality=high&width=180&height=180', 'link': '/product/Restore-Powder-Free-Nitrile-Exam-Gloves-with-Oatmeal/SensiCare/Z05-PF88156;ecomsessionid=sX4N8IskisxTruFKifHnaQM2wDnvaTBQFFnD7YP3.OC8-b2b?question=&index=P68&indexCount=68', 'title': '\nRestore Powder-Free Nitrile Exam Gloves with Oatmeal'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_06/PF159466/PF159466_PRI04.JPG?quality=high&width=180&height=180', 'link': '/product/MediGuard-ES-Powder-Free-Nitrile-Exam-Gloves/MediGuard/Z05-PF159466;ecomsessionid=sX4N8IskisxTruFKifHnaQM2wDnvaTBQFFnD7YP3.OC8-b2b?question=&index=P69&indexCount=69', 'title': '\nMediGuard ES Powder-Free Nitrile Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_10/PF114581/PF114581_PRI06.JPG?quality=high&width=180&height=180', 'link': '/product/FitGuard-Touch-Powder-Free-Nitrile-Exam-Gloves/Medline/Z05-PF114581;ecomsessionid=sX4N8IskisxTruFKifHnaQM2wDnvaTBQFFnD7YP3.OC8-b2b?question=&index=P70&indexCount=70', 'title': '\nFitGuard Touch Powder-Free Nitrile Exam Gloves'}, {'img': '/media/catalog/CA02/CA02_03/CA02_03_01/CA02_03_01_08/PF145694/PF145694_PRI09.JPG?quality=high&width=180&height=180', 'link': '/product/Restore-Sense-Powder-Free-Nitrile-Exam-Gloves-with-Oatmeal/SensiCare/Z05-PF145694;ecomsessionid=sX4N8IskisxTruFKifHnaQM2wDnvaTBQFFnD7YP3.OC8-b2b?question=&index=P71&indexCount=71', 'title': '\nRestore Sense Powder-Free Nitrile Exam Gloves with Oatmeal'}, {'img': '/media/catalog/sku/EQP/EQPT4131_PRI01.JPG?quality=high&width=180&height=180', 'link': '/product/EQPT-Powder-Free-Disposable-Vinyl-Gloves/Medline/Z05-PF146920;ecomsessionid=sX4N8IskisxTruFKifHnaQM2wDnvaTBQFFnD7YP3.OC8-b2b?question=&index=P72&indexCount=72', 'title': '\nEQPT Powder-Free Disposable Vinyl Gloves'}]]

从网页https://www.medline.com/catalog/category-products.jsp?itemId=Z05-CA02_03&N=111079+4294770643&iclp=Z05-CA02_03打开 and scrape it you have to induce WebDriverWait for the number_of_windows_to_be(2) and you can use the following 中的每个产品:

  • 代码块:

      from selenium import webdriver
      from selenium.webdriver.support.ui import WebDriverWait
      from selenium.webdriver.common.by import By
      from selenium.webdriver.support import expected_conditions as EC
      import time
    
      chrome_options = webdriver.ChromeOptions() 
      chrome_options.add_argument("start-maximized")
      driver = webdriver.Chrome(options=chrome_options, executable_path=r'C:\WebDrivers\chromedriver.exe')
    
      driver.get("https://www.medline.com/catalog/category-products.jsp?itemId=Z05-CA02_03&N=111079+4294770643&iclp=Z05-CA02_03")
      my_hrefs = [my_elem.get_attribute("href") for my_elem in WebDriverWait(driver, 5).until(EC.visibility_of_all_elements_located((By.XPATH, "//div[contains(@class, 'resultGalleryViewRow')]//div[@class='medGridProdTitle']//a")))]
      windows_before  = driver.current_window_handle # Store the parent_window_handle for future use
      for my_href in my_hrefs:
          driver.execute_script("window.open('" + my_href +"');")
          WebDriverWait(driver, 10).until(EC.number_of_windows_to_be(2)) # Induce  WebDriverWait for the number_of_windows_to_be 2
          windows_after = driver.window_handles
          new_window = [x for x in windows_after if x != windows_before][0] # Identify the newly opened window
          driver.switch_to.window(new_window) # switch_to the new window
          time.sleep(3) # perform your webscraping here
          print(driver.title) # print the page title or your perform your webscraping
          driver.close() # close the window
          driver.switch_to.window(windows_before) # switch_to the parent_window_handle
      driver.quit() #quit your program
    
  • 控制台输出:

      SensiCare Powder-Free Nitrile Exam Gloves | Medline Industries, Inc.
      MediGuard Vinyl Synthetic Exam Gloves | Medline Industries, Inc.
      CURAD Stretch Vinyl Exam Gloves | Medline Industries, Inc.
      CURAD Nitrile Exam Gloves | Medline Industries, Inc.
      SensiCare Ice Blue Powder-Free Nitrile Exam Gloves | Medline Industries, Inc.
      MediGuard Synthetic Exam Gloves | Medline Industries, Inc.
      Accutouch Synthetic Exam Gloves | Medline Industries, Inc.
      Aloetouch Ice Powder-Free Nitrile Exam Gloves | Medline Industries, Inc.
      Aloetouch 3G Powder-Free Synthetic Exam Gloves | Medline Industries, Inc.
      SensiCare Powder-Free Stretch Vinyl Sterile Exam Gloves | Medline Industries, Inc.
      CURAD Powder-Free Textured Latex Exam Gloves | Medline Industries, Inc.
      Accutouch Chemo Nitrile Exam Gloves | Medline Industries, Inc.
      Aloetouch 12" Powder-Free Nitrile Exam Gloves | Medline Industries, Inc.
      Ultra Stretch Synthetic Exam Gloves | Medline Industries, Inc.
      Generation Pink 3G Synthetic Exam Gloves | Medline Industries, Inc.
      SensiCare Extended Cuff Powder-Free Nitrile Exam Gloves | Medline Industries, Inc.
      Eudermic MP High-Risk Powder-Free Latex Exam Gloves | Medline Industries, Inc.
      Aloetouch Powder-Free Latex Exam Gloves | Medline Industries, Inc.
      CURAD Powder-Free Nitrile Exam Gloves | Medline Industries, Inc.
      Medline Sterile Powder-Free Latex Exam Gloves | Medline Industries, Inc.
      SensiCare Silk Powder-Free Nitrile Exam Gloves | Medline Industries, Inc.
      Medline Sterile Powder-Free Latex Exam Glove Pairs | Medline Industries, Inc.
      MediGuard 2.0 Nitrile Exam Gloves | Medline Industries, Inc.
      Designer Boxed Vinyl Exam Gloves | Medline Industries, Inc.
    

参考资料

您可以在以下位置找到一些相关的详细讨论:

  • StaleElementReferenceException even after adding the wait while collecting the data from the wikipedia using web-scraping