使用美汤刮位置
Scraping location using beautiful soup
我是 beautifulsoup 的超级新手，我已经制作了大量的在线视频，现在我正在探索我的第一个项目。
无论如何，我的目标是抓取 https://www.mastermindtoys.com/apps/store/find-a-store 的位置。所有位置都在一个 class“clearfix 大型容器”下。我想知道如何从所有“address-sec”中提取地址信息。 “address-sec”是“clearfix large-container”下的class。如果有人有内联或建议的视频，那就太好了！
您看到的数据是从外部源加载的，因此beautifulsoup看不到。但是你可以用 requests 模块模拟 Ajax 请求来加载它：
import json
import requests
from bs4 import BeautifulSoup
url = "https://bopis.mastermindtoys.com/ajax/getstorelocation.php"
headers = {
"Referer": "https://www.mastermindtoys.com/",
}
payload = {
"doAction": "getNearStoreData",
"latitude": "",
"longitude": "",
"flagDisplayFirstRecord": "Y",
"shopifyCustomerId": "",
"selectedStoreInCookies": "",
}
data = requests.post(url, headers=headers, data=payload).json()
# uncomment this to print all data:
# print(json.dumps(data, indent=4))
for i, d in enumerate(data["details"], 1):
print(
"{:<3} {:<25} {:<30} {:<30}".format(
i, d["name"], d["address1"], d["address2"]
)
)
打印：
1 Abbotsford 32470 South Fraser Way Clearbrook Town Square
2 Langley 20085 Langley Bypass Langley Centre
3 Coquitlam 3000 Lougheed Hwy Westwood Mall
4 Tsawwassen 5000 Canoe Pass Way Tsawwassen Mills
5 Langford 2945 Jacklin Road Westshore Town Centre
6 Prince George 3034 Recplace Drive River Point Landing
7 Westhills 600 Stewart Green SW Westhills Town Centre
8 Shawnessy 16061 MacLeod Trail SE Boulevard Shopping Centre
9 McKenzie Towne 4307 130 Ave SE South Trail Crossing
10 Country Hills 450 Country Hills Blvd NE Country Hills Town Centre
11 Lethbridge 1250 2A Avenue North
12 Red Deer 5250 22nd Street
13 Grande Prairie 11521 Westgate Drive Westgate Centre
14 Windermere 5236 Windermere Boulevard NW Currents of Windermere
15 Terra Losa 9752 170 Street Terra Losa Centre
16 South Edmonton 2004 99th Street NW
17 St. Albert 398 St. Albert Trail Mission Hill Plaza
18 Sherwood Park 222 Baseline Road Sherwood Park Baseline Village Plaza
19 Medicine Hat 1840 Strachan Road SE
20 Saskatoon South 3011 Clarence Avenue South Unit 170
21 Preston Crossing 1718 Preston Avenue North
22 Regina East 2820 East Quance Street
23 Kenaston 1655 Kenaston Boulevard
24 Kildonan 1561 Regent Avenue West Kildonan Mall
25 Sudbury 1332 Kingsway Boulevard
26 Windsor 3175 Howard Avenue Roundhouse Centre
27 Owen Sound 1969 16th Street East
28 London North 685 Fanshawe Park Road West
29 London South 3120 Wonderland Road South Wonderland Plaza
30 Waterloo 405 King Street North
31 Barrie 488 Bayfield Street North Supercentre
32 Cambridge 612 Hespeler Road
33 Guelph 370 Stone Road West Stone Square Centre
34 Milton 1079 Maple Avenue
35 Newmarket 18267 Yonge Street
36 Aurora 650 Wellington Street East Aurora Gateway Centre
37 Ancaster 821 Golf Links Road Meadowlands Entertainment Centre
38 Vaughan 16 Famous Avenue Colossus Centre
39 Hurontario 26 Eglinton Avenue West
40 Burlington 1035 Brant Street
41 Oakville 1011 Upper Middle Road East Upper Oakville Shopping Centre
42 Richmond Hill 9350 Yonge Street
43 Mississauga South 970 Southdown Road Clarkson Crossing Centre
44 Etobicoke 4242 Dundas Street West Kingsway Mills Shopping Centre
45 Hamilton 2180 Rymal Road East
46 Bayview Village 2901 Bayview Avenue Bayview Village Shopping Centre
47 Yonge 3350 Yonge Street
48 Markham 8555 McCowan Rd
49 Mount Pleasant 639 Mount Pleasant Road
50 Rosedale 1133 Yonge Street
51 The Beach 2134 Queen Street East
52 Pickering 1355 Kingston Road Pickering Town Centre
53 Ajax 170 Kingston Road East
54 North Oshawa 991 Taunton Road East
55 St. Catharines 210 Glendale Avenue
56 Peterborough 995 Lansdowne Street West
57 Kanata 145 Roland Michener Drive Kanata Centrum Plaza
58 Kingston 2511 Princess Street
59 Barrhaven 71 Marketplace Avenue Chapman Mills Marketplace
60 St. Laurent 1091 St Laurent Boulevard
61 Orléans 1315 Place d'Orleans Drive Place d'Orléans
62 Fredericton 38 Trinity Avenue
63 Saint John 70 Consumers Drive Westmorland Shopping Centre
64 Moncton 84 Wyse Street Mapleton Centre
65 Halifax 201 Chain Lake Drive
66 Dartmouth 29 Countryview Drive Dartmouth Crossing
67 Sydney 20 Sydney Port Access Road
68 St. John's North 50 White Rose Drive
我是 beautifulsoup 的超级新手，我已经制作了大量的在线视频，现在我正在探索我的第一个项目。
无论如何，我的目标是抓取 https://www.mastermindtoys.com/apps/store/find-a-store 的位置。所有位置都在一个 class“clearfix 大型容器”下。我想知道如何从所有“address-sec”中提取地址信息。 “address-sec”是“clearfix large-container”下的class。如果有人有内联或建议的视频，那就太好了！
您看到的数据是从外部源加载的，因此beautifulsoup看不到。但是你可以用 requests 模块模拟 Ajax 请求来加载它：
import json
import requests
from bs4 import BeautifulSoup
url = "https://bopis.mastermindtoys.com/ajax/getstorelocation.php"
headers = {
"Referer": "https://www.mastermindtoys.com/",
}
payload = {
"doAction": "getNearStoreData",
"latitude": "",
"longitude": "",
"flagDisplayFirstRecord": "Y",
"shopifyCustomerId": "",
"selectedStoreInCookies": "",
}
data = requests.post(url, headers=headers, data=payload).json()
# uncomment this to print all data:
# print(json.dumps(data, indent=4))
for i, d in enumerate(data["details"], 1):
print(
"{:<3} {:<25} {:<30} {:<30}".format(
i, d["name"], d["address1"], d["address2"]
)
)
打印：
1 Abbotsford 32470 South Fraser Way Clearbrook Town Square
2 Langley 20085 Langley Bypass Langley Centre
3 Coquitlam 3000 Lougheed Hwy Westwood Mall
4 Tsawwassen 5000 Canoe Pass Way Tsawwassen Mills
5 Langford 2945 Jacklin Road Westshore Town Centre
6 Prince George 3034 Recplace Drive River Point Landing
7 Westhills 600 Stewart Green SW Westhills Town Centre
8 Shawnessy 16061 MacLeod Trail SE Boulevard Shopping Centre
9 McKenzie Towne 4307 130 Ave SE South Trail Crossing
10 Country Hills 450 Country Hills Blvd NE Country Hills Town Centre
11 Lethbridge 1250 2A Avenue North
12 Red Deer 5250 22nd Street
13 Grande Prairie 11521 Westgate Drive Westgate Centre
14 Windermere 5236 Windermere Boulevard NW Currents of Windermere
15 Terra Losa 9752 170 Street Terra Losa Centre
16 South Edmonton 2004 99th Street NW
17 St. Albert 398 St. Albert Trail Mission Hill Plaza
18 Sherwood Park 222 Baseline Road Sherwood Park Baseline Village Plaza
19 Medicine Hat 1840 Strachan Road SE
20 Saskatoon South 3011 Clarence Avenue South Unit 170
21 Preston Crossing 1718 Preston Avenue North
22 Regina East 2820 East Quance Street
23 Kenaston 1655 Kenaston Boulevard
24 Kildonan 1561 Regent Avenue West Kildonan Mall
25 Sudbury 1332 Kingsway Boulevard
26 Windsor 3175 Howard Avenue Roundhouse Centre
27 Owen Sound 1969 16th Street East
28 London North 685 Fanshawe Park Road West
29 London South 3120 Wonderland Road South Wonderland Plaza
30 Waterloo 405 King Street North
31 Barrie 488 Bayfield Street North Supercentre
32 Cambridge 612 Hespeler Road
33 Guelph 370 Stone Road West Stone Square Centre
34 Milton 1079 Maple Avenue
35 Newmarket 18267 Yonge Street
36 Aurora 650 Wellington Street East Aurora Gateway Centre
37 Ancaster 821 Golf Links Road Meadowlands Entertainment Centre
38 Vaughan 16 Famous Avenue Colossus Centre
39 Hurontario 26 Eglinton Avenue West
40 Burlington 1035 Brant Street
41 Oakville 1011 Upper Middle Road East Upper Oakville Shopping Centre
42 Richmond Hill 9350 Yonge Street
43 Mississauga South 970 Southdown Road Clarkson Crossing Centre
44 Etobicoke 4242 Dundas Street West Kingsway Mills Shopping Centre
45 Hamilton 2180 Rymal Road East
46 Bayview Village 2901 Bayview Avenue Bayview Village Shopping Centre
47 Yonge 3350 Yonge Street
48 Markham 8555 McCowan Rd
49 Mount Pleasant 639 Mount Pleasant Road
50 Rosedale 1133 Yonge Street
51 The Beach 2134 Queen Street East
52 Pickering 1355 Kingston Road Pickering Town Centre
53 Ajax 170 Kingston Road East
54 North Oshawa 991 Taunton Road East
55 St. Catharines 210 Glendale Avenue
56 Peterborough 995 Lansdowne Street West
57 Kanata 145 Roland Michener Drive Kanata Centrum Plaza
58 Kingston 2511 Princess Street
59 Barrhaven 71 Marketplace Avenue Chapman Mills Marketplace
60 St. Laurent 1091 St Laurent Boulevard
61 Orléans 1315 Place d'Orleans Drive Place d'Orléans
62 Fredericton 38 Trinity Avenue
63 Saint John 70 Consumers Drive Westmorland Shopping Centre
64 Moncton 84 Wyse Street Mapleton Centre
65 Halifax 201 Chain Lake Drive
66 Dartmouth 29 Countryview Drive Dartmouth Crossing
67 Sydney 20 Sydney Port Access Road
68 St. John's North 50 White Rose Drive