使用 Beautiful Soup 抓取 Yelp 评论显示不同标签的内容

Scraping Yelp review content displaying different tags using Beautiful Soup

我正在练习网络抓取并尝试从以下页面获取评论:https://www.yelp.com/biz/jajaja-plantas-mexicana-new-york-2?osq=Vegetarian+Food

这是我在检查网页上的名称元素后得到的结果:

page = requests.get('https://www.yelp.com/biz/jajaja-plantas-mexicana-new-york-2?osq=Vegetarian+Food', headers={'User-Agent':'Mozilla/5.0'}).text

parsed_page = BeautifulSoup(page, 'lxml')

# print(parsed_page)

for x in parsed_page.find_all('a', class_='css-1422juy'):
    print(x)

但是好像不行,输出的不是名字而是:

<a class="css-1422juy" href="/c/mexican">Mexican</a>
<a class="css-1422juy" href="/c/vegan">Vegan</a>
<a class="css-1422juy" href="/c/bars">Bars</a>
<a class="css-1422juy" href="https://www.yelp.com/menu/jajaja-plantas-mexicana-new-york-2" role="link">View full menu<span class="display--inline__09f24__c6N_k margin-l1__09f24__m8GL9 border-color--default__09f24__NPAKY"><span aria-hidden="true" class="icon--14-chevron-right-outline css-mpwjkl"><svg class="icon_svg" height="14" width="14"><path d="M5.043 11.5a.498.498 0 00.353-.146L9.75 7 5.396 2.646a.5.5 0 00-.707.708L8.336 7l-3.647 3.646a.502.502 0 00.354.854z"></path></svg></span></span></a>
<a class="css-1422juy" href="/questions/XipQLDbyTl5tsLlyzAWzug" role="link">Ask a question<span class="display--inline__09f24__c6N_k margin-l1__09f24__m8GL9 border-color--default__09f24__NPAKY"><span aria-hidden="true" class="icon--24-add-v2 css-106vfgv"><svg class="icon_svg" height="24" width="24"><path d="M19 11h-6V5a1 1 0 10-2 0v6H5a1 1 0 100 2h6v6a1 1 0 102 0v-6h6a1 1 0 100-2z"></path></svg></span></span></a>

如果我使用 html.parser.prettify() 而不是控制台中名称、评级和评论字段的解析输出看起来非常不同:

<script type="application/ld+json">
     {"@context":"https://schema.org","@type":"Restaurant","name":"Jajaja Plantas Mexicana","image":"https://s3-media0.fl.yelpcdn.com/bphoto/OkWKXxOZBLJO7hRjOlIMig/l.jpg","priceRange":"-30","telephone":"(646) 883-5453","address":{"streetAddress":"162 E Broadway","addressLocality":"New York","addressCountry":"US","addressRegion":"NY","postalCode":"10002"},"review":[{"author":"Caroline J.","datePublished":"2021-11-24","reviewRating":{"ratingValue":5},"description":"I typically don&apos;t write reviews for restaurants, but I&apos;ll make an exception! I have been vegan for almost 4 years now and meatless for about 8 years now. My sister told me about this place and we didn&apos;t get a chance to go until last night. My sister is non-vegan, my mom is also non-vegan. I went with my sister, and we got food to share. My sister who is a very skeptical about vegan food absolutely loved it. My mom who is also skeptical about vegan food had my sisters leftovers and absolutely loved it as well! Everything was delicious and I love the atmosphere in the restaurant as well! Definitely will be going back because they have other stuff on the menu I want to try! It&apos;s both vegan and non vegan approved for a 100% vegan Mexican  restaurant!!"},{"author":"Melissa W.","datePublished":"2021-12-24","reviewRating":{"ratingValue":5},"description":"The food was great. I went with my partner for a birthday dinner and while I have adopted a plant based lifestyle, he has not but he enjoyed EVERYTHING that we ordered."},{"author":"Sooji L.","datePublished":"2022-01-09","reviewRating":{"ratingValue":5},"description":"NACHOS NACHOS NACHOS ( with guac). I prefer having these nachos than other nachos that aren&apos;t vegan. Soooooooo gooooood. I have to say that to pay extra  for guac, they can add more than just a scoop. The portions are enough for 3 people! \n\nFish tacos were great. Replaced with squash and flavorful all together.  for two tacos. You may add an extra taco for an extra /4 bucks. \n\nMy husband ordered the Gorditas. The shell was so crunchy and good. It was supposed to have &quot;bacon&quot; but it was quite underwhelming. Wouldn&apos;t quite recommend this dish.\n\nBoth drinks were delish! Almond horchata... I would get that the next time I come back. It is a bit gritty though.\nI ordered the matcha y coconut cocktail. Cocktails are $$. It was . Some drinks can be turned into a mock tail for the same price which I find that to be unreasonable. Regardless, the drinks are worth getting.\n\n****ASK FOR HOT SAUCE! and you&apos;ll get three choices. I love my hot sauce and the variety they provided. The mild (orange) was my favorite. \n\n(Rating was not based on this following experience) on our visit, there was a homeless man in their outdoor seating area which we were able to view from our seat inside. At some moments, he pulled down his pants. Not pleasant, I&apos;d say. The staff did try to remove him but he did not budge. I hope they reached out to homeless services. He was there during our entire dining experience. \n\nMy rating is purely on the food and service. That situation was out of their control but do please keep in mind that the neighborhood is not the best area. They do have multiple locations so please consider checking them out."},{"author":"Roshni P.","datePublished":"2021-11-11","reviewRating":{"ratingValue":5},"description":"I was skeptical because I am from Southern California and Mexican has to hit right for me. But holy crap this is amazing!!!! \n\nBirria tacos was our least favorite but that&apos;s not saying much because I would give it 9.5/10 \n\nBurrito was 10/10 . As you eat it there is more and more flavor and so so good! I like the red sauce side more \n\nNachos is hands down the best item!!! Plenty of food for your meal. I would give it a 11/10. The chorizo is soo good. Ask for hot sauces on the side. The orange one is really good"},{"author":"Whitney L.","datePublished":"2021-11-03","reviewRating":{"ratingValue":5},"description":"This place hits the spot on so many levels.\n\n Service was attentive, efficient and personable. Check!\n\nFood---outstanding. I ordered the Tavon Taco bowl, pumpkin and beet empanada and the Matcha and Coconut Cocktail. Everything was delicious and seasoned well. If you like matcha, don&apos;t sleep on that Matcha +Coconut Cocktail....OUTSTANDING. Everything tasted truly unique---like something you can&apos;t get other places. And even better, lots of veggies and healthy options. Double- check!!\n\nTheir homemade chorizo is yum yum yum. Make sure to try that!\n\nDecor and ambiance was fun, colorful and on brand. This place has great energy. Highly recommend and can&apos;t wait to go back."},{"author":"Johnny G.","datePublished":"2021-12-27","reviewRating":{"ratingValue":4},"description":"The nachos are to die for. Unbelievable, so clean and not heavy. Couldn&apos;t tell the difference from any carnivorous nacho I&apos;ve had. I loved it and wished I stopped there.\nI had the Coconut Queso Quesadilla and wasn&apos;t really a fan. Tasted like a pasta dish (heavy on the pesto) and it had a sweetness. Nonetheless great food overall."},{"author":"Kathy X.","datePublished":"2021-09-25","reviewRating":{"ratingValue":4},"description":"Recently, I came here with a friend for lunch. I was looking forward to trying plant-based food and drink. The interior was lovely, bright, and modern. There is a fully stocked bar with plants all around, orange lights dropping down from the ceiling, and nice wood tables. \n\nFor my meal, I ordered Almond Horchata (always have to get the Horchata if it&apos;s on the menu), Crispy Chayote Fish, and Enchiladas Mole. I also got to try my friend&apos;s Chorizo Nachos and Mexican Street Corn. \n\nThe Almond Horchata came in a wine glass with a beautiful deep purple orchid flower. The color of the drink was pale brown with bits of coconut on top. It was delightfully creamy, cinnamon-y, and sweet. \n\nChayote Fish consisted of hemp and flax seed buttered squash, and the taco also came with chipotle almond butter and red onion. The squash was fried to perfection in a deliciously well-seasoned batter, and truly resembled the taste of fish. The chipotle-flavored butter added a depth of flavor to the tacos, while the red onion gave it an additional crunch and fresh element. \n\nEnchiladas Moles had shredded palm carnitas, coconut queso, guajillo, sour cream, and also came with Spanish rice. This was absolutely wonderful! The palm carnitas was tender, succulent, with plenty of flavorful spices. The sauce that accompanied the enchiladas was yummy as well. In addition, the Spanish rice was cooked well. \n\nNachos came with plant based chorizo, fermented black beans, corn, turmeric, queso fundido, and sour cream. What a mouthwatering medley of flavors that mingled together to create very tasty bites. The chorizo was juicy, succulent, and packed with bold flavor. The cheese sauce with the sour cream was addicting and was well distributed throughout all the chips. \n\nThe Mexican Street Corn was covered with a powdery type of cheese, and slightly charred (which is how I like it) and was good.  \n\nService was top notch. Our waiter was friendly and kind. Everything was made with such care and attention to detail. I also loved the unique menu offerings. I would definitely like to return and try the Coconut Queso Quesadilla, and perhaps the Peanut Chocolate Cake."},{"author":"Alissa M.","datePublished":"2022-01-09","reviewRating":{"ratingValue":4},"description":"Came here for lunch with my wife on a Saturday afternoon. The restaurant is absolutely adorable inside! I love the plants and the decor all throughout. The nachos are to die for! Definitely the best nachos I have had since becoming dairy free. The coquito was also very good! The display of the chorizo burrito and barbacoa tacos was cute but I wasn&apos;t a huge fan of the texture of the barbacoa or the flavor of the protein inside the burrito. We will definitely be back for the Nachos and I must try the mini churros next time!"},{"author":"Steven Z.","datePublished":"2021-08-12","reviewRating":{"ratingValue":5},"description":"Great Mexican food with a cool interior playing Hispanic music which made for a nice ambience. Prices are fair\n\nMy friend and I were starving after a long day and ordered a lot of food: nachos, chorizo burrito, fish tacos and a quesadilla.  \n\nThe nachos (with guac) were really good and made for a great appetizer, the chorizo burrito was delicious (but very messy since it&apos;s covered in sauce), the quesadilla was really good too. But the highlight of the meal was the fish tacos - nice crispy outside but soft and yummy inside. If I had to get one dish again, it&apos;d be the fish taco. \n\nThe frozen agave margarita was a refreshing as well; it&apos;s pretty strong so you&apos;ll definitely taste the alcohol."},{"author":"Lily C.","datePublished":"2021-10-30","reviewRating":{"ratingValue":5},"description":"We got the fish tacos, carnitas, nachos with guacamole, and churros! Everything was so good and we couldn&apos;t believe it was all vegan! My boyfriend and I are both meat eaters, but we noticed that we didn&apos;t feel sluggish/sleepy (as we would with our non-vegan meals) after finishing our food. Definitely get the churros if you&apos;re craving for something sweet after your meal! \n\n5/5 Highly recommend whether you&apos;re a vegan or not! :D"},{"author":"Rebecca R.","datePublished":"2021-12-26","reviewRating":{"ratingValue":5},"description":"Yummy food. Inventive and interesting. Wide variety. The nachos were great. Good sized portions. Accommodating for preferences and such. Even the kiddos loved it!"},{"author":"Shubhi M.","datePublished":"2021-07-18","reviewRating":{"ratingValue":5},"description":"I&apos;ve never tried a vegan mexican food that tasted this good. It&apos;s crazy some of the vegan taco options they had (birria, carnitas, chorizo, barbacoa, and fish), I have never had an opportunity to try vegan version of those dishes. I&apos;ve never had barbacoa or fish but I can imagine this is may somewhat taste like. It was almost a weird (but definitely good) experience how meaty the texture was of the tacos. \n\nThe nachos were beyond amazing, a pet peeve of mine is when I get served a bunch of chips with toppings only on the top and the bottom of the pile is just plain chips but these were perfectly layered with so much vegan chorizo. They recently opened a location in a whole food which is very close to me so i&apos;m definitely going to go back!"},{"author":"Likitha M.","datePublished":"2021-08-11","reviewRating":{"ratingValue":4},"description":"My friend picked out this place for us to try and I didn&apos;t realize that it was all plant-based until our waitress mentioned it. We didn&apos;t have a reservation but we were seated as soon as we walked in and given a QR code to scan for the menus. \n\nIt felt like there were so many options to choose from but we ultimately narrowed it down and ordered the nachos with added guac, chayote &quot;fish&quot; tacos, chorizo tacos, barbacoa tacos, and almond horchata. We LOVED the nachos -- the chips were nice and crispy and all the toppings were so cohesive. There was a really good topping/chip ratio so we never ran out of one or the other. In addition, the sauces added were delicious -- if you come by and only want one thing to order, get the nachos!! The tacos were all so flavorful and delicious as well, but so small and expensive. I personally don&apos;t think they were worth  for 2 tacos, especially if it only took like 3 bites to consume but I understand the pricing as it is vegan and located in NYC. Finally, the horchata was very refreshing and I loved it! \n\nOverall, they had really delicious food and it was a pretty decent experience for my first time having an all plant-based meal!"},{"author":"Fionna L.","datePublished":"2021-07-14","reviewRating":{"ratingValue":4},"description":"Link to menu: https://qrcgcustomers.s3-eu-west-1.amazonaws.com/account8682491/15091203_1.pdf?0.4352589218767642 \n\n(4.5/5) The lively, communal atmosphere made eating here a pleasure! We sat at the bar so service was slower but never neglectful. Their nachos and tacos are must-trys and great to share with friends!\n\n - Nachos:  +  guac: the BEST plant-based nachos I&apos;ve had! Their chorizo was chewy and flavorful, the fermented black beans were yum and very filling, and the guac was definitely worth it for added depth and creaminess. Small thing, but I wish they had more than just a couple red and purple nachos.\n - Chorizo Burrito: : too salty! We could barely taste the ingredients or differentiate the sauces because salt dominated the flavor of every bite. Quite disappointed in this one. \n - Crispy Chayote Fish Tacos (2): : how ingenious to use &quot;hemp &amp; flax seed battered squash&quot; as a substitute for fish! The description sounds sophisticated but it tasted very much like BBQ Lay&apos;s, even down to the delicious crunch. Due to the strength of its flavorings, we couldn&apos;t taste any of the squash&apos;s original flavor. The ring of red onion was aesthetic but would have been easier to eat if chopped.\n\nSide note: their restrooms downstairs demonstrate how they turn rustic elements unique through modern and innovative design. \n\nAll in all a great experience, though I wouldn&apos;t recommend the burrito. Would love to go back to try the Coconut Queso Quesadilla and Spicy Birria Tacos!!"},{"author":"Sharon J.","datePublished":"2021-10-16","reviewRating":{"ratingValue":4},"description":"Kinda overrated food, but great vibes! Pretty good for a vegan restaurant, but I&apos;ve had better vegan food and Mexican food. \n\nI got the chorizo burrito but it was very salty with not a lot of other flavor besides that - almost as if they were using salt to cover up the lack of flavor. Definitely very pretty food though, nice interior, and great service - just wish the food was more flavorful. Maybe I&apos;ll try the tacos next time and see if those are better..."},{"author":"Srini V.","datePublished":"2021-09-24","reviewRating":{"ratingValue":5},"description":"I have been meaning to visit this location of Jajaja for some time now.  And the opportunity presented itself as a few of us were looking for a neighborhood restaurant in the middle of a torrential downpour.\n\nWe chose to take a table inside.  The QR code for their menu was pasted on the wall by our table.  We had the guacamole y chips to start.   I then got the sopa del día, which happened to be a green soup (sopa verde).  This was the highlight of my meal.  The soup was thick, had a generous sprinkling of hot sauce and was just perfect, especially after the rain.  I then had the chipotle sweet potato street tacos that were delicious ... the fermented beans made for a hearty filling.\n\nGreat service.  Nice ambiance.  Everyone was happy with their food and drinks.  I will be back soon!"},{"author":"Julianna Y.","datePublished":"2021-09-29","reviewRating":{"ratingValue":5},"description":"This place is DELICIOUS! I&apos;ve been ordering my lunch here for a while now, and I have to say that everything I&apos;ve had has been so good! This is a vegan Mexican heaven for foodies! Things you should order when you come here are: \n\n-Turmeric Cauliflower Rice  \n-Taco Tazón bowl \n-Nachos! \n-chorizo Burrito \n\nAside from these yummy dishes, their hot sauces are amazing! Definitely recommend getting the mild (orange) and black bean sauce!"},{"author":"Saasha G.","datePublished":"2021-09-14","reviewRating":{"ratingValue":4},"description":"I have became a certified vegan food lover! Never have I tasted such great food and no meat in anything. I was astonished at the flavors. This was my first time dining here and I will return. The place is small and minimal decor but honestly I don&apos;t care just give me the food!"},{"author":"Edward K.","datePublished":"2021-11-20","reviewRating":{"ratingValue":5},"description":"My friend and I get dinner once a month, we alternate choosing and this month my friend selected Jajaja. We arrived a half hour before our 6:45 reservation on a Thursday. The gentleman at the door was able to get us seated right away.\n\nThe restaurant has a trendy vibe, cool furniture and lighting. We did get wedged into a slightly uncomfortable corner table but it may have been uncomfortable mainly because I&apos;m on the taller side.\n\nThe goal of our monthly outings is to continually challenge our palates. We&apos;ve had the vegan/vegetarian genre before but we hadn&apos;t tried vegan Mexican before. The menu is cool, lots of choices that are able to be shared. We selected 5 starters, 2 entree style dishes, and three tacos. The heart of palm ceviche was very fresh. The beet and pumpkin empanada was piping hot and flavorful. I enjoyed the tamale with jackfruit. The Gordita and Burrito entrees were pretty substantial and also tasty. We found the tacos to be a highlight, the cauliflower especially. The birria and &quot;fish&quot; tacos were also quite nice, the birria was perhaps the dish with the most kick and the &quot;fish&quot; had a good texture.\n\nService was solid throughout the meal. Although we ate a solid amount of dishes for two people, I didn&apos;t feel overstuffed. Definitely a place worth returning to."},{"author":"Pamela L.","datePublished":"2021-02-15","reviewRating":{"ratingValue":4},"description":"As I was walking to my mom&apos;s place, this new restaurant stopped me in my tracks. OMG...I remember that this spot used to be a Chinese bakery. As a kid, I used to frequent the old Golden Carriage to grab a drink after a day at the park with my best friend. When I got to my mom&apos;s I check the place out on Yelp. Asked my son if he would be interested in trying Vegan food. He frowned until I convinced him by showing him the photos.\n\nNYC just opened their indoor dining again on the 14th. Temps were taken before we were allowed to sit down. Right off the bat, let me just say that our waiter was amazing, very attentive and kept asking how we liked everything. Decor-wise...I can&apos;t believe this is the same place, it&apos;s an amazing renovation.\n\nOne thing I read and saw was that their nachos w/chorizo is a must order. When it came to the table, I was like...no way we will finish that. Ummm, it was so frickin delish that we had no trouble. The first thing my son said was...I can&apos;t tell that it&apos;s Vegan and he&apos;s a huge meat eater. Our waiter asked if we wanted hot sauce with our nachos. He came by with 3 different ones and explained the level of hottest. The most mild one reminded me of a mole. I thought that was interesting.\n\nWe all enjoyed the tacos we ordered. The portions were generous and the ingredients fresh. I&apos;m not familiar with Vegan cuisine but now I&apos;m game to venture more into it. \n\nOn a last note: there is major gentrification that has been happening in the Lower East Side of Manhattan for years and I for one do welcome it. Some might argue about it but I think it makes the neighborhood more upscale."}],"aggregateRating":{"@type":"AggregateRating","ratingValue":4.5,"reviewCount":1078},"servesCuisine":"Mexican"}
    </script>

我如何使用 BeautifulSoup 访问此特定脚本标签中的内容以及其中的 json 信息?或者有没有办法通过带有 lxml.find_all()<a> 标签来实现?

您可以使用 json 模块来解析脚本标签的内容,可通过 .text 字段

访问

这里是解析所有脚本jsons并打印名称的例子:

import json

import requests
from bs4 import BeautifulSoup

url = 'https://www.yelp.com/biz/jajaja-plantas-mexicana-new-york-2?osq=Vegetarian+Food'

r = requests.get(
    url,
    headers={'User-Agent':'Mozilla/5.0'},
)

parsed_page = BeautifulSoup(r.text, 'html.parser')

page_jsons = []
for x in parsed_page.select('script[type="application/ld+json"]'):
    try:
        data = json.loads(x.text)
    except:
        continue

    page_jsons.append(data)

for d in page_jsons:
    if d.get('name'):
        print(d['name'])  # => Jajaja Plantas Mexicana