无法获取页面的所有 P 标签
having trouble in getting all the P tags of page
文章段落 div被编入不同的 div 标签
就像在图片中你可以看到它写着 data-page-number="2"
就像数据被 divided 到 8 个不同的 divs 而问题是 divs 进一步进入 div 然后有一个前置标签
当我写代码的时候
from bs4 import BeautifulSoup
headers = {
'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:55.0) Gecko/20100101 Firefox/55.0',
}
link="https://www.wattpad.com/1163185219-the-amazing-charlie-wade-volume-2-chapter-2001"
url=f"{link}"
r=requests.get(url,headers=headers)
soup=BeautifulSoup(r.content,'html.parser')
#print(soup.prettify()
div=soup.find("div",{"id":"sp1163185219-pg1"})
print(div.get_text())
输出为
Mrs. Willson had never been so happy to shake her whole body to eat.
According to the foreman, she had been busy at the grocery store, and at seven in the afternoon, she finally received the food coupon she was thinking of.
Although she had already had a full meal at noon, Ms. Willson still looked like a starving dog that had been confined for three days with this meal in the mess hall. He had a big belly and couldn't stand up.
I could no longer eat. He rested on the canteen seat for half an hour before reluctantly getting up with his hands on the table.
After eating and drinking, the whole person has energy.
Mrs. Willson wiped her mouth with the back of her hand and touched her belly with the palm of her hand, feeling self-satisfied: "Unexpectedly, eating after this delivery is so delicious! Today I had enough to eat! "
Immediately, he looked at the time and realized that it was already past eight, so he planned to go home quickly.
Although he has always been selfish, after living through all the hardships and hardships during this period, he has more or less cared for his children and grandchildren.
Thinking of her son and grandson still lying in bed waiting to be fed, Mrs. Willson rushed to the cafeteria window and asked the staff member who was about to get off work, "Excuse me, do you still need these leftovers, steamed buns and rice? "
The other person looked at her and said helplessly, "Didn't you, old lady, have you eaten three people's food alone and haven't eaten enough?"
s. Willson wiped her mouth in embarrassment and said with a smile, "Hey, I'm full and the kids in the family are still hungry. You know, they haven't eaten anything for almost two days and two nights. "
the other party asked in surprise, "Really? When is there someone who still cannot eat? "
Ms. Willson had a hot face and said sheepishly, "Hey, something happened to my family during this period of time. The oldest daughter-in-law in the family has breast cancer. It is in an advanced stage. The ball is about to die, my eldest son spent all the money at home to cure his illness ... "
With that said, Ms. Willson suddenly felt very refreshed in her heart.
It's pretty cool!
had been a long time since I had seen Horiyah that my eyes did not like him. If Regnar hadn't stopped her, she would have wanted to beat Horiyah to death herself.
Especially recently, Horiyah has disobeyed her in various ways and even forced her to work. She has long hated her for this in her heart.
Therefore, in front of the others, manufacturing the rumor that Horiyah was dying of breast cancer made her feel the pleasure of revenge.
When the cafeteria staff heard this, they suddenly exclaimed, "Huh? Advanced breast cancer, that would be too miserable ... "
"That's not a big deal!" Mrs. Willson sighed, "Hey ... My oldest daughter-in-law has advanced breast cancer and my second daughter-in-law is even more miserable!"
he staff member hurriedly asked, "Ma'am, what happened to your second daughter-in-law?"
Ms Willson said regretfully: "The second daughter-in-law has uremia and is on dialysis all day. However, on the way to the hospital for her dialysis some time ago, she was hit by a car, and her limbs were amputated under her $ s... "
he staff were scared and dumbfounded: "Isn't this ... so miserable?"
Mrs. Willson said with a face, "What is this? The worst is yet to come!
要么这里是下一段
我得到了部分结果
当我改变
div=soup.find("div",{"id":"sp1163185219-pg1"}) 至 div=soup.find("div",{"id":"sp1163185219-pg2"})
并重新 运行 失败的代码。
这里是 link 网站 =https://www.wattpad.com/1163185219-the-amazing-charlie-wade-volume-2-chapter-2001
如果我能完成这个，那将是很大的帮助
预先感谢您的帮助
尝试使用正则表达式进行匹配，然后找到子 p 个标签。
import re
soup = BeautifulSoup(r.content,'html.parser')
divs = soup.findAll("div",{"id": re.compile(r"sp1163185219-pg\d+")})
for div in divs:
p_tags = div.findAll("p")
for p in p_tags:
print(p.get_text())
输出：
Chapter 2001
Mrs. Willson had never been so happy to shake her whole body to eat.
According to the foreman, she had been busy at the grocery store, and at seven in the afternoon, she finally received the food coupon she was thinking of.
Although she had already had a full meal at noon, Ms. Willson still looked like a starving dog that had been confined for three days with this meal in the mess hall. He had a big belly and couldn't stand up.
I could no longer eat. He rested on the canteen seat for half an hour before reluctantly getting up with his hands on the table.
After eating and drinking, the whole person has energy.
Mrs. Willson wiped her mouth with the back of her hand and touched her belly with the palm of her hand, feeling self-satisfied: "Unexpectedly, eating after this delivery is so delicious! Today I had enough to eat! "
Immediately, he looked at the time and realized that it was already past eight, so he planned to go home quickly.
Although he has always been selfish, after living through all the hardships and hardships during this period, he has more or less cared for his children and grandchildren.
Thinking of her son and grandson still lying in bed waiting to be fed, Mrs. Willson rushed to the cafeteria window and asked the staff member who was about to get off work, "Excuse me, do you still need these leftovers, steamed buns and rice? "
The other person looked at her and said helplessly, "Didn't you, old lady, have you eaten three people's food alone and haven't eaten enough?"
Ms. Willson wiped her mouth in embarrassment and said with a smile, "Hey, I'm full and the kids in the family are still hungry. You know, they haven't eaten anything for almost two days and two nights. "
The other party asked in surprise, "Really? When is there someone who still cannot eat? "
Ms. Willson had a hot face and said sheepishly, "Hey, something happened to my family during this period of time. The oldest daughter-in-law in the family has breast cancer. It is in an advanced stage. The ball is about to die, my eldest son spent all the money at home to cure his illness ... "
With that said, Ms. Willson suddenly felt very refreshed in her heart.
It's pretty cool!
It had been a long time since I had seen Horiyah that my eyes did not like him. If Regnar hadn't stopped her, she would have wanted to beat Horiyah to death herself.
Especially recently, Horiyah has disobeyed her in various ways and even forced her to work. She has long hated her for this in her heart.
Therefore, in front of the others, manufacturing the rumor that Horiyah was dying of breast cancer made her feel the pleasure of revenge.
When the cafeteria staff heard this, they suddenly exclaimed, "Huh? Advanced breast cancer, that would be too miserable ... "
"That's not a big deal!" Mrs. Willson sighed, "Hey ... My oldest daughter-in-law has advanced breast cancer and my second daughter-in-law is even more miserable!"
The staff member hurriedly asked, "Ma'am, what happened to your second daughter-in-law?"
Ms Willson said regretfully: "The second daughter-in-law has uremia and is on dialysis all day. However, on the way to the hospital for her dialysis some time ago, she was hit by a car, and her limbs were amputated under her $ s... "
The staff were scared and dumbfounded: "Isn't this ... so miserable?"
Mrs. Willson said with a face, "What is this? The worst is yet to come! "
数据也来自 API
import requests
api_url = 'https://www.wattpad.com/api/v3/stories/293954234/recommended?limit=7&fields=id%2Ctitle%2CvoteCount%2CreadCount%2CcreateDate%2ClastPublishedPart%2Cuser(name%2Cavatar)%2Ccover%2Curl%2CnumParts%2CisPaywalled%2Ccompleted%2Cmature%2Cdescription'
headers={
"content-type": "application/json",
"x-requested-with": "XMLHttpRequest",
'User-Agent': 'Mozilla/5.0'
}
req =requests.get(api_url,headers=headers).json()
#print(req)
for para in req:
txt=para['description']
print(txt)
或
urlchapter-2001 的端点包含 10 页
import requests
from bs4 import BeautifulSoup
headers = {'User-Agent': 'Mozilla/5.0'}
url="https://www.wattpad.com/1163185219-the-amazing-charlie-wade-volume-2-chapter-2001/page/{page}"
for page in range(1,11):
r=requests.get(url.format(page=page),headers=headers)
soup=BeautifulSoup(r.content,'html.parser')
#print(soup.prettify()
div=soup.find("div",{"id":"sp1163185219-pg1"})
print(div.get_text(strip=True).replace('Chapter 2001',''))
文章段落 div被编入不同的 div 标签
就像在图片中你可以看到它写着 data-page-number="2"
就像数据被 divided 到 8 个不同的 divs 而问题是 divs 进一步进入 div 然后有一个前置标签
当我写代码的时候
from bs4 import BeautifulSoup
headers = {
'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:55.0) Gecko/20100101 Firefox/55.0',
}
link="https://www.wattpad.com/1163185219-the-amazing-charlie-wade-volume-2-chapter-2001"
url=f"{link}"
r=requests.get(url,headers=headers)
soup=BeautifulSoup(r.content,'html.parser')
#print(soup.prettify()
div=soup.find("div",{"id":"sp1163185219-pg1"})
print(div.get_text())
输出为
Mrs. Willson had never been so happy to shake her whole body to eat.
According to the foreman, she had been busy at the grocery store, and at seven in the afternoon, she finally received the food coupon she was thinking of.
Although she had already had a full meal at noon, Ms. Willson still looked like a starving dog that had been confined for three days with this meal in the mess hall. He had a big belly and couldn't stand up.
I could no longer eat. He rested on the canteen seat for half an hour before reluctantly getting up with his hands on the table.
After eating and drinking, the whole person has energy.
Mrs. Willson wiped her mouth with the back of her hand and touched her belly with the palm of her hand, feeling self-satisfied: "Unexpectedly, eating after this delivery is so delicious! Today I had enough to eat! "
Immediately, he looked at the time and realized that it was already past eight, so he planned to go home quickly.
Although he has always been selfish, after living through all the hardships and hardships during this period, he has more or less cared for his children and grandchildren.
Thinking of her son and grandson still lying in bed waiting to be fed, Mrs. Willson rushed to the cafeteria window and asked the staff member who was about to get off work, "Excuse me, do you still need these leftovers, steamed buns and rice? "
The other person looked at her and said helplessly, "Didn't you, old lady, have you eaten three people's food alone and haven't eaten enough?"
s. Willson wiped her mouth in embarrassment and said with a smile, "Hey, I'm full and the kids in the family are still hungry. You know, they haven't eaten anything for almost two days and two nights. "
the other party asked in surprise, "Really? When is there someone who still cannot eat? "
Ms. Willson had a hot face and said sheepishly, "Hey, something happened to my family during this period of time. The oldest daughter-in-law in the family has breast cancer. It is in an advanced stage. The ball is about to die, my eldest son spent all the money at home to cure his illness ... "
With that said, Ms. Willson suddenly felt very refreshed in her heart.
It's pretty cool!
had been a long time since I had seen Horiyah that my eyes did not like him. If Regnar hadn't stopped her, she would have wanted to beat Horiyah to death herself.
Especially recently, Horiyah has disobeyed her in various ways and even forced her to work. She has long hated her for this in her heart.
Therefore, in front of the others, manufacturing the rumor that Horiyah was dying of breast cancer made her feel the pleasure of revenge.
When the cafeteria staff heard this, they suddenly exclaimed, "Huh? Advanced breast cancer, that would be too miserable ... "
"That's not a big deal!" Mrs. Willson sighed, "Hey ... My oldest daughter-in-law has advanced breast cancer and my second daughter-in-law is even more miserable!"
he staff member hurriedly asked, "Ma'am, what happened to your second daughter-in-law?"
Ms Willson said regretfully: "The second daughter-in-law has uremia and is on dialysis all day. However, on the way to the hospital for her dialysis some time ago, she was hit by a car, and her limbs were amputated under her $ s... "
he staff were scared and dumbfounded: "Isn't this ... so miserable?"
Mrs. Willson said with a face, "What is this? The worst is yet to come!
要么这里是下一段
我得到了部分结果
当我改变
div=soup.find("div",{"id":"sp1163185219-pg1"}) 至 div=soup.find("div",{"id":"sp1163185219-pg2"})
并重新 运行 失败的代码。
这里是 link 网站 =https://www.wattpad.com/1163185219-the-amazing-charlie-wade-volume-2-chapter-2001
如果我能完成这个，那将是很大的帮助 预先感谢您的帮助
尝试使用正则表达式进行匹配，然后找到子 p 个标签。
import re
soup = BeautifulSoup(r.content,'html.parser')
divs = soup.findAll("div",{"id": re.compile(r"sp1163185219-pg\d+")})
for div in divs:
p_tags = div.findAll("p")
for p in p_tags:
print(p.get_text())
输出：
Chapter 2001
Mrs. Willson had never been so happy to shake her whole body to eat.
According to the foreman, she had been busy at the grocery store, and at seven in the afternoon, she finally received the food coupon she was thinking of.
Although she had already had a full meal at noon, Ms. Willson still looked like a starving dog that had been confined for three days with this meal in the mess hall. He had a big belly and couldn't stand up.
I could no longer eat. He rested on the canteen seat for half an hour before reluctantly getting up with his hands on the table.
After eating and drinking, the whole person has energy.
Mrs. Willson wiped her mouth with the back of her hand and touched her belly with the palm of her hand, feeling self-satisfied: "Unexpectedly, eating after this delivery is so delicious! Today I had enough to eat! "
Immediately, he looked at the time and realized that it was already past eight, so he planned to go home quickly.
Although he has always been selfish, after living through all the hardships and hardships during this period, he has more or less cared for his children and grandchildren.
Thinking of her son and grandson still lying in bed waiting to be fed, Mrs. Willson rushed to the cafeteria window and asked the staff member who was about to get off work, "Excuse me, do you still need these leftovers, steamed buns and rice? "
The other person looked at her and said helplessly, "Didn't you, old lady, have you eaten three people's food alone and haven't eaten enough?"
Ms. Willson wiped her mouth in embarrassment and said with a smile, "Hey, I'm full and the kids in the family are still hungry. You know, they haven't eaten anything for almost two days and two nights. "
The other party asked in surprise, "Really? When is there someone who still cannot eat? "
Ms. Willson had a hot face and said sheepishly, "Hey, something happened to my family during this period of time. The oldest daughter-in-law in the family has breast cancer. It is in an advanced stage. The ball is about to die, my eldest son spent all the money at home to cure his illness ... "
With that said, Ms. Willson suddenly felt very refreshed in her heart.
It's pretty cool!
It had been a long time since I had seen Horiyah that my eyes did not like him. If Regnar hadn't stopped her, she would have wanted to beat Horiyah to death herself.
Especially recently, Horiyah has disobeyed her in various ways and even forced her to work. She has long hated her for this in her heart.
Therefore, in front of the others, manufacturing the rumor that Horiyah was dying of breast cancer made her feel the pleasure of revenge.
When the cafeteria staff heard this, they suddenly exclaimed, "Huh? Advanced breast cancer, that would be too miserable ... "
"That's not a big deal!" Mrs. Willson sighed, "Hey ... My oldest daughter-in-law has advanced breast cancer and my second daughter-in-law is even more miserable!"
The staff member hurriedly asked, "Ma'am, what happened to your second daughter-in-law?"
Ms Willson said regretfully: "The second daughter-in-law has uremia and is on dialysis all day. However, on the way to the hospital for her dialysis some time ago, she was hit by a car, and her limbs were amputated under her $ s... "
The staff were scared and dumbfounded: "Isn't this ... so miserable?"
Mrs. Willson said with a face, "What is this? The worst is yet to come! "
数据也来自 API
import requests
api_url = 'https://www.wattpad.com/api/v3/stories/293954234/recommended?limit=7&fields=id%2Ctitle%2CvoteCount%2CreadCount%2CcreateDate%2ClastPublishedPart%2Cuser(name%2Cavatar)%2Ccover%2Curl%2CnumParts%2CisPaywalled%2Ccompleted%2Cmature%2Cdescription'
headers={
"content-type": "application/json",
"x-requested-with": "XMLHttpRequest",
'User-Agent': 'Mozilla/5.0'
}
req =requests.get(api_url,headers=headers).json()
#print(req)
for para in req:
txt=para['description']
print(txt)
或
urlchapter-2001 的端点包含 10 页
import requests
from bs4 import BeautifulSoup
headers = {'User-Agent': 'Mozilla/5.0'}
url="https://www.wattpad.com/1163185219-the-amazing-charlie-wade-volume-2-chapter-2001/page/{page}"
for page in range(1,11):
r=requests.get(url.format(page=page),headers=headers)
soup=BeautifulSoup(r.content,'html.parser')
#print(soup.prettify()
div=soup.find("div",{"id":"sp1163185219-pg1"})
print(div.get_text(strip=True).replace('Chapter 2001',''))