如何解决'POST data should be bytes, an iterable of bytes, or a file object. It cannot be of type str.'？

Question

我需要从站点获取硬币分析。跨度为 class 'speedometerSignal-DPgs-R4s sellColor-DPgs-R4s'。它应该可以工作，但事实并非如此。

我的代码：

import urllib.request as urllib2
from bs4 import BeautifulSoup

soup = BeautifulSoup(urllib2.urlopen("https://ru.tradingview.com/symbols/DOTUSDT/technicals/", 'html'))
result = soup.find_all("span", {"class":"speedometerSignal-DPgs-R4s sellColor-DPgs-R4s"})

for res in result:
    print(res.decode_contents().strip())

和错误：

    Traceback (most recent call last):
  File "/home/andrey/PycharmProjects/pfkegf/analis.py", line 4, in <module>
    soup = BeautifulSoup(urllib2.urlopen("https://ru.tradingview.com/symbols/DOTUSDT/technicals/", 'html').json())
  File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.8/urllib/request.py", line 522, in open
    req = meth(req)
  File "/usr/lib/python3.8/urllib/request.py", line 1285, in do_request_
    raise TypeError(msg)
TypeError: POST data should be bytes, an iterable of bytes, or a file object. It cannot be of type str.

请帮忙。

Answer 1

soup = BeautifulSoup(urllib2.urlopen("https://ru.tradingview.com/symbols/DOTUSDT/technicals/", 'html').read())

这应该可以解决您的问题。

Answer 2

如评论中所述，您需要在抓取之前对其进行渲染。这可以使用 selenium 网络驱动程序来完成，但首先需要进行一些设置。

安装硒

pip install selenium

为您的浏览器安装驱动程序。这是一个二进制文件，特定于您的 OS 和浏览器。例如，chrome 网络驱动程序可以在 https://chromedriver.chromium.org/downloads
下载
在您的代码中提供安装到 selenium 的网络驱动程序的路径：

from selenium import webdriver
from bs4 import BeautifulSoup

web_driver = webdriver.Chrome(<path-to-your-installed-web-driver>)

url = "https://ru.tradingview.com/symbols/DOTUSDT/technicals/"

# initiate scrape of website page data
web_driver.get(url)

# get the (dynamically generated) content
content = web_driver.page_source

# use soup to make page content easily searchable
soup = BeautifulSoup(content, 'html.parser')

spans = soup.findAll('span', {'class':'speedometerSignal-DPgs-R4s sellColor-DPgs-R4s'})

for span in spans:
    print(span.text)

在撰写本文时，它打印：

Продавать
Активно продавать

如何解决'POST data should be bytes, an iterable of bytes, or a file object. It cannot be of type str.'？

How to slove 'POST data should be bytes, an iterable of bytes, or a file object. It cannot be of type str.'?

python

urllib

beautifulsoup