multiprocessing.Process 不符合我的要求
multiprocessing.Process doesn't work as I want
我写了一个程序,我想 运行 一个带有 multiprocessing.Process 的函数,但是我没有得到结果。
import sys, urllib
from urllib.request import urlopen, Request
def disallow(line):
if "Disallow" in line:
print(line.replace("Disallow: ", ""))
def parse_robot_txt(url):
url += "robots.txt"
req = Request(url, data=None,
headers={"User-Agent": "Mozilla/5.0"},)
robots = urlopen(req).read().decode("UTF-8")
lines = str(robots).splitlines()
for l in lines:
multiprocessing.Process(target=disallow, args=(l,))
if __name__ == "__main__":
parse_robot_txt("https://moz.com/")
输出:
我没有得到结果,
但是如果我 运行 没有多处理:
import sys, urllib
from urllib.request import urlopen, Request
def disallow(line):
if "Disallow" in line:
print(line.replace("Disallow: ", ""))
def parse_robot_txt(url):
url += "robots.txt"
req = Request(url, data=None,
headers={"User-Agent": "Mozilla/5.0"},)
robots = urlopen(req).read().decode("UTF-8")
lines = str(robots).splitlines()
for l in lines:
disallow(l)
if __name__ == "__main__":
parse_robot_txt("https://moz.com/")
输出:
/products/content/
/local/enterprise/confirm
/researchtools/ose/
/page-strength/*
/thumbs/*
/api/user?*
/checkout/freetrial/*
/local/search/
/local/details/
/messages/
/content/audit/*
/content/search/*
/marketplace/
/cpresources/
/vendor/
/community/q/questions/*/view_counts
/admin-preview/*
那么,多处理的问题是什么?我怎样才能使用它来获得相同的结果?
您需要启动流程,只需将其更改为:
for l in lines:
proc = multiprocessing.Process(target=disallow, args=(l,))
proc.start()
我写了一个程序,我想 运行 一个带有 multiprocessing.Process 的函数,但是我没有得到结果。
import sys, urllib
from urllib.request import urlopen, Request
def disallow(line):
if "Disallow" in line:
print(line.replace("Disallow: ", ""))
def parse_robot_txt(url):
url += "robots.txt"
req = Request(url, data=None,
headers={"User-Agent": "Mozilla/5.0"},)
robots = urlopen(req).read().decode("UTF-8")
lines = str(robots).splitlines()
for l in lines:
multiprocessing.Process(target=disallow, args=(l,))
if __name__ == "__main__":
parse_robot_txt("https://moz.com/")
输出:
我没有得到结果, 但是如果我 运行 没有多处理:
import sys, urllib
from urllib.request import urlopen, Request
def disallow(line):
if "Disallow" in line:
print(line.replace("Disallow: ", ""))
def parse_robot_txt(url):
url += "robots.txt"
req = Request(url, data=None,
headers={"User-Agent": "Mozilla/5.0"},)
robots = urlopen(req).read().decode("UTF-8")
lines = str(robots).splitlines()
for l in lines:
disallow(l)
if __name__ == "__main__":
parse_robot_txt("https://moz.com/")
输出:
/products/content/
/local/enterprise/confirm
/researchtools/ose/
/page-strength/*
/thumbs/*
/api/user?*
/checkout/freetrial/*
/local/search/
/local/details/
/messages/
/content/audit/*
/content/search/*
/marketplace/
/cpresources/
/vendor/
/community/q/questions/*/view_counts
/admin-preview/*
那么,多处理的问题是什么?我怎样才能使用它来获得相同的结果?
您需要启动流程,只需将其更改为:
for l in lines:
proc = multiprocessing.Process(target=disallow, args=(l,))
proc.start()