如何从 greenlet 调用异步方法(编剧)
How to call async method from greenlet (playwright)
我的框架(Locust,https://github.com/locustio/locust) is based on gevent and greenlets. But I would like to leverage Playwright (https://playwright.dev/python/),它是基于 asyncio 构建的。
天真地使用 Playwrights 同步 api 不起作用并给出异常:
playwright._impl._api_types.Error: It looks like you are using Playwright Sync API inside the asyncio loop.
Please use the Async API instead.
我正在寻找有关如何将异步与 gevent 结合使用的最佳实践。
我尝试了几种不同的方法,但我不知道我是否接近或者我尝试做的事情是否可行(我对 gevent 有一些经验,但之前没有真正使用过 asyncio)
编辑:我现在有点工作了(我删除了 Locust 并直接生成了一些 greenlets 以使其更容易理解)。这已经很好了,还是有更好的解决方案?
import asyncio
import threading
from playwright.async_api import async_playwright
import gevent
def thr(i):
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
loop.run_until_complete(do_stuff(i))
loop.close()
async def do_stuff(i):
playwright = await async_playwright().start()
browser = await playwright.chromium.launch(headless=False)
page = await browser.new_page()
await page.wait_for_timeout(5000)
await page.goto(f"https://google.com")
await page.close()
print(i)
def green(i):
t = threading.Thread(target=thr, args=(i,))
t.start()
# t.join() # joining doesnt work, but I couldnt be bothered right now :)
g1 = gevent.spawn(green, 1)
g2 = gevent.spawn(green, 2)
g1.join()
g2.join()
受@user4815162342 的评论启发,我做了这样的事情:
from playwright.async_api import async_playwright # need to import this first
from gevent import monkey, spawn
import asyncio
import gevent
monkey.patch_all()
loop = asyncio.new_event_loop()
async def f():
print("start")
playwright = await async_playwright().start()
browser = await playwright.chromium.launch(headless=True)
context = await browser.new_context()
page = await context.new_page()
await page.goto(f"https://www.google.com")
print("done")
def greeny():
while True: # and not other_exit_condition
future = asyncio.run_coroutine_threadsafe(f(), loop)
while not future.done():
gevent.sleep(1)
greenlet1 = spawn(greeny)
greenlet2 = spawn(greeny)
loop.run_forever()
实际实施将在 Locust 某天结束,可能经过一些优化(重用浏览器实例等)
我的框架(Locust,https://github.com/locustio/locust) is based on gevent and greenlets. But I would like to leverage Playwright (https://playwright.dev/python/),它是基于 asyncio 构建的。
天真地使用 Playwrights 同步 api 不起作用并给出异常:
playwright._impl._api_types.Error: It looks like you are using Playwright Sync API inside the asyncio loop.
Please use the Async API instead.
我正在寻找有关如何将异步与 gevent 结合使用的最佳实践。
我尝试了几种不同的方法,但我不知道我是否接近或者我尝试做的事情是否可行(我对 gevent 有一些经验,但之前没有真正使用过 asyncio)
编辑:我现在有点工作了(我删除了 Locust 并直接生成了一些 greenlets 以使其更容易理解)。这已经很好了,还是有更好的解决方案?
import asyncio
import threading
from playwright.async_api import async_playwright
import gevent
def thr(i):
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
loop.run_until_complete(do_stuff(i))
loop.close()
async def do_stuff(i):
playwright = await async_playwright().start()
browser = await playwright.chromium.launch(headless=False)
page = await browser.new_page()
await page.wait_for_timeout(5000)
await page.goto(f"https://google.com")
await page.close()
print(i)
def green(i):
t = threading.Thread(target=thr, args=(i,))
t.start()
# t.join() # joining doesnt work, but I couldnt be bothered right now :)
g1 = gevent.spawn(green, 1)
g2 = gevent.spawn(green, 2)
g1.join()
g2.join()
受@user4815162342 的评论启发,我做了这样的事情:
from playwright.async_api import async_playwright # need to import this first
from gevent import monkey, spawn
import asyncio
import gevent
monkey.patch_all()
loop = asyncio.new_event_loop()
async def f():
print("start")
playwright = await async_playwright().start()
browser = await playwright.chromium.launch(headless=True)
context = await browser.new_context()
page = await context.new_page()
await page.goto(f"https://www.google.com")
print("done")
def greeny():
while True: # and not other_exit_condition
future = asyncio.run_coroutine_threadsafe(f(), loop)
while not future.done():
gevent.sleep(1)
greenlet1 = spawn(greeny)
greenlet2 = spawn(greeny)
loop.run_forever()
实际实施将在 Locust 某天结束,可能经过一些优化(重用浏览器实例等)