我想在异步函数中使用 boto3，python

Question

我在playwright开发web-scraper，想异步上传图片到aws-s3。

但 boto3 不是异步函数..如何解决？


class Boto3:
   def __init__(self, key, id):
       self.S3 = boto3.client('s3', aws_access_key_id=aws_key_id, aws_secret_access_key=aws_secret)

   def upload_stream(self, stream, bucket_name, key):
       self.S3.put_object(Body=stream, Bucket=bucket_name, Key=key)

...
...

class Scraper:
   def __init__(self, key, id):
      self.S3 = boto3.client('s3', aws_access_key_id=id, aws_secret_access_key=key)

   asnyc _save_image(res):
      buffer = await res.body()
      # S3.put_object is not an async function.
      self.S3.put_object(
            Body=buffer, 
            Bucket=bucket_name, 
            Key=bucket_key,
      )

   async def scrape():
      playwright = await async_playwright().start()
      browser = await playwright.chromium.launch( headless = True, devtools = False )
      page = browser.new_page()

      page.on('response', _save_image)
      await page.goto('https://www.example.com')

scraper = Scraper(key, id)
asyncio.run(scraper.scrape())

self.S3.put_object ：这个函数不是 asnyc 所以想在异步版本中改变它。如何解决？

提前致谢。

Answer 1

How to fix it?

你不能，因为 boto3 不是异步的。充其量您可以尝试使用第三方 non-AWS 库，例如 aioboto3 代替 boto3。

我想在异步函数中使用 boto3，python

I want to use boto3 in async function, python

python

amazon-s3

amazon-web-services

python-asyncio

boto3