使用 python boto 仅下载 S3 存储桶中的特定文件夹

Question

下面的 link 显示了如何下载整个 S3 内容。但是，如何获取子文件夹内容。假设我的 S3 文件夹具有以下模拟结构。

S3Folder/S1/file1.c

S3Folder/S1/file2.h

S3Folder/S1/file1.h

S3Folder/S2/file.exe

S3Folder/S2/resource.数据

假设我只对 S2 文件夹感兴趣。如何隔离存储桶列表中的键？

local backup of an S3 content

conn = boto.connect_s3(AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY)
bucket = conn.get_bucket(bucket_name)

# go through the list of files
bucket_list = bucket.list()
for l in bucket_list:
  keyString = str(l.key)
  d = LOCAL_PATH + keyString
  try:
    l.get_contents_to_filename(d)
  except OSError:
    # check if dir exists
    if not os.path.exists(d):
      os.mkdir(d)

Answer 1

您可以通过在键值中添加它的前缀来下载s3对象。

所以，根据你的问题，你只需要在下载对象时添加前缀'/S2'

仅供参考：s3 download object 使用 boto3

更多检查

Answer 2

您可以执行以下操作：

import os
import boto3
s3_resource = boto3.resource("s3", region_name="us-east-1")

    def download_objects():
        root_dir = 'D:/' # local machine location
        s3_bucket_name = 'S3_Bucket_Name' #s3 bucket name
        s3_root_folder_prefix = 'sample' # bucket inside root folder
        s3_folder_list = ['s3_folder_1','s3_folder_2','s3_folder_3'] # root folder inside sub folders list

        my_bucket = self.s3_resource.Bucket(s3_bucket_name)
        for file in my_bucket.objects.filter(Prefix=s3_root_folder_prefix):
            if any(s in file.key for s in s3_folder_list):
                try:
                    path, filename = os.path.split(file.key)
                    try:
                        os.makedirs(root_dir + path)
                    except Exception as err:
                        pass
                    my_bucket.download_file(file.key, root_dir + path + '/' + filename)
                except Exception as err:
                    print(err)

if __name__ == '__main__':
    download_objects()

使用 python boto 仅下载 S3 存储桶中的特定文件夹

Download only specific folder in S3 bucket using python boto

amazon-s3

boto

amazon-web-services