AWS Lambda NodeJS s3.DeleteObjects 有时会调用两次

Question

我想使用 Lambda 函数 (NodeJS) 删除特定存储桶中最旧的文件。我正在开发该功能并且仍在测试（因此，目前没有触发器）。

每件事都适用于所有承诺，但函数 s3.deleteObjects 有时会为单个对象调用两次。例如，该函数为对象“folder-A”调用两次，为对象“Folder-B”调用一次。如果我执行一项新测试，也许该函数会被调用一次（如我所料）。这似乎是一种随机行为。我没有错误。

这是我的代码：

// Get folder's last modified date
   const allFolders = ['object-1', 'object-2', 'object-3'];

   const promises = allFolders.map(folderName => new Promise((resolve, reject) => {
       
       s3.getObject({ Bucket: bucketName, Key: folderName }, function(err, data) { 
           
           if(err){
               console.log('** ERRORE **', bucketName, folderName, err);
               return reject(err);
           }else{
               resolve({Key: folderName, LastModified: data.LastModified});
           }
       });
       
    }));
   
   Promise.all(promises).then(foldersData => {
         
      'here I sort the objects by LastModified... and the result is var foldersToDelete'
       
       const foldersToDelete = ['object-2', 'object-3'];
     
       //delete objects
       // I call listAndDeleteObjects from an external file called functions.js
       functions.listAndDeleteObjects(bucketName, foldersToDelete);
    });

function.js这里

function listAndDeleteObjects (bucketName, foldersToDelete){
    if (foldersToDelete.length === 0){
        console.log('No objects found: exit');
        return;
    }
    
    //list and delete all objects inside every folder
    foldersToDelete.forEach( async (folderName, index) => {
            //get all folder's objects
            // listObjects returns all objects inside path folderName. In my example it returns only 'object-2' and 'object-3'
            const folderObjects = (await listObjects(bucketName, folderName)).Contents.map( item => {
                return {Key: item.Key}
            });
            
            //delete folder's objects
            await mydeleteObjects(bucketName, folderObjects);
        }
    );
}

async function mydeleteObjects (bucketName, folderObjects) {
    const deleteParams = {
        Bucket: bucketName,
        Delete: {
            Objects: folderObjects
        }
    }
    
    // **HERE THE PROBLEM**. s3.deleteObjects sometimes is called twice!
    // for example 3.deleteObjects will "delete" object-2 two times and object-3 one time.
    // if I execute a test again (after recreating object in s3 bucket), maybe it will delete object-3 twice and object-2 one, or object-3 and 2 one time...

    await s3.deleteObjects(deleteParams, function(err,data){

        if (deleteParams.Delete.Objects.length === 0){
            console.log('No objects to delete found: exit');
            return;
        }
        
        if (err)    console.log(err,err.stack);
        else{
            console.log("End deleting objects:",data);
        }
    }).promise();
}

谁能解释一下为什么？谢谢

Answer 1

我找到了解决方案。我以错误的方式使用 'async' 和 'await'，而不是在 s3.getObject 中使用回调函数，现在我使用 .then .

这里是代码：

try{
    const listBucketParams = {
        Bucket: bucketName,
        Prefix: prefix,
        Delimiter: '/'
    };

    // Get all bucket folders
    allFolders = (await s3.listObjectsV2(listBucketParams).promise()).CommonPrefixes.map(item => item.Prefix);

    const promises = allFolders.map(folderName => new Promise( (resolve, reject) => {

        //get folder's LastModified date
        s3.getObject({ Bucket: bucketName, Key: folderName })
            .promise()
            .then(response => {
                resolve({Key: folderName, LastModified: response.LastModified});
            })
            .catch(async e => {

                if(e.code === 'NoSuchKey'){
                  //this function retrieves one object from the folderName s3 folder 
                  functions.listFolderObject(bucketName, folderName, resolve);
                }
            });
     }));

    await Promise.all(promises).then(async foldersData => {

        // sort folders by date desc
        foldersData.sort((a,b) => (a.LastModified < b.LastModified) ? 1 : ((b.LastModified < a.LastModified) ? -1 : 0));

        // get folders to delete
        const foldersToDelete = functions.getfoldersToDelete(foldersData, objectsToPreserve, isDevEnv);


        //delete objects
        await functions.listAndDeleteObjects(bucketName, foldersToDelete);
     });
} catch (e){
}

基本上我遵循了这个：https://docs.aws.amazon.com/sdk-for-javascript/v3/developer-guide/using-promises.html#multiple-promises

AWS Lambda NodeJS s3.DeleteObjects 有时会调用两次

AWS Lambda NodeJS s3.DeleteObjects sometimes called twice

amazon-s3

node.js

es6-promise

aws-lambda

aws-sdk-js