AWS S3 事件通知是否保证送达?
Are AWS S3 Event Notifications guaranteed to be delivered?
我正在将 AWS S3 事件通知连接到 AWS Lambda 处理管道。
我找不到有关 S3 事件通知保证的文档。我可以确定所有 S3 事件都会触发 lambda 进程吗?命令与我无关,只是我的 lambda 最终被解雇了。
如果不存在这样的保证,架构师如何处理?
2020 年更新
事件现在至少一次:https://docs.aws.amazon.com/AmazonS3/latest/dev/NotificationHowTo.html
该页面的相关引用:
Amazon S3 event notifications are designed to be delivered at least once. Typically, event notifications are delivered in seconds but can sometimes take a minute or longer.
原回答
原则上可以。但是,Lambda 有 99.9% 的 SLA,S3 也有 99.9% 的正常运行时间 SLA。因此,理论上某些事件可能会被错过,但只有在服务中断时才会发生。当 Lambda 函数失败时,它会自动重试最多 3 次。
我将 S3 用作 Lambda 的 'queue',并且从未错过任何一条消息(数以百万计)。
我认为您无能为力。部分使用 'serverless' 基础设施意味着您也会失去一些控制权(但是,我认为 AWS 可以比我自己和一些专用服务器做得更好)。
From the Documentation
重要
Amazon S3 事件通知通常在几秒钟内传送事件,但有时可能需要一分钟或更长时间。 在极少数情况下,事件可能会丢失。
我们现在看到丢失的事件。数量很多,但百分比低。
AWS 仅通过文档而非产品将语义从最多一次升级到至少一次,这非常有趣。
在 2020 年之前,根据这个 link:
Amazon S3 event notifications typically deliver events in seconds but
can sometimes take a minute or longer. On very rare occasions, events
might be lost. If your application requires particular semantics (for
example, ensuring that no events are missed, or that operations run
only once), we recommend that you account for missed and duplicate
events when designing your application. You can audit for missed
events by using the LIST Objects API or Amazon S3 Inventory reports.
The LIST Objects API and Amazon S3 inventory reports are subject to
eventual consistency and might not reflect recently added or deleted
objects.
2020年改为至少一次,如下:
Amazon S3 event notifications are designed to be delivered at least once. Typically, event notifications are delivered in seconds but can sometimes take a minute or longer.
2021 年 11 月,它再次降级为 仅一次,据此 PR。
By default, Amazon S3 event notifications are delivered only once. Most of the time, event notifications are delivered in seconds. However, sometimes it can take a minute or longer for them to be delivered.
2021 年 12 月,我们指出 AWS 进行了向后不兼容的更改,AWS 解释说这是文档更改的事件,但我不相信。于是AWS很快将文档改回如下:
Amazon S3 event notifications are designed to be delivered at least once. Typically, event notifications are delivered in seconds but can sometimes take a minute or longer
但是 AWS support 也告诉我们,它被设计为至少一次,但不保证任何事情。
所以这不仅令人困惑,而且具有误导性。
我正在将 AWS S3 事件通知连接到 AWS Lambda 处理管道。
我找不到有关 S3 事件通知保证的文档。我可以确定所有 S3 事件都会触发 lambda 进程吗?命令与我无关,只是我的 lambda 最终被解雇了。
如果不存在这样的保证,架构师如何处理?
2020 年更新
事件现在至少一次:https://docs.aws.amazon.com/AmazonS3/latest/dev/NotificationHowTo.html
该页面的相关引用:
Amazon S3 event notifications are designed to be delivered at least once. Typically, event notifications are delivered in seconds but can sometimes take a minute or longer.
原回答
原则上可以。但是,Lambda 有 99.9% 的 SLA,S3 也有 99.9% 的正常运行时间 SLA。因此,理论上某些事件可能会被错过,但只有在服务中断时才会发生。当 Lambda 函数失败时,它会自动重试最多 3 次。
我将 S3 用作 Lambda 的 'queue',并且从未错过任何一条消息(数以百万计)。
我认为您无能为力。部分使用 'serverless' 基础设施意味着您也会失去一些控制权(但是,我认为 AWS 可以比我自己和一些专用服务器做得更好)。
From the Documentation 重要 Amazon S3 事件通知通常在几秒钟内传送事件,但有时可能需要一分钟或更长时间。 在极少数情况下,事件可能会丢失。
我们现在看到丢失的事件。数量很多,但百分比低。
AWS 仅通过文档而非产品将语义从最多一次升级到至少一次,这非常有趣。
在 2020 年之前,根据这个 link:
Amazon S3 event notifications typically deliver events in seconds but can sometimes take a minute or longer. On very rare occasions, events might be lost. If your application requires particular semantics (for example, ensuring that no events are missed, or that operations run only once), we recommend that you account for missed and duplicate events when designing your application. You can audit for missed events by using the LIST Objects API or Amazon S3 Inventory reports. The LIST Objects API and Amazon S3 inventory reports are subject to eventual consistency and might not reflect recently added or deleted objects.
2020年改为至少一次,如下:
Amazon S3 event notifications are designed to be delivered at least once. Typically, event notifications are delivered in seconds but can sometimes take a minute or longer.
2021 年 11 月,它再次降级为 仅一次,据此 PR。
By default, Amazon S3 event notifications are delivered only once. Most of the time, event notifications are delivered in seconds. However, sometimes it can take a minute or longer for them to be delivered.
2021 年 12 月,我们指出 AWS 进行了向后不兼容的更改,AWS 解释说这是文档更改的事件,但我不相信。于是AWS很快将文档改回如下:
Amazon S3 event notifications are designed to be delivered at least once. Typically, event notifications are delivered in seconds but can sometimes take a minute or longer
但是 AWS support 也告诉我们,它被设计为至少一次,但不保证任何事情。
所以这不仅令人困惑,而且具有误导性。