解决 AWS CloudFormation 中的循环依赖

Question

下面的AWS CloudFormation给出了一个循环依赖错误。我的理解是依赖关系是这样流动的：rawUploads -> generatePreview -> previewPipeline -> rawUploads。虽然看起来 rawUploads 并不依赖于 generatePreview，但我猜想 CF 在创建存储桶时需要知道要触发什么 lambda，即使触发器是在 CloudFormation 模板的 lambda 部分定义的。

我在网上找到了一些讨论类似问题的资源，但它似乎不适用于此处。 https://aws.amazon.com/premiumsupport/knowledge-center/unable-validate-circular-dependency-cloudformation/

要打破这个循环依赖链，我有哪些选择？可编写脚本的解决方案是可行的，但手动更改的多个部署不适合我的用例。

AWSTemplateFormatVersion: '2010-09-09'
Transform: 'AWS::Serverless-2016-10-31'
Resources:
  rawUploads:
    Type: 'AWS::S3::Bucket'
  previewAudioFiles:
    Type: 'AWS::S3::Bucket'

  generatePreview:
    Type: AWS::Serverless::Function
    Properties:
      Handler: generatePreview.handler
      Runtime: nodejs6.10
      CodeUri: .
      Environment:
        Variables:
          PipelineId: !Ref previewPipeline
      Events:
        BucketrawUploads:
          Type: S3
          Properties:
            Bucket: !Ref rawUploads
            Events: 's3:ObjectCreated:*'

  previewPipeline:
    Type: Custom::ElasticTranscoderPipeline
    Version: '1.0'
    Properties:
      ServiceToken:
        Fn::Join:
        - ":"
        - - arn:aws:lambda
          - Ref: AWS::Region
          - Ref: AWS::AccountId
          - function
          - aws-cloudformation-elastic-transcoder-pipeline-1-0-0
      Name: transcoderPipeline
      InputBucket:
        Ref: rawUploads
      OutputBucket:
        Ref: previewAudioFiles

Answer 1

一种方法是为 S3 存储桶指定明确的名称，这样以后您就可以简单地使用存储桶名称，而不是依赖 Ref: bucketname。如果您想要自动生成存储桶名称，这显然是有问题的，在这种情况下，谨慎的做法是从某个前缀加上（唯一的）堆栈名称生成存储桶名称，例如：

InputBucket: !Join ["-", ['rawuploads', Ref: 'AWS::StackName']]

另一种选择是使用单个 CloudFormation 模板，但分 2 个阶段 - 第一个阶段创建基础资源（以及任何非循环引用），然后将剩余的引用添加到模板并进行堆栈更新。显然不理想，所以我更喜欢第一种方法。

您也可以在需要引用 ARN 的情况下使用第一种技术，例如：

!Join ['/', ['arn:aws:s3:::logsbucket', 'AWSLogs', Ref: 'AWS:AccountId', '*']]

使用此技术时，您可能还想考虑使用 DependsOn，因为您已经删除了有时会导致问题的隐式依赖项。

Answer 2

这个post最终帮助了我：https://aws.amazon.com/premiumsupport/knowledge-center/unable-validate-destination-s3/

我最终在 CloudFormation 中配置了一个 SNS 主题。存储桶会推送关于该主题的事件，而 Lambda 函数会监听该主题。这样依赖关系图如下：

S3 bucket -> SNS topic -> SNS topic policy
Lambda function -> SNS topic
Lambda function -> transcoder pipeline

类似的东西（省略了一些政策）

AWSTemplateFormatVersion: '2010-09-09'
Transform: 'AWS::Serverless-2016-10-31'
Resources:
  SNSTopic:
    Type: AWS::SNS::Topic
  SNSTopicPolicy:
    Type: AWS::SNS::TopicPolicy
    Properties:
      PolicyDocument:
        Id: MyTopicPolicy
        Version: '2012-10-17'
        Statement:
        - Sid: Statement-id
          Effect: Allow
          Principal:
            AWS: "*"
          Action: sns:Publish
          Resource:
            Ref: SNSTopic
          Condition:
            ArnLike:
              aws:SourceArn:
                !Join ["-", ['arn:aws:s3:::rawuploads', Ref: 'AWS::StackName']]
      Topics:
      - Ref: SNSTopic

  rawUploads:
    Type: 'AWS::S3::Bucket'
    DependsOn: SNSTopicPolicy
    Properties:
      BucketName: !Join ["-", ['rawuploads', Ref: 'AWS::StackName']]
      NotificationConfiguration:
        TopicConfigurations:
          - Topic:
              Ref: "SNSTopic"
            Event: 's3:ObjectCreated:*'

  previewAudioFiles:
    Type: 'AWS::S3::Bucket'


  generatePreview:
    Type: AWS::Serverless::Function
    Properties:
      FunctionName: !Join ["-", ['generatepreview', Ref: 'AWS::StackName']]
      Handler: generatePreview.handler
      Runtime: nodejs6.10
      CodeUri: .
      Environment:
        Variables:
          PipelineId: !Ref previewPipeline
      Events:
        BucketrawUploads:
          Type: SNS
          Properties:
            Topic: !Ref "SNSTopic"

  previewPipeline:
    Type: Custom::ElasticTranscoderPipeline
    DependsOn: 'rawUploads'
    Version: '1.0'
    Properties:
      ServiceToken:
        Fn::Join:
        - ":"
        - - arn:aws:lambda
          - Ref: AWS::Region
          - Ref: AWS::AccountId
          - function
          - aws-cloudformation-elastic-transcoder-pipeline-1-0-0
      Name: transcoderPipeline
      InputBucket:
        !Join ["-", ['arn:aws:s3:::rawuploads', Ref: 'AWS::StackName']]
      OutputBucket:
        Ref: previewAudioFiles

解决 AWS CloudFormation 中的循环依赖

Work around circular dependency in AWS CloudFormation

amazon-web-services

amazon-cloudformation

aws-lambda

serverless-framework

serverless