AWS Cloud Formation error: ElasticMapReduce Cluster failed to stabilize

Question

尽管我的研究告诉我这是亚马逊内部错误，但我一直收到此错误。我不知道从哪里开始这个错误，或者我什至可以做些什么来帮助它。

我一直得到它的事实让我觉得我的脚本有问题。这是：

{
  "Description": "Demo pipeline.",
  "Resources": {
    "s3Demo": {
        "Type" : "AWS::S3::Bucket",
        "Properties" : {
            "BucketName" : "example-dna-demo"
        }
    },

    "s3Access": {
        "Type": "AWS::IAM::Role",
        "Properties": {
            "ManagedPolicyArns": [
                "arn:aws:iam::aws:policy/AmazonS3FullAccess"
            ],
            "AssumeRolePolicyDocument": {
                "Version": "2012-10-17",
                "Statement": [{
                    "Effect": "Allow",
                    "Action": "sts:AssumeRole",
                    "Principal":{
                        "Service": "firehose.amazonaws.com"
                    }
                }]
            }, 
            "RoleName": "kinesisS3Access"
        },
        "DependsOn": "s3Demo"
    },

    "kinesisDemo": {
        "Type": "AWS::KinesisFirehose::DeliveryStream",
        "Properties": {
            "DeliveryStreamName": "Demo-Stream",
            "S3DestinationConfiguration": {
                "BucketARN" : "arn:aws:s3:::example-dna-demo",
                "BufferingHints" : {
                    "IntervalInSeconds" : 300,
                    "SizeInMBs" : 5
                },
                "CompressionFormat" : "UNCOMPRESSED",
                "Prefix" : "twitter",
                "RoleARN" : { "Fn::GetAtt": [ "s3Access", "Arn" ]}
            }
        },
        "DependsOn": "s3Access"
    },

    "S3LambdaAccess":{
        "Type": "AWS::IAM::Role",
        "Properties": {
            "ManagedPolicyArns": [
                "arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess"
            ],
            "AssumeRolePolicyDocument": {
                "Version": "2012-10-17",
                "Statement": [{
                    "Effect": "Allow",
                    "Action": "sts:AssumeRole",
                    "Principal":{
                        "Service": "lambda.amazonaws.com"
                    }
                }]
            }, 
            "RoleName": "lambdaS3Access"
        }
    },
    "LambdaDemo": {
        "Type" : "AWS::Lambda::Function",
        "Properties" : {
            "Code" : {
                "S3Bucket" : "example-dna-cloud-formation",
                "S3Key" : "lambda_function.py.zip"
            },
            "Description" : "Looks for S3 writes and loads them into another resource",
            "FunctionName" : "DemoLambdaFunction",
            "Handler" : "lambda-handler",
            "Role" : { "Fn::GetAtt": [ "S3LambdaAccess", "Arn" ]},
            "Runtime" : "python2.7"
        },
        "DependsOn": "S3LambdaAccess"
    },
    "EMRClusterJobFlowRole": {
        "Type": "AWS::IAM::Role",
        "Properties": {
            "AssumeRolePolicyDocument": {  
                "Version": "2012-10-17",
                "Statement": [{
                    "Effect": "Allow",
                    "Action": "sts:AssumeRole",
                    "Principal":{
                        "Service": "ec2.amazonaws.com"
                    }
                }] 
            },
            "RoleName": "ClusterRole"
        }
    },
    "EMRServiceRole": {
        "Type": "AWS::IAM::Role",
        "Properties": {
            "AssumeRolePolicyDocument": { 
              "Version": "2012-10-17",
                "Statement": [{
                    "Effect": "Allow",
                    "Action": "sts:AssumeRole",
                    "Principal":{
                        "Service": "ec2.amazonaws.com"
                    }
                }]
            },
            "RoleName": "EC2InstanceRole"
        }
    },
    "EMR":{
        "Type" : "AWS::EMR::Cluster",
        "Properties" : {
            "Applications": [
                {
                    "Name" : "Spark"
                }
            ],
            "ReleaseLabel": "emr-5.0.0",
            "Instances" : {
                "CoreInstanceGroup" : {
                    "BidPrice": 0.06,
                    "InstanceCount" : 1,
                    "InstanceType" : "m4.large",
                    "Market": "SPOT"
                    },
                "MasterInstanceGroup" : {
                    "BidPrice": 0.06,
                    "InstanceCount" : 1,
                    "InstanceType" : "m4.large",
                    "Market": "SPOT"
                    }
                },
            "JobFlowRole" : "EMRClusterJobFlowRole",
            "Name" : "DemoEMR",
            "ServiceRole" : "EMRServiceRole",
            "LogUri":"s3://toyota-dna-cloud-formation/cf-logging"
        },
        "DependsOn": ["EMRServiceRole", "EMRServiceRole"]
    }
  }
}

我想您可能无法运行因为我有一个从 S3 存储桶获取代码的 lambda 函数，我在这里更改了它的名称。我只是在学习云形成脚本，我知道有很多东西我不会在这里做，但我只是想构建一个可以工作的小东西，然后再填充一点。

我知道我的脚本一直运行到两个 IAM 角色和 EMR 集群。提前致谢。

编辑：我指定了最近的实例版本并选择了 ReleaseLabel 属性。没有运气。同样的错误。

Answer 1

可能是您的帐户已达到您尝试部署到的区域的 EC2 限制。您尝试过其他地区吗？

Answer 2

所以事实证明，在我运行脚本所在的区域中没有默认 VPC，这就是我的 EMR 集群无法稳定的原因。

当我在另一个区域尝试运行它时，它起作用了，但是因为那个区域确实有一个默认的 VPC。

Answer 3

在我的例子中，这是由于缺少自动缩放角色，称为 EMR_AutoScaling_DefaultRole。

一旦我通过 aws emr create-default-roles 将其部署到位，我的 cloudformation 堆栈再次开始很好地部署（在我添加自动缩放内容之前部署正常）。

AWS Cloud Formation error: ElasticMapReduce Cluster failed to stabilize

AWS Cloud Formation error: ElasticMapReduce Cluster failed to stabilize

amazon-web-services

amazon-cloudformation