尝试使用 AmazonElasticMapReduce 运行 作业流时出错

Question

我正在尝试 运行 使用 AmazonElasticMapReduce 的作业流程，但我收到以下错误：

Exception in thread "main" com.amazonaws.services.elasticmapreduce.model.AmazonElasticMapReduceException:
The supplied ami version is invalid.
(Service: AmazonElasticMapReduce; Status Code: 400; Error Code: ValidationException; Request ID:
9e5d5d67-cc98-4002-989e-a0df55517d62)

这是我的代码：

 public static void main(String[] args) throws Exception {
        AmazonElasticMapReduce mapReduce = new AmazonElasticMapReduceClient();
        HadoopJarStepConfig hadoopJarStep = new HadoopJarStepConfig()
                .withJar("s3n://bucket/jar")
                .withMainClass("some.pack.MainClass")
                .withArgs("s3n://bucket/input/", "s3n://bucket/output/");

        StepConfig stepConfig = new StepConfig()
                .withName("stepname")
                .withHadoopJarStep(hadoopJarStep)
                .withActionOnFailure("TERMINATE_JOB_FLOW");

        JobFlowInstancesConfig instances = new JobFlowInstancesConfig()
                .withInstanceCount(2)
                .withMasterInstanceType(InstanceType.M1_SMALL.toString())
                .withSlaveInstanceType(InstanceType.M1_SMALL.toString())
                .withHadoopVersion("2.6.0").withEc2KeyName("mykey")
                .withKeepJobFlowAliveWhenNoSteps(false)
                .withPlacement(new PlacementType("us-east-1a"));

        RunJobFlowRequest runFlowRequest = new RunJobFlowRequest()
                .withName("jobname")
                .withInstances(instances)
                .withSteps(stepConfig)
                .withLogUri("s3n://bucket/logs/");
        runFlowRequest.setServiceRole("EMR_DefaultRole");
        runFlowRequest.setJobFlowRole("EMR_EC2_DefaultRole");

        RunJobFlowResult runJobFlowResult = mapReduce.runJobFlow(runFlowRequest);
        String jobFlowId = runJobFlowResult.getJobFlowId();
        System.out.println("Ran job flow with id: " + jobFlowId);
    }

我尝试使用不同的 ami 版本将 .withAmiVersion() 添加到 RunJobFlowRequest，但这没有帮助。

Answer 1

我通过将 .withReleaseLabel("emr-4.0.0") 添加到 RunJobFlowRequest 来解决它。 我在这里找到了与我的 Hadoop 版本兼容的 emr 发布版本： https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-release-components.html