CDK 部署结果 "Bucket named 'x' exists, but not in account 067685711111. Wrong account?"
CDK deploy results in "Bucket named 'x' exists, but not in account 067685711111. Wrong account?"
我正在修改一个有效的 CDK 管道。我正在添加一个 lambda，当进行此代码修改时，管道失败并出现错误。
import * as cdk from '@aws-cdk/core';
import * as lambda from '@aws-cdk/aws-lambda';
import * as path from 'path';
import { Rule, Schedule } from '@aws-cdk/aws-events';
import { LambdaFunction } from '@aws-cdk/aws-events-targets';
import { Secret } from '@aws-cdk/aws-secretsmanager';
import { Effect, Policy, PolicyStatement } from '@aws-cdk/aws-iam';
interface LambdaStackProps extends cdk.StackProps {
logzioToken: string;
}
export class Lambda extends cdk.Stack {
constructor(scope: cdk.Construct, id: string, props: LambdaStackProps) {
super(scope, id, props);
const logzioSecret = Secret.fromSecretAttributes(this, 'sv-logzioToken', {
secretPartialArn: props.logzioToken,
});
// Here is the func itself
const func = new lambda.Function(this, 'sv-http-cache-clear-lambda', {
runtime: lambda.Runtime.NODEJS_14_X,
handler: 'index.handler',
// The "build" folder requires `yarn build` to have been run in the lambda folder
code: lambda.Code.fromAsset(path.join(__dirname, '../../lambdas/HttpCacheClear/build')),
environment: {
LOGZIO_TOKEN: logzioSecret.secretValueFromJson('logzioToken').toString(),
},
});
// Here are the Dynamo permissions I think we need
const policyStatement = new PolicyStatement({
actions: ['dynamodb:GetItem', 'dynamodb:Query', 'dynamodb:Scan', 'dynamodb:DeleteItem'],
effect: Effect.ALLOW,
resources: ['*'],
});
const policy = new Policy(this, 'sv-http-cache-clear-policy', {
statements: [policyStatement],
});
func.role?.attachInlinePolicy(policy);
new Rule(this, 'sv-http-cache-clear-schedule-rule', {
description: 'Schedule a Lambda that regularly clears the cache',
schedule: Schedule.cron({
year: '*',
month: '*',
day: '*',
hour: '1',
minute: '0',
}),
targets: [new LambdaFunction(func)],
});
}
}
这是在 CDK 中从这里实例化的：
#!/usr/bin/env node
import 'source-map-support/register';
import * as cdk from '@aws-cdk/core';
import { AppStack } from '../lib/app-stack';
import { ECR } from '../lib/ecr';
import { Lambda } from '../lib/lambda';
import { getConfig } from '../lib/config';
const app = new cdk.App();
// Stateful infra
const { repository } = new ECR(app, 'sv-ecr-stack');
// Get app config
const environment = process.env.ENVIRONMENT || '';
const { certificateArn, domainName, sIdProgramId, logzioToken, optimizelyUrl } = getConfig(environment);
new AppStack(app, 'sv-stack', {
repository,
domainName,
certificateArn,
sIdProgramId,
logzioToken,
optimizelyUrl,
environment,
});
const lambda = new Lambda(app, 'sv-http-scheduled-cache-clear', {
logzioToken,
});
为了简洁起见，我没有将 AppStack 添加到此 post。
最后我有了一组相当简单的 NPM 依赖项：
{
"name": "cdk",
"version": "0.1.0",
"bin": {
"cdk": "bin/cdk.js"
},
"scripts": {
"build": "tsc",
"watch": "tsc -w",
"cdk": "cdk",
"synth": "cdk synth -q",
"deploy": "cdk deploy --all --progress=events --require-approval=never"
},
"devDependencies": {
"@aws-cdk/assert": "^1.100.0",
"@aws-cdk/aws-certificatemanager": "^1.100.0",
"@aws-cdk/aws-ec2": "^1.100.0",
"@aws-cdk/aws-ecr": "^1.100.0",
"@aws-cdk/aws-ecs": "^1.100.0",
"@aws-cdk/aws-ecs-patterns": "^1.100.0",
"@aws-cdk/aws-iam": "^1.100.0",
"@aws-cdk/aws-route53": "^1.100.0",
"@types/node": "10.17.27",
"aws-cdk": "^1.100.0",
"ts-node": "^9.0.0",
"typescript": "~3.9.7"
},
"dependencies": {
"@aws-cdk/core": "^1.100.0",
"source-map-support": "^0.5.16"
}
}
如果我在本地 运行 npm run synth 然后它 运行 s synth 命令，并成功。我的编辑器中没有 TypeScript 错误。
这是 cdk deploy 错误：
export AWS_ACCESS_KEY_ID=$DEV_ACCESS_KEY_ID
export AWS_SECRET_ACCESS_KEY=$DEV_SECRET_ACCESS_KEY
export ENVIRONMENT=dev
cd cdk
npm install
npm run deploy
npm WARN read-shrinkwrap This version of npm is compatible with lockfileVersion@1, but package-lock.json was generated for lockfileVersion@2. I'll try to do my best with it!
npm WARN cdk@0.1.0 No repository field.
npm WARN cdk@0.1.0 No license field.
added 1363 packages from 232 contributors and audited 1366 packages in 19.228s
18 packages are looking for funding
run `npm fund` for details
found 0 vulnerabilities
> cdk@0.1.0 deploy /home/circleci/project/cdk
> cdk deploy --all --progress=events --require-approval=never
sv-ecr-stack
sv-ecr-stack: deploying...
✅ sv-ecr-stack (no changes)
Outputs:
sv-ecr-stack.ExportsOutputFnGetAttsvrepositoryXXX = arn:aws:ecr:*********:067685711111:repository/student-validation-app
sv-ecr-stack.ExportsOutputRefsvrepositoryXXX = student-validation-app
Stack ARN:
arn:aws:cloudformation:*********:067685711111:stack/sv-ecr-stack/aabbccddeeffgghhiijjkk
sv-http-scheduled-cache-clear
sv-http-scheduled-cache-clear: deploying...
[0%] start: Publishing abcdefghijklmnop:current
[100%] fail: Bucket named 'cdktoolkit-stagingbucket-7k57fjeur592' exists, but not in account 067685711111. Wrong account?
❌ sv-http-scheduled-cache-clear failed: Error: Failed to publish one or more assets. See the error messages above for more information.
at Object.publishAssets (/home/circleci/project/cdk/node_modules/aws-cdk/lib/util/asset-publishing.ts:25:11)
at Object.deployStack (/home/circleci/project/cdk/node_modules/aws-cdk/lib/api/deploy-stack.ts:235:3)
at CdkToolkit.deploy (/home/circleci/project/cdk/node_modules/aws-cdk/lib/cdk-toolkit.ts:180:24)
at initCommandLine (/home/circleci/project/cdk/node_modules/aws-cdk/bin/cdk.ts:208:9)
***************************************************
*** Newer version of CDK is available [1.110.0] ***
*** Upgrade recommended ***
***************************************************
Failed to publish one or more assets. See the error messages above for more information.
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! cdk@0.1.0 deploy: `cdk deploy --all --progress=events --require-approval=never`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the cdk@0.1.0 deploy script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:
npm ERR! /home/circleci/.npm/_logs/2021-06-25T13_34_37_706Z-debug.log
Exited with code exit status 1
一些研究表明 CDK 使用 S3 创建一个“暂存桶”，我假设这就是 cdktoolkit-stagingbucket-* 是什么（它不是我明确创建的工件）。
我是否需要创建此存储桶 remotely/explicitly 或 locally/manually？
更新
我发现 this bug report 这表明错误消息很糟糕 - 它需要 S3 访问权限，但消息并没有说清楚。我已经为 CI/CD 用户找到了这个策略文件，我想知道我是否需要确定要添加什么策略以允许使用 S3。
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "StmtXXXYYYZZZ4501",
"Action": [
"cloudformation:*",
"ec2:*",
"ecs:*",
"ecr:*",
"application-autoscaling:*",
"elasticloadbalancing:*",
"iam:CreateRole",
"iam:DeleteRole",
"iam:GetRole",
"iam:TagRole",
"iam:UpdateRole",
"iam:GetRolePolicy",
"iam:DeleteRolePolicy",
"iam:PutRolePolicy",
"iam:PassRole",
"logs:*",
"route53:*",
"secretsmanager:*",
"sts:*",
"dynamodb:*",
],
"Effect": "Allow",
"Resource": "*"
}
]
}
错误消息具有误导性。如果您有权访问多个帐户，则 S3 存储桶很可能存在于另一个帐户中 - 这是因为它是一个自动生成的存储桶，CDK 使用它来打包和部署 lambda。我假设它在所有 AWS 账户中都是相同的名称。
简单的解决方案是将 "s3:*" 添加到针对 "*" 资源的允许操作。但是，我决定添加一个新的 Policy，以便资源规范更严格一些。在我的例子中，这意味着添加一个具有不同资源规范的新子句：
{
"Action": [
"s3:*"
],
"Effect": "Allow",
"Resource": "arn:aws:s3:::cdktoolkit-stagingbucket-*"
}
您似乎不必显式创建此存储桶 - CDK 会为您完成。似乎“Sid”也不是必填字段。
就我而言，添加此策略显示我还缺少多少其他权限，但至少这些错误消息更明智！
触发该错误的操作是 s3:GetBucketLocation。仅提供 CDK 所需的最小权限的策略是：
{
"Action": [
"s3:ListBucket",
"s3:GetObject",
"s3:GetBucketLocation",
"s3:PutObject",
"s3:DeleteObject"
],
"Resource": "arn:aws:s3:::cdktoolkit-stagingbucket-*",
"Effect": "Allow"
}
我正在修改一个有效的 CDK 管道。我正在添加一个 lambda，当进行此代码修改时，管道失败并出现错误。
import * as cdk from '@aws-cdk/core';
import * as lambda from '@aws-cdk/aws-lambda';
import * as path from 'path';
import { Rule, Schedule } from '@aws-cdk/aws-events';
import { LambdaFunction } from '@aws-cdk/aws-events-targets';
import { Secret } from '@aws-cdk/aws-secretsmanager';
import { Effect, Policy, PolicyStatement } from '@aws-cdk/aws-iam';
interface LambdaStackProps extends cdk.StackProps {
logzioToken: string;
}
export class Lambda extends cdk.Stack {
constructor(scope: cdk.Construct, id: string, props: LambdaStackProps) {
super(scope, id, props);
const logzioSecret = Secret.fromSecretAttributes(this, 'sv-logzioToken', {
secretPartialArn: props.logzioToken,
});
// Here is the func itself
const func = new lambda.Function(this, 'sv-http-cache-clear-lambda', {
runtime: lambda.Runtime.NODEJS_14_X,
handler: 'index.handler',
// The "build" folder requires `yarn build` to have been run in the lambda folder
code: lambda.Code.fromAsset(path.join(__dirname, '../../lambdas/HttpCacheClear/build')),
environment: {
LOGZIO_TOKEN: logzioSecret.secretValueFromJson('logzioToken').toString(),
},
});
// Here are the Dynamo permissions I think we need
const policyStatement = new PolicyStatement({
actions: ['dynamodb:GetItem', 'dynamodb:Query', 'dynamodb:Scan', 'dynamodb:DeleteItem'],
effect: Effect.ALLOW,
resources: ['*'],
});
const policy = new Policy(this, 'sv-http-cache-clear-policy', {
statements: [policyStatement],
});
func.role?.attachInlinePolicy(policy);
new Rule(this, 'sv-http-cache-clear-schedule-rule', {
description: 'Schedule a Lambda that regularly clears the cache',
schedule: Schedule.cron({
year: '*',
month: '*',
day: '*',
hour: '1',
minute: '0',
}),
targets: [new LambdaFunction(func)],
});
}
}
这是在 CDK 中从这里实例化的：
#!/usr/bin/env node
import 'source-map-support/register';
import * as cdk from '@aws-cdk/core';
import { AppStack } from '../lib/app-stack';
import { ECR } from '../lib/ecr';
import { Lambda } from '../lib/lambda';
import { getConfig } from '../lib/config';
const app = new cdk.App();
// Stateful infra
const { repository } = new ECR(app, 'sv-ecr-stack');
// Get app config
const environment = process.env.ENVIRONMENT || '';
const { certificateArn, domainName, sIdProgramId, logzioToken, optimizelyUrl } = getConfig(environment);
new AppStack(app, 'sv-stack', {
repository,
domainName,
certificateArn,
sIdProgramId,
logzioToken,
optimizelyUrl,
environment,
});
const lambda = new Lambda(app, 'sv-http-scheduled-cache-clear', {
logzioToken,
});
为了简洁起见，我没有将 AppStack 添加到此 post。
最后我有了一组相当简单的 NPM 依赖项：
{
"name": "cdk",
"version": "0.1.0",
"bin": {
"cdk": "bin/cdk.js"
},
"scripts": {
"build": "tsc",
"watch": "tsc -w",
"cdk": "cdk",
"synth": "cdk synth -q",
"deploy": "cdk deploy --all --progress=events --require-approval=never"
},
"devDependencies": {
"@aws-cdk/assert": "^1.100.0",
"@aws-cdk/aws-certificatemanager": "^1.100.0",
"@aws-cdk/aws-ec2": "^1.100.0",
"@aws-cdk/aws-ecr": "^1.100.0",
"@aws-cdk/aws-ecs": "^1.100.0",
"@aws-cdk/aws-ecs-patterns": "^1.100.0",
"@aws-cdk/aws-iam": "^1.100.0",
"@aws-cdk/aws-route53": "^1.100.0",
"@types/node": "10.17.27",
"aws-cdk": "^1.100.0",
"ts-node": "^9.0.0",
"typescript": "~3.9.7"
},
"dependencies": {
"@aws-cdk/core": "^1.100.0",
"source-map-support": "^0.5.16"
}
}
如果我在本地 运行 npm run synth 然后它 运行 s synth 命令，并成功。我的编辑器中没有 TypeScript 错误。
这是 cdk deploy 错误：
export AWS_ACCESS_KEY_ID=$DEV_ACCESS_KEY_ID
export AWS_SECRET_ACCESS_KEY=$DEV_SECRET_ACCESS_KEY
export ENVIRONMENT=dev
cd cdk
npm install
npm run deploy
npm WARN read-shrinkwrap This version of npm is compatible with lockfileVersion@1, but package-lock.json was generated for lockfileVersion@2. I'll try to do my best with it!
npm WARN cdk@0.1.0 No repository field.
npm WARN cdk@0.1.0 No license field.
added 1363 packages from 232 contributors and audited 1366 packages in 19.228s
18 packages are looking for funding
run `npm fund` for details
found 0 vulnerabilities
> cdk@0.1.0 deploy /home/circleci/project/cdk
> cdk deploy --all --progress=events --require-approval=never
sv-ecr-stack
sv-ecr-stack: deploying...
✅ sv-ecr-stack (no changes)
Outputs:
sv-ecr-stack.ExportsOutputFnGetAttsvrepositoryXXX = arn:aws:ecr:*********:067685711111:repository/student-validation-app
sv-ecr-stack.ExportsOutputRefsvrepositoryXXX = student-validation-app
Stack ARN:
arn:aws:cloudformation:*********:067685711111:stack/sv-ecr-stack/aabbccddeeffgghhiijjkk
sv-http-scheduled-cache-clear
sv-http-scheduled-cache-clear: deploying...
[0%] start: Publishing abcdefghijklmnop:current
[100%] fail: Bucket named 'cdktoolkit-stagingbucket-7k57fjeur592' exists, but not in account 067685711111. Wrong account?
❌ sv-http-scheduled-cache-clear failed: Error: Failed to publish one or more assets. See the error messages above for more information.
at Object.publishAssets (/home/circleci/project/cdk/node_modules/aws-cdk/lib/util/asset-publishing.ts:25:11)
at Object.deployStack (/home/circleci/project/cdk/node_modules/aws-cdk/lib/api/deploy-stack.ts:235:3)
at CdkToolkit.deploy (/home/circleci/project/cdk/node_modules/aws-cdk/lib/cdk-toolkit.ts:180:24)
at initCommandLine (/home/circleci/project/cdk/node_modules/aws-cdk/bin/cdk.ts:208:9)
***************************************************
*** Newer version of CDK is available [1.110.0] ***
*** Upgrade recommended ***
***************************************************
Failed to publish one or more assets. See the error messages above for more information.
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! cdk@0.1.0 deploy: `cdk deploy --all --progress=events --require-approval=never`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the cdk@0.1.0 deploy script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:
npm ERR! /home/circleci/.npm/_logs/2021-06-25T13_34_37_706Z-debug.log
Exited with code exit status 1
一些研究表明 CDK 使用 S3 创建一个“暂存桶”，我假设这就是 cdktoolkit-stagingbucket-* 是什么（它不是我明确创建的工件）。
我是否需要创建此存储桶 remotely/explicitly 或 locally/manually？
更新
我发现 this bug report 这表明错误消息很糟糕 - 它需要 S3 访问权限，但消息并没有说清楚。我已经为 CI/CD 用户找到了这个策略文件，我想知道我是否需要确定要添加什么策略以允许使用 S3。
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "StmtXXXYYYZZZ4501",
"Action": [
"cloudformation:*",
"ec2:*",
"ecs:*",
"ecr:*",
"application-autoscaling:*",
"elasticloadbalancing:*",
"iam:CreateRole",
"iam:DeleteRole",
"iam:GetRole",
"iam:TagRole",
"iam:UpdateRole",
"iam:GetRolePolicy",
"iam:DeleteRolePolicy",
"iam:PutRolePolicy",
"iam:PassRole",
"logs:*",
"route53:*",
"secretsmanager:*",
"sts:*",
"dynamodb:*",
],
"Effect": "Allow",
"Resource": "*"
}
]
}
错误消息具有误导性。如果您有权访问多个帐户，则 S3 存储桶很可能存在于另一个帐户中 - 这是因为它是一个自动生成的存储桶，CDK 使用它来打包和部署 lambda。我假设它在所有 AWS 账户中都是相同的名称。
简单的解决方案是将 "s3:*" 添加到针对 "*" 资源的允许操作。但是，我决定添加一个新的 Policy，以便资源规范更严格一些。在我的例子中，这意味着添加一个具有不同资源规范的新子句：
{
"Action": [
"s3:*"
],
"Effect": "Allow",
"Resource": "arn:aws:s3:::cdktoolkit-stagingbucket-*"
}
您似乎不必显式创建此存储桶 - CDK 会为您完成。似乎“Sid”也不是必填字段。
就我而言，添加此策略显示我还缺少多少其他权限，但至少这些错误消息更明智！
触发该错误的操作是 s3:GetBucketLocation。仅提供 CDK 所需的最小权限的策略是：
{
"Action": [
"s3:ListBucket",
"s3:GetObject",
"s3:GetBucketLocation",
"s3:PutObject",
"s3:DeleteObject"
],
"Resource": "arn:aws:s3:::cdktoolkit-stagingbucket-*",
"Effect": "Allow"
}