当我将此 BucketDeployment 添加到我的 CDK CodePipeline 时，cdk synth 永远不会完成
When I add this BucketDeployment to my CDK CodePipeline, cdk synth never finishes
我正在尝试使用 CDK 和 CodePipeline 构建 React 应用程序并将其部署到 S3。在 CodePipeline 阶段之后，在我自己的堆栈中，我这样定义了 S3 存储桶：
const bucket = new Bucket(this, "Bucket", {
websiteIndexDocument: "index.html",
websiteErrorDocument: "error.html",
})
有效。然后我这样定义了我构建的 React 应用程序的部署：
new BucketDeployment(this, "WebsiteDeployment", {
sources: [Source.asset("./")],
destinationBucket: bucket
})
这似乎不起作用。 BucketDeployment 的用法正确吗？
当我添加 BucketDeployment 行时发生的一些奇怪的事情是 cdk synth 或 cdk deploy，它们永远不会完成并且它们似乎在 [=21= 中生成无限递归树], 所以那里肯定有问题。
如果我更改为 Source.asset("./build")，我会收到错误消息：
> cdk synth
C:\Users\pupeno\Code\ww3fe\node_modules\aws-cdk-lib\core\lib\asset-staging.ts:109
throw new Error(`Cannot find asset at ${this.sourcePath}`);
^
Error: Cannot find asset at C:\Users\pupeno\Code\ww3fe\build
at new AssetStaging (C:\Users\pupeno\Code\ww3fe\node_modules\aws-cdk-lib\core\lib\asset-staging.ts:109:13)
at new Asset (C:\Users\pupeno\Code\ww3fe\node_modules\aws-cdk-lib\aws-s3-assets\lib\asset.ts:72:21)
at Object.bind (C:\Users\pupeno\Code\ww3fe\node_modules\aws-cdk-lib\aws-s3-deployment\lib\source.ts:55:23)
at C:\Users\pupeno\Code\ww3fe\node_modules\aws-cdk-lib\aws-s3-deployment\lib\bucket-deployment.ts:170:83
at Array.map (<anonymous>)
at new BucketDeployment (C:\Users\pupeno\Code\ww3fe\node_modules\aws-cdk-lib\aws-s3-deployment\lib\bucket-deployment.ts:170:51)
at new MainStack (C:\Users\pupeno\Code\ww3fe\infra\pipeline-stack.ts:16:9)
at new DeployStage (C:\Users\pupeno\Code\ww3fe\infra\pipeline-stack.ts:28:26)
at new PipelineStack (C:\Users\pupeno\Code\ww3fe\infra\pipeline-stack.ts:56:24)
at Object.<anonymous> (C:\Users\pupeno\Code\ww3fe\infra\pipeline.ts:6:1)
Subprocess exited with error 1
这表明这是非常错误的。为什么它在我的机器上搜索 build 目录？它应该在构建后在 CodePipeline 中搜索它。
我的整个管道是：
import {Construct} from "constructs"
import {CodeBuildStep, CodePipeline, CodePipelineSource} from "aws-cdk-lib/pipelines"
import {Stage, CfnOutput, StageProps, Stack, StackProps} from "aws-cdk-lib"
import {Bucket} from "aws-cdk-lib/aws-s3"
import {BucketDeployment, Source} from "aws-cdk-lib/aws-s3-deployment"
export class MainStack extends Stack {
constructor(scope: Construct, id: string, props?: StageProps) {
super(scope, id, props)
const bucket = new Bucket(this, "Bucket", {
websiteIndexDocument: "index.html",
websiteErrorDocument: "error.html",
})
new BucketDeployment(this, "WebsiteDeployment", {
sources: [Source.asset("./")],
destinationBucket: bucket
})
new CfnOutput(this, "BucketOutput", {value: bucket.bucketArn})
}
}
export class DeployStage extends Stage {
public readonly mainStack: MainStack
constructor(scope: Construct, id: string, props?: StageProps) {
super(scope, id, props)
this.mainStack = new MainStack(this, "example")
}
}
export class PipelineStack extends Stack {
constructor(scope: Construct, id: string, props?: StackProps) {
super(scope, id, props)
const pipeline = new CodePipeline(this, id, {
pipelineName: id,
synth: new CodeBuildStep("Synth", {
input: CodePipelineSource.connection("user/example", "main", {
connectionArn: "arn:aws:codestar-connections:....",
}),
installCommands: [
"npm install -g aws-cdk"
],
commands: [
"npm ci",
"npm run build",
"npx cdk synth"
]
}
),
})
const deploy = new DeployStage(this, "Staging")
const deployStage = pipeline.addStage(deploy)
}
}
我在 AWS 中遇到的实际错误是：
[Container] 2022/01/25 20:28:27 Phase complete: BUILD State: SUCCEEDED
[Container] 2022/01/25 20:28:27 Phase context status code: Message:
[Container] 2022/01/25 20:28:27 Entering phase POST_BUILD
[Container] 2022/01/25 20:28:27 Phase complete: POST_BUILD State: SUCCEEDED
[Container] 2022/01/25 20:28:27 Phase context status code: Message:
[Container] 2022/01/25 20:28:27 Expanding base directory path: cdk.out
[Container] 2022/01/25 20:28:27 Assembling file list
[Container] 2022/01/25 20:28:27 Expanding cdk.out
[Container] 2022/01/25 20:28:27 Skipping invalid file path cdk.out
[Container] 2022/01/25 20:28:27 Phase complete: UPLOAD_ARTIFACTS State: FAILED
[Container] 2022/01/25 20:28:27 Phase context status code: CLIENT_ERROR Message: no matching base directory path found for cdk.out
以防万一，cdk.json，在我的 repo 的根目录中，包含：
{
"app": "npx ts-node --project infra/tsconfig.json --prefer-ts-exts infra/pipeline.ts",
"watch": {
"include": [
"**"
],
"exclude": [
"README.md",
"cdk*.json",
"**/*.d.ts",
"**/*.js",
"tsconfig.json",
"package*.json",
"yarn.lock",
"node_modules",
"test"
]
},
"context": {
"@aws-cdk/aws-apigateway:usagePlanKeyOrderInsensitiveId": true,
"@aws-cdk/core:stackRelativeExports": true,
"@aws-cdk/aws-rds:lowercaseDbIdentifier": true,
"@aws-cdk/aws-lambda:recognizeVersionProps": true,
"@aws-cdk/aws-cloudfront:defaultSecurityPolicyTLSv1.2_2021": true
}
}
第一个问题：
And if I change to Source.asset("./build") I get the error: ... Why is it searching for the build directory on my machine?
当您在本地 运行 cdk synth 时，就会发生这种情况。请记住，cdk synth 将始终引用此命令所在的文件系统 运行。在本地它将是您的机器，在管道中它将在 AWS CodePipeline 使用的容器或环境中。
深入了解 BucketDeployment
而且，这里发生的一些有趣的事情可能会有所帮助。 BucketDeployment 不只是从您在 BucketDeployment.sources 中引用的源中提取并将其上传到您在 BucketDeployment.destinationBucket 中指定的存储桶。根据 BucketDeployment docs ，资产被上传到一个中间桶，然后合并到你的桶。这很重要，因为它将解释您收到的错误 Error: Cannot find asset at C:\Users\pupeno\Code\ww3fe\build，因为当您 运行 cdk synth 时，它将期望 Source.asset("./build") 中所述的目录 ./build 存在。
当尝试使用 CodePipeline 构建和部署单页应用程序（如您的案例中的 React）时，这会变得非常有趣。默认情况下，CodePipeline 将执行 Source 步骤，然后执行 Synth 步骤，然后是您在其后添加的任何 waves 或阶段。添加构建您的 React 应用程序的 wave 不会立即起作用，因为我们现在看到在 Synth 步骤中需要构建您的 React 应用程序的输出目录，因为 BucketDeployment 的工作方式。我们需要能够让顺序为 Source -> Build -> Synth -> Deploy。正如发现 ，我们可以通过使用输入和输出来控制步骤的顺序。 CodePipeline 将对步骤进行排序以确保满足 input/output 依赖关系。所以我们需要让我们的 Synth 步骤使用 Build 的输出作为它的输入。
关注当前定义的管道
我相信您当前的管道缺少一个 CodeBuildStep，它将捆绑您的 React 应用程序并将其输出到您在 BucketDeployment.sources 中指定的目录。我们还需要设置输入以正确排序这些操作。以下是对管道定义的一些更新，但可能需要进行一些更改才能获得正确的文件路径。另外，将 BucketDeployment.sources 设置为写入应用程序包的目录。
export class PipelineStack extends Stack {
constructor(scope: Construct, id: string, props?: StackProps) {
super(scope, id, props)
const sourceAction = CodePipelineSource.connection("user/example", "main", {
connectionArn: "arn:aws:codestar-connections:....",
});
const buildAction = new CodeBuildStep("Build", {
input: sourceAction, // This places Source first
installCommands: [ "npm ci" ],
commands: [ "npm run build" ], // Change this to your build command for your react app
// We need to output the entire contents of our file structure
// this is both the CDK code needed for synth and the bundled app
primaryOutputDirectory: "./",
});
const synthAction = new ShellStep("Synth", {
input: buildAction, // This places Synth after Build
installCommands: [
"npm install -g aws-cdk"
],
commands: [
"npm ci",
"npm run build",
"npx cdk synth"
],
// Synth step must output to cdk.out
// if your CDK code is nested, this will have to match that structure
primaryOutputDirectory: "./cdk.out",
});
const pipeline = new CodePipeline(this, id, {
pipelineName: id,
synth: synthAction,
});
const deploy = new DeployStage(this, "Staging");
const deployStage = pipeline.addStage(deploy);
}
}
TL;DR 通过两个更改，管道成功部署了 React 应用程序：(1) Source.asset 需要 build 目录的完整路径(2) 需要将 React 构建命令添加到合成步骤。
给 Source.asset React build 目录的完整路径：
new BucketDeployment(this, "WebsiteDeployment", {
sources: [Source.asset(path.join(__dirname, './build'))], // relative to the Stack dir
destinationBucket: bucket
})
React 构建工件通常是 .gitignored，因此 CodePipeline 需要构建 React 应用程序。我的版本有一个单独的 package.json 用于 React 应用程序，因此构建步骤 1 需要更多命令：
synth: new pipelines.CodeBuildStep('Synth', {
commands: [
// build react (new)
'cd react-app', // path from project root to React app package.json
'npm ci',
'npm run build',
'cd ..',
// synth cdk (as in OP)
"npm ci",
"npm run build",
"npx cdk synth" // synth must run AFTER the React build step
],
React 应用程序部署到 S3 URL：
// MainStack
new cdk.CfnOutput(this, 'WebsiteUrl', {
value: `http://${this.websiteBucket.bucketName}.s3-website-${this.region}.amazonaws.com`,
});
(1) pipelines.CodePipeline 是一个 opinionated construct for deploying Stacks。 lower-level codepipeline.Pipeline 构造具有许多应用程序需要的功能，例如分离构建步骤和在步骤之间传递 build-time 环境变量（例如注入 API URL使用 REACT_APP_API_URL env var).
进入客户端包
如果 json 或 html 等文件未提交推送到 Repo，请查看您的 gitignore 文件。
我正在尝试使用 CDK 和 CodePipeline 构建 React 应用程序并将其部署到 S3。在 CodePipeline 阶段之后，在我自己的堆栈中，我这样定义了 S3 存储桶：
const bucket = new Bucket(this, "Bucket", {
websiteIndexDocument: "index.html",
websiteErrorDocument: "error.html",
})
有效。然后我这样定义了我构建的 React 应用程序的部署：
new BucketDeployment(this, "WebsiteDeployment", {
sources: [Source.asset("./")],
destinationBucket: bucket
})
这似乎不起作用。 BucketDeployment 的用法正确吗？
当我添加 BucketDeployment 行时发生的一些奇怪的事情是 cdk synth 或 cdk deploy，它们永远不会完成并且它们似乎在 [=21= 中生成无限递归树], 所以那里肯定有问题。
如果我更改为 Source.asset("./build")，我会收到错误消息：
> cdk synth
C:\Users\pupeno\Code\ww3fe\node_modules\aws-cdk-lib\core\lib\asset-staging.ts:109
throw new Error(`Cannot find asset at ${this.sourcePath}`);
^
Error: Cannot find asset at C:\Users\pupeno\Code\ww3fe\build
at new AssetStaging (C:\Users\pupeno\Code\ww3fe\node_modules\aws-cdk-lib\core\lib\asset-staging.ts:109:13)
at new Asset (C:\Users\pupeno\Code\ww3fe\node_modules\aws-cdk-lib\aws-s3-assets\lib\asset.ts:72:21)
at Object.bind (C:\Users\pupeno\Code\ww3fe\node_modules\aws-cdk-lib\aws-s3-deployment\lib\source.ts:55:23)
at C:\Users\pupeno\Code\ww3fe\node_modules\aws-cdk-lib\aws-s3-deployment\lib\bucket-deployment.ts:170:83
at Array.map (<anonymous>)
at new BucketDeployment (C:\Users\pupeno\Code\ww3fe\node_modules\aws-cdk-lib\aws-s3-deployment\lib\bucket-deployment.ts:170:51)
at new MainStack (C:\Users\pupeno\Code\ww3fe\infra\pipeline-stack.ts:16:9)
at new DeployStage (C:\Users\pupeno\Code\ww3fe\infra\pipeline-stack.ts:28:26)
at new PipelineStack (C:\Users\pupeno\Code\ww3fe\infra\pipeline-stack.ts:56:24)
at Object.<anonymous> (C:\Users\pupeno\Code\ww3fe\infra\pipeline.ts:6:1)
Subprocess exited with error 1
这表明这是非常错误的。为什么它在我的机器上搜索 build 目录？它应该在构建后在 CodePipeline 中搜索它。
我的整个管道是：
import {Construct} from "constructs"
import {CodeBuildStep, CodePipeline, CodePipelineSource} from "aws-cdk-lib/pipelines"
import {Stage, CfnOutput, StageProps, Stack, StackProps} from "aws-cdk-lib"
import {Bucket} from "aws-cdk-lib/aws-s3"
import {BucketDeployment, Source} from "aws-cdk-lib/aws-s3-deployment"
export class MainStack extends Stack {
constructor(scope: Construct, id: string, props?: StageProps) {
super(scope, id, props)
const bucket = new Bucket(this, "Bucket", {
websiteIndexDocument: "index.html",
websiteErrorDocument: "error.html",
})
new BucketDeployment(this, "WebsiteDeployment", {
sources: [Source.asset("./")],
destinationBucket: bucket
})
new CfnOutput(this, "BucketOutput", {value: bucket.bucketArn})
}
}
export class DeployStage extends Stage {
public readonly mainStack: MainStack
constructor(scope: Construct, id: string, props?: StageProps) {
super(scope, id, props)
this.mainStack = new MainStack(this, "example")
}
}
export class PipelineStack extends Stack {
constructor(scope: Construct, id: string, props?: StackProps) {
super(scope, id, props)
const pipeline = new CodePipeline(this, id, {
pipelineName: id,
synth: new CodeBuildStep("Synth", {
input: CodePipelineSource.connection("user/example", "main", {
connectionArn: "arn:aws:codestar-connections:....",
}),
installCommands: [
"npm install -g aws-cdk"
],
commands: [
"npm ci",
"npm run build",
"npx cdk synth"
]
}
),
})
const deploy = new DeployStage(this, "Staging")
const deployStage = pipeline.addStage(deploy)
}
}
我在 AWS 中遇到的实际错误是：
[Container] 2022/01/25 20:28:27 Phase complete: BUILD State: SUCCEEDED
[Container] 2022/01/25 20:28:27 Phase context status code: Message:
[Container] 2022/01/25 20:28:27 Entering phase POST_BUILD
[Container] 2022/01/25 20:28:27 Phase complete: POST_BUILD State: SUCCEEDED
[Container] 2022/01/25 20:28:27 Phase context status code: Message:
[Container] 2022/01/25 20:28:27 Expanding base directory path: cdk.out
[Container] 2022/01/25 20:28:27 Assembling file list
[Container] 2022/01/25 20:28:27 Expanding cdk.out
[Container] 2022/01/25 20:28:27 Skipping invalid file path cdk.out
[Container] 2022/01/25 20:28:27 Phase complete: UPLOAD_ARTIFACTS State: FAILED
[Container] 2022/01/25 20:28:27 Phase context status code: CLIENT_ERROR Message: no matching base directory path found for cdk.out
以防万一，cdk.json，在我的 repo 的根目录中，包含：
{
"app": "npx ts-node --project infra/tsconfig.json --prefer-ts-exts infra/pipeline.ts",
"watch": {
"include": [
"**"
],
"exclude": [
"README.md",
"cdk*.json",
"**/*.d.ts",
"**/*.js",
"tsconfig.json",
"package*.json",
"yarn.lock",
"node_modules",
"test"
]
},
"context": {
"@aws-cdk/aws-apigateway:usagePlanKeyOrderInsensitiveId": true,
"@aws-cdk/core:stackRelativeExports": true,
"@aws-cdk/aws-rds:lowercaseDbIdentifier": true,
"@aws-cdk/aws-lambda:recognizeVersionProps": true,
"@aws-cdk/aws-cloudfront:defaultSecurityPolicyTLSv1.2_2021": true
}
}
第一个问题：
And if I change to Source.asset("./build") I get the error: ... Why is it searching for the build directory on my machine?
当您在本地 运行 cdk synth 时，就会发生这种情况。请记住，cdk synth 将始终引用此命令所在的文件系统 运行。在本地它将是您的机器，在管道中它将在 AWS CodePipeline 使用的容器或环境中。
深入了解 BucketDeployment
而且，这里发生的一些有趣的事情可能会有所帮助。 BucketDeployment 不只是从您在 BucketDeployment.sources 中引用的源中提取并将其上传到您在 BucketDeployment.destinationBucket 中指定的存储桶。根据 BucketDeployment docs ，资产被上传到一个中间桶，然后合并到你的桶。这很重要，因为它将解释您收到的错误 Error: Cannot find asset at C:\Users\pupeno\Code\ww3fe\build，因为当您 运行 cdk synth 时，它将期望 Source.asset("./build") 中所述的目录 ./build 存在。
当尝试使用 CodePipeline 构建和部署单页应用程序（如您的案例中的 React）时，这会变得非常有趣。默认情况下，CodePipeline 将执行 Source 步骤，然后执行 Synth 步骤，然后是您在其后添加的任何 waves 或阶段。添加构建您的 React 应用程序的 wave 不会立即起作用，因为我们现在看到在 Synth 步骤中需要构建您的 React 应用程序的输出目录，因为 BucketDeployment 的工作方式。我们需要能够让顺序为 Source -> Build -> Synth -> Deploy。正如发现
关注当前定义的管道
我相信您当前的管道缺少一个 CodeBuildStep，它将捆绑您的 React 应用程序并将其输出到您在 BucketDeployment.sources 中指定的目录。我们还需要设置输入以正确排序这些操作。以下是对管道定义的一些更新，但可能需要进行一些更改才能获得正确的文件路径。另外，将 BucketDeployment.sources 设置为写入应用程序包的目录。
export class PipelineStack extends Stack {
constructor(scope: Construct, id: string, props?: StackProps) {
super(scope, id, props)
const sourceAction = CodePipelineSource.connection("user/example", "main", {
connectionArn: "arn:aws:codestar-connections:....",
});
const buildAction = new CodeBuildStep("Build", {
input: sourceAction, // This places Source first
installCommands: [ "npm ci" ],
commands: [ "npm run build" ], // Change this to your build command for your react app
// We need to output the entire contents of our file structure
// this is both the CDK code needed for synth and the bundled app
primaryOutputDirectory: "./",
});
const synthAction = new ShellStep("Synth", {
input: buildAction, // This places Synth after Build
installCommands: [
"npm install -g aws-cdk"
],
commands: [
"npm ci",
"npm run build",
"npx cdk synth"
],
// Synth step must output to cdk.out
// if your CDK code is nested, this will have to match that structure
primaryOutputDirectory: "./cdk.out",
});
const pipeline = new CodePipeline(this, id, {
pipelineName: id,
synth: synthAction,
});
const deploy = new DeployStage(this, "Staging");
const deployStage = pipeline.addStage(deploy);
}
}
TL;DR 通过两个更改，管道成功部署了 React 应用程序：(1) Source.asset 需要 build 目录的完整路径(2) 需要将 React 构建命令添加到合成步骤。
给 Source.asset React build 目录的完整路径：
new BucketDeployment(this, "WebsiteDeployment", {
sources: [Source.asset(path.join(__dirname, './build'))], // relative to the Stack dir
destinationBucket: bucket
})
React 构建工件通常是 .gitignored，因此 CodePipeline 需要构建 React 应用程序。我的版本有一个单独的 package.json 用于 React 应用程序，因此构建步骤 1 需要更多命令：
synth: new pipelines.CodeBuildStep('Synth', {
commands: [
// build react (new)
'cd react-app', // path from project root to React app package.json
'npm ci',
'npm run build',
'cd ..',
// synth cdk (as in OP)
"npm ci",
"npm run build",
"npx cdk synth" // synth must run AFTER the React build step
],
React 应用程序部署到 S3 URL：
// MainStack
new cdk.CfnOutput(this, 'WebsiteUrl', {
value: `http://${this.websiteBucket.bucketName}.s3-website-${this.region}.amazonaws.com`,
});
(1) pipelines.CodePipeline 是一个 opinionated construct for deploying Stacks。 lower-level codepipeline.Pipeline 构造具有许多应用程序需要的功能，例如分离构建步骤和在步骤之间传递 build-time 环境变量（例如注入 API URL使用 REACT_APP_API_URL env var).
如果 json 或 html 等文件未提交推送到 Repo，请查看您的 gitignore 文件。