如何从 Kubernetes Engine 访问 Google KMS？
How to access Google KMS from Kubernetes Engine?
我不得不将我的 .Net Core 应用程序从 Google App Engine 移动到 Google Kubernetes Engine，因为我需要静态 IP，遗憾的是 Google App Engine 没有选项。
我已经成功创建了一个集群和一些 pod，但在日志中我看到：
Unhandled exception. Grpc.Core.RpcException: Status(StatusCode=PermissionDenied, Detail="Request had insufficient authentication scopes.")
at Grpc.Core.Internal.AsyncCall`2.UnaryCall(TRequest msg)
at Grpc.Core.Calls.BlockingUnaryCall[TRequest,TResponse](CallInvocationDetails`2 call, TRequest req)
at Grpc.Core.DefaultCallInvoker.BlockingUnaryCall[TRequest,TResponse](Method`2 method, String host, CallOptions options, TRequest request)
at Grpc.Core.Interceptors.InterceptingCallInvoker.<BlockingUnaryCall>b__3_0[TRequest,TResponse](TRequest req, ClientInterceptorContext`2 ctx)
at Grpc.Core.ClientBase.ClientBaseConfiguration.ClientBaseConfigurationInterceptor.BlockingUnaryCall[TRequest,TResponse](TRequest request, ClientInterceptorContext`2 context, BlockingUnaryCallContinuation`2 continuation)
at Grpc.Core.Interceptors.InterceptingCallInvoker.BlockingUnaryCall[TRequest,TResponse](Method`2 method, String host, CallOptions options, TRequest request)
at Google.Cloud.Kms.V1.KeyManagementService.KeyManagementServiceClient.Decrypt(DecryptRequest request, CallOptions options)
at Google.Api.Gax.Grpc.ApiCall.GrpcCallAdapter`2.CallSync(TRequest request, CallSettings callSettings)
at Google.Api.Gax.Grpc.ApiCallRetryExtensions.<>c__DisplayClass1_0`2.<WithRetry>b__0(TRequest request, CallSettings callSettings)
at Google.Api.Gax.Grpc.ApiCall`2.<>c__DisplayClass10_0.<WithCallSettingsOverlay>b__1(TRequest req, CallSettings cs)
at Google.Api.Gax.Grpc.ApiCall`2.Sync(TRequest request, CallSettings perCallCallSettings)
at Google.Cloud.Kms.V1.KeyManagementServiceClientImpl.Decrypt(DecryptRequest request, CallSettings callSettings)
at Google.Cloud.Kms.V1.KeyManagementServiceClient.Decrypt(CryptoKeyName name, ByteString ciphertext, CallSettings callSettings)
at Neo.Services.Kms.EncryptedFileInfo.CreateReadStream() in /app/Services/Kms/EncryptedFileInfo.cs:line 81
at Microsoft.Extensions.Configuration.FileConfigurationProvider.Load(Boolean reload)
at Microsoft.Extensions.Configuration.FileConfigurationProvider.Load()
at Microsoft.Extensions.Configuration.ConfigurationRoot..ctor(IList`1 providers)
at Microsoft.Extensions.Configuration.ConfigurationBuilder.Build()
at Neo.Services.Config.ConfigurationProvider.get_AppConfig() in /app/Services/Config/ConfigurationProvider.cs:line 27
at Neo.Startup.LogAppChecks() in /app/Startup.cs:line 197
at Neo.Startup.Run() in /app/Startup.cs:line 24
at Neo.Program.Main() in /app/Program.cs:line 5
我设法发现存在这个问题，因为我无法在 Kubernetes 中解密我的 appsettings.json 文件。它适用于 Google App Engine。
我猜那是因为 GAE 默认设置了 GOOGLE_APPLICATION_CREDENTIALS 环境变量。
我找到了 that 篇文章，但它似乎描述了不同的内容。
如何从 Kubernetes Engine 访问 Google KMS？
@更新
我的节点池的输出描述：
config:
diskSizeGb: 100
diskType: pd-standard
imageType: COS
machineType: n1-standard-1
metadata:
disable-legacy-endpoints: 'true'
oauthScopes:
- https://www.googleapis.com/auth/devstorage.read_only
- https://www.googleapis.com/auth/logging.write
- https://www.googleapis.com/auth/monitoring
- https://www.googleapis.com/auth/service.management.readonly
- https://www.googleapis.com/auth/servicecontrol
- https://www.googleapis.com/auth/trace.append
serviceAccount: default
我已经使用此命令将范围添加到我的新节点池
gcloud container node-pools create your-pool-name --zone europe-west1-b --cluster
your-cluster-name --num-nodes 1 --scopes default,bigquery,cloud-platform,compute-rw,datastore,storage-full,taskqueue,userinfo-email,sql-admin
现在我得到：
Unhandled exception. Grpc.Core.RpcException: Status(StatusCode=PermissionDenied, Detail="Permission 'cloudkms.cryptoKeyVersions.useToDecrypt' denied on resource 'projects/project-name/locations/global/keyRings/webapp/cryptoKeys/appsecrets' (or it may not exist).")
at Grpc.Core.Internal.AsyncCall`2.UnaryCall(TRequest msg)
at Grpc.Core.Calls.BlockingUnaryCall[TRequest,TResponse](CallInvocationDetails`2 call, TRequest req)
at Grpc.Core.DefaultCallInvoker.BlockingUnaryCall[TRequest,TResponse](Method`2 method, String host, CallOptions options, TRequest request)
at Grpc.Core.Interceptors.InterceptingCallInvoker.<BlockingUnaryCall>b__3_0[TRequest,TResponse](TRequest req, ClientInterceptorContext`2 ctx)
at Grpc.Core.ClientBase.ClientBaseConfiguration.ClientBaseConfigurationInterceptor.BlockingUnaryCall[TRequest,TResponse](TRequest request, ClientInterceptorContext`2 context, BlockingUnaryCallContinuation`2 continuation)
at Grpc.Core.Interceptors.InterceptingCallInvoker.BlockingUnaryCall[TRequest,TResponse](Method`2 method, String host, CallOptions options, TRequest request)
at Google.Cloud.Kms.V1.KeyManagementService.KeyManagementServiceClient.Decrypt(DecryptRequest request, CallOptions options)
at Google.Api.Gax.Grpc.ApiCall.GrpcCallAdapter`2.CallSync(TRequest request, CallSettings callSettings)
at Google.Api.Gax.Grpc.ApiCallRetryExtensions.<>c__DisplayClass1_0`2.<WithRetry>b__0(TRequest request, CallSettings callSettings)
at Google.Api.Gax.Grpc.ApiCall`2.<>c__DisplayClass10_0.<WithCallSettingsOverlay>b__1(TRequest req, CallSettings cs)
at Google.Api.Gax.Grpc.ApiCall`2.Sync(TRequest request, CallSettings perCallCallSettings)
at Google.Cloud.Kms.V1.KeyManagementServiceClientImpl.Decrypt(DecryptRequest request, CallSettings callSettings)
at Google.Cloud.Kms.V1.KeyManagementServiceClient.Decrypt(CryptoKeyName name, ByteString ciphertext, CallSettings callSettings)
at Neo.Services.Kms.EncryptedFileInfo.CreateReadStream() in /app/Services/Kms/EncryptedFileInfo.cs:line 81
at Microsoft.Extensions.Configuration.FileConfigurationProvider.Load(Boolean reload)
at Microsoft.Extensions.Configuration.FileConfigurationProvider.Load()
at Microsoft.Extensions.Configuration.ConfigurationRoot..ctor(IList`1 providers)
at Microsoft.Extensions.Configuration.ConfigurationBuilder.Build()
at Neo.Services.Config.ConfigurationProvider.get_AppConfig() in /app/Services/Config/ConfigurationProvider.cs:line 27
at Neo.Startup.LogAppChecks() in /app/Startup.cs:line 197
at Neo.Startup.Run() in /app/Startup.cs:line 24
at Neo.Program.Main() in /app/Program.cs:line 5
在我将 Google KMS Decypt/Encrypt 角色添加到服务帐户后，它起作用了！
感谢@sethvargo
现在可以使用了
错误信息如下：
Request had insufficient authentication scopes
因此需要添加范围 https://www.googleapis.com/auth/cloud-platform。
服务帐户需要 IAM 角色 roles/cloudkms.cryptoKeyEncrypterDecrypter。
我不得不将我的 .Net Core 应用程序从 Google App Engine 移动到 Google Kubernetes Engine，因为我需要静态 IP，遗憾的是 Google App Engine 没有选项。
我已经成功创建了一个集群和一些 pod，但在日志中我看到：
Unhandled exception. Grpc.Core.RpcException: Status(StatusCode=PermissionDenied, Detail="Request had insufficient authentication scopes.")
at Grpc.Core.Internal.AsyncCall`2.UnaryCall(TRequest msg)
at Grpc.Core.Calls.BlockingUnaryCall[TRequest,TResponse](CallInvocationDetails`2 call, TRequest req)
at Grpc.Core.DefaultCallInvoker.BlockingUnaryCall[TRequest,TResponse](Method`2 method, String host, CallOptions options, TRequest request)
at Grpc.Core.Interceptors.InterceptingCallInvoker.<BlockingUnaryCall>b__3_0[TRequest,TResponse](TRequest req, ClientInterceptorContext`2 ctx)
at Grpc.Core.ClientBase.ClientBaseConfiguration.ClientBaseConfigurationInterceptor.BlockingUnaryCall[TRequest,TResponse](TRequest request, ClientInterceptorContext`2 context, BlockingUnaryCallContinuation`2 continuation)
at Grpc.Core.Interceptors.InterceptingCallInvoker.BlockingUnaryCall[TRequest,TResponse](Method`2 method, String host, CallOptions options, TRequest request)
at Google.Cloud.Kms.V1.KeyManagementService.KeyManagementServiceClient.Decrypt(DecryptRequest request, CallOptions options)
at Google.Api.Gax.Grpc.ApiCall.GrpcCallAdapter`2.CallSync(TRequest request, CallSettings callSettings)
at Google.Api.Gax.Grpc.ApiCallRetryExtensions.<>c__DisplayClass1_0`2.<WithRetry>b__0(TRequest request, CallSettings callSettings)
at Google.Api.Gax.Grpc.ApiCall`2.<>c__DisplayClass10_0.<WithCallSettingsOverlay>b__1(TRequest req, CallSettings cs)
at Google.Api.Gax.Grpc.ApiCall`2.Sync(TRequest request, CallSettings perCallCallSettings)
at Google.Cloud.Kms.V1.KeyManagementServiceClientImpl.Decrypt(DecryptRequest request, CallSettings callSettings)
at Google.Cloud.Kms.V1.KeyManagementServiceClient.Decrypt(CryptoKeyName name, ByteString ciphertext, CallSettings callSettings)
at Neo.Services.Kms.EncryptedFileInfo.CreateReadStream() in /app/Services/Kms/EncryptedFileInfo.cs:line 81
at Microsoft.Extensions.Configuration.FileConfigurationProvider.Load(Boolean reload)
at Microsoft.Extensions.Configuration.FileConfigurationProvider.Load()
at Microsoft.Extensions.Configuration.ConfigurationRoot..ctor(IList`1 providers)
at Microsoft.Extensions.Configuration.ConfigurationBuilder.Build()
at Neo.Services.Config.ConfigurationProvider.get_AppConfig() in /app/Services/Config/ConfigurationProvider.cs:line 27
at Neo.Startup.LogAppChecks() in /app/Startup.cs:line 197
at Neo.Startup.Run() in /app/Startup.cs:line 24
at Neo.Program.Main() in /app/Program.cs:line 5
我设法发现存在这个问题，因为我无法在 Kubernetes 中解密我的 appsettings.json 文件。它适用于 Google App Engine。
我猜那是因为 GAE 默认设置了 GOOGLE_APPLICATION_CREDENTIALS 环境变量。
我找到了 that 篇文章，但它似乎描述了不同的内容。
如何从 Kubernetes Engine 访问 Google KMS？
@更新
我的节点池的输出描述：
config:
diskSizeGb: 100
diskType: pd-standard
imageType: COS
machineType: n1-standard-1
metadata:
disable-legacy-endpoints: 'true'
oauthScopes:
- https://www.googleapis.com/auth/devstorage.read_only
- https://www.googleapis.com/auth/logging.write
- https://www.googleapis.com/auth/monitoring
- https://www.googleapis.com/auth/service.management.readonly
- https://www.googleapis.com/auth/servicecontrol
- https://www.googleapis.com/auth/trace.append
serviceAccount: default
我已经使用此命令将范围添加到我的新节点池
gcloud container node-pools create your-pool-name --zone europe-west1-b --cluster
your-cluster-name --num-nodes 1 --scopes default,bigquery,cloud-platform,compute-rw,datastore,storage-full,taskqueue,userinfo-email,sql-admin
现在我得到：
Unhandled exception. Grpc.Core.RpcException: Status(StatusCode=PermissionDenied, Detail="Permission 'cloudkms.cryptoKeyVersions.useToDecrypt' denied on resource 'projects/project-name/locations/global/keyRings/webapp/cryptoKeys/appsecrets' (or it may not exist).")
at Grpc.Core.Internal.AsyncCall`2.UnaryCall(TRequest msg)
at Grpc.Core.Calls.BlockingUnaryCall[TRequest,TResponse](CallInvocationDetails`2 call, TRequest req)
at Grpc.Core.DefaultCallInvoker.BlockingUnaryCall[TRequest,TResponse](Method`2 method, String host, CallOptions options, TRequest request)
at Grpc.Core.Interceptors.InterceptingCallInvoker.<BlockingUnaryCall>b__3_0[TRequest,TResponse](TRequest req, ClientInterceptorContext`2 ctx)
at Grpc.Core.ClientBase.ClientBaseConfiguration.ClientBaseConfigurationInterceptor.BlockingUnaryCall[TRequest,TResponse](TRequest request, ClientInterceptorContext`2 context, BlockingUnaryCallContinuation`2 continuation)
at Grpc.Core.Interceptors.InterceptingCallInvoker.BlockingUnaryCall[TRequest,TResponse](Method`2 method, String host, CallOptions options, TRequest request)
at Google.Cloud.Kms.V1.KeyManagementService.KeyManagementServiceClient.Decrypt(DecryptRequest request, CallOptions options)
at Google.Api.Gax.Grpc.ApiCall.GrpcCallAdapter`2.CallSync(TRequest request, CallSettings callSettings)
at Google.Api.Gax.Grpc.ApiCallRetryExtensions.<>c__DisplayClass1_0`2.<WithRetry>b__0(TRequest request, CallSettings callSettings)
at Google.Api.Gax.Grpc.ApiCall`2.<>c__DisplayClass10_0.<WithCallSettingsOverlay>b__1(TRequest req, CallSettings cs)
at Google.Api.Gax.Grpc.ApiCall`2.Sync(TRequest request, CallSettings perCallCallSettings)
at Google.Cloud.Kms.V1.KeyManagementServiceClientImpl.Decrypt(DecryptRequest request, CallSettings callSettings)
at Google.Cloud.Kms.V1.KeyManagementServiceClient.Decrypt(CryptoKeyName name, ByteString ciphertext, CallSettings callSettings)
at Neo.Services.Kms.EncryptedFileInfo.CreateReadStream() in /app/Services/Kms/EncryptedFileInfo.cs:line 81
at Microsoft.Extensions.Configuration.FileConfigurationProvider.Load(Boolean reload)
at Microsoft.Extensions.Configuration.FileConfigurationProvider.Load()
at Microsoft.Extensions.Configuration.ConfigurationRoot..ctor(IList`1 providers)
at Microsoft.Extensions.Configuration.ConfigurationBuilder.Build()
at Neo.Services.Config.ConfigurationProvider.get_AppConfig() in /app/Services/Config/ConfigurationProvider.cs:line 27
at Neo.Startup.LogAppChecks() in /app/Startup.cs:line 197
at Neo.Startup.Run() in /app/Startup.cs:line 24
at Neo.Program.Main() in /app/Program.cs:line 5
在我将 Google KMS Decypt/Encrypt 角色添加到服务帐户后，它起作用了！
感谢@sethvargo
现在可以使用了
错误信息如下：
Request had insufficient authentication scopes
因此需要添加范围 https://www.googleapis.com/auth/cloud-platform。
服务帐户需要 IAM 角色 roles/cloudkms.cryptoKeyEncrypterDecrypter。