GCP 数据融合 StatusRuntimeException：INVALID_ARGUMENT：'DISKS_TOTAL_GB' 配额不足。请求 3000.0，可用 2048.0
GCP Data Fusion StatusRuntimeException: INVALID_ARGUMENT: Insufficient 'DISKS_TOTAL_GB' quota. Requested 3000.0, available 2048.0
我正在尝试在 GCP Data Fusion 中部署管道。我最初使用的是免费帐户，但升级是为了按照以下问题中的建议增加配额。
但是，根据公认的答案，我仍然不清楚要在 GCE 中增加哪些具体配额才能使管道能够到达 运行。有人可以在上面的链接问题中提供更清晰的信息，或者在此处回复以详细说明需要增加 IAM 配额中的哪些内容以解决此处看到的问题：
com.google.api.gax.rpc.InvalidArgumentException: io.grpc.StatusRuntimeException: INVALID_ARGUMENT: Insufficient 'DISKS_TOTAL_GB' quota. Requested 3000.0, available 2048.0.
at com.google.api.gax.rpc.ApiExceptionFactory.createException(ApiExceptionFactory.java:49) ~[na:na]
at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:72) ~[na:na]
at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:60) ~[na:na]
at com.google.api.gax.grpc.GrpcExceptionCallable$ExceptionTransformingFuture.onFailure(GrpcExceptionCallable.java:97) ~[na:na]
at com.google.api.core.ApiFutures.onFailure(ApiFutures.java:68) ~[na:na]
at com.google.common.util.concurrent.Futures.run(Futures.java:1123) ~[com.google.guava.guava-13.0.1.jar:na]
at com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:435) ~[na:na]
at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:900) ~[com.google.guava.guava-13.0.1.jar:na]
at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:811) ~[com.google.guava.guava-13.0.1.jar:na]
at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:675) ~[com.google.guava.guava-13.0.1.jar:na]
at io.grpc.stub.ClientCalls$GrpcFuture.setException(ClientCalls.java:515) ~[na:na]
at io.grpc.stub.ClientCalls$UnaryStreamToFuture.onClose(ClientCalls.java:490) ~[na:na]
at io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39) ~[na:na]
at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23) ~[na:na]
at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40) ~[na:na]
at io.grpc.internal.CensusStatsModule$StatsClientInterceptor.onClose(CensusStatsModule.java:700) ~[na:na]
at io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39) ~[na:na]
at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23) ~[na:na]
at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40) ~[na:na]
at io.grpc.internal.CensusTracingModule$TracingClientInterceptor.onClose(CensusTracingModule.java:399) ~[na:na]
at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:510) ~[na:na]
at io.grpc.internal.ClientCallImpl.access0(ClientCallImpl.java:66) ~[na:na]
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.close(ClientCallImpl.java:630) ~[na:na]
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.access0(ClientCallImpl.java:518) ~[na:na]
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImplStreamClosed.runInternal(ClientCallImpl.java:692) ~[na:na]
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImplStreamClosed.runInContext(ClientCallImpl.java:681) ~[na:na]
at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37) ~[na:na]
at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123) ~[na:na]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_232]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_232]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access1(ScheduledThreadPoolExecutor.java:180) ~[na:1.8.0_232]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) ~[na:1.8.0_232]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[na:1.8.0_232]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[na:1.8.0_232]
at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_232]
Caused by: io.grpc.StatusRuntimeException: INVALID_ARGUMENT: Insufficient 'DISKS_TOTAL_GB' quota. Requested 3000.0, available 2048.0.
at io.grpc.Status.asRuntimeException(Status.java:533) ~[na:na]
与DISKS_TOTAL_GB相关的具体配额是Persistent disk standard (GB)，如您在Disk quotas documentation中所见。
您可以在项目的 Cloud Console 中按区域编辑此配额，方法是转到 IAM 和管理页面 => 配额 并且 select 只有指标 永久性磁盘标准 (GB)。
- 部署管道。
- 按"Configure"并选择"Compute config"。
- 按"Customize"。
- 配置 "Master Nodes" 和 "Worker Nodes" 设置 "Master Disk Size (GB)" 和 "Worker Disk size (GB)" 以使用更少的 space。
- 列表项
计算配置
无需从免费试用版升级，Genavs 的答案是正确的。请记住，可能的最低值为 15GB（否则会给您一个配置错误）。
即使使用比标准要求更高的管道，每个节点默认的 1TB 磁盘大小也是一个很大的禁忌。在调整配额之前，请务必检查为什么要超出配额。
我正在尝试在 GCP Data Fusion 中部署管道。我最初使用的是免费帐户，但升级是为了按照以下问题中的建议增加配额
但是，根据公认的答案，我仍然不清楚要在 GCE 中增加哪些具体配额才能使管道能够到达 运行。有人可以在上面的链接问题中提供更清晰的信息，或者在此处回复以详细说明需要增加 IAM 配额中的哪些内容以解决此处看到的问题：
com.google.api.gax.rpc.InvalidArgumentException: io.grpc.StatusRuntimeException: INVALID_ARGUMENT: Insufficient 'DISKS_TOTAL_GB' quota. Requested 3000.0, available 2048.0.
at com.google.api.gax.rpc.ApiExceptionFactory.createException(ApiExceptionFactory.java:49) ~[na:na]
at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:72) ~[na:na]
at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:60) ~[na:na]
at com.google.api.gax.grpc.GrpcExceptionCallable$ExceptionTransformingFuture.onFailure(GrpcExceptionCallable.java:97) ~[na:na]
at com.google.api.core.ApiFutures.onFailure(ApiFutures.java:68) ~[na:na]
at com.google.common.util.concurrent.Futures.run(Futures.java:1123) ~[com.google.guava.guava-13.0.1.jar:na]
at com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:435) ~[na:na]
at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:900) ~[com.google.guava.guava-13.0.1.jar:na]
at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:811) ~[com.google.guava.guava-13.0.1.jar:na]
at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:675) ~[com.google.guava.guava-13.0.1.jar:na]
at io.grpc.stub.ClientCalls$GrpcFuture.setException(ClientCalls.java:515) ~[na:na]
at io.grpc.stub.ClientCalls$UnaryStreamToFuture.onClose(ClientCalls.java:490) ~[na:na]
at io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39) ~[na:na]
at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23) ~[na:na]
at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40) ~[na:na]
at io.grpc.internal.CensusStatsModule$StatsClientInterceptor.onClose(CensusStatsModule.java:700) ~[na:na]
at io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39) ~[na:na]
at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23) ~[na:na]
at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40) ~[na:na]
at io.grpc.internal.CensusTracingModule$TracingClientInterceptor.onClose(CensusTracingModule.java:399) ~[na:na]
at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:510) ~[na:na]
at io.grpc.internal.ClientCallImpl.access0(ClientCallImpl.java:66) ~[na:na]
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.close(ClientCallImpl.java:630) ~[na:na]
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.access0(ClientCallImpl.java:518) ~[na:na]
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImplStreamClosed.runInternal(ClientCallImpl.java:692) ~[na:na]
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImplStreamClosed.runInContext(ClientCallImpl.java:681) ~[na:na]
at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37) ~[na:na]
at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123) ~[na:na]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_232]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_232]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access1(ScheduledThreadPoolExecutor.java:180) ~[na:1.8.0_232]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) ~[na:1.8.0_232]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[na:1.8.0_232]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[na:1.8.0_232]
at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_232]
Caused by: io.grpc.StatusRuntimeException: INVALID_ARGUMENT: Insufficient 'DISKS_TOTAL_GB' quota. Requested 3000.0, available 2048.0.
at io.grpc.Status.asRuntimeException(Status.java:533) ~[na:na]
与DISKS_TOTAL_GB相关的具体配额是Persistent disk standard (GB)，如您在Disk quotas documentation中所见。
您可以在项目的 Cloud Console 中按区域编辑此配额，方法是转到 IAM 和管理页面 => 配额 并且 select 只有指标 永久性磁盘标准 (GB)。
- 部署管道。
- 按"Configure"并选择"Compute config"。
- 按"Customize"。
- 配置 "Master Nodes" 和 "Worker Nodes" 设置 "Master Disk Size (GB)" 和 "Worker Disk size (GB)" 以使用更少的 space。
- 列表项
计算配置
无需从免费试用版升级，Genavs 的答案是正确的。请记住，可能的最低值为 15GB（否则会给您一个配置错误）。 即使使用比标准要求更高的管道，每个节点默认的 1TB 磁盘大小也是一个很大的禁忌。在调整配额之前，请务必检查为什么要超出配额。