为什么 Datastore 在获取 URL 时产生超时？

Question

今天很短的时间，我的代码运行在灵活环境 "compat" 中使用 Google Cloud Datastore API 经历了 java.net.SocketTimeoutException : 在另一个 GAE 项目中将项目插入数据存储时获取 URL 时超时。

另外，Dataflow当时插入Items失败；几乎可以肯定是同样的问题。

简单的键查询也会出现这种情况，所以不是大数据的问题。

在此之前和之后，确实正确插入了许多其他数据；包括对相同数据的重新运行。

谷歌搜索错误表明它可能是由 Google 云中的停机引起的，但 Google 云状态仪表板显示绿色。

这是什么原因造成的？以后怎么避免呢？

com.freightos.backup.datastore.gcloudapi.GCloudApiDSBackup$CopyEntities run: ERROR in CopyEntities(commerceDocs/RFQ) java.lang.RuntimeException: com.google.cloud.datastore.DatastoreException: I/O error at 
com.freightos.backup.datastore.gcloudapi.GCloudApiDSBackup$CopyEntities.retryIfAllowed(GCloudApiDSBackup.java:963) at 
com.freightos.backup.datastore.gcloudapi.GCloudApiDSBackup$CopyEntities.putEntities(GCloudApiDSBackup.java:857) at 
com.freightos.backup.datastore.gcloudapi.GCloudApiDSBackup$CopyEntities.retryIfAllowed(GCloudApiDSBackup.java:959) at 
com.freightos.backup.datastore.gcloudapi.GCloudApiDSBackup$CopyEntities.putEntities(GCloudApiDSBackup.java:857) at 
com.freightos.backup.datastore.gcloudapi.GCloudApiDSBackup$CopyEntities.putEntitiesByParts(GCloudApiDSBackup.java:991) at 
com.freightos.backup.datastore.gcloudapi.GCloudApiDSBackup$CopyEntities.run(GCloudApiDSBackup.java:801) at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at 
java.util.concurrent.FutureTask.run(FutureTask.java:266) at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at 
java.lang.Thread.run(Thread.java:745) Caused by: com.google.cloud.datastore.DatastoreException: I/O error at 
com.google.cloud.datastore.spi.DefaultDatastoreRpc.translate(DefaultDatastoreRpc.java:105) at 
com.google.cloud.datastore.spi.DefaultDatastoreRpc.commit(DefaultDatastoreRpc.java:133) at 
com.google.cloud.datastore.DatastoreImpl.call(DatastoreImpl.java:390) at 
com.google.cloud.datastore.DatastoreImpl.call(DatastoreImpl.java:387) at 
com.google.cloud.RetryHelper.doRetry(RetryHelper.java:179) at 
com.google.cloud.RetryHelper.runWithRetries(RetryHelper.java:244) at 
com.google.cloud.datastore.DatastoreImpl.commit(DatastoreImpl.java:386) at 
com.google.cloud.datastore.DatastoreImpl.commitMutation(DatastoreImpl.java:380) at 
com.google.cloud.datastore.DatastoreImpl.put(DatastoreImpl.java:340) at 
com.freightos.backup.datastore.gcloudapi.GCloudApiDSBackup$CopyEntities.putEntities(GCloudApiDSBackup.java:836) ... 9 more Caused by: com.google.datastore.v1.client.DatastoreException: I/O error, code=UNAVAILABLE at 
com.google.datastore.v1.client.RemoteRpc.makeException(RemoteRpc.java:126) at 
com.google.datastore.v1.client.RemoteRpc.call(RemoteRpc.java:95) at 
com.google.datastore.v1.client.Datastore.commit(Datastore.java:84) at 
com.google.cloud.datastore.spi.DefaultDatastoreRpc.commit(DefaultDatastoreRpc.java:131) ... 17 more Caused by: java.net.SocketTimeoutException: Timeout while fetching URL: https://datastore.googleapis.com/v1/projects/freightos-prod-backup2:commit at 
com.google.appengine.api.urlfetch.URLFetchServiceImpl.convertApplicationException(URLFetchServiceImpl.java:173) at 
com.google.appengine.api.urlfetch.URLFetchServiceImpl.fetch(URLFetchServiceImpl.java:45) at 
com.google.api.client.extensions.appengine.http.UrlFetchRequest.execute(UrlFetchRequest.java:74) at 
com.google.api.client.http.HttpRequest.execute(HttpRequest.java:981) at 
com.google.datastore.v1.client.RemoteRpc.call(RemoteRpc.java:87) ... 19 more

Answer 1

看起来 UrlFetchTransport 在不应该的时候被使用了。

我已提交 this issue 以修复 google-cloud-java 库。

与此同时，您应该可以在构造 DatastoreOptions 对象时强制它使用 NetHttpTransport：

import com.google.api.client.http.HttpTransport;
import com.google.api.client.http.javanet.NetHttpTransport;
import com.google.auth.http.HttpTransportFactory;

DatastoreOptions.newBuilder()
    .setHttpTransportFactory(new HttpTransportFactory() {
      @Override
      public HttpTransport create() {
        return new NetHttpTransport();
      }
    })
    ...

为什么 Datastore 在获取 URL 时产生超时？

Why does Datastore produce Timeout while fetching URL?

java

google-app-engine

google-cloud-datastore

app-engine-flexible