Spring 在 Couchbase 中调用删除文档（带分页）时的数据 "TimeoutException"

Question

我们的 Spring Boot 应用正在使用 Couchbase 数据库并使用 Spring-Data

访问它

为了从存储桶中删除记录，我们在存储库中创建了以下方法：

Slice<Dog> deleteAllByOwnerIdAndName(String ownerId, String name, Pageable pageable);

我们在桶上也有相关索引：

CREATE INDEX `dogs_by_ownerId_and_name_idx` ON `dogs`(`ownerId`,`name`) WHERE (`_class` = "com.example.Dog")

我们的代码在尝试删除元素时使用了分页：

 Slice<Dog> dogsSlice = null;
 Pageable pageable = PageRequest.of(0, 1000, Sort.by("id"));
 int pageCounter = 0;
 do {
   log.debug("Deleting page No. {} of dogs", pageCounter++);
   dogsSlice = dogsRepository.deleteAllByOwnerIdAndName("3243242", "Max", pageable);
 } while (dogsSlice.hasNext());

但是，我们得到的次数太多了 Timeoutexceptioin:

Deleting page No. 0 of dogs

o.s.s.s.TaskUtils$LoggingErrorHandler : Unexpected error occurred in scheduled task.

org.springframework.data.couchbase.core.CouchbaseQueryExecutionException: Unable to execute query due to the following n1ql errors: {"msg":"Timeout 7.5s exceeded","code":1080} at org.springframework.data.couchbase.core.CouchbaseTemplate.findByN1QL(CouchbaseTemplate.java:458) ~[classes!/:5.1.40] at org.springframework.data.couchbase.repository.query.AbstractN1qlBasedQuery.executeSliced(AbstractN1qlBasedQuery.java:189) ~[classes!/:5.1.40] at org.springframework.data.couchbase.repository.query.AbstractN1qlBasedQuery.executeDependingOnType(AbstractN1qlBasedQuery.java:129) ~[classes!/:5.1.40] at org.springframework.data.couchbase.repository.query.AbstractN1qlBasedQuery.execute(AbstractN1qlBasedQuery.java:106) ~[classes!/:5.1.40] at org.springframework.data.repository.core.support.RepositoryFactorySupport$QueryExecutorMethodInterceptor.doInvoke(RepositoryFactorySupport.java:590) ~[classes!/:5.1.40] at org.springframework.data.repository.core.support.RepositoryFactorySupport$QueryExecutorMethodInterceptor.invoke(RepositoryFactorySupport.java:578) ~[classes!/:5.1.40] at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185) ~[classes!/:5.1.40] at org.springframework.data.projection.DefaultMethodInvokingMethodInterceptor.invoke(DefaultMethodInvokingMethodInterceptor.java:59) ~[classes!/:5.1.40] at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185) ~[classes!/:5.1.40] at org.springframework.data.couchbase.repository.support.ViewPostProcessor$ViewInterceptor.invoke(ViewPostProcessor.java:87) ~[classes!/:5.1.40] at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185) ~[classes!/:5.1.40] at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92) ~[classes!/:5.1.40] at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185) ~[classes!/:5.1.40] at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92) ~[classes!/:5.1.40] at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185) ~[classes!/:5.1.40] at org.springframework.data.repository.core.support.SurroundingTransactionDetectorMethodInterceptor.invoke(SurroundingTransactionDetectorMethodInterceptor.java:61) ~[classes!/:5.1.40] at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185) ~[classes!/:5.1.40] at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) ~[classes!/:5.1.40] at com.sun.proxy.$Proxy130.deleteAllByOwnerIdAndName(Unknown Source) ~[na:na]

还有什么需要我们做的吗？

Answer 1

您可以改进一些事情：

1) 将您的索引更改为按 ownerId 排序，然后在删除时也按 ownerId 排序

CREATE INDEX `dogs_by_ownerId_and_name_idx` ON `dogs`(`ownerId` ASC,`name`) WHERE (`_class` = "com.example.Dog")

由于您的索引已经排序，couchbase 不会在删除过程中花费额外的时间对其进行排序。

2) 你真的需要 return 所有删除的对象吗？ Couchbase 在将文档发回给您之前必须带上所有不在您的索引中的属性，并且此操作将花费一些额外的时间。最好的方法是 return 只有 id。

@Override
public void updateFamilyName(String familyName, String familyId) {

    String queryString = "Delete from "+getBucketName()+" WHERE "+getClassFilter()+" " +
            " and familyId = '"+familyId+"' RETURNING meta().id";

    N1qlParams params = N1qlParams.build().consistency(ScanConsistency.REQUEST_PLUS).adhoc(true);
    ParameterizedN1qlQuery query = N1qlQuery.parameterized(queryString, JsonObject.create(), params);
    checklistRepository.getCouchbaseOperations().getCouchbaseBucket().query(query);
}

private String getBucketName(){
    return checklistRepository.getCouchbaseOperations().getCouchbaseBucket().bucketManager().info().name();
}

private String getClassFilter(){
    return "_class = '" + Checklist.class.getName() + "' ";
}

3) 您还可以改进分页，但我认为您的情况没有必要。

https://blog.couchbase.com/offset-keyset-pagination-n1ql-query-couchbase/

Answer 2

Sort.by("id") 导致查询延迟，因为 Couchbase 似乎正在按该标准对整个文档集合进行排序。

所以如果没有真正需要对结果进行排序，最好使用

Pageable pageable = PageRequest.of(0, 1000);

Answer 3

如果查询有 ORDER BY，如果可能优化器尝试使用索引顺序。如果不可能，它必须生成所有可能的数据集并对数据进行排序以满足查询，即使分页需要很少的项目。

检查规则 #7 https://blog.couchbase.com/create-right-index-get-right-performance/

另见这篇文章https://blog.couchbase.com/offset-keyset-pagination-n1ql-query-couchbase/

Spring 在 Couchbase 中调用删除文档（带分页）时的数据 "TimeoutException"

Spring Data "TimeoutException" when calling delete documents (with pagination) in Couchbase

java

couchbase

spring-data

spring-boot

n1ql