Spring Data Elasticsearch 是否支持 Amazon Elasticsearch？

Question

根据我所做的研究，这两者似乎不能一起工作，因为 HTTP 仅支持 Amazon Elasticsearch。

希望有人能澄清一下 Spring Data Elasticsearch 是否真的无法做到这一点。

Answer 1

来自不同的讨论：
- Spring data ES and searchly
- port for the transport protocol

和AWS documentation on ES service limitations；转到底部，最后一行说：

The service supports HTTP on port 80, but does not support TCP transport.

尚无法使用 Spring Data ES，它使用 Java API，亚马逊服务只能通过 REST 获得。

Answer 2

有一个不错的项目提供 Spring 与 AWS 托管 ES 服务一起工作的数据 ES 的 Jest 实现。

结帐 https://github.com/VanRoy/spring-data-jest

Answer 3

看起来 Spring 版本 3.2.0 的数据弹性搜索与 http rest 客户端一起工作，因此可以通过 Rest API 和端口 443 连接到 aws elastic 实例。他们以某种方式集成了spring-data-jest 方法进入 spring 数据。我使用 RestHighLevelClient:

    @Bean
    public RestHighLevelClient client() {
        return new RestHighLevelClient(RestClient.builder(HttpHost.create(awsUrl)));
    }

awsUrl 格式为：https://some_aws_generated_address.us-east-n.es.amazonaws.com:443

注意：如果您使用 spring 默认启动 bom.xml，您需要将 spring 启动升级到 2.2.1.RELEASE 或更高版本

Answer 4

It is possible to use Spring Data Elasticsearch with Amazon Elasticsearch

摘自Spring-数据弹性搜索doc

TransportClient is deprecated as of Elasticsearch 7 and will be removed in Elasticsearch 8. This is in favor of Java High Level REST Client. Spring Data Elasticsearch will support the TransportClient as long as it is available in the Elasticsearch.

The Java High Level REST Client now is the default client of Elasticsearch, it provides a straight forward replacement for the TransportClient as it accepts and returns the very same request/response objects and therefore depends on the Elasticsearch core project

Spring 数据 ElasticSearch 已根据 ElasticSearch 的最新标准进行了简化，因此从 spring-data-elasticsearch:3.2.X 开始它提供实现自定义 RestHighLevelClient.(link) 的灵活方式即使可以使用基于 HTTP 的 elastic search API 调用有或没有身份验证，它不会解决与 AWS elastic-search API 调用相关的问题。

因为任何 HTTP 请求 到 AWS 服务或 APIGW 支持的服务 必须遵循 "Signature Version 4 Signing Process(SigV4)" 最终将身份验证信息添加到通过 HTTP 发送的 AWS 请求中。为了安全起见，大多数对 AWS 的请求必须使用访问密钥进行签名，该访问密钥由 accesskey ID 和 secret access key 组成.因此，我们调用AWS ElasticSearch服务时必须遵循标准。

让我们亲自动手编写代码并深入研究实现

请按照以下步骤操作：

第 1 步：添加所需的依赖项

       <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-elasticsearch</artifactId>
            <version>2.2.2.RELEASE</version>
        </dependency>
        <dependency>
            <groupId>com.amazonaws</groupId>
            <artifactId>aws-java-sdk-elasticsearch</artifactId>
            <version>1.11.346</version>
        </dependency>

步骤 2：添加 AWS CredentialsProvider

import com.amazonaws.auth.AWSStaticCredentialsProvider;
import com.amazonaws.auth.BasicAWSCredentials;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
public class AWSCredentialsConfiguration {

    @Value("${aws.es.accessKey}")
    private String esAccessKey = null;

    @Value("${aws.es.secretKey}")
    private String esSecretKey = null;

    @Bean
    public AWSStaticCredentialsProvider awsDynamoCredentialsProviderDevelopment() {
        return new AWSStaticCredentialsProvider(new BasicAWSCredentials(
                esAccessKey, esSecretKey));
    }
}

或者如果您的应用程序运行在 AWS 实例上并且您不想使用 property-driven/hardcoded AccessKey 和 SecretKey 那么您必须分配 IAM 角色 到您的 Amazon ECS 任务 for more.

import com.amazonaws.auth.AWSCredentialsProvider;
import com.amazonaws.auth.EC2ContainerCredentialsProviderWrapper;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
public class AWSCredentialsConfiguration {

    @Bean
    public AWSCredentialsProvider amazonAWSCredentialsProvider() {
        return new EC2ContainerCredentialsProviderWrapper();
    }

}

步骤 3：添加 ElasticSearchRestClientConfiguration

如果您观察下面的代码，我们正在为抽象方法 **AbstractElasticsearchConfiguration#elasticsearchClient()** 提供 **RestHighLevelClient** 的自定义实现。通过这种方式，我们将 "elasticsearchOperations"、"elasticsearchTemplate" bean 的合并 customRestHighLevelClient 注入到 spring容器。
HttpRequestInterceptor 是另一个需要注意的重要事项。特别感谢 AWSRequestSigningApacheInterceptor.java，AWSLabs 提供的示例实现帮助我们使用 [=25= 将拦截器添加到 RestClient ]机制。
@EnableElasticsearchRepositories 注释有助于启用 elasticsearch 数据存储库。

import com.amazonaws.auth.AWS4Signer;
import com.amazonaws.auth.AWSCredentialsProvider;
import org.apache.http.HttpHost;
import org.apache.http.HttpRequestInterceptor;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestHighLevelClient;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.elasticsearch.config.AbstractElasticsearchConfiguration;
import org.springframework.data.elasticsearch.repository.config.EnableElasticsearchRepositories;


@Configuration
@EnableElasticsearchRepositories(basePackages = "com.demo.aws.elasticsearch.data.repository")
public class ElasticSearchRestClientConfiguration extends AbstractElasticsearchConfiguration {

    @Value("${aws.es.endpoint}")
    private String endpoint = null;

    @Value("${aws.es.region}")
    private String region = null;

    @Autowired
    private AWSCredentialsProvider credentialsProvider = null;

    @Override
    @Bean
    public RestHighLevelClient elasticsearchClient() {
        AWS4Signer signer = new AWS4Signer();
        String serviceName = "es";
        signer.setServiceName(serviceName);
        signer.setRegionName(region);
        HttpRequestInterceptor interceptor = new AWSRequestSigningApacheInterceptor(serviceName, signer, credentialsProvider);
        return new RestHighLevelClient(RestClient.builder(HttpHost.create(endpoint)).setHttpClientConfigCallback(e -> e.addInterceptorLast(interceptor)));
    }
}

真棒祖鲁！而已。我们已经完成了配置部分。现在有了这个解决方案，您可以利用 spring-data 弹性优势以及 Amazon 弹性搜索服务。 Medium Post

中记录了完整的解决方案

如果有任何与访问索引权限相关的问题（例如：刷新），您可以使用 answer

添加权限

Answer 5

我在与本文中所示的 aws es stack 相同的配置中收到以下错误 https://medium.com/@prasanth_rajendran/how-to-integrate-spring-boot-elasticsearch-data-with-aws-445e6fc72998

Error creating bean with name 'supplierContacts' defined in file ... Unsatisfied dependency expressed through constructor parameter 1; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'supplierContactListDaoImpl' defined in file [SupplierContactListDaoImpl.class]: Unsatisfied dependency expressed through constructor parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'supplierContactListRepository': Cannot resolve reference to bean 'elasticsearchTemplate' while setting bean property 'elasticsearchOperations'; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'elasticsearchOperations' defined in class path resource [ElasticSearchConfig.class]: Unsatisfied dependency expressed through method 'elasticsearchOperations' parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'elasticsearchEntityMapper' defined in class path resource [ElasticSearchConfig.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.data.elasticsearch.core.convert.ElasticsearchConverter]: Factory method 'elasticsearchEntityMapper' threw exception; nested exception is java.lang.NoClassDefFoundError: org/springframework/data/mapping/model/EntityInstantiators

Spring Data Elasticsearch 是否支持 Amazon Elasticsearch？

Does Spring Data Elasticsearch support Amazon Elasticsearch?

spring

spring-data

spring-data-elasticsearch

amazon-elasticsearch