可以使用 Python 的请求访问 Elasticsearch 但不能使用官方客户端

Question

我在 Kubernetes 上有一个 Elasticsearch DB 运行作为 Istio 虚拟服务暴露给 my_domain.com/elastic，我可以通过浏览器访问它没有问题（因为我成功登录到端点).我还可以使用 Python 的请求查询数据库。但是如果我使用 my_domain.com/elastic，我就无法使用官方 python 客户端访问数据库。 LoadBalancer IP 即使在客户端也能完美运行。我错过了什么？我通过 Cert-Manager 和 CloudFlare 为 my_domain.com 设置了 SSL 证书。

这个有效：

import requests
import os
data = ' { "query": { "match_all": {} } }'
headers = {'Content-Type': 'application/json'}
auth= ('elastic', os.environ['ELASTIC_PASSWORD']) 
response = requests.post('https://mydomain.cloud/elastic/_search', auth=auth,  data=data, headers=headers)
print(response.text)

这行不通（我尝试了很多不同的参数）：

from datetime import datetime
import os
from elasticsearch import Elasticsearch, RequestsHttpConnection


es = Elasticsearch(,
   [{'host': 'mydomain.cloud', 'port': 443, 'url_prefix': 'elastic', 'use_ssl': True}],
    http_auth=('elastic', os.environ['ELASTIC_PASSWORD']), # 1Password or kubectl get secret elastic-cluster-es-elastic-user -o go-template='{{.data.elastic | base64decode}}' -n elastic-system
    schema='https'#, verify_certs=False,
    # use_ssl=True,
    # connection_class = RequestsHttpConnection,
    # port=443,
)

# if not es.ping():
#     raise ValueError("Connection failed")

doc = {
    'author': 'kimchy',
    'text': 'Elasticsearch: cool. bonsai cool.',
    'timestamp': datetime.now(),
}
res = es.index(index="test-index", id=1, document=doc)
print(res['result'])

res = es.get(index="test-index", id=1)
print(res['_source'])

es.indices.refresh(index="test-index")

res = es.search(index="test-index", query={"match_all": {}})
print("Got %d Hits:" % res['hits']['total']['value'])
for hit in res['hits']['hits']:
    print("%(timestamp)s %(author)s: %(text)s" % hit["_source"])

产生的错误：

elasticsearch.exceptions.RequestError: RequestError(400, 'no handler found for uri [//test-index/_doc/1] and method [PUT]', 'no handler found for uri [//test-index/_doc/1] and method [PUT]')

cluster.yaml

apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
  name: elastic-cluster
  namespace: elastic-system
spec:
  version: 7.15.2
  http:
    # tls:
    #   selfSignedCertificate:
    #      disabled: true
    service:
      spec:
        type: LoadBalancer
  nodeSets:
  - name: master-nodes
    count: 2
    config:
      node.roles: ["master"]
    volumeClaimTemplates:
    - metadata:
        name: elasticsearch-data # Do not change this name unless you set up a volume mount for the data path.
      spec:
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: 5Gi
        storageClassName: local-storage
  - name: data-nodes
    count: 2
    config:
      node.roles: ["data"]
    volumeClaimTemplates:
    - metadata:
        name: elasticsearch-data # Do not change this name unless you set up a volume mount for the data path.
      spec:
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: 10Gi
        storageClassName: local-storage
    podTemplate:
      # metadata:
      #   annotations:
      #     traffic.sidecar.istio.io/includeInboundPorts: "*"
      #     traffic.sidecar.istio.io/excludeOutboundPorts: "9300" 
      #     traffic.sidecar.istio.io/excludeInboundPorts: "9300"
      spec:
        # automountServiceAccountToken: true
        containers:
        - name: elasticsearch
          resources:
            requests:
              memory: 4Gi
              cpu: 3
            limits:
              memory: 4Gi
              cpu: 3

虚拟-service.yaml

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: elastic-vts
  namespace: elastic-system
spec:
  hosts:
  - "mydomain.cloud"
  gateways:
  - istio-system/gateway
  http:
  - match:
    - port: 443
    - uri:
        prefix: /elastic
    rewrite:
      uri: /
    route:
    - destination:
        host: elastic-cluster-es-http.elastic-system.svc.cluster.local
        port:
          number: 9200

目的地-rule.yaml

apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: elastic-destination-rule
  namespace: elastic-system
spec:
  host: elastic-cluster-es-http.elastic-system.svc.cluster.local
  trafficPolicy:
    tls:
      mode: SIMPLE

gateway.yaml

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: gateway
  namespace: istio-system
spec:
  selector:
    istio: ingressgateway
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - 'mydomain.cloud'
    tls:
      httpsRedirect: true
  - port:
      number: 443
      name: https
      protocol: HTTPS
    hosts:
      - 'mydomain.cloud'
    tls:
      mode: SIMPLE
      credentialName: letsencrypt-staging-tls

Answer 1

我已经复现了你的问题，解决方法如下。首先，注意你的yaml文件：

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: elastic-vts
  namespace: elastic-system
spec:
  hosts:
  - "mydomain.cloud"
  gateways:
  - istio-system/gateway
  http:
  - match:
    - port: 443
    - uri:
        prefix: /elastic   <---- here is the problem
    rewrite:
      uri: /
      ...

出现的错误如下所示：

elasticsearch.exceptions.RequestError: RequestError(400, 'no handler found for uri [//test-index/_doc/1] and method [PUT]', 'no handler found for uri [//test-index/_doc/1] and method [PUT]')

问题就在那里：[//test-index/_doc/1]（关于重复/字符）。我认为这与提到的问题 here 类似。要解决此问题，我建议将 / 添加到 prefix: /elastic 行，您的 yaml 将类似于此示例：

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: elastic-vts
  namespace: elastic-system
spec:
  hosts:
  - "mydomain.cloud"
  gateways:
  - istio-system/gateway
  http:
  - match:
    - port: 443
    - uri:
        prefix: /elastic/ <---- here
    rewrite:
      uri: /
      ...

此时 Elastic 的答案如下：

Got 1 Hits:
2021-12-30T09:20:12.038004 kimchy: Elasticsearch: cool. bonsai cool.

可以使用 Python 的请求访问 Elasticsearch 但不能使用官方客户端

Able to access Elasticsearch with Python's Requests but not with official client

http

elasticsearch

kubernetes

istio