fluentd 无法连接到 elasticsearch

Question

I 运行堆栈 EFK 通过 docker-compose，堆栈启动，kibana 和 elasticsearch 完成初始设置。 fluentd 在启动时抛出以下错误：

"The client is unable to verify that the server is Elasticsearch. Some functionality may not be compatible if the server is running an unsupported product."

kibana 8.2.0 弹性搜索 8.2.0

可能fluent无法通过https登录，请问哪里有问题？

Dockerfile 流畅：

FROM fluentd:latest

# Use root account to use apk
USER root

# below RUN includes plugin as examples elasticsearch is not required
# you may customize including plugins as you wish
RUN apk add --no-cache --update --virtual .build-deps \
        sudo build-base ruby-dev \
 && sudo gem install fluent-plugin-elasticsearch \
 && sudo gem sources --clear-all \
 && apk del .build-deps \
 && rm -rf /tmp/* /var/tmp/* /usr/lib/ruby/gems/*/cache/*.gem

COPY conf/fluent.conf /fluentd/etc/
COPY entrypoint.sh /bin/

USER fluent

fluentd.conf

<source>
  @type forward
  port 24224
  bind 0.0.0.0
</source>

<match *.**>
  @type elasticsearch
  host elasticsearch
  port 9200
  user fluent
  password 6M9eXThhypVjV8h
  logstash_format true
  logstash_prefix fluentd
  logstash_dateformat %Y%m%d
</match>

fluent 的用户名和密码是在 kibana 中创建的

Answer 1

通过降级EFK版本和修改Dockerfile解决问题

Docker 文件

FROM fluent/fluentd:v1.12.0-debian-1.0
USER root
RUN gem uninstall -I elasticsearch && gem install elasticsearch -v 7.17.0
RUN ["gem", "install", "fluent-plugin-elasticsearch", "--no-document", "--version", "5.0.3"]
USER fluent

docker-compose.yml

## EFK Stack
  fluentd:
    build: ./fluentd
    volumes:
      - ./fluentd/conf/fluent.conf:/fluentd/etc/fluent.conf
    ports:
      - "24224:24224"
      - "24224:24224/udp"
    depends_on:
      - elasticsearch
      - kibana
    networks:
      - efk

  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.13.1
    container_name: elasticsearch
    environment:
      - "discovery.type=single-node"
    expose:
      - "9200"
    ports:
      - "9200:9200"
    networks:
      - efk

  kibana:
    image: docker.elastic.co/kibana/kibana:7.13.1
    ports:
      - "5601:5601"
    depends_on:
      - elasticsearch
    networks:
      - efk

networks:
  efk:
    driver: bridge

fluent.conf

<source>
  @type forward
  port 24224
  bind 0.0.0.0
</source>

<match *.**>
  @type copy

  <store>
    @type elasticsearch
    host elasticsearch
    port 9200
    logstash_format true
    logstash_prefix fluentd
    logstash_dateformat %Y%m%d
    include_tag_key true
    type_name access_log
    tag_key @log_name
    flush_interval 1s
  </store>

  <store>
    @type stdout
  </store>
</match>

fluentd 无法连接到 elasticsearch

fluentd cannot connect to elasticsearch

elasticsearch

docker-compose

efk