未在 Filebeat 中将索引名称设置为 Elasticsearch - ELK .NET Docker ElasticHQ

Question

我正在试验一些根据 Elasticsearch 格式化的 json，所以我直接从 Filebeat 转到 Elasticsearch，而不是通过 [=16] =].这是使用 docker-compose:

version: '2.2'
services:
  elasticsearch:
    container_name: elasticsearch
    image: docker.elastic.co/elasticsearch/elasticsearch:7.5.2
    ports:
      - 9200:9200
      - 9300:9300
    environment:
      - discovery.type=single-node
      - cluster.name=docker-
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    networks:
      - esnet
  filebeat:
    container_name: filebeat
    build:
      context: .
      dockerfile: filebeat.Dockerfile
    volumes:
      - ./logs:/var/log
      - ./filebeat/filebeat.yml:/usr/share/filebeat/filebeat.yml
    networks:
      - esnet  
  elastichq:
    container_name: elastichq
    image: elastichq/elasticsearch-hq
    ports:
      - 8080:5000
    environment:
      - HQ_DEFAULT_URL=http://elasticsearch:9200
      - HQ_ENABLE_SSL=False
      - HQ_DEBUG=FALSE
    networks:
      - esnet  
networks:
  esnet:

但是，当我打开 ElasticHQ 时，索引名称已被标记为 filebeat-7.5.2-2020.02.10-000001 并带有日期戳。我在 filebeat.yml 中将 index 名称指定为 Sample。是不是我遗漏了什么，或者这种行为正常吗？

这是我的 filebeat.yml

filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /var/log/*.json
  json.keys_under_root: true
  json.add_error_key: true 

#----------------------------- Elasticsearch output --------------------------------

output.elasticsearch:
  hosts: ["elasticsearch:9200"]
  index: "sample-%{+YYYY.MM.dd}"


setup.template.name: "sample"
setup.template.pattern: "sample-*"

知道一些预定义的东西会更实用，所以如果我使用 Postman 而不是 ElasticHQ，我可以开始查询我的数据而无需查找 index 名称。

Answer 1

我认为 Filebeat ILM 可能会取代配置的索引名称。

Starting with version 7.0, Filebeat uses index lifecycle management by default when it connects to a cluster that supports lifecycle management. Filebeat loads the default policy automatically and applies it to any indices created by Filebeat.

并且当 ilm 启用时 Filebeat Elasticsearch output 索引设置被忽略

The index setting is ignored when index lifecycle management is enabled. If you’re sending events to a cluster that supports index lifecycle management, see Configure index lifecycle management to learn how to change the index name.

您可能需要禁用 ILM 或更好地使用 ILM 配置所需的文件名 rollover_alias。

未在 Filebeat 中将索引名称设置为 Elasticsearch - ELK .NET Docker ElasticHQ

Index Name Not Being Set in Filebeat to Elasticsearch - ELK .NET Docker ElasticHQ

elasticsearch

docker-compose

filebeat