Httpbeat 指标未显示在 Kibana 中

Question

亲爱的，

我是 Kibana/Elasticsearch/Httpbeat 的新手，设置它让我有点头疼...

Httpbeat 运行s 并将数据泵入 Elasticsearch：

虽然，当我尝试创建可视化时，我运行遇到了麻烦；不知何故数据不存在......

这也可能有用：

和模板json：

{
  "mappings": {
    "_default_": {
      "_meta": {
        "version": "5.4.0"
      },
      "dynamic_templates": [
        {
          "strings_as_keyword": {
            "mapping": {
              "ignore_above": 1024,
              "type": "keyword"
            },
            "match_mapping_type": "string"
          }
        }
      ],
      "properties": {
        "@timestamp": {
          "type": "date"
        },
        "beat": {
          "properties": {
            "hostname": {
              "ignore_above": 1024,
              "type": "keyword"
            },
            "name": {
              "ignore_above": 1024,
              "type": "keyword"
            },
            "version": {
              "ignore_above": 1024,
              "type": "keyword"
            }
          }
        },
        "meta": {
          "properties": {
            "cloud": {
              "properties": {
                "availability_zone": {
                  "ignore_above": 1024,
                  "type": "keyword"
                },
                "instance_id": {
                  "ignore_above": 1024,
                  "type": "keyword"
                },
                "machine_type": {
                  "ignore_above": 1024,
                  "type": "keyword"
                },
                "project_id": {
                  "ignore_above": 1024,
                  "type": "keyword"
                },
                "provider": {
                  "ignore_above": 1024,
                  "type": "keyword"
                },
                "region": {
                  "ignore_above": 1024,
                  "type": "keyword"
                }
              }
            }
          }
        },
        "request": {
          "properties": {
            "body": {
              "ignore_above": 1024,
              "type": "keyword"
            },
            "headers": {
              "properties": {},
              "type": "nested"
            },
            "method": {
              "ignore_above": 1024,
              "type": "keyword"
            },
            "url": {
              "ignore_above": 1024,
              "type": "keyword"
            }
          }
        },
        "response": {
          "properties": {
            "body": {
              "ignore_above": 1024,
              "type": "keyword"
            },
            "code": {
              "ignore_above": 1024,
              "type": "keyword"
            },
            "headers": {
              "properties": {},
              "type": "nested"
            },
            "jsonBody": {
              "properties": {
                  "globalTime": {
                      "type": "long"
                  }
              }
            }
          }
        },
        "tags": {
          "ignore_above": 1024,
          "type": "keyword"
        }
      }
    }
  },
  "order": 0,
  "settings": {
    "index.mapping.total_fields.limit": 10000,
    "index.refresh_interval": "1m"
  },
  "template": "httpbeat-*"
}

httpbeat.yml

######################## Httpbeat Configuration Example ########################

############################## Httpbeat ########################################
httpbeat:

  hosts:
    # Each - Host endpoints to call. Below are the host endpoint specific configurations
    -
      # Optional cron expression, defines when to poll the host endpoint.
      # Default is every 1 minute.
      schedule: "@every 1m"

      # The URL endpoint to call by Httpbeat
      url: (a correct url)

      # HTTP method to use.
      # Possible options are:
      # * get
      # * delete
      # * head
      # * patch
      # * post
      # * put
      method: get

      # Optional additional headers to send to the endpoint
      #headers:
        #Accept: application/json

      # Optional basic authentication
      basic_auth:
        # Basic authentication username
        username: theetsa
        # Basic authentication password
        password: (a very secret password)

      # Type to be published in the 'type' field. For Elasticsearch output,
      # the type defines the document type these entries should be stored
      # in. Default: httpbeat
      #document_type:

      # Optional output format for the response body.
      # Possible options are:
      # * string
      # * json
      # Default output format is 'string'
      output_format: json

      # Optional convertion of dots in keys in JSON response body. By default is off.
      # Possible options are:
      # * replace - replaces dots with a different character. The default value is `_`.
      # * unflatten - converts {"foo.bar":false} to {"foo":{"bar":false}}
      #json_dot_mode: replace

      # Optional additional headers to send to the endpoint
      #headers:
        #Accept: application/json
      # Enable SSL support. SSL is automatically enabled, if any SSL setting is set.
      #ssl.enabled: true

      # Configure SSL verification mode. If `none` is configured, all server hosts
      # and certificates will be accepted. In this mode, SSL based connections are
      # susceptible to man-in-the-middle attacks. Use only for testing. Default is
      # `full`.
      #ssl.verification_mode: full

      # List of supported/valid TLS versions. By default all TLS versions 1.0 up to
      # 1.2 are enabled.
      #ssl.supported_protocols: [TLSv1.0, TLSv1.1, TLSv1.2]

      # Optional SSL configuration options. SSL is off by default.
      # List of root certificates for HTTPS server verifications
      #ssl.certificate_authorities: ["/etc/pki/root/ca.pem"]

      # Certificate for SSL client authentication
      #ssl.certificate: "/etc/pki/client/cert.pem"

      # Client Certificate Key
      #ssl.key: "/etc/pki/client/cert.key"

      # Optional passphrase for decrypting the Certificate Key.
      #ssl.key_passphrase: ''

      # Configure cipher suites to be used for SSL connections
      #ssl.cipher_suites: []

      # Configure curve types for ECDHE based cipher suites
      #ssl.curve_types: []


#================================ General =====================================

# The name of the shipper that publishes the network data. It can be used to group
# all the transactions sent by a single shipper in the web interface.
#name:

# The tags of the shipper are included in their own field with each
# transaction published.
#tags: ["service-X", "web-tier"]

# Optional fields that you can specify to add additional information to the
# output.
#fields:
#  env: staging

#================================ Outputs =====================================

# Configure what outputs to use when sending the data collected by the beat.
# Multiple outputs may be used.

#-------------------------- Elasticsearch output ------------------------------
output.elasticsearch:
  # Array of hosts to connect to.
  hosts: ["localhost:9200"]

  # Optional protocol and basic auth credentials.
  #protocol: "https"
  #username: "elastic"
  #password: "changeme"

#----------------------------- Logstash output --------------------------------
#output.logstash:
  # The Logstash hosts
  #hosts: ["localhost:5044"]

  # Optional SSL. By default is off.
  # List of root certificates for HTTPS server verifications
  #ssl.certificate_authorities: ["/etc/pki/root/ca.pem"]

  # Certificate for SSL client authentication
  #ssl.certificate: "/etc/pki/client/cert.pem"

  # Client Certificate Key
  #ssl.key: "/etc/pki/client/cert.key"

#================================ Logging =====================================

# Sets log level. The default log level is info.
# Available log levels are: critical, error, warning, info, debug
#logging.level: debug

# At debug level, you can selectively enable logging only for some components.
# To enable all selectors use ["*"]. Examples of other selectors are "beat",
# "publish", "service".
#logging.selectors: ["*"]

我真的不知道我做错了什么:-/

我尝试使用与 Metricbeat 中相同的设置；在图表有效的地方，我也查看了日志，但在那里找不到任何有用的东西...

我注意到beat版本是4.0.0；这可能是问题所在，我真的不知道 :-/

感谢任何帮助或指点...

S.

Answer 1

我不确定是什么把戏，但我

已停止 httpBeat
已停止 elasticSearch
删除所有索引
rm -Rf data/nodes/0/*
重启elasticSearch
使用了这个模板：

httpbeat.template-es2x.json:

{
  "mappings": {
    "my_type": {
        "_meta": {
                "version": "5.4.0"
        },
        "dynamic_templates": [
          {
            "integers": {
              "match_mapping_type": "long",
              "mapping": {
                "type": "integer"
              }
            }
          }
        ],
        "properties": {
                "@timestamp": {
                  "type": "date"
                },
                "response": {
                        "properties": {
                                "jsonBody": {
                                        "properties": {
                                                "globalTime": {
                                                        "type": "long"
                                                }
                                        }
                                }
                        }
                }
        },
        "fields": {
          "properties": {}
        }
             -> more about this below...
    }
  },
  "order": 0,
  "settings": {
    "index.mapping.total_fields.limit": 10000,
    "index.refresh_interval": "1m"
  },
  "template": "httpbeat-*"
}

然后重新启动一切

我认为主要是 'fields' 很重要；当我在没有使用模板的情况下使用时，我在 Kibana 中遇到了关于 'fields' 和：

的错误

        "fields": {
          "properties": {}
        }

是存在于 metricbeat.template-es2x.json 而不是 httpbeat.template-es2x.json 的东西，它似乎与 httpbeat.template- 中的那个字段一起工作es2x.json 而不是 httpbeat.template.json...

格茨，

S.

ps：如果您的答案不是基于试错，我会接受这个答案而不是这个...

Httpbeat 指标未显示在 Kibana 中

Httpbeat Metrics not showing up in Kibana

kibana

kibana-5