Logstash 不使用 Shield 写入 Elasticsearch

Question

我一直在尝试使用 shield 使 logstash 写入 elasticseach，但没有成功。

在将 shield 插件安装到 elasticsearch 之前，我的设置运行正常。我已关注 this guide from elastic.co 并使用以下方法为 logstash user role 创建了一个新用户：

esusers useradd logstashadmin -r logstash

我还按照指南中的建议更新了 logstash output configuration 并添加了 protocol、user 和 password。

重新启动 logstash 和 elasticsearch 后，我仍然没有收到来自 logstash 的关于 elasticsearch 的任何信息。我错过了什么吗？

这是我的设置：

$ esusers roles logstashadmin                         
logstashadmin  : logstash

$ cat shield/roles.yml
...
# The required role for logstash users
logstash:
  cluster: indices:admin/template/get, indices:admin/template/put
  indices:
    'logstash-*': indices:data/write/bulk, indices:data/write/delete, indices:data/write/update, indices:data/read/search, indices:data/read/scroll, create_index
...


$ cat logstash/output.conf

output {
  elasticsearch {
    protocol => "http"
    cluster => "logstash"
    user => "logstashadmin"
    password => "logstashadmin123"
  }
}

注意：我还在 logstash 中安装了 transport 插件并尝试用它代替 protocol => "http"，但结果相同。

如果您需要更多信息，请告诉我。谢谢

编辑 1：

弹性搜索日志：

[2015-06-12 05:59:16,952][INFO ][node                     ] [Silver Sable] stopping ...
[2015-06-12 05:59:17,087][INFO ][shield.license           ] [Silver Sable] DISABLING LICENSE FOR [shield]
[2015-06-12 05:59:17,088][INFO ][node                     ] [Silver Sable] stopped
[2015-06-12 05:59:17,088][INFO ][node                     ] [Silver Sable] closing ...
[2015-06-12 05:59:17,104][INFO ][node                     ] [Silver Sable] closed
[2015-06-12 05:59:20,479][INFO ][node                     ] [Lionheart] version[1.4.5], pid[28662], build[2aaf797/2015-04-27T08:06:06Z]
[2015-06-12 05:59:20,480][INFO ][node                     ] [Lionheart] initializing ...
[2015-06-12 05:59:20,586][INFO ][plugins                  ] [Lionheart] loaded [license, shield], sites []
[2015-06-12 05:59:21,301][INFO ][transport                ] [Lionheart] Using [org.elasticsearch.shield.transport.ShieldServerTransportService] as transport service, overridden by [shield]
[2015-06-12 05:59:21,301][INFO ][transport                ] [Lionheart] Using [org.elasticsearch.shield.transport.netty.ShieldNettyTransport] as transport, overridden by [shield]
[2015-06-12 05:59:21,301][INFO ][http                     ] [Lionheart] Using [org.elasticsearch.shield.transport.netty.ShieldNettyHttpServerTransport] as http transport, overridden by [shield]
[2015-06-12 05:59:27,166][INFO ][node                     ] [Lionheart] initialized
[2015-06-12 05:59:27,166][INFO ][node                     ] [Lionheart] starting ...
[2015-06-12 05:59:28,148][INFO ][shield.transport         ] [Lionheart] bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/10.1.0.25:9300]}
[2015-06-12 05:59:28,209][INFO ][discovery                ] [Lionheart] logstash/uuDCpM6hTKKvLNd2oFGYpA
[2015-06-12 05:59:32,032][INFO ][cluster.service          ] [Lionheart] new_master [Lionheart][uuDCpM6hTKKvLNd2oFGYpA][0ba2a1c6e1de][inet[/10.1.0.25:9300]], reason: zen-disco-join (elected_as_master)
[2015-06-12 05:59:32,119][INFO ][http                     ] [Lionheart] bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address {inet[/10.1.0.25:9200]}
[2015-06-12 05:59:32,119][INFO ][node                     ] [Lionheart] started
[2015-06-12 05:59:33,007][INFO ][shield.license           ] [Lionheart] enabling license for [shield]
[2015-06-12 05:59:33,013][INFO ][license.plugin.core      ] [Lionheart] license for [shield] - valid
[2015-06-12 05:59:33,028][ERROR][shield.license           ] [Lionheart] 
#
# Shield license will expire on [Saturday, July 11, 2015]. Cluster health, cluster stats and indices stats operations are
# blocked on Shield license expiration. All data operations (read and write) continue to work. If you
# have a new license, please update it. Otherwise, please reach out to your support contact.
#
[2015-06-12 05:59:33,162][INFO ][gateway                  ] [Lionheart] recovered [2] indices into cluster_state

Logstash 日志：（这部分重复了很多次）

NotImplementedError: block device detection unsupported or native support failed to load
       blockdev? at org/jruby/RubyFileTest.java:67
         device? at /opt/logstash/vendor/bundle/jruby/1.9/gems/filewatch-0.6.2/lib/filewatch/helper.rb:67
  _sincedb_write at /opt/logstash/vendor/bundle/jruby/1.9/gems/filewatch-0.6.2/lib/filewatch/tail.rb:230
   sincedb_write at /opt/logstash/vendor/bundle/jruby/1.9/gems/filewatch-0.6.2/lib/filewatch/tail.rb:203
        teardown at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-file-0.1.10/lib/logstash/inputs/file.rb:151
     inputworker at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.0-java/lib/logstash/pipeline.rb:203
     synchronize at org/jruby/ext/thread/Mutex.java:149
     inputworker at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.0-java/lib/logstash/pipeline.rb:203
     start_input at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.0-java/lib/logstash/pipeline.rb:171

Answer 1

我认为这是一个与屏蔽无关的问题。检查这个问题： https://github.com/elastic/logstash/issues/3127

就像 post 提到的那样，执行以下命令对我有用：

ln -s /lib/x86_64-linux-gnu/libcrypt.so.1 /usr/lib/x86_64-linux-gnu/libcrypt.so

Logstash 不使用 Shield 写入 Elasticsearch

Logstash not writing to Elasticsearch with Shield

elasticsearch

logstash

elasticsearch-plugin

logstash-configuration

elasticsearch-shield

编辑 1：