为什么 Fluentd Azure blob 插件在 kubernetes 中不起作用

Question

我已经使用以下命令 helm install fluentd bitnami-azure/fluentd --namespace mynamespace --set forwarder.configMap=fluentd-aksconfig 在我的 AKS 群集中安装了 fluentd。

下面是我的configmap.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: fluentd-aksconfig
  namespace: mynamespace
data:
  fluentd.conf: |
    <match **>
          @type azure-storage-append-blob
          azure_storage_account    mystorageaccount
          azure_storage_access_key myaccesskey
          azure_container          aksfluentd
          auto_create_container    true
          path logs/
          azure_object_key_format  %{path}%{time_slice}_%{index}.log
          time_slice_format        %Y%m%d-%H
          # if you want to use %{tag} or %Y/%m/%d/ like syntax in path / azure_blob_name_format,
          # need to specify tag for %{tag} and time for %Y/%m/%d in <buffer> argument.
          <buffer tag,time>
            @type file
            path /var/log/fluent/azurestorageappendblob
            timekey 120 # 2 minutes
            timekey_wait 60
            timekey_use_utc true # use utc
          </buffer>
    </match>

下面是从 pods 中得到的错误。它显示没有名为 "azure-storage-append-blob" 的插件。如何解决这个问题？

日志：

[38;5;6mfluentd [38;5;5m10:03:45.15 [0m [38;5;6mfluentd [38;5;5m10:03:45.15 [0m[1mWelcome to the Bitnami fluentd container[0m [38;5;6mfluentd [38;5;5m10:03:45.16 [0mSubscribe to project updates by watching [1mhttps://github.com/bitnami/bitnami-docker-fluentd[0m [38;5;6mfluentd [38;5;5m10:03:45.16 [0mSubmit issues and feature requests at [1mhttps://github.com/bitnami/bitnami-docker-fluentd/issues[0m [38;5;6mfluentd [38;5;5m10:03:45.16 [0mSend us your feedback at [1mcontainers@bitnami.com[0m [38;5;6mfluentd [38;5;5m10:03:45.16 [0m [38;5;6mfluentd [38;5;5m10:03:45.16 [0m[38;5;2mINFO [0m ==> ** Starting Fluentd setup ** [38;5;6mfluentd [38;5;5m10:03:45.38 [0m[38;5;2mINFO [0m ==> ** Fluentd setup finished! ** [38;5;6mfluentd [38;5;5m10:03:45.39 [0m[38;5;2mINFO [0m ==> ** Starting Fluentd ** 2020-02-18 10:03:46 +0000 [info]: parsing config file is succeeded path="/opt/bitnami/fluentd/conf/fluentd.conf" 2020-02-18 10:03:46 +0000 [info]: gem 'fluent-plugin-concat' version '2.4.0' 2020-02-18 10:03:46 +0000 [info]: gem 'fluent-plugin-detect-exceptions' version '0.0.13' 2020-02-18 10:03:46 +0000 [info]: gem 'fluent-plugin-elasticsearch' version '4.0.1' 2020-02-18 10:03:46 +0000 [info]: gem 'fluent-plugin-kafka' version '0.12.2' 2020-02-18 10:03:46 +0000 [info]: gem 'fluent-plugin-kubernetes_metadata_filter' version '2.4.2' 2020-02-18 10:03:46 +0000 [info]: gem 'fluent-plugin-multi-format-parser' version '1.0.0' 2020-02-18 10:03:46 +0000 [info]: gem 'fluent-plugin-prometheus' version '1.7.0' 2020-02-18 10:03:46 +0000 [info]: gem 'fluent-plugin-s3' version '1.2.1' 2020-02-18 10:03:46 +0000 [info]: gem 'fluent-plugin-systemd' version '1.0.2' 2020-02-18 10:03:46 +0000 [info]: gem 'fluentd' version '1.9.1' 2020-02-18 10:03:46 +0000 [error]: config error file="/opt/bitnami/fluentd/conf/fluentd.conf" error_class=Fluent::ConfigError error="Unknown output plugin 'azure-storage-append-blob'. Run 'gem search -rd fluent-plugin' to find plugins"

Answer 1

azure-storage-append-blob plugin is not installed by default. So you need to install it. Docs on how to install Fluentd plugin is here.

Answer 2

正如@ArghyaSadhu 在他的回答中所说，Currently fluentd docker image 没有预装 azure blob 插件。我们可以获取docker图像并对其进行自定义。关于如何自定义 fluentd 的 docker 图像，请参考以下 link。 https://github.com/bitnami/bitnami-docker-fluentd#customize-this-image

为什么 Fluentd Azure blob 插件在 kubernetes 中不起作用

Why is Fluentd Azure blob plugin not working in kubernetes

azure

kubernetes

azure-blob-storage

fluentd

azure-aks