Google Cloud Container Optimized OS 主机日志到 stackdriver

Question

TL;DR
将容器优化 os host 日志（ssh 和执行的 shell 命令）发送到 Stackdriver 的最佳做法是什么？

背景：
我正在使用 Googles Container Optimized OS，效果很好。将容器日志发送到 Stackdriver 非常容易，但是如何将 host 日志发送到 Stackdriver？

它用于审计 purposes，我需要记录所有 SSH 连接（接受或拒绝）和通过 shell 执行的所有命令。以前我只是通过 stackdriver host 记录器包将 rsyslogd (auth,authpriv) 发送到 stackdriver。

这是针对托管实例组 (mig) 中的容器优化 OS VM:s 运行，而不是 Google Kubernetes Engine。

它可能非常明显，但我似乎找不到任何关于它的文档。

Answer 1

Google 团队回答：

To have journald logs in stackdriver, you will need to configure Fluentd to do so. See these lines of fluentd configmap for some examples. Note that the "node-journal" filter in the configmap is not enabled by default on GKE.

To have audits on COS nodes, additionally, you will need to enable COS audit logging system service. On COS nodes: Run "systemctl start cloud-audit-setup". Then you will have audit logs like SSH login in the journald log.

Answer 2

在高层次上，这就是您需要为任何 GCP COS 实例执行的操作，以将 OS 审计日志发送到 Google stackdriver:

首先，您需要使用以下命令在 COS 上启用审计日志： systemctl 启动云审核设置 这将允许在计算实例日志中生成和捕获审计日志，您可以使用 journalctl 命令查看结果

其次，您需要在实例上安装 Google Stackdriver 代理并配置为将审计日志从实例日志发送到堆栈驱动程序。这可以通过 docker 容器 运行 fluentd-gcp google 容器镜像来实现。

我正在分享下面的 cloud-init 来为你完成整个工作。您需要做的就是拥有一个带有键 "user-data" 且值为以下脚本的实例元数据：

#cloud-config
users:
- name: logger
  uid: 2001
  groups: docker

write_files:

- path: /etc/google-fluentd/fluentd.conf
  permissions: 0644
  owner: root
  content: |
    # This config comes from a heavily trimmed version of the
    # container-engine-customize-fluentd project. The upstream config is here:
    # https://github.com/GoogleCloudPlatform/container-engine-customize-fluentd/blob/6a46d72b29f3d8e8e495713bc3382ce28caf744e/kubernetes/fluentd- 
configmap.yaml
    <source>
        type systemd
        path /var/log/journal
        pos_file /var/log/gcp-journald.pos
        filters [{ "SYSLOG_IDENTIFIER": "audit" }]  
        tag node-journal
        read_from_head true
    </source>
    <match **>
      @type copy
       <store>
        @type google_cloud
        # Set the buffer type to file to improve the reliability
        # and reduce the memory consumption
        buffer_type file
        buffer_path /var/log/google-fluentd/cos-system.buffer
        # Set queue_full action to block because we want to pause gracefully
        # in case of the off-the-limits load instead of throwing an exception
        buffer_queue_full_action block
        # Set the chunk limit conservatively to avoid exceeding the GCL limit
        # of 10MiB per write request.
        buffer_chunk_limit 2M
        # Cap the combined memory usage of this buffer and the one below to
        # 2MiB/chunk * (6 + 2) chunks = 16 MiB
        buffer_queue_limit 6
        # Never wait more than 5 seconds before flushing logs in the non-error
        # case.
        flush_interval 5s
        # Never wait longer than 30 seconds between retries.
        max_retry_wait 30
        # Disable the limit on the number of retries (retry forever).
        disable_retry_limit
        # Use multiple threads for processing.
        num_threads 2
      </store>
    </match>
- path: /etc/systemd/system/logger.service
  permissions: 0644
  owner: root
  content: |
    [Unit]
    Description=logging docker container
    Requires=network-online.target
    After=network-online.target

    [Service]
    Environment="HOME=/home/logger"
    ExecStartPre=/usr/share/google/dockercfg_update.sh
    ExecStartPre=/bin/mkdir -p /var/log/google-fluentd/
    ExecStartPre=-/usr/bin/docker rm -fv logger
    ExecStart=/usr/bin/docker run --rm -u 0 \
       --name=logger \
       -v /var/log/:/var/log/ \
       -v /var/lib/docker/containers:/var/lib/docker/containers \
       -v /etc/google-fluentd/:/etc/fluent/config.d/ \
       --env='FLUENTD_ARGS=-q' \
       gcr.io/google-containers/fluentd-gcp:2.0.17
    Restart=always
    RestartSec=1
runcmd:
- systemctl daemon-reload
- systemctl start logger.service
- systemctl start cloud-audit-setup

Answer 3

how do I send host logs to Stackdriver?

Here 是 COS 打包 Stackdriver Logging 代理的一些代码。您可以通过 sudo systemctl start stackdriver-logging.

启动它