为什么挂载的 hostPath 在 GKE 的 kubernetes 上不起作用

Question

我在GKE上部署了这2种服务。只是想确认nginx数据是否已经挂载到主机上。

Yaml

Nginx 部署

---
apiVersion: apps/v1
kind: Deployment
metadata:
  namespace: beats
  name: nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.7.9
        ports:
        - containerPort: 80
        volumeMounts:
        - name: nginx-data
          mountPath: /var/log/nginx
      volumes:
      - name: nginx-data
        hostPath:
          path: /var/lib/nginx-data
          type: DirectoryOrCreate

Filebeat

---
apiVersion: v1
kind: ConfigMap
metadata:
  name: filebeat-config
  namespace: beats
  labels:
    k8s-app: filebeat
data:
  filebeat.yml: |-
    filebeat.modules:
      - module: nginx

    filebeat.autodiscover:
      providers:
        - type: kubernetes
          hints.enabled: false
          templates:
            - condition.contains:
                kubernetes.namespace: beats
              config:
                - module: nginx
                  access:
                    enabled: true
                    var.paths: ["/var/lib/nginx-data/access.log*"]
                    subPath: access.log
                    tags: ["access"]
                  error:
                    enabled: true
                    var.paths: ["/var/lib/nginx-data/error.log*"]
                    subPath: error.log
                    tags: ["error"]

    output.logstash:
      hosts: ["logstash.beats.svc.cluster.local:5044"]

---
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: filebeat
  namespace: beats
  labels:
    k8s-app: filebeat
spec:
  selector:
    matchLabels:
      k8s-app: filebeat
  template:
    metadata:
      labels:
        k8s-app: filebeat
    spec:
      serviceAccountName: filebeat
      terminationGracePeriodSeconds: 30
      containers:
        - name: filebeat
          image: docker.elastic.co/beats/filebeat:7.10.0
          args: [
            "-c", "/etc/filebeat.yml",
            "-e",
          ]
          env:
            - name: NODE_NAME
              valueFrom:
                fieldRef:
                  fieldPath: spec.nodeName
          securityContext:
            runAsUser: 0
          resources:
            limits:
              memory: 200Mi
            requests:
              cpu: 100m
              memory: 100Mi
          volumeMounts:
            - name: config
              mountPath: /etc/filebeat.yml
              subPath: filebeat.yml
              readOnly: true
            - name: data
              mountPath: /usr/share/filebeat/data
            - name: varlibdockercontainers
              mountPath: /var/lib/docker/containers
              readOnly: true
            - name: varlog
              mountPath: /var/log
              readOnly: true
            - name: nginx-data
              mountPath: /var/lib/nginx-data
      volumes:
        - name: config
          configMap:
            defaultMode: 0600
            name: filebeat-config
        - name: varlibdockercontainers
          hostPath:
            path: /var/lib/docker/containers
        - name: varlog
          hostPath:
            path: /var/log
        - name: data
          hostPath:
            path: /var/lib/filebeat-data
            type: DirectoryOrCreate
        - name: nginx-data
          hostPath:
            path: /var/lib/nginx-data
            type: DirectoryOrCreate

检查部署

Nginx

kubectl describe po nginx-658f45f77-dpflp -n beats

...
Volumes:
  nginx-data:
    Type:          HostPath (bare host directory volume)
    Path:          /var/lib/nginx-data
    HostPathType:  DirectoryOrCreate

Filebeat pod

kubectl describe po filebeat-42wh7 -n beats

....
Volumes:
  ....
  nginx-data:
    Type:          HostPath (bare host directory volume)
    Path:          /var/lib/nginx-data
    HostPathType:  DirectoryOrCreate

检查 nginx pod

# mount | grep nginx
/dev/sda1 on /var/log/nginx type ext4 (rw,nosuid,nodev,noexec,relatime,commit=30)
/dev/sda1 on /var/cache/nginx type ext4 (rw,nosuid,nodev,relatime,commit=30)

root@nginx-658f45f77-dpflp:/# ls /var/log/nginx/
access.log  error.log

检查 filebeat pod

# mount | grep nginx
/dev/sda1 on /var/lib/nginx-data type ext4 (rw,nosuid,nodev,noexec,relatime,commit=30)

# ls /var/lib/nginx-data
(NULL)

hostPath - /var/lib/nginx-data 不起作用。如果使用minikube，它可以工作。我可以使用 minikube ssh 检查主机上的路径。但是在 GKE 上，如何检查远程机器上的主机路径？

Answer 1

在 GKE（以及来自 public-云提供商的其他托管 Kubernetes 产品）上，您无法直接连接到节点。您必须使用 kubectl exec 之类的调试工具确认内容正在从一个 pod 传输到另一个 pod；因为你运行 filebeat 作为 DaemonSet，所以你需要检查运行在与 nginx pod 相同的节点上的特定 pod。

标准 Docker 集线器 nginx 映像配置为将其日志发送到容器 stdout/stderr（更具体地说，没有卷安装，/var/log/nginx/access.log 是一个符号链接/proc/self/stdout）。在 Kubernetes 环境中，您展示的基本日志收集器设置将能够收集其日志。我只想删除您在这个问题中询问的自定义设置——不要创建 hostPath 目录，不要在容器的 /var/log/nginx 上安装任何东西，也不要有特殊情况这个 pod 的日志收集。

为什么挂载的 hostPath 在 GKE 的 kubernetes 上不起作用

Why mounted hostPath doesn't work on kubernetes of GKE

linux

nginx

kubernetes

google-kubernetes-engine

mount