如何在 istio kubernetes 集群中拒绝默认但允许 HTTP 和 TCP 流量？

Question

我有一个启用了 istio 注入并定义了 cockroach db 有状态集的集群：

apiVersion: v1
kind: ServiceAccount
metadata:
  name: cockroachdb-serviceaccount
---
apiVersion: v1
kind: Service
metadata:
  # This service is meant to be used by clients of the database. It exposes a ClusterIP that will
  # automatically load balance connections to the different database pods.
  name: cockroachdb-public
  labels:
    app: cockroachdb
spec:
  ports:
  # The main port, served by gRPC, serves Postgres-flavor SQL, internode
  # traffic and the cli.
  - port: 26257
    targetPort: 26257
    name: tcp
  # The secondary port serves the UI as well as health and debug endpoints.
  - port: 8080
    targetPort: 8080
    name: http
  selector:
    app: cockroachdb
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: cockroachdb-statefulset
  labels:
    version: v20.1.2
spec:
  serviceName: cockroachdb
  replicas: 3
  selector:
    matchLabels:
      app: cockroachdb
  template:
    metadata:
      labels:
        app: cockroachdb
        version: v20.1.2
    spec:
      serviceAccountName: cockroachdb-serviceaccount
      containers:
      - name: cockroachdb
        image: cockroachdb/cockroach:v20.1.2
        ports:
        - containerPort: 26257
          name: tcp
        - containerPort: 8080
          name: http
        volumeMounts:
        - name: datadir
          mountPath: /cockroach/cockroach-data
        env:
        - name: COCKROACH_CHANNEL
          value: kubernetes-insecure
        command:
          - "/bin/bash"
          - "-ecx"
          # The use of qualified `hostname -f` is crucial:
          # Other nodes aren't able to look up the unqualified hostname.
          - "exec /cockroach/cockroach start --logtostderr --insecure --advertise-host $(hostname -f) --http-addr 0.0.0.0 --join cockroachdb-statefulset-0.cockroachdb,cockroachdb-statefulset-1.cockroachdb,cockroachdb-statefulset-2.cockroachdb --cache 25% --max-sql-memory 25%"
      # No pre-stop hook is required, a SIGTERM plus some time is all that's
      # needed for graceful shutdown of a node.
      terminationGracePeriodSeconds: 5
      volumes:
      - name: datadir
        persistentVolumeClaim:
          claimName: datadir
  podManagementPolicy: Parallel
  updateStrategy:
    type: RollingUpdate
  volumeClaimTemplates:
  - metadata:
      name: datadir
    spec:
      accessModes:
        - "ReadWriteOnce"
      resources:
        requests:
          storage: 4Gi
---
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: cockroachdb-public
spec:
  host: cockroachdb-public
  trafficPolicy:
    tls:
      mode: ISTIO_MUTUAL
---
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: cockroachdb-public
spec:
  hosts:
  - cockroachdb-public
  http:
  - match:
    - port: 8080
    route:
    - destination:
        host: cockroachdb-public
        port:
          number: 8080
  tcp:
  - match:
    - port: 26257
    route:
    - destination:
        host: cockroachdb-public
        port:
          number: 26257

以及访问它的服务：

apiVersion: v1
kind: ServiceAccount
metadata:
  name: downstream-serviceaccount
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: downstream-deployment-v1
  labels:
    app: downstream
    version: v1
spec:
  replicas: 1
  selector:
    matchLabels:
      app: downstream
      version: v1
  template:
    metadata:
      labels:
        app: downstream
        version: v1
    spec:
      serviceAccountName: downstream-serviceaccount
      containers:
      - name: downstream
        image: downstream:0.1
        ports:
        - containerPort: 80
        env:
          - name: DATABASE_URL
            value: postgres://roach@cockroachdb-public:26257/roach?sslmode=disable
---
apiVersion: v1
kind: Service
metadata:
  name: downstream-service
  labels:
    app: downstream
spec:
  type: ClusterIP
  selector:
    app: downstream
  ports:
    - port: 80
      targetPort: 80
      name: http
      protocol: TCP
---
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: downstream-service
spec:
  host: downstream-service
  trafficPolicy:
    tls:
      mode: ISTIO_MUTUAL
---
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: downstream-service
spec:
  hosts:
  - downstream-service
  http:
  - name: "downstream-service-routes"
    match:
    - port: 80
    route:
    - destination:
        host: downstream-service
        port:
          number: 80

现在我想将对 cockroach db 的访问限制为 downstream-service 和 cockroachdb 本身（因为节点之间需要相互通信）。

我正在尝试通过以下方式限制流量：

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
 name: default-deny-all
 namespace: default
spec:
  {}
---
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
 name: cockroachdb-authorizationpolicy-allow-from-downstream
 namespace: default
spec:
 selector:
   matchLabels:
     app: cockroachdb
 action: ALLOW
 rules:
  - from:
    - source:
       principals: ["cluster.local/ns/default/sa/downstream-serviceaccount"]
  - to:
    - operation:
       ports: ["26257"]
---
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
 name: cockroachdb-authorizationpolicy-allow-from-cockroachdb
 namespace: default
spec:
 selector:
   matchLabels:
     app: cockroachdb
 action: ALLOW
 rules:
  - from:
    - source:
       principals: ["cluster.local/ns/default/sa/cockroachdb-serviceaccount"]
  - to:
    - operation:
       ports: ["26257"]

但似乎没有做任何事情。我仍然可以从下游服务访问 cockroachdb-public:8080 集群 HTTP UI。

现在当我添加以下内容时：

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
 name: default-deny-all-to-cockroachdb
 namespace: default
spec:
  selector:
    matchLabels:
      app: cockroachdb
  action: DENY
  rules:
  - to:
    - operation:
        ports: ["26257"]

然后所有流量都被阻塞（包括cockroachdb节点之间的流量）。

我做错了什么？

Answer 1

你和几天前的一个人遇到了同样的问题。在您的授权政策中，您有两个政策：

服务帐户 downstream-serviceaccount（以及其他授权策略的 cockroachdb-serviceaccount）可以在 default 命名空间的任何端口上访问带有标签 app: cockroachdb 的服务。
来自任何命名空间的任何服务帐户都可以在端口 26257 上访问带有标签 app: cockroachdb 的服务。

为了使其成为 AND，您可以这样做：

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
 name: cockroachdb-authorizationpolicy-allow-from-cockroachdb
 namespace: default
spec:
 selector:
   matchLabels:
     app: cockroachdb
 action: ALLOW
 rules:
  - from:
    - source:
       principals: ["cluster.local/ns/default/sa/cockroachdb-serviceaccount"]
    to:                  <- remove the dash from here
    - operation:
       ports: ["26257"]

与另一个 AuthorizationPolicy 对象相同。另请注意，您无需显式创建 DENY 策略。当你创建一个 ALLOW 时，它会自动拒绝其他所有内容。

如何在 istio kubernetes 集群中拒绝默认但允许 HTTP 和 TCP 流量？

How to deny default but allow HTTP and TCP traffic in istio kubernetes cluster?

authorization

kubernetes

istio