服务帐户不尊重其集群角色
Service account not respecting its cluster role
我创建了一个具有 只读访问权限的用户的上下文 但是当我以该用户身份登录时，我仍然可以做任何我想做的事，比如部署和killing pods, 等等...这是为什么呢？
我关注了这个tutorial。
1) 首先我创建了一个服务账号：
kubectl create sa myserviceaccount
2) 现在我想要一个具有最低权限的角色（只需阅读）所以我将从 kube-system 中获取一个名为“view"
$ kubectl describe clusterrole view
Resources Non-Resource URLs Resource Names Verbs
--------- ----------------- -------------- -----
bindings [] [] [get list watch]
configmaps [] [] [get list watch]
[...]
3) 现在我必须创建一个 clusterRoleBinding 来将服务帐户绑定到角色 "view"
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: crbmyserviceaccount
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: view
subjects:
- kind: ServiceAccount
name: myserviceaccount
namespace: default
4) 现在我们必须找到关联的秘密名称
kubectl get secrets -> myserviceaccount-token-bmwwd
5) 将显示的令牌保存在某处（以备后用）
kubectl describe secret myserviceaccount-token-xxxxx
既然我们拥有了所需的一切，我们就可以在 kubernetes 客户端上创建上下文了。
6) 在 kubeconfig 中配置集群：
kubectl config set-cluster myawesomecluster --server=IP-OF-MY-CLUSTER
7) 创建凭据：
kubectl config set-credentials myawesomecluster-myserviceaccount --token=TOKEN-FROM-STEP-5
8) 创建上下文
kubectl config set-context myawesomecluster --cluster=myawesomecluster --user=myawesomecluster-myserviceaccount --namespace=default
kubectl config use-context myawesomecluster
太棒了！
既然设置了上下文，我应该能够阅读每个资源，但不能创建任何资源。不幸的是，我仍然可以使用 kubectl 进行部署，甚至可以删除 pods 等
这应该 return 我拒绝访问： kubectl create -f someFileWithDeployment
我做错了什么？
谢谢！
编辑 - 为调试目的添加命名空间和配置视图的输出：
$kubectl get sa
NAME SECRETS AGE
api-explorer 1 39h
default 1 5d22h
myserviceaccount 1 17h
$kubectl config view
apiVersion: v1
clusters:
- cluster:
certificate-authority: /home/xxxx/rbac/accountTest/api- explorer/context/team-a-decoded.crt
server: http://127.0.0.1:8080
name: cfc
- cluster:
server: http://127.0.0.1:8080
name: myawesomecluster
- cluster:
certificate-authority-data: DATA+OMITTED
server: http://localhost:8080
name: test-cluster
contexts:
- context:
cluster: cfc
user: user
name: cfc
- context:
cluster: ""
user: ""
name: default
- context:
cluster: myawesomecluster
namespace: default
user: myawesomecluster-myserviceaccount
name: myawesomecluster
current-context: myawesomecluster
kind: Config
preferences: {}
users:
- name: api-explorer
user:
token: ZXlKaGJHY2l[...]
- name: myawesomecluster-myserviceaccount
user:
token: eyJhbGci [...]
- name: user
user:
token: ZXlKaGJH
编辑 2：显示 get pod kube-apiserver-nodemaster1 的输出
$kubectl get pod kube-apiserver-nodemaster1 -n kube-system -o yaml
apiVersion: v1
kind: Pod
metadata:
annotations:
kubernetes.io/config.hash: 034c3[...]
kubernetes.io/config.mirror: 034b3[...]
kubernetes.io/config.seen: 2018-11-23T09:48:59.766423346Z
kubernetes.io/config.source: file
scheduler.alpha.kubernetes.io/critical-pod: ""
creationTimestamp: 2018-11-23T09:50:29Z
labels:
component: kube-apiserver
tier: control-plane
name: kube-apiserver-nodemaster1
namespace: kube-system
resourceVersion: "804213"
selfLink: /api/v1/namespaces/kube-system/pods/kube-apiserver-nodemaster1
uid: 36340f[...]
spec:
containers:
- command:
- kube-apiserver
- --allow-privileged=true
- --apiserver-count=3
- --authorization-mode=Node,RBAC
- --bind-address=0.0.0.0
- --endpoint-reconciler-type=lease
- --insecure-bind-address=127.0.0.1
- --insecure-port=8080
- --kubelet-preferred-address-types=InternalDNS,InternalIP,Hostname,ExternalDNS,ExternalIP
- --runtime-config=admissionregistration.k8s.io/v1alpha1
- --service-node-port-range=30000-32767
- --storage-backend=etcd3
- --advertise-address=10.10.10.101
- --client-ca-file=/etc/kubernetes/ssl/ca.crt
- --enable-admission-plugins=NodeRestriction
- --enable-bootstrap-token-auth=true
- --etcd-cafile=/etc/kubernetes/ssl/etcd/ca.pem
- --etcd-certfile=/etc/kubernetes/ssl/etcd/node-nodemaster1.pem
- --etcd-keyfile=/etc/kubernetes/ssl/etcd/node-nodemaster1-key.pem
- --etcd-servers=https://10.10.10.101:2379,https://10.10.10.102:2379,https://10.10.10.103:2379
- --kubelet-client-certificate=/etc/kubernetes/ssl/apiserver-kubelet-client.crt
- --kubelet-client-key=/etc/kubernetes/ssl/apiserver-kubelet-client.key
- --proxy-client-cert-file=/etc/kubernetes/ssl/front-proxy-client.crt
- --proxy-client-key-file=/etc/kubernetes/ssl/front-proxy-client.key
- --requestheader-allowed-names=front-proxy-client
- --requestheader-client-ca-file=/etc/kubernetes/ssl/front-proxy-ca.crt
- --requestheader-extra-headers-prefix=X-Remote-Extra-
- --requestheader-group-headers=X-Remote-Group
- --requestheader-username-headers=X-Remote-User
- --secure-port=6443
- --service-account-key-file=/etc/kubernetes/ssl/sa.pub
- --service-cluster-ip-range=10.233.0.0/18
- --tls-cert-file=/etc/kubernetes/ssl/apiserver.crt
- --tls-private-key-file=/etc/kubernetes/ssl/apiserver.key
image: gcr.io/google-containers/kube-apiserver:v1.12.2
imagePullPolicy: IfNotPresent
livenessProbe:
failureThreshold: 8
httpGet:
host: 10.10.10.101
path: /healthz
port: 6443
scheme: HTTPS
initialDelaySeconds: 15
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 15
name: kube-apiserver
resources:
requests:
cpu: 250m
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /etc/kubernetes/ssl
name: k8s-certs
readOnly: true
- mountPath: /etc/ssl/certs
name: ca-certs
readOnly: true
- mountPath: /etc/pki
name: etc-pki
readOnly: true
dnsPolicy: ClusterFirst
hostNetwork: true
nodeName: nodemaster1
priority: 2000000000
priorityClassName: system-cluster-critical
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
terminationGracePeriodSeconds: 30
tolerations:
- effect: NoExecute
operator: Exists
volumes:
- hostPath:
path: /etc/kubernetes/ssl
type: DirectoryOrCreate
name: k8s-certs
- hostPath:
path: /etc/ssl/certs
type: DirectoryOrCreate
name: ca-certs
- hostPath:
path: /etc/pki
type: DirectoryOrCreate
name: etc-pki
status:
conditions:
- lastProbeTime: null
lastTransitionTime: 2018-11-23T09:55:05Z
status: "True"
type: Initialized
- lastProbeTime: null
lastTransitionTime: 2018-11-23T09:55:05Z
status: "True"
type: Ready
- lastProbeTime: null
lastTransitionTime: 2018-11-23T09:55:05Z
status: "True"
type: ContainersReady
- lastProbeTime: null
lastTransitionTime: 2018-11-23T09:55:05Z
status: "True"
type: PodScheduled
containerStatuses:
- containerID: docker://8287[...]
image: gcr.io/google-containers/kube-apiserver:v1.12.2
imageID: docker-pullable://gcr.io/google-containers/kube-apiserver@sha256:0949[...]
lastState:
terminated:
containerID: docker://e97[...]
exitCode: 0
finishedAt: 2018-11-27T14:18:24Z
reason: Completed
startedAt: 2018-11-23T09:49:00Z
name: kube-apiserver
ready: true
restartCount: 1
state:
running:
startedAt: 2018-11-27T14:18:24Z
hostIP: 10.10.10.101
phase: Running
podIP: 10.10.10.101
qosClass: Burstable
startTime: 2018-11-23T09:55:05Z
我假设当你在 kube-apiserver 配置中使用 --insecure-bind-address=127.0.0.1 和 --insecure-port=8080 标志时，所有对 Kubernetes API 服务器的请求都会绕过 RBAC 模块授权，如官方 Kubernetes 中所述documentation:
Localhost Port:
- is intended for testing and bootstrap, and for other components of the master node (scheduler, controller-manager) to talk to the API
- no TLS
- default is port 8080, change with
--insecure-port flag.
- default IP is localhost, change with
--insecure-bind-address flag.
- request bypasses authentication and authorization modules.
- request handled by admission control module(s).
- protected by need to have host access
正如@mk_sta 所解释的那样，我必须使用 HTTPS 端点而不是默认的 http 端点。
Http 端点被考虑弃用，可能会按照说明删除 here
为了通过 https 端点替换第 6 步：
kubectl config set-cluster myawesomecluster --server=https://127.0.0.1:6443
现在您可能会遇到 SLL 错误。
我所做的是更改我的 kubeconfig 文件（在 ~/.kube/config）。
我添加了密钥 "certificate-authority" 以及 kubernetes 存储其主证书的 路径的值 .
最终的集群配置现在看起来像这样：
- cluster:
certificate-authority: /etc/kubernetes/ssl/ca.crt
server: https://127.0.0.1:6443
name: secureRemote
我创建了一个具有 只读访问权限的用户的上下文 但是当我以该用户身份登录时，我仍然可以做任何我想做的事，比如部署和killing pods, 等等...这是为什么呢？
我关注了这个tutorial。
1) 首先我创建了一个服务账号：
kubectl create sa myserviceaccount
2) 现在我想要一个具有最低权限的角色（只需阅读）所以我将从 kube-system 中获取一个名为“view"
$ kubectl describe clusterrole view
Resources Non-Resource URLs Resource Names Verbs
--------- ----------------- -------------- -----
bindings [] [] [get list watch]
configmaps [] [] [get list watch]
[...]
3) 现在我必须创建一个 clusterRoleBinding 来将服务帐户绑定到角色 "view"
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: crbmyserviceaccount
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: view
subjects:
- kind: ServiceAccount
name: myserviceaccount
namespace: default
4) 现在我们必须找到关联的秘密名称
kubectl get secrets -> myserviceaccount-token-bmwwd
5) 将显示的令牌保存在某处（以备后用）
kubectl describe secret myserviceaccount-token-xxxxx
既然我们拥有了所需的一切，我们就可以在 kubernetes 客户端上创建上下文了。
6) 在 kubeconfig 中配置集群：
kubectl config set-cluster myawesomecluster --server=IP-OF-MY-CLUSTER
7) 创建凭据：
kubectl config set-credentials myawesomecluster-myserviceaccount --token=TOKEN-FROM-STEP-5
8) 创建上下文
kubectl config set-context myawesomecluster --cluster=myawesomecluster --user=myawesomecluster-myserviceaccount --namespace=default
kubectl config use-context myawesomecluster
太棒了！
既然设置了上下文，我应该能够阅读每个资源，但不能创建任何资源。不幸的是，我仍然可以使用 kubectl 进行部署，甚至可以删除 pods 等
这应该 return 我拒绝访问： kubectl create -f someFileWithDeployment
我做错了什么？
谢谢！
编辑 - 为调试目的添加命名空间和配置视图的输出：
$kubectl get sa
NAME SECRETS AGE
api-explorer 1 39h
default 1 5d22h
myserviceaccount 1 17h
$kubectl config view
apiVersion: v1
clusters:
- cluster:
certificate-authority: /home/xxxx/rbac/accountTest/api- explorer/context/team-a-decoded.crt
server: http://127.0.0.1:8080
name: cfc
- cluster:
server: http://127.0.0.1:8080
name: myawesomecluster
- cluster:
certificate-authority-data: DATA+OMITTED
server: http://localhost:8080
name: test-cluster
contexts:
- context:
cluster: cfc
user: user
name: cfc
- context:
cluster: ""
user: ""
name: default
- context:
cluster: myawesomecluster
namespace: default
user: myawesomecluster-myserviceaccount
name: myawesomecluster
current-context: myawesomecluster
kind: Config
preferences: {}
users:
- name: api-explorer
user:
token: ZXlKaGJHY2l[...]
- name: myawesomecluster-myserviceaccount
user:
token: eyJhbGci [...]
- name: user
user:
token: ZXlKaGJH
编辑 2：显示 get pod kube-apiserver-nodemaster1 的输出
$kubectl get pod kube-apiserver-nodemaster1 -n kube-system -o yaml
apiVersion: v1
kind: Pod
metadata:
annotations:
kubernetes.io/config.hash: 034c3[...]
kubernetes.io/config.mirror: 034b3[...]
kubernetes.io/config.seen: 2018-11-23T09:48:59.766423346Z
kubernetes.io/config.source: file
scheduler.alpha.kubernetes.io/critical-pod: ""
creationTimestamp: 2018-11-23T09:50:29Z
labels:
component: kube-apiserver
tier: control-plane
name: kube-apiserver-nodemaster1
namespace: kube-system
resourceVersion: "804213"
selfLink: /api/v1/namespaces/kube-system/pods/kube-apiserver-nodemaster1
uid: 36340f[...]
spec:
containers:
- command:
- kube-apiserver
- --allow-privileged=true
- --apiserver-count=3
- --authorization-mode=Node,RBAC
- --bind-address=0.0.0.0
- --endpoint-reconciler-type=lease
- --insecure-bind-address=127.0.0.1
- --insecure-port=8080
- --kubelet-preferred-address-types=InternalDNS,InternalIP,Hostname,ExternalDNS,ExternalIP
- --runtime-config=admissionregistration.k8s.io/v1alpha1
- --service-node-port-range=30000-32767
- --storage-backend=etcd3
- --advertise-address=10.10.10.101
- --client-ca-file=/etc/kubernetes/ssl/ca.crt
- --enable-admission-plugins=NodeRestriction
- --enable-bootstrap-token-auth=true
- --etcd-cafile=/etc/kubernetes/ssl/etcd/ca.pem
- --etcd-certfile=/etc/kubernetes/ssl/etcd/node-nodemaster1.pem
- --etcd-keyfile=/etc/kubernetes/ssl/etcd/node-nodemaster1-key.pem
- --etcd-servers=https://10.10.10.101:2379,https://10.10.10.102:2379,https://10.10.10.103:2379
- --kubelet-client-certificate=/etc/kubernetes/ssl/apiserver-kubelet-client.crt
- --kubelet-client-key=/etc/kubernetes/ssl/apiserver-kubelet-client.key
- --proxy-client-cert-file=/etc/kubernetes/ssl/front-proxy-client.crt
- --proxy-client-key-file=/etc/kubernetes/ssl/front-proxy-client.key
- --requestheader-allowed-names=front-proxy-client
- --requestheader-client-ca-file=/etc/kubernetes/ssl/front-proxy-ca.crt
- --requestheader-extra-headers-prefix=X-Remote-Extra-
- --requestheader-group-headers=X-Remote-Group
- --requestheader-username-headers=X-Remote-User
- --secure-port=6443
- --service-account-key-file=/etc/kubernetes/ssl/sa.pub
- --service-cluster-ip-range=10.233.0.0/18
- --tls-cert-file=/etc/kubernetes/ssl/apiserver.crt
- --tls-private-key-file=/etc/kubernetes/ssl/apiserver.key
image: gcr.io/google-containers/kube-apiserver:v1.12.2
imagePullPolicy: IfNotPresent
livenessProbe:
failureThreshold: 8
httpGet:
host: 10.10.10.101
path: /healthz
port: 6443
scheme: HTTPS
initialDelaySeconds: 15
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 15
name: kube-apiserver
resources:
requests:
cpu: 250m
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /etc/kubernetes/ssl
name: k8s-certs
readOnly: true
- mountPath: /etc/ssl/certs
name: ca-certs
readOnly: true
- mountPath: /etc/pki
name: etc-pki
readOnly: true
dnsPolicy: ClusterFirst
hostNetwork: true
nodeName: nodemaster1
priority: 2000000000
priorityClassName: system-cluster-critical
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
terminationGracePeriodSeconds: 30
tolerations:
- effect: NoExecute
operator: Exists
volumes:
- hostPath:
path: /etc/kubernetes/ssl
type: DirectoryOrCreate
name: k8s-certs
- hostPath:
path: /etc/ssl/certs
type: DirectoryOrCreate
name: ca-certs
- hostPath:
path: /etc/pki
type: DirectoryOrCreate
name: etc-pki
status:
conditions:
- lastProbeTime: null
lastTransitionTime: 2018-11-23T09:55:05Z
status: "True"
type: Initialized
- lastProbeTime: null
lastTransitionTime: 2018-11-23T09:55:05Z
status: "True"
type: Ready
- lastProbeTime: null
lastTransitionTime: 2018-11-23T09:55:05Z
status: "True"
type: ContainersReady
- lastProbeTime: null
lastTransitionTime: 2018-11-23T09:55:05Z
status: "True"
type: PodScheduled
containerStatuses:
- containerID: docker://8287[...]
image: gcr.io/google-containers/kube-apiserver:v1.12.2
imageID: docker-pullable://gcr.io/google-containers/kube-apiserver@sha256:0949[...]
lastState:
terminated:
containerID: docker://e97[...]
exitCode: 0
finishedAt: 2018-11-27T14:18:24Z
reason: Completed
startedAt: 2018-11-23T09:49:00Z
name: kube-apiserver
ready: true
restartCount: 1
state:
running:
startedAt: 2018-11-27T14:18:24Z
hostIP: 10.10.10.101
phase: Running
podIP: 10.10.10.101
qosClass: Burstable
startTime: 2018-11-23T09:55:05Z
我假设当你在 kube-apiserver 配置中使用 --insecure-bind-address=127.0.0.1 和 --insecure-port=8080 标志时，所有对 Kubernetes API 服务器的请求都会绕过 RBAC 模块授权，如官方 Kubernetes 中所述documentation:
Localhost Port:
- is intended for testing and bootstrap, and for other components of the master node (scheduler, controller-manager) to talk to the API
- no TLS
- default is port 8080, change with
--insecure-portflag.- default IP is localhost, change with
--insecure-bind-addressflag.- request bypasses authentication and authorization modules.
- request handled by admission control module(s).
- protected by need to have host access
正如@mk_sta 所解释的那样，我必须使用 HTTPS 端点而不是默认的 http 端点。 Http 端点被考虑弃用，可能会按照说明删除 here
为了通过 https 端点替换第 6 步：
kubectl config set-cluster myawesomecluster --server=https://127.0.0.1:6443
现在您可能会遇到 SLL 错误。
我所做的是更改我的 kubeconfig 文件（在 ~/.kube/config）。
我添加了密钥 "certificate-authority" 以及 kubernetes 存储其主证书的 路径的值 .
最终的集群配置现在看起来像这样：
- cluster:
certificate-authority: /etc/kubernetes/ssl/ca.crt
server: https://127.0.0.1:6443
name: secureRemote