API Server: Unable to authenticate the request due to an error: invalid bearer token

Question

我设置了一个示例 Kubernetes 集群，其中包含 3 个主节点和 2 个工作节点。我试图将它连接到一个 OpenId 提供商（在我的例子中是 Keycloak）。但是在查询 API 时，我从 kubectl 收到以下消息：

error: You must be logged in to the server (Unauthorized)

或通过 curl:

curl -H "Authorization: Bearer $token" -k https://192.168.178.30:6443/api/v1/namespaces/default/pods
{
  "kind": "Status",
  "apiVersion": "v1",
  "metadata": {
    
  },
  "status": "Failure",
  "message": "Unauthorized",
  "reason": "Unauthorized",
  "code": 401
}

并且在 API 服务器日志中显示

Unable to authenticate the request due to an error: invalid bearer token

根据 jwt.io 我的令牌似乎有效。

API 服务器的配置

在 API 服务器中我指定了以下参数：

oidc-issuer-url: https://192.168.178.25:8443/auth/realms/master
oidc-client-id: account
oidc-ca-file: /etc/kubernetes/ssl/keycloak-rootCA.pem
oidc-username-claim: sub
oidc-groups-claim: groups

我必须指定 CA 文件，因为用于 keycloak 的证书是自签名的

证书似乎有效，因为没有 CA 文件的 curl 会导致错误；没有指定 CA 文件的地方：

ansible@node1:~$ curl https://192.168.178.25:8443/auth/realms/master
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.haxx.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.


ansible@node1:~$ curl --cacert /etc/kubernetes/ssl/keycloak-rootCA.pem https://192.168.178.25:8443/auth/realms/master
{"realm":"master","public_key":"...","token-service":"https://192.168.178.25:8443/auth/realms/master/protocol/openid-connect","account-service":"https://192.168.178.25:8443/auth/realms/master/account","tokens-not-before":0}

智威汤逊令牌

我的 JWT 令牌的有效负载如下所示：

{
  "jti": "455feab7-5828-49f3-9243-a2ce0a1ae6f9",
  "exp": 1557246081,
  "nbf": 0,
  "iat": 1557246021,
  "iss": "https://192.168.178.25:8443/auth/realms/master",
  "aud": "account",
  "sub": "78f9bdc8-d1a4-43c4-ab83-f1b4645519f6",
  "typ": "ID",
  "azp": "account",
  "auth_time": 1557246020,
  "session_state": "ccafaa62-a888-43dd-9135-1c5a31766e0b",
  "acr": "1",
  "email_verified": true,
  "name": "Testuser",
  "groups": [
    "group1",
    "k8s-admin"
  ],
  "preferred_username": "dummy-username",
  "given_name": "Firstname1",
  "family_name": "Lastname1",
  "email": "me@example.com"
}

当我尝试使用生成的服务帐户（来自 kubernetes）的令牌登录时，一切正常。

补充信息

环境是通过 Kubespray 设置的，API 服务器参数按以下方式指定：

kube_oidc_url: https://192.168.178.25:8443/auth/realms/master
kube_oidc_client_id: account
kube_oidc_ca_file: "{{ kube_cert_dir }}/keycloak-rootCA.pem"
kube_oidc_username_claim: sub
kube_oidc_groups_claim: groups
#kube_oidc_username_prefix: "oidc:"
#kube_oidc_groups_prefix: "oidc:"

感谢您的帮助。

Answer 1

我认为你的情况 kube_oidc_url 应该是 https://192.168.178.25:8443

同时记下 documentation 的 CA 证书要求：

A note about requirement #3 above, requiring a CA signed certificate. If you deploy your own identity provider (as opposed to one of the cloud providers like Google or Microsoft) you MUST have your identity provider’s web server certificate signed by a certificate with the CA flag set to TRUE, even if it is self signed. This is due to GoLang’s TLS client implementation being very strict to the standards around certificate validation. If you don’t have a CA handy, you can use this script from the CoreOS team to create a simple CA and a signed certificate and key pair. Or you can use this similar script that generates SHA256 certs with a longer life and larger key size.

Answer 2

问题是 Kubespray 没有将我的设置覆盖到 api-server (/etc/kubernetes/manifests/kube-apiserver.yaml) 的 Pod 规范中。当我在每个 api-server 的 Pod 规范中添加参数时（我有一个多主设置），它按预期工作：

- --oidc-issuer-url=https://192.168.178.25:8443/auth/realms/master
- --oidc-client-id=account
- --oidc-username-claim=email
- '--oidc-username-prefix=oidc:'
- --oidc-groups-claim=groups
- '--oidc-groups-prefix=oidc:'
- --oidc-ca-file=/etc/kubernetes/ssl/keycloak-rootCA.pem

API Server: Unable to authenticate the request due to an error: invalid bearer token