问：在 Kubernetes 中配置自定义调度程序插件
Q: Configuration custom scheduler plugin in Kubernetes
我正在尝试在 Kubernetes 中添加带有调度程序插件的自定义调度程序。
由于Kubernetes文档中没有详细示例(scheduling-framework, 20180409-scheduling-framework)
所以我只是为了测试而制作的。但是，我得到了 error: unable to recognize "test.yaml": no matches for kind "KubeSchedulerConfiguration" in version "kubescheduler.config.k8s.io/v1alpha1".
如何配置和使用调度程序插件？
在我的例子中，测试配置是
apiVersion: kubescheduler.config.k8s.io/v1alpha1
kind: KubeSchedulerConfiguration
plugins:
queueSort:
enabled:
- name: test
pluginConfig:
- name: test-
kubectl 命令
$ kubectl apply -f test.yaml
error: unable to recognize "test.yaml": no matches for kind "KubeSchedulerConfiguration" in version "kubescheduler.config.k8s.io/v1alpha1"
当我创建为 pod 时，
$ kubectl create -f test.yaml
$ kubectl logs pod test
E0920 15:55:47.970707 1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20190918160344-1fbdaa4c8d90/tools/cache/reflector.go:96: Failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope
E0920 15:55:47.971808 1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20190918160344-1fbdaa4c8d90/tools/cache/reflector.go:96: Failed to list *v1.StorageClass: storageclasses.storage.k8s.io is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "storageclasses" in API group "storage.k8s.io" at the cluster scope
E0920 15:55:48.962810 1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20190918160344-1fbdaa4c8d90/tools/cache/reflector.go:96: Failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "nodes" in API group "" at the cluster scope
E0920 15:55:48.963476 1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20190918160344-1fbdaa4c8d90/tools/cache/reflector.go:96: Failed to list *v1beta1.CSINode: csinodes.storage.k8s.io is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "csinodes" in API group "storage.k8s.io" at the cluster scope
E0920 15:55:48.964615 1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20190918160344-1fbdaa4c8d90/tools/cache/reflector.go:96: Failed to list *v1.StatefulSet: statefulsets.apps is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "statefulsets" in API group "apps" at the cluster scope
E0920 15:55:48.965674 1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20190918160344-1fbdaa4c8d90/tools/cache/reflector.go:96: Failed to list *v1.ReplicaSet: replicasets.apps is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "replicasets" in API group "apps" at the cluster scope
E0920 15:55:48.966800 1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20190918160344-1fbdaa4c8d90/tools/cache/reflector.go:96: Failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "replicationcontrollers" in API group "" at the cluster scope
E0920 15:55:48.967967 1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20190918160344-1fbdaa4c8d90/tools/cache/reflector.go:96: Failed to list *v1.Service: services is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "services" in API group "" at the cluster scope
E0920 15:55:48.969030 1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20190918160344-1fbdaa4c8d90/tools/cache/reflector.go:96: Failed to list *v1.Pod: pods is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "pods" in API group "" at the cluster scope
E0920 15:55:48.970095 1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20190918160344-1fbdaa4c8d90/tools/cache/reflector.go:96: Failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "persistentvolumes" in API group "" at the cluster scope
E0920 15:55:48.971150 1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20190918160344-1fbdaa4c8d90/tools/cache/reflector.go:96: Failed to list *v1beta1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope
E0920 15:55:48.972252 1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20190918160344-1fbdaa4c8d90/tools/cache/reflector.go:96: Failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope
E0920 15:55:48.973396 1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20190918160344-1fbdaa4c8d90/tools/cache/reflector.go:96: Failed to list *v1.StorageClass: storageclasses.storage.k8s.io is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "storageclasses" in API group "storage.k8s.io" at the cluster scope
E0920 15:55:49.964805 1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20190918160344-1fbdaa4c8d90/tools/cache/reflector.go:96: Failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "nodes" in API group "" at the cluster scope
E0920 15:55:49.965338 1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20190918160344-1fbdaa4c8d90/tools/cache/reflector.go:96: Failed to list *v1beta1.CSINode: csinodes.storage.k8s.io is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "csinodes" in API group "storage.k8s.io" at the cluster scope
E0920 15:55:49.966313 1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20190918160344-1fbdaa4c8d90/tools/cache/reflector.go:96: Failed to list *v1.StatefulSet: statefulsets.apps is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "statefulsets" in API group "apps" at the cluster scope
E0920 15:55:49.967467 1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20190918160344-1fbdaa4c8d90/tools/cache/reflector.go:96: Failed to list *v1.ReplicaSet: replicasets.apps is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "replicasets" in API group "apps" at the cluster scope
E0920 15:55:49.968448 1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20190918160344-1fbdaa4c8d90/tools/cache/reflector.go:96: Failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "replicationcontrollers" in API group "" at the cluster scope
E0920 15:55:49.969534 1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20190918160344-1fbdaa4c8d90/tools/cache/reflector.go:96: Failed to list *v1.Service: services is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "services" in API group "" at the cluster scope
E0920 15:55:49.970775 1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20190918160344-1fbdaa4c8d90/tools/cache/reflector.go:96: Failed to list *v1.Pod: pods is f^C
环境
$ kubectl version
Client Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.3", GitCommit:"2d3c76f9091b6bec110a5e63777c332469e0cba2", GitTreeState:"clean", BuildDate:"2019-08-19T11:13:54Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.3", GitCommit:"2d3c76f9091b6bec110a5e63777c332469e0cba2", GitTreeState:"clean", BuildDate:"2019-08-19T11:05:50Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}
去
$ go version
go version go1.13 linux/amd64
进入模块
$ cat go.sum
...
k8s.io/kubernetes v1.16.0 h1:WPaqle2JWogVzLxhN6IK67u62IHKKrtYF7MS4FVR4/E=
k8s.io/kube-scheduler v0.0.0-20190918162820-3b5c1246eb18 h1:WnbesbtFrWlZ8wwgDuP1KN+6E+XLDGzG4ztojMxfHsc=
...
调度程序配置未通过 kubectl apply 命令应用于 API 服务器，它通过 kube-scheduler --config=/path/to/config 传递给调度程序进程。在 Github.
上阅读更多相关信息
--config 是您应该指定调度程序将采用哪些参数的条目。从 API 的角度来看，该配置文件应该包含一个 KubeSchedulerConfiguration 对象：
# content of the file passed to "--config"
apiVersion: kubescheduler.config.k8s.io/v1alpha1
kind: KubeSchedulerConfiguration
clientConnection:
kubeconfig: "/var/run/kubernetes/scheduler.kubeconfig"
algorithmSource:
policy:
file:
path: "/root/config/scheduler-extender-policy.json"
参考这个article, which describe how to create custom Kubernetes scheduler and configuration of Scheduler.
KubeSchedulerConfiguration是一种configmap，如果要申请的话必须像下面这样命令-
kubectl create -n kube-system configmap my-scheduler-config --from-file=/root/my-scheduler-config.yaml
这里-
kube-system = 你的目标命名空间
my-scheduler-config = 你的config map的名字，可以是任何名字
/root/my-scheduler-config.yaml = 替换为您的 yaml 配置文件名和位置
我正在尝试在 Kubernetes 中添加带有调度程序插件的自定义调度程序。 由于Kubernetes文档中没有详细示例(scheduling-framework, 20180409-scheduling-framework)
所以我只是为了测试而制作的。但是，我得到了 error: unable to recognize "test.yaml": no matches for kind "KubeSchedulerConfiguration" in version "kubescheduler.config.k8s.io/v1alpha1".
如何配置和使用调度程序插件？
在我的例子中，测试配置是
apiVersion: kubescheduler.config.k8s.io/v1alpha1
kind: KubeSchedulerConfiguration
plugins:
queueSort:
enabled:
- name: test
pluginConfig:
- name: test-
kubectl 命令
$ kubectl apply -f test.yaml
error: unable to recognize "test.yaml": no matches for kind "KubeSchedulerConfiguration" in version "kubescheduler.config.k8s.io/v1alpha1"
当我创建为 pod 时，
$ kubectl create -f test.yaml
$ kubectl logs pod test
E0920 15:55:47.970707 1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20190918160344-1fbdaa4c8d90/tools/cache/reflector.go:96: Failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope
E0920 15:55:47.971808 1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20190918160344-1fbdaa4c8d90/tools/cache/reflector.go:96: Failed to list *v1.StorageClass: storageclasses.storage.k8s.io is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "storageclasses" in API group "storage.k8s.io" at the cluster scope
E0920 15:55:48.962810 1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20190918160344-1fbdaa4c8d90/tools/cache/reflector.go:96: Failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "nodes" in API group "" at the cluster scope
E0920 15:55:48.963476 1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20190918160344-1fbdaa4c8d90/tools/cache/reflector.go:96: Failed to list *v1beta1.CSINode: csinodes.storage.k8s.io is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "csinodes" in API group "storage.k8s.io" at the cluster scope
E0920 15:55:48.964615 1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20190918160344-1fbdaa4c8d90/tools/cache/reflector.go:96: Failed to list *v1.StatefulSet: statefulsets.apps is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "statefulsets" in API group "apps" at the cluster scope
E0920 15:55:48.965674 1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20190918160344-1fbdaa4c8d90/tools/cache/reflector.go:96: Failed to list *v1.ReplicaSet: replicasets.apps is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "replicasets" in API group "apps" at the cluster scope
E0920 15:55:48.966800 1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20190918160344-1fbdaa4c8d90/tools/cache/reflector.go:96: Failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "replicationcontrollers" in API group "" at the cluster scope
E0920 15:55:48.967967 1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20190918160344-1fbdaa4c8d90/tools/cache/reflector.go:96: Failed to list *v1.Service: services is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "services" in API group "" at the cluster scope
E0920 15:55:48.969030 1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20190918160344-1fbdaa4c8d90/tools/cache/reflector.go:96: Failed to list *v1.Pod: pods is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "pods" in API group "" at the cluster scope
E0920 15:55:48.970095 1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20190918160344-1fbdaa4c8d90/tools/cache/reflector.go:96: Failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "persistentvolumes" in API group "" at the cluster scope
E0920 15:55:48.971150 1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20190918160344-1fbdaa4c8d90/tools/cache/reflector.go:96: Failed to list *v1beta1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope
E0920 15:55:48.972252 1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20190918160344-1fbdaa4c8d90/tools/cache/reflector.go:96: Failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope
E0920 15:55:48.973396 1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20190918160344-1fbdaa4c8d90/tools/cache/reflector.go:96: Failed to list *v1.StorageClass: storageclasses.storage.k8s.io is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "storageclasses" in API group "storage.k8s.io" at the cluster scope
E0920 15:55:49.964805 1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20190918160344-1fbdaa4c8d90/tools/cache/reflector.go:96: Failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "nodes" in API group "" at the cluster scope
E0920 15:55:49.965338 1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20190918160344-1fbdaa4c8d90/tools/cache/reflector.go:96: Failed to list *v1beta1.CSINode: csinodes.storage.k8s.io is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "csinodes" in API group "storage.k8s.io" at the cluster scope
E0920 15:55:49.966313 1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20190918160344-1fbdaa4c8d90/tools/cache/reflector.go:96: Failed to list *v1.StatefulSet: statefulsets.apps is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "statefulsets" in API group "apps" at the cluster scope
E0920 15:55:49.967467 1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20190918160344-1fbdaa4c8d90/tools/cache/reflector.go:96: Failed to list *v1.ReplicaSet: replicasets.apps is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "replicasets" in API group "apps" at the cluster scope
E0920 15:55:49.968448 1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20190918160344-1fbdaa4c8d90/tools/cache/reflector.go:96: Failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "replicationcontrollers" in API group "" at the cluster scope
E0920 15:55:49.969534 1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20190918160344-1fbdaa4c8d90/tools/cache/reflector.go:96: Failed to list *v1.Service: services is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "services" in API group "" at the cluster scope
E0920 15:55:49.970775 1 reflector.go:123] pkg/mod/k8s.io/client-go@v0.0.0-20190918160344-1fbdaa4c8d90/tools/cache/reflector.go:96: Failed to list *v1.Pod: pods is f^C
环境
$ kubectl version
Client Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.3", GitCommit:"2d3c76f9091b6bec110a5e63777c332469e0cba2", GitTreeState:"clean", BuildDate:"2019-08-19T11:13:54Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.3", GitCommit:"2d3c76f9091b6bec110a5e63777c332469e0cba2", GitTreeState:"clean", BuildDate:"2019-08-19T11:05:50Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}
去
$ go version
go version go1.13 linux/amd64
进入模块
$ cat go.sum
...
k8s.io/kubernetes v1.16.0 h1:WPaqle2JWogVzLxhN6IK67u62IHKKrtYF7MS4FVR4/E=
k8s.io/kube-scheduler v0.0.0-20190918162820-3b5c1246eb18 h1:WnbesbtFrWlZ8wwgDuP1KN+6E+XLDGzG4ztojMxfHsc=
...
调度程序配置未通过 kubectl apply 命令应用于 API 服务器，它通过 kube-scheduler --config=/path/to/config 传递给调度程序进程。在 Github.
--config 是您应该指定调度程序将采用哪些参数的条目。从 API 的角度来看，该配置文件应该包含一个 KubeSchedulerConfiguration 对象：
# content of the file passed to "--config"
apiVersion: kubescheduler.config.k8s.io/v1alpha1
kind: KubeSchedulerConfiguration
clientConnection:
kubeconfig: "/var/run/kubernetes/scheduler.kubeconfig"
algorithmSource:
policy:
file:
path: "/root/config/scheduler-extender-policy.json"
参考这个article, which describe how to create custom Kubernetes scheduler and configuration of Scheduler.
KubeSchedulerConfiguration是一种configmap，如果要申请的话必须像下面这样命令-
kubectl create -n kube-system configmap my-scheduler-config --from-file=/root/my-scheduler-config.yaml
这里-
kube-system = 你的目标命名空间
my-scheduler-config = 你的config map的名字，可以是任何名字
/root/my-scheduler-config.yaml = 替换为您的 yaml 配置文件名和位置