kublet 不以云提供商开头 "openstack"
kublet doesn't start with cloud provider "openstack"
为了使用 cinder 卷，我将选项 --cloud-provider 和 --cloud-config 添加到我的 kubelet 配置中：
$ cat /etc/kubernetes/kubelet
###
# kubernetes kubelet (node) config
# The address for the info server to serve on (set to 0.0.0.0 or "" for all interfaces)
KUBELET_ADDRESS="--address=0.0.0.0"
# The port for the info server to serve on
# KUBELET_PORT="--port=10250"
# You may leave this blank to use the actual hostname
KUBELET_HOSTNAME="--hostname_override=192.168.100.76"
# location of the api-server
KUBELET_API_SERVER="--api_servers=https://localhost:6443"
# Add your own!
KUBELET_ARGS="--cluster_dns=10.100.0.10 --cluster_domain=cluster.local --kubeconfig=/etc/kubernetes/kubelet.kubeconfig --config=/etc/kubernetes/manifests --cloud-provider=openstack --cloud-config=/etc/kubernetes/cloud_config"
我已经创建了一个包含必要凭据的文件：
$ sudo cat /etc/kubernetes/cloud_config
[Global]
auth-url=https://api.*********.com:5000/v2.0
user-id=kubecindertest
username=kubecindertest
password=*****
region=RegionOne
tenant-name=kubecindertest
tenant-id=6568768756a7886767e676f7efe76fe7
project-name=kubecindertest
当启动kubelet（手动）时，进程只记录unknown cloud provider "openstack"并且存在：
source /etc/kubernetes/kubelet; sudo /usr/bin/kubelet $KUBE_LOGTOSTDERR $KUBE_LOG_LEVEL $KUBELET_API_SERVER $KUBELET_ADDRESS $KUBELET_PORT $KUBELET_HOSTNAME $KUBE_ALLOW_PRIV $KUBELET_ARGS
unknown cloud provider "openstack"
在 kubernetes 存储库中定义 openstack 提供程序的 openstack.go 具有完全相同的小写名称：
const ProviderName = "openstack"
更新
原来错误是在解析配置文件时发生的预期错误。我删除了所有可选或不需要的键，现在我将其用作我的配置文件：
$ sudo cat /etc/kubernetes/cloud_config
[Global]
auth-url=https://api.*********.com:5000/v2.0
username=kubecindertest
password=*****
region=RegionOne
tenant-id=6568768756a7886767e676f7efe76fe7
然而，启动 kublet 只会导致另一个错误：
I0923 07:14:33.315311 23743 manager.go:127] cAdvisor running in container: "/user.slice"
I0923 07:14:33.316263 23743 fs.go:93] Filesystem partitions: map[/dev/vda1:{mountpoint:/ major:253 minor:1}]
I0923 07:14:33.358848 23743 manager.go:158] Machine: {NumCores:2 CpuFrequency:2099998 MemoryCapacity:4144640000 MachineID:dae72fe0cc064eb0b7797f25bfaf69df SystemUUID:BEDAF943-624D-C04A-B92C-4EB07258246C BootID:e2d988e2-9aba-49bf-a344-fd62607a6754 Filesystems:[{Device:/dev/vda1 Capacity:21456445440}] DiskMap:map[252:0:{Name:dm-0 Major:252 Minor:0 Size:107374182400 Scheduler:none} 252:1:{Name:dm-1 Major:252 Minor:1 Size:10737418240 Scheduler:none} 252:2:{Name:dm-2 Major:252 Minor:2 Size:10737418240 Scheduler:none} 253:0:{Name:vda Major:253 Minor:0 Size:21474836480 Scheduler:none}] NetworkDevices:[{Name:eth0 MacAddress:fa:16:3e:64:fa:9a Speed:0 Mtu:1500} {Name:eth1 MacAddress:fa:16:3e:01:00:79 Speed:0 Mtu:1500} {Name:flannel.1 MacAddress:d2:9c:ad:29:df:c5 Speed:0 Mtu:1450}] Topology:[{Id:0 Memory:4294434816 Cores:[{Id:0 Threads:[0] Caches:[{Size:32768 Type:Data Level:1} {Size:32768 Type:Instruction Level:1} {Size:4194304 Type:Unified Level:2}]}] Caches:[]} {Id:1 Memory:0 Cores:[{Id:0 Threads:[1] Caches:[{Size:32768 Type:Data Level:1} {Size:32768 Type:Instruction Level:1} {Size:4194304 Type:Unified Level:2}]}] Caches:[]}] CloudProvider:Unknown InstanceType:Unknown}
I0923 07:14:33.363915 23743 manager.go:164] Version: {KernelVersion:3.10.0-229.11.1.el7.x86_64 ContainerOsVersion:CentOS Linux 7 (Core) DockerVersion:1.8.1.el7 CadvisorVersion:0.16.0}
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x0 pc=0x8559cd]
goroutine 1 [running]:
k8s.io/kubernetes/pkg/cloudprovider/providers/openstack.(*OpenStack).Instances(0x0, 0x0, 0x0, 0xe)
/builddir/build/BUILD/kubernetes-196f58b9cb25a2222c7f9aacd624737910b03acb/_output/local/go/src/k8s.io/kubernetes/pkg/cloudprovider/providers/openstack/openstack.go:167 +0x8ed
k8s.io/kubernetes/cmd/kubelet/app.RunKubelet(0xc820144900, 0x0, 0x0, 0x0)
/builddir/build/BUILD/kubernetes-196f58b9cb25a2222c7f9aacd624737910b03acb/_output/local/go/src/k8s.io/kubernetes/cmd/kubelet/app/server.go:628 +0x13c
k8s.io/kubernetes/cmd/kubelet/app.(*KubeletServer).Run(0xc8202c2000, 0xc820144900, 0x0, 0x0)
/builddir/build/BUILD/kubernetes-196f58b9cb25a2222c7f9aacd624737910b03acb/_output/local/go/src/k8s.io/kubernetes/cmd/kubelet/app/server.go:420 +0x84b
main.main()
/builddir/build/BUILD/kubernetes-196f58b9cb25a2222c7f9aacd624737910b03acb/_output/local/go/src/k8s.io/kubernetes/cmd/kubelet/kubelet.go:46 +0xab
goroutine 17 [syscall, locked to thread]:
runtime.goexit()
/usr/lib/golang/src/runtime/asm_amd64.s:1696 +0x1
goroutine 5 [chan receive]:
github.com/golang/glog.(*loggingT).flushDaemon(0x1dc7000)
/builddir/build/BUILD/kubernetes-196f58b9cb25a2222c7f9aacd624737910b03acb/Godeps/_workspace/src/github.com/golang/glog/glog.go:879 +0x67
created by github.com/golang/glog.init.1
/builddir/build/BUILD/kubernetes-196f58b9cb25a2222c7f9aacd624737910b03acb/Godeps/_workspace/src/github.com/golang/glog/glog.go:410 +0x297
goroutine 37 [runnable]:
syscall.Syscall6(0x36, 0x4, 0x29, 0x1a, 0xc820035a7c, 0x4, 0x0, 0x0, 0x1a, 0x0)
/usr/lib/golang/src/syscall/asm_linux_amd64.s:44 +0x5
syscall.setsockopt(0x4, 0x29, 0x1a, 0xc820035a7c, 0x4, 0x0, 0x0)
/usr/lib/golang/src/syscall/zsyscall_linux_amd64.go:1655 +0x73
syscall.SetsockoptInt(0x4, 0x29, 0x1a, 0x0, 0x0, 0x0)
/usr/lib/golang/src/syscall/syscall_unix.go:267 +0x61
net.setDefaultSockopts(0x4, 0xa, 0x1, 0x0, 0x0, 0x0)
/usr/lib/golang/src/net/sockopt_linux.go:17 +0x7f
net.socket(0x135f188, 0x3, 0xa, 0x1, 0x0, 0x0, 0x7fe031c3cd50, 0xc8205e43f0, 0x0, 0x0, ...)
/usr/lib/golang/src/net/sock_posix.go:42 +0xcb
net.internetSocket(0x135f188, 0x3, 0x7fe031c3cd50, 0xc8205e43f0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x1, ...)
/usr/lib/golang/src/net/ipsock_posix.go:160 +0x141
net.ListenTCP(0x135f188, 0x3, 0xc8205e43f0, 0x7fe02fbf6bc0, 0x0, 0x0)
/usr/lib/golang/src/net/tcpsock_posix.go:324 +0x19b
net.Listen(0x135f188, 0x3, 0xc8205ee100, 0x5, 0x0, 0x0, 0x0, 0x0)
/usr/lib/golang/src/net/dial.go:393 +0x462
net/http.(*Server).ListenAndServe(0xc82007c660, 0x0, 0x0)
/usr/lib/golang/src/net/http/server.go:1827 +0x8e
k8s.io/kubernetes/pkg/kubelet/cadvisor.(*cadvisorClient).exportHTTP.func1(0xc82007c660, 0x1062)
/builddir/build/BUILD/kubernetes-196f58b9cb25a2222c7f9aacd624737910b03acb/_output/local/go/src/k8s.io/kubernetes/pkg/kubelet/cadvisor/cadvisor_linux.go:100 +0x7b
created by k8s.io/kubernetes/pkg/kubelet/cadvisor.(*cadvisorClient).exportHTTP
/builddir/build/BUILD/kubernetes-196f58b9cb25a2222c7f9aacd624737910b03acb/_output/local/go/src/k8s.io/kubernetes/pkg/kubelet/cadvisor/cadvisor_linux.go:106 +0x3b6
goroutine 6 [syscall]:
os/signal.loop()
/usr/lib/golang/src/os/signal/signal_unix.go:22 +0x18
created by os/signal.init.1
/usr/lib/golang/src/os/signal/signal_unix.go:28 +0x37
goroutine 27 [sleep]:
time.Sleep(0x12a05f200)
/usr/lib/golang/src/runtime/time.go:59 +0xf9
k8s.io/kubernetes/pkg/util.Until(0x164e780, 0x12a05f200, 0xc82007cc00)
/builddir/build/BUILD/kubernetes-196f58b9cb25a2222c7f9aacd624737910b03acb/_output/local/go/src/k8s.io/kubernetes/pkg/util/util.go:117 +0x61
created by k8s.io/kubernetes/pkg/util.InitLogs
/builddir/build/BUILD/kubernetes-196f58b9cb25a2222c7f9aacd624737910b03acb/_output/local/go/src/k8s.io/kubernetes/pkg/util/logs.go:49 +0xba
更新 2
第二个问题是证书问题。
openstack cloud provider 插件中的错误消息不是很清楚。
如果配置文件中有任何意外字段，它不会注册，它会抛出一个错误，不幸的是没有正确冒泡。
这是插件识别的字段（及其标识符）
[Global]
AuthUrl stringgcfg:"auth-url"
Username string
UserId stringgcfg:"user-id"
Password string
ApiKey stringgcfg:"api-key"
TenantId stringgcfg:"tenant-id"
TenantName stringgcfg:"tenant-name"
DomainId stringgcfg:"domain-id"
DomainName stringgcfg:"domain-name"
Region string
当然不是所有的都是必需的。我通常
[Global]
auth-url
username
password
region
tenant-id
为了使用 cinder 卷，我将选项 --cloud-provider 和 --cloud-config 添加到我的 kubelet 配置中：
$ cat /etc/kubernetes/kubelet
###
# kubernetes kubelet (node) config
# The address for the info server to serve on (set to 0.0.0.0 or "" for all interfaces)
KUBELET_ADDRESS="--address=0.0.0.0"
# The port for the info server to serve on
# KUBELET_PORT="--port=10250"
# You may leave this blank to use the actual hostname
KUBELET_HOSTNAME="--hostname_override=192.168.100.76"
# location of the api-server
KUBELET_API_SERVER="--api_servers=https://localhost:6443"
# Add your own!
KUBELET_ARGS="--cluster_dns=10.100.0.10 --cluster_domain=cluster.local --kubeconfig=/etc/kubernetes/kubelet.kubeconfig --config=/etc/kubernetes/manifests --cloud-provider=openstack --cloud-config=/etc/kubernetes/cloud_config"
我已经创建了一个包含必要凭据的文件：
$ sudo cat /etc/kubernetes/cloud_config
[Global]
auth-url=https://api.*********.com:5000/v2.0
user-id=kubecindertest
username=kubecindertest
password=*****
region=RegionOne
tenant-name=kubecindertest
tenant-id=6568768756a7886767e676f7efe76fe7
project-name=kubecindertest
当启动kubelet（手动）时，进程只记录unknown cloud provider "openstack"并且存在：
source /etc/kubernetes/kubelet; sudo /usr/bin/kubelet $KUBE_LOGTOSTDERR $KUBE_LOG_LEVEL $KUBELET_API_SERVER $KUBELET_ADDRESS $KUBELET_PORT $KUBELET_HOSTNAME $KUBE_ALLOW_PRIV $KUBELET_ARGS
unknown cloud provider "openstack"
在 kubernetes 存储库中定义 openstack 提供程序的 openstack.go 具有完全相同的小写名称：
const ProviderName = "openstack"
更新
原来错误是在解析配置文件时发生的预期错误。我删除了所有可选或不需要的键，现在我将其用作我的配置文件：
$ sudo cat /etc/kubernetes/cloud_config
[Global]
auth-url=https://api.*********.com:5000/v2.0
username=kubecindertest
password=*****
region=RegionOne
tenant-id=6568768756a7886767e676f7efe76fe7
然而，启动 kublet 只会导致另一个错误：
I0923 07:14:33.315311 23743 manager.go:127] cAdvisor running in container: "/user.slice"
I0923 07:14:33.316263 23743 fs.go:93] Filesystem partitions: map[/dev/vda1:{mountpoint:/ major:253 minor:1}]
I0923 07:14:33.358848 23743 manager.go:158] Machine: {NumCores:2 CpuFrequency:2099998 MemoryCapacity:4144640000 MachineID:dae72fe0cc064eb0b7797f25bfaf69df SystemUUID:BEDAF943-624D-C04A-B92C-4EB07258246C BootID:e2d988e2-9aba-49bf-a344-fd62607a6754 Filesystems:[{Device:/dev/vda1 Capacity:21456445440}] DiskMap:map[252:0:{Name:dm-0 Major:252 Minor:0 Size:107374182400 Scheduler:none} 252:1:{Name:dm-1 Major:252 Minor:1 Size:10737418240 Scheduler:none} 252:2:{Name:dm-2 Major:252 Minor:2 Size:10737418240 Scheduler:none} 253:0:{Name:vda Major:253 Minor:0 Size:21474836480 Scheduler:none}] NetworkDevices:[{Name:eth0 MacAddress:fa:16:3e:64:fa:9a Speed:0 Mtu:1500} {Name:eth1 MacAddress:fa:16:3e:01:00:79 Speed:0 Mtu:1500} {Name:flannel.1 MacAddress:d2:9c:ad:29:df:c5 Speed:0 Mtu:1450}] Topology:[{Id:0 Memory:4294434816 Cores:[{Id:0 Threads:[0] Caches:[{Size:32768 Type:Data Level:1} {Size:32768 Type:Instruction Level:1} {Size:4194304 Type:Unified Level:2}]}] Caches:[]} {Id:1 Memory:0 Cores:[{Id:0 Threads:[1] Caches:[{Size:32768 Type:Data Level:1} {Size:32768 Type:Instruction Level:1} {Size:4194304 Type:Unified Level:2}]}] Caches:[]}] CloudProvider:Unknown InstanceType:Unknown}
I0923 07:14:33.363915 23743 manager.go:164] Version: {KernelVersion:3.10.0-229.11.1.el7.x86_64 ContainerOsVersion:CentOS Linux 7 (Core) DockerVersion:1.8.1.el7 CadvisorVersion:0.16.0}
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x0 pc=0x8559cd]
goroutine 1 [running]:
k8s.io/kubernetes/pkg/cloudprovider/providers/openstack.(*OpenStack).Instances(0x0, 0x0, 0x0, 0xe)
/builddir/build/BUILD/kubernetes-196f58b9cb25a2222c7f9aacd624737910b03acb/_output/local/go/src/k8s.io/kubernetes/pkg/cloudprovider/providers/openstack/openstack.go:167 +0x8ed
k8s.io/kubernetes/cmd/kubelet/app.RunKubelet(0xc820144900, 0x0, 0x0, 0x0)
/builddir/build/BUILD/kubernetes-196f58b9cb25a2222c7f9aacd624737910b03acb/_output/local/go/src/k8s.io/kubernetes/cmd/kubelet/app/server.go:628 +0x13c
k8s.io/kubernetes/cmd/kubelet/app.(*KubeletServer).Run(0xc8202c2000, 0xc820144900, 0x0, 0x0)
/builddir/build/BUILD/kubernetes-196f58b9cb25a2222c7f9aacd624737910b03acb/_output/local/go/src/k8s.io/kubernetes/cmd/kubelet/app/server.go:420 +0x84b
main.main()
/builddir/build/BUILD/kubernetes-196f58b9cb25a2222c7f9aacd624737910b03acb/_output/local/go/src/k8s.io/kubernetes/cmd/kubelet/kubelet.go:46 +0xab
goroutine 17 [syscall, locked to thread]:
runtime.goexit()
/usr/lib/golang/src/runtime/asm_amd64.s:1696 +0x1
goroutine 5 [chan receive]:
github.com/golang/glog.(*loggingT).flushDaemon(0x1dc7000)
/builddir/build/BUILD/kubernetes-196f58b9cb25a2222c7f9aacd624737910b03acb/Godeps/_workspace/src/github.com/golang/glog/glog.go:879 +0x67
created by github.com/golang/glog.init.1
/builddir/build/BUILD/kubernetes-196f58b9cb25a2222c7f9aacd624737910b03acb/Godeps/_workspace/src/github.com/golang/glog/glog.go:410 +0x297
goroutine 37 [runnable]:
syscall.Syscall6(0x36, 0x4, 0x29, 0x1a, 0xc820035a7c, 0x4, 0x0, 0x0, 0x1a, 0x0)
/usr/lib/golang/src/syscall/asm_linux_amd64.s:44 +0x5
syscall.setsockopt(0x4, 0x29, 0x1a, 0xc820035a7c, 0x4, 0x0, 0x0)
/usr/lib/golang/src/syscall/zsyscall_linux_amd64.go:1655 +0x73
syscall.SetsockoptInt(0x4, 0x29, 0x1a, 0x0, 0x0, 0x0)
/usr/lib/golang/src/syscall/syscall_unix.go:267 +0x61
net.setDefaultSockopts(0x4, 0xa, 0x1, 0x0, 0x0, 0x0)
/usr/lib/golang/src/net/sockopt_linux.go:17 +0x7f
net.socket(0x135f188, 0x3, 0xa, 0x1, 0x0, 0x0, 0x7fe031c3cd50, 0xc8205e43f0, 0x0, 0x0, ...)
/usr/lib/golang/src/net/sock_posix.go:42 +0xcb
net.internetSocket(0x135f188, 0x3, 0x7fe031c3cd50, 0xc8205e43f0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x1, ...)
/usr/lib/golang/src/net/ipsock_posix.go:160 +0x141
net.ListenTCP(0x135f188, 0x3, 0xc8205e43f0, 0x7fe02fbf6bc0, 0x0, 0x0)
/usr/lib/golang/src/net/tcpsock_posix.go:324 +0x19b
net.Listen(0x135f188, 0x3, 0xc8205ee100, 0x5, 0x0, 0x0, 0x0, 0x0)
/usr/lib/golang/src/net/dial.go:393 +0x462
net/http.(*Server).ListenAndServe(0xc82007c660, 0x0, 0x0)
/usr/lib/golang/src/net/http/server.go:1827 +0x8e
k8s.io/kubernetes/pkg/kubelet/cadvisor.(*cadvisorClient).exportHTTP.func1(0xc82007c660, 0x1062)
/builddir/build/BUILD/kubernetes-196f58b9cb25a2222c7f9aacd624737910b03acb/_output/local/go/src/k8s.io/kubernetes/pkg/kubelet/cadvisor/cadvisor_linux.go:100 +0x7b
created by k8s.io/kubernetes/pkg/kubelet/cadvisor.(*cadvisorClient).exportHTTP
/builddir/build/BUILD/kubernetes-196f58b9cb25a2222c7f9aacd624737910b03acb/_output/local/go/src/k8s.io/kubernetes/pkg/kubelet/cadvisor/cadvisor_linux.go:106 +0x3b6
goroutine 6 [syscall]:
os/signal.loop()
/usr/lib/golang/src/os/signal/signal_unix.go:22 +0x18
created by os/signal.init.1
/usr/lib/golang/src/os/signal/signal_unix.go:28 +0x37
goroutine 27 [sleep]:
time.Sleep(0x12a05f200)
/usr/lib/golang/src/runtime/time.go:59 +0xf9
k8s.io/kubernetes/pkg/util.Until(0x164e780, 0x12a05f200, 0xc82007cc00)
/builddir/build/BUILD/kubernetes-196f58b9cb25a2222c7f9aacd624737910b03acb/_output/local/go/src/k8s.io/kubernetes/pkg/util/util.go:117 +0x61
created by k8s.io/kubernetes/pkg/util.InitLogs
/builddir/build/BUILD/kubernetes-196f58b9cb25a2222c7f9aacd624737910b03acb/_output/local/go/src/k8s.io/kubernetes/pkg/util/logs.go:49 +0xba
更新 2
第二个问题是证书问题。
openstack cloud provider 插件中的错误消息不是很清楚。 如果配置文件中有任何意外字段，它不会注册，它会抛出一个错误，不幸的是没有正确冒泡。
这是插件识别的字段（及其标识符）
[Global]
AuthUrl stringgcfg:"auth-url"
Username string
UserId stringgcfg:"user-id"
Password string
ApiKey stringgcfg:"api-key"
TenantId stringgcfg:"tenant-id"
TenantName stringgcfg:"tenant-name"
DomainId stringgcfg:"domain-id"
DomainName stringgcfg:"domain-name"
Region string
当然不是所有的都是必需的。我通常
[Global]
auth-url
username
password
region
tenant-id