Skaffold dev 挂起并出现首次部署消息失败
Skaffold dev just hangs with first deploy message failure
我正在向我的 Kubernetes 集群添加一个 NextJS 前端。我添加了以下文件：
apiVersion: apps/v1
kind: Deployment
metadata:
name: client-depl
spec:
replicas: 1
selector:
matchLabels:
app: client
template:
metadata:
labels:
app: client
spec:
containers:
- name: client
image: ldco2016/client
---
apiVersion: v1
kind: Service
metadata:
name: client-srv
spec:
selector:
app: client
ports:
- name: client
protocol: TCP
port: 3000
targetPort: 3000
到我的 infra/k8s/ 目录，然后像这样重新配置 ingress-srv.yml：
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-service
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/use-regex: "true"
spec:
rules:
- host: ticketing.dev
http:
paths:
- path: /api/users/?(.*)
backend:
serviceName: auth-srv
servicePort: 3000
- path: /?(.*)
backend:
serviceName: client-srv
servicePort: 3000
和 skaffold.yml 文件：
apiVersion: skaffold/v2alpha3
kind: Config
deploy:
kubectl:
manifests:
- ./infra/k8s*
build:
local:
push: false
artifacts:
- image: ldco2016/auth
context: auth
docker:
dockerfile: Dockerfile
sync:
manual:
- src: "src/**/*.ts"
dest: .
- image: ldco2016/client
context: client
docker:
dockerfile: Dockerfile
sync:
manual:
- src: "**/*.js"
dest: .
当我 运行 skaffold dev 它挂在这里：
starting deploy...
- deployment.apps/auth-depl created
- service/auth-srv created
- deployment.apps/auth-mongo-depl created
- service/auth-mongo-srv created
- deployment.apps/client-depl created
- service/client-srv created
- ingress.extensions/ingress-service created
Waiting for deployments to stabilize...
- deployment/auth-depl: waiting for rollout to finish: 0 of 1 updated replicas are available...
- deployment/auth-mongo-depl: waiting for rollout to finish: 0 of 1 updated replicas are available...
- deployment/client-depl: waiting for rollout to finish: 0 of 1 updated replicas are available...
- deployment/client-depl is ready. [2/3 deployment(s) still pending]
- deployment/auth-mongo-depl is ready. [1/3 deployment(s) still pending]
有什么想法吗？
我也在 运行宁 Docker 桌面和 Kubernetes。由于这是一个微服务应用程序，我认为 Kubernetes 可能需要更多资源。我尝试添加更多资源，但并没有解决问题。
我怀疑问题出在我的一个 pods 上，所以我 运行: kubectl get pods
NAME READY STATUS RESTARTS AGE
auth-depl-5867ffb6bd-n5s6w 0/1 CreateContainerConfigError 0 2m7s
auth-depl-669fc8fd66-qr8kj 0/1 CreateContainerConfigError 0 6m11s
auth-mongo-depl-585f5f978c-tnc9w 1/1 Running 0 2m7s
所以问题似乎出在我的 auth-depl 上，所以我查看了它的 yaml 文件，我怀疑问题出在我添加的密钥上，所以我将其注释掉：
apiVersion: apps/v1
kind: Deployment
metadata:
name: auth-depl
spec:
replicas: 1
selector:
matchLabels:
app: auth
template:
metadata:
labels:
app: auth
spec:
containers:
- name: auth
image: ldco2016/auth
# env:
# - name: JWT_KEY
# valueFrom:
# secretKeyRef:
# name: jwt-secret
# key: JWT_KEY
---
apiVersion: v1
kind: Service
metadata:
name: auth-srv
spec:
selector:
app: auth
ports:
- name: auth
protocol: TCP
port: 3000
targetPort: 3000
然后我 运行 skaffold dev --cleanup=false 看到了：
Listing files to watch...
- ldco2016/auth
Generating tags...
- ldco2016/auth -> ldco2016/auth:latest
Some taggers failed. Rerun with -vdebug for errors.
Checking cache...
- ldco2016/auth: Found Locally
Tags used in deployment:
- ldco2016/auth -> ldco2016/auth:367e6b2171c5c8477a3f3458d23dd73030f35716df45a290aa54baa5f4dcdaa1
Starting deploy...
- deployment.apps/auth-depl configured
- service/auth-srv configured
- deployment.apps/auth-mongo-depl configured
- service/auth-mongo-srv configured
- ingress.extensions/ingress-service configured
Waiting for deployments to stabilize...
- deployment/auth-depl: waiting for rollout to finish: 1 old replicas are pending termination...
- deployment/auth-mongo-depl: waiting for rollout to finish: 1 old replicas are pending termination...
- deployment/auth-depl is ready. [1/2 deployment(s) still pending]
- deployment/auth-mongo-depl is ready.
Deployments stabilized in 3.633465001s
Watching for changes...
[auth-depl-5c59699679-tnzk2 auth]
[auth-depl-5c59699679-tnzk2 auth] > auth@1.0.0 start /app
[auth-depl-5c59699679-tnzk2 auth] > nodemon ./src/index.ts
[auth-depl-5c59699679-tnzk2 auth]
[auth-depl-5c59699679-tnzk2 auth] [nodemon] 2.0.5
[auth-depl-5c59699679-tnzk2 auth] [nodemon] to restart at any time, enter `rs`
[auth-depl-5c59699679-tnzk2 auth] [nodemon] watching path(s): *.*
[auth-depl-5c59699679-tnzk2 auth] [nodemon] watching extensions: ts,json
[auth-depl-5c59699679-tnzk2 auth] [nodemon] starting `ts-node ./src/index.ts`
[auth-depl-5c59699679-tnzk2 auth] (node:40) UnhandledPromiseRejectionWarning: Error: JWT must be defined
这是一个很大的线索，因为当我去 kubectl get secrets 我发现我的 JWT 不再在 Kubernetes 秘密中，我相信那是因为我的机器最近无意中重启了，这意味着我忘记了单击前述直到稍后时间，它在那天晚上晚些时候重新启动，这重新启动了我使用 Kubernetes 的 Docker 桌面的本地副本。
所以我再次 运行 kubectl create secret... 命令然后 运行 kubectl get secrets 再次看到我的密钥。
我在我的 auth-depl.yml 文件中使用密钥或该密钥的值添加回那些环境变量，然后再次 运行 skaffold dev --cleanup=false 并且：
Listing files to watch...
- ldco2016/auth
Generating tags...
- ldco2016/auth -> ldco2016/auth:latest
Some taggers failed. Rerun with -vdebug for errors.
Checking cache...
- ldco2016/auth: Found Locally
Tags used in deployment:
- ldco2016/auth -> ldco2016/auth:367e6b2171c5c8477a3f3458d23dd73030f35716df45a290aa54baa5f4dcdaa1
Starting deploy...
- deployment.apps/auth-depl configured
- service/auth-srv configured
- deployment.apps/auth-mongo-depl configured
- service/auth-mongo-srv configured
- ingress.extensions/ingress-service configured
Waiting for deployments to stabilize...
- deployment/auth-depl: waiting for rollout to finish: 1 old replicas are pending termination...
- deployment/auth-mongo-depl: waiting for rollout to finish: 1 old replicas are pending termination...
- deployment/auth-depl is ready. [1/2 deployment(s) still pending]
- deployment/auth-mongo-depl is ready.
Deployments stabilized in 3.612848017s
Watching for changes...
[auth-depl-5c59699679-tnzk2 auth] Error from server (BadRequest): container "auth" in pod "auth-depl-5c59699679-tnzk2" is terminated
[auth-depl-7d9bf44d9f-n9rcq auth]
[auth-depl-7d9bf44d9f-n9rcq auth] > auth@1.0.0 start /app
[auth-depl-7d9bf44d9f-n9rcq auth] > nodemon ./src/index.ts
[auth-depl-7d9bf44d9f-n9rcq auth]
[auth-depl-7d9bf44d9f-n9rcq auth] [nodemon] 2.0.5
[auth-depl-7d9bf44d9f-n9rcq auth] [nodemon] to restart at any time, enter `rs`
[auth-depl-7d9bf44d9f-n9rcq auth] [nodemon] watching path(s): *.*
WARN[0004] exit status 1
[auth-depl-7d9bf44d9f-n9rcq auth] [nodemon] watching extensions: ts,json
[auth-depl-7d9bf44d9f-n9rcq auth] [nodemon] starting `ts-node ./src/index.ts`
[auth-depl-7d9bf44d9f-n9rcq auth] Connected to MongoDB
[auth-depl-7d9bf44d9f-n9rcq auth] Listening on port 3000!!!!!
恢复业务。
更改所有 yaml 配置文件，类似于这些更改。我们必须为每个 yaml 文件添加资源限制：
apiVersion: apps/v1
kind: Deployment
metadata:
name: client-depl
spec:
replicas: 1
selector:
matchLabels:
app: client
template:
metadata:
labels:
app: client
spec:
containers:
- name: client
image: ldco2016/client
resources:
limits:
memory: 512Mi
cpu: "1"
requests:
memory: 256Mi
cpu: "0.2"
---
apiVersion: v1
kind: Service
metadata:
name: client-srv
spec:
type: ClusterIP
selector:
app: client
ports:
- name: client
protocol: TCP
port: 3000
targetPort: 3000
我正在向我的 Kubernetes 集群添加一个 NextJS 前端。我添加了以下文件：
apiVersion: apps/v1
kind: Deployment
metadata:
name: client-depl
spec:
replicas: 1
selector:
matchLabels:
app: client
template:
metadata:
labels:
app: client
spec:
containers:
- name: client
image: ldco2016/client
---
apiVersion: v1
kind: Service
metadata:
name: client-srv
spec:
selector:
app: client
ports:
- name: client
protocol: TCP
port: 3000
targetPort: 3000
到我的 infra/k8s/ 目录，然后像这样重新配置 ingress-srv.yml：
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-service
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/use-regex: "true"
spec:
rules:
- host: ticketing.dev
http:
paths:
- path: /api/users/?(.*)
backend:
serviceName: auth-srv
servicePort: 3000
- path: /?(.*)
backend:
serviceName: client-srv
servicePort: 3000
和 skaffold.yml 文件：
apiVersion: skaffold/v2alpha3
kind: Config
deploy:
kubectl:
manifests:
- ./infra/k8s*
build:
local:
push: false
artifacts:
- image: ldco2016/auth
context: auth
docker:
dockerfile: Dockerfile
sync:
manual:
- src: "src/**/*.ts"
dest: .
- image: ldco2016/client
context: client
docker:
dockerfile: Dockerfile
sync:
manual:
- src: "**/*.js"
dest: .
当我 运行 skaffold dev 它挂在这里：
starting deploy...
- deployment.apps/auth-depl created
- service/auth-srv created
- deployment.apps/auth-mongo-depl created
- service/auth-mongo-srv created
- deployment.apps/client-depl created
- service/client-srv created
- ingress.extensions/ingress-service created
Waiting for deployments to stabilize...
- deployment/auth-depl: waiting for rollout to finish: 0 of 1 updated replicas are available...
- deployment/auth-mongo-depl: waiting for rollout to finish: 0 of 1 updated replicas are available...
- deployment/client-depl: waiting for rollout to finish: 0 of 1 updated replicas are available...
- deployment/client-depl is ready. [2/3 deployment(s) still pending]
- deployment/auth-mongo-depl is ready. [1/3 deployment(s) still pending]
有什么想法吗？
我也在 运行宁 Docker 桌面和 Kubernetes。由于这是一个微服务应用程序，我认为 Kubernetes 可能需要更多资源。我尝试添加更多资源，但并没有解决问题。
我怀疑问题出在我的一个 pods 上，所以我 运行: kubectl get pods
NAME READY STATUS RESTARTS AGE
auth-depl-5867ffb6bd-n5s6w 0/1 CreateContainerConfigError 0 2m7s
auth-depl-669fc8fd66-qr8kj 0/1 CreateContainerConfigError 0 6m11s
auth-mongo-depl-585f5f978c-tnc9w 1/1 Running 0 2m7s
所以问题似乎出在我的 auth-depl 上，所以我查看了它的 yaml 文件，我怀疑问题出在我添加的密钥上，所以我将其注释掉：
apiVersion: apps/v1
kind: Deployment
metadata:
name: auth-depl
spec:
replicas: 1
selector:
matchLabels:
app: auth
template:
metadata:
labels:
app: auth
spec:
containers:
- name: auth
image: ldco2016/auth
# env:
# - name: JWT_KEY
# valueFrom:
# secretKeyRef:
# name: jwt-secret
# key: JWT_KEY
---
apiVersion: v1
kind: Service
metadata:
name: auth-srv
spec:
selector:
app: auth
ports:
- name: auth
protocol: TCP
port: 3000
targetPort: 3000
然后我 运行 skaffold dev --cleanup=false 看到了：
Listing files to watch...
- ldco2016/auth
Generating tags...
- ldco2016/auth -> ldco2016/auth:latest
Some taggers failed. Rerun with -vdebug for errors.
Checking cache...
- ldco2016/auth: Found Locally
Tags used in deployment:
- ldco2016/auth -> ldco2016/auth:367e6b2171c5c8477a3f3458d23dd73030f35716df45a290aa54baa5f4dcdaa1
Starting deploy...
- deployment.apps/auth-depl configured
- service/auth-srv configured
- deployment.apps/auth-mongo-depl configured
- service/auth-mongo-srv configured
- ingress.extensions/ingress-service configured
Waiting for deployments to stabilize...
- deployment/auth-depl: waiting for rollout to finish: 1 old replicas are pending termination...
- deployment/auth-mongo-depl: waiting for rollout to finish: 1 old replicas are pending termination...
- deployment/auth-depl is ready. [1/2 deployment(s) still pending]
- deployment/auth-mongo-depl is ready.
Deployments stabilized in 3.633465001s
Watching for changes...
[auth-depl-5c59699679-tnzk2 auth]
[auth-depl-5c59699679-tnzk2 auth] > auth@1.0.0 start /app
[auth-depl-5c59699679-tnzk2 auth] > nodemon ./src/index.ts
[auth-depl-5c59699679-tnzk2 auth]
[auth-depl-5c59699679-tnzk2 auth] [nodemon] 2.0.5
[auth-depl-5c59699679-tnzk2 auth] [nodemon] to restart at any time, enter `rs`
[auth-depl-5c59699679-tnzk2 auth] [nodemon] watching path(s): *.*
[auth-depl-5c59699679-tnzk2 auth] [nodemon] watching extensions: ts,json
[auth-depl-5c59699679-tnzk2 auth] [nodemon] starting `ts-node ./src/index.ts`
[auth-depl-5c59699679-tnzk2 auth] (node:40) UnhandledPromiseRejectionWarning: Error: JWT must be defined
这是一个很大的线索，因为当我去 kubectl get secrets 我发现我的 JWT 不再在 Kubernetes 秘密中，我相信那是因为我的机器最近无意中重启了，这意味着我忘记了单击前述直到稍后时间，它在那天晚上晚些时候重新启动，这重新启动了我使用 Kubernetes 的 Docker 桌面的本地副本。
所以我再次 运行 kubectl create secret... 命令然后 运行 kubectl get secrets 再次看到我的密钥。
我在我的 auth-depl.yml 文件中使用密钥或该密钥的值添加回那些环境变量，然后再次 运行 skaffold dev --cleanup=false 并且：
Listing files to watch...
- ldco2016/auth
Generating tags...
- ldco2016/auth -> ldco2016/auth:latest
Some taggers failed. Rerun with -vdebug for errors.
Checking cache...
- ldco2016/auth: Found Locally
Tags used in deployment:
- ldco2016/auth -> ldco2016/auth:367e6b2171c5c8477a3f3458d23dd73030f35716df45a290aa54baa5f4dcdaa1
Starting deploy...
- deployment.apps/auth-depl configured
- service/auth-srv configured
- deployment.apps/auth-mongo-depl configured
- service/auth-mongo-srv configured
- ingress.extensions/ingress-service configured
Waiting for deployments to stabilize...
- deployment/auth-depl: waiting for rollout to finish: 1 old replicas are pending termination...
- deployment/auth-mongo-depl: waiting for rollout to finish: 1 old replicas are pending termination...
- deployment/auth-depl is ready. [1/2 deployment(s) still pending]
- deployment/auth-mongo-depl is ready.
Deployments stabilized in 3.612848017s
Watching for changes...
[auth-depl-5c59699679-tnzk2 auth] Error from server (BadRequest): container "auth" in pod "auth-depl-5c59699679-tnzk2" is terminated
[auth-depl-7d9bf44d9f-n9rcq auth]
[auth-depl-7d9bf44d9f-n9rcq auth] > auth@1.0.0 start /app
[auth-depl-7d9bf44d9f-n9rcq auth] > nodemon ./src/index.ts
[auth-depl-7d9bf44d9f-n9rcq auth]
[auth-depl-7d9bf44d9f-n9rcq auth] [nodemon] 2.0.5
[auth-depl-7d9bf44d9f-n9rcq auth] [nodemon] to restart at any time, enter `rs`
[auth-depl-7d9bf44d9f-n9rcq auth] [nodemon] watching path(s): *.*
WARN[0004] exit status 1
[auth-depl-7d9bf44d9f-n9rcq auth] [nodemon] watching extensions: ts,json
[auth-depl-7d9bf44d9f-n9rcq auth] [nodemon] starting `ts-node ./src/index.ts`
[auth-depl-7d9bf44d9f-n9rcq auth] Connected to MongoDB
[auth-depl-7d9bf44d9f-n9rcq auth] Listening on port 3000!!!!!
恢复业务。
更改所有 yaml 配置文件，类似于这些更改。我们必须为每个 yaml 文件添加资源限制：
apiVersion: apps/v1
kind: Deployment
metadata:
name: client-depl
spec:
replicas: 1
selector:
matchLabels:
app: client
template:
metadata:
labels:
app: client
spec:
containers:
- name: client
image: ldco2016/client
resources:
limits:
memory: 512Mi
cpu: "1"
requests:
memory: 256Mi
cpu: "0.2"
---
apiVersion: v1
kind: Service
metadata:
name: client-srv
spec:
type: ClusterIP
selector:
app: client
ports:
- name: client
protocol: TCP
port: 3000
targetPort: 3000