通过 Kubernetes 安装卷时 Neo4j 关闭
Neo4j shutdown when mounting volume via Kubernetes
我已经为导入必要插件的 neo4j 创建了一个自定义 dockerfile。接下来，我在 Azure 上的 Kubernetes 上创建了一个部署 yaml 运行。部署效果很好，我可以按预期 URL 访问浏览器，效果很好。然而，当我安装一个卷时，neo4j 变得无法启动，并出现下面提供的错误堆栈。似乎 Azure 正在使用干扰 neo4j 的所需目录？但是我不确定如何解决这个问题。
有什么想法吗？
两个注意事项：
- 注意mount的目录是空的，所以还没有数据库。
- 持久卷（声明）已经在像 nginx 这样的简单镜像上进行了测试，并且工作正常。
在错误堆栈和部署 yaml 下方找到。
我的部署yaml
apiVersion: v1
kind: Service
metadata:
name: myservice
spec:
selector:
app: mygraph
ports:
- name: bolt
protocol: TCP
port: 7687
targetPort: 7687
nodePort: 30076
- name: http
protocol: TCP
port: 7474
targetPort: 7474
nodePort: 30074
- name: https
protocol: TCP
port: 7473
targetPort: 7473
nodePort: 30073
type: LoadBalancer
loadBalancerIP: 40.118.172.46
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: mygraph-deployment
labels:
app: mygraph
spec:
replicas: 1
selector:
matchLabels:
app: mygraph
template:
metadata:
labels:
app: mygraph
spec:
containers:
- name: mygraph
image: mycontainer.azurecr.io/mygraph:dev
securityContext:
privileged: true
ports:
- containerPort: 7687
- containerPort: 7474
- containerPort: 7473
env:
- name: NEO4J_dbms_jvm_additional
value: "-Xms512m -Xmx6g"
# When adding the below lines the deployment failes. Without it, it works great!
volumeMounts:
- name: neo4jdata
mountPath: /data
volumes:
- name: neo4jdata
persistentVolumeClaim:
claimName: azurefile
以及对应的错误堆栈
2019-09-05 07:02:12.522+0000 INFO ======== Neo4j 3.3.9 ========
2019-09-05 07:02:12.560+0000 INFO Starting...
2019-09-05 07:02:14.122+0000 INFO Bolt enabled on 0.0.0.0:7687.
2019-09-05 07:02:14.274+0000 ERROR Failed to start Neo4j: Starting Neo4j failed: Component 'org.neo4j.server.database.LifecycleManagingDatabase@149dd36b' was successfully initialized, but failed to start. Please see the attached cause exception "Unable to create directory path [/var/lib/neo4j/data/databases/graph.db] for Neo4j store.". Starting Neo4j failed: Component 'org.neo4j.server.database.LifecycleManagingDatabase@149dd36b' was successfully initialized, but failed to start. Please see the attached cause exception "Unable to create directory path [/var/lib/neo4j/data/databases/graph.db] for Neo4j store.".
org.neo4j.server.ServerStartupException: Starting Neo4j failed: Component 'org.neo4j.server.database.LifecycleManagingDatabase@149dd36b' was successfully initialized, but failed to start. Please see the attached cause exception "Unable to create directory path [/var/lib/neo4j/data/databases/graph.db] for Neo4j store.".
at org.neo4j.server.exception.ServerStartupErrors.translateToServerStartupError(ServerStartupErrors.java:68)
at org.neo4j.server.AbstractNeoServer.start(AbstractNeoServer.java:220)
at org.neo4j.server.ServerBootstrapper.start(ServerBootstrapper.java:111)
at org.neo4j.server.ServerBootstrapper.start(ServerBootstrapper.java:79)
at org.neo4j.server.CommunityEntryPoint.main(CommunityEntryPoint.java:32)
Caused by: org.neo4j.kernel.lifecycle.LifecycleException: Component 'org.neo4j.server.database.LifecycleManagingDatabase@149dd36b' was successfully initialized, but failed to start. Please see the attached cause exception "Unable to create directory path [/var/lib/neo4j/data/databases/graph.db] for Neo4j store.".
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:466)
at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:107)
at org.neo4j.server.AbstractNeoServer.start(AbstractNeoServer.java:212)
... 3 more
Caused by: java.lang.RuntimeException: Error starting org.neo4j.kernel.impl.factory.GraphDatabaseFacadeFactory, /var/lib/neo4j/data/databases/graph.db
at org.neo4j.kernel.impl.factory.GraphDatabaseFacadeFactory.initFacade(GraphDatabaseFacadeFactory.java:211)
at org.neo4j.kernel.impl.factory.GraphDatabaseFacadeFactory.newFacade(GraphDatabaseFacadeFactory.java:126)
at org.neo4j.server.CommunityNeoServer.lambda$static[=11=](CommunityNeoServer.java:58)
at org.neo4j.server.database.LifecycleManagingDatabase.start(LifecycleManagingDatabase.java:88)
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:445)
... 5 more
Caused by: org.neo4j.kernel.lifecycle.LifecycleException: Component 'org.neo4j.kernel.internal.locker.StoreLockerLifecycleAdapter@134d26af' was successfully initialized, but failed to start. Please see the attached cause exception "Unable to create directory path [/var/lib/neo4j/data/databases/graph.db] for Neo4j store.".
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:466)
at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:107)
at org.neo4j.kernel.impl.factory.GraphDatabaseFacadeFactory.initFacade(GraphDatabaseFacadeFactory.java:207)
... 9 more
Caused by: org.neo4j.kernel.StoreLockException: Unable to create path for store dir: /var/lib/neo4j/data/databases/graph.db. Please ensure no other process is using this database, and that the directory is writable (required even for read-only access)
at org.neo4j.kernel.internal.locker.StoreLocker.storeLockException(StoreLocker.java:116)
at org.neo4j.kernel.internal.locker.StoreLocker.checkLock(StoreLocker.java:79)
at org.neo4j.kernel.internal.locker.GlobalStoreLocker.checkLock(GlobalStoreLocker.java:60)
at org.neo4j.kernel.internal.locker.StoreLockerLifecycleAdapter.start(StoreLockerLifecycleAdapter.java:36)
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:445)
... 11 more
Caused by: java.io.IOException: Unable to create directory path [/var/lib/neo4j/data/databases/graph.db] for Neo4j store.
at org.neo4j.io.fs.DefaultFileSystemAbstraction.mkdirs(DefaultFileSystemAbstraction.java:124)
at org.neo4j.kernel.internal.locker.StoreLocker.checkLock(StoreLocker.java:73)
... 14 more
2019-09-05 07:02:14.317+0000 INFO Neo4j Server shutdown initiated by request
发现是Azure写入数据的权限问题。特别是在 StorageClass 的设置中，您可以设置额外的 mountOptions。其中之一是 uid 应该设置为 100 - 用于 neo4j 的用户 ID。
存储类 yaml 示例：
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: azurefile-neo4j
provisioner: kubernetes.io/azure-file
reclaimPolicy: Retain
mountOptions:
- dir_mode=0755
- file_mode=0755
- uid=100 # Allow write access for neo4j
- gid=1000
parameters:
skuName: Standard_LRS
奇怪的是之前/data/dbms文件写成功了，而/data/databases写不出来。似乎 dbms 和数据库是由不同的用户创建的？但那将是一个单独的问题。
我已经为导入必要插件的 neo4j 创建了一个自定义 dockerfile。接下来，我在 Azure 上的 Kubernetes 上创建了一个部署 yaml 运行。部署效果很好，我可以按预期 URL 访问浏览器，效果很好。然而，当我安装一个卷时，neo4j 变得无法启动，并出现下面提供的错误堆栈。似乎 Azure 正在使用干扰 neo4j 的所需目录？但是我不确定如何解决这个问题。
有什么想法吗？
两个注意事项：
- 注意mount的目录是空的，所以还没有数据库。
- 持久卷（声明）已经在像 nginx 这样的简单镜像上进行了测试，并且工作正常。
在错误堆栈和部署 yaml 下方找到。
我的部署yaml
apiVersion: v1
kind: Service
metadata:
name: myservice
spec:
selector:
app: mygraph
ports:
- name: bolt
protocol: TCP
port: 7687
targetPort: 7687
nodePort: 30076
- name: http
protocol: TCP
port: 7474
targetPort: 7474
nodePort: 30074
- name: https
protocol: TCP
port: 7473
targetPort: 7473
nodePort: 30073
type: LoadBalancer
loadBalancerIP: 40.118.172.46
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: mygraph-deployment
labels:
app: mygraph
spec:
replicas: 1
selector:
matchLabels:
app: mygraph
template:
metadata:
labels:
app: mygraph
spec:
containers:
- name: mygraph
image: mycontainer.azurecr.io/mygraph:dev
securityContext:
privileged: true
ports:
- containerPort: 7687
- containerPort: 7474
- containerPort: 7473
env:
- name: NEO4J_dbms_jvm_additional
value: "-Xms512m -Xmx6g"
# When adding the below lines the deployment failes. Without it, it works great!
volumeMounts:
- name: neo4jdata
mountPath: /data
volumes:
- name: neo4jdata
persistentVolumeClaim:
claimName: azurefile
以及对应的错误堆栈
2019-09-05 07:02:12.522+0000 INFO ======== Neo4j 3.3.9 ========
2019-09-05 07:02:12.560+0000 INFO Starting...
2019-09-05 07:02:14.122+0000 INFO Bolt enabled on 0.0.0.0:7687.
2019-09-05 07:02:14.274+0000 ERROR Failed to start Neo4j: Starting Neo4j failed: Component 'org.neo4j.server.database.LifecycleManagingDatabase@149dd36b' was successfully initialized, but failed to start. Please see the attached cause exception "Unable to create directory path [/var/lib/neo4j/data/databases/graph.db] for Neo4j store.". Starting Neo4j failed: Component 'org.neo4j.server.database.LifecycleManagingDatabase@149dd36b' was successfully initialized, but failed to start. Please see the attached cause exception "Unable to create directory path [/var/lib/neo4j/data/databases/graph.db] for Neo4j store.".
org.neo4j.server.ServerStartupException: Starting Neo4j failed: Component 'org.neo4j.server.database.LifecycleManagingDatabase@149dd36b' was successfully initialized, but failed to start. Please see the attached cause exception "Unable to create directory path [/var/lib/neo4j/data/databases/graph.db] for Neo4j store.".
at org.neo4j.server.exception.ServerStartupErrors.translateToServerStartupError(ServerStartupErrors.java:68)
at org.neo4j.server.AbstractNeoServer.start(AbstractNeoServer.java:220)
at org.neo4j.server.ServerBootstrapper.start(ServerBootstrapper.java:111)
at org.neo4j.server.ServerBootstrapper.start(ServerBootstrapper.java:79)
at org.neo4j.server.CommunityEntryPoint.main(CommunityEntryPoint.java:32)
Caused by: org.neo4j.kernel.lifecycle.LifecycleException: Component 'org.neo4j.server.database.LifecycleManagingDatabase@149dd36b' was successfully initialized, but failed to start. Please see the attached cause exception "Unable to create directory path [/var/lib/neo4j/data/databases/graph.db] for Neo4j store.".
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:466)
at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:107)
at org.neo4j.server.AbstractNeoServer.start(AbstractNeoServer.java:212)
... 3 more
Caused by: java.lang.RuntimeException: Error starting org.neo4j.kernel.impl.factory.GraphDatabaseFacadeFactory, /var/lib/neo4j/data/databases/graph.db
at org.neo4j.kernel.impl.factory.GraphDatabaseFacadeFactory.initFacade(GraphDatabaseFacadeFactory.java:211)
at org.neo4j.kernel.impl.factory.GraphDatabaseFacadeFactory.newFacade(GraphDatabaseFacadeFactory.java:126)
at org.neo4j.server.CommunityNeoServer.lambda$static[=11=](CommunityNeoServer.java:58)
at org.neo4j.server.database.LifecycleManagingDatabase.start(LifecycleManagingDatabase.java:88)
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:445)
... 5 more
Caused by: org.neo4j.kernel.lifecycle.LifecycleException: Component 'org.neo4j.kernel.internal.locker.StoreLockerLifecycleAdapter@134d26af' was successfully initialized, but failed to start. Please see the attached cause exception "Unable to create directory path [/var/lib/neo4j/data/databases/graph.db] for Neo4j store.".
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:466)
at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:107)
at org.neo4j.kernel.impl.factory.GraphDatabaseFacadeFactory.initFacade(GraphDatabaseFacadeFactory.java:207)
... 9 more
Caused by: org.neo4j.kernel.StoreLockException: Unable to create path for store dir: /var/lib/neo4j/data/databases/graph.db. Please ensure no other process is using this database, and that the directory is writable (required even for read-only access)
at org.neo4j.kernel.internal.locker.StoreLocker.storeLockException(StoreLocker.java:116)
at org.neo4j.kernel.internal.locker.StoreLocker.checkLock(StoreLocker.java:79)
at org.neo4j.kernel.internal.locker.GlobalStoreLocker.checkLock(GlobalStoreLocker.java:60)
at org.neo4j.kernel.internal.locker.StoreLockerLifecycleAdapter.start(StoreLockerLifecycleAdapter.java:36)
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:445)
... 11 more
Caused by: java.io.IOException: Unable to create directory path [/var/lib/neo4j/data/databases/graph.db] for Neo4j store.
at org.neo4j.io.fs.DefaultFileSystemAbstraction.mkdirs(DefaultFileSystemAbstraction.java:124)
at org.neo4j.kernel.internal.locker.StoreLocker.checkLock(StoreLocker.java:73)
... 14 more
2019-09-05 07:02:14.317+0000 INFO Neo4j Server shutdown initiated by request
发现是Azure写入数据的权限问题。特别是在 StorageClass 的设置中，您可以设置额外的 mountOptions。其中之一是 uid 应该设置为 100 - 用于 neo4j 的用户 ID。
存储类 yaml 示例：
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: azurefile-neo4j
provisioner: kubernetes.io/azure-file
reclaimPolicy: Retain
mountOptions:
- dir_mode=0755
- file_mode=0755
- uid=100 # Allow write access for neo4j
- gid=1000
parameters:
skuName: Standard_LRS
奇怪的是之前/data/dbms文件写成功了，而/data/databases写不出来。似乎 dbms 和数据库是由不同的用户创建的？但那将是一个单独的问题。