领导节点失败并显示 /tmp/spark-jobserver/filedao/data/jars.data(权限被拒绝)
Lead node fails with /tmp/spark-jobserver/filedao/data/jars.data (Permission denied)
SnappyData v.0-5
我以非 root 用户身份登录 Ubuntu,'foo'。
SnappyData directory/install 由 'foo' 用户和 'foo' 组所有。
我在此处使用脚本启动所有节点(定位器、领导、服务器):
SNAPPY_HOME/sbin/snappy-start-all.sh
定位器启动。
服务器启动。
Lead 死于此错误。
16/07/21 23:12:26.883 UTC serverConnector INFO JobFileDAO:
rootDir is /tmp/spark-jobserver/filedao/data 16/07/21 23:12:26.888 UTC
serverConnector ERROR JobServer$: Unable to start Spark
JobServer: java.lang.reflect.InvocationTargetException at
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at spark.jobserver.JobServer$.start(JobServer.scala:69) at
io.snappydata.impl.LeadImpl.startAddOnServices(LeadImpl.scala:283) at
io.snappydata.impl.LeadImpl$.invokeLeadStartAddonService(LeadImpl.scala:360)
at
io.snappydata.ToolsCallbackImpl$.invokeLeadStartAddonService(ToolsCallbackImpl.scala:28)
at
org.apache.spark.sql.SnappyContext$.invokeServices(SnappyContext.scala:1362)
at
org.apache.spark.sql.SnappyContext$.initGlobalSnappyContext(SnappyContext.scala:1340)
at org.apache.spark.sql.SnappyContext.(SnappyContext.scala:104)
at org.apache.spark.sql.SnappyContext.(SnappyContext.scala:95)
at
org.apache.spark.sql.SnappyContext$.newSnappyContext(SnappyContext.scala:1221)
at
org.apache.spark.sql.SnappyContext$.apply(SnappyContext.scala:1249)
at
org.apache.spark.scheduler.SnappyTaskSchedulerImpl.postStartHook(SnappyTaskSchedulerImpl.scala:25)
at org.apache.spark.SparkContext.(SparkContext.scala:601) at
io.snappydata.impl.LeadImpl.start(LeadImpl.scala:129) at
io.snappydata.impl.ServerImpl.start(ServerImpl.scala:32) at
io.snappydata.tools.LeaderLauncher.startServerVM(LeaderLauncher.scala:91)
at
com.pivotal.gemfirexd.tools.internal.GfxdServerLauncher.connect(GfxdServerLauncher.java:174)
at
com.gemstone.gemfire.internal.cache.CacheServerLauncher$AsyncServerLauncher.run(CacheServerLauncher.java:1003)
at java.lang.Thread.run(Thread.java:745) Caused by:
java.io.FileNotFoundException:
/tmp/spark-jobserver/filedao/data/jars.data (Permission denied) at
java.io.FileOutputStream.open0(Native Method) at
java.io.FileOutputStream.open(FileOutputStream.java:270) at
java.io.FileOutputStream.(FileOutputStream.java:213) at
spark.jobserver.io.JobFileDAO.init(JobFileDAO.scala:90) at
spark.jobserver.io.JobFileDAO.(JobFileDAO.scala:30) ... 22 more
16/07/21 23:12:26.891 UTC Distributed system shutdown hook
INFO snappystore: VM is exiting - shutting down distributed system
我是否需要成为其他用户才能启动 Lead 节点?使用 'sudo'?配置一个属性 告诉Spark 使用一个目录'foo' 有权限?提前自己创建这个目录?
/tmp/spark-jobserver 的当前所有者似乎是其他用户。检查该目录的权限并将其删除。
如果多个用户将 运行 在同一台机器上引导,您可以将作业服务器目录配置到其他地方,如提到的 here. The relevant properties can be found in application.conf 来源。这可能是麻烦多于价值,所以现在只需确保单个用户在一台机器上启动领先节点会更容易。
我们将在下一版本 (SNAP-69) 中将默认设置固定在 work/ 目录中。
SnappyData v.0-5
我以非 root 用户身份登录 Ubuntu,'foo'。 SnappyData directory/install 由 'foo' 用户和 'foo' 组所有。
我在此处使用脚本启动所有节点(定位器、领导、服务器):
SNAPPY_HOME/sbin/snappy-start-all.sh
定位器启动。 服务器启动。 Lead 死于此错误。
16/07/21 23:12:26.883 UTC serverConnector INFO JobFileDAO: rootDir is /tmp/spark-jobserver/filedao/data 16/07/21 23:12:26.888 UTC serverConnector ERROR JobServer$: Unable to start Spark JobServer: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at spark.jobserver.JobServer$.start(JobServer.scala:69) at io.snappydata.impl.LeadImpl.startAddOnServices(LeadImpl.scala:283) at io.snappydata.impl.LeadImpl$.invokeLeadStartAddonService(LeadImpl.scala:360) at io.snappydata.ToolsCallbackImpl$.invokeLeadStartAddonService(ToolsCallbackImpl.scala:28) at org.apache.spark.sql.SnappyContext$.invokeServices(SnappyContext.scala:1362) at org.apache.spark.sql.SnappyContext$.initGlobalSnappyContext(SnappyContext.scala:1340) at org.apache.spark.sql.SnappyContext.(SnappyContext.scala:104) at org.apache.spark.sql.SnappyContext.(SnappyContext.scala:95) at org.apache.spark.sql.SnappyContext$.newSnappyContext(SnappyContext.scala:1221) at org.apache.spark.sql.SnappyContext$.apply(SnappyContext.scala:1249) at org.apache.spark.scheduler.SnappyTaskSchedulerImpl.postStartHook(SnappyTaskSchedulerImpl.scala:25) at org.apache.spark.SparkContext.(SparkContext.scala:601) at io.snappydata.impl.LeadImpl.start(LeadImpl.scala:129) at io.snappydata.impl.ServerImpl.start(ServerImpl.scala:32) at io.snappydata.tools.LeaderLauncher.startServerVM(LeaderLauncher.scala:91) at com.pivotal.gemfirexd.tools.internal.GfxdServerLauncher.connect(GfxdServerLauncher.java:174) at com.gemstone.gemfire.internal.cache.CacheServerLauncher$AsyncServerLauncher.run(CacheServerLauncher.java:1003) at java.lang.Thread.run(Thread.java:745) Caused by: java.io.FileNotFoundException: /tmp/spark-jobserver/filedao/data/jars.data (Permission denied) at java.io.FileOutputStream.open0(Native Method) at java.io.FileOutputStream.open(FileOutputStream.java:270) at java.io.FileOutputStream.(FileOutputStream.java:213) at spark.jobserver.io.JobFileDAO.init(JobFileDAO.scala:90) at spark.jobserver.io.JobFileDAO.(JobFileDAO.scala:30) ... 22 more 16/07/21 23:12:26.891 UTC Distributed system shutdown hook INFO snappystore: VM is exiting - shutting down distributed system
我是否需要成为其他用户才能启动 Lead 节点?使用 'sudo'?配置一个属性 告诉Spark 使用一个目录'foo' 有权限?提前自己创建这个目录?
/tmp/spark-jobserver 的当前所有者似乎是其他用户。检查该目录的权限并将其删除。
如果多个用户将 运行 在同一台机器上引导,您可以将作业服务器目录配置到其他地方,如提到的 here. The relevant properties can be found in application.conf 来源。这可能是麻烦多于价值,所以现在只需确保单个用户在一台机器上启动领先节点会更容易。
我们将在下一版本 (SNAP-69) 中将默认设置固定在 work/ 目录中。