名称节点和辅助名称节点未启动
Name node and secondary name nodes are not starting
我是 Hadoop 世界的新手,我正在尝试以伪分布式方式在我的机器上从这个 tutorial. I have freshly installed Hadoop 2.9.2 学习基础知识,并进行了以下设置。
核心-site.xml
<configuration>
<propeerty>
<name>fs.defaultFS</name>
<value>hdfs://localhost:9000</value>
</propeerty>
</configuration>
hdfs-site.xml
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
</configuration>
mapred-site.xml
<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
</configuration>
纱-site.xml
<configuration>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
</configuration>
运行 bin/hdfs namenode -format
一切顺利,没有任何错误,最后 运行 sbin/start-dfs.sh
没有给我以下日志
的错误
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.apache.hadoop.security.authentication.util.KerberosUtil (file:/Users/myusername/hadoop-install/hadoop-2.9.2/share/hadoop/common/lib/hadoop-auth-2.9.2.jar) to method sun.security.krb5.Config.getInstance()
WARNING: Please consider reporting this to the maintainers of org.apache.hadoop.security.authentication.util.KerberosUtil
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
20/03/30 21:44:16 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Incorrect configuration: namenode address dfs.namenode.servicerpc-address or dfs.namenode.rpc-address is not configured.
Starting namenodes on []
Password:
localhost: starting namenode, logging to /Users/myusername/hadoop-install/hadoop-2.9.2/logs/hadoop-myusername-namenode-C02WT020HTDG.out
Password:
localhost: starting datanode, logging to /Users/myusername/hadoop-install/hadoop-2.9.2/logs/hadoop-myusername-datanode-C02WT020HTDG.out
localhost: WARNING: An illegal reflective access operation has occurred
localhost: WARNING: Illegal reflective access by org.apache.hadoop.security.authentication.util.KerberosUtil (file:/Users/myusername/hadoop-install/hadoop-2.9.2/share/hadoop/common/lib/hadoop-auth-2.9.2.jar) to method sun.security.krb5.Config.getInstance()
localhost: WARNING: Please consider reporting this to the maintainers of org.apache.hadoop.security.authentication.util.KerberosUtil
localhost: WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
localhost: WARNING: All illegal access operations will be denied in a future release
Starting secondary namenodes [0.0.0.0]
The authenticity of host '0.0.0.0 (0.0.0.0)' can't be established.
ECDSA key fingerprint is SHA256:M1QP8tl98stYKNcIBmKYTuRoasil3AafGqIq3FZ1Vv8.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
0.0.0.0: Warning: Permanently added '0.0.0.0' (ECDSA) to the list of known hosts.
Password:
0.0.0.0: starting secondarynamenode, logging to /Users/myusername/hadoop-install/hadoop-2.9.2/logs/hadoop-myusername-secondarynamenode-C02WT020HTDG.out
0.0.0.0: WARNING: An illegal reflective access operation has occurred
0.0.0.0: WARNING: Illegal reflective access by org.apache.hadoop.security.authentication.util.KerberosUtil (file:/Users/myusername/hadoop-install/hadoop-2.9.2/share/hadoop/common/lib/hadoop-auth-2.9.2.jar) to method sun.security.krb5.Config.getInstance()
0.0.0.0: WARNING: Please consider reporting this to the maintainers of org.apache.hadoop.security.authentication.util.KerberosUtil
0.0.0.0: WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
0.0.0.0: WARNING: All illegal access operations will be denied in a future release
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.apache.hadoop.security.authentication.util.KerberosUtil (file:/Users/myusername/hadoop-install/hadoop-2.9.2/share/hadoop/common/lib/hadoop-auth-2.9.2.jar) to method sun.security.krb5.Config.getInstance()
WARNING: Please consider reporting this to the maintainers of org.apache.hadoop.security.authentication.util.KerberosUtil
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
20/03/30 21:44:48 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[1]: https://app.pluralsight.com/library/courses/building-blocks-hadoop-hdfs-mapreduce-yarn/table-of-contents
名称节点日志中的错误:
2020-03-31 09:35:56,962 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: Exception in secureMain
java.io.IOException: Incorrect configuration: namenode address dfs.namenode.servicerpc-address or dfs.namenode.rpc-address is not configured.
at org.apache.hadoop.hdfs.DFSUtil.getNNServiceRpcAddressesForCluster(DFSUtil.java:576)
at org.apache.hadoop.hdfs.server.datanode.BlockPoolManager.refreshNamenodes(BlockPoolManager.java:152)
at org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:1392)
at org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:495)
at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:2695)
at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:2598)
at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:2645)
at org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:2789)
at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:2813)
2020-03-31 09:35:56,966 INFO org.apache.hadoop.util.ExitUtil: Exiting with status 1: java.io.IOException: Incorrect configuration: namenode address dfs.namenode.servicerpc-address or dfs.namenode.rpc-address is not configured.
二级名称节点日志:
2020-03-31 09:36:05,237 FATAL org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode: Failed to start secondary namenode
java.lang.IllegalArgumentException: Invalid URI for NameNode address (check fs.defaultFS): file:/// has no authority.
at org.apache.hadoop.hdfs.DFSUtilClient.getNNAddress(DFSUtilClient.java:626)
at org.apache.hadoop.hdfs.DFSUtilClient.getNNAddressCheckLogical(DFSUtilClient.java:655)
at org.apache.hadoop.hdfs.DFSUtilClient.getNNAddress(DFSUtilClient.java:617)
at org.apache.hadoop.hdfs.server.namenode.NameNode.getServiceAddress(NameNode.java:519)
at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.initialize(SecondaryNameNode.java:231)
at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.<init>(SecondaryNameNode.java:194)
at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.main(SecondaryNameNode.java:690)
2020-03-31 09:36:05,241 INFO org.apache.hadoop.util.ExitUtil: Exiting with status 1: ExitException
当我转到 localhost:50070
时,我什么也没得到(无法访问站点),当我点击 jps
时,我没有看到任何名称节点或辅助名称节点 运行。
我做了什么?
- 参考 Incorrect configuration: namenode address dfs.namenode.rpc-address is not configured 并确保任何配置中都没有空格(已在说明中列出)
是否有需要配置的东西或我缺少的东西?
提前致谢!
core-site.xml
属性 标签有错字!。必须是 property
而不是 propeerty
.
<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://localhost:9000</value>
</property>
</configuration>
此外,在 hdfs-site.xml
中您没有提到 dfs.namenode.name.dir
和 dfs.datanode.data.dir
属性。始终建议配置这些属性,否则将 hadoop.tmp.dir
属性(在 core-site.xml
中)配置到 /tmp
以外的某个目录。
更新这些属性,格式化名称节点并启动服务。
我是 Hadoop 世界的新手,我正在尝试以伪分布式方式在我的机器上从这个 tutorial. I have freshly installed Hadoop 2.9.2 学习基础知识,并进行了以下设置。
核心-site.xml
<configuration>
<propeerty>
<name>fs.defaultFS</name>
<value>hdfs://localhost:9000</value>
</propeerty>
</configuration>
hdfs-site.xml
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
</configuration>
mapred-site.xml
<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
</configuration>
纱-site.xml
<configuration>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
</configuration>
运行 bin/hdfs namenode -format
一切顺利,没有任何错误,最后 运行 sbin/start-dfs.sh
没有给我以下日志
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.apache.hadoop.security.authentication.util.KerberosUtil (file:/Users/myusername/hadoop-install/hadoop-2.9.2/share/hadoop/common/lib/hadoop-auth-2.9.2.jar) to method sun.security.krb5.Config.getInstance()
WARNING: Please consider reporting this to the maintainers of org.apache.hadoop.security.authentication.util.KerberosUtil
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
20/03/30 21:44:16 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Incorrect configuration: namenode address dfs.namenode.servicerpc-address or dfs.namenode.rpc-address is not configured.
Starting namenodes on []
Password:
localhost: starting namenode, logging to /Users/myusername/hadoop-install/hadoop-2.9.2/logs/hadoop-myusername-namenode-C02WT020HTDG.out
Password:
localhost: starting datanode, logging to /Users/myusername/hadoop-install/hadoop-2.9.2/logs/hadoop-myusername-datanode-C02WT020HTDG.out
localhost: WARNING: An illegal reflective access operation has occurred
localhost: WARNING: Illegal reflective access by org.apache.hadoop.security.authentication.util.KerberosUtil (file:/Users/myusername/hadoop-install/hadoop-2.9.2/share/hadoop/common/lib/hadoop-auth-2.9.2.jar) to method sun.security.krb5.Config.getInstance()
localhost: WARNING: Please consider reporting this to the maintainers of org.apache.hadoop.security.authentication.util.KerberosUtil
localhost: WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
localhost: WARNING: All illegal access operations will be denied in a future release
Starting secondary namenodes [0.0.0.0]
The authenticity of host '0.0.0.0 (0.0.0.0)' can't be established.
ECDSA key fingerprint is SHA256:M1QP8tl98stYKNcIBmKYTuRoasil3AafGqIq3FZ1Vv8.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
0.0.0.0: Warning: Permanently added '0.0.0.0' (ECDSA) to the list of known hosts.
Password:
0.0.0.0: starting secondarynamenode, logging to /Users/myusername/hadoop-install/hadoop-2.9.2/logs/hadoop-myusername-secondarynamenode-C02WT020HTDG.out
0.0.0.0: WARNING: An illegal reflective access operation has occurred
0.0.0.0: WARNING: Illegal reflective access by org.apache.hadoop.security.authentication.util.KerberosUtil (file:/Users/myusername/hadoop-install/hadoop-2.9.2/share/hadoop/common/lib/hadoop-auth-2.9.2.jar) to method sun.security.krb5.Config.getInstance()
0.0.0.0: WARNING: Please consider reporting this to the maintainers of org.apache.hadoop.security.authentication.util.KerberosUtil
0.0.0.0: WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
0.0.0.0: WARNING: All illegal access operations will be denied in a future release
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.apache.hadoop.security.authentication.util.KerberosUtil (file:/Users/myusername/hadoop-install/hadoop-2.9.2/share/hadoop/common/lib/hadoop-auth-2.9.2.jar) to method sun.security.krb5.Config.getInstance()
WARNING: Please consider reporting this to the maintainers of org.apache.hadoop.security.authentication.util.KerberosUtil
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
20/03/30 21:44:48 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[1]: https://app.pluralsight.com/library/courses/building-blocks-hadoop-hdfs-mapreduce-yarn/table-of-contents
名称节点日志中的错误:
2020-03-31 09:35:56,962 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: Exception in secureMain
java.io.IOException: Incorrect configuration: namenode address dfs.namenode.servicerpc-address or dfs.namenode.rpc-address is not configured.
at org.apache.hadoop.hdfs.DFSUtil.getNNServiceRpcAddressesForCluster(DFSUtil.java:576)
at org.apache.hadoop.hdfs.server.datanode.BlockPoolManager.refreshNamenodes(BlockPoolManager.java:152)
at org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:1392)
at org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:495)
at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:2695)
at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:2598)
at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:2645)
at org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:2789)
at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:2813)
2020-03-31 09:35:56,966 INFO org.apache.hadoop.util.ExitUtil: Exiting with status 1: java.io.IOException: Incorrect configuration: namenode address dfs.namenode.servicerpc-address or dfs.namenode.rpc-address is not configured.
二级名称节点日志:
2020-03-31 09:36:05,237 FATAL org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode: Failed to start secondary namenode
java.lang.IllegalArgumentException: Invalid URI for NameNode address (check fs.defaultFS): file:/// has no authority.
at org.apache.hadoop.hdfs.DFSUtilClient.getNNAddress(DFSUtilClient.java:626)
at org.apache.hadoop.hdfs.DFSUtilClient.getNNAddressCheckLogical(DFSUtilClient.java:655)
at org.apache.hadoop.hdfs.DFSUtilClient.getNNAddress(DFSUtilClient.java:617)
at org.apache.hadoop.hdfs.server.namenode.NameNode.getServiceAddress(NameNode.java:519)
at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.initialize(SecondaryNameNode.java:231)
at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.<init>(SecondaryNameNode.java:194)
at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.main(SecondaryNameNode.java:690)
2020-03-31 09:36:05,241 INFO org.apache.hadoop.util.ExitUtil: Exiting with status 1: ExitException
当我转到 localhost:50070
时,我什么也没得到(无法访问站点),当我点击 jps
时,我没有看到任何名称节点或辅助名称节点 运行。
我做了什么?
- 参考 Incorrect configuration: namenode address dfs.namenode.rpc-address is not configured 并确保任何配置中都没有空格(已在说明中列出)
是否有需要配置的东西或我缺少的东西?
提前致谢!
core-site.xml
属性 标签有错字!。必须是 property
而不是 propeerty
.
<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://localhost:9000</value>
</property>
</configuration>
此外,在 hdfs-site.xml
中您没有提到 dfs.namenode.name.dir
和 dfs.datanode.data.dir
属性。始终建议配置这些属性,否则将 hadoop.tmp.dir
属性(在 core-site.xml
中)配置到 /tmp
以外的某个目录。
更新这些属性,格式化名称节点并启动服务。