直接使用 FileSystem 在标准输出上显示 Hadoop 文件系统中的文件时出错

Question

我是 Hadoop 的新手，我试图直接使用文件系统在标准输出上显示 Hadoop 文件系统中的文件内容，但出现以下错误。

import java.io.InputStream;
import java.net.URI;

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.IOUtils;


public class FileSystemCat {

  public static void main(String[] args) throws Exception {
   Configuration conf = new Configuration();
      conf.addResource(new Path("/usr/local/hadoop/etc/hadoop/core-site.xml"));
      conf.addResource(new Path("/usr/local/hadoop/etc/hadoop/hdfs-site.xml"));
    String uri = "/books/pg5000.txt";
  FileSystem fs = FileSystem.get(URI.create(uri), conf);
  InputStream in = null;
  try {
   in = fs.open(new Path(uri));
   IOUtils.copyBytes(in, System.out, 4096, false);
   } finally {
   IOUtils.closeStream(in);
   }
  }
}

错误 -

Exception in thread "main" java.lang.NoClassDefFoundError: org/slf4j/LoggerFactory
    at org.apache.hadoop.security.authentication.util.KerberosName.<clinit>(KerberosName.java:43)
    at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:261)
    at org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:248)
    at org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(UserGroupInformation.java:763)
    at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:748)
    at 

    org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:621)
        at org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:2753)
        at org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:2745)
        at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2611)
        at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:370)
        at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:169)
        at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:354)
        at FileSystemCat.main(FileSystemCat.java:17)
    Caused by: java.lang.ClassNotFoundException: org.slf4j.LoggerFactory
        at java.net.URLClassLoader.run(URLClassLoader.java:366)
        at java.net.URLClassLoader.run(URLClassLoader.java:355)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
        ... 13 more

谁能指导我，我哪里错了？

Answer 1

java.lang.ClassNotFoundException: org.slf4j.LoggerFactory

如果您的类路径中没有 slf4j-api-x.x.x，则会出现此错误。首先尝试将此 jar 添加到您的类路径中。这个jar可以在$HADOOP_HOME/share/hadoop/common/lib中找到（我的hadoop版本是2.6.0）。

你也可以试试这个：

java -cp /path/to/hadoop/common/lib/slf4j-api-x.x.x.jar:. yourjavaclassname

直接使用 FileSystem 在标准输出上显示 Hadoop 文件系统中的文件时出错

Error displaying files in Hadoop filesystem on standard output using FileSystem directly

hadoop

hdfs