调试 MapReduce 代码以查找 NULLPointerException 错误的原因

Question

我正在尝试运行一个 MapReduce 代码来计算每个部门的平均满意度。以下是示例数据。该文件有 headers 并且为了删除 headers 我将它们放在另一个 txt 文件中并用它来检查行值是否不等于 headers.

示例数据 satisfaction_level,last_evaluation,number_project,average_montly_hours,time_spend_company,Work_accident,left,promotion_last_5years,sales,salary

0.38,0.53,2,157,3,0,1,0,销量,低 0.8,0.86,5,262,6,0,1,0,销售额,中等 0.11,0.88,7,272,4,0,1,0,销售额,中等 0.72,0.87,5,223,5,0,1,0,销售额,低 0.37,0.52,2,159,3,0,1,0,销量,低 0.41,0.5,2,153,3,0,1,0,销量,低 0.1,0.77,6,247,4,0,1,0,销量,低 0.92,0.85,5,259,5,0,1,0,销量,低 0.89,1,5,224,5,0,1,0,销售额,低

下面是我的映射器代码。

public class SatisfactionLevelMapper extends Mapper<LongWritable, Text, Text, FloatWritable>
{
     String header;
 @Override
 protected void setup(Mapper.Context context) throws IOException, InterruptedException 
/* Headers are placed in a separate header.txt file for which the location is specified in the driver */
{
    BufferedReader bufferedReader = new BufferedReader(new FileReader("header.txt"));
    header = bufferedReader.readLine();

}
    public void map(LongWritable key, Text text, Context context) throws IOException, InterruptedException
{
    String row = text.toString();
    String [] values = row.trim().split(","); //dataset is a CSV file with 10 columns

    float satisfaction = 0.0f;
    String dept = null;
    try
    {
        if(values.length == 9 && header != row)
        {
            satisfaction = Float.parseFloat(values[0]);         }
    }
    catch (Exception e)
    {
        e.printStackTrace();
    }
    context.write(new Text(dept), new FloatWritable(satisfaction));
    }
}

我收到如下 NullPointerException 错误。

Error: java.lang.NullPointerException
    at org.apache.hadoop.io.Text.encode(Text.java:450)
    at org.apache.hadoop.io.Text.set(Text.java:198)
    at org.apache.hadoop.io.Text.<init>(Text.java:88)
    at com.df.hra.SatisfactionLevelMapper.map SatisfactionLevelMapper.java:62)
    at com.df.hra.SatisfactionLevelMapper.map(SatisfactionLevelMapper.java:1)
    at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
    at org.apache.hadoop.mapred.YarnChild.run(YarnChild.java:168)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1642)
    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)

有没有办法找出我的代码的哪一部分有问题？我是 Java 的新手，正在尝试找出一种调试 MapReduce 代码的方法。任何帮助，将不胜感激。谢谢！

Answer 1

您不能像在此处（使用 dept）那样使用 null 初始化 Text 值。考虑改用空字符串 ("")。

调试 MapReduce 代码以查找 NULLPointerException 错误的原因

Debug MapReduce code to find the cause of NULLPointerException error

java

mapreduce