报告任务管理器中的processedWithError是什么意思？

Question

我已经将文件摄取到druid中，很高兴显示摄取成功。但是，当我检查摄取报告时，所有行都经过错误处理，但数据源显示在 "Datasource" 选项卡中。

我尝试将行数从 20M 减少到 20 行。这是我的配置文件：

"type" : "index",
  "spec" : {
    "ioConfig" : {
      "type" : "index",
      "firehose" : {
        "type" : "local",
        "baseDir" : "/home/data/Salutica",
      "filter" : "outDashboard2RawV3.csv"
      }
    },
    "dataSchema" : {
        "dataSource": "DaTRUE2_Dashboard_V3",
      "granularitySpec" : {
        "type" : "uniform",
        "segmentGranularity" : "WEEK",
        "queryGranularity" : "none",
        "intervals" : ["2017-05-08/2019-05-17"],
        "rollup" : false
      },
      "parser" : {
        "type" : "string",
        "parseSpec": {
          "format" : "csv",
           "timestampSpec" : {
            "column" : "Date_Time",
            "format" : "auto"
         },
        "columns" : [
          "Main_ID","Parameter_ID","Date_Time","Serial_Number","Status","Station_ID",
          "Station_Type","Parameter_Name","Failed_Date_Time","Failed_Measurement",
          "Database_Name","Date_Time_Year","Date_Time_Month",
          "Date_Time_Day","Date_Time_Hour","Date_Time_Weekday","Status_New"
        ],
        "dimensionsSpec" : {
          "dimensions" : [
              "Date_Time","Serial_Number","Status","Station_ID",
              "Station_Type","Parameter_Name","Failed_Date_Time",
              "Failed_Measurement","Database_Name","Status_New",
              {
                  "name" : "Main_ID",
                  "type" : "long"
              },
              {
                  "name" : "Parameter_ID",
                  "type" : "long"
              },
              {
                  "name" : "Date_Time_Year",
                  "type" : "long"
              },
              {
                  "name" : "Date_Time_Month",
                  "type" : "long"
              },
              {
                  "name" : "Date_Time_Day",
                  "type" : "long"
              },
              {
                  "name" : "Date_Time_Hour",
                  "type" : "long"
              },
              {
                  "name" : "Date_Time_Weekday",
                  "type" : "long"
              }
                  ]
           }
        }
      },
      "metricsSpec" : [
        {
          "name" : "count",
          "type" : "count"
        }
      ]
    },
    "tuningConfig" : {
      "type" : "index",
      "partitionsSpec" : {
        "type" : "hashed",
        "targetPartitionSize" : 5000000
      },
      "jobProperties" : {}
    }
  }
}

举报：

{"ingestionStatsAndErrors":{"taskId":"index_DaTRUE2_Dashboard_V3_2019-09-10T01:16:47.113Z","payload":{"ingestionState":"COMPLETED","unparseableEvents":{},"rowStats":{"determinePartitions":{"processed":0,"processedWithError":0,"thrownAway":0,"unparseable":0},"buildSegments":{"processed":0,"processedWithError":20606701,"thrownAway":0,"unparseable":1}},"errorMsg":null},"type":"ingestionStatsAndErrors"}}

我期待这个：

{"processed":20606701,"processedWithError":0,"thrownAway":0,"unparseable":1}},"errorMsg":null},"type":"ingestionStatsAndErrors"}}

而不是这个：

{"processed":0,"processedWithError":20606701,"thrownAway":0,"unparseable":1}},"errorMsg":null},"type":"ingestionStatsAndErrors"}}

Answer 1

下面是我从 csv 中输入的数据；

    "Main_ID","Parameter_ID","Date_Time","Serial_Number","Status","Station_ID","Station_Type","Parameter_Name","Failed_Date_Time","Failed_Measurement","Database_Name","Date_Time_Year","Date_Time_Month","Date_Time_Day","Date_Time_Hour","Date_Time_Weekday","Status_New"
1,3,"2018-10-05 15:00:55","1840SDF00038","Passed","ST1","BLTBoard","1.8V","","","DaTRUE2Left",2018,10,5,15,"Friday","Passed"
1,4,"2018-10-05 15:00:55","1840SDF00038","Passed","ST1","BLTBoard","1.35V","","","DaTRUE2Left",2018,10,5,15,"Friday","Passed"
1,5,"2018-10-05 15:00:55","1840SDF00038","Passed","ST1","BLTBoard","Isc_VChrg","","","DaTRUE2Left",2018,10,5,15,"Friday","Passed"
1,6,"2018-10-05 15:00:55","1840SDF00038","Passed","ST1","BLTBoard","Isc_VBAT","","","DaTRUE2Left",2018,10,5,15,"Friday","Passed"

报告任务管理器中的processedWithError是什么意思？

What is meant by processedWithError in the report task manager?

java

druid