尝试从 AVRO 文件创建 table 时 BQ 加载作业失败

Question

我正在尝试从 AVRO 文件创建 BQ Table。当我运行 BQ 加载作业时出现此错误：

"Error while reading data, error message: The Apache Avro library failed to parse the header with the following error: Unexpected type for default value. Expected long, but found null: null"

AVRO 文件的架构是：

{
  "type" : "record",
  "name" : "Pair",
  "namespace" : "org.apache.avro.mapred",
  "fields" : [ {
    "name" : "key",
    "type" : "int",
    "doc" : ""
  }, {
    "name" : "value",
    "type" : {
      "type" : "record",
      "name" : "CustomerInventoryOrderItems",
      "namespace" : "com.test.customer.order",
      "fields" : [ {
        "name" : "updated_at",
        "type" : "long"
      }, {
        "name" : "inventory_order_items",
        "type" : {
          "type" : "map",
          "values" : {
            "type" : "array",
            "items" : {
              "type" : "record",
              "name" : "CustomerInventoryOrderItem",
              "fields" : [ {
                "name" : "order_item_id",
                "type" : "int",
                "default" : null
              }, {
                "name" : "updated_at",
                "type" : "long"
              }, {
                "name" : "created_at",
                "type" : "long"
              }, {
                "name" : "product_id",
                "type" : [ "null", "int" ],
                "default" : null
              }, {
                "name" : "type_id",
                "type" : "int",
                "default" : null
              }, {
                "name" : "event_id",
                "type" : [ "null", "int" ],
                "default" : null
              }, {
                "name" : "price",
                "type" : [ "null", "double" ],
                "default" : null
              }, {
                "name" : "tags",
                "type" : [ "null", "string" ],
                "default" : null
              }, {
                "name" : "estimated_ship_date",
                "type" : [ "null", "long" ],
                "default" : null
              } ]
            }
          }
        }
      } ]
    },
    "doc" : "",
    "order" : "ignore"
  } ]
}

我不确定架构或其他任何问题有什么问题，因此我无法加载数据。

Answer 1

问题很可能是类型为 int 的字段，但您将 null 作为默认值。例如：

                "name" : "type_id",
                "type" : "int",
                "default" : null

默认值应更改为整数，或者类型应更改为包含 null 的联合（与许多其他字段一样）。

尝试从 AVRO 文件创建 table 时 BQ 加载作业失败

BQ load job failing when trying to create table from AVRO file

schema

avro

google-bigquery

bq-load