HIVe 中的嵌套 JSON 错误

Question

我试图在配置单元中加载这个 json 数据

{
    "id": "0001",
    "type": "donut",
     "name": "Cake",
     "ppu": 0.55,
     "batters":
         {
             "batter":
                 [
                     { "id": "1001", "type": "Regular" },
                     { "id": "1002", "type": "Chocolate" },
                    { "id": "1003", "type": "Blueberry" },
                    { "id": "1004", "type": "Devil's Food" }
                ]
        },
    "topping":
        [
            { "id": "5001", "type": "None" },
            { "id": "5002", "type": "Glazed" },
            { "id": "5005", "type": "Sugar" },
            { "id": "5007", "type": "Powdered Sugar" },
            { "id": "5006", "type": "Chocolate with Sprinkles" },
            { "id": "5003", "type": "Chocolate" },
            { "id": "5004", "type": "Maple" }
        ]
}

使用 DDL 命令

ADD JAR /home/cloudera/Downloads/json-serde-1.3.6-SNAPSHOT-jar-with-dependencies.jar;

CREATE EXTERNAL TABLE format.json_serde (
  `id` string,
  `type` string,
  `name` string,
 `ppu` float,       
  batters` struct < `batter`:array < struct <`bid`:string, `btype`:string >>>,
  `topping`:array < struct<`tid`:int, `ttype`:string>>
)
ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe';

给我报错

FAILED: ParseException line 7:11 cannot recognize input near ':' 'array' '<' in column type </b>

Answer 1

你有错别字
ttype`:string 应该是 ttype:string
battersstruct 应该是 batters struct
topping:array 应该是 topping array
JSON SerDe 映射是按名称完成的。
您的结构字段名称应与实际名称相匹配，例如id 而不是 bid 或 tid，否则您将获得这些字段的 NULL 值。
已经有一个 JSON SerDe 是 Hive 安装的一部分。 https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-RowFormats&SerDe

create external table json_serde 
( 
    id      string
   ,type    string 
   ,name    string 
   ,ppu     float
   ,batters struct<batter:array<struct<id:string,type:string>>>
   ,topping array<struct<id:string,type:string>>
) 
row format serde 
'org.apache.hive.hcatalog.data.JsonSerDe' 
stored as textfile
;

select * from json_serde
;

+------+-------+------+-------------------+--------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|  id  | type  | name |        ppu        |                                                                     batters                                                                      |                                                                                                                  topping                                                                                                                  |
+------+-------+------+-------------------+--------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| 0001 | donut | Cake | 0.550000011920929 | {"batter":[{"id":"1001","type":"Regular"},{"id":"1002","type":"Chocolate"},{"id":"1003","type":"Blueberry"},{"id":"1004","type":"Devil'sFood"}]} | [{"id":"5001","type":"None"},{"id":"5002","type":"Glazed"},{"id":"5005","type":"Sugar"},{"id":"5007","type":"PowderedSugar"},{"id":"5006","type":"ChocolatewithSprinkles"},{"id":"5003","type":"Chocolate"},{"id":"5004","type":"Maple"}] |
+------+-------+------+-------------------+--------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

Answer 2

当我删除顶部附近的分号时，它起作用了。谢谢

创建外部 TABLE format.json_serde (
id 字符串，
type 字符串，
name 字符串，
ppu 浮动，

batters 结构<batter:数组< 结构<bid：字符串，btype：字符串>>>，

topping array< struct<tid:string, ttype:string>> )

HIVe 中的嵌套 JSON 错误

Error in Nested JSON in HIve

json

hadoop

hive

nested

hive-serde