U- SQL 无法从 JSON 文件中提取数据
U- SQL Unable to extract data from JSON file
我试图使用 USQL 从 JSON 文件中提取数据。查询成功运行但未生成任何输出数据或结果为 "vertex failed fast error".
JSON 文件如下所示:
{
"results": [
{
"name": "Sales/Account",
"id": "7367e3f2-e1a5-11e5-80e8-0933ecd4cd8c",
"deviceName": "HP",
"deviceModel": "g6-pavilion",
"clientip": "0.41.4.1"
},
{
"name": "Sales/Account",
"id": "c01efba0-e0d5-11e5-ae20-af6dc1f2c036",
"deviceName": "acer",
"deviceModel": "veriton",
"clientip": "10.10.14.36"
}
]
}
而我的 U-SQL 脚本是
REFERENCE ASSEMBLY [Newtonsoft.Json];
REFERENCE ASSEMBLY [Microsoft.Analytics.Samples.Formats];
DECLARE @in string="adl://xyz.azuredatalakestore.net/todelete.json";
DECLARE @out string="adl://xyz.azuredatalakestore.net/todelete.tsv";
@trail2=EXTRACT results string FROM @in USING new Microsoft.Analytics.Samples.Formats.Json.JsonExtractor();
@jsonify=SELECT Microsoft.Analytics.Samples.Formats.Json.JsonFunctions.JsonTuple(results,"name","id","deviceName","deviceModel","clientip") AS rec FROM @trail2;
@logSchema=SELECT rec["name"] AS sysName,
rec["id"] AS sysId,
rec["deviceName"] AS domainDeviceName,
rec["deviceModel"] AS domainDeviceModel,
rec["clientip"] AS domainClientIp
FROM @jsonify;
OUTPUT @logSchema TO @out USING Outputters.Tsv();
萨拉斯,
问题是你的@trail2 输出是一个 json 数组“[{...},{...}]”,据我所知,JsonFunction 无法解析它.所以我把它输出到一个文件并用输入器重新读取它,它可以解析数组。
REFERENCE ASSEMBLY [Newtonsoft.Json];
REFERENCE ASSEMBLY [Microsoft.Analytics.Samples.Formats];
DECLARE @in string="adl://xyz.azuredatalakestore.net/todelete.json";
DECLARE @out string="adl://xyz.azuredatalakestore.net/todelete.tsv";
DECLARE @mid string="adl://xyz.azuredatalakestore.net/intermediate.txt";
@trail2=EXTRACT results string FROM @in USING new Microsoft.Analytics.Samples.Formats.Json.JsonExtractor();
OUTPUT @trail2 TO @mid USING Outputters.Text(quoting:false);
@jsonify=EXTRACT name string,
id string,
deviceName string ,
deviceModel string,
clientip string
FROM @mid USING new Microsoft.Analytics.Samples.Formats.Json.JsonExtractor();
@logSchema=SELECT name AS sysName,
id AS sysId,
deviceName AS domainDeviceName,
deviceModel AS domainDeviceModel,
clientip AS domainClientIp
FROM @jsonify;
OUTPUT @logSchema TO @out USING Outputters.Tsv();
实际上 JSONExtractor
支持 JSONPath 中表示的行路径参数,使您能够识别要映射的 JSON 对象或 JSON 数组项成行。因此,您可以使用一条语句从 JSON 文档中提取数据:
@logSchema =
EXTRACT name string, id string, deviceName string, deviceModel string, clientip string
FROM @input
USING new Microsoft.Analytics.Samples.Formats.Json.JsonExtractor("results[*]");
我试图使用 USQL 从 JSON 文件中提取数据。查询成功运行但未生成任何输出数据或结果为 "vertex failed fast error".
JSON 文件如下所示:
{
"results": [
{
"name": "Sales/Account",
"id": "7367e3f2-e1a5-11e5-80e8-0933ecd4cd8c",
"deviceName": "HP",
"deviceModel": "g6-pavilion",
"clientip": "0.41.4.1"
},
{
"name": "Sales/Account",
"id": "c01efba0-e0d5-11e5-ae20-af6dc1f2c036",
"deviceName": "acer",
"deviceModel": "veriton",
"clientip": "10.10.14.36"
}
]
}
而我的 U-SQL 脚本是
REFERENCE ASSEMBLY [Newtonsoft.Json];
REFERENCE ASSEMBLY [Microsoft.Analytics.Samples.Formats];
DECLARE @in string="adl://xyz.azuredatalakestore.net/todelete.json";
DECLARE @out string="adl://xyz.azuredatalakestore.net/todelete.tsv";
@trail2=EXTRACT results string FROM @in USING new Microsoft.Analytics.Samples.Formats.Json.JsonExtractor();
@jsonify=SELECT Microsoft.Analytics.Samples.Formats.Json.JsonFunctions.JsonTuple(results,"name","id","deviceName","deviceModel","clientip") AS rec FROM @trail2;
@logSchema=SELECT rec["name"] AS sysName,
rec["id"] AS sysId,
rec["deviceName"] AS domainDeviceName,
rec["deviceModel"] AS domainDeviceModel,
rec["clientip"] AS domainClientIp
FROM @jsonify;
OUTPUT @logSchema TO @out USING Outputters.Tsv();
萨拉斯,
问题是你的@trail2 输出是一个 json 数组“[{...},{...}]”,据我所知,JsonFunction 无法解析它.所以我把它输出到一个文件并用输入器重新读取它,它可以解析数组。
REFERENCE ASSEMBLY [Newtonsoft.Json];
REFERENCE ASSEMBLY [Microsoft.Analytics.Samples.Formats];
DECLARE @in string="adl://xyz.azuredatalakestore.net/todelete.json";
DECLARE @out string="adl://xyz.azuredatalakestore.net/todelete.tsv";
DECLARE @mid string="adl://xyz.azuredatalakestore.net/intermediate.txt";
@trail2=EXTRACT results string FROM @in USING new Microsoft.Analytics.Samples.Formats.Json.JsonExtractor();
OUTPUT @trail2 TO @mid USING Outputters.Text(quoting:false);
@jsonify=EXTRACT name string,
id string,
deviceName string ,
deviceModel string,
clientip string
FROM @mid USING new Microsoft.Analytics.Samples.Formats.Json.JsonExtractor();
@logSchema=SELECT name AS sysName,
id AS sysId,
deviceName AS domainDeviceName,
deviceModel AS domainDeviceModel,
clientip AS domainClientIp
FROM @jsonify;
OUTPUT @logSchema TO @out USING Outputters.Tsv();
实际上 JSONExtractor
支持 JSONPath 中表示的行路径参数,使您能够识别要映射的 JSON 对象或 JSON 数组项成行。因此,您可以使用一条语句从 JSON 文档中提取数据:
@logSchema =
EXTRACT name string, id string, deviceName string, deviceModel string, clientip string
FROM @input
USING new Microsoft.Analytics.Samples.Formats.Json.JsonExtractor("results[*]");