Azure 数据摄取到数据资源管理器

Question

我有一个 APIM 策略，它像这样记录到事件中心：-

<log-to-eventhub logger-id="eventhublogger" partition-id="0">
@{

    var body = context.Request.Body?.As<string>(true);
    if (body != null && body.Length > 1024)
    {
        body = body.Substring(0, 1024);
    }

    var json = new JObject( 
        new JProperty("DateTime", DateTime.Now),
        new JProperty("Method", context.Request.Method),
        new JProperty("Path", context.Request.Url.Path + context.Request.Url.QueryString),
        new JProperty("RequestBody", body) 
    ); 
    return json.ToString();


}
</log-to-eventhub>

我想尝试将此数据从事件中心提取到 DataExplorer

我不确定应该如何创建我的 Table 映射

我设置了一些 blob 存储并将事件中心数据转发给它，这样我就可以看到它的样子。在 blob 中它看起来像这样：-

Objavro.codecnullavro.schema�{"type":"record","name":"EventData","namespace":"Microsoft.ServiceBus.Messaging","fields":[{"name":"SequenceNumber","type":"long"},{"name":"Offset","type": "string"},{"name":"EnqueuedTimeUtc","type":"string"},{"name":"SystemProperties","type":{"type":"map","values":["long","double","string","bytes"]}},{ "name":"Properties","type":{"type":"map","values":["long","double", "string","bytes","null"]}},{"name":"Body","type":["null","bytes"]}]}

�106408(8/23/2019 4:41:18 上午

{ "DateTime": "2019-08-23T04:40:53.9151977+00:00", "Method": "POST", "Path": "/api/FuncCreateLead", "RequestBody": "{\r\n \"Title\": \"Miss\",\r\n \"FirstName\": \"Alice\"} }

我是否需要使用以下字段创建 table 映射？

SequenceNumber、Offset、EnqueuedTimeUtc、SystemProperties、Properties、Body？

Answer 1

如果这是您的负载：

{
    "DateTime": "2019-08-23T04:40:53.9151977+00:00",
    "Method": "POST",
    "Path": "/api/FuncCreateLead",
    "RequestBody": {
        "Title": "Miss",
        "FirstName": "Alice"
    }
}

这是您用来创建 Kusto/ADX table:

的 table 创建命令

{ "DateTime": "2019-08-23T04:40:53.9151977+00:00", "Method": "POST", "Path": "/api/FuncCreateLead", "RequestBody"：{ "Title": "Miss", "FirstName": "Alice" } }

.create table TableName (
    DateTime: datetime,
    Method: string,
    Path: string,
    RequestBody: dynamic
)

那么这就是您要创建的映射

.create table TableName ingestion json mapping 'mapping_name' 
'['
'   {'
'       "column": "DateTime",'
'       "path": "$.DateTime",'
'       "datatype": "datetime"'
'   },'
'   {'
'       "column": "Method",'
'       "path": "$.Method",'
'       "datatype": "string"'
'   },'
'   {'
'       "column": "Path",'
'       "path": "$.Path",'
'       "datatype": "string"'
'   },'
'   {'
'       "column": "RequestBody",'
'       "path": "$.RequestBody",'
'       "datatype": "dynamic"'
'   }'
']'

相应地，您可以根据自己的实际需求，从table definition/mapping中选择add/remove列，以上仅为示例。

Azure 数据摄取到数据资源管理器

Azure Data Ingest to Data Explorer

azure-api-management

azure-data-explorer