CosmosDb 与 DocumentDB API 通过 Azure API 管理
CosmosDb with DocumentDB API through Azure API Management
我正在尝试通过 API 管理查询我的新 CosmosDB collection。一旦证明，这将是一个 front-end 供用户访问记录的数据。因此，我按订阅 ID 对数据进行了分区。在我的 WebApi 数据库的日志 collection 的 Azure 门户中，我看到分区键为 /api_subscription_key。我有来自 API Mgt 的数据。 -> 事件中心 -> 流分析 -> Cosmos。
使用 Azure 门户中的查询资源管理器，我可以尝试如下查询：
SELECT * FROM c WHERE c.api_subscription_key = '573a1c65bceb52192c140131'
这带回了我已经成功写入 CosmosDB 很多天的预期文档
[
{
"eventenqueuedutctimesecond": "2017-07-27T15:09:02Z",
"business_unit_key": null,
"user_key": null,
"api_message_id": "1718ea66-d225-45ec-b3fc-5daff4c7f426",
"api_identifier": "21926e9d-9206-42b0-b4b1-7e7f1eb4e7dd",
"api_id": "58d94cc622be39392343d4b6",
"api_operation_id": "58e682bde055cd0ba4215d4b",
"api_adapter_id": "573a1c64bceb520aac127ee5",
"api_subscription_id": "573a1c65bceb52192c140131",
"api_policy_id": "64BC4270-54AC-42DA-835C-E285F35BCA81",
"basic_username": "",
"message_version": "10",
"claim_business_unit_key": null,
"claim_user_key": null,
...
"lasterrorsource": null,
"lasterrorreason": null,
"lasterrorscope": null,
"lasterrorsection": null,
"lasterrorpolicyid": null,
"id": "7/27/2017 3:09:02 PM",
"_rid": "9Fc0ANW4fwAoAAAAAAAADA==",
"_self": "dbs/9Fc0AA==/colls/9Fc0ANW4fwA=/docs/9Fc0ANW4fwAoAAAAAAAADA==/",
"_etag": "\"0700d90c-0000-0000-0000-597a020e0000\"",
"_attachments": "attachments/",
"_ts": 1501168140
}...
我的 CosmosDB 实例是 plexconnectcosmos。通过 API 管理及其政策，我发布到
https://plexconnectcosmos.documents.azure.com/dbs/WebApi/colls/Logs/docs
这些 headers（许多残留，希望没有效果）：
[
{
name: "Postman-Token",
value: "756c2c21-ef23-4e5a-a63a-ae6aed961d35"
},
{
name: "Ocp-Apim-Subscription-Key",
value: "a2a05eff128943bc89f62b81a63aa368"
},
{
name: "Accept-Charset",
value: "UTF-8"
},
{
name: "Cache-Control",
value: "no-cache"
},
{
name: "Content-Type",
value: "application/query+json"
},
{
name: "Accept",
value: "application/json;odata=nometadata"
},
{
name: "Accept-Encoding",
value: "gzip,deflate"
},
{
name: "Cookie",
value: "x-ms-gateway-slice=008; stsservicecookie=ests; BIGipServerpmc_rest_webservices_http_prod=1242575370.20480.0000"
},
{
name: "User-Agent",
value: "PostmanRuntime/6.2.5"
},
{
name: "x-ms-date",
value: "Wed, 09 Aug 2017 20:10:09 GMT"
},
{
name: "x-ms-version",
value: "2017-02-22"
},
{
name: "MaxDataServiceVersion",
value: "3.0"
},
{
name: "DataServiceVersion",
value: "1.0;NetFx"
},
{
name: "Api-Message-Id",
value: "12427ae7-7704-44cb-b4af-d7e622898b99"
},
{
name: "Api-Identifier",
value: "461f0c19-8df3-4272-9ac7-c64bb776dd56"
},
{
name: "Api-Id",
value: "58987927bceb5204c4e59168"
},
{
name: "Api-Operation-Id",
value: "598b3c72e055cd14fc3abdd1"
},
{
name: "Api-Adapter-Id",
value: "573a1c64bceb520aac127ee5"
},
{
name: "Api-Subscription-Id",
value: "573a1c65bceb52192c140131"
},
{
name: "Api-Policy-Id",
value: "64BC4270-54AC-42DA-835C-E285F35BCA81"
},
{
name: "X-Basic-Username",
value: ""
},
{
name: "x-ms-documentdb-isquery",
value: "True"
},
{
name: "x-ms-documentdb-query-enablecrosspartition",
value: "False"
},
{
name: "x-ms-max-item-count",
value: "1000"
},
{
name: "x-ms-documentdb-partitionkey",
value: "573a1c65bceb52192c140131"
},
{
name: "x-ms-partition-key",
value: "573a1c65bceb52192c140131"
},
{
name: "Authorization",
value: "type=master&ver=1.0&sig=Ke...Q="
},
{
name: "X-Forwarded-For",
value: "75.39.38.67"
}
]
我得到的回复是
{
"code": "BadRequest",
"message": "Partition key 573a1c65bceb52192c140131 is invalid.\r\nActivityId: 61836599-fe4b-4232-b55b-2c568eecc767"
}
或
{
"code": "Unauthorized",
"message": "The input authorization token can't serve the request. Please check that the expected payload is built as per the protocol, and check the key being used. Server used the following payload to sign: 'post\ndocs\ndbs/WebApi/colls/Logs\nwed, 09 aug 2017 20:35:41 gmt\n\n'\r\nActivityId: 429....2e2"
}
这些似乎给我解决了两个问题。首先，我该如何解决这个分区的故障？从我的分析来看，它似乎是一个有效的分区，通过门户中的查询和 headers "x-ms-documentdb-partitionkey" 和 "x-ms-partition-key" 进行验证。 （我在 MS 文档中看到了两个 header 名字，所以我用两个来覆盖我的基础。）
"The input authorization token can't serve the request." 消息提示我查询中有一些错误。我怀疑可能是数据值？我的政策与我用于 Azure Table Storage REST API 的政策略有不同，我从来没有遇到过这个问题。我正在使用从 Azure 门户获取并存储在 API 管理的命名值中的 read-only 主键：
<policies>
<inbound>
<base />
<set-variable name="Content-Type" value="application/query+json" />
<set-variable name="x-ms-documentdb-isquery" value="True" />
<set-variable name="x-ms-documentdb-query-enablecrosspartition" value="False" />
<set-variable name="x-ms-max-item-count" value="1000" />
<set-variable name="x-ms-version" value="2017-02-22" />
<set-header name="Content-Type" exists-action="override">
<value>@((string)context.Variables["Content-Type"])</value>
</set-header>
<set-header name="x-ms-documentdb-isquery" exists-action="override">
<value>@((string)context.Variables["x-ms-documentdb-isquery"])</value>
</set-header>
<set-header name="x-ms-documentdb-query-enablecrosspartition" exists-action="override">
<value>@((string)context.Variables["x-ms-documentdb-query-enablecrosspartition"])</value>
</set-header>
<set-header name="x-ms-max-item-count" exists-action="override">
<value>@((string)context.Variables["x-ms-max-item-count"])</value>
</set-header>
<set-header name="x-ms-version" exists-action="override">
<value>@((string)context.Variables["x-ms-version"])</value>
</set-header>
<!-- MS docs may conflict here. Possibly "x-ms-documentdb-partitionkey" req'd and "x-ms-partition-key" not supported -->
<set-header name="x-ms-documentdb-partitionkey" exists-action="override">
<value>@(context.Subscription.Id)</value>
</set-header>
<set-header name="x-ms-partition-key" exists-action="override">
<value>@(context.Subscription.Id)</value>
</set-header>
<set-variable name="StringToSign" value="@(string.Format("post\ndocs\ndbs/WebApi/colls/Logs\n{0}\n\n", ((string)context.Variables["x-ms-date"]).ToLowerInvariant()))" />
<set-variable name="cosmosreadonlykey" value="{{CosmosReadOnlyKey}}" />
<set-variable name="SharedKey" value="@{
// https://docs.microsoft.com/en-us/rest/api/documentdb/access-control-on-documentdb-resources#constructkeytoken
System.Security.Cryptography.HMACSHA256 hasher = new System.Security.Cryptography.HMACSHA256(Convert.FromBase64String((string)context.Variables["cosmosreadonlykey"]));
return Convert.ToBase64String(hasher.ComputeHash(System.Text.Encoding.UTF8.GetBytes((string)context.Variables["StringToSign"])));
}" />
<set-variable name="Authorization" value="@(string.Format("type=master&ver=1.0&sig={0}", (string)context.Variables["SharedKey"]))" />
<set-header name="Authorization" exists-action="override">
<value>@((string)context.Variables["Authorization"])</value>
</set-header>
<set-backend-service base-url="https://plexconnectcosmos.documents.azure.com" />
<rewrite-uri template="/dbs/WebApi/colls/Logs/docs" />
</inbound>
我想知道的一些事情：返回的 ActivityId 能否帮助我获取更多详细信息，如何获取更多详细信息？即使没有它，我还没有发现 Azure 中是否有一些日志可以揭示更多细节。
如果我在这里做的有什么明显的错误，请告诉我。
我明白了 运行 一些小的调整。
<policies>
<inbound>
<base />
<set-variable name="Content-Type" value="application/query+json" />
<set-variable name="x-ms-documentdb-isquery" value="True" />
<set-variable name="x-ms-documentdb-query-enablecrosspartition" value="False" />
<set-variable name="x-ms-max-item-count" value="1000" />
<set-variable name="x-ms-version" value="2017-02-22" />
<set-variable name="x-ms-date" value="@( DateTime.UtcNow.ToString("R") )" />
<set-header name="Content-Type" exists-action="override">
<value>@((string)context.Variables["Content-Type"])</value>
</set-header>
<set-header name="x-ms-documentdb-isquery" exists-action="override">
<value>@((string)context.Variables["x-ms-documentdb-isquery"])</value>
</set-header>
<set-header name="x-ms-documentdb-query-enablecrosspartition" exists-action="override">
<value>@((string)context.Variables["x-ms-documentdb-query-enablecrosspartition"])</value>
</set-header>
<set-header name="x-ms-max-item-count" exists-action="override">
<value>@((string)context.Variables["x-ms-max-item-count"])</value>
</set-header>
<set-header name="x-ms-version" exists-action="override">
<value>@((string)context.Variables["x-ms-version"])</value>
</set-header>
<set-header name="x-ms-documentdb-partitionkey" exists-action="override">
<value>@("[\""+context.Subscription.Id+"\"]")</value>
</set-header>
<set-header name="x-ms-date" exists-action="override">
<value>@( (string)context.Variables["x-ms-date"] )</value>
</set-header>
<set-variable name="StringToSign" value="@(string.Format("post\ndocs\ndbs/WebApi/colls/Logs\n{0}\n\n", ((string)context.Variables["x-ms-date"]).ToLowerInvariant()))" />
<set-variable name="cosmosreadonlykey" value="{{CosmosReadOnlyKey}}" />
<set-variable name="SharedKey" value="@{
// https://docs.microsoft.com/en-us/rest/api/documentdb/access-control-on-documentdb-resources#constructkeytoken
System.Security.Cryptography.HMACSHA256 hasher = new System.Security.Cryptography.HMACSHA256(Convert.FromBase64String((string)context.Variables["cosmosreadonlykey"]));
return Convert.ToBase64String(hasher.ComputeHash(System.Text.Encoding.UTF8.GetBytes((string)context.Variables["StringToSign"])));
}" />
<set-variable name="Authorization" value="@(string.Format("type=master&ver=1.0&sig={0}", ((string)context.Variables["SharedKey"]).Replace("&","%26").Replace("+","%2B").Replace("=","%3D")))" />
<set-header name="Authorization" exists-action="override">
<value>@((string)context.Variables["Authorization"])</value>
</set-header>
<set-backend-service base-url="https://mycosmosdb.documents.azure.com" />
<rewrite-uri template="/dbs/WebApi/colls/Logs/docs" />
</inbound>
</policies>
- the partition key needs to be formatted as an array
- 日期被放入header和基于相同值的StringToSign
- did some hacky URL hex encoding - 可以通过适当的十六进制编码进行改进
我正在尝试通过 API 管理查询我的新 CosmosDB collection。一旦证明，这将是一个 front-end 供用户访问记录的数据。因此，我按订阅 ID 对数据进行了分区。在我的 WebApi 数据库的日志 collection 的 Azure 门户中，我看到分区键为 /api_subscription_key。我有来自 API Mgt 的数据。 -> 事件中心 -> 流分析 -> Cosmos。
使用 Azure 门户中的查询资源管理器，我可以尝试如下查询：
SELECT * FROM c WHERE c.api_subscription_key = '573a1c65bceb52192c140131'
这带回了我已经成功写入 CosmosDB 很多天的预期文档
[
{
"eventenqueuedutctimesecond": "2017-07-27T15:09:02Z",
"business_unit_key": null,
"user_key": null,
"api_message_id": "1718ea66-d225-45ec-b3fc-5daff4c7f426",
"api_identifier": "21926e9d-9206-42b0-b4b1-7e7f1eb4e7dd",
"api_id": "58d94cc622be39392343d4b6",
"api_operation_id": "58e682bde055cd0ba4215d4b",
"api_adapter_id": "573a1c64bceb520aac127ee5",
"api_subscription_id": "573a1c65bceb52192c140131",
"api_policy_id": "64BC4270-54AC-42DA-835C-E285F35BCA81",
"basic_username": "",
"message_version": "10",
"claim_business_unit_key": null,
"claim_user_key": null,
...
"lasterrorsource": null,
"lasterrorreason": null,
"lasterrorscope": null,
"lasterrorsection": null,
"lasterrorpolicyid": null,
"id": "7/27/2017 3:09:02 PM",
"_rid": "9Fc0ANW4fwAoAAAAAAAADA==",
"_self": "dbs/9Fc0AA==/colls/9Fc0ANW4fwA=/docs/9Fc0ANW4fwAoAAAAAAAADA==/",
"_etag": "\"0700d90c-0000-0000-0000-597a020e0000\"",
"_attachments": "attachments/",
"_ts": 1501168140
}...
我的 CosmosDB 实例是 plexconnectcosmos。通过 API 管理及其政策，我发布到
https://plexconnectcosmos.documents.azure.com/dbs/WebApi/colls/Logs/docs
这些 headers（许多残留，希望没有效果）：
[
{
name: "Postman-Token",
value: "756c2c21-ef23-4e5a-a63a-ae6aed961d35"
},
{
name: "Ocp-Apim-Subscription-Key",
value: "a2a05eff128943bc89f62b81a63aa368"
},
{
name: "Accept-Charset",
value: "UTF-8"
},
{
name: "Cache-Control",
value: "no-cache"
},
{
name: "Content-Type",
value: "application/query+json"
},
{
name: "Accept",
value: "application/json;odata=nometadata"
},
{
name: "Accept-Encoding",
value: "gzip,deflate"
},
{
name: "Cookie",
value: "x-ms-gateway-slice=008; stsservicecookie=ests; BIGipServerpmc_rest_webservices_http_prod=1242575370.20480.0000"
},
{
name: "User-Agent",
value: "PostmanRuntime/6.2.5"
},
{
name: "x-ms-date",
value: "Wed, 09 Aug 2017 20:10:09 GMT"
},
{
name: "x-ms-version",
value: "2017-02-22"
},
{
name: "MaxDataServiceVersion",
value: "3.0"
},
{
name: "DataServiceVersion",
value: "1.0;NetFx"
},
{
name: "Api-Message-Id",
value: "12427ae7-7704-44cb-b4af-d7e622898b99"
},
{
name: "Api-Identifier",
value: "461f0c19-8df3-4272-9ac7-c64bb776dd56"
},
{
name: "Api-Id",
value: "58987927bceb5204c4e59168"
},
{
name: "Api-Operation-Id",
value: "598b3c72e055cd14fc3abdd1"
},
{
name: "Api-Adapter-Id",
value: "573a1c64bceb520aac127ee5"
},
{
name: "Api-Subscription-Id",
value: "573a1c65bceb52192c140131"
},
{
name: "Api-Policy-Id",
value: "64BC4270-54AC-42DA-835C-E285F35BCA81"
},
{
name: "X-Basic-Username",
value: ""
},
{
name: "x-ms-documentdb-isquery",
value: "True"
},
{
name: "x-ms-documentdb-query-enablecrosspartition",
value: "False"
},
{
name: "x-ms-max-item-count",
value: "1000"
},
{
name: "x-ms-documentdb-partitionkey",
value: "573a1c65bceb52192c140131"
},
{
name: "x-ms-partition-key",
value: "573a1c65bceb52192c140131"
},
{
name: "Authorization",
value: "type=master&ver=1.0&sig=Ke...Q="
},
{
name: "X-Forwarded-For",
value: "75.39.38.67"
}
]
我得到的回复是
{
"code": "BadRequest",
"message": "Partition key 573a1c65bceb52192c140131 is invalid.\r\nActivityId: 61836599-fe4b-4232-b55b-2c568eecc767"
}
或
{
"code": "Unauthorized",
"message": "The input authorization token can't serve the request. Please check that the expected payload is built as per the protocol, and check the key being used. Server used the following payload to sign: 'post\ndocs\ndbs/WebApi/colls/Logs\nwed, 09 aug 2017 20:35:41 gmt\n\n'\r\nActivityId: 429....2e2"
}
这些似乎给我解决了两个问题。首先，我该如何解决这个分区的故障？从我的分析来看，它似乎是一个有效的分区，通过门户中的查询和 headers "x-ms-documentdb-partitionkey" 和 "x-ms-partition-key" 进行验证。 （我在 MS 文档中看到了两个 header 名字，所以我用两个来覆盖我的基础。）
"The input authorization token can't serve the request." 消息提示我查询中有一些错误。我怀疑可能是数据值？我的政策与我用于 Azure Table Storage REST API 的政策略有不同，我从来没有遇到过这个问题。我正在使用从 Azure 门户获取并存储在 API 管理的命名值中的 read-only 主键：
<policies>
<inbound>
<base />
<set-variable name="Content-Type" value="application/query+json" />
<set-variable name="x-ms-documentdb-isquery" value="True" />
<set-variable name="x-ms-documentdb-query-enablecrosspartition" value="False" />
<set-variable name="x-ms-max-item-count" value="1000" />
<set-variable name="x-ms-version" value="2017-02-22" />
<set-header name="Content-Type" exists-action="override">
<value>@((string)context.Variables["Content-Type"])</value>
</set-header>
<set-header name="x-ms-documentdb-isquery" exists-action="override">
<value>@((string)context.Variables["x-ms-documentdb-isquery"])</value>
</set-header>
<set-header name="x-ms-documentdb-query-enablecrosspartition" exists-action="override">
<value>@((string)context.Variables["x-ms-documentdb-query-enablecrosspartition"])</value>
</set-header>
<set-header name="x-ms-max-item-count" exists-action="override">
<value>@((string)context.Variables["x-ms-max-item-count"])</value>
</set-header>
<set-header name="x-ms-version" exists-action="override">
<value>@((string)context.Variables["x-ms-version"])</value>
</set-header>
<!-- MS docs may conflict here. Possibly "x-ms-documentdb-partitionkey" req'd and "x-ms-partition-key" not supported -->
<set-header name="x-ms-documentdb-partitionkey" exists-action="override">
<value>@(context.Subscription.Id)</value>
</set-header>
<set-header name="x-ms-partition-key" exists-action="override">
<value>@(context.Subscription.Id)</value>
</set-header>
<set-variable name="StringToSign" value="@(string.Format("post\ndocs\ndbs/WebApi/colls/Logs\n{0}\n\n", ((string)context.Variables["x-ms-date"]).ToLowerInvariant()))" />
<set-variable name="cosmosreadonlykey" value="{{CosmosReadOnlyKey}}" />
<set-variable name="SharedKey" value="@{
// https://docs.microsoft.com/en-us/rest/api/documentdb/access-control-on-documentdb-resources#constructkeytoken
System.Security.Cryptography.HMACSHA256 hasher = new System.Security.Cryptography.HMACSHA256(Convert.FromBase64String((string)context.Variables["cosmosreadonlykey"]));
return Convert.ToBase64String(hasher.ComputeHash(System.Text.Encoding.UTF8.GetBytes((string)context.Variables["StringToSign"])));
}" />
<set-variable name="Authorization" value="@(string.Format("type=master&ver=1.0&sig={0}", (string)context.Variables["SharedKey"]))" />
<set-header name="Authorization" exists-action="override">
<value>@((string)context.Variables["Authorization"])</value>
</set-header>
<set-backend-service base-url="https://plexconnectcosmos.documents.azure.com" />
<rewrite-uri template="/dbs/WebApi/colls/Logs/docs" />
</inbound>
我想知道的一些事情：返回的 ActivityId 能否帮助我获取更多详细信息，如何获取更多详细信息？即使没有它，我还没有发现 Azure 中是否有一些日志可以揭示更多细节。
如果我在这里做的有什么明显的错误，请告诉我。
我明白了 运行 一些小的调整。
<policies>
<inbound>
<base />
<set-variable name="Content-Type" value="application/query+json" />
<set-variable name="x-ms-documentdb-isquery" value="True" />
<set-variable name="x-ms-documentdb-query-enablecrosspartition" value="False" />
<set-variable name="x-ms-max-item-count" value="1000" />
<set-variable name="x-ms-version" value="2017-02-22" />
<set-variable name="x-ms-date" value="@( DateTime.UtcNow.ToString("R") )" />
<set-header name="Content-Type" exists-action="override">
<value>@((string)context.Variables["Content-Type"])</value>
</set-header>
<set-header name="x-ms-documentdb-isquery" exists-action="override">
<value>@((string)context.Variables["x-ms-documentdb-isquery"])</value>
</set-header>
<set-header name="x-ms-documentdb-query-enablecrosspartition" exists-action="override">
<value>@((string)context.Variables["x-ms-documentdb-query-enablecrosspartition"])</value>
</set-header>
<set-header name="x-ms-max-item-count" exists-action="override">
<value>@((string)context.Variables["x-ms-max-item-count"])</value>
</set-header>
<set-header name="x-ms-version" exists-action="override">
<value>@((string)context.Variables["x-ms-version"])</value>
</set-header>
<set-header name="x-ms-documentdb-partitionkey" exists-action="override">
<value>@("[\""+context.Subscription.Id+"\"]")</value>
</set-header>
<set-header name="x-ms-date" exists-action="override">
<value>@( (string)context.Variables["x-ms-date"] )</value>
</set-header>
<set-variable name="StringToSign" value="@(string.Format("post\ndocs\ndbs/WebApi/colls/Logs\n{0}\n\n", ((string)context.Variables["x-ms-date"]).ToLowerInvariant()))" />
<set-variable name="cosmosreadonlykey" value="{{CosmosReadOnlyKey}}" />
<set-variable name="SharedKey" value="@{
// https://docs.microsoft.com/en-us/rest/api/documentdb/access-control-on-documentdb-resources#constructkeytoken
System.Security.Cryptography.HMACSHA256 hasher = new System.Security.Cryptography.HMACSHA256(Convert.FromBase64String((string)context.Variables["cosmosreadonlykey"]));
return Convert.ToBase64String(hasher.ComputeHash(System.Text.Encoding.UTF8.GetBytes((string)context.Variables["StringToSign"])));
}" />
<set-variable name="Authorization" value="@(string.Format("type=master&ver=1.0&sig={0}", ((string)context.Variables["SharedKey"]).Replace("&","%26").Replace("+","%2B").Replace("=","%3D")))" />
<set-header name="Authorization" exists-action="override">
<value>@((string)context.Variables["Authorization"])</value>
</set-header>
<set-backend-service base-url="https://mycosmosdb.documents.azure.com" />
<rewrite-uri template="/dbs/WebApi/colls/Logs/docs" />
</inbound>
</policies>
- the partition key needs to be formatted as an array
- 日期被放入header和基于相同值的StringToSign
- did some hacky URL hex encoding - 可以通过适当的十六进制编码进行改进