从 Azure Table 存储 OData 筛选器查询中读取超过 1000 行？

Question

我们如何从 Azure Table 存储中读取超过 1000 行？这是我用于从 Table 存储读取数据的代码，它只检索 1000 行：

readData()
{
    var s = @$"https://{storageAccountName}.table.core.windows.net/{tableName}()";          
    var baseurl = $"{s}<sas-token>&$filter=Name%20eq%20'XYZ'";
    var data = GetForOData(baseurl);
    var responseData = data.Data.Replace(".", "_");
    var odata = JsonConvert.DeserializeObject<ODataResponse>(responseData); 
}

GetForOData(string url) {return InvokeForOData<Object>("GET", url, null, null);}

private static HttpResponseData InvokeForOData<T>(string method, string url, Object id, T data)
{
    var Response = new HttpResponseData()
    {
         Code = HttpStatusCode.RequestTimeout, Data = string.Empty, Message = string.Empty
    };
            
    var PostParam = string.Empty;
    if (data != null) { PostParam = data.ToString(); }
    var postData = Encoding.UTF8.GetBytes(PostParam);
    var request = (HttpWebRequest)WebRequest.Create(new Uri(url + (id == null ? "" : '/' + id.ToString())));
     request.Method = method;
     // add headers
     if (postData.Length > 0)
     {
        using (Stream requestStream = request.GetRequestStream())
        { requestStream.Write(postData, 0, postData.Length); }
     }
     using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
     {
        Response.Code = response.StatusCode;
        using (var stream = new StreamReader(response.GetResponseStream(), Encoding.UTF8))
        { Response.Data = stream.ReadToEnd(); }
     }            
     return Response;
}

我在哪里检查 x-ms-continuation-NextPartitionKey 和 x-ms-continuation-NextRowKey 并在下一个请求中使用它们？

更新：我能够找到 nextPartitionKey 和 nextRowKey header 值。如何在下一个请求中传递这些值？

                using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
                {
                    Response.Code = response.StatusCode;
                    var nextPartitionKey = response.Headers["x-ms-continuation-NextPartitionKey"];
                    var nextRowKey = response.Headers["x-ms-continuation-NextRowKey"];
                    using (StreamReader stream = new StreamReader(response.GetResponseStream(), Encoding.UTF8))
                    {
                        Response.Data = stream.ReadToEnd();
                    }
                }

Answer 1

对 Azure Table 存储的单个调用将 return 最多 1000 个实体。如果有更多实体与查询匹配，您将获得一个延续令牌，您将需要使用它来获取下一组实体。

因此，在您的情况下，为了读取超过 1000 个实体，您必须发送请求、获取数据并检查响应中的继续令牌（x-ms-continuation-NextPartitionKey 和 x-ms-continuation-NextRowKey）并在下一个请求中使用它们。

您可以在此处了解有关 Azure Table 存储中分页的更多信息：https://docs.microsoft.com/en-us/rest/api/storageservices/query-timeout-and-pagination。

更新

请看下面的代码（虽然未经测试）：

private static HttpResponseData InvokeForOData<T>(string method, string url, Object id, T data)
{
  var Response = new HttpResponseData()
  {
        Code = HttpStatusCode.RequestTimeout, 
        Data = string.Empty, 
        Message = string.Empty,
        NextPartitionKey = string.Empty,
        NextRowKey = string.Empty
  };
            
  var PostParam = string.Empty;
  if (data != null) { PostParam = data.ToString(); }
  var postData = Encoding.UTF8.GetBytes(PostParam);
  var request = (HttpWebRequest)WebRequest.Create(new Uri(url + (id == null ? "" : '/' + id.ToString())));
    request.Method = method;
    // add headers
    if (postData.Length > 0)
    {
      using (Stream requestStream = request.GetRequestStream())
      { requestStream.Write(postData, 0, postData.Length); }
    }
  using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
  {
      Response.Code = response.StatusCode;
      var nextPartitionKey = response.Headers["x-ms-continuation-NextPartitionKey"];
      var nextRowKey = response.Headers["x-ms-continuation-NextRowKey"];
      using (StreamReader stream = new StreamReader(response.GetResponseStream(), Encoding.UTF8))
      {
          Response.Data = stream.ReadToEnd();
          Response.NextPartitionKey = nextPartitionKey;
          Response.NextRowKey = nextRowKey;
      }
  }     
  return Response;
}

基本上我所做的是在您的 HttpResponseData 对象中添加两个属性（NextPartitionKey 和 NextRowKey）并用 nextPartitionKey 和 [=17= 填充它] 分别从响应头中得到。

您在代码（处理响应的地方）中必须做的是检查这两个值中的任何一个是否不为 null 或为空。非 null/empty 值表示存在更多实体。

如果是这种情况，那么您需要做的是通过附加 NextPartitionKey 和 NextRowKey 值作为查询字符串参数来修改 URL（请参阅 link 上面的详细信息）并再次发送请求。您将需要执行此操作，直到您将这两个值都作为 null 或空字符串。

从 Azure Table 存储 OData 筛选器查询中读取超过 1000 行？

Read more than 1000 rows from Azure Table Storage OData filter query?

c#

azure-table-storage