无法使用 FNV-1 算法检查密钥分发

Question

根据很多地方的推荐，我正在使用 FNV-1 方法为具有 5 个分区的 Service Fabric 集群生成分区键，如下所示：

[
  {
    "lowKey": -5534023222112865000,
    "highKey": -1844674407370955300,
    "id": "92b4c32e-cb3c-432f-aea0-4579fc72850c",
    "kind": 2
  },
  {
    "lowKey": 5534023222112865000,
    "highKey": 9223372036854776000,
    "id": "dd9b084d-88fa-4773-94d9-f634d22a2c03",
    "kind": 2
  },
  {
    "lowKey": -9223372036854776000,
    "highKey": -5534023222112865000,
    "id": "d7322e52-cf1e-47c2-b713-e983f443b533",
    "kind": 2
  },
  {
    "lowKey": -1844674407370955300,
    "highKey": 1844674407370955300,
    "id": "dfe2f994-bdc7-4b32-b54f-78a0245e4c4c",
    "kind": 2
  },
  {
    "lowKey": 1844674407370955300,
    "highKey": 5534023222112865000,
    "id": "6ad81ffb-5f3c-48ba-91d9-e65b23c528a4",
    "kind": 2
  }
]

我想检查我的数据分布情况。

但是，我看不到如何执行此操作，因为数字太大了。

我假设我必须为此使用 LowKey="-9223372036854775808" HighKey="9223372036854775807"？

如果是这样，这些数字太 big/small 我无法在 SQL 或 C# 中处理（C# 因为常量太大）。

有办法解决这个问题吗？如果我尝试将这些值保存在我的数据库中，我会遇到溢出问题。

我正在使用 SQL Server 2017。

DECLARE @tbl TABLE(LowKey BIGINT, HighKey BIGINT, TotalITems INT)

INSERT INTO @tbl(LowKey, HighKey, TotalITems) VALUES (5534023222112865000, 9223372036854776000, 0)
INSERT INTO @tbl(LowKey, HighKey, TotalITems) VALUES (-9223372036854776000, -5534023222112865000, 0)

INSERT INTO @tbl(LowKey, HighKey, TotalITems) VALUES (-5534023222112865000, -1844674407370955300, 0)
INSERT INTO @tbl(LowKey, HighKey, TotalITems) VALUES (-1844674407370955300, 1844674407370955300, 0)
INSERT INTO @tbl(LowKey, HighKey, TotalITems) VALUES (1844674407370955300, 5534023222112865000, 0)

前两行因错误而失败

Msg 8115, Level 16, State 2, Line 168
Arithmetic overflow error converting expression to data type bigint.

这是我用来查看分区的逻辑：

[ActionName("GetPartitions")]
public async Task<ActionResult> GetPartitionsAsync()
{
   var result = new List<ServicePartitionInformation>();
   ServicePartitionList partitions = await FabricClient.QueryManager.GetPartitionListAsync(FullServiceName).ConfigureAwait(false);

   foreach (var partition in partitions)
   {
       result.Add(partition.PartitionInformation);
   }

   return new JsonResult(result);
}

保罗

Answer 1

Service Fabric 可以使用 UniformInt64Partition。这意味着最低价为 -9223372036854775808，最高价为 9223372036854775807。

您可以使用

检查

Console.WriteLine(long.MinValue);
Console.WriteLine(long.MaxValue);

所以你的json不正确。例如，-9223372036854776000 不是有效的 long/Int64 值。

I am assuming that I have to use LowKey="-9223372036854775808" HighKey="9223372036854775807" for this?

If so, these numbers are to big/small for me to process in either SQL or C# (C# because the constants are too big)

不可能，这些是基于 Int64 的有状态服务的默认值。

<Service Name="Stateful1" ServicePackageActivationMode="ExclusiveProcess">
  <StatefulService ServiceTypeName="Stateful1Type" TargetReplicaSetSize="[Stateful1_TargetReplicaSetSize]" MinReplicaSetSize="[Stateful1_MinReplicaSetSize]">
    <UniformInt64Partition PartitionCount="[Stateful1_PartitionCount]" LowKey="-9223372036854775808" HighKey="9223372036854775807" />
  </StatefulService>
</Service>

当我运行你的代码针对新的默认有状态服务时，我得到

[
    {
        "LowKey": 5534023222112865485,
        "HighKey": 9223372036854775807,
        "Id": "ef982303-dd13-49f0-ad3c-5de7428eccad",
        "Kind": 2
    },
    {
        "LowKey": -1844674407370955161,
        "HighKey": 1844674407370955161,
        "Id": "e2e5f51b-d3f5-4381-b7ff-6267124d0adb",
        "Kind": 2
    },
    {
        "LowKey": -5534023222112865484,
        "HighKey": -1844674407370955162,
        "Id": "4715db41-124a-4eee-b5b9-2587256ed6e1",
        "Kind": 2
    },
    {
        "LowKey": -9223372036854775808,
        "HighKey": -5534023222112865485,
        "Id": "10e9d445-53e0-4f27-9434-0ea5621ac188",
        "Kind": 2
    },
    {
        "LowKey": 1844674407370955162,
        "HighKey": 5534023222112865484,
        "Id": "687c1c4d-9fc1-4574-a916-66ac5ebfed25",
        "Kind": 2
    }
]

如您所见，您的高调 9223372036854776000 和低调 -9223372036854776000 无效，不在我生成的 json 中。不知道那是从哪里来的。

我要重现的代码：

    protected override async Task RunAsync(CancellationToken cancellationToken)
    {
        var result = new List<ServicePartitionInformation>();
        ServicePartitionList partitions = await new FabricClient().QueryManager.GetPartitionListAsync(Context.ServiceName).ConfigureAwait(false);

        foreach (var partition in partitions)
        {
            result.Add(partition.PartitionInformation);
        }

        var info = JsonConvert.SerializeObject(result);
        Console.WriteLine(info);
    }

无法使用 FNV-1 算法检查密钥分发

Can't check key distribution with FNV-1 algorithm

c#

hash

azure

azure-service-fabric

service-fabric-stateful