不等于 Core 2.1 中 Encoding.Default.GetString 的字符串和等于字节数组的 .net5

Not equal strings by Encoding.Default.GetString in Core 2.1 and .net5 from equals byte arrays

我在将服务重写为 .net5 时遇到了这个麻烦。我检查了密码哈希,它与数据库中的哈希不相等。例如,我编写了小型控制台程序。 Core 2.1 和 .net5 的工作结果不同

static void Main(string[] args)
{
   var password = "MlsU37z*!";
   var bytesOfHash = GetBytesOfHash(password);

   Console.WriteLine(BitConverter.ToString(bytesOfHash));
   var hashString = Encoding.Default.GetString(bytesOfHash);
   Console.WriteLine(string.Join("-", hashString.Select(ch => $"{(ushort)ch:X4}")));
   Console.WriteLine($"Hash string length = {hashString.Length}");
           
   Console.ReadKey();
}

Core2.1 的结果:

29-E7-2A-D5-85-F7-ED-5F-3E-E2-F4-23-7E-09-D0-48-7E-0E-E9-B0-70-9E-D5 -08-B0-A9-BA-75-30-C7-8E-B9-A9-8B-93-71-FC-7D-E8-B5-24-C2-80-EE-41-58-8C-D2 -7E-0D-78-87-30-C1-A2-2D-FF-D9-A4-95-B3-97-95-BF-AA-44-62-06-02-46-3F-96-0E -0F-C3-86-DE-18-97-AB-A9-59-CF-E6-14-F8-DE-66-0D-44-CF-1B-16-5D-5F-8A-75-96 -58-FC-FE-A4-14-B1-17-6D-DE-CF-B4-FE-0B-95-DB-96-39-49-48-0C-7B-8A-C1-6F-62 -8F-63-E6-B1-77-BC-41-B6-FC-D3-5A-4B-BB-05-E6-02-F4-9D-40-C5-9B-97-33-6A-6D -21-83-D6-F3-56-87-68-56-C3-1A-97-57-95-ED-CC-47-DB-EA-35-73-EE-83-3B-40-E3 -95-93-0C-87-10-64-28-8F-39-B6-8B-FD-15-7D-1D-AB-43-AC-97-9F-23-FF-1F-60-E8 -8A-21-A1-40-E6-4F-D4-E7-05-BC-52-E9-5B-0F-8D-0E-F4-EA-29-05-B6-2A-10-44-CF -D6-8A-5A-71-36-C6-4C-99-8B-B4-CC-39-89-F9-B2-3A-C3-1D-A3-AD-17-78-FA-E4-5E -89-25-07-55-C1-21-E2-EB-C4-AD-96-76-38-5E-6F-8F-DC-0F-04-E9-DF-20-B5-A1-C2 -85-C4-0E-F7-AF-9A-5F-C5-E9-BE-16-98-D9-F0-B1-48-77-0E-1D-E5-05-02-0A-EE-4F -F6-27-0A-B7-3C-20-BA-FA-B3-21-A2-8E-D7-51-C5-14-E1-CB-61-D2-36-1B-47-8D-57 -17-CB-61-D9-FE-F3-93-F8-2 F-2E-2E-80-B0-D3-F7-F6-3C-06-CE-92-7E-49-88-C8-4D-38-F5-BF-61-4D-33-2E-51- 53-DC-33-E4​​-8E-33-31-57-F4-BF-E1-A9-B8-13-A0-AB-5A-C0-D5-DC-0E-61-26-AD-4C- A3-48-46-C6-05-C5-79-47-CC-4B-CB-C8-21-31-56-34-B5-BA-52-85-87-F2-DB-71-FF- 3D-D6-FD-C0-23-E4-6E-23-F2-78-A1-04-B3-1B-B4-A9-A2-AE-EB- 55-75-89-08-01-6D-B8-00-49-1D-44-C4-8E-9B-21-E2-5B-4F-59-41-00-43-4F-C5-B3- C3-53-E5-2F-95-06-24-30-A3-B6-87-0F-DB-31-C4-AE-D7-8C-AE-D9-BD-C8-9F-F1-04- E5-5F-A9-D7-9B-3B-21-51-FC-B4-44-23-D3-75-34-3B-1F-93-99-6C-D5-EE-D4-65-96- B4-16-2D-30-4E-B0-DD-D3-31-CF-53-0E

0029-FFFD-002A-0545-FFFD-FFFD-005F-003E-FFFD-FFFD-0023-007E-0009-FFFD-0048-007E-000E-FFFD-0070-FFFD-FFFD-0008-FFFD -FFFD-FFFD-0075-0030-01CE-FFFD-FFFD-FFFD-FFFD-0071-FFFD-007D-FFFD-0024-0080-FFFD-0041-0058-FFFD-FFFD-007E-000D-0078-FFFD-0030 -FFFD-FFFD-002D-FFFD-0664-FFFD-FFFD-FFFD-FFFD-FFFD-FFFD-0044-0062-0006-0002-0046-003F-FFFD-000E-000F-00C6-FFFD-0018-FFFD-FFFD -FFFD-0059-FFFD-FFFD-0014-FFFD-FFFD-0066-000D-0044-FFFD-001B-0016-005D-005F-FFFD-0075-FFFD-0058-FFFD-FFFD-FFFD-0014-FFFD-0017 -006D-FFFD-03F4-FFFD-000B-FFFD-06D6-0039-0049-0048-000C-007B-FFFD-FFFD-006F-0062-FFFD-0063-FFFD-0077-FFFD-0041-FFFD-FFFD-FFFD -005A-004B-FFFD-0005-FFFD-0002-FFFD-0040-015B-FFFD-0033-006A-006D-0021-FFFD-FFFD-FFFD-0056-FFFD-0068-0056-FFFD-001A-FFFD-0057 -FFFD-FFFD-FFFD-0047-FFFD-FFFD-0035-0073-FFFD-003B-0040-3553-000C-FFFD-0010-0064-0028-FFFD-0039-FFFD-FFFD-FFFD-0015-007D-001D -FFFD-0043-FFFD-FFFD-FFFD-0023-FFFD-001F-0060-FFFD-0021-FFFD-0040-FFFD-004F-FFFD-FFFD-0005-FFFD-0052-FFFD-005B-000F-FFFD-000E -FFFD -FFFD-0029-0005-FFFD-002A-0010-0044-FFFD-058A-005A-0071-0036-FFFD-004C-FFFD-FFFD-FFFD-FFFD-0039-FFFD-FFFD-FFFD-003A-FFFD-001D -FFFD-FFFD-0017-0078-FFFD-FFFD-005E-FFFD-0025-0007-0055-FFFD-0021-FFFD-FFFD-012D-FFFD-0076-0038-005E-006F-FFFD-FFFD-000F-0004 -FFFD-FFFD-0020-FFFD-FFFD-0085- FFFD-000E-FFFD-FFFD-FFFD-005F-FFFD-FFFD-0016-FFFD-FFFD-FFFD-0048-0077-000E-001D-FFFD-0005-0002-000A-FFFD-004F-FFFD-0027-000A- FFFD-003C-0020-FFFD-FFFD-FFFD-0021-FFFD-FFFD-FFFD-0051-FFFD-0014-FFFD-FFFD-0061-FFFD-0036-001B-0047-FFFD-0057-0017-FFFD-0061- FFFD-FFFD-FFFD-FFFD-002F-002E-002E-FFFD-FFFD-FFFD-FFFD-FFFD-003C-0006-0392-007E-0049-FFFD-FFFD-004D-0038-FFFD-FFFD-0061-004D- 0033-002E-0051-0053-FFFD-0033-FFFD-0033-0031-0057-FFFD-1A78-0013-FFFD-FFFD-005A-FFFD-FFFD-FFFD-000E-0061-0026-FFFD-004C-FFFD- 0048-0046-FFFD-0005-FFFD-0079-0047-FFFD-004B-FFFD-FFFD-0021-0031-0056-0034-FFFD-FFFD-0052-FFFD-FFFD-FFFD-FFFD-0071-FFFD-003D- FFFD-FFFD-FFFD-0023-FFFD-006E-0023-FFFD-0078-FFFD-0004-FFFD-001B-FFFD-FFFD-FFFD-FFFD-FFFD-0055-0075-FFFD-0008-0001-006D-FFFD- 0000-0049-001D-0044-010E-FFFD-0021-FFFD-005B-004F-0059-0041-0000-0043-004F-0173-FFFD-0053-FFFD-002F-FFFD-0006-0024-0030-FFFD- FFFD-FFFD-000F-FFFD-0031-012E-05CC-FFFD-067D-021F-FFFD-0004-FFFD-005F-FFFD-05DB-003B-0021-0051-FFFD-FFFD-0044-0023-FFFD-0075- 0034-003B-001F-FFFD-FFFD-006C-FFFD-FFFD-FFFD-0065-FFFD-FFFD-0016-002D-0030-004E-FFFD-FFFD-FFFD-0031-FFFD-0053-000E

哈希字符串长度 = 478

.net5 的结果:

29-E7-2A-D5-85-F7-ED-5F-3E-E2-F4-23-7E-09-D0-48-7E-0E-E9-B0-70-9E-D5 -08-B0-A9-BA-75-30-C7-8E-B9-A9-8B-93-71-FC-7D-E8-B5-24-C2-80-EE-41-58-8C-D2 -7E-0D-78-87-30-C1-A2-2D-FF-D9-A4-95-B3-97-95-BF-AA-44-62-06-02-46-3F-96-0E -0F-C3-86-DE-18-97-AB-A9-59-CF-E6-14-F8-DE-66-0D-44-CF-1B-16-5D-5F-8A-75-96 -58-FC-FE-A4-14-B1-17-6D-DE-CF-B4-FE-0B-95-DB-96-39-49-48-0C-7B-8A-C1-6F-62 -8F-63-E6-B1-77-BC-41-B6-FC-D3-5A-4B-BB-05-E6-02-F4-9D-40-C5-9B-97-33-6A-6D -21-83-D6-F3-56-87-68-56-C3-1A-97-57-95-ED-CC-47-DB-EA-35-73-EE-83-3B-40-E3 -95-93-0C-87-10-64-28-8F-39-B6-8B-FD-15-7D-1D-AB-43-AC-97-9F-23-FF-1F-60-E8 -8A-21-A1-40-E6-4F-D4-E7-05-BC-52-E9-5B-0F-8D-0E-F4-EA-29-05-B6-2A-10-44-CF -D6-8A-5A-71-36-C6-4C-99-8B-B4-CC-39-89-F9-B2-3A-C3-1D-A3-AD-17-78-FA-E4-5E -89-25-07-55-C1-21-E2-EB-C4-AD-96-76-38-5E-6F-8F-DC-0F-04-E9-DF-20-B5-A1-C2 -85-C4-0E-F7-AF-9A-5F-C5-E9-BE-16-98-D9-F0-B1-48-77-0E-1D-E5-05-02-0A-EE-4F -F6-27-0A-B7-3C-20-BA-FA-B3-21-A2-8E-D7-51-C5-14-E1-CB-61-D2-36-1B-47-8D-57 -17-CB-61-D9-FE-F3-93-F8-2 F-2E-2E-80-B0-D3-F7-F6-3C-06-CE-92-7E-49-88-C8-4D-38-F5-BF-61-4D-33-2E-51- 53-DC-33-E4​​-8E-33-31-57-F4-BF-E1-A9-B8-13-A0-AB-5A-C0-D5-DC-0E-61-26-AD-4C- A3-48-46-C6-05-C5-79-47-CC-4B-CB-C8-21-31-56-34-B5-BA-52-85-87-F2-DB-71-FF- 3D-D6-FD-C0-23-E4-6E-23-F2-78-A1-04-B3-1B-B4-A9-A2-AE-EB- 55-75-89-08-01-6D-B8-00-49-1D-44-C4-8E-9B-21-E2-5B-4F-59-41-00-43-4F-C5-B3- C3-53-E5-2F-95-06-24-30-A3-B6-87-0F-DB-31-C4-AE-D7-8C-AE-D9-BD-C8-9F-F1-04- E5-5F-A9-D7-9B-3B-21-51-FC-B4-44-23-D3-75-34-3B-1F-93-99-6C-D5-EE-D4-65-96- B4-16-2D-30-4E-B0-DD-D3-31-CF-53-0E

0029-FFFD-002A-0545-FFFD-FFFD-005F-003E-FFFD-FFFD-0023-007E-0009-FFFD-0048-007E-000E-FFFD-0070-FFFD-FFFD-0008-FFFD -FFFD-FFFD-0075-0030-01CE-FFFD-FFFD-FFFD-FFFD-0071-FFFD-007D-FFFD-0024-0080-FFFD-0041-0058-FFFD-FFFD-007E-000D-0078-FFFD-0030 -FFFD-FFFD-002D-FFFD-0664-FFFD-FFFD-FFFD-FFFD-FFFD-FFFD-0044-0062-0006-0002-0046-003F-FFFD-000E-000F-00C6-FFFD-0018-FFFD-FFFD -FFFD-0059-FFFD-FFFD-0014-FFFD-FFFD-0066-000D-0044-FFFD-001B-0016-005D-005F-FFFD-0075-FFFD-0058-FFFD-FFFD-FFFD-0014-FFFD-0017 -006D-FFFD-03F4-FFFD-000B-FFFD-06D6-0039-0049-0048-000C-007B-FFFD-FFFD-006F-0062-FFFD-0063-FFFD-0077-FFFD-0041-FFFD-FFFD-FFFD -005A-004B-FFFD-0005-FFFD-0002-FFFD-FFFD-0040-015B-FFFD-0033-006A-006D-0021-FFFD-FFFD-FFFD-0056-FFFD-0068-0056-FFFD-001A-FFFD -0057-FFFD-FFFD-FFFD-0047-FFFD-FFFD-0035-0073-FFFD-003B-0040-3553-000C-FFFD-0010-0064-0028-FFFD-0039-FFFD-FFFD-FFFD-0015-007D -001D-FFFD-0043-FFFD-FFFD-FFFD-0023-FFFD-001F-0060-FFFD-0021-FFFD-0040-FFFD-004F-FFFD-FFFD-0005-FFFD-0052-FFFD-005B-000F-FFFD -000E -FFFD-FFFD-0029-0005-FFFD-002A-0010-0044-FFFD-058A-005A-0071-0036-FFFD-004C-FFFD-FFFD-FFFD-FFFD-0039-FFFD-FFFD-FFFD-003A-FFFD -001D-FFFD-FFFD-0017-0078-FFFD-FFFD-005E-FFFD-0025-0007-0055-FFFD-0021-FFFD-FFFD-012D-FFFD-0076-0038-005E-006F-FFFD-FFFD-000F -0004-FFFD-FFFD-0020-FFFD-FFFD- 0085-FFFD-000E-FFFD-FFFD-FFFD-005F-FFFD-FFFD-0016-FFFD-FFFD-FFFD-0048-0077-000E-001D-FFFD-0005-0002-000A-FFFD-004F-FFFD-0027- 000A-FFFD-003C-0020-FFFD-FFFD-FFFD-0021-FFFD-FFFD-FFFD-0051-FFFD-0014-FFFD-FFFD-0061-FFFD-0036-001B-0047-FFFD-0057-0017-FFFD- 0061-FFFD-FFFD-FFFD-FFFD-002F-002E-002E-FFFD-FFFD-FFFD-FFFD-FFFD-003C-0006-0392-007E-0049-FFFD-FFFD-004D-0038-FFFD-FFFD-0061- 004D-0033-002E-0051-0053-FFFD-0033-FFFD-0033-0031-0057-FFFD-FFFD-1A78-0013-FFFD-FFFD-005A-FFFD-FFFD-FFFD-000E-0061-0026-FFFD- 004C-FFFD-0048-0046-FFFD-0005-FFFD-0079-0047-FFFD-004B-FFFD-FFFD-0021-0031-0056-0034-FFFD-FFFD-0052-FFFD-FFFD-FFFD-FFFD-0071- FFFD-003D-FFFD-FFFD-FFFD-0023-FFFD-006E-0023-FFFD-0078-FFFD-0004-FFFD-001B-FFFD-FFFD-FFFD-FFFD-FFFD-0055-0075-FFFD-0008-0001- 006D-FFFD-0000-0049-001D-0044-010E-FFFD-0021-FFFD-005B-004F-0059-0041-0000-0043-004F-0173-FFFD-0053-FFFD-002F-FFFD-0006-0024- 0030-FFFD-FFFD-FFFD-000F-FFFD-0031-012E-05CC-FFFD-067D-021F-FFFD-0004-FFFD-005F-FFFD-05DB-003B-0021-0051-FFFD-FFFD-0044-0023- FFFD-0075-0034-003B-001F-FFFD-FFFD-006C-FFFD-FFFD-FFFD-0065-FFFD-FFFD-0016-002D-0030-004E-FFFD-FFFD-FFFD-0031-FFFD-0053-000E

哈希字符串长度 = 480

我运行两个控制台程序在同一台计算机上。

你在字节数组上使用 Encoding.GetString 是没有意义的,而且是错误的,它绝不代表文本字符串。所以永远不要那样做!这个答案的其余部分是关于相同(非法!)字节序列在两个版本的 .NET 中给出不同字符串的方式。


经过评论并向问题添加实际数据后,看起来原因确实是 UTF-8 解码器在遇到非法字节序列时的差异。特别是,当他们第一次看到一个以位 11110 开头的正确字节表示 4 字节编码的开始时,然后看到一个以 10 开头的正确字节表示第一个纯“有效负载”字节(两个更多该类型的预期),但随后在上下文中看到一个错误的字节,一个不是以 10 开头的字节,然后解码器不同意发出多少 U+FFFD 'REPLACEMENT CHARACTER'

特别是,提问者在评论中确认 Encoding.Default.GetString(new byte[] { 0xF4, 0x9D, 0x40, }) 在 .NET Core 2.1 中给出 "�@",但在 .NET 5 中给出 "��@"。字节 F4承诺 4 字节序列的开始,字节 9D 看起来和字节 2 一样正确,但随后纯 ASCII 字节 40(意思是 @)打断了序列。所以分歧在于它是显示为 "�@" 还是 "��@".

字符串不同的另一种情况类似,除了这里 4 字节序列被一个以 1110 开头的字节打断,而不是一个以 0 开头的字节。

我想你可以更普遍地使用 Encoding.UTF8.GetString(new byte[] { 0xF4, 0x9D, 0x40, })(因为如果你的 OS 设置为 language/locale 可以用遗留的 1 字节编码,例如 Windows-1252). Addition: I see in the documentation of Default,它将在所有 .NET Core 变体(包括 .NET Core 2.1 和 .NET 5)上提供 UTF-8。它仅在旧的 .NET Framework(如 .NET 框架)上使用。 NET Framework 4.8) Default 可能会产生一个 ANSI 代码页,例如 Windows-1252.