您如何正确快速地比较数据行/数据表？

Question

更新：解释我正在比较的数据类型table- “比较具有相同列的两个数据 table，一个数据 table 正在从外部服务器中拉出并插入最初，从那时起，只有最近 6 个月的记录从外部数据库中拉出（对于各种reasons) ，并将数据与本地数据（6 个月的日期范围）进行比较，以查看 DataRow 是否已更改，需要删除或添加行标识符 (PKey) 本质上是 SalesID + LineRow 匹配和其他列是要比较的值，以查看该行是否需要 re-added/deleted，因为传入列与当前列不同，并且还会删除传入数据不包含这些行的行

所以基本上我想要一个 Exclusive Left Join [插入数据] 和独占权加入[删除该数据] “

我一直在做一些数据库编码以及 JSON 拉动，我想知道什么是标准的/正确的做事方式，我从 2 小时的比较时间开始（在虚拟数据库上 table) 减少到 1 小时到 1 秒（在将我的 janky 方法应用到 DB table Compare 之后），然后最终将它用于实时拉取，结果似乎是正确且一致的，所以我开始进行测试在虚拟数据上，它从 1 小时到 26 分钟到最后 <1 秒（使用我自己的简陋方式），Table 测试和假设的大小在 100,000 和 200,000 行之间所以让我们通过我尝试过的标准方法，然后继续我制作的简陋解决方案。

第一个明显的想法是使用两次 ForEach 迭代（即使在心理上这似乎会很慢，但我认为考虑到 Add 的速度有多快以及您可以多快在遍历 Jarrays 时比较 JSON 个标记）。代码类似于以下内容：

DataTable dtQueryItemsDiff = dtItems.Clone();
DataTable dtItemsDiff = dtItems.Clone();

int maxRowCountCache = dtItems.AsEnumerable().OrderBy(row => Convert.ToDateTime(row.Field<String>("Date"))).ThenBy(row => row.Field<String>("Name")).Count();
int rowcountCCache = 0;
var query = dtQuery.AsEnumerable().OrderBy(row => Convert.ToDateTime(row.Field<String>("Date"))).ThenBy(row => row.Field<String>("Name"));
foreach (DataRow drDTI in dtItems.AsEnumerable().OrderBy(row => Convert.ToDateTime(row.Field<String>("Date"))).ThenBy(row => row.Field<String>("Name")))
{
    int innerrowcount = 0;
    bool rowfound = false;
    if (query.Count() != 0)
    {
        foreach (DataRow drDTQ in query)
        {
            if (drDTI["SalesID"].ToString() == drDTI["SalesID"].ToString() && drDTI["LineNumber"].ToString() == drDTI["LineNumber"].ToString())
            {
                rowfound = true;
                break;
            }
            innerrowcount++;
        }
    }
    else
    {
        dtItemsDiff.ImportRow(drDTI);
        continue;
    }
    if (rowfound == true)
    {
        orderedDtquery.ElementAt(innerrowcount).Delete();
    }
    else
    {
        dtItemsDiff.ImportRow(drDTI);
    }
    rowcountCCache++;
    BeginInvoke(new MethodInvoker(delegate
    {
        lblDataLoadC.Text = rowcountCCache.ToString() + " / " + maxRowCountCache.ToString();
    }));
}
if (query.Count() != 0)
{
    foreach (DataRow drDTQ in query)
    {
        dtQueryItemsDiff.ImportRow(drDTQ);
    }
}

这花费了大约 1H（1 小时）到 1.5H 的相当长的时间，具体取决于数据的排序方式等。好处是我可以精细地更改代码，它在两个方面都为我提供了不匹配的数据tables，它还减少了搜索的查询大小，但这对我来说还不够快，所以我尝试了 Linq 搜索，但我并没有减少列表大小（先删除再搜索再慢一些）只是搜索），这花了大约 40-50 分钟，看起来像：

int maxRowCountCache = dtItems.AsEnumerable().OrderBy(row => Convert.ToDateTime(row.Field<String>("Date"))).ThenBy(row => row.Field<String>("Name")).Count();
int rowcountCCache = 0;
dtItems.AcceptChanges();
foreach (DataRow drDTI in dtItems.AsEnumerable().OrderBy(row => Convert.ToDateTime(row.Field<String>("Date"))).ThenBy(row => row.Field<String>("Name")))
{
    var checkIfRecordInIDB = progSettings.query.AsEnumerable().Where(row => row.Field<string>("CardRecordID") == drDTI["CardRecordID"].ToString()
    && row.Field<string>("Date") == drDTI["Date"].ToString() && row.Field<string>("SaleID") == drDTI["SaleID"].ToString()
    && row.Field<string>("ItemID") == drDTI["ItemID"].ToString() && row.Field<Int64>("LineNumber") == Convert.ToInt64(drDTI["LineNumber"].ToString())).FirstOrDefault();
    if (checkIfRecordInIDB != null)
    {
        drDTI.Delete();
    }
    rowcountCCache++;
    BeginInvoke(new MethodInvoker(delegate
    {
        lblDataLoadC.Text = rowcountCCache.ToString() + " / " + maxRowCountCache.ToString();
    }));
}
dtItems.AcceptChanges();

这样做的好处是它稍微更懒惰、更快和简洁，但是它只为您提供一个 table 中的数据，就像 Except 所做的一样，这正是我接下来使用 ~100,000 行虚拟数据尝试的, 这花了 26 分 35 秒。

dtItems.Rows.Clear();
query.Rows.Clear();
Thread start = new Thread(timerAndUIupdate);
start.Start();

dtItems.Rows.Add("4 Beans Cafe", "2af0f4bf-52ea-44fb-b1b3-36181fe7bfdf", "2019-07-01", "2019-07-01", "7fc4f98a-35af-4da3-afe3-f7cfcd922ea7", "72421ee8-459b-46fb-bf5a-f51e80976e5a", "Pioneer 1kg (FT), RRP ", "100115", 1, 25.0, "N");
dtItems.Rows.Add("4 Beans Cafe", "2af0f4bf-52ea-44fb-b1b3-36181fe7bfdf", "2019-07-01", "2019-07-01", "7fc4f98a-35af-4da3-afe3-f7cfcd922ea7", "8885a911-8d32-4dfe-93e5-2e453fd54db9", "Decaf Beans 250g FT", "1002302", 2, 2.0, "N");
dtItems.Rows.Add("4 Beans Cafe", "2af0f4bf-52ea-44fb-b1b3-36181fe7bfdf", "2019-07-01", "2019-07-01", "7fc4f98a-35af-4da3-afe3-f7cfcd922ea7", "e3aa4b15-b774-4f6a-ac21-77fa05a4332f", "P&R Cups 06oz (1000)", "30056", 3, 1.0, "N");
dtItems.Rows.Add("4 Beans Cafe", "2af0f4bf-52ea-44fb-b1b3-36181fe7bfdf", "2019-07-01", "2019-07-01", "7fc4f98a-35af-4da3-afe3-f7cfcd922ea7", "51e1a867-4079-4a3c-9ddc-e93d87d80b46", "P&R Cups 12oz (1000)", "30058", 4, 1.0, "N");
query.Rows.Add("4 Beans Cafe", "2af0f4bf-52ea-44fb-b1b3-36181fe7bfdf", "2019-07-01", "2019-07-01", "7fc4f98a-35af-4da3-afe3-f7cfcd922ea7", "72421ee8-459b-46fb-bf5a-f51e80976e5a", "Pioneer 1kg (FT), RRP ", "100115", 1, 25.0, "N");
query.Rows.Add("4 Beans Cafe", "2af0f4bf-52ea-44fb-b1b3-36181fe7bfdf", "2019-07-01", "2019-07-01", "7fc4f98a-35af-4da3-afe3-f7cfcd922ea7", "8885a911-8d32-4dfe-93e5-2e453fd54db9", "Decaf Beans 250g FT", "1002302", 2, 2.0, "N");
query.Rows.Add("4 Beans Cafe", "2af0f4bf-52ea-44fb-b1b3-36181fe7bfdf", "2019-07-01", "2019-07-01", "7fc4f98a-35af-4da3-afe3-f7cfcd922ea7", "e3aa4b15-b774-4f6a-ac21-77fa05a4332f", "P&R Cups 06oz (1000)", "30056", 3, 1.0, "N");
query.Rows.Add("4 Beans Cafe", "2af0f4bf-52ea-44fb-b1b3-36181fe7bfdf", "2019-07-01", "2019-07-01", "7fc4f98a-35af-4da3-afe3-f7cfcd922ea7", "51e1a867-4079-4a3c-9ddc-e93d87d80b46", "P&R Cups 12oz (1000)", "30058", 4, 1.0, "N");
for (int i = 1; i < 100000; i++)
{
    dtItems.Rows.Add("Bennett St Dairy", "ed0c8d30-6469-4e13-af5a-36d7357a4a70", "2019-07-01", "2019-07-01", "8b909a4b-a07b-4a06-bebc-6a3387433aaf", "c8cc1115-da02-42cf-b427-accc1b6d07e3", "Trailblazer 1Kg, RRP ", "10011", i, (i * 4), "N");
    query.Rows.Add("Bennett St Dairy", "ed0c8d30-6469-4e13-af5a-36d7357a4a70", "2019-07-01", "2019-07-01", "8b909a4b-a07b-4a06-bebc-6a3387433aaf", "c8cc1115-da02-42cf-b427-accc1b6d07e3", "Trailblazer 1Kg, RRP ", "10011", i, (i * 4), "N");
}
dtItems.Rows.Add("Air Coffee International Cafe Pty Ltd", "bb4fa724-9759-4c60-93fe-70fbdfd00417", "2019-07-01", "2019-07-01", "b972f020-3740-4ef2-941f-78b1a9edefa8", "0be54733-ac0e-43f9-8ea5-204c7cdb5f48", "Custom 1kg", "100116", 1, 4.0, "N");
dtItems.Rows.Add("Allure Cafe & Co.", "f76f383f-e9f4-45c9-bb93-81102629b9c3", "2019-07-01", "2019-07-01", "2ad0667f-2254-4df5-8b24-eb36736cabb0", "6edc584b-a8eb-4f0b-a449-dbcb76a40a24", "Porter St 1Kg, RRP ", "100111", 1, 10.0, "N");
dtItems.Rows.Add("Mad Hatter Wine Co", "49340e5f-c7ef-41d9-9f1b-200711e6e629", "2021-07-28", "2021-07-28", "e16cbbac-c319-45f3-ac53-89d979fbcdc1", "6edc584b-a8eb-4f0b-a449-dbcb76a40a24", "Porter St 1Kg, RRP ", "100111", 1, 30.0, "N");
dtItems.Rows.Add("Mad Hatter Wine Co", "49340e5f-c7ef-41d9-9f1b-200711e6e629", "2021-07-28", "2021-07-28", "e16cbbac-c319-45f3-ac53-89d979fbcdc1", "51e1a867-4079-4a3c-9ddc-e93d87d80b46", "P&R Cups 12oz (1000)", "30058", 2, 12.0, "N");
dtItems.Rows.Add("Mad Hatter Wine Co", "49340e5f-c7ef-41d9-9f1b-200711e6e629", "2021-07-28", "2021-07-28", "e16cbbac-c319-45f3-ac53-89d979fbcdc1", "401ce902-e158-4f21-85a5-3312c32457fc", "Lids 06/08/12oz (White) (1000)", "30062", 3, 7.0, "N");
dtItems.Rows.Add("Mad Hatter Wine Co", "49340e5f-c7ef-41d9-9f1b-200711e6e629", "2021-07-28", "2021-07-28", "e16cbbac-c319-45f3-ac53-89d979fbcdc1", "9b80c825-6e9f-4f6b-9c77-f3378cc220e4", "4-Cup Cardboard Holders (300)", "41003", 4, 1.0, "N");
dtItems.Rows.Add("Mad Hatter Wine Co", "49340e5f-c7ef-41d9-9f1b-200711e6e629", "2021-07-28", "2021-07-28", "e16cbbac-c319-45f3-ac53-89d979fbcdc1", "ea4c906e-fab1-4b15-8845-619f20e53c6a", "Organic Panela 1kg", "20014", 5, 2.0, "N");
dtItems.Rows.Add("Mad Hatter Wine Co", "49340e5f-c7ef-41d9-9f1b-200711e6e629", "2021-07-28", "2021-07-28", "e16cbbac-c319-45f3-ac53-89d979fbcdc1", "bb3e1c10-9e67-46d3-99b4-17df45dead90", "Chocolate Powder 1Kg, RRP ", "20034", 6, 1.0, "N");

query.Rows.Add("Aussie Bites Cafe", "30389aca-9089-4b37-9a1e-5fbc3c2af485", "2019-07-01", "2019-07-01", "85df1af6-3d1e-4e04-8fe9-d90462a59d4c", "ea89ade4-c7ff-4d79-abcd-dcdbb8122562", "X Blend 1Kg, RRP ", "100112", 1, 4.0, "N");
query.Rows.Add("Aussie Bites Cafe", "30389aca-9089-4b37-9a1e-5fbc3c2af485", "2019-07-01", "2019-07-01", "85df1af6-3d1e-4e04-8fe9-d90462a59d4c", "21fe57ad-08f9-4c8b-81d0-d7b88b291571", "webfreight", "webfreight", 2, 1.0, "N");
query.Rows.Add("Mad Hatter Wine Co", "49340e5f-c7ef-41d9-9f1b-200711e6e629", "2021-07-28", "2021-07-28", "e16cbbac-c319-45f3-ac53-89d979fbcdc1", "6edc584b-a8eb-4f0b-a449-dbcb76a40a24", "Porter St 1Kg, RRP ", "100111", 1, 30.0, "N");
query.Rows.Add("Mad Hatter Wine Co", "49340e5f-c7ef-41d9-9f1b-200711e6e629", "2021-07-28", "2021-07-28", "e16cbbac-c319-45f3-ac53-89d979fbcdc1", "51e1a867-4079-4a3c-9ddc-e93d87d80b46", "P&R Cups 12oz (1000)", "30058", 2, 1.0, "N");
query.Rows.Add("Mad Hatter Wine Co", "49340e5f-c7ef-41d9-9f1b-200711e6e629", "2021-07-28", "2021-07-28", "e16cbbac-c319-45f3-ac53-89d979fbcdc1", "401ce902-e158-4f21-85a5-3312c32457fc", "Lids 06/08/12oz (White) (1000)", "30062", 3, 2.0, "N");
query.Rows.Add("Mad Hatter Wine Co", "49340e5f-c7ef-41d9-9f1b-200711e6e629", "2021-07-28", "2021-07-28", "e16cbbac-c319-45f3-ac53-89d979fbcdc1", "9b80c825-6e9f-4f6b-9c77-f3378cc220e4", "4-Cup Cardboard Holders (300)", "41003", 4, 1.0, "N");
Stopwatch pullTime = new();
pullTime.Start();

BeginInvoke(new MethodInvoker(delegate
{
    lblTimerAddRowEnd.Text = "Start Time,Except: " + pullTime.Elapsed.ToString("mm\:ss\.ff");
}));

var orderedDtItems = dtItems.AsEnumerable().OrderBy(row => Convert.ToDateTime(row.Field<String>("Date"))).ThenBy(row => row.Field<String>("Name"));
var orderedDtquery = query.AsEnumerable().OrderBy(row => Convert.ToDateTime(row.Field<String>("Date"))).ThenBy(row => row.Field<String>("Name"));
DataTable excepteditems = orderedDtItems.Except(orderedDtquery, DataRowComparer.Default).CopyToDataTable();

BeginInvoke(new MethodInvoker(delegate
{
    labelControl1.Text = "End Time,Except: " + pullTime.Elapsed.ToString("mm\:ss\.ff");
}));

BeginInvoke(new MethodInvoker(delegate
{
    dgvResults.DataSource = excepteditems;
    btnStart.Enabled = true;
    simpleButton1.Enabled = true;
}));

使用此 UI 的更新程序代码（已线程化并用于所有测试比较）：

private void timerAndUIupdate()
{
    Stopwatch pullTime = new();
    pullTime.Start();
    do
    {
        Thread.Sleep(500);
        BeginInvoke(new MethodInvoker(delegate
        {
            lblTimer.Text = "Timer: " + pullTime.Elapsed.ToString("mm\:ss\.ff");
            Application.DoEvents();
        }));
    } while (btnStart.Enabled == false);
    pullTime.Stop();
    BeginInvoke(new MethodInvoker(delegate
    {
        lblTimer.Text = "Timer: " + pullTime.Elapsed.ToString("mm\:ss\.ff");
        Application.DoEvents();
    }));
}

Winforms 上的结果如下所示：

然后我做了我的简陋的方法，结果非常快而且看起来很准确，而且因为它只花了几分之一秒我可以多次执行这个来得到，新行，旧行不在拉那个不应删除和应删除的旧行 -> 代码如下所示

dtItems.Rows.Clear();
query.Rows.Clear();
Thread start = new Thread(timerAndUIupdate);
start.Start();

      
dtItems.Rows.Add("4 Beans Cafe", "2af0f4bf-52ea-44fb-b1b3-36181fe7bfdf", "2019-07-01", "2019-07-01", "7fc4f98a-35af-4da3-afe3-f7cfcd922ea7", "72421ee8-459b-46fb-bf5a-f51e80976e5a", "Pioneer 1kg (FT), RRP ", "100115", 1, 25.0, "N");
dtItems.Rows.Add("4 Beans Cafe", "2af0f4bf-52ea-44fb-b1b3-36181fe7bfdf", "2019-07-01", "2019-07-01", "7fc4f98a-35af-4da3-afe3-f7cfcd922ea7", "8885a911-8d32-4dfe-93e5-2e453fd54db9", "Decaf Beans 250g FT", "1002302", 2, 2.0, "N");
dtItems.Rows.Add("4 Beans Cafe", "2af0f4bf-52ea-44fb-b1b3-36181fe7bfdf", "2019-07-01", "2019-07-01", "7fc4f98a-35af-4da3-afe3-f7cfcd922ea7", "e3aa4b15-b774-4f6a-ac21-77fa05a4332f", "P&R Cups 06oz (1000)", "30056", 3, 1.0, "N");
dtItems.Rows.Add("4 Beans Cafe", "2af0f4bf-52ea-44fb-b1b3-36181fe7bfdf", "2019-07-01", "2019-07-01", "7fc4f98a-35af-4da3-afe3-f7cfcd922ea7", "51e1a867-4079-4a3c-9ddc-e93d87d80b46", "P&R Cups 12oz (1000)", "30058", 4, 1.0, "N");
query.Rows.Add("4 Beans Cafe", "2af0f4bf-52ea-44fb-b1b3-36181fe7bfdf", "2019-07-01", "2019-07-01", "7fc4f98a-35af-4da3-afe3-f7cfcd922ea7", "72421ee8-459b-46fb-bf5a-f51e80976e5a", "Pioneer 1kg (FT), RRP ", "100115", 1, 25.0, "N");
query.Rows.Add("4 Beans Cafe", "2af0f4bf-52ea-44fb-b1b3-36181fe7bfdf", "2019-07-01", "2019-07-01", "7fc4f98a-35af-4da3-afe3-f7cfcd922ea7", "8885a911-8d32-4dfe-93e5-2e453fd54db9", "Decaf Beans 250g FT", "1002302", 2, 2.0, "N");
query.Rows.Add("4 Beans Cafe", "2af0f4bf-52ea-44fb-b1b3-36181fe7bfdf", "2019-07-01", "2019-07-01", "7fc4f98a-35af-4da3-afe3-f7cfcd922ea7", "e3aa4b15-b774-4f6a-ac21-77fa05a4332f", "P&R Cups 06oz (1000)", "30056", 3, 1.0, "N");
query.Rows.Add("4 Beans Cafe", "2af0f4bf-52ea-44fb-b1b3-36181fe7bfdf", "2019-07-01", "2019-07-01", "7fc4f98a-35af-4da3-afe3-f7cfcd922ea7", "51e1a867-4079-4a3c-9ddc-e93d87d80b46", "P&R Cups 12oz (1000)", "30058", 4, 1.0, "N");
for (int i = 1; i < 100000; i++)
{
    dtItems.Rows.Add("Bennett St Dairy", "ed0c8d30-6469-4e13-af5a-36d7357a4a70", "2019-07-01", "2019-07-01", "8b909a4b-a07b-4a06-bebc-6a3387433aaf", "c8cc1115-da02-42cf-b427-accc1b6d07e3", "Trailblazer 1Kg, RRP ", "10011", i, (i * 4), "N");
    query.Rows.Add("Bennett St Dairy", "ed0c8d30-6469-4e13-af5a-36d7357a4a70", "2019-07-01", "2019-07-01", "8b909a4b-a07b-4a06-bebc-6a3387433aaf", "c8cc1115-da02-42cf-b427-accc1b6d07e3", "Trailblazer 1Kg, RRP ", "10011", i, (i * 4), "N");
}
dtItems.Rows.Add("Air Coffee International Cafe Pty Ltd", "bb4fa724-9759-4c60-93fe-70fbdfd00417", "2019-07-01", "2019-07-01", "b972f020-3740-4ef2-941f-78b1a9edefa8", "0be54733-ac0e-43f9-8ea5-204c7cdb5f48", "Custom 1kg", "100116", 1, 4.0, "N");
dtItems.Rows.Add("Allure Cafe & Co.", "f76f383f-e9f4-45c9-bb93-81102629b9c3", "2019-07-01", "2019-07-01", "2ad0667f-2254-4df5-8b24-eb36736cabb0", "6edc584b-a8eb-4f0b-a449-dbcb76a40a24", "Porter St 1Kg, RRP ", "100111", 1, 10.0, "N");
dtItems.Rows.Add("Mad Hatter Wine Co", "49340e5f-c7ef-41d9-9f1b-200711e6e629", "2021-07-28", "2021-07-28", "e16cbbac-c319-45f3-ac53-89d979fbcdc1", "6edc584b-a8eb-4f0b-a449-dbcb76a40a24", "Porter St 1Kg, RRP ", "100111", 1, 30.0, "N");
dtItems.Rows.Add("Mad Hatter Wine Co", "49340e5f-c7ef-41d9-9f1b-200711e6e629", "2021-07-28", "2021-07-28", "e16cbbac-c319-45f3-ac53-89d979fbcdc1", "51e1a867-4079-4a3c-9ddc-e93d87d80b46", "P&R Cups 12oz (1000)", "30058", 2, 12.0, "N");
dtItems.Rows.Add("Mad Hatter Wine Co", "49340e5f-c7ef-41d9-9f1b-200711e6e629", "2021-07-28", "2021-07-28", "e16cbbac-c319-45f3-ac53-89d979fbcdc1", "401ce902-e158-4f21-85a5-3312c32457fc", "Lids 06/08/12oz (White) (1000)", "30062", 3, 7.0, "N");
dtItems.Rows.Add("Mad Hatter Wine Co", "49340e5f-c7ef-41d9-9f1b-200711e6e629", "2021-07-28", "2021-07-28", "e16cbbac-c319-45f3-ac53-89d979fbcdc1", "9b80c825-6e9f-4f6b-9c77-f3378cc220e4", "4-Cup Cardboard Holders (300)", "41003", 4, 1.0, "N");
dtItems.Rows.Add("Mad Hatter Wine Co", "49340e5f-c7ef-41d9-9f1b-200711e6e629", "2021-07-28", "2021-07-28", "e16cbbac-c319-45f3-ac53-89d979fbcdc1", "ea4c906e-fab1-4b15-8845-619f20e53c6a", "Organic Panela 1kg", "20014", 5, 2.0, "N");
dtItems.Rows.Add("Mad Hatter Wine Co", "49340e5f-c7ef-41d9-9f1b-200711e6e629", "2021-07-28", "2021-07-28", "e16cbbac-c319-45f3-ac53-89d979fbcdc1", "bb3e1c10-9e67-46d3-99b4-17df45dead90", "Chocolate Powder 1Kg, RRP ", "20034", 6, 1.0, "N");
        
query.Rows.Add("Aussie Bites Cafe", "30389aca-9089-4b37-9a1e-5fbc3c2af485", "2019-07-01", "2019-07-01", "85df1af6-3d1e-4e04-8fe9-d90462a59d4c", "ea89ade4-c7ff-4d79-abcd-dcdbb8122562", "X Blend 1Kg, RRP ", "100112", 1, 4.0, "N");
query.Rows.Add("Aussie Bites Cafe", "30389aca-9089-4b37-9a1e-5fbc3c2af485", "2019-07-01", "2019-07-01", "85df1af6-3d1e-4e04-8fe9-d90462a59d4c", "21fe57ad-08f9-4c8b-81d0-d7b88b291571", "webfreight", "webfreight", 2, 1.0, "N");
query.Rows.Add("Mad Hatter Wine Co", "49340e5f-c7ef-41d9-9f1b-200711e6e629", "2021-07-28", "2021-07-28", "e16cbbac-c319-45f3-ac53-89d979fbcdc1", "6edc584b-a8eb-4f0b-a449-dbcb76a40a24", "Porter St 1Kg, RRP ", "100111", 1, 30.0, "N");
query.Rows.Add("Mad Hatter Wine Co", "49340e5f-c7ef-41d9-9f1b-200711e6e629", "2021-07-28", "2021-07-28", "e16cbbac-c319-45f3-ac53-89d979fbcdc1", "51e1a867-4079-4a3c-9ddc-e93d87d80b46", "P&R Cups 12oz (1000)", "30058", 2, 1.0, "N");
query.Rows.Add("Mad Hatter Wine Co", "49340e5f-c7ef-41d9-9f1b-200711e6e629", "2021-07-28", "2021-07-28", "e16cbbac-c319-45f3-ac53-89d979fbcdc1", "401ce902-e158-4f21-85a5-3312c32457fc", "Lids 06/08/12oz (White) (1000)", "30062", 3, 2.0, "N");
query.Rows.Add("Mad Hatter Wine Co", "49340e5f-c7ef-41d9-9f1b-200711e6e629", "2021-07-28", "2021-07-28", "e16cbbac-c319-45f3-ac53-89d979fbcdc1", "9b80c825-6e9f-4f6b-9c77-f3378cc220e4", "4-Cup Cardboard Holders (300)", "41003", 4, 1.0, "N");

Stopwatch pullTime = new();
pullTime.Start();

BeginInvoke(new MethodInvoker(delegate
{
    lblTimerAddRowEnd.Text = "Start Time,Except: " + pullTime.Elapsed.ToString("mm\:ss\.ff");
}));
var orderedDtItems = dtItems.AsEnumerable().OrderBy(row => Convert.ToDateTime(row.Field<String>("Date"))).ThenBy(row => row.Field<String>("Name"));
var orderedDtquery = query.AsEnumerable().OrderBy(row => Convert.ToDateTime(row.Field<String>("Date"))).ThenBy(row => row.Field<String>("Name"));
      
dtOnlyNewRows.Rows.Clear();
HashSet<String> orderedDtItemsHS = new();
HashSet<String> orderedDtqueryHS = new();
HashSet<String> orderedDtItemsHSRemains = new();
HashSet<String> orderedDtqueryHSRemains = new();
foreach (DataRow dr in orderedDtquery)
{
    orderedDtqueryHSRemains.Add(dr["CardRecordID"].ToString() + "⌁" + dr["Date"].ToString() + "⌁" + dr["SaleID"].ToString() + "⌁" + dr["ItemID"].ToString()
        + "⌁" + dr["LineNumber"].ToString() + "⌁" + dr["Quantity"].ToString());
    orderedDtqueryHS.Add(dr["CardRecordID"].ToString() + "⌁" + dr["Date"].ToString() + "⌁" + dr["SaleID"].ToString() + "⌁" + dr["ItemID"].ToString()
        + "⌁" + dr["LineNumber"].ToString() + "⌁" + dr["Quantity"].ToString());
}
foreach (DataRow dr in orderedDtItems)
{
    orderedDtItemsHSRemains.Add(dr["CardRecordID"].ToString() + "⌁" + dr["Date"].ToString() + "⌁" + dr["SaleID"].ToString() + "⌁" + dr["ItemID"].ToString()
        + "⌁" + dr["LineNumber"].ToString() + "⌁" + dr["Quantity"].ToString());
    orderedDtItemsHS.Add(dr["CardRecordID"].ToString() + "⌁" + dr["Date"].ToString() + "⌁" + dr["SaleID"].ToString() + "⌁" + dr["ItemID"].ToString()
        + "⌁" + dr["LineNumber"].ToString() + "⌁" + dr["Quantity"].ToString());
    bool added = orderedDtqueryHSRemains.Add(dr["CardRecordID"].ToString() + "⌁" + dr["Date"].ToString() + "⌁" + dr["SaleID"].ToString() + "⌁" + dr["ItemID"].ToString()
        + "⌁" + dr["LineNumber"].ToString() + "⌁" + dr["Quantity"].ToString());
    if (added == false)
    {
        orderedDtqueryHSRemains.Remove(dr["CardRecordID"].ToString() + "⌁" + dr["Date"].ToString() + "⌁" + dr["SaleID"].ToString() + "⌁" + dr["ItemID"].ToString()
        + "⌁" + dr["LineNumber"].ToString() + "⌁" + dr["Quantity"].ToString());
    }
    else if (added == true)
    {
        dtOnlyNewRows.ImportRow(dr);
        orderedDtqueryHSRemains.Remove(dr["CardRecordID"].ToString() + "⌁" + dr["Date"].ToString() + "⌁" + dr["SaleID"].ToString() + "⌁" + dr["ItemID"].ToString()
        + "⌁" + dr["LineNumber"].ToString() + "⌁" + dr["Quantity"].ToString());
    }
}

foreach (DataRow dr in orderedDtquery)
{
    bool added = orderedDtItemsHSRemains.Add(dr["CardRecordID"].ToString() + "⌁" + dr["Date"].ToString() + "⌁" + dr["SaleID"].ToString() + "⌁" + dr["ItemID"].ToString()
        + "⌁" + dr["LineNumber"].ToString() + "⌁" + dr["Quantity"].ToString());
    if (added == false)
    {
        orderedDtItemsHSRemains.Remove(dr["CardRecordID"].ToString() + "⌁" + dr["Date"].ToString() + "⌁" + dr["SaleID"].ToString() + "⌁" + dr["ItemID"].ToString()
        + "⌁" + dr["LineNumber"].ToString() + "⌁" + dr["Quantity"].ToString());
    }
    else if (added == true)
    {
        DateTime rowTime = Convert.ToDateTime(dr["date"].ToString());
        if (rowTime <= MonthCutOff)
        {
            dtOnlyLeftoverRows.ImportRow(dr);
        }
        else
        {
            dtOnlyDeleteRows.ImportRow(dr);
        }
        orderedDtItemsHSRemains.Remove(dr["CardRecordID"].ToString() + "⌁" + dr["Date"].ToString() + "⌁" + dr["SaleID"].ToString() + "⌁" + dr["ItemID"].ToString()
        + "⌁" + dr["LineNumber"].ToString() + "⌁" + dr["Quantity"].ToString());
    }
}

Debug.WriteLine(dtOnlyNewRows.Rows.Count.ToString());
BeginInvoke(new MethodInvoker(delegate
{
    labelControl1.Text = "End Time,Except: " + pullTime.Elapsed.ToString("mm\:ss\.ff");
}));
pullTime.Stop();
BeginInvoke(new MethodInvoker(delegate
{
    dgvRowsRemaing.DataSource = dtOnlyLeftoverRows;
    dgvResults.DataSource = dtOnlyNewRows;
    dgvDeleteRows.DataSource = dtOnlyDeleteRows;
    btnStart.Enabled = true;
}));

最终结果如下：

所有这些解释之后我的问题是：

我在其他方法中做错了什么，它们可以更快吗？
如果我的 janky 方法不对，我应该如何比较数据table？
只要能正常工作，即使卡顿也很快，就可以吗？
我的 janky 方法可能存在哪些问题？

编辑时间：2021-08-03 11:25 下午 AEST（澳大利亚东部标准时间）

Juris 编写的代码更简洁、更快捷，应用于我的虚拟数据时的样子 Windows Forms

3 倍更快，更少混乱的代码，更短这正是我要找的谢谢

Answer 1

我将通过使用一对字典为数据表编制索引来完成此操作。 DataTable 可以定义主键并执行在内部使用字典的快速查找，但通常使用数据表是非常难看的东西，所以没有必要向它添加更多 PK 丑陋

所以我们在右边有一些数据表，它是从数据库下载的，你已经决定“Foo”和“Bar”列是 PK。 Foo 是一个字符串，Bar 是一个整数：

Dim rIndex = new Dictionary(Of (ValueTuple(Of String, Integer), DataRow)

For Each r as DataRow In rightDt.Rows
  Dim key = ( r.Field(Of String)("Foo"), r.Field(Of Integer)("Bar") )
  rIndex(key) = r
Next r

我们有一些文件已读入左侧数据表。该文件的列恰好称为 Wit（字符串）和 Woo（int）

Dim lIndex = new Dictionary(Of (ValueTuple(Of String, Integer), DataRow)

For Each r as DataRow In leftDt.Rows
  Dim key = (r.Field(Of String)("Wit"), r.Field(Of Integer)("Woo") )
  lIndex(key) = r
Next r

现在，如果我们将密钥也存储到哈希集中，可能会让生活变得轻松；这代表左右结合

Dim allKeys as New HashSet(Of ValueTuple(Of String, Integer))

Dim rIndex = new Dictionary(Of (ValueTuple(Of String, Integer), DataRow)

For Each r as DataRow In rightDt.Rows
  Dim key = ( r.Field(Of String)("Foo"), r.Field(Of Integer)("Bar") )
  rIndex(key) = r
  allKeys.Add(key)
Next r

Dim lIndex = new Dictionary(Of (ValueTuple(Of String, Integer), DataRow)

For Each r as DataRow In leftDt.Rows
  Dim key = (r.Field(Of String)("Wit"), r.Field(Of Integer)("Woo") )
  lIndex(key) = r
  allKeys.Add(key)
Next r

剩下的就是枚举 allKeys 并询问字典是否包含它并决定做什么

For Each k in allKeys
  Dim inL = lIndex.ContainsKey(k)
  Dim inR = rIndex.ContainsKey(k)

  If inL AndAlso inR Then
    Dim updateRo = lIndex(k) 'update the db using this datarow
    ...
  ElseIf inL Then
    Dim insertRo = lIndex(k) 'insert this row to the db
    ...
  Else
    Dim deleteRo = rIndex(k) 'delete this row from the db
    ...
  End If
Next k

--

哈，我才发现我的大脑还处于VB模式。这是上面的 C# 版本：

var allKeys = new HashSet<(string, int)>();

var rIndex = new Dictionary<(string, int), DataRow>();

foreach(DataRow r in rightDt.Rows){
  var key = (r.Field<string>("Foo"), r.Field<int>("Bar"));
  rIndex[key] = r;
  allKeys.Add(key);
}

var lIndex = new Dictionary<(string, int), DataRow>();

foreach(DataRow r in leftDt.Rows){
  var key = (r.Field<string>("Wit"), r.Field<int>("Woo"));
  lIndex[key] = r;
  allKeys.Add(key);
}

foreach(var k in allKeys){
  var inL = lIndex.ContainsKey(k);
  var inR = rIndex.ContainsKey(k);

  if(inL && inR){
    var updateRo = lIndex[k]; //update the db using this datarow
    ...
  } else if(inL){
    var insertRo = lIndex[k]; //insert this row to the db
    ...
  } else {
    var deleteRo = rIndex[k]; //delete this row from the db
    ...
  }
}

您可以在 https://dotnetfiddle.net/3jfrPl

查看工作示例

您如何正确快速地比较数据行/数据表？

How do you correctly and quickly compare Datarows / Datatables?

c#

performance

linq

datatable

datarow