由于 Large-List.Contains() 上的 WHERE 导致的 EF 超时异常

EF Timeout Exception due to WHERE on a Large-List.Contains()

当我向以 LINQ 风格编写的查询添加条件过滤器(或 WHERE)时出现异常。

我得到这个异常:

问题过滤器:

if (filterCriteria.ContainsKey(isExcludeKiosk) && filterCriteria.TryGetValue(isExcludeKiosk, out var actualExcludeKiosk))
{
    if ((bool)actualExcludeKiosk)
    {
        var kioskRecordsList = await context.View_BillInfoDetailWithKiosks.Where(w => w.KioskMadeId != null).Select(s => s.KioskMadeId ?? -1).ToListAsync();

        result = result.Where(where => !kioskRecordsList.Contains(where.REFERENCE_ID));
    }
}

这个文件管理器是要取出某一个kiosk生成的bill details。没有直接关系。我在我的数据库中生成了一个视图,该视图使用 SQL 链接服务器来生成由 kiosk 生成的可能 ID 列表。问题是该列表大约有 12,000 条记录,而 bill details 也超过 500,000 条。我这样做是为了获得 EF 执行的 SQL 查询的转储,并且该查询在 Management Studio 中执行大约需要十分钟:

SELECT 
    [Project1].[PAY_MODE_ID] AS [PAY_MODE_ID], 
    [Project1].[BILL_NUMBER] AS [BILL_NUMBER], 
    [Project1].[CASHIER_ID] AS [CASHIER_ID], 
    [Project1].[SERVICE_CODE] AS [SERVICE_CODE], 
    [Project1].[REFERENCE_ID] AS [REFERENCE_ID], 
    [Project1].[ACCOUNT_CODE] AS [ACCOUNT_CODE], 
    [Project1].[SERVICE_FEE] AS [SERVICE_FEE], 
    [Project1].[REQUESTED_QUANTITY] AS [REQUESTED_QUANTITY], 
    [Project1].[CURRENT_QUANTITY] AS [CURRENT_QUANTITY], 
    [Project1].[VAT_AMOUNT] AS [VAT_AMOUNT], 
    [Project1].[KNOWLEDGE_FEE] AS [KNOWLEDGE_FEE], 
    [Project1].[KNOWLEDGE_FEE_TOTAL] AS [KNOWLEDGE_FEE_TOTAL], 
    [Project1].[INNOVATION_FEE] AS [INNOVATION_FEE], 
    [Project1].[INNOVATION_FEE_TOTAL] AS [INNOVATION_FEE_TOTAL], 
    [Project1].[ITEM_TOTAL] AS [ITEM_TOTAL], 
    [Project1].[LAST_MODIFIED_BY] AS [LAST_MODIFIED_BY], 
    [Project1].[INPUT_STATUS] AS [INPUT_STATUS], 
    [Project1].[STAMP_DATE] AS [STAMP_DATE], 
    [Project1].[BILL_NUMBER1] AS [BILL_NUMBER1], 
    [Project1].[PAY_MODE_ID1] AS [PAY_MODE_ID1], 
    [Project1].[CASHIER_ID1] AS [CASHIER_ID1], 
    [Project1].[FOCUS_BILL_NUMBER] AS [FOCUS_BILL_NUMBER], 
    [Project1].[BILL_DATE] AS [BILL_DATE], 
    [Project1].[CUSTOMER_CODE] AS [CUSTOMER_CODE], 
    [Project1].[CUSTOMER_NAME] AS [CUSTOMER_NAME], 
    [Project1].[CUSTOMER_KIND_ID] AS [CUSTOMER_KIND_ID], 
    [Project1].[BILL_AMOUNT] AS [BILL_AMOUNT], 
    [Project1].[BANK_ID] AS [BANK_ID], 
    [Project1].[BANK_BRANCH_ID] AS [BANK_BRANCH_ID], 
    [Project1].[BANK_TRANSFER_TRANSACTION_NUMBER] AS [BANK_TRANSFER_TRANSACTION_NUMBER], 
    [Project1].[CHEQUE_NUMBER] AS [CHEQUE_NUMBER], 
    [Project1].[CHEQUE_DUE_DATE] AS [CHEQUE_DUE_DATE], 
    [Project1].[REMARKS] AS [REMARKS], 
    [Project1].[DEPOSIT_ID] AS [DEPOSIT_ID], 
    [Project1].[GRP_IS_SUBMITTED] AS [GRP_IS_SUBMITTED], 
    [Project1].[GRP_BATCH_FULL_ID] AS [GRP_BATCH_FULL_ID], 
    [Project1].[GRP_BATCH_ID] AS [GRP_BATCH_ID], 
    [Project1].[GRP_BATCH_DATE] AS [GRP_BATCH_DATE], 
    [Project1].[GRP_INVOICE_NUMBER] AS [GRP_INVOICE_NUMBER], 
    [Project1].[GRP_RECEIPT_NUMBER] AS [GRP_RECEIPT_NUMBER], 
    [Project1].[GRP_DOCUMENT_NUMBER] AS [GRP_DOCUMENT_NUMBER], 
    [Project1].[REFUND_ID] AS [REFUND_ID], 
    [Project1].[BILL_TYPE_ID] AS [BILL_TYPE_ID], 
    [Project1].[REASON_ID] AS [REASON_ID], 
    [Project1].[ACCOUNT_GROUP] AS [ACCOUNT_GROUP], 
    [Project1].[ACCOUNT_CODE1] AS [ACCOUNT_CODE1], 
    [Project1].[REFERENCE_NUMBER] AS [REFERENCE_NUMBER], 
    [Project1].[CREDIT_CARD_TYPE_ID] AS [CREDIT_CARD_TYPE_ID], 
    [Project1].[CREDIT_CARD_CHARGES] AS [CREDIT_CARD_CHARGES], 
    [Project1].[CREDIT_CARD_AUTH_CODE] AS [CREDIT_CARD_AUTH_CODE], 
    [Project1].[EDIRHAM_CARD_TYPE_ID] AS [EDIRHAM_CARD_TYPE_ID], 
    [Project1].[EDIRHAM_AUTH_CODE] AS [EDIRHAM_AUTH_CODE], 
    [Project1].[PAID_CURRENCY] AS [PAID_CURRENCY], 
    [Project1].[AMOUNT_IN_WORD] AS [AMOUNT_IN_WORD], 
    [Project1].[TRANSACTION_LOCATION_ID] AS [TRANSACTION_LOCATION_ID], 
    [Project1].[DA_TRN] AS [DA_TRN], 
    [Project1].[CUSTOMER_TRN] AS [CUSTOMER_TRN], 
    [Project1].[LAST_MODIFIED_BY1] AS [LAST_MODIFIED_BY1], 
    [Project1].[INPUT_STATUS1] AS [INPUT_STATUS1], 
    [Project1].[STAMP_DATE1] AS [STAMP_DATE1], 
    [Project1].[ID] AS [ID], 
    [Project1].[NAME] AS [NAME], 
    [Project1].[REPORT_NAME] AS [REPORT_NAME], 
    [Project1].[INPUT_STATUS2] AS [INPUT_STATUS2], 
    [Project1].[SERVICE_CODE1] AS [SERVICE_CODE1], 
    [Project1].[NAME1] AS [NAME1], 
    [Project1].[FEE] AS [FEE], 
    [Project1].[ACCOUNT_CODE2] AS [ACCOUNT_CODE2], 
    [Project1].[TERMINAL] AS [TERMINAL], 
    [Project1].[DURATION] AS [DURATION], 
    [Project1].[KNOWLEDGE_FEE1] AS [KNOWLEDGE_FEE1], 
    [Project1].[INNOVATION_FEE1] AS [INNOVATION_FEE1], 
    [Project1].[IS_VAT] AS [IS_VAT], 
    [Project1].[VAT_VALUE] AS [VAT_VALUE], 
    [Project1].[VAT_TYPE] AS [VAT_TYPE], 
    [Project1].[INPUT_STATUS3] AS [INPUT_STATUS3], 
    [Project1].[STAMP_DATE2] AS [STAMP_DATE2], 
    [Project1].[GRP_MEMO_LINE] AS [GRP_MEMO_LINE], 
    [Project1].[PROCESS_TYPE_ID] AS [PROCESS_TYPE_ID]
    FROM ( SELECT 
        [Filter1].[BILL_NUMBER1] AS [BILL_NUMBER], 
        [Filter1].[PAY_MODE_ID1] AS [PAY_MODE_ID], 
        [Filter1].[CASHIER_ID1] AS [CASHIER_ID], 
        [Filter1].[SERVICE_CODE] AS [SERVICE_CODE], 
        [Filter1].[REFERENCE_ID] AS [REFERENCE_ID], 
        [Filter1].[ACCOUNT_CODE1] AS [ACCOUNT_CODE], 
        [Filter1].[SERVICE_FEE] AS [SERVICE_FEE], 
        [Filter1].[REQUESTED_QUANTITY] AS [REQUESTED_QUANTITY], 
        [Filter1].[CURRENT_QUANTITY] AS [CURRENT_QUANTITY], 
        [Filter1].[VAT_AMOUNT] AS [VAT_AMOUNT], 
        [Filter1].[KNOWLEDGE_FEE] AS [KNOWLEDGE_FEE], 
        [Filter1].[KNOWLEDGE_FEE_TOTAL] AS [KNOWLEDGE_FEE_TOTAL], 
        [Filter1].[INNOVATION_FEE] AS [INNOVATION_FEE], 
        [Filter1].[INNOVATION_FEE_TOTAL] AS [INNOVATION_FEE_TOTAL], 
        [Filter1].[ITEM_TOTAL] AS [ITEM_TOTAL], 
        [Filter1].[LAST_MODIFIED_BY1] AS [LAST_MODIFIED_BY], 
        [Filter1].[INPUT_STATUS1] AS [INPUT_STATUS], 
        [Filter1].[STAMP_DATE1] AS [STAMP_DATE], 
        [Filter1].[BILL_NUMBER2] AS [BILL_NUMBER1], 
        [Filter1].[PAY_MODE_ID2] AS [PAY_MODE_ID1], 
        [Filter1].[CASHIER_ID2] AS [CASHIER_ID1], 
        [Filter1].[FOCUS_BILL_NUMBER] AS [FOCUS_BILL_NUMBER], 
        [Filter1].[BILL_DATE] AS [BILL_DATE], 
        [Filter1].[CUSTOMER_CODE] AS [CUSTOMER_CODE], 
        [Filter1].[CUSTOMER_NAME] AS [CUSTOMER_NAME], 
        [Filter1].[CUSTOMER_KIND_ID] AS [CUSTOMER_KIND_ID], 
        [Filter1].[BILL_AMOUNT] AS [BILL_AMOUNT], 
        [Filter1].[BANK_ID] AS [BANK_ID], 
        [Filter1].[BANK_BRANCH_ID] AS [BANK_BRANCH_ID], 
        [Filter1].[BANK_TRANSFER_TRANSACTION_NUMBER] AS [BANK_TRANSFER_TRANSACTION_NUMBER], 
        [Filter1].[CHEQUE_NUMBER] AS [CHEQUE_NUMBER], 
        [Filter1].[CHEQUE_DUE_DATE] AS [CHEQUE_DUE_DATE], 
        [Filter1].[REMARKS] AS [REMARKS], 
        [Filter1].[DEPOSIT_ID] AS [DEPOSIT_ID], 
        [Filter1].[GRP_IS_SUBMITTED] AS [GRP_IS_SUBMITTED], 
        [Filter1].[GRP_BATCH_FULL_ID] AS [GRP_BATCH_FULL_ID], 
        [Filter1].[GRP_BATCH_ID] AS [GRP_BATCH_ID], 
        [Filter1].[GRP_BATCH_DATE] AS [GRP_BATCH_DATE], 
        [Filter1].[GRP_INVOICE_NUMBER] AS [GRP_INVOICE_NUMBER], 
        [Filter1].[GRP_RECEIPT_NUMBER] AS [GRP_RECEIPT_NUMBER], 
        [Filter1].[GRP_DOCUMENT_NUMBER] AS [GRP_DOCUMENT_NUMBER], 
        [Filter1].[REFUND_ID] AS [REFUND_ID], 
        [Filter1].[BILL_TYPE_ID] AS [BILL_TYPE_ID], 
        [Filter1].[REASON_ID] AS [REASON_ID], 
        [Filter1].[ACCOUNT_GROUP] AS [ACCOUNT_GROUP], 
        [Filter1].[ACCOUNT_CODE2] AS [ACCOUNT_CODE1], 
        [Filter1].[REFERENCE_NUMBER] AS [REFERENCE_NUMBER], 
        [Filter1].[CREDIT_CARD_TYPE_ID] AS [CREDIT_CARD_TYPE_ID], 
        [Filter1].[CREDIT_CARD_CHARGES] AS [CREDIT_CARD_CHARGES], 
        [Filter1].[CREDIT_CARD_AUTH_CODE] AS [CREDIT_CARD_AUTH_CODE], 
        [Filter1].[EDIRHAM_CARD_TYPE_ID] AS [EDIRHAM_CARD_TYPE_ID], 
        [Filter1].[EDIRHAM_AUTH_CODE] AS [EDIRHAM_AUTH_CODE], 
        [Filter1].[PAID_CURRENCY] AS [PAID_CURRENCY], 
        [Filter1].[AMOUNT_IN_WORD] AS [AMOUNT_IN_WORD], 
        [Filter1].[TRANSACTION_LOCATION_ID] AS [TRANSACTION_LOCATION_ID], 
        [Filter1].[DA_TRN] AS [DA_TRN], 
        [Filter1].[CUSTOMER_TRN] AS [CUSTOMER_TRN], 
        [Filter1].[LAST_MODIFIED_BY2] AS [LAST_MODIFIED_BY1], 
        [Filter1].[INPUT_STATUS2] AS [INPUT_STATUS1], 
        [Filter1].[STAMP_DATE2] AS [STAMP_DATE1], 
        [Filter1].[ID] AS [ID], 
        [Filter1].[NAME] AS [NAME], 
        [Filter1].[REPORT_NAME] AS [REPORT_NAME], 
        [Filter1].[INPUT_STATUS3] AS [INPUT_STATUS2], 
        [Extent4].[SERVICE_CODE] AS [SERVICE_CODE1], 
        [Extent4].[NAME] AS [NAME1], 
        [Extent4].[FEE] AS [FEE], 
        [Extent4].[PROCESS_TYPE_ID] AS [PROCESS_TYPE_ID], 
        [Extent4].[ACCOUNT_CODE] AS [ACCOUNT_CODE2], 
        [Extent4].[TERMINAL] AS [TERMINAL], 
        [Extent4].[DURATION] AS [DURATION], 
        [Extent4].[KNOWLEDGE_FEE] AS [KNOWLEDGE_FEE1], 
        [Extent4].[INNOVATION_FEE] AS [INNOVATION_FEE1], 
        [Extent4].[IS_VAT] AS [IS_VAT], 
        [Extent4].[VAT_VALUE] AS [VAT_VALUE], 
        [Extent4].[VAT_TYPE] AS [VAT_TYPE], 
        [Extent4].[GRP_MEMO_LINE] AS [GRP_MEMO_LINE], 
        [Extent4].[INPUT_STATUS] AS [INPUT_STATUS3], 
        [Extent4].[STAMP_DATE] AS [STAMP_DATE2]
        FROM   (SELECT [Extent1].[BILL_NUMBER] AS [BILL_NUMBER1], [Extent1].[PAY_MODE_ID] AS [PAY_MODE_ID1], [Extent1].[CASHIER_ID] AS [CASHIER_ID1], [Extent1].[SERVICE_CODE] AS [SERVICE_CODE], [Extent1].[REFERENCE_ID] AS [REFERENCE_ID], [Extent1].[ACCOUNT_CODE] AS [ACCOUNT_CODE1], [Extent1].[SERVICE_FEE] AS [SERVICE_FEE], [Extent1].[REQUESTED_QUANTITY] AS [REQUESTED_QUANTITY], [Extent1].[CURRENT_QUANTITY] AS [CURRENT_QUANTITY], [Extent1].[VAT_AMOUNT] AS [VAT_AMOUNT], [Extent1].[KNOWLEDGE_FEE] AS [KNOWLEDGE_FEE], [Extent1].[KNOWLEDGE_FEE_TOTAL] AS [KNOWLEDGE_FEE_TOTAL], [Extent1].[INNOVATION_FEE] AS [INNOVATION_FEE], [Extent1].[INNOVATION_FEE_TOTAL] AS [INNOVATION_FEE_TOTAL], [Extent1].[ITEM_TOTAL] AS [ITEM_TOTAL], [Extent1].[LAST_MODIFIED_BY] AS [LAST_MODIFIED_BY1], [Extent1].[INPUT_STATUS] AS [INPUT_STATUS1], [Extent1].[STAMP_DATE] AS [STAMP_DATE1], [Extent2].[BILL_NUMBER] AS [BILL_NUMBER2], [Extent2].[PAY_MODE_ID] AS [PAY_MODE_ID2], [Extent2].[CASHIER_ID] AS [CASHIER_ID2], [Extent2].[FOCUS_BILL_NUMBER] AS [FOCUS_BILL_NUMBER], [Extent2].[BILL_DATE] AS [BILL_DATE], [Extent2].[CUSTOMER_CODE] AS [CUSTOMER_CODE], [Extent2].[CUSTOMER_NAME] AS [CUSTOMER_NAME], [Extent2].[CUSTOMER_KIND_ID] AS [CUSTOMER_KIND_ID], [Extent2].[BILL_AMOUNT] AS [BILL_AMOUNT], [Extent2].[BANK_ID] AS [BANK_ID], [Extent2].[BANK_BRANCH_ID] AS [BANK_BRANCH_ID], [Extent2].[BANK_TRANSFER_TRANSACTION_NUMBER] AS [BANK_TRANSFER_TRANSACTION_NUMBER], [Extent2].[CHEQUE_NUMBER] AS [CHEQUE_NUMBER], [Extent2].[CHEQUE_DUE_DATE] AS [CHEQUE_DUE_DATE], [Extent2].[REMARKS] AS [REMARKS], [Extent2].[DEPOSIT_ID] AS [DEPOSIT_ID], [Extent2].[GRP_IS_SUBMITTED] AS [GRP_IS_SUBMITTED], [Extent2].[GRP_BATCH_FULL_ID] AS [GRP_BATCH_FULL_ID], [Extent2].[GRP_BATCH_ID] AS [GRP_BATCH_ID], [Extent2].[GRP_BATCH_DATE] AS [GRP_BATCH_DATE], [Extent2].[GRP_INVOICE_NUMBER] AS [GRP_INVOICE_NUMBER], [Extent2].[GRP_RECEIPT_NUMBER] AS [GRP_RECEIPT_NUMBER], [Extent2].[GRP_DOCUMENT_NUMBER] AS [GRP_DOCUMENT_NUMBER], [Extent2].[REFUND_ID] AS [REFUND_ID], [Extent2].[BILL_TYPE_ID] AS [BILL_TYPE_ID], [Extent2].[REASON_ID] AS [REASON_ID], [Extent2].[ACCOUNT_GROUP] AS [ACCOUNT_GROUP], [Extent2].[ACCOUNT_CODE] AS [ACCOUNT_CODE2], [Extent2].[REFERENCE_NUMBER] AS [REFERENCE_NUMBER], [Extent2].[CREDIT_CARD_TYPE_ID] AS [CREDIT_CARD_TYPE_ID], [Extent2].[CREDIT_CARD_CHARGES] AS [CREDIT_CARD_CHARGES], [Extent2].[CREDIT_CARD_AUTH_CODE] AS [CREDIT_CARD_AUTH_CODE], [Extent2].[EDIRHAM_CARD_TYPE_ID] AS [EDIRHAM_CARD_TYPE_ID], [Extent2].[EDIRHAM_AUTH_CODE] AS [EDIRHAM_AUTH_CODE], [Extent2].[PAID_CURRENCY] AS [PAID_CURRENCY], [Extent2].[AMOUNT_IN_WORD] AS [AMOUNT_IN_WORD], [Extent2].[TRANSACTION_LOCATION_ID] AS [TRANSACTION_LOCATION_ID], [Extent2].[DA_TRN] AS [DA_TRN], [Extent2].[CUSTOMER_TRN] AS [CUSTOMER_TRN], [Extent2].[LAST_MODIFIED_BY] AS [LAST_MODIFIED_BY2], [Extent2].[INPUT_STATUS] AS [INPUT_STATUS2], [Extent2].[STAMP_DATE] AS [STAMP_DATE2], [Extent3].[ID] AS [ID], [Extent3].[NAME] AS [NAME], [Extent3].[REPORT_NAME] AS [REPORT_NAME], [Extent3].[INPUT_STATUS] AS [INPUT_STATUS3]
            FROM   [dbo].[BILL_INFO_DETAIL] AS [Extent1]
            INNER JOIN [dbo].[BILL_INFO] AS [Extent2] ON ([Extent1].[BILL_NUMBER] = [Extent2].[BILL_NUMBER]) AND ([Extent1].[PAY_MODE_ID] = [Extent2].[PAY_MODE_ID]) AND ([Extent1].[CASHIER_ID] = [Extent2].[CASHIER_ID])
            INNER JOIN [dbo].[MASTER_PAY_MODE] AS [Extent3] ON [Extent2].[PAY_MODE_ID] = [Extent3].[ID]
            WHERE ([Extent1].[PAY_MODE_ID] IN (1, 2, 5, 10)) 
            AND ( NOT ([Extent1].[REFERENCE_ID] IN (328645, 325892, 325892, 325892, ... ~12,000 Ids))) AND ([Extent2].[DEPOSIT_ID] IS NOT NULL) AND ([Extent2].[DEPOSIT_ID] > 0) ) AS [Filter1]
        LEFT OUTER JOIN [dbo].[SERVICE_INFO] AS [Extent4] ON [Filter1].[SERVICE_CODE] = [Extent4].[SERVICE_CODE]
        WHERE ([Filter1].[INPUT_STATUS2] = 1) AND (([Filter1].[GRP_IS_SUBMITTED] = 0) OR (1 = 0))
    )  AS [Project1]
    ORDER BY [Project1].[DEPOSIT_ID] DESC


-- p__linq__0: 'True' (Type = Boolean, IsNullable = false)

-- p__linq__1: 'False' (Type = Boolean)

我执行了一个执行计划并注意到最后一个排序占用了 80%,所以我删除了它。我把它减少到 1.11 分钟。但这还有一段时间。

计划:https://www.brentozar.com/pastetheplan/?id=Bk-H9bCBB

还有优化空间吗?

@哈桑, 首先,您不应该从数据库中获取 Kiosk 列表到内存中,然后将其传递给 linq query.This 导致您的查询未优化。 尝试以所有命令在数据库中为 运行 的方式编写查询。 这样,您使用的 linq 查询将具有 EXISTS 而不是 IN 运算符。

var conditionSubQuery=context.View_BillInfoDetailWithKiosks.Where(w => w.KioskMadeId != null);
result = await result.Where(r => !conditionSubQuery.Any(c=>r.REFERENCE_ID==c.KioskMadeId)).ToListAsync();

此致, 马苏德