用于钻取操作和处理的退化维度的性能

Performance on Degenerated Dimension for drillthrough Action and Processing

有很多信息不适合作为度量,也没有必要的维度,所以我决定将这些数据集成到 FactTable 中以供以后的钻取操作(文档编号、文档行等信息.).所以我使用 FactTable 作为事实维度(或 Kimball 称之为退化维度)。 Fact维度创建后与Measure Group关联如下图:

FactTable/Fact 维度有 140.000.000 行,因此我决定使用 ROLAP 作为存储模式,试图避免 MOLAP 处理,但现在性能问题已转移到钻取操作中。所有其他维度都在 MOLAP 中。 分析安装在具有 98GB RAM 的 64x-Server 上,Memory\TotalMemoryLimit 设置为 70%。

我还在执行钻取操作(在 rolap 中的退化维度上)时激活了探查器,因此我得到了 SQL-Query。大量聚合和分组 - 难怪。

这种情况下如何处理性能,才能及时执行钻取Action和退化维度的处理?

更新 13.04

我在下面附上了 Profiler 中收到的查询的执行计划:

SELECT  
SUM ( [dbo_FactCdbSAP_Details].[Amount] ) AS Amount, 
SUM ( [dbo_FactCdbSAP_Details].[SharedAmount] ) AS SharedAmount,
[dbo_FactCdbSAP_Details].[Pk_id] ,
[dbo_FactCdbSAP_Details].[DocumentNo] ,
[dbo_FactCdbSAP_Details].[DocumentLine] ,
[dbo_FactCdbSAP_Details].[DocumentHeader] ,
[dbo_FactCdbSAP_Details].[DocumentType] ,
[dbo_FactCdbSAP_Details].[Reference] ,
[dbo_FactCdbSAP_Details].[DocumentDate] ,
[dbo_FactCdbSAP_Details].[EntryDate] ,
[dbo_FactCdbSAP_Details].[FiscalPeriod] ,
[dbo_FactCdbSAP_Details].[StornoDocNo] ,
[dbo_FactCdbSAP_Details].[DocumentCurrency] ,
[dbo_FactCdbSAP_Details].[CustomerNumber] ,
[dbo_FactCdbSAP_Details].[EnteredBy] ,
[dbo_FactCdbSAP_Details].[PartnerSegment] ,
[dbo_FactCdbSAP_Details].[PartnerBusinessArea] ,
[dbo_FactCdbSAP_Details].[ItemText] ,
[dbo_FactCdbSAP_Details].[ID_Date] ,
[dbo_FactCdbSAP_Details].[ID_CostCategory] ,
[dbo_FactCdbSAP_Details].[ID_CostCenter] ,
[dbo_FactCdbSAP_Details].[ID_Currency] ,
[dbo_FactCdbSAP_Details].[ID_Branch] ,
[dbo_FactCdbSAP_Details].[ID_Customer] ,
[dbo_FactCdbSAP_Details].[ID_Scenario] ,
[dbo_DimCostCategory_3].[AccountNo] ,
[dbo_DimCostCategory_3].[AccountNameDEU] ,
[dbo_DimCostCategory_3].[AccountNameEng] ,
[dbo_DimCostCategory_3].[AccountType] ,
[dbo_DimCostCategory_3].[AccountSetSAP] ,
[dbo_DimCostCenter_4].[CostCenterNo] ,
[dbo_DimCostCenter_4].[CostCenterName] ,
[dbo_DimCostCenter_4].[CostCenterAliasDEU] ,
[dbo_DimCostCenter_4].[CostCenterAliasENG] ,
[dbo_DimCurrency_5].[CurrencyCode] ,
[dbo_DimCurrency_5].[CurrencyENG] ,
[dbo_DimBranchShare_6].[Branch No] ,
[dbo_DimBranchShare_6].[Branch Name DE] ,
[dbo_DimBranchShare_6].[Branch Name TM1] ,
[dbo_DimBranchShare_6].[Branch Name ENG] ,
[dbo_DimBranchShare_6].[BranchId] ,
[dbo_DimBranchShare_6].[SharePercentage] ,
[dbo_DimBranchShare_6].[Branch Name ASL] ,
[dbo_DimBranchShare_6].[Country] ,
[dbo_DimBranchShare_6].[Currency] ,
[dbo_DimBranchShare_6].[IsSAP] ,
[dbo_DimCustomers_7].[Customer No] ,
[dbo_DimCustomers_7].[Customer Name1] ,
[dbo_DimCustomers_7].[Short Name] ,
[dbo_DimCustomers_7].[Street] ,
[dbo_DimCustomers_7].[Country] ,
[dbo_DimCustomers_7].[Postal Code] ,
[dbo_DimCustomers_7].[Telefon No] ,
[dbo_DimCustomers_7].[Fax TeletexNo] ,
[dbo_DimCustomers_7].[Attending BST] ,
[dbo_DimCustomers_7].[Key Industry Sector] ,
[dbo_DimCustomers_7].[Booking No] ,
[dbo_DimCustomers_7].[Status Inactiv] ,
[dbo_DimCustomers_7].[Company Key] ,
[dbo_DimCustomers_7].[Direct Mailing Forwarder] ,
[dbo_DimCustomers_7].[Direct Mailing BKeeping] ,
[dbo_DimCustomers_7].[Direct Mailing Sales] ,
[dbo_DimCustomers_7].[Direct Mailing Magazines] ,
[dbo_DimCustomers_7].[Customer Name2] ,
[dbo_DimCustomers_7].[Customer Name3] ,
[dbo_DimScenario_8].[ScenarioTypeENG] ,
[dbo_DimDate_2].[Quarter] ,
[dbo_DimDate_2].[Jan-Feb] ,
[dbo_DimDate_2].[Jan-Mrz] ,
[dbo_DimDate_2].[Jan-Apr] ,
[dbo_DimDate_2].[Jan-Mai] ,
[dbo_DimDate_2].[Jan-Jun] ,
[dbo_DimDate_2].[Jan-Jul] ,
[dbo_DimDate_2].[Jan-Aug] ,
[dbo_DimDate_2].[Jan-Sep] ,
[dbo_DimDate_2].[Jan-Okt] ,
[dbo_DimDate_2].[Jan-Nov] ,
[dbo_DimDate_2].[Jan-Dez] ,
[dbo_DimDate_2].[MonthName] ,
[dbo_DimDate_2].[Semester]
FROM (
      SELECT 
             [dbo].[FactCdbSAP_Details].[Pk_id],
             [dbo].[FactCdbSAP_Details].[ID_Date],
             [dbo].[FactCdbSAP_Details].[ID_Scenario],
             [dbo].[FactCdbSAP_Details].[ID_Branch],
             [dbo].[FactCdbSAP_Details].[ID_CostCategory],
             [dbo].[FactCdbSAP_Details].[ID_CostCenter],
             [dbo].[FactCdbSAP_Details].[ID_Customer],
             [dbo].[FactCdbSAP_Details].[ID_Currency],
             [dbo].[FactCdbSAP_Details].[DocumentNo],
             [dbo].[FactCdbSAP_Details].[DocumentLine],
             [dbo].[FactCdbSAP_Details].[DocumentHeader],
             [dbo].[FactCdbSAP_Details].[DocumentType],
             [dbo].[FactCdbSAP_Details].[Reference],
             [dbo].[FactCdbSAP_Details].[DocumentDate],
             [dbo].[FactCdbSAP_Details].[EntryDate],
             [dbo].[FactCdbSAP_Details].[FiscalPeriod],
             [dbo].[FactCdbSAP_Details].[StornoDocNo],
             [dbo].[FactCdbSAP_Details].[DocumentCurrency],
             [dbo].[FactCdbSAP_Details].[CustomerNumber],
             [dbo].[FactCdbSAP_Details].[EnteredBy],
             [dbo].[FactCdbSAP_Details].[PartnerSegment],
             [dbo].[FactCdbSAP_Details].[PartnerBusinessArea],
             [dbo].[FactCdbSAP_Details].[ItemText],
             [dbo].[FactCdbSAP_Details].[Amount],
             [dbo].[FactCdbSAP_Details].[SharedAmount]
        FROM [dbo].[FactCdbSAP_Details]
        WHERE 
            id_date >201509
            )  AS [dbo_FactCdbSAP_Details], 
                [dbo].[DimCostCategory] AS [dbo_DimCostCategory_3],
                [dbo].[DimCostCenter] AS [dbo_DimCostCenter_4],
                [dbo].[DimCurrency] AS [dbo_DimCurrency_5],
                [dbo].[DimBranchShare] AS [dbo_DimBranchShare_6],
                [dbo].[DimCustomers] AS [dbo_DimCustomers_7],
                [dbo].[DimScenario] AS [dbo_DimScenario_8],
                [dbo].[DimDate] AS [dbo_DimDate_2]

WHERE 

            [dbo_FactCdbSAP_Details].[ID_Date] = [dbo_DimDate_2].[ID_Date]
            AND     
            [dbo_FactCdbSAP_Details].[ID_CostCategory]  = [dbo_DimCostCategory_3].[PK_Cost]
            AND     
            [dbo_FactCdbSAP_Details].[ID_CostCenter] = [dbo_DimCostCenter_4].[Pk_CostCenter]
            AND     
            [dbo_FactCdbSAP_Details].[ID_Currency] = [dbo_DimCurrency_5].[Pk_Currency]
            AND     
            [dbo_FactCdbSAP_Details].[ID_Branch] =  [dbo_DimBranchShare_6].[PK_ShareBranch]
            AND     
            [dbo_FactCdbSAP_Details].[ID_Customer] = [dbo_DimCustomers_7].[Pk_Customer]
            AND     
            [dbo_FactCdbSAP_Details].[ID_Scenario] = [dbo_DimScenario_8].[Pk_Scenario]
            AND     
            [dbo_DimCurrency_5].[CurrencyDEU] = 'Lokale Währung'
            AND     
            [dbo_DimScenario_8].[ScenarioTypeDEU] = 'Ist'       
            AND     
            [dbo_DimDate_2].[Year] = 2016   
            AND     
            [dbo_DimDate_2].[Month] = 2
group by 
....

为了获得良好的钻取操作和处理性能,找到并实施了以下解决方案:

  • 我在MOLAP中更改了退化维度的存储方式
  • AttributeHierarchyOptimizedState=FullyOptimized对于退化维度的所有属性
  • AttributeHierarchyOrdered=false为退化维度的主键
  • 我实施了进程添加。创建了 Delta Table Today-Yesterday 以找出 ProcessAdd 的 suitable 数据(在这种情况下,旧数据不会更改)
  • 使用 DataFlowTask 创建了一个 SSIS 包。在 DataFlowTask 内部,delta table 被设置为 OLEDB Source,Dimension Processing Task 被设置为 Destination(这意味着直接在 MOLAP Dimension 中增量添加)。下图显示:

  • 最后将只处理受影响的多维数据集分区,也使用 ProcessAdd 选项。

非常感谢 Greg Galloway 在此 post http://www.artisconsulting.com/blogs/greggalloway/2007/4/20/processadd-on-large-dimensions

上描述 ProcessAdd ond 大维度