Powerquery：展开其中有记录的所有列
Powerquery: Expand all columns of that have records in them
使用 Microsoft Excel 2013 中的 Power Query，我创建了一个 table，如下所示：
// To insert this in Power Query, append a '=' before the 'Table.FromRows'
Table.FromRows(
{
{"0", "Tom", "null", "null"},
{"1", "Bob", [ name="Berlin" , street="BarStreet" ], [ name="Mary", age=25 ]},
{"2", "Jim", [ name="Hamburg", street="FooStreet" ], [ name="Marta", age=30 ]}
},
{"ID", "Name", "Address", "Wife"}
)
现在，我想使用 name 属性扩展 Address 和 Wife 列
在两个记录上。手动，我会这样做：
// To insert this in Power Query, append a '=' before the 'Table.FromRows'
let
t = Table.FromRows(
{
{"0", "Tom", "null", "null"},
{"1", "Bob", [ name="Berlin" , street="BarStreet" ], [ name="Mary", age=25 ]},
{"2", "Jim", [ name="Hamburg", street="FooStreet" ], [ name="Marta", age=30 ]}
},
{"ID", "Name", "Address", "Wife"}
),
expAddress = Table.ExpandRecordColumn(t, "Address", {"name"}, {"Address → name"}),
expWife = Table.ExpandRecordColumn(expAddress, "Wife", {"name"}, {"Wife → name"})
in
expWife
背景
只要我的数据 table 具有不同的布局，我就需要重写
询问。在幻想世界中，您可以展开所有包含 Records 的列
他们使用特定的密钥。 理想情况下，您将拥有以下库
函数：
// Returns a list with the names of the columns that match the secified type.
// Will also try to infer the type of a column if the table is untyped.
Table.ColumnsOfTypeInfer(
table as table,
listOfTypes as list
) as list
// Expands a column of records into columns with each of the values.
Table.ExpandRecordColumnByKey(
table as table,
columns as list,
key as text,
) as table
然后，我可以打电话给
// To insert this in Power Query, append a '=' before the 'Table.FromRows'
let
t = Table.FromRows(
{
{"0", "Tom", "null", "null"},
{"1", "Bob", [ name="Berlin" , street="BarStreet" ], [ name="Mary", age=25 ]},
{"2", "Jim", [ name="Hamburg", street="FooStreet" ], [ name="Marta", age=30 ]}
},
{"ID", "Name", "Address", "Wife"}
),
recordColumns = Table.ColumnsOfTypeInfer(t, {type record}),
expAll = Table.ExpandRecordColumnByKey(t, recordColumns, "name")
in
expAll
问题
- 您能否获得具有未在 table 中指定的特定类型的列的列表，也就是推断它？
- 你能使该记录扩展通用吗？
编辑：添加了带有两个空单元格的第 0 行。
（首先，感谢清晰的解释和示例数据及建议！）
1) M代码无法进行类型推断。这个限制几乎可以被认为是 "feature"，因为如果源数据以导致推断类型不同的方式发生变化，它几乎肯定会破坏您的查询。
加载未类型化数据后，应该可以快速使用 Detect Data Type 按钮为此生成 M。或者，如果您正在从 JSON 读取数据，它应该已经输入了足够多的数据。
如果您在特定情况下无法解决此问题，是否想更新您的问题？ :)
2) 只要 table 的单元格值是记录，就很有可能使记录扩展成为通用的，只是有点复杂。这会找到所有行都是 null 或记录的列，并展开 name 列。
以下是您可以添加到库中的一些简单实现：
let
t = Table.FromRows(
{
{"0", "Tom", null, null},
{"1", "Bob", [ name="Berlin" , street="BarStreet" ], [ name="Mary", age=25 ]},
{"2", "Jim", [ name="Hamburg", street="FooStreet" ], [ name="Marta", age=30 ]}
},
{"ID", "Name", "Address", "Wife"}
),
Table.ColumnsOfAllRowType = (table as table, typ as type) as list => let
ColumnNames = Table.ColumnNames(table),
ColumnsOfType = List.Select(ColumnNames, (name) =>
List.AllTrue(List.Transform(Table.Column(table, name), (cell) => Type.Is(Value.Type(cell), typ))))
in
ColumnsOfType,
Table.ExpandRecordColumnByKey = (table as table, columns as list, key as text) as table =>
List.Accumulate(columns, table, (state, columnToExpand) =>
Table.ExpandRecordColumn(state, columnToExpand, {key}, { columnToExpand & " → " & key })),
recordColumns = Table.ColumnsOfAllRowType(t, type nullable record),
expAll = Table.ExpandRecordColumnByKey(t, recordColumns, "name")
in
expAll
如果一个新的库函数可以在 M 中实现，我们不太可能将它添加到我们的标准库中，但如果您觉得缺少它，请随时在以下位置提出建议：https://ideas.powerbi.com/forums/265200-power-bi/
您可能有一个很好的理由来添加类似 Table.ReplaceTypeFromFirstRow(table as table) as table 的内容，因为使用 M 构造类型非常混乱。
抱歉来晚了一点，但我刚刚遇到了类似的挑战。我尝试使用 Chris Webb 的 ExpandAll 函数：
http://blog.crossjoin.co.uk/2014/05/21/expanding-all-columns-in-a-table-in-power-query/
... 但这只适用于 Table 类型的列，不适用于记录类型的列，但我已经设法将其破解到那个目的。我将 Chris 的函数复制为 "ExpandAllRecords" 并进行了 3 次编辑：:
- 将
each if _ is table then Table.ColumnNames(_) 替换为 each if _ is record then Record.FieldNames(_)
- 将
Table.ExpandTableColumn 替换为 Table.ExpandRecordColumn
- 将
ExpandAll 替换为 ExpandAllRecords
我尝试在一个函数中同时扩展表和记录，但我一直遇到类型错误。
无论如何，有了这个，最终的查询就是：
let
t = Table.FromRows(
{
{"1", "Tom", null, [ name="Jane", age=35 ]},
{"2", "Bob", [ name="Berlin" , street="BarStreet" ], [ name="Mary", age=25 ]},
{"3", "Jim", [ name="Hamburg", street="FooStreet" ], [ name="Marta", age=30 ]}
},
{"ID", "Name", "Address", "Wife"}
),
Output = ExpandAllRecords(t)
in
Output
编辑：
出于担心有一天伟大的片段（Chris Webb 所著，@MikeHoney 提到）有一天会消失），我将在此处复制整个代码：
let
//Define function taking two parameters - a table and an optional column number
Source = (TableToExpand as table, optional ColumnNumber as number) =>
let
//If the column number is missing, make it 0
ActualColumnNumber = if (ColumnNumber=null) then 0 else ColumnNumber,
//Find the column name relating to the column number
ColumnName = Table.ColumnNames(TableToExpand){ActualColumnNumber},
//Get a list containing all of the values in the column
ColumnContents = Table.Column(TableToExpand, ColumnName),
//Iterate over each value in the column and then
//If the value is of type table get a list of all of the columns in the table
//Then get a distinct list of all of these column names
ColumnsToExpand = List.Distinct(List.Combine(List.Transform(ColumnContents,
each if _ is table then Table.ColumnNames(_) else {}))),
//Append the original column name to the front of each of these column names
NewColumnNames = List.Transform(ColumnsToExpand, each ColumnName & "." & _),
//Is there anything to expand in this column?
CanExpandCurrentColumn = List.Count(ColumnsToExpand)>0,
//If this column can be expanded, then expand it
ExpandedTable = if CanExpandCurrentColumn
then
Table.ExpandTableColumn(TableToExpand, ColumnName,
ColumnsToExpand, NewColumnNames)
else
TableToExpand,
//If the column has been expanded then keep the column number the same, otherwise add one to it
NextColumnNumber = if CanExpandCurrentColumn then ActualColumnNumber else ActualColumnNumber+1,
//If the column number is now greater than the number of columns in the table
//Then return the table as it is
//Else call the ExpandAll function recursively with the expanded table
OutputTable = if NextColumnNumber>(Table.ColumnCount(ExpandedTable)-1)
then
ExpandedTable
else
ExpandAll(ExpandedTable, NextColumnNumber)
in
OutputTable
in
Source
You can then use this function on the XML file as follows:
let
//Load XML file
Source = Xml.Tables(File.Contents("C:\Users\Chris\Documents\PQ XML Expand All Demo.xml")),
ChangedType = Table.TransformColumnTypes(Source,{{"companyname", type text}}),
//Call the ExpandAll function to expand all columns
Output = ExpandAll(ChangedType)
in
Output
(Source and downloadable example: Chris Webb's Bi Blog, 2014-05-21)
使用 Microsoft Excel 2013 中的 Power Query，我创建了一个 table，如下所示：
// To insert this in Power Query, append a '=' before the 'Table.FromRows'
Table.FromRows(
{
{"0", "Tom", "null", "null"},
{"1", "Bob", [ name="Berlin" , street="BarStreet" ], [ name="Mary", age=25 ]},
{"2", "Jim", [ name="Hamburg", street="FooStreet" ], [ name="Marta", age=30 ]}
},
{"ID", "Name", "Address", "Wife"}
)
现在，我想使用 name 属性扩展 Address 和 Wife 列
在两个记录上。手动，我会这样做：
// To insert this in Power Query, append a '=' before the 'Table.FromRows'
let
t = Table.FromRows(
{
{"0", "Tom", "null", "null"},
{"1", "Bob", [ name="Berlin" , street="BarStreet" ], [ name="Mary", age=25 ]},
{"2", "Jim", [ name="Hamburg", street="FooStreet" ], [ name="Marta", age=30 ]}
},
{"ID", "Name", "Address", "Wife"}
),
expAddress = Table.ExpandRecordColumn(t, "Address", {"name"}, {"Address → name"}),
expWife = Table.ExpandRecordColumn(expAddress, "Wife", {"name"}, {"Wife → name"})
in
expWife
背景
只要我的数据 table 具有不同的布局，我就需要重写 询问。在幻想世界中，您可以展开所有包含 Records 的列 他们使用特定的密钥。 理想情况下，您将拥有以下库 函数：
// Returns a list with the names of the columns that match the secified type.
// Will also try to infer the type of a column if the table is untyped.
Table.ColumnsOfTypeInfer(
table as table,
listOfTypes as list
) as list
// Expands a column of records into columns with each of the values.
Table.ExpandRecordColumnByKey(
table as table,
columns as list,
key as text,
) as table
然后，我可以打电话给
// To insert this in Power Query, append a '=' before the 'Table.FromRows'
let
t = Table.FromRows(
{
{"0", "Tom", "null", "null"},
{"1", "Bob", [ name="Berlin" , street="BarStreet" ], [ name="Mary", age=25 ]},
{"2", "Jim", [ name="Hamburg", street="FooStreet" ], [ name="Marta", age=30 ]}
},
{"ID", "Name", "Address", "Wife"}
),
recordColumns = Table.ColumnsOfTypeInfer(t, {type record}),
expAll = Table.ExpandRecordColumnByKey(t, recordColumns, "name")
in
expAll
问题
- 您能否获得具有未在 table 中指定的特定类型的列的列表，也就是推断它？
- 你能使该记录扩展通用吗？
编辑：添加了带有两个空单元格的第 0 行。
（首先，感谢清晰的解释和示例数据及建议！）
1) M代码无法进行类型推断。这个限制几乎可以被认为是 "feature"，因为如果源数据以导致推断类型不同的方式发生变化，它几乎肯定会破坏您的查询。
加载未类型化数据后，应该可以快速使用 Detect Data Type 按钮为此生成 M。或者，如果您正在从 JSON 读取数据，它应该已经输入了足够多的数据。
如果您在特定情况下无法解决此问题，是否想更新您的问题？ :)
2) 只要 table 的单元格值是记录，就很有可能使记录扩展成为通用的，只是有点复杂。这会找到所有行都是 null 或记录的列，并展开 name 列。
以下是您可以添加到库中的一些简单实现：
let
t = Table.FromRows(
{
{"0", "Tom", null, null},
{"1", "Bob", [ name="Berlin" , street="BarStreet" ], [ name="Mary", age=25 ]},
{"2", "Jim", [ name="Hamburg", street="FooStreet" ], [ name="Marta", age=30 ]}
},
{"ID", "Name", "Address", "Wife"}
),
Table.ColumnsOfAllRowType = (table as table, typ as type) as list => let
ColumnNames = Table.ColumnNames(table),
ColumnsOfType = List.Select(ColumnNames, (name) =>
List.AllTrue(List.Transform(Table.Column(table, name), (cell) => Type.Is(Value.Type(cell), typ))))
in
ColumnsOfType,
Table.ExpandRecordColumnByKey = (table as table, columns as list, key as text) as table =>
List.Accumulate(columns, table, (state, columnToExpand) =>
Table.ExpandRecordColumn(state, columnToExpand, {key}, { columnToExpand & " → " & key })),
recordColumns = Table.ColumnsOfAllRowType(t, type nullable record),
expAll = Table.ExpandRecordColumnByKey(t, recordColumns, "name")
in
expAll
如果一个新的库函数可以在 M 中实现，我们不太可能将它添加到我们的标准库中，但如果您觉得缺少它，请随时在以下位置提出建议：https://ideas.powerbi.com/forums/265200-power-bi/
您可能有一个很好的理由来添加类似 Table.ReplaceTypeFromFirstRow(table as table) as table 的内容，因为使用 M 构造类型非常混乱。
抱歉来晚了一点，但我刚刚遇到了类似的挑战。我尝试使用 Chris Webb 的 ExpandAll 函数：
http://blog.crossjoin.co.uk/2014/05/21/expanding-all-columns-in-a-table-in-power-query/
... 但这只适用于 Table 类型的列，不适用于记录类型的列，但我已经设法将其破解到那个目的。我将 Chris 的函数复制为 "ExpandAllRecords" 并进行了 3 次编辑：:
- 将
each if _ is table then Table.ColumnNames(_)替换为each if _ is record then Record.FieldNames(_) - 将
Table.ExpandTableColumn替换为Table.ExpandRecordColumn - 将
ExpandAll替换为ExpandAllRecords
我尝试在一个函数中同时扩展表和记录，但我一直遇到类型错误。
无论如何，有了这个，最终的查询就是：
let
t = Table.FromRows(
{
{"1", "Tom", null, [ name="Jane", age=35 ]},
{"2", "Bob", [ name="Berlin" , street="BarStreet" ], [ name="Mary", age=25 ]},
{"3", "Jim", [ name="Hamburg", street="FooStreet" ], [ name="Marta", age=30 ]}
},
{"ID", "Name", "Address", "Wife"}
),
Output = ExpandAllRecords(t)
in
Output
编辑：
出于担心有一天伟大的片段（Chris Webb 所著，@MikeHoney 提到）有一天会消失），我将在此处复制整个代码：
let //Define function taking two parameters - a table and an optional column number Source = (TableToExpand as table, optional ColumnNumber as number) => let //If the column number is missing, make it 0 ActualColumnNumber = if (ColumnNumber=null) then 0 else ColumnNumber, //Find the column name relating to the column number ColumnName = Table.ColumnNames(TableToExpand){ActualColumnNumber}, //Get a list containing all of the values in the column ColumnContents = Table.Column(TableToExpand, ColumnName), //Iterate over each value in the column and then //If the value is of type table get a list of all of the columns in the table //Then get a distinct list of all of these column names ColumnsToExpand = List.Distinct(List.Combine(List.Transform(ColumnContents, each if _ is table then Table.ColumnNames(_) else {}))), //Append the original column name to the front of each of these column names NewColumnNames = List.Transform(ColumnsToExpand, each ColumnName & "." & _), //Is there anything to expand in this column? CanExpandCurrentColumn = List.Count(ColumnsToExpand)>0, //If this column can be expanded, then expand it ExpandedTable = if CanExpandCurrentColumn then Table.ExpandTableColumn(TableToExpand, ColumnName, ColumnsToExpand, NewColumnNames) else TableToExpand, //If the column has been expanded then keep the column number the same, otherwise add one to it NextColumnNumber = if CanExpandCurrentColumn then ActualColumnNumber else ActualColumnNumber+1, //If the column number is now greater than the number of columns in the table //Then return the table as it is //Else call the ExpandAll function recursively with the expanded table OutputTable = if NextColumnNumber>(Table.ColumnCount(ExpandedTable)-1) then ExpandedTable else ExpandAll(ExpandedTable, NextColumnNumber) in OutputTable in SourceYou can then use this function on the XML file as follows:
let //Load XML file Source = Xml.Tables(File.Contents("C:\Users\Chris\Documents\PQ XML Expand All Demo.xml")), ChangedType = Table.TransformColumnTypes(Source,{{"companyname", type text}}), //Call the ExpandAll function to expand all columns Output = ExpandAll(ChangedType) in Output(Source and downloadable example: Chris Webb's Bi Blog, 2014-05-21)