Google oauth：在 Power Query 中刷新令牌
Google oauth: Refresh tokens in Power Query
我正在尝试使用 oauth 方法和 power bi 连接到 youtube/google 分析。
我已经完成了一半，我需要一些帮助。这是我所在的位置：
我使用手动获取授权令牌：
https://accounts.google.com/o/oauth2/auth?scope=https://www.googleapis.com/auth/yt-analytics.readonly&response_type=code&access_type=offline&redirect_uri=urn:ietf:wg:oauth:2.0:oob&approval_prompt=force&client_id={clientid}
一旦我有了它，我就把它放在我的查询中，我能够同时获得 access_token 和 refresh_token:
现在，如果我正确理解文档，当 access_token 一小时后过期时，我可以使用我得到的 refresh_token 自动创建一个新的 access_token，
这可以在 Power Query 中完成吗？有人试过吗？
我完全不知道该怎么做，而且我不是开发人员，所以我的技能有限:(
感谢任何帮助！
我不知道 power bia 但如果你可以发送 HTTP POST 你应该能够使用刷新令牌获得新的访问令牌
POST https://accounts.google.com/o/oauth2/token
client_id={ClientId}&client_secret={ClientSecret}&refresh_token={RefreshToken}&grant_type=refresh_token
响应应该是这样的
{
"access_token" : "ya29.1.AADtN_XK16As2ZHlScqOxGtntIlevNcasMSPwGiE3pe5ANZfrmJTcsI3ZtAjv4sDrPDRnQ",
"token_type" : "Bearer",
"expires_in" : 3600
}
弄清楚如何自动执行 OAuth 流程对我们 Power BI 开发人员团队来说也不容易 ;)
PowerBI Desktop has a built-in data source connector GoogleAnalytics.Accounts() 自动处理 OAuth 令牌。
（Google Power Query 今天不提供分析功能，抱歉。）
对于 YouTube 分析，PowerBI UserVoice 此功能有线程跟踪需求。在那里表达你的支持！
我们有类似的需要直接连接到 Analytics API，以规避内置连接器的缺点。让 Web 版本的 PowerBI 接受身份验证端点作为 "anonymous" 源有点尴尬，但是反向代理可以通过使用 [=20= 响应 'probe' GET 请求来欺骗它].
这是主要的 PowerQuery / M 逻辑，分解为函数：
GetAccessToken_GA
let
Source = (optional nonce as text) as text => let
// use `nonce` to force a fresh fetch
someNonce = if nonce = null or nonce = ""
then "nonce"
else nonce,
// Reverse proxy required to trick PowerBI Cloud into allowing its malformed "anonymous" requests to return `200 OK`.
// We can skip this and connect directly to GA, but then the Web version will not be able to refresh.
url = "https://obfuscated.herokuapp.com/oauth2/v4/token",
GetJson = Web.Contents(url,
[
Headers = [
#"Content-Type"="application/x-www-form-urlencoded"
],
Content = Text.ToBinary(
// "code=" & #"Google API - Auth Code"
// "&redirect_uri=urn:ietf:wg:oauth:2.0:oob"
"refresh_token=" & #"Google API - Refresh Token"
& "&client_id=" & #"Google API - Client ID"
& "&client_secret=" & #"Google API - Client Secret"
// & "&scope=https://www.googleapis.com/auth/analytics.readonly"
& "&grant_type=refresh_token"
& "&nonce=" & someNonce
)
]
),
FormatAsJson = Json.Document(GetJson),
// Gets token from the Json response
AccessToken = FormatAsJson[access_token],
AccessTokenHeader = "Bearer " & AccessToken
in
AccessTokenHeader
in
Source
returnAccessHeaders_GA
随机数未被 GA API 使用，我在这里使用它来允许 Power BI 最多缓存 API 请求一分钟。
let
returnAccessHeaders = () as text => let
nonce = DateTime.ToText(DateTime.LocalNow(), "yyyyMMddhhmm"),
AccessTokenHeader = GetAccessToken_GA(nonce)
in
AccessTokenHeader
in
returnAccessHeaders
parseJsonResponse_GA
let
fetcher = (jsonResponse as binary) as table => let
FormatAsJsonQuery = Json.Document(jsonResponse),
columnHeadersGA = FormatAsJsonQuery[columnHeaders],
listRows = Record.FieldOrDefault(
FormatAsJsonQuery,
"rows",
{List.Transform(columnHeadersGA, each null)}
// a list of (lists of length exactly matching the # of columns) of null
),
columnNames = List.Transform(columnHeadersGA, each Record.Field(_, "name")),
matchTypes = (column as record) as list => let
values = {
{ "STRING", type text },
{ "FLOAT", type number },
{ "INTEGER", Int64.Type },
{ "TIME", type number },
{ "PERCENT", type number },
{ column[dataType], type text } // default type
},
columnType = List.First(
List.Select(
values,
each _{0} = column[dataType]
)
){1},
namedColumnType = { column[name], columnType }
in namedColumnType,
recordRows = List.Transform(
listRows,
each Record.FromList(_, columnNames)
),
columnTypes = List.Transform(columnHeadersGA, each matchTypes(_)),
rowsTable = Table.FromRecords(recordRows),
typedRowsTable = Table.TransformColumnTypes(rowsTable, columnTypes)
in typedRowsTable
in fetcher
fetchAndParseGA
Web.Contents() 的第一个参数必须是字符串文字，否则悲伤随之而来。
let
AccessTokenHeader = returnAccessHeaders_GA(),
fetchAndParseGA_fn = (url as text) as table => let
JsonQuery = Web.Contents(
"https://gapis-powerbi-revproxy.herokuapp.com/analytics",
[
RelativePath = url,
Headers = [
#"Authorization" = AccessTokenHeader
]
]
),
Response = parseJsonResponse_GA(JsonQuery)
in
Response
in
fetchAndParseGA_fn
queryUrlHelper
允许我们使用 Power BI 的 'Step Editor' UI 调整查询参数，自动 URL 编码。
let
safeString = (s as nullable text) as text => let
result = if s = null
then ""
else s
in
result,
uriEncode = (s as nullable text) as text => let
result = Uri.EscapeDataString(safeString(s))
in
result,
optionalParam = (name as text, s as nullable text) => let
result = if s = null or s = ""
then ""
else "&" & name & "=" & uriEncode(s)
in
result,
queryUrlHelper = (
gaID as text,
startDate as text,
endDate as text,
metrics as text,
dimensions as nullable text,
sort as nullable text,
filters as nullable text,
segment as nullable text,
otherParameters as nullable text
) as text => let
result = "/v3/data/ga?ids=" & uriEncode(gaID)
& "&start-date=" & uriEncode(startDate)
& "&end-date=" & uriEncode(endDate)
& "&metrics=" & uriEncode(metrics)
& optionalParam("dimensions", dimensions)
& optionalParam("sort", sort)
& optionalParam("filters", filters)
& optionalParam("segment", segment)
& safeString(otherParameters)
in
result,
Example = queryUrlHelper(
"ga:59361446", // gaID
"MONTHSTART", // startDate
"MONTHEND", // endDate
"ga:sessions,ga:pageviews", // metrics
"ga:userGender", // dimensions
"-ga:sessions", // sort
null, // filters
"gaid::BD_Im9YKTJeO9xDxV4w6Kw", // segment
null // otherParameters (must be manually url-encoded, and start with "&")
)
in
queryUrlHelper
getLinkForQueryExplorer
只是为了方便，在 the Query Explorer.
中打开一个查询
let
getLinkForQueryExplorer = (querySuffixUrl as text) as text => let
// querySuffixUrl should start like `/v3/data/ga?ids=ga:132248814&...`
link = Text.Replace(
querySuffixUrl,
"/v3/data/ga",
"https://ga-dev-tools.appspot.com/query-explorer/"
)
in
link
in
getLinkForQueryExplorer
Identity
Returns其输入不变；这个函数的主要用途是通过方便的 'Step Editor' UI.
允许另一种方式 update query variables
let
Identity = (x as any) as any => let
x = x
in
x
in
Identity
getMonthBoundary
// Get a list of the start and end dates of the relative month, as ISO 8601 formatted dates.
//
// The end date of the current month is considered to be the current date.
//
// E.g.:
// ```
// {
// "2016-09-01",
// "2016-09-31"
// }
// ```
//
// Source: <https://gist.github.com/r-k-b/db1eb0e00364cb592e1d8674bb03cb5c>
let
GetMonthDates = (monthOffset as number) as list => let
now = DateTime.LocalNow(),
otherMonth = Date.AddMonths(now, monthOffset),
month1Start = Date.StartOfMonth(otherMonth),
month1End = Date.AddDays(Date.EndOfMonth(otherMonth), -1),
dates = {
month1Start,
month1End
},
result = List.Transform(
dates,
each DateTime.ToText(_, "yyyy-MM-dd")
)
in
result
in
GetMonthDates
replaceUrlDates
//
// E.g., on 2016-10-19 this is the result:
// ```
// replaceDates(-1, "/foo?s=MONTHSTART&e=MONTHEND") === "/foo?s=2016-09-01&e=2016-09-28"
// ```
let
replaceDates = (monthOffset as number, rawUrl as text) as text => let
boundaryList = getMonthBoundary(monthOffset),
stage01 = Text.Replace(
rawUrl,
"MONTHSTART",
boundaryList{0}
),
stage02 = Text.Replace(
stage01,
"MONTHEND",
boundaryList{1}
),
stage03 = replaceViewNames(stage02)
in
stage03
in
replaceDates
示例查询
let
QueryBase = queryUrlHelper("All Web Site Data", "MONTHSTART", "today", "ga:sessions,ga:pageviews,ga:pageviewsPerSession", "ga:deviceCategory,ga:yearMonth", null, null, null, null),
MonthOffset = Identity(#"Months Back to Query"),
QueryURL = replaceUrlDates(MonthOffset, QueryBase),
CopyableLinkToQueryExplorer = getLinkForQueryExplorer(QueryURL),
Source = fetchAndParseGA(QueryURL)
in
Source
作为奖励，这可以推广到任何 OAuthV2 数据源，并且还需要进行最少的调整才能与 the powerful V4 API.
一起使用
我正在尝试使用 oauth 方法和 power bi 连接到 youtube/google 分析。 我已经完成了一半，我需要一些帮助。这是我所在的位置：
我使用手动获取授权令牌：
https://accounts.google.com/o/oauth2/auth?scope=https://www.googleapis.com/auth/yt-analytics.readonly&response_type=code&access_type=offline&redirect_uri=urn:ietf:wg:oauth:2.0:oob&approval_prompt=force&client_id={clientid}
一旦我有了它，我就把它放在我的查询中，我能够同时获得 access_token 和 refresh_token:
现在，如果我正确理解文档，当 access_token 一小时后过期时，我可以使用我得到的 refresh_token 自动创建一个新的 access_token，
这可以在 Power Query 中完成吗？有人试过吗？
我完全不知道该怎么做，而且我不是开发人员，所以我的技能有限:(
感谢任何帮助！
我不知道 power bia 但如果你可以发送 HTTP POST 你应该能够使用刷新令牌获得新的访问令牌
POST https://accounts.google.com/o/oauth2/token
client_id={ClientId}&client_secret={ClientSecret}&refresh_token={RefreshToken}&grant_type=refresh_token
响应应该是这样的
{
"access_token" : "ya29.1.AADtN_XK16As2ZHlScqOxGtntIlevNcasMSPwGiE3pe5ANZfrmJTcsI3ZtAjv4sDrPDRnQ",
"token_type" : "Bearer",
"expires_in" : 3600
}
弄清楚如何自动执行 OAuth 流程对我们 Power BI 开发人员团队来说也不容易 ;)
PowerBI Desktop has a built-in data source connector GoogleAnalytics.Accounts() 自动处理 OAuth 令牌。
（Google Power Query 今天不提供分析功能，抱歉。）
对于 YouTube 分析，PowerBI UserVoice 此功能有线程跟踪需求。在那里表达你的支持！
我们有类似的需要直接连接到 Analytics API，以规避内置连接器的缺点。让 Web 版本的 PowerBI 接受身份验证端点作为 "anonymous" 源有点尴尬，但是反向代理可以通过使用 [=20= 响应 'probe' GET 请求来欺骗它]. 这是主要的 PowerQuery / M 逻辑，分解为函数：
GetAccessToken_GA
let
Source = (optional nonce as text) as text => let
// use `nonce` to force a fresh fetch
someNonce = if nonce = null or nonce = ""
then "nonce"
else nonce,
// Reverse proxy required to trick PowerBI Cloud into allowing its malformed "anonymous" requests to return `200 OK`.
// We can skip this and connect directly to GA, but then the Web version will not be able to refresh.
url = "https://obfuscated.herokuapp.com/oauth2/v4/token",
GetJson = Web.Contents(url,
[
Headers = [
#"Content-Type"="application/x-www-form-urlencoded"
],
Content = Text.ToBinary(
// "code=" & #"Google API - Auth Code"
// "&redirect_uri=urn:ietf:wg:oauth:2.0:oob"
"refresh_token=" & #"Google API - Refresh Token"
& "&client_id=" & #"Google API - Client ID"
& "&client_secret=" & #"Google API - Client Secret"
// & "&scope=https://www.googleapis.com/auth/analytics.readonly"
& "&grant_type=refresh_token"
& "&nonce=" & someNonce
)
]
),
FormatAsJson = Json.Document(GetJson),
// Gets token from the Json response
AccessToken = FormatAsJson[access_token],
AccessTokenHeader = "Bearer " & AccessToken
in
AccessTokenHeader
in
Source
returnAccessHeaders_GA
随机数未被 GA API 使用，我在这里使用它来允许 Power BI 最多缓存 API 请求一分钟。
let
returnAccessHeaders = () as text => let
nonce = DateTime.ToText(DateTime.LocalNow(), "yyyyMMddhhmm"),
AccessTokenHeader = GetAccessToken_GA(nonce)
in
AccessTokenHeader
in
returnAccessHeaders
parseJsonResponse_GA
let
fetcher = (jsonResponse as binary) as table => let
FormatAsJsonQuery = Json.Document(jsonResponse),
columnHeadersGA = FormatAsJsonQuery[columnHeaders],
listRows = Record.FieldOrDefault(
FormatAsJsonQuery,
"rows",
{List.Transform(columnHeadersGA, each null)}
// a list of (lists of length exactly matching the # of columns) of null
),
columnNames = List.Transform(columnHeadersGA, each Record.Field(_, "name")),
matchTypes = (column as record) as list => let
values = {
{ "STRING", type text },
{ "FLOAT", type number },
{ "INTEGER", Int64.Type },
{ "TIME", type number },
{ "PERCENT", type number },
{ column[dataType], type text } // default type
},
columnType = List.First(
List.Select(
values,
each _{0} = column[dataType]
)
){1},
namedColumnType = { column[name], columnType }
in namedColumnType,
recordRows = List.Transform(
listRows,
each Record.FromList(_, columnNames)
),
columnTypes = List.Transform(columnHeadersGA, each matchTypes(_)),
rowsTable = Table.FromRecords(recordRows),
typedRowsTable = Table.TransformColumnTypes(rowsTable, columnTypes)
in typedRowsTable
in fetcher
fetchAndParseGA
Web.Contents() 的第一个参数必须是字符串文字，否则悲伤随之而来。
let
AccessTokenHeader = returnAccessHeaders_GA(),
fetchAndParseGA_fn = (url as text) as table => let
JsonQuery = Web.Contents(
"https://gapis-powerbi-revproxy.herokuapp.com/analytics",
[
RelativePath = url,
Headers = [
#"Authorization" = AccessTokenHeader
]
]
),
Response = parseJsonResponse_GA(JsonQuery)
in
Response
in
fetchAndParseGA_fn
queryUrlHelper
允许我们使用 Power BI 的 'Step Editor' UI 调整查询参数，自动 URL 编码。
let
safeString = (s as nullable text) as text => let
result = if s = null
then ""
else s
in
result,
uriEncode = (s as nullable text) as text => let
result = Uri.EscapeDataString(safeString(s))
in
result,
optionalParam = (name as text, s as nullable text) => let
result = if s = null or s = ""
then ""
else "&" & name & "=" & uriEncode(s)
in
result,
queryUrlHelper = (
gaID as text,
startDate as text,
endDate as text,
metrics as text,
dimensions as nullable text,
sort as nullable text,
filters as nullable text,
segment as nullable text,
otherParameters as nullable text
) as text => let
result = "/v3/data/ga?ids=" & uriEncode(gaID)
& "&start-date=" & uriEncode(startDate)
& "&end-date=" & uriEncode(endDate)
& "&metrics=" & uriEncode(metrics)
& optionalParam("dimensions", dimensions)
& optionalParam("sort", sort)
& optionalParam("filters", filters)
& optionalParam("segment", segment)
& safeString(otherParameters)
in
result,
Example = queryUrlHelper(
"ga:59361446", // gaID
"MONTHSTART", // startDate
"MONTHEND", // endDate
"ga:sessions,ga:pageviews", // metrics
"ga:userGender", // dimensions
"-ga:sessions", // sort
null, // filters
"gaid::BD_Im9YKTJeO9xDxV4w6Kw", // segment
null // otherParameters (must be manually url-encoded, and start with "&")
)
in
queryUrlHelper
getLinkForQueryExplorer
只是为了方便，在 the Query Explorer.
中打开一个查询let
getLinkForQueryExplorer = (querySuffixUrl as text) as text => let
// querySuffixUrl should start like `/v3/data/ga?ids=ga:132248814&...`
link = Text.Replace(
querySuffixUrl,
"/v3/data/ga",
"https://ga-dev-tools.appspot.com/query-explorer/"
)
in
link
in
getLinkForQueryExplorer
Identity
Returns其输入不变；这个函数的主要用途是通过方便的 'Step Editor' UI.
允许另一种方式 update query variableslet
Identity = (x as any) as any => let
x = x
in
x
in
Identity
getMonthBoundary
// Get a list of the start and end dates of the relative month, as ISO 8601 formatted dates.
//
// The end date of the current month is considered to be the current date.
//
// E.g.:
// ```
// {
// "2016-09-01",
// "2016-09-31"
// }
// ```
//
// Source: <https://gist.github.com/r-k-b/db1eb0e00364cb592e1d8674bb03cb5c>
let
GetMonthDates = (monthOffset as number) as list => let
now = DateTime.LocalNow(),
otherMonth = Date.AddMonths(now, monthOffset),
month1Start = Date.StartOfMonth(otherMonth),
month1End = Date.AddDays(Date.EndOfMonth(otherMonth), -1),
dates = {
month1Start,
month1End
},
result = List.Transform(
dates,
each DateTime.ToText(_, "yyyy-MM-dd")
)
in
result
in
GetMonthDates
replaceUrlDates
//
// E.g., on 2016-10-19 this is the result:
// ```
// replaceDates(-1, "/foo?s=MONTHSTART&e=MONTHEND") === "/foo?s=2016-09-01&e=2016-09-28"
// ```
let
replaceDates = (monthOffset as number, rawUrl as text) as text => let
boundaryList = getMonthBoundary(monthOffset),
stage01 = Text.Replace(
rawUrl,
"MONTHSTART",
boundaryList{0}
),
stage02 = Text.Replace(
stage01,
"MONTHEND",
boundaryList{1}
),
stage03 = replaceViewNames(stage02)
in
stage03
in
replaceDates
示例查询
let
QueryBase = queryUrlHelper("All Web Site Data", "MONTHSTART", "today", "ga:sessions,ga:pageviews,ga:pageviewsPerSession", "ga:deviceCategory,ga:yearMonth", null, null, null, null),
MonthOffset = Identity(#"Months Back to Query"),
QueryURL = replaceUrlDates(MonthOffset, QueryBase),
CopyableLinkToQueryExplorer = getLinkForQueryExplorer(QueryURL),
Source = fetchAndParseGA(QueryURL)
in
Source
作为奖励，这可以推广到任何 OAuthV2 数据源，并且还需要进行最少的调整才能与 the powerful V4 API.
一起使用