SDK 调用的 Google API 个端点列表

Question

我们之前通过 grep 源代码库生成了 SDK 使用的 Google 的 API 端点列表。既然那个似乎不可用，有没有其他人找到了获得这样一个列表的方法？我们需要能够在公司 firewall/proxy 上将这些端点列入白名单。

谢谢！

Answer 1

第 1 部分

如果您的 objective 是为您的防火墙将 URL 列入白名单，那么 URL *.googleapis.com 将覆盖您需要的 99%。只剩下几个端点：

bookstore.endpoints.endpoints-portal-demo.cloud.goog
cloudvolumesgcp-api.netapp.com
echo-api.endpoints.endpoints-portal-demo.cloud.goog
elasticsearch-service.gcpmarketplace.elastic.co
gcp.redisenterprise.com
payg-prod.gcpmarketplace.confluent.cloud
prod.cloud.datastax.com

第 2 部分

使用此命令列出项目的 Google API 个端点：available：

gcloud services list --available --format json | jq -r ".[].config.name"

https://cloud.google.com/sdk/gcloud/reference/services/list

有关生成类似列表的 PowerShell 脚本，请参阅第 5 部分。

第 3 部分

处理提供机器可读信息的Discovery Document：

Google API Discovery Service

curl https://www.googleapis.com/discovery/v1/apis | jq -r ".items[].discoveryRestUrl"

获得发现文档列表后，处理每个文档并提取 rootUrl 键。

curl https://youtubereporting.googleapis.com/$discovery/rest?version=v1 | jq -r ".rootUrl"

第 4 部分

用于处理发现文档并生成 API 端点列表的 PowerShell 脚本：

将此代码复制到名为 list_google_apis.ps1 的文件中。运行命令如下：

powershell ".\list_google_apis.ps1 | Sort-Object -Unique | Out-File -Encoding ASCII -FilePath apilist.txt"

将显示一些错误，因为某些发现文档 URL 会导致 404（未找到）错误。

$url_discovery = "https://www.googleapis.com/discovery/v1/apis"

$params = @{
    Uri = $url_discovery
    ContentType = 'application/json'
}

$r = Invoke-RestMethod @params

foreach($item in $r.items) {
    $url = $item.discoveryRestUrl

    try {
        $p = @{
            Uri = $url
            ContentType = 'application/json'
        }

        $doc = Invoke-RestMethod @p

        $doc.rootUrl
    } catch {
        Write-Host "Failed:" $url -ForegroundColor Red
    }
}

第 5 部分

我不久前写的 PowerShell 脚本，它产生与 gcloud services list 类似的输出。

API 的文档：

https://cloud.google.com/service-usage/docs/reference/rest/v1/services/list

<#
.SYNOPSIS
    This program displays a list of Google Cloud services

.DESCRIPTION
    Google Service Management allows service producers to publish their services on
    Google Cloud Platform so that they can be discovered and used by service consumers.

.NOTES
    This program requires the Google Cloud SDK CLI is installed and set up.
    https://cloud.google.com/sdk/docs/quickstarts

.LINK
    PowerShell Invoke-RestMethod
    https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.utility/invoke-restmethod?view=powershell-5.1
    Google Cloud CLI print-access-token Documentation
    https://cloud.google.com/sdk/gcloud/reference/auth/print-access-token
    Google Cloud API Documentation
    https://cloud.google.com/service-infrastructure/docs/service-management/reference/rest
    https://cloud.google.com/service-usage/docs/reference/rest/v1/services
    https://cloud.google.com/service-infrastructure/docs/service-management/reference/rest/v1/services/list
#>

function Get-AccessToken {
    # Get an OAuth Access Token
    $accessToken=gcloud auth print-access-token

    return $accessToken
}

function Display-ServiceTable {
    Param([array][Parameter(Position = 0, Mandatory = $true)] $serviceList)

    if ($serviceList.Count -lt 1) {
        Write-Output "No services were found"
        return
    }

    # Display as a table
    $serviceList.serviceConfig | Select name, title | Format-Table -Wrap | more
}

function Get-ServiceList {
    Param([string][Parameter(Position = 0, Mandatory = $true)] $accessToken)

    # Build the url
    # https://cloud.google.com/service-infrastructure/docs/service-management/reference/rest/v1/services/list
    $url="https://servicemanagement.googleapis.com/v1/services"

    # Build the Invoke-RestMethod parameters
    # https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.utility/invoke-restmethod?view=powershell-5.1
    $params = @{
        Headers = @{
            Authorization = "Bearer " + $accessToken
        }
        Method = 'Get'
        ContentType = "application/json"
    }

    # Create an array to store the API output which is an array of services
    $services = @()

    # Google APIs page the output
    $nextPageToken = $null

    do {
        if ($nextPageToken -eq $null)
        {
            $uri = $url
        } else {
            $uri = $url + "?pageToken=$nextPageToken"
        }

        try {
            # Get the list of services
            $output = Invoke-RestMethod @params -Uri $uri
        } catch {
            Write-Host "Error: REST API failed." -ForegroundColor Red
            Write-Host "URL: $url" -ForegroundColor Red
            Write-Host $_.Exception.Message -ForegroundColor Red

            return $services
        }

        # Debug: Display as JSON
        # $output | ConvertTo-Json

        # Append services to list
        $services += $output.services

        $nextPageToken = $output.nextPageToken
    } while ($nextPageToken -ne $null)

    return $services
}

############################################################
# Main Program
############################################################

$accessToken = Get-AccessToken

$serviceList = Get-ServiceList $accessToken

Display-ServiceTable $serviceList

Command-line tool JQ

SDK 调用的 Google API 个端点列表

List of Google API endpoints call by the SDK

google-api

google-cloud-platform

google-cloud-sdk