使用 PowerShell 脚本从 Azure Blob 存储读取 JSON 个文件并写回 Blob 存储中的另一个文件

Question

我有以下 JSON 文件 (product.json) 存储在 Azure Blob 存储中。是否可以编写 PowerShell 脚本从 blob 存储中读取此文件进行一些更改并写回另一个文件。我希望输出文件发生以下更改：

用“id”替换所有“_id”
删除所有“_rev”及其值。

Product.json

[
  {
    "_id": "9f4da9d6babeb9d411c896baa68c94c8",
    "_rev": "1-4259271795225df18768ab68baacc96c",
    "account_id": 692278,
    "limit": 10000,
    "products": [
      "Commodity",
      "InvestmentStock"
    ]
  },
  {
    "_id": "cc4b59f585b8556a2bedca78294a0797",
    "_rev": "1-410e479257faba0457bd9b4816c4dc95",
    "account_id": 328304,
    "limit": 10000,
    "products": [
      "Derivatives",
      "InvestmentStock",
      "CurrencyService"
    ]
  },
  {
    "_id": "d7e2a72963cff2760514ff772969ffe0",
    "_rev": "1-2ec6e2679eae13b76410c93f49c14c4a",
    "account_id": 674364,
    "limit": 10000,
    "products": [
      "InvestmentStock"
    ]
  }
]

outputfile.json应该是这样的：

[
  {
    "id": "9f4da9d6babeb9d411c896baa68c94c8",
    "account_id": 692278,
    "limit": 10000,
    "products": [
      "Commodity",
      "InvestmentStock"
    ]
  },
  {
    "id": "cc4b59f585b8556a2bedca78294a0797",
    "account_id": 328304,
    "limit": 10000,
    "products": [
      "Derivatives",
      "InvestmentStock",
      "CurrencyService"
    ]
  },
  {
    "id": "d7e2a72963cff2760514ff772969ffe0",
    "account_id": 674364,
    "limit": 10000,
    "products": [
      "InvestmentStock"
    ]
  }
]

Answer 1

这不是最优雅的方式，但应该可以解决问题：

$accountName = "account-name"
$accountKey = "account-key"
$containerName = "container-name"
$blobName = "Product.json"
$outputBlobName = "Output.json"

$ctx = New-AzStorageContext -StorageAccountName $accountName -StorageAccountKey $accountKey

#Read blob and save it to local file
Get-AzStorageBlobContent -Blob $blobName -Container $containerName -Destination "Product.json" -Context $ctx

#Read local file and get JSON object
$jsonContent = Get-Content -Raw -Path "Product.json" | ConvertFrom-Json

#Loop through json content and manipulate it
For ($i=0; $i -lt $jsonContent.Length; $i++) {
    $jsonContent[$i] | Add-Member -NotePropertyName "id" -NotePropertyValue $jsonContent[$i]._id
    $jsonContent[$i].PsObject.Properties.Remove("_id")
    $jsonContent[$i].PsObject.Properties.Remove("_rev")
}
$jsonContent

#Write content back to local disk
$jsonContent | ConvertTo-Json | Set-Content -Path $outputBlobName

#Upload into Azure Storage
$properties = @{"ContentType" = "application/json"}
Set-AzStorageBlobContent -File $outputBlobName -Container $containerName -Blob $outputBlobName -Properties $properties -Context $ctx

Answer 2

如果要将文件存入内存，可以使用DownloadText()方法将内容下载到内存中。

示例代码：

$accountName = "xxx"
$accountKey = "xxx"
$containerName = "xxx"
$blobName = "Product.json"
$outputBlobName = "Output.json"    

$context = New-AzStorageContext -StorageAccountName $accountName -StorageAccountKey $accountKey
$container_client = Get-AzStorageContainer -Name $containerName -Context $context
$source_blob_client = $container_client.CloudBlobContainer.GetBlockBlobReference($blobName)

#download the blob as text into memory
$download_file = $source_blob_client.DownloadText()

$jsonContent = $download_file | ConvertFrom-Json

#Loop through json content and manipulate it
For ($i=0; $i -lt $jsonContent.Length; $i++) {
    $jsonContent[$i] | Add-Member -NotePropertyName "id" -NotePropertyValue $jsonContent[$i]._id
    $jsonContent[$i].PsObject.Properties.Remove("_id")
    $jsonContent[$i].PsObject.Properties.Remove("_rev")
}

$replaced_json = $jsonContent | ConvertTo-Json

#upload the json file
$dest_blob_client =  $container_client.CloudBlobContainer.GetBlockBlobReference($outputBlobName)
$dest_blob_client.Properties.ContentType = "application/json"
$dest_blob_client.UploadText($replaced_json)

Write-Output("**completed**")

使用 PowerShell 脚本从 Azure Blob 存储读取 JSON 个文件并写回 Blob 存储中的另一个文件

Read JSON file from Azure Blob Storage using PowerShell script and write back to another file in blob storage

powershell

azure-powershell

azure-blob-storage