Azure-CLI Blob 存储删除多个 blob
Azure-CLI Blob Storage remove multiple blobs
我需要使用 Azure-CLI 删除多个 files/blobs。但是官方不支持。所以我写了一个 for 循环来使用 az 命令删除多个文件。它看起来像这样：
for i in `az storage blob list --container-name $BUCKET --output table | awk '{print }'| sed '1,2d' | sed '/^$/d'` ; do
if az storage blob delete --name $i --container-name $BUCKET --output table ; then
echo "Files inside the bucket (container) ${BUCKET} have been erased"
else
echo "Unable to erase the files inside the bucket (container) ${BUCKET}." && exit 1
fi
done
但这不能正常工作，我认为循环挂起。但是不知道为什么。
我希望有人能帮助我。提前致谢。
编辑：
感谢您的帮助。但什么也没有发生。我正在使用 Xubuntu 17.04 并全新安装了 azure cli。
如果我按 CTRL+C:
我会收到这条消息
Error in sys.excepthook:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/applicationinsights/channel/SenderBase.py", line 118, in send
File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.5/urllib/request.py", line 466, in open
response = self._open(req, data)
File "/usr/lib/python3.5/urllib/request.py", line 484, in _open
'_open', req)
File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain
result = func(*args)
File "/usr/lib/python3.5/urllib/request.py", line 1297, in https_open
context=self._context, check_hostname=self._check_hostname)
File "/usr/lib/python3.5/urllib/request.py", line 1254, in do_open
h.request(req.get_method(), req.selector, req.data, headers)
File "/usr/lib/python3.5/http/client.py", line 1107, in request
self._send_request(method, url, body, headers)
File "/usr/lib/python3.5/http/client.py", line 1152, in _send_request
self.endheaders(body)
File "/usr/lib/python3.5/http/client.py", line 1103, in endheaders
self._send_output(message_body)
File "/usr/lib/python3.5/http/client.py", line 934, in _send_output
self.send(msg)
File "/usr/lib/python3.5/http/client.py", line 877, in send
self.connect()
File "/usr/lib/python3.5/http/client.py", line 1253, in connect
super().connect()
File "/usr/lib/python3.5/http/client.py", line 849, in connect
(self.host,self.port), self.timeout, self.source_address)
File "/usr/lib/python3.5/socket.py", line 703, in create_connection
sock.connect(sa)
KeyboardInterrupt
Original exception was:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/azure/cli/core/telemetry.py", line 271, in <module>
File "/usr/lib/python3/dist-packages/azure/cli/core/telemetry.py", line 263, in upload_telemetry
File "/usr/lib/python3/dist-packages/applicationinsights/TelemetryClient.py", line 55, in flush
File "/usr/lib/python3/dist-packages/applicationinsights/channel/TelemetryChannel.py", line 71, in flush
File "/usr/lib/python3/dist-packages/applicationinsights/channel/SynchronousQueue.py", line 39, in flush
File "/usr/lib/python3/dist-packages/applicationinsights/channel/SenderBase.py", line 118, in send
File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.5/urllib/request.py", line 466, in open
response = self._open(req, data)
File "/usr/lib/python3.5/urllib/request.py", line 484, in _open
'_open', req)
File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain
result = func(*args)
File "/usr/lib/python3.5/urllib/request.py", line 1297, in https_open
context=self._context, check_hostname=self._check_hostname)
File "/usr/lib/python3.5/urllib/request.py", line 1254, in do_open
h.request(req.get_method(), req.selector, req.data, headers)
File "/usr/lib/python3.5/http/client.py", line 1107, in request
self._send_request(method, url, body, headers)
File "/usr/lib/python3.5/http/client.py", line 1152, in _send_request
self.endheaders(body)
File "/usr/lib/python3.5/http/client.py", line 1103, in endheaders
self._send_output(message_body)
File "/usr/lib/python3.5/http/client.py", line 934, in _send_output
self.send(msg)
File "/usr/lib/python3.5/http/client.py", line 877, in send
self.connect()
File "/usr/lib/python3.5/http/client.py", line 1253, in connect
super().connect()
File "/usr/lib/python3.5/http/client.py", line 849, in connect
(self.host,self.port), self.timeout, self.source_address)
File "/usr/lib/python3.5/socket.py", line 703, in create_connection
sock.connect(sa)
KeyboardInterrupt
我运行这个脚本完全没有必要。我总是得到同样的错误:-(
--阿鲁卡
在我的测试中，你的脚本对我有用，也许我们可以试试这个脚本：
export AZURE_STORAGE_ACCOUNT=jasontest321
export AZURE_STORAGE_ACCESS_KEY=+v6/ZQy74QhIwdvEMbnMV7HMaDaNUAkCDizBTOVdZkBUlw==
export container_name=vhds
for i in `az storage blob list --container-name $container_name --output table | awk '{print }'| sed '1,2d' | sed '/^$/d'` ; do
if az storage blob delete --name $i --container-name $container_name >/dev/null ; then
echo "Files $i inside the $container_name have been erased"
else
echo "Unable to erase the files $i inside the $container_name." && exit 1
fi
done
这里是关于结果的截图：
更新：
#az --version
.
role (2.0.5)
sf (1.0.2)
sql (2.0.4)
storage (2.0.7)
vm (2.0.7)
Python (Linux) 2.7.5 (default, Nov 6 2016, 00:28:07)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-11)]
Python location '/root/lib/azure-cli/bin/python'
[root@jasoncli@jasonye ~]# python
Python 2.7.5 (default, Nov 6 2016, 00:28:07)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-11)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>
在我的实验室中，我的root 帐户版本是2.7.5。我们应该为您的用户 aruka.
检查您的 python 版本
更新2：
我们可以使用 Azure 门户 cloud shell 来测试这个脚本：
更新3：
root@jasonvm:~# history
1 echo "deb [arch=amd64] https://packages.microsoft.com/repos/azure-cli/ wheezy main" | sudo tee /etc/apt/sources.list.d/azure-cli.list
2 sudo apt-key adv --keyserver packages.microsoft.com --recv-keys 417A0893
3 sudo apt-get install apt-transport-https
4 sudo apt-get update && sudo apt-get install azure-cli
5 export AZURE_STORAGE_ACCOUNT=jasondisk321
6 export AZURE_STORAGE_ACCESS_KEY=O1/xxxWRzOlWep78iRcxxx/htgxHImKxxytlsv7gnn0hg1aGvw==
7 export container_name=vhds
8 for i in `az storage blob list --container-name $container_name --output table | awk '{print }'| sed '1,2d' | sed '/^$/d'` ; do if az storage blob delete --name $i --container-name $container_name >/dev/null ; then echo "Files $i inside the $container_name have been erased"; else echo "Unable to erase the files $i inside the $container_name." && exit 1; fi; done
9 az login -u jason.ye@xxxx.onmicrosoft.com -p xxxx
10 export AZURE_STORAGE_ACCOUNT=jasondisk321
11 export AZURE_STORAGE_ACCESS_KEY=O1/2D1UxeGcOlWep78iRcR5PxxgxxxzRbWoytlsv7gnn0hg1aGvw==
12 export container_name=vhds
13 for i in `az storage blob list --container-name $container_name --output table | awk '{print }'| sed '1,2d' | sed '/^$/d'` ; do if az storage blob delete --name $i --container-name $container_name >/dev/null ; then echo "Files $i inside the $container_name have been erased"; else echo "Unable to erase the files $i inside the $container_name." && exit 1; fi; done
14 export AZURE_STORAGE_ACCOUNT=jasondisk321
15 export AZURE_STORAGE_ACCESS_KEY=O1/2D1UC0xxx/ztABmMyQFCeGcWRzOlWep78iRcR5Pt/K9yKxMT/htgxHImKqgznzRbWoytlsv7gnn0hg1aGvw==
16 export container_name=vhds
17 for i in `az storage blob list --container-name $container_name --output table | awk '{print }'| sed '1,2d' | sed '/^$/d'` ; do if az storage blob delete --name $i --container-name $container_name >/dev/null ; then echo "Files $i inside the $container_name have been erased"; else echo "Unable to erase the files $i inside the $container_name." && exit 1; fi; done
18 history
第一个命令（我自己的天蓝色）：
$ az storage blob list --container-name testbucket --output table
Name Blob Type Length Content Type Last Modified
----------- ----------- -------- -------------- -------------------------
results.csv BlockBlob 254 text/csv 2017-08-02T22:30:05+00:00
和版本：
$ az --version
azure-cli (2.0.12)
acr (2.0.9)
acs (2.0.11)
appservice (0.1.11)
batch (3.0.3)
billing (0.1.3)
cdn (0.0.6)
cloud (2.0.7)
cognitiveservices (0.1.6)
command-modules-nspkg (2.0.1)
component (2.0.6)
configure (2.0.10)
consumption (0.1.3)
container (0.1.7)
core (2.0.12)
cosmosdb (0.1.11)
dla (0.0.10)
dls (0.0.11)
feedback (2.0.6)
find (0.2.6)
interactive (0.3.7)
iot (0.1.10)
keyvault (2.0.8)
lab (0.0.9)
monitor (0.0.8)
network (2.0.11)
nspkg (3.0.1)
profile (2.0.9)
rdbms (0.0.5)
redis (0.2.7)
resource (2.0.11)
role (2.0.9)
sf (1.0.5)
sql (2.0.8)
storage (2.0.11)
vm (2.0.11)
Python (Linux) 2.7.13 (default, Jan 19 2017, 14:48:08)
[GCC 6.3.0 20170118]
Python location '/home/aruka/lib/azure-cli/bin/python'
一些提示（与删除命令挂起的问题根源无关）- 您不必使用 sed/awk 舞蹈来获取 blob 名称。有一种特定的输出格式 (tsv) 旨在为您提供更加脚本友好的输出（无需排除 headers 或其他分散注意力的内容）。结合 --query，您可以 select 您对输出的哪些字段感兴趣：
az storage blob list -c templates --account-name johanstetemplates --query "[].name" --output tsv
将为您提供给定容器中的 blob 列表。
就个人而言，如果我想并行处理许多项目并控制诸如我发出的并发请求数等事情，我偏向于使用 xargs。
az storage blob list -c templates --account-name johanstetemplates --query "[].name" --output tsv | xargs -L1 -P10 -I{} az storage blob show -c templates --account-name johanstetemplates -n {}
（我将删除命令更改为 show 以防止用户盲目 copy/paste 命令并在他们所有的 blob 都被删除时感到非常沮丧）
我需要使用 Azure-CLI 删除多个 files/blobs。但是官方不支持。所以我写了一个 for 循环来使用 az 命令删除多个文件。它看起来像这样：
for i in `az storage blob list --container-name $BUCKET --output table | awk '{print }'| sed '1,2d' | sed '/^$/d'` ; do
if az storage blob delete --name $i --container-name $BUCKET --output table ; then
echo "Files inside the bucket (container) ${BUCKET} have been erased"
else
echo "Unable to erase the files inside the bucket (container) ${BUCKET}." && exit 1
fi
done
但这不能正常工作，我认为循环挂起。但是不知道为什么。
我希望有人能帮助我。提前致谢。
编辑：
感谢您的帮助。但什么也没有发生。我正在使用 Xubuntu 17.04 并全新安装了 azure cli。
如果我按 CTRL+C:
我会收到这条消息Error in sys.excepthook:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/applicationinsights/channel/SenderBase.py", line 118, in send
File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.5/urllib/request.py", line 466, in open
response = self._open(req, data)
File "/usr/lib/python3.5/urllib/request.py", line 484, in _open
'_open', req)
File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain
result = func(*args)
File "/usr/lib/python3.5/urllib/request.py", line 1297, in https_open
context=self._context, check_hostname=self._check_hostname)
File "/usr/lib/python3.5/urllib/request.py", line 1254, in do_open
h.request(req.get_method(), req.selector, req.data, headers)
File "/usr/lib/python3.5/http/client.py", line 1107, in request
self._send_request(method, url, body, headers)
File "/usr/lib/python3.5/http/client.py", line 1152, in _send_request
self.endheaders(body)
File "/usr/lib/python3.5/http/client.py", line 1103, in endheaders
self._send_output(message_body)
File "/usr/lib/python3.5/http/client.py", line 934, in _send_output
self.send(msg)
File "/usr/lib/python3.5/http/client.py", line 877, in send
self.connect()
File "/usr/lib/python3.5/http/client.py", line 1253, in connect
super().connect()
File "/usr/lib/python3.5/http/client.py", line 849, in connect
(self.host,self.port), self.timeout, self.source_address)
File "/usr/lib/python3.5/socket.py", line 703, in create_connection
sock.connect(sa)
KeyboardInterrupt
Original exception was:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/azure/cli/core/telemetry.py", line 271, in <module>
File "/usr/lib/python3/dist-packages/azure/cli/core/telemetry.py", line 263, in upload_telemetry
File "/usr/lib/python3/dist-packages/applicationinsights/TelemetryClient.py", line 55, in flush
File "/usr/lib/python3/dist-packages/applicationinsights/channel/TelemetryChannel.py", line 71, in flush
File "/usr/lib/python3/dist-packages/applicationinsights/channel/SynchronousQueue.py", line 39, in flush
File "/usr/lib/python3/dist-packages/applicationinsights/channel/SenderBase.py", line 118, in send
File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.5/urllib/request.py", line 466, in open
response = self._open(req, data)
File "/usr/lib/python3.5/urllib/request.py", line 484, in _open
'_open', req)
File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain
result = func(*args)
File "/usr/lib/python3.5/urllib/request.py", line 1297, in https_open
context=self._context, check_hostname=self._check_hostname)
File "/usr/lib/python3.5/urllib/request.py", line 1254, in do_open
h.request(req.get_method(), req.selector, req.data, headers)
File "/usr/lib/python3.5/http/client.py", line 1107, in request
self._send_request(method, url, body, headers)
File "/usr/lib/python3.5/http/client.py", line 1152, in _send_request
self.endheaders(body)
File "/usr/lib/python3.5/http/client.py", line 1103, in endheaders
self._send_output(message_body)
File "/usr/lib/python3.5/http/client.py", line 934, in _send_output
self.send(msg)
File "/usr/lib/python3.5/http/client.py", line 877, in send
self.connect()
File "/usr/lib/python3.5/http/client.py", line 1253, in connect
super().connect()
File "/usr/lib/python3.5/http/client.py", line 849, in connect
(self.host,self.port), self.timeout, self.source_address)
File "/usr/lib/python3.5/socket.py", line 703, in create_connection
sock.connect(sa)
KeyboardInterrupt
我运行这个脚本完全没有必要。我总是得到同样的错误:-(
--阿鲁卡
在我的测试中，你的脚本对我有用，也许我们可以试试这个脚本：
export AZURE_STORAGE_ACCOUNT=jasontest321
export AZURE_STORAGE_ACCESS_KEY=+v6/ZQy74QhIwdvEMbnMV7HMaDaNUAkCDizBTOVdZkBUlw==
export container_name=vhds
for i in `az storage blob list --container-name $container_name --output table | awk '{print }'| sed '1,2d' | sed '/^$/d'` ; do
if az storage blob delete --name $i --container-name $container_name >/dev/null ; then
echo "Files $i inside the $container_name have been erased"
else
echo "Unable to erase the files $i inside the $container_name." && exit 1
fi
done
这里是关于结果的截图：
更新：
#az --version
.
role (2.0.5)
sf (1.0.2)
sql (2.0.4)
storage (2.0.7)
vm (2.0.7)
Python (Linux) 2.7.5 (default, Nov 6 2016, 00:28:07)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-11)]
Python location '/root/lib/azure-cli/bin/python'
[root@jasoncli@jasonye ~]# python
Python 2.7.5 (default, Nov 6 2016, 00:28:07)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-11)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>
在我的实验室中，我的root 帐户版本是2.7.5。我们应该为您的用户 aruka.
更新2：
我们可以使用 Azure 门户 cloud shell 来测试这个脚本：
更新3：
root@jasonvm:~# history
1 echo "deb [arch=amd64] https://packages.microsoft.com/repos/azure-cli/ wheezy main" | sudo tee /etc/apt/sources.list.d/azure-cli.list
2 sudo apt-key adv --keyserver packages.microsoft.com --recv-keys 417A0893
3 sudo apt-get install apt-transport-https
4 sudo apt-get update && sudo apt-get install azure-cli
5 export AZURE_STORAGE_ACCOUNT=jasondisk321
6 export AZURE_STORAGE_ACCESS_KEY=O1/xxxWRzOlWep78iRcxxx/htgxHImKxxytlsv7gnn0hg1aGvw==
7 export container_name=vhds
8 for i in `az storage blob list --container-name $container_name --output table | awk '{print }'| sed '1,2d' | sed '/^$/d'` ; do if az storage blob delete --name $i --container-name $container_name >/dev/null ; then echo "Files $i inside the $container_name have been erased"; else echo "Unable to erase the files $i inside the $container_name." && exit 1; fi; done
9 az login -u jason.ye@xxxx.onmicrosoft.com -p xxxx
10 export AZURE_STORAGE_ACCOUNT=jasondisk321
11 export AZURE_STORAGE_ACCESS_KEY=O1/2D1UxeGcOlWep78iRcR5PxxgxxxzRbWoytlsv7gnn0hg1aGvw==
12 export container_name=vhds
13 for i in `az storage blob list --container-name $container_name --output table | awk '{print }'| sed '1,2d' | sed '/^$/d'` ; do if az storage blob delete --name $i --container-name $container_name >/dev/null ; then echo "Files $i inside the $container_name have been erased"; else echo "Unable to erase the files $i inside the $container_name." && exit 1; fi; done
14 export AZURE_STORAGE_ACCOUNT=jasondisk321
15 export AZURE_STORAGE_ACCESS_KEY=O1/2D1UC0xxx/ztABmMyQFCeGcWRzOlWep78iRcR5Pt/K9yKxMT/htgxHImKqgznzRbWoytlsv7gnn0hg1aGvw==
16 export container_name=vhds
17 for i in `az storage blob list --container-name $container_name --output table | awk '{print }'| sed '1,2d' | sed '/^$/d'` ; do if az storage blob delete --name $i --container-name $container_name >/dev/null ; then echo "Files $i inside the $container_name have been erased"; else echo "Unable to erase the files $i inside the $container_name." && exit 1; fi; done
18 history
第一个命令（我自己的天蓝色）：
$ az storage blob list --container-name testbucket --output table
Name Blob Type Length Content Type Last Modified
----------- ----------- -------- -------------- -------------------------
results.csv BlockBlob 254 text/csv 2017-08-02T22:30:05+00:00
和版本：
$ az --version
azure-cli (2.0.12)
acr (2.0.9)
acs (2.0.11)
appservice (0.1.11)
batch (3.0.3)
billing (0.1.3)
cdn (0.0.6)
cloud (2.0.7)
cognitiveservices (0.1.6)
command-modules-nspkg (2.0.1)
component (2.0.6)
configure (2.0.10)
consumption (0.1.3)
container (0.1.7)
core (2.0.12)
cosmosdb (0.1.11)
dla (0.0.10)
dls (0.0.11)
feedback (2.0.6)
find (0.2.6)
interactive (0.3.7)
iot (0.1.10)
keyvault (2.0.8)
lab (0.0.9)
monitor (0.0.8)
network (2.0.11)
nspkg (3.0.1)
profile (2.0.9)
rdbms (0.0.5)
redis (0.2.7)
resource (2.0.11)
role (2.0.9)
sf (1.0.5)
sql (2.0.8)
storage (2.0.11)
vm (2.0.11)
Python (Linux) 2.7.13 (default, Jan 19 2017, 14:48:08)
[GCC 6.3.0 20170118]
Python location '/home/aruka/lib/azure-cli/bin/python'
一些提示（与删除命令挂起的问题根源无关）- 您不必使用 sed/awk 舞蹈来获取 blob 名称。有一种特定的输出格式 (tsv) 旨在为您提供更加脚本友好的输出（无需排除 headers 或其他分散注意力的内容）。结合 --query，您可以 select 您对输出的哪些字段感兴趣：
az storage blob list -c templates --account-name johanstetemplates --query "[].name" --output tsv
将为您提供给定容器中的 blob 列表。
就个人而言，如果我想并行处理许多项目并控制诸如我发出的并发请求数等事情，我偏向于使用 xargs。
az storage blob list -c templates --account-name johanstetemplates --query "[].name" --output tsv | xargs -L1 -P10 -I{} az storage blob show -c templates --account-name johanstetemplates -n {}
（我将删除命令更改为 show 以防止用户盲目 copy/paste 命令并在他们所有的 blob 都被删除时感到非常沮丧）