curl 请求后压缩文件损坏

Question

我正在使用 Qualtrics API 按预定时间间隔检索调查数据。下面是我的 shell 脚本 (bash) get_responses.sh，其中 post 导出、测量下载完成率、获取导出和 stores/unzips 文件。

STARTDATE=$(date -v-7d "+%Y-%m-%d")
STARTDATESTRING=$STARTDATE"T00:00:00-07:00:00"
ENDDATE=$(date "+%Y-%m-%d")
ENDDATESTRING=$ENDDATE"T00:00:00-07:00:00"

result=$(curl -X POST -H 'X-API-TOKEN: MYAPITOKEN' -H 'Content-Type: application/json' -d '{
    "surveyId": "SV_000000000000",
    "startDate": "startDate": "'"$STARTDATESTRING"'",
    "endDate": "endDate": "'"$ENDDATESTRING"'",
    "format": "csv",
    "useLocalTime": true,
    "useLabels": true
}' "https://myorg.qualtrics.com/API/v3/responseexports")

es_id=$(echo "$result" | /usr/local/Cellar/jq/1.5_2/bin/jq --raw-output '.result.id')

curl -H "X-API-TOKEN: MYAPITOKEN" "https://myorg.qualtrics.com/API/v3/responseexports/${es_id}"

curl -X GET -H "Content-Type: application/json" -H "X-API-TOKEN: MYAPITOKEN" "https://myorg.qualtrics.com/API/v3/responseexports/${es_id}/file" -o "/Users/myname/Desktop/WEA/response.zip"

unzip "/Users/myname/Desktop/WEA/response.zip" -d "/Users/myname/Desktop/WEA/"

我间歇性地收到以下错误-

% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 339 100 133 100 206 165 256 --:--:-- --:--:-- --:--:-- 256 {"result":{"percentComplete":0.0,"file":null,"status":"in progress"},"meta":{"httpStatus":"200 - OK","requestId":"2c55524c-03aa-495c-8de7-b54d5b441b34"}} % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 129 100 129 0 0 405 0 --:--:-- --:--:-- --:--:-- 405

End-of-central-directory signature not found. Either this file is not a zipfile, or it constitutes one disk of a multi-part archive. In the latter case the central directory and zipfile comment will be found on the last disk(s) of this archive.

unzip: cannot find zipfile directory in one of /Users/myname/Desktop/response.zip or /Users/myname/Desktop/WEA/response.zip.zip, and cannot find /Users/myname/Desktop/WEA/response.zip.ZIP, period.

奇怪的是，这个错误通常只在我第一次运行脚本时发生。如果我重新运行它，不做任何更改，它将通过而没有错误。据我了解，此错误消息表示 zip 文件已损坏。我是否通过将 POST 请求保存到变量中来破坏它？我需要以某种方式捕获 post 请求输出，因为它提供了后续 GET 请求所需的 es_id。我会硬编码，但 es_id 每周刷新一次。

Answer 1

使用@T.Gibbons 建议，我能够通过在脚本中加入一个 while 循环来消除初始运行的错误。

STARTDATE=$(date -v-7d "+%Y-%m-%d")
STARTDATESTRING=$STARTDATE"T00:00:00-07:00:00"
ENDDATE=$(date "+%Y-%m-%d")
ENDDATESTRING=$ENDDATE"T00:00:00-07:00:00"
post_response=$(curl -X POST -H 'X-API-TOKEN: MYAPITOKEN' -H 'Content-Type: application/json' -d '{
    "surveyId": "SV_00000000000000",
    "startDate": "'"$STARTDATESTRING"'",
    "endDate": "'"$ENDDATESTRING"'",
    "format": "csv",
    "useLocalTime": true,
    "useLabels": true
}' "https://myorg.qualtrics.com/API/v3/responseexports")

es_id=$(echo "$post_response" | /usr/local/Cellar/jq/1.5_2/bin/jq --raw-output '.result.id')

percent_complete=0
while [ $percent_complete -ne 100 ]
do
    response=$(curl -H "X-API-TOKEN: MYAPITOKEN" "https://myorg.qualtrics.com/API/v3/responseexports/${es_id}")
    percent_complete=$(echo "$response" | /usr/local/Cellar/jq/1.5_2/bin/jq --raw-output '.result.percentComplete')
done

curl -X GET -H "Content-Type: application/json" -H "X-API-TOKEN: MYAPITOKEN" "https://myorg.qualtrics.com/API/v3/responseexports/${es_id}/file" -o "/Users/myname/Desktop/WEA/response.zip"

unzip "/Users/myname/Desktop/WEA/response.zip" -d "/Users/myname/Desktop/WEA"

rm "/Users/myname/Desktop/WEA/response.zip"

curl 请求后压缩文件损坏

Corrupted zipfile after curl request

bash

curl

qualtrics