使用 Python 在 SFTP 服务器上保存时截断文件？

Question

这是

的扩展问题

with pysftp.Connection(host=myHostName, username=myUsername, 
                      password=myPassword, cnopts=cnopts) as sftp:
    storage_client = storage.Client()
    bucket = storage_client.bucket(bucket_name)
    blob = bucket.blob(filename) #source_blob_name
    with sftp.open(remotepath, 'w+', 32768) as f:
        blob.download_to_file(f)

W+/w 命令没有截断之前的文件并抛出错误。这是当 SFTP 服务器中已经存在同名文件时！最佳解决方案应该是什么？

完整的错误代码日志为

gcs-to-sftp
4wz44jq3oe2g
Traceback (most recent call last):
  File "/env/local/lib/python3.7/site-packages/google/cloud/storage/client.py", line 728, in download_blob_to_file
    checksum=checksum,
  File "/env/local/lib/python3.7/site-packages/google/cloud/storage/blob.py", line 986, in _do_download
    response = download.consume(transport, timeout=timeout)
  File "/env/local/lib/python3.7/site-packages/google/resumable_media/requests/download.py", line 168, in consume
    self._process_response(result)
  File "/env/local/lib/python3.7/site-packages/google/resumable_media/_download.py", line 186, in _process_response
    response, _ACCEPTABLE_STATUS_CODES, self._get_status_code
  File "/env/local/lib/python3.7/site-packages/google/resumable_media/_helpers.py", line 104, in require_status_code
    *status_codes
google.resumable_media.common.InvalidResponse: ('Request failed with status code', 404, 'Expected one of', <HTTPStatus.OK: 200>, <HTTPStatus.PARTIAL_CONTENT: 206>)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker_v2.py", line 449, in run_background_function
    _function_handler.invoke_user_function(event_object)
  File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker_v2.py", line 268, in invoke_user_function
    return call_user_function(request_or_event)
  File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker_v2.py", line 265, in call_user_function
    event_context.Context(**request_or_event.context))
  File "/user_code/main.py", line 20, in hello_sftp
    copy_file_on_ftp(myHostName, myUsername, myPassword, bucket_name, filename)
  File "/user_code/main.py", line 42, in copy_file_on_ftp
    file_to_export.download_to_file(f)
  File "/env/local/lib/python3.7/site-packages/google/cloud/storage/blob.py", line 1128, in download_to_file
    checksum=checksum,
  File "/env/local/lib/python3.7/site-packages/google/cloud/storage/client.py", line 731, in download_blob_to_file
    _raise_from_invalid_response(exc)
  File "/env/local/lib/python3.7/site-packages/google/cloud/storage/blob.py", line 4100, in _raise_from_invalid_response
    raise exceptions.from_http_status(response.status_code, message, response=response)
google.api_core.exceptions.NotFound: 404 GET https://storage.googleapis.com/download/storage/v1/b/sftp__test/o/test_file.csv?alt=media: No such object: sftp__test/test_file.csv: ('Request failed with status code', 404, 'Expected one of', <HTTPStatus.OK: 200>, <HTTPStatus.PARTIAL_CONTENT: 206>)

Answer 1

根据@Martin-Prikryl 的建议回答

替换

with sftp.open(remotepath, 'w+', 32768) as f:

和

sftp.open(remotepath, "wb")

或使用

from io import BytesIO

flo = BytesIO() 
blob.download_to_file(flo) 
flo.seek(0) 
sftp.putfo(flo,remotepath)

Answer 2

w+模式下的+不应该存在。它打开文件以进行读写，这在您的方案中不需要。

虽然 w 应该截断文件（在普通的 OpenSSH Linux 服务器上也是如此），但当与 +.

使用 Python 在 SFTP 服务器上保存时截断文件？

Truncating the file while saving on SFTP server using Python?

python

google-cloud-functions

google-cloud-storage

sftp

pysftp