'gsutil compose'和'gsutil cp'作为并行复合上传有什么区别
What is the difference between 'gsutil compose' and 'gsutil cp' as a parallel composite upload
gsutil compose
和gsutil -o
GSUtil:parallel_composite_upload_threshold=150M cp bigfile
gs://your-bucket
有什么区别?
- 只有当我们有许多较小的源文件时,
gsutil compose
才有效吗?
- 我们如何获得这些较小的文件?
组合操作
The compose command creates a new object whose content is the
concatenation of a given sequence of source objects under the same
bucket.
gsutil compose gs://your-bucket/file1.txt gs://your-bucket/file2.txt gs://your-bucket/file3.txt
上面的命令会将file1.txt
和file2.txt
的内容连接成file3.txt
。大文件也可以连接起来,尽管在单个 gsutil compose
命令中最多只能连接 32 个文件。您需要事先指定并拥有这些文件。关于此的更多信息 here.
并行复合上传
If enabled (see below), a large file will be split into component
pieces that are uploaded in parallel and then composed in the cloud
(and the temporary components finally deleted). The maximum size of
each component piece is determined by the variable
parallel_composite_upload_component_size
.
gsutil -o GSUtil:parallel_composite_upload_threshold=150M cp bigfile gs://your-bucket
上面的命令会将bigfile
拆分成最多32块,上传后会合成为一块,删除临时文件,类似于合成操作。关于此的更多信息 here.
简而言之,第一个命令用于组合对象;后者将其拆分,上传片段,然后在云存储中进行组合。
gsutil compose
和gsutil -o GSUtil:parallel_composite_upload_threshold=150M cp bigfile gs://your-bucket
有什么区别?- 只有当我们有许多较小的源文件时,
gsutil compose
才有效吗? - 我们如何获得这些较小的文件?
组合操作
The compose command creates a new object whose content is the concatenation of a given sequence of source objects under the same bucket.
gsutil compose gs://your-bucket/file1.txt gs://your-bucket/file2.txt gs://your-bucket/file3.txt
上面的命令会将file1.txt
和file2.txt
的内容连接成file3.txt
。大文件也可以连接起来,尽管在单个 gsutil compose
命令中最多只能连接 32 个文件。您需要事先指定并拥有这些文件。关于此的更多信息 here.
并行复合上传
If enabled (see below), a large file will be split into component pieces that are uploaded in parallel and then composed in the cloud (and the temporary components finally deleted). The maximum size of each component piece is determined by the variable
parallel_composite_upload_component_size
.
gsutil -o GSUtil:parallel_composite_upload_threshold=150M cp bigfile gs://your-bucket
上面的命令会将bigfile
拆分成最多32块,上传后会合成为一块,删除临时文件,类似于合成操作。关于此的更多信息 here.
简而言之,第一个命令用于组合对象;后者将其拆分,上传片段,然后在云存储中进行组合。