如何将远程hive查询的结果存储到文件中

Question

我正在尝试运行 Google Compute Engine 上的配置单元查询。我的 Hadoop 服务在 Google Dataproc 上。我使用此命令提交配置单元作业 -

gcloud dataproc jobs submit hive --region=my-region --cluster=my-cluster-name -f file.hql > result.txt

和

gcloud dataproc jobs submit hive --region=my-region --cluster=my-cluster-name -e="use test;select * from emp;" > result.txt

我希望在 result.txt 中看到查询结果，但这就是我在 txt 文件中得到的全部结果 -

done: true
driverControlFilesUri: gs://my-gcs-bucket-for-dataproc/google-cloud-dataproc-metainfo/27f9-f4a5-4df2-a311-e41a92/jobs/ea7ab2164/
driverOutputResourceUri: gs://my-gcs-bucket-for-dataproc/google-cloud-dataproc-metainfo/1f309-f4a5-4df2-a311-e4182/jobs/eafab0e2164/driveroutput
hiveJob:
queryFileUri: gs://my-gcs-bucket-for-dataproc/google-cloud-dataproc-metainfo/1ff9-f4a5-4df2-a311-e412/jobs/ea781f64/staging/file.hql
jobUuid: 91db33-ee0e-391b-b46d-37b276
placement:
  clusterName: my-cluster-name
  clusterUuid: my-cluster-uuid
reference:
  jobId: ea7ab0e2164
  projectId: my-project
status:
  state: DONE
  stateStartTime: '2022-02-07T09:33:44.317237Z'
statusHistory:
- state: PENDING
  stateStartTime: '2022-02-07T09:33:16.724561Z'
- state: SETUP_DONE
  stateStartTime: '2022-02-07T09:33:16.762680Z'
- details: Agent reported job success
  state: RUNNING
  stateStartTime: '2022-02-07T09:33:18.403518Z'
yarnApplications:
- name: HIVE-94a5b7-8bc7-4dc9-a016-81ab721
  progress: 1.0
  state: RUNNING
  trackingUrl: http://my-cluster-name:8088/proxy/application_1692_0008/

如有任何帮助，我们将不胜感激。谢谢

Answer 1

查询结果在标准错误中。尝试 &> result.txt 重定向 stdout 和 stderr，或 2> result.txt 仅重定向 stderr。

如何将远程hive查询的结果存储到文件中

How to store the result of remote hive query to a file

hive

hiveql

google-cloud-dataproc