无法导入气流提供程序包

Question

我无法为 Google 导入气流提供程序包。我使用的命令是

pip3 install apache-airflow-backport-providers-google

它给了我错误

ERROR: Could not find a version that satisfies the requirement apache-airflow-backport-providers-google (from versions: none)
ERROR: No matching distribution found for apache-airflow-backport-providers-google

因此我的 DAG 卡住了，错误消息是

Broken DAGS: No module named 'airflow.providers'

我查了airflow库：env/lib/python3.5/site-packages/airflow

找不到提供程序文件夹。

编辑： 当我按照答案中的建议将 python 版本升级到 3.6+ 时，上述问题已解决。但是现在，面临一个新的价值错误问题。我在导入 airflow.providers.google.cloud.operators.dataproc.

后使用 DataprocInstantiateWorkflowTemplateOperator

请在下面找到错误回溯。

Traceback (most recent call last):
  File "workflow_dag.py", line 6, in <module>
    from airflow.providers.google.cloud.operators.dataproc import DataprocInstantiateWorkflowTemplateOperator
  File "/home/kshitij/env/lib/python3.7/site-packages/airflow/providers/google/cloud/operators/dataproc.py", line 36, in <module>
    from google.cloud.dataproc_v1beta2.types import (  # pylint: disable=no-name-in-module
  File "/home/kshitij/env/lib/python3.7/site-packages/google/cloud/dataproc_v1beta2/__init__.py", line 18, in <module>
    from .services.autoscaling_policy_service import AutoscalingPolicyServiceClient
  File "/home/kshitij/env/lib/python3.7/site-packages/google/cloud/dataproc_v1beta2/services/autoscaling_policy_service/__init__.py", line 18, in <module>
    from .client import AutoscalingPolicyServiceClient
  File "/home/kshitij/env/lib/python3.7/site-packages/google/cloud/dataproc_v1beta2/services/autoscaling_policy_service/client.py", line 33, in <module>
    from google.cloud.dataproc_v1beta2.services.autoscaling_policy_service import pagers
  File "/home/kshitij/env/lib/python3.7/site-packages/google/cloud/dataproc_v1beta2/services/autoscaling_policy_service/pagers.py", line 20, in <module>
    from google.cloud.dataproc_v1beta2.types import autoscaling_policies
  File "/home/kshitij/env/lib/python3.7/site-packages/google/cloud/dataproc_v1beta2/types/__init__.py", line 18, in <module>
    from .autoscaling_policies import (
  File "/home/kshitij/env/lib/python3.7/site-packages/google/cloud/dataproc_v1beta2/types/autoscaling_policies.py", line 41, in <module>
    class AutoscalingPolicy(proto.Message):
  File "/home/kshitij/env/lib/python3.7/site-packages/proto/message.py", line 139, in __new__
    field.descriptor.oneof_index = oneofs[field.oneof]
  File "/home/kshitij/env/lib/python3.7/site-packages/proto/fields.py", line 113, in descriptor
    proto3_optional=self.optional,
ValueError: Protocol message FieldDescriptorProto has no "proto3_optional" field.

Answer 1

您的 Python 版本是 python3.5，在此之前（参见 documentation 和下图），您需要将 python 升级到 3.6+如果你想使用 apache-airflow-backport-providers-google 包。

Python Cloud Composer and you could try either upgrade your enviroment 支持 3.6 或创建一个新版本。

无法导入气流提供程序包

Unable to import airflow providers package

python

google-cloud-platform

google-cloud-dataproc

airflow