在 Conda 环境中安装 Apache-Airflow
Installing Apache-Airflow in Conda Environment
情况
我正在尝试使用 pip 将 apache-airflow 安装到 conda 环境中。我做不到，因为 setup.py egg_info 一直失败：
Command "python setup.py egg_info" failed with error code 1 in
C:\Users\joshu\AppData\Local\Temp\pip-install-3efyslfh\apache-airflow\
当我 运行
时的消息
pip install "apache-airflow[s3, postgres]"
我 运行 在 Windows cmd 中执行此操作，然后在 cmd 中以管理员身份再次执行所有步骤（打开 cmd 时按 shift+enter）。
(myVenv) C:\Users\joshu\Documents>pip install "apache-airflow[s3, postgres]"
Collecting apache-airflow[postgres,s3]
Using cached https://files.pythonhosted.org/packages/9e/12/6c70f9ef852b3061a3a6c9af03bd9dcdcaecb7d75c8898f82e3a54ad5f87/apache-airflow-1.9.0.tar.gz
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\Users\joshu\AppData\Local\Temp\pip-install-3efyslfh\apache-airflow\setup.py", line 102
async = [
^
SyntaxError: invalid syntax
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in C:\Users\joshu\AppData\Local\Temp\pip-install-3efyslfh\apache-airflow\
我采取的步骤
1）我首先创建了一个conda环境，并在环境中安装了pip和setuptools：
C:\Users\joshu\Documents>conda create -n airflow pip setuptools
Solving environment: done
==> WARNING: A newer version of conda exists. <==
current version: 4.5.4
latest version: 4.5.10
Please update conda by running
$ conda update -n base conda
## Package Plan ##
environment location: C:\Users\joshu\Anaconda3\envs\airflow
added / updated specs:
- pip
- setuptools
The following NEW packages will be INSTALLED:
certifi: 2018.8.13-py37_0
pip: 10.0.1-py37_0
python: 3.7.0-hea74fb7_0
setuptools: 40.0.0-py37_0
vc: 14.1-h0510ff6_3
vs2015_runtime: 15.5.2-3
wheel: 0.31.1-py37_0
wincertstore: 0.2-py37_0
C:\Users\joshu\Documents>activate airflow
(airflow) C:\Users\joshu\Documents> pip freeze
certifi==2018.8.13
wincertstore==0.2
(airflow) C:\Users\joshu\Documents>conda list
# packages in environment at C:\Users\joshu\Anaconda3\envs\airflow:
#
# Name Version Build Channel
certifi 2018.8.13 py37_0
pip 10.0.1 py37_0
python 3.7.0 hea74fb7_0
setuptools 40.0.0 py37_0
vc 14.1 h0510ff6_3
vs2015_runtime 15.5.2 3
wheel 0.31.1 py37_0
wincertstore 0.2 py37_0
(airflow) C:\Users\joshu\Documents>where pip
C:\Users\joshu\Anaconda3\envs\airflow\Scripts\pip.exe
C:\Users\joshu\Anaconda3\Scripts\pip.exe
(airflow) C:\Users\joshu\Documents>which pip
/c/Users/joshu/Anaconda3/envs/airflow/Scripts/pip
在这个设置之后，运行ning pip install apache-airflow 给我这个页面顶部的错误
我采取的其他步骤
- 正在我的系统上安装 apache-airflow（无 conda 环境）。这可行，但我希望将其包含在环境中。
- 正在升级设置工具：
代码：
(airflow) C:\Users\joshu\Documents>pip install --upgrade setuptools
Collecting setuptools
Using cached https://files.pythonhosted.org/packages/66/e8/570bb5ca88a8bcd2a1db9c6246bb66615750663ffaaeada95b04ffe74e12/setuptools-40.2.0-py2.py3-none-any.whl
Installing collected packages: setuptools
Found existing installation: setuptools 40.0.0
Uninstalling setuptools-40.0.0:
Successfully uninstalled setuptools-40.0.0
Successfully installed setuptools-40.2.0
You are using pip version 10.0.1, however version 18.0 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' command.
(airflow) C:\Users\joshu\Documents>pip freeze
certifi==2018.8.13
wincertstore==0.2
You are using pip version 10.0.1, however version 18.0 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' command.
(airflow) C:\Users\joshu\Documents>conda list
# packages in environment at C:\Users\joshu\Anaconda3\envs\airflow:
#
# Name Version Build Channel
certifi 2018.8.13 py37_0
pip 10.0.1 py37_0
python 3.7.0 hea74fb7_0
setuptools 40.2.0 <pip>
setuptools 40.0.0 py37_0
vc 14.1 h0510ff6_3
vs2015_runtime 15.5.2 3
wheel 0.31.1 py37_0
wincertstore 0.2 py37_0
(airflow) C:\Users\joshu\Documents>pip install "apache-airflow[s3, postgres]"
Collecting apache-airflow[postgres,s3]
Using cached https://files.pythonhosted.org/packages/9e/12/6c70f9ef852b3061a3a6c9af03bd9dcdcaecb7d75c8898f82e3a54ad5f87/apache-airflow-1.9.0.tar.gz
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\Users\joshu\AppData\Local\Temp\pip-install-n1v4sa6d\apache-airflow\setup.py", line 102
async = [
^
SyntaxError: invalid syntax
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in C:\Users\joshu\AppData\Local\Temp\pip-install-n1v4sa6d\apache-airflow\
我也试过升级 pip 然后 运行ning install -- 运气不好
(airflow) C:\Users\joshu\Documents>python -m pip install --upgrade pip
Collecting pip
Using cached https://files.pythonhosted.org/packages/5f/25/e52d3f31441505a5f3af41213346e5b6c221c9e086a166f3703d2ddaf940/pip-18.0-py2.py3-none-any.whl
Installing collected packages: pip
Found existing installation: pip 10.0.1
Uninstalling pip-10.0.1:
Successfully uninstalled pip-10.0.1
Successfully installed pip-18.0
(airflow) C:\Users\joshu\Documents>which pip
/c/Users/joshu/Anaconda3/envs/airflow/Scripts/pip
(airflow) C:\Users\joshu\Documents>pip --version
pip 18.0 from c:\users\joshu\anaconda3\envs\airflow\lib\site-packages\pip (python 3.7)
(airflow) C:\Users\joshu\Documents>pip install "apache-airflow[s3, postgres]"
最终我想要在 conda 环境中为我的项目提供气流和相关包。
错误似乎是因为 async is a reserved keyword in Python 3.7 并且此软件包尚未更新其 setup.py 脚本。在软件包更新之前，您可以使用 Python 3.6
conda create -n airflow pip setuptools python=3.6
conda activate airflow
pip install "apache-airflow[s3, postgres]"
我运行陷入同样的问题。 Apache Airflow 1.10 不兼容。
我的解决方案是构建源代码。我通过 git 获得了最新的源代码。此命令在 dist 目录中创建一个 wheel 文件。
python3 setup.py bdist_wheel
然后你安装
pip3 install apache_airflow-2.0.0.dev0+incubating-py3-none-any.whl
我也遇到了同样的问题。我通过使用 conda 而不是 pip 安装来解决它：
conda 安装-c conda-forge 气流
情况
我正在尝试使用 pip 将 apache-airflow 安装到 conda 环境中。我做不到，因为 setup.py egg_info 一直失败：
Command "python setup.py egg_info" failed with error code 1 in C:\Users\joshu\AppData\Local\Temp\pip-install-3efyslfh\apache-airflow\
当我 运行
时的消息pip install "apache-airflow[s3, postgres]"
我 运行 在 Windows cmd 中执行此操作，然后在 cmd 中以管理员身份再次执行所有步骤（打开 cmd 时按 shift+enter）。
(myVenv) C:\Users\joshu\Documents>pip install "apache-airflow[s3, postgres]"
Collecting apache-airflow[postgres,s3]
Using cached https://files.pythonhosted.org/packages/9e/12/6c70f9ef852b3061a3a6c9af03bd9dcdcaecb7d75c8898f82e3a54ad5f87/apache-airflow-1.9.0.tar.gz
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\Users\joshu\AppData\Local\Temp\pip-install-3efyslfh\apache-airflow\setup.py", line 102
async = [
^
SyntaxError: invalid syntax
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in C:\Users\joshu\AppData\Local\Temp\pip-install-3efyslfh\apache-airflow\
我采取的步骤
1）我首先创建了一个conda环境，并在环境中安装了pip和setuptools：
C:\Users\joshu\Documents>conda create -n airflow pip setuptools
Solving environment: done
==> WARNING: A newer version of conda exists. <==
current version: 4.5.4
latest version: 4.5.10
Please update conda by running
$ conda update -n base conda
## Package Plan ##
environment location: C:\Users\joshu\Anaconda3\envs\airflow
added / updated specs:
- pip
- setuptools
The following NEW packages will be INSTALLED:
certifi: 2018.8.13-py37_0
pip: 10.0.1-py37_0
python: 3.7.0-hea74fb7_0
setuptools: 40.0.0-py37_0
vc: 14.1-h0510ff6_3
vs2015_runtime: 15.5.2-3
wheel: 0.31.1-py37_0
wincertstore: 0.2-py37_0
C:\Users\joshu\Documents>activate airflow
(airflow) C:\Users\joshu\Documents> pip freeze
certifi==2018.8.13
wincertstore==0.2
(airflow) C:\Users\joshu\Documents>conda list
# packages in environment at C:\Users\joshu\Anaconda3\envs\airflow:
#
# Name Version Build Channel
certifi 2018.8.13 py37_0
pip 10.0.1 py37_0
python 3.7.0 hea74fb7_0
setuptools 40.0.0 py37_0
vc 14.1 h0510ff6_3
vs2015_runtime 15.5.2 3
wheel 0.31.1 py37_0
wincertstore 0.2 py37_0
(airflow) C:\Users\joshu\Documents>where pip
C:\Users\joshu\Anaconda3\envs\airflow\Scripts\pip.exe
C:\Users\joshu\Anaconda3\Scripts\pip.exe
(airflow) C:\Users\joshu\Documents>which pip
/c/Users/joshu/Anaconda3/envs/airflow/Scripts/pip
在这个设置之后，运行ning pip install apache-airflow 给我这个页面顶部的错误
我采取的其他步骤
- 正在我的系统上安装 apache-airflow（无 conda 环境）。这可行，但我希望将其包含在环境中。
- 正在升级设置工具：
(airflow) C:\Users\joshu\Documents>pip install --upgrade setuptools
Collecting setuptools
Using cached https://files.pythonhosted.org/packages/66/e8/570bb5ca88a8bcd2a1db9c6246bb66615750663ffaaeada95b04ffe74e12/setuptools-40.2.0-py2.py3-none-any.whl
Installing collected packages: setuptools
Found existing installation: setuptools 40.0.0
Uninstalling setuptools-40.0.0:
Successfully uninstalled setuptools-40.0.0
Successfully installed setuptools-40.2.0
You are using pip version 10.0.1, however version 18.0 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' command.
(airflow) C:\Users\joshu\Documents>pip freeze
certifi==2018.8.13
wincertstore==0.2
You are using pip version 10.0.1, however version 18.0 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' command.
(airflow) C:\Users\joshu\Documents>conda list
# packages in environment at C:\Users\joshu\Anaconda3\envs\airflow:
#
# Name Version Build Channel
certifi 2018.8.13 py37_0
pip 10.0.1 py37_0
python 3.7.0 hea74fb7_0
setuptools 40.2.0 <pip>
setuptools 40.0.0 py37_0
vc 14.1 h0510ff6_3
vs2015_runtime 15.5.2 3
wheel 0.31.1 py37_0
wincertstore 0.2 py37_0
(airflow) C:\Users\joshu\Documents>pip install "apache-airflow[s3, postgres]"
Collecting apache-airflow[postgres,s3]
Using cached https://files.pythonhosted.org/packages/9e/12/6c70f9ef852b3061a3a6c9af03bd9dcdcaecb7d75c8898f82e3a54ad5f87/apache-airflow-1.9.0.tar.gz
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\Users\joshu\AppData\Local\Temp\pip-install-n1v4sa6d\apache-airflow\setup.py", line 102
async = [
^
SyntaxError: invalid syntax
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in C:\Users\joshu\AppData\Local\Temp\pip-install-n1v4sa6d\apache-airflow\
我也试过升级 pip 然后 运行ning install -- 运气不好
(airflow) C:\Users\joshu\Documents>python -m pip install --upgrade pip
Collecting pip
Using cached https://files.pythonhosted.org/packages/5f/25/e52d3f31441505a5f3af41213346e5b6c221c9e086a166f3703d2ddaf940/pip-18.0-py2.py3-none-any.whl
Installing collected packages: pip
Found existing installation: pip 10.0.1
Uninstalling pip-10.0.1:
Successfully uninstalled pip-10.0.1
Successfully installed pip-18.0
(airflow) C:\Users\joshu\Documents>which pip
/c/Users/joshu/Anaconda3/envs/airflow/Scripts/pip
(airflow) C:\Users\joshu\Documents>pip --version
pip 18.0 from c:\users\joshu\anaconda3\envs\airflow\lib\site-packages\pip (python 3.7)
(airflow) C:\Users\joshu\Documents>pip install "apache-airflow[s3, postgres]"
最终我想要在 conda 环境中为我的项目提供气流和相关包。
错误似乎是因为 async is a reserved keyword in Python 3.7 并且此软件包尚未更新其 setup.py 脚本。在软件包更新之前，您可以使用 Python 3.6
conda create -n airflow pip setuptools python=3.6
conda activate airflow
pip install "apache-airflow[s3, postgres]"
我运行陷入同样的问题。 Apache Airflow 1.10 不兼容。
我的解决方案是构建源代码。我通过 git 获得了最新的源代码。此命令在 dist 目录中创建一个 wheel 文件。
python3 setup.py bdist_wheel
然后你安装
pip3 install apache_airflow-2.0.0.dev0+incubating-py3-none-any.whl
我也遇到了同样的问题。我通过使用 conda 而不是 pip 安装来解决它：
conda 安装-c conda-forge 气流