如何在 spark HDInsight 集群上安装 tensorflow？

Question

我正在尝试在 spark HDInsight 集群上安装 tensorflow。但面临问题。

我从头节点使用 pip install tensorflow。

我可以从 python.

导入 tensorflow

Python 2.7.12 (default, Dec  4 2017, 14:50:18)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow
>>> exit()

但是当我尝试从头节点或 pyspark 控制台导入 tensorflow 时，它抛出错误

SPARK_MAJOR_VERSION is set to 2, using Spark2
Python 2.7.12 |Anaconda custom (64-bit)| (default, Jul  2 2016, 17:42:40)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel).
Welcome to
      ____              __
     / __/__  ___ _____/ /__
    _\ \/ _ \/ _ `/ __/  '_/
   /__ / .__/\_,_/_/ /_/\_\   version 2.0.2.2.5.6.2-9
      /_/

Using Python version 2.7.12 (default, Jul  2 2016 17:42:40)
SparkSession available as 'spark'.
>>> import tensorflow
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named tensorflow

谁能帮我在 spark HDinsight 集群上安装 tensorflow。

谢谢。

Answer 1

您可以使用脚本操作在头节点和工作节点上安装 TensorFlow，并在 "Use external packages with Jupyter notebooks" 部分找到 here 的说明，提供了 TensorFlow bash 脚本

如何在 spark HDInsight 集群上安装 tensorflow？

how to install tensorflow on a spark HDInsight cluster?

python

apache-spark

pyspark

tensorflow

azure-hdinsight