无法加载 ipython 魔术扩展 sparkmagic：找不到模块

Question

我不熟悉将自定义 magics 加载到 ipython。载入说明

sparkmagics

扩展包括以下已成功完成的步骤：

pip install sparkmagic
pip show sparkmagic
jupyter-kernelspec install sparkmagic/kernels/sparkkernel
jupyter-kernelspec install sparkmagic/kernels/pysparkkernel
jupyter-kernelspec install sparkmagic/kernels/pyspark3kernel
jupyter-kernelspec install sparkmagic/kernels/sparkrkernel

命令pip show sparkmagic似乎是健康的：

$pip show sparkmagic
Name: sparkmagic
Version: 0.10.1
Summary: SparkMagic: Spark execution via Livy
Home-page: https://github.com/jupyter-incubator/sparkmagic/sparkmagic
Author: Jupyter Development Team
Author-email: jupyter@googlegroups.org
License: BSD 3-clause
Location: /usr/local/lib/python2.7/site-packages
Requires: mock, requests, tornado, nose, notebook, ipywidgets, ipykernel, pandas, hdijupyterutils, autovizwidget, ipython, numpy

没有的东西 - 但列为可选的是;

$ jupyter serverextension enable --py sparkmagic

Traceback (most recent call last):
  File "/usr/local/bin/jupyter-serverextension", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.6/site-packages/jupyter_core/application.py", line 267, in launch_instance
    return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/traitlets/config/application.py", line 658, in launch_instance
    app.start()
  File "/usr/local/lib/python3.6/site-packages/notebook/serverextensions.py", line 300, in start
    super(ServerExtensionApp, self).start()
  File "/usr/local/lib/python3.6/site-packages/jupyter_core/application.py", line 256, in start
    self.subapp.start()
  File "/usr/local/lib/python3.6/site-packages/notebook/serverextensions.py", line 217, in start
    self.toggle_server_extension_python(arg)
  File "/usr/local/lib/python3.6/site-packages/notebook/serverextensions.py", line 206, in toggle_server_extension_python
    m, server_exts = _get_server_extension_metadata(package)
  File "/usr/local/lib/python3.6/site-packages/notebook/serverextensions.py", line 334, in _get_server_extension_metadata
    m = import_item(module)
  File "/usr/local/lib/python3.6/site-packages/traitlets/utils/importstring.py", line 42, in import_item
    return __import__(parts[0])
ModuleNotFoundError: No module named 'sparkmagic'

错误ModuleNotFoundError: No module named 'sparkmagic'似乎是不祥之兆。鉴于 show sparkmagic 是健康的，那么为什么要在这里投诉？ 正确设置 sparkmagic 模块还缺少什么？

无论如何，我也在 ipython 内部进行了尝试，但不出所料，它失败了：

%load_ext sparkmagic.magics

这是（很长的）堆栈跟踪的第一部分。让我知道是否需要整个跟踪。

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-1-2b579b518c56> in <module>()
----> 1 get_ipython().magic(u'load_ext sparkmagic.magics')

/usr/local/lib/python2.7/site-packages/IPython/core/interactiveshell.pyc in magic(self, arg_s)
   2161         magic_name, _, magic_arg_s = arg_s.partition(' ')
   2162         magic_name = magic_name.lstrip(prefilter.ESC_MAGIC)
-> 2163         return self.run_line_magic(magic_name, magic_arg_s)
   2164
   2165     #-------------------------------------------------------------------------

/usr/local/lib/python2.7/site-packages/IPython/core/interactiveshell.pyc in run_line_magic(self, magic_name, line)
   2082                 kwargs['local_ns'] = sys._getframe(stack_depth).f_locals
   2083             with self.builtin_trap:
-> 2084                 result = fn(*args,**kwargs)
   2085             return result
   2086

<decorator-gen-64> in load_ext(self, module_str)

/usr/local/lib/python2.7/site-packages/IPython/core/magic.pyc in <lambda>(f, *a, **k)
    191     # but it's overkill for just that one bit of state.
    192     def magic_deco(arg):
--> 193         call = lambda f, *a, **k: f(*a, **k)
    194
    195         if callable(arg):

/usr/local/lib/python2.7/site-packages/IPython/core/magics/extension.pyc in load_ext(self, module_str)
     64         if not module_str:
     65             raise UsageError('Missing module name.')
---> 66         res = self.shell.extension_manager.load_extension(module_str)
     67
     68         if res == 'already loaded':

总而言之，installing/using任何魔法通常都需要一个步骤。感谢指点。

Answer 1

根据我在您的错误和安装消息中看到的情况，您为 python2.7 安装了 spark magic，并试图在 python 3.6 中运行它。它令人困惑，当我开始时，我也多次遇到同样的问题。我建议您通过笔记本上的运行ning !python --version 检查您使用的 python 版本。如果 python 的版本正确，则使用 !pip install sparkmagic 安装 sparkmagic 并检查安装消息是否安装到 python 的正确版本。

您还可以设置一个环境，让生活更轻松。更多内容 https://www.youtube.com/watch?v=LjQlmee58hg

无法加载 ipython 魔术扩展 sparkmagic：找不到模块

Unable to load an ipython magic extension sparkmagic: module not found

python

ipython

ipython-notebook

jupyter-notebook