从位于另一个目录的模块中导入本地函数,并使用python3在jupyter笔记本中导入相关的导入。

时间:2022-02-20 13:13:31

I have a directory structure similar to the following

我有一个类似于下面的目录结构

meta_project
    project1
        __init__.py
        lib
            module.py
            __init__.py
    notebook_folder
        notebook.jpynb

When working in notebook.jpynb if I try to use a relative import to access a function function() in module.py with:

在使用note .jpynb时,如果我试图使用相对导入访问模块中的函数()。py:

from ..project1.lib.module import function

I get the following error

我得到以下错误

SystemError                               Traceback (most recent call last)
<ipython-input-7-6393744d93ab> in <module>()
----> 1 from ..project1.lib.module import function

SystemError: Parent module '' not loaded, cannot perform relative import

Is there any way to get this to work using relative imports?

有什么方法可以让它使用相对进口来工作吗?

Note, the notebook server is instantiated at the level of the meta_project directory, so it should have access to the information in those files.

注意,在meta_project目录级别上实例化了notebook服务器,因此它应该有权访问这些文件中的信息。

Note, also, that at least as originally intended project1 wasn't thought of as a module and therefore does not have an __init__.py file, it was just meant as a file-system directory. If the solution to the problem requires treating it as a module and including an __init__.py file (even a blank one) that is fine, but doing so is not enough to solve the problem.

请注意,至少最初打算的project1没有被认为是一个模块,因此没有__init__。py文件,它只是一个文件系统目录。如果这个问题的解决方案需要把它作为一个模块,包括一个__init__。py文件(即使是空白文件)也可以,但是这样做还不足以解决问题。

I share this directory between machines and relative imports allow me to use the same code everywhere, & I often use notebooks for quick prototyping, so suggestions that involve hacking together absolute paths are unlikely to be helpful.

我在机器和相关导入之间共享这个目录,这允许我在任何地方都使用相同的代码,而且我经常使用笔记本进行快速原型设计,因此涉及到将绝对路径组合在一起的建议不太可能有帮助。


Edit: This is unlike Relative imports in Python 3, which talks about relative imports in Python 3 in general and – in particular – running a script from within a package directory. This has to do with working within a jupyter notebook trying to call a function in a local module in another directory which has both different general and particular aspects.

编辑:这与Python 3中的相对导入不同,后者一般都讨论Python 3中的相对导入,特别是在包目录中运行脚本。这与在jupyter记事本中工作有关,试图调用另一个目录中的本地模块中的函数,该目录具有不同的通用和特定方面。

3 个解决方案

#1


66  

I had almost the same example as you in this notebook where I wanted to illustrate the usage of an adjacent module's function in a DRY manner.

我在这个笔记本上有一个与您几乎相同的示例,我想以一种枯燥的方式说明相邻模块函数的用法。

My solution was to tell Python of that additional module import path by adding a snippet like this one to the notebook:

我的解决方案是通过在笔记本上添加一个这样的代码片段来告诉Python这个额外的模块导入路径:

import os
import sys
module_path = os.path.abspath(os.path.join('..'))
if module_path not in sys.path:
    sys.path.append(module_path)

This allows you to import the desired function from the module hierarchy:

这允许您从模块层次结构中导入所需的函数:

from project1.lib.module import function
# use the function normally
function(...)

Note that it is necessary to add empty __init__.py files to project1/ and lib/ folders if you don't have them already.

注意,需要添加空__init__。py文件到project1/和lib/文件夹,如果你还没有的话。

#2


5  

Came here searching for best practices in abstracting code to submodules when working in Notebooks. I'm not sure that there is a best practice. I have been proposing this.

来到这里是为了寻找在笔记本上工作时将代码抽象为子模块的最佳实践。我不确定是否有最佳实践。我一直在提议。

A project hierarchy as such:

项目层次结构如下:

├── ipynb
│   ├── 20170609-Examine_Database_Requirements.ipynb
│   └── 20170609-Initial_Database_Connection.ipynb
└── lib
    ├── __init__.py
    └── postgres.py

And from 20170609-Initial_Database_Connection.ipynb:

从20170609 - initial_database_connection.ipynb:

    In [1]: cd ..

    In [2]: from lib.postgres import database_connection

This works because by default the Jupyter Notebook can parse the cd command. Note that this does not make use of Python Notebook magic. It simply works without prepending %bash.

因为默认情况下,Jupyter笔记本可以解析cd命令。注意,这并没有利用Python笔记本的魔力。它只是在没有预挂%bash的情况下工作。

Considering that 99 times out of a 100 I am working in Docker using one of the Project Jupyter Docker images, the following modification is idempotent

考虑到我在Docker工作的99次使用的是Jupyter Docker的一个项目,下面的修改是幂等的

    In [1]: cd /home/jovyan

    In [2]: from lib.postgres import database_connection

#3


1  

So far, the accepted answer has worked best for me. However, my concern has always been that there is a likely scenario where I might refactor the notebooks directory into subdirectories, requiring to change the module_path in every notebook. I decided to add a python file within each notebook directory to import the required modules.

到目前为止,公认的答案对我最有效。然而,我一直担心的是,有一种可能的情况是,我可能将notebook目录重构为子目录,需要更改每个notebook中的module_path。我决定在每个笔记本目录中添加一个python文件,以导入所需的模块。

Thus, having the following project structure:

因此,具有以下项目结构:

project
|__notebooks
   |__explore
      |__ notebook1.ipynb
      |__ notebook2.ipynb
      |__ project_path.py
   |__ explain
       |__notebook1.ipynb
       |__project_path.py
|__lib
   |__ __init__.py
   |__ module.py

I added the file project_path.py in each notebook subdirectory (notebooks/explore and notebooks/explain). This file contains the code for relative imports (from @metakermit):

我添加了文件project_path。每个笔记本子目录中的py(笔记本/探索和笔记本/解释)。此文件包含相对导入的代码(来自@metakermit):

import sys
import os

module_path = os.path.abspath(os.path.join(os.pardir, os.pardir))
if module_path not in sys.path:
    sys.path.append(module_path)

This way, I just need to do relative imports within the project_path.py file, and not in the notebooks. The notebooks files would then just need to import project_path before importing lib. For example in 0.0-notebook.ipynb:

这样,我只需要在project_path中执行相对导入。py文件,不在笔记本上。在导入lib之前,笔记本文件只需要导入project_path,例如在0-notebook.ipynb中:

import project_path
import lib

The caveat here is that reversing the imports would not work. THIS DOES NOT WORK:

这里的警告是,逆转进口不会起作用。这并不工作:

import lib
import project_path

Thus care must be taken during imports.

因此,进口时必须小心。

#1


66  

I had almost the same example as you in this notebook where I wanted to illustrate the usage of an adjacent module's function in a DRY manner.

我在这个笔记本上有一个与您几乎相同的示例,我想以一种枯燥的方式说明相邻模块函数的用法。

My solution was to tell Python of that additional module import path by adding a snippet like this one to the notebook:

我的解决方案是通过在笔记本上添加一个这样的代码片段来告诉Python这个额外的模块导入路径:

import os
import sys
module_path = os.path.abspath(os.path.join('..'))
if module_path not in sys.path:
    sys.path.append(module_path)

This allows you to import the desired function from the module hierarchy:

这允许您从模块层次结构中导入所需的函数:

from project1.lib.module import function
# use the function normally
function(...)

Note that it is necessary to add empty __init__.py files to project1/ and lib/ folders if you don't have them already.

注意,需要添加空__init__。py文件到project1/和lib/文件夹,如果你还没有的话。

#2


5  

Came here searching for best practices in abstracting code to submodules when working in Notebooks. I'm not sure that there is a best practice. I have been proposing this.

来到这里是为了寻找在笔记本上工作时将代码抽象为子模块的最佳实践。我不确定是否有最佳实践。我一直在提议。

A project hierarchy as such:

项目层次结构如下:

├── ipynb
│   ├── 20170609-Examine_Database_Requirements.ipynb
│   └── 20170609-Initial_Database_Connection.ipynb
└── lib
    ├── __init__.py
    └── postgres.py

And from 20170609-Initial_Database_Connection.ipynb:

从20170609 - initial_database_connection.ipynb:

    In [1]: cd ..

    In [2]: from lib.postgres import database_connection

This works because by default the Jupyter Notebook can parse the cd command. Note that this does not make use of Python Notebook magic. It simply works without prepending %bash.

因为默认情况下,Jupyter笔记本可以解析cd命令。注意,这并没有利用Python笔记本的魔力。它只是在没有预挂%bash的情况下工作。

Considering that 99 times out of a 100 I am working in Docker using one of the Project Jupyter Docker images, the following modification is idempotent

考虑到我在Docker工作的99次使用的是Jupyter Docker的一个项目,下面的修改是幂等的

    In [1]: cd /home/jovyan

    In [2]: from lib.postgres import database_connection

#3


1  

So far, the accepted answer has worked best for me. However, my concern has always been that there is a likely scenario where I might refactor the notebooks directory into subdirectories, requiring to change the module_path in every notebook. I decided to add a python file within each notebook directory to import the required modules.

到目前为止,公认的答案对我最有效。然而,我一直担心的是,有一种可能的情况是,我可能将notebook目录重构为子目录,需要更改每个notebook中的module_path。我决定在每个笔记本目录中添加一个python文件,以导入所需的模块。

Thus, having the following project structure:

因此,具有以下项目结构:

project
|__notebooks
   |__explore
      |__ notebook1.ipynb
      |__ notebook2.ipynb
      |__ project_path.py
   |__ explain
       |__notebook1.ipynb
       |__project_path.py
|__lib
   |__ __init__.py
   |__ module.py

I added the file project_path.py in each notebook subdirectory (notebooks/explore and notebooks/explain). This file contains the code for relative imports (from @metakermit):

我添加了文件project_path。每个笔记本子目录中的py(笔记本/探索和笔记本/解释)。此文件包含相对导入的代码(来自@metakermit):

import sys
import os

module_path = os.path.abspath(os.path.join(os.pardir, os.pardir))
if module_path not in sys.path:
    sys.path.append(module_path)

This way, I just need to do relative imports within the project_path.py file, and not in the notebooks. The notebooks files would then just need to import project_path before importing lib. For example in 0.0-notebook.ipynb:

这样,我只需要在project_path中执行相对导入。py文件,不在笔记本上。在导入lib之前,笔记本文件只需要导入project_path,例如在0-notebook.ipynb中:

import project_path
import lib

The caveat here is that reversing the imports would not work. THIS DOES NOT WORK:

这里的警告是,逆转进口不会起作用。这并不工作:

import lib
import project_path

Thus care must be taken during imports.

因此,进口时必须小心。