5

I am trying to put the back-end of a simple web-scraping application that I'm working on into a package, but this application relies on loading from a pickled python object, which I'm unable to load into the file with importlib. Before, when I had all the code in a single file, and relied on open(), all worked fine, but now I get an error when I import the package. After this error, I tried loading the file with importlib, and could not make it work.

I am following the steps in this answer from a similar question: How to read a (static) file from inside a Python package?.

The file structure in my package is:

mypackage\
        __init__.py
        parse.py
        search.py
        categories\
                categories.pickle
                generate_categories_if_corrupted.py

The content of init.py:

from %mymodule% import search

The code where the error happens:

import importlib.resources as resources
from pickle import load
from . import categories

try:
    with resources.open_binary(categories, "categories.pickle") as cat:
        CATS = load(cat)
except FileNotFoundError:
    raise FileNotFoundError("")

The error:

Traceback (most recent call last):
  File "%mypackage%\parse.py", line 15, in <module>
    with resources.open_binary(categories, "categories.pickle") as cat:
  File "C:\Users\%me%\AppData\Local\Programs\Python\Python38\lib\importlib\resources.py", line 92, in open_binary
    _check_location(package)
  File "C:\Users\%me%\AppData\Local\Programs\Python\Python38\lib\importlib\resources.py", line 82, in _check_location
    raise FileNotFoundError(f'Package has no location {package!r}')
FileNotFoundError: Package has no location <module '%mypackage%.categories' (namespace)>

During handling of the above exception, another exception occurred:

## just a  FileNotFoundError with an error message, as expected.

How do I fix this? It's my first time trying to make my code into a package in Python.

Thanks in advance for your answers.

Luiz Cavalcanti
  • 53
  • 4
  • out of curiosity, why are there % in the import line in init.py – ewong Jul 16 '20 at 02:30
  • I just didn't find that the name of my package was relevant to the question, so it's just a placeholder. There is a regular windows folder path in place of it. – Luiz Cavalcanti Jul 16 '20 at 03:26
  • you mean something like ```from c:\foobar import search```? – ewong Jul 16 '20 at 03:27
  • Not sure it's the issue, but I would add a `mypackage/categories/__init__.py` file (file can stay empty). -- Edit: I'd suggest reading [this answer](https://stackoverflow.com/a/58941536/11138259) completely, which indeed seems to suggest that the `__init__.py` file is needed for `importlib.resources` to do its job. – sinoroc Jul 16 '20 at 09:28

1 Answers1

6

As per this answer on a related question (you might want to go and upvote this answer), for importlib.resources to do its job there has to be a __init__.py file in the packages.

So in your case, I believe there should be a mypackage/categories/__init__.py file (as always, that file can be left empty but it has to exist).

sinoroc
  • 12,887
  • 1
  • 23
  • 54