6

When a package becomes large, it can be hard to remember where things are, and dot-paths to the object we want can be cumbersome. One way that authors seem to address this is to bring references to the "best of" objects to the top, though there code can actually live several package levels below.

This allows one to say:

from pandas import wide_to_long

instead of

from pandas.core.reshape.melt import wide_to_long

But what are the ins and outs of doing this, and the best practices around the method? Doesn't loading the top __init__.py with many imports (in order to make them available at the top level) mean that any import of a single object of the package suddenly takes much more memory than needed - since everything mentioned in the __init__.py is automatically loaded?

Yet, packages do it. See, for example, what can be imported from top level numpy or pandas below (code to run your own diagnosis can be found in this gist).

$ python print_top_level_diagnosis.py numpy
--------- numpy ---------
599 objects can be imported from top level numpy:
  19 modules
  300 functions
  104 types

depth   count
0   162
1   406
2   2
3   29
4   1

$ python print_top_level_diagnosis.py pandas
--------- pandas ---------
115 objects can be imported from top level pandas:
  12 modules
  55 functions
  40 types

depth   count
0   12
3   37
4   65
5   1
WolfGirl
  • 39
  • 9
thorwhalen
  • 1,303
  • 9
  • 19
  • 2
    I think it depends on the project. Some simple ones may need to load resources at the entry-level init but other larger projects should probably remain modular and only import when needed or have the developer import a module within the package like package.module1.module2. Django is pretty big and it uses the many dot functionality. I prefer this because it means I know where I'm importing from – bherbruck May 14 '20 at 15:36
  • Related: [Can I put a class definition into __init__.py?](https://stackoverflow.com/q/47740935/6862601) – codeforester Mar 05 '21 at 03:14
  • 1
    Memory is really not a concern here. Usually, in any case, most of a module/package is loaded into memory, you can't just pick and choose single objects. What changes is *what objects are available in what namespace* – juanpa.arrivillaga Mar 05 '21 at 03:32
  • 2
    Some of the best practices are discussed in this answer by the most amazing Aaron Hall: [Can someone explain __all__ in Python?](https://stackoverflow.com/a/35710527/6862601). – codeforester Mar 05 '21 at 05:42

0 Answers0