What's Python good practice for importing and offering optional features?

Question

I'm writing a piece of software over on github. It's basically a tray icon with some extra features. I want to provide a working piece of code without actually having to make the user install what are essentially dependencies for optional features and I don't actually want to import things I'm not going to use so I thought code like this would be "good solution":

---- IN LOADING FUNCTION ----
features = []

for path in sys.path:
       if os.path.exists(os.path.join(path, 'pynotify')):
              features.append('pynotify')
       if os.path.exists(os.path.join(path, 'gnomekeyring.so')):
              features.append('gnome-keyring')

#user dialog to ask for stuff
#notifications available, do you want them enabled?
dlg = ConfigDialog(features)

if not dlg.get_notifications():
    features.remove('pynotify')


service_start(features ...)

---- SOMEWHERE ELSE ------

def service_start(features, other_config):

        if 'pynotify' in features:
               import pynotify
               #use pynotify...

There are some issues however. If a user formats his machine and installs the newest version of his OS and redeploys this application, features suddenly disappear without warning. The solution is to present this on the configuration window:

if 'pynotify' in features:
    #gtk checkbox
else:
    #gtk label reading "Get pynotify and enjoy notification pop ups!"

But if this is say, a mac, how do I know I'm not sending the user on a wild goose chase looking for a dependency they can never fill?

The second problem is the:

if os.path.exists(os.path.join(path, 'gnomekeyring.so')):

issue. Can I be sure that the file is always called gnomekeyring.so across all the linux distros?

How do other people test these features? The problem with the basic

try:
    import pynotify
except:
    pynotify = disabled

is that the code is global, these might be littered around and even if the user doesn't want pynotify....it's loaded anyway.

So what do people think is the best way to solve this problem?

Answer 1

The try: method does not need to be global — it can be used in any scope and so modules can be "lazy-loaded" at runtime. For example:

def foo():
    try:
        import external_module
    except ImportError:
        external_module = None 

    if external_module:
        external_module.some_whizzy_feature()
    else:
        print("You could be using a whizzy feature right now, if you had external_module.")

When your script is run, no attempt will be made to load external_module. The first time foo() is called, external_module is (if available) loaded and inserted into the function's local scope. Subsequent calls to foo() reinsert external_module into its scope without needing to reload the module.

In general, it's best to let Python handle import logic — it's been doing it for a while. :-)

Answer 2

You might want to have a look at the imp module, which basically does what you do manually above. So you can first look for a module with find_module() and then load it via load_module() or by simply importing it (after checking the config).

And btw, if using except: I always would add the specific exception to it (here ImportError) to not accidently catch unrelated errors.

Answer 3

Not sure if this is good practice, but I created a function that does the optional import (using importlib) and error handling:

def _optional_import(module: str, name: str = None, package: str = None):
    import importlib
    try:
        module = importlib.import_module(module)
        return module if name is None else getattr(module, name)
    except ImportError as e:
        if package is None:
            package = module
        msg = f"install the '{package}' package to make use of this feature"
        raise ValueError(msg) from e

If an optional module is not available, the user will at least get the idea what to do. E.g.

# code ...

if file.endswith('.json'):
    from json import load
elif file.endswith('.yaml'):
    # equivalent to 'from yaml import safe_load as load'
    load = _optional_import('yaml', 'safe_load', package='pyyaml')

# code using load ...

The main disadvantage with this approach is that your imports have to be done in-line and are not all on the top of your file. Therefore, it might be considered better practice to use a slight adaptation of this function (assuming that you are importing a function or the like):

def _optional_import_(module: str, name: str = None, package: str = None):
    import importlib
    try:
        module = importlib.import_module(module)
        return module if name is None else getattr(module, name)
    except ImportError as e:
        if package is None:
            package = module
        msg = f"install the '{package}' package to make use of this feature"
        import_error = e

        def _failed_import(*args):
            raise ValueError(msg) from import_error

        return _failed_import

Now, you can make the imports with the rest of your imports and the error will only be raised when the function that failed to import is actually used. E.g.

from utils import _optional_import_  # let's assume we import the function
from json import load as json_load
yaml_load = _optional_import_('yaml', 'safe_load', package='pyyaml')

# unimportant code ...

with open('test.txt', 'r') as fp:
    result = yaml_load(fp)    # will raise a value error if import was not successful

PS: sorry for the late answer!

Answer 4

One way to handle the problem of different dependencies for different features is to implement the optional features as plugins. That way the user has control over which features are activated in the app but isn't responsible for tracking down the dependencies herself. That task then gets handled at the time of each plugin's installation.