How can I separate the functions of a class into multiple files?

Question

I have a main class that has a ton of different functions in it. It's getting hard to manage. I'd like to be able to separate those functions into a separate file, but I'm finding it hard to come up with a good way to do so.

Here's what I've done so far:

File main.py

import separate

class MainClass(object):
    self.global_var_1 = ...
    self.global_var_2 = ...

    def func_1(self, x, y):
        ...
    def func_2(self, z):
        ...
    # tons of similar functions, and then the ones I moved out:

    def long_func_1(self, a, b):
        return separate.long_func_1(self, a, b)

File separate.py

def long_func_1(obj, a, b):
    if obj.global_var_1:
        ...
    obj.func_2(z)
    ...
    return ...
# Lots of other similar functions that use info from MainClass

I do this because if I do:

obj_1 = MainClass()

I want to be able to do:

obj_1.long_func_1(a, b)

instead of:

separate.long_func_1(obj_1, a, b)

I know this seems kind of nit-picky, but I want just about all of the code to start with obj_1., so there isn't confusion.

Is there a better solution that what I'm currently doing? The only issues that I have with my current setup are:

1. I have to change arguments for both instances of the function
2. It seems needlessly repetitive

_{I know this has been asked a couple of times, but I couldn't quite understand the previous answers and/or I don't think the solution quite represents what I'm shooting for. I'm still pretty new to Python, so I'm having a tough time figuring this out.}

Answer 1

Here is how I do it:

1. Class (or group of) is actually a full module. You don't have to do it this way, but if you're splitting a class on multiple files I think this is 'cleanest' (opinion).

2. The definition is in __init__.py, methods are split into files by a meaningful grouping.

3. A method file is just a regular Python file with functions, except you can't forget 'self' as a first argument. You can have auxiliary methods here, both taking self and not.

4. Methods are imported directly into the class definition.

Suppose my class is some fitting GUI (this is actually what I did this for first time). So my file hierarchy may look something like

mymodule/
     __init__.py
     _plotstuff.py
     _fitstuff.py
     _datastuff.py

So plot stuff will have plotting methods, fit stuff contains fitting methods, and data stuff contains methods for loading and handling of data - you get the point. By convention I mark the files with a _ to indicate these really aren't meant to be imported directly anywhere outside the module. So _plotsuff.py for example may look like:

def plot(self,x,y):
     #body
def clear(self):
     #body

etc. Now the important thing is file __init__.py:

class Fitter(object):
     def __init__(self,whatever):
         self.field1 = 0
         self.field2 = whatever

     # Imported methods
     from ._plotstuff import plot, clear
     from ._fitstuff  import fit
     from ._datastuff import load

     # static methods need to be set
     from ._static_example import something
     something = staticmethod(something)

     # Some more small functions
     def printHi(self):
         print("Hello world")

Tom Sawyer mentions PEP-8 recommends putting all imports at the top, so you may wish to put them before __init__, but I prefer it this way. I have to say, my Flake8 checker does not complain, so likely this is PEP-8 compliant.

Note the from ... import ... is particularly useful to hide some 'helper' functions to your methods you don't want accessible through objects of the class. I usually also place the custom exceptions for the class in the different files, but import them directly so they can be accessed as Fitter.myexception.

If this module is in your path then you can access your class with

from mymodule import Fitter
f = Fitter()
f.load('somefile') # Imported method
f.plot()           # Imported method

It is not completely intuitive, but not too difficult either. The short version for your specific problem was you were close - just move the import into the class, and use

from separate import long_func_1

and don't forget your self!

Answer 2

I use the approach I found here. It shows many different approaches, but if you scroll down to the end, the preferred method is to basically go the opposite direction of @Martin Pieter's suggestion which is have a base class that inherits other classes with your methods in those classes.

So the folder structure is something like:

_DataStore/
    __init__.py
    DataStore.py
    _DataStore.py

So your base class would be:

File DataStore.py

import _DataStore

class DataStore(_DataStore.Mixin): # Could inherit many more mixins

    def __init__(self):
        self._a = 1
        self._b = 2
        self._c = 3

    def small_method(self):
        return self._a

Then your Mixin class:

File _DataStore.py

class Mixin:

    def big_method(self):
        return self._b

    def huge_method(self):
        return self._c

Your separate methods would be located in other appropriately named files, and in this example it is just _DataStore.

I am interested to hear what others think about this approach. I showed it to someone at work and they were scared by it, but it seemed to be a clean and easy way to separate a class into multiple files.

Answer 3

Here is an implementation of Martijn Pieters's comment to use subclasses:

File main.py

from separate import BaseClass

class MainClass(BaseClass):
    def long_func_1(self, a, b):
        if self.global_var_1:
            ...
        self.func_2(z)
        ...
        return ...
    # Lots of other similar functions that use info from BaseClass

File separate.py

class BaseClass(object):

    # You almost always want to initialize instance variables in the `__init__` method.
    def __init__(self):
        self.global_var_1 = ...
        self.global_var_2 = ...

    def func_1(self, x, y):
        ...
    def func_2(self, z):
        ...
    # tons of similar functions, and then the ones I moved out:
    #
    # Why are there "tons" of _similar_ functions?
    # Remember that functions can be defined to take a
    # variable number of/optional arguments, lists/tuples
    # as arguments, dicts as arguments, etc.

from main import MainClass
m = MainClass()
m.func_1(1, 2)
....