1

Something rather odd is happening in an interaction between bound methods, inheritance, and getattr that I am failing to understand.

I have a directory setup like:

/a
__init__.py
start_module.py
  /b
  __init__.py
  imported_module.py

imported_module.py contains a number of class objects one of which is of this form:

class Foo(some_parent_class):
    def bar(self):
       return [1,2,3]

A function in start_module.py uses inspect to get a list of strings representing the classes in imported_module.py. "Foo" is the first of those strings. The goal is to run bar in start_module.py using that string and getattr.*

To do this I use code in start_module of the form:

for class_name in class_name_list:
  instance = getattr(b.imported_module, class_name)()
  function = getattr(instance, "bar")
  for doodad in [x for x in function()]:
     print doodad

Which does successfully start to iterate over the list comprehension, but on the first string, "bar", I get a bizarre error. Despite bar being a bound method, and so as far as I understand expecting an instance of Foo as an argument, I am told:

TypeError: bar() takes no arguments (1 given)

This makes it seem like my call to function() is passing the Foo instance, but the code is not expecting to receive it.

I really have no idea what is going on here and couldn't parse out an explanation through looking on Google and Stack Overflow. Is the double getattr causing some weird interaction? Is my understanding of class objects in Python too hazy? I'd love to hear your thoughts.

*To avoid the anti-pattern, the real end objective is to have start_module.py automatically have access to all methods of name bar across a variety of classes similar to Foo in imported_module.py. I am doing this in the hopes of avoiding making my successors maintain a list for what could be a very large number of Foo-resembling classes.

Answered below: I think the biggest takeaways here are that inspect is very useful, and that if there is a common cause for the bug you are experiencing, make absolutely sure you've ruled that out before moving on to search for other possibilities. In this case I overlooked the fact that the module I was looking at that had correct code might not be the one being imported due to recent edits to the file structure.

Mark Garcia
  • 16,898
  • 3
  • 55
  • 94
Tim Wilder
  • 1,547
  • 1
  • 17
  • 26
  • `return = [1,2,3]` isn't valid Python. Please make sure you are posting the actual code you are using – John La Rooy Jan 10 '13 at 01:31
  • 2
    Probably a better idea to make a metaclass for `Foo` that can "register" all the `bar` eg. store them in a list – John La Rooy Jan 10 '13 at 01:32
  • Oops. Sorry about that. Fixing it. I added the class description in a very simplified form so as to make it legible and accidentally slipped an extra character in. – Tim Wilder Jan 10 '13 at 01:34
  • I agree with gnibbler. [Here](http://python-3-patterns-idioms-test.readthedocs.org/en/latest/Metaprogramming.html#example-self-registration-of-subclasses) is an example of that. – asermax Jan 10 '13 at 01:36
  • I do like that idea gnibbler, but the main point of this effort is to automatically get access to all Foo.bar whenever a Foo is added to the code without having the maintainer need to go to any additional effort or understand more of the code. – Tim Wilder Jan 10 '13 at 01:37
  • That and I am genuinely curious about what is going on here. – Tim Wilder Jan 10 '13 at 01:38
  • @Tim, that's why you use a metaclass. subclasses of `Foo` will inherit the metaclass. – John La Rooy Jan 10 '13 at 01:39
  • Asermax, that looks promising. If you had meant an approach like that originally gnibbler, I take back all reservations. I would like to know what is going on with the Exception though if anyone has any ideas. – Tim Wilder Jan 10 '13 at 01:40
  • I'm sold guys. I will do that. Any guesses on what is going wrong in this case? It's definitely not the omission of self in one of the functions definitions. – Tim Wilder Jan 10 '13 at 01:44
  • @Tim, I can't think of another way to get Python to tell you something takes no arguments. Maybe use the debugger and inspect to see what is going on. – John La Rooy Jan 10 '13 at 01:46
  • 2
    This doesn't address the main point of the question at all, but the list comprehension in `for doodad in [x for x in function()]` is completely unnecessary. Just use `for doodad in function()`! – Blckknght Jan 10 '13 at 03:19
  • Agreed blckknght. I don't know why I did that. Changed. – Tim Wilder Jan 10 '13 at 17:42

3 Answers3

3

Since the sample code you posted is wrong, I'm guessing that you have another module with the Foo class somewhere - maybe bar is defined like this

class Foo(object):
    def bar():   # <-- missing self parameter
        return [1,2,3]

This does give that error message

>>> Foo().bar()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: bar() takes no arguments (1 given)
John La Rooy
  • 281,034
  • 50
  • 354
  • 495
  • I wish that was the case. I just went backed and double checked again, and both bar and the function it overrides are coded with self included. Sorry about the sample code. – Tim Wilder Jan 10 '13 at 01:42
  • @TimWilder: (1) does `print inspect.getsource(function)` right before the `for` loop show you the code that it should? (2) Does your code work when run under `python -tt` (to rule out possible indentation errors due to inconsistent whitespace)? (3) If not, then I think you're going to have to construct an [SSCCE](http://sscce.org/). – DSM Jan 10 '13 at 01:48
  • I'll take a look tomorrow. Good suggestion. – Tim Wilder Jan 10 '13 at 07:36
  • 1
    The problem was that in my drowsiness I had moved imported_module.py, and was importing the moved copy with start_module. I was still editing the old copy. DSM's suggestion to use inspect revealed this very quickly. Gnibbler, your conviction was spot on. Thanks guys! – Tim Wilder Jan 10 '13 at 17:38
  • @TimWilder Thank you for writing that. Had the exact same problem, and exactly the same solution. – chwi Aug 11 '15 at 08:33
2

Class method have a self argument that is essentially automatically passed. It is just the class instance on which you are calling the method. You don't need to pass another parameter.

Volatility
  • 29,514
  • 10
  • 78
  • 87
2

I'm not able to reproduce the error you're getting. Here's my attempt at a short, self-contained compilable example, run from the Python shell:

>>> class Foo(object):
    def bar(self):
        print("Foo.bar!")

>>> import __main__ as mod
>>> cls = getattr(mod, "Foo")
>>> inst = cls()
>>> func = getattr(inst, "bar")
>>> func()
Foo.bar!

Perhaps you can try adapting your inspect based code to an example like this one and see where it is going wrong.

Blckknght
  • 93,977
  • 11
  • 112
  • 159