Modular class methods with type hints

Question

In this question How can I separate functions of class into multiple files? the top answer suggests to use

from method_file import method

Inside a class definition to have class methods defined in separate files. However, for a class like this

my_number.py

class MyNumber:
    def __init__(self):
        self.x = 5

    from my_method import my_method

my_method.py

def my_method(self):
    print(self.x)

It would not be clear to the IDE that self refers to a MyNumber object. As a consequence code completion (for e.g. self.x) is not available in my_method. A type hint for self could solve this, i.e.

my_method.py

from my_number import MyNumber

def my_method(self: MyNumber):
    print(self.x)

but this leads to a circular import.

Is there any workaround or best practice for such a situation?

Answer 1

There is an approach that combines a __future__ import to disregard type annotations at runtime, with a if TYPE_CHECKING clause that "imports" the code from your IDE's point of view only, so that code completion is available.

Example:

my_number.py

class MyNumber:
    def __init__(self):
        self.x = 5

    from my_method import my_method

my_method.py

from __future__ import annotations
from typing import TYPE_CHECKING

if TYPE_CHECKING:
    from my_number import MyNumber

def my_method(self: MyNumber):
    print(self.x)

With the from __future__ import annotations, we postpone the evaluation of type hints - in other words, we can type hint my_method even if we don't actually import MyNumber. This behavior was planned to be the default in Python 3.10, but it got postponed, so we need this import for now.

Now depending on your editor/IDE, you will still get a warning complaining that MyNumber isn't defined, and its methods/attributes may not show up on the autocomplete. Here's where the TYPE_CHECKING comes into play: this is simply a constant False value, which means that our clause is:

if False:
    from my_number import MyNumber

In other words, we're "tricking" the IDE into thinking we're importing MyNumber, but in reality that line never executes. Thus we avoid the circular import altogether.

This might feel a little hacky, but it works :-) the whole point of the TYPE_CHECKING constant is to allow type checkers to do their job, while not actually importing code at runtime, and to do so in a clear way (Zen of Python: There should be one-- and preferably only one --obvious way to do it).

This approach has worked for me consistently in PyCharm, not sure about other IDEs/editors.

Answer 2

Is there any workaround or best practice for such a situation?

The best practice is not to do this. If a method implementation is specific to a class, it should be part of the class definition.

---

If a method is not specific to a class, it should be defined across all valid types. A Protocol is appropriate to express this :

from typing import Protocol, Any

class HasX(Protocol):
    x: Any  # might need a TypeVar for complex cases

def my_method(self: HasX):
    print(self.x)

If a method extends a class separate of its definition, it should not be patched in. Use functools.singledispatch to externally define single dispatch functions, which are logically similar to methods:

from functools import singledispatch
from my_number import MyNumber

# not imported into MyNumber
@singledispatch
def my_method(self):
    raise NotImplementedError(f"no dispatch for {type(self}") 

@my_method.register
def _(self: MyNumber):
    print(self.x)

Answer 3

Typically, the self keyword is used to represent an instance of the class. It doesn't make sense to type hint self. Second, you can't access the instance variable x via self if your function is not a method to a class that is a MyNumber object.

I would suggest two options to accomplish what you want to. You can accept a MyNumber object as a parameter to the my_method() function or you can create a new class and inherit the MyNumber class. Make sure the files are in the same directory, otherwise update the import statement in File 2.

Option #1

class MyNumber:
    def __init__(self):
        self.x = 5

def my_method(my_number: MyNumber):
    print(my_number.x)

my_method(MyNumber())

Option #2

#my_number.py
class MyNumber:
    def __init__(self):
        self.x = 5

#my_method.py
from my_number import MyNumber

class MyMethod(MyNumber):
    def __init__(self):
        super().__init__()
        
    def my_method(self):
        print(self.x)

MyMethod().my_method()

Answer 4

I think you are having problems regarding object-oriented concepts in python. Your "my_method" function doesn't need the "self: MyNumber" as a parameter, in fact, you need to create an object of the MyNumber class and consequently this class will have an attribute that is the "x" since you defined the "x" in the constructor of the MyNumber class. It would look something like this:

#my_number.py
class MyNumber:
  def __init__(self):
    self.x = 5

#my_method.py
from my_number import MyNumber
def my_method():
  mm = MyNumber()
  print(mm.x)