Use type comments to narrow typing of already declared Python variable

Question

How can one use type comments in Python to change or narrow the type of an already declared variable, in such a way as to make pycharm or other type-aware systems understand the new type.

For instance, I might have two classes:

class A:
   is_b = False
   ...

class B(A):
   is_b = True

   def flummox(self):
       return '?'

and another function elsewhere:

def do_something_to_A(a_in: A):
    ...
    if a_in.is_b:
       assert isinstance(a_in, B)  # THIS IS THE LINE...
       a_in.flummox()

As long as I have the assert statement, PyCharm will understand that I've narrowed a_in to be of class B, and not complain about .flummox(). Without it, errors/warnings such as a_in has no method flummox will appear.

The question I have is, is there a PEP 484 (or successor) way of showing that a_in (which might have originally been of type A or B or something else) is now of type B without having the assert statement. The statement b_in : B = a_in also gives type errors.

In TypeScript I could do something like this:

if a_in.is_b:
   const b_in = <B><any> a_in;
   b_in.flummox()

// or

if a_in.is_b:
   (a_in as B).flummox()

There are two main reasons I don't want to use the assert line is (1) speed is very important to this part of code, and having an extra is_instance call for every time the line is run slows it down too much, and (2) a project code style that forbids bare assert statements.

A question: Why is `A`, a parent, even aware of the existence of it child class `B` such that is has a method to determine whether or not an instance of it is in fact a `B`? Would it not be clearer to rely on the type system directly and replace `is_b` with `ininstance` calls? — Brian61354270, Feb 17 '20 at 23:27
the `is_b` is for simplification. It could be any other function or anything (and I'm writing the `do_something_to_A` function, not designing the classes A or B themself. And `isinstance` is really slow compared to attribute lookup (faster in recent Python, but still about 3x slower than checking a property) — Michael Scott Asato Cuthbert, Feb 17 '20 at 23:33
I'm confused why you'd add code for the sole purpose of avoiding warnings in the editor. — rcriii, Feb 17 '20 at 23:40
This is totally unnecessary and bad style; the `is_b` and `assert`. If you only want to avoid an exception if `flummox()` is ever called on an instance of base class `A`, then just declare a default base implementation `def flummox(self): return None` in class `A`, and override it with the real implementation in class `B`. You don't need type annotations and you don't need to placate the editor from objecting to a non-existent method on `A`. — smci, Feb 18 '20 at 00:48

Brian61354270 · Accepted Answer · 2020-02-18T00:32:10.323

So long as you are using Python 3.6+, you can "re-annotate" the type of a variable arbitrarily using the same syntax as you would use to "declare" the type of a variable without initializing it (PEP 526).

In the example you have provided, the following snippet has the behavior you expect:

def do_something_to_A(a_in: A):
    ...
    if a_in.is_b:
       a_in: B
       a_in.flummox()

I have tested that this technique is properly detected by PyCharm 2019.2.

It is worth noting that this incurs no runtime cost since the same bytecode is generated with or without this added annotation statement. Given the following defintions,

def do_something_with_annotation(a_in: A): 
     if a_in.is_b: 
        a_in: B 
        a_in.flummox() 


def do_something_without_annotation(a_in: A): 
     if a_in.is_b: 
        a_in.flummox()

dis produce the following bytecode:

>>> dis.dis(do_something_with_annotation)
  3           0 LOAD_FAST                0 (a_in)
              2 LOAD_ATTR                0 (is_b)
              4 POP_JUMP_IF_FALSE       14

  5           6 LOAD_FAST                0 (a_in)
              8 LOAD_ATTR                1 (flummox)
             10 CALL_FUNCTION            0
             12 POP_TOP
        >>   14 LOAD_CONST               0 (None)
             16 RETURN_VALUE
>>> dis.dis(do_something_without_annotation)
  3           0 LOAD_FAST                0 (a_in)
              2 LOAD_ATTR                0 (is_b)
              4 POP_JUMP_IF_FALSE       14

  4           6 LOAD_FAST                0 (a_in)
              8 LOAD_ATTR                1 (flummox)
             10 CALL_FUNCTION            0
             12 POP_TOP
        >>   14 LOAD_CONST               0 (None)
             16 RETURN_VALUE

As a side note, you could also keep the assertion statements and disable assertions in your production environment by invoking the interpreter with the -O flag. This may or may not be considered more readable by your colleagues, depending on their familiarity with type hinting in Python.

Thank you Brian -- this is actually what I was looking for, but for some reason had only tried it with type comments. Just to add that it suppressed warnings in PyCharm also with `a_in: 'B'` in cases where `B` cannot be imported for whatever reason (often the other reason not to do `isinstance()` checking) — Michael Scott Asato Cuthbert, Feb 18 '20 at 19:41

Use type comments to narrow typing of already declared Python variable

1 Answers1