86

Is there any rule about which built-in and standard library classes are not subclassable ("final")?

As of Python 3.3, here are a few examples:

  • bool
  • function
  • operator.itemgetter
  • slice

I found a question which deals with the implementation of "final" classes, both in C and pure Python.

I would like to understand what reasons may explain why a class is chosen to be "final" in the first place.

Community
  • 1
  • 1
max
  • 49,282
  • 56
  • 208
  • 355
  • 6
    `NoneType` is another example. – Duncan Apr 08 '12 at 09:57
  • 4
    Can a final class in one Python implementation be subclassable in another implementation? I hope someone can confirm that it never happens. Otherwise, code written for one implementation may break when ported to another (very painfully, too: imagine if someone subclassed `function`, and now needs to refactor the code to avoid this inheritance). – max Apr 09 '12 at 20:14
  • 3
    Note that PyPy refuses to subclass your four examples, too... even though it doesn't have the CPython restriction. They might have a reason documented in their codebase. – mbauman Apr 09 '12 at 21:24
  • 4
    `NotImplementedType` (i.e., `type(NotImplemented)`) and `ellipsis` (i.e., `type(...)`) are two more examples. Like `None`, there is no reason to have more than one instance of these classes, and if this was allowed, it would be more awkward to check for them (`if x is None` would have to become `if isinstance(x, type(None))`). I suppose, in principle, you could get a complete list by looking though the source for the value of `tp_flags` in type definitions. – James Apr 10 '12 at 11:46
  • Matt B: PyPy needs to run all python code. Even RPython can run on a python interpreter (which from everything I have seen on it seems like a TERRIBLE idea and a pain in the ass to code, but they make it work). Therefore it wouldn't make sense if they allowed code that couldn't be run in Python. – Garrett Berg Apr 10 '12 at 19:25
  • Are there any classes you _can't subclass_ where subclassing would be _useful_ and not _break anything_? Subclassing a singleton might break things, and subclassing `function` seems unnecessary -- you can use the `__call__` magic method instead. If the answer is "No", then the rule is that "only classes that can usefully be subclassed are subclassable". – agf Apr 11 '12 at 17:28
  • 2
    @agf: See [this post](http://code.activestate.com/lists/python-ideas/14590/) for why you'd want to subclass `itemgetter`; and [this post](http://grokbase.com/t/python/python-list/033r5nks47/type-function-does-not-subtype#20030324zd54avealbfyrkqlu5ke3yril4) for why you might want to subclass `function` (this one is old, so some of these arguments may not apply). – max Apr 11 '12 at 18:05
  • 1
    @max I don't see that as a reason to subclass `itemgetter` -- there is no use case for the current equality behavior (since if you want that behavior you use `is`) so there is no reason to sublcass rather than change the behavior, other than as a workaround. As far as `function` goes, all of that is either possible without subclassing `function`, or better done with a callable user-defined class instance. Yes, if everything was subclassable we'd have more flexibility, but flexibility without a use case isn't a reason in Python. – agf Apr 11 '12 at 18:15
  • 1
    @agf: on `itemgetter` agreed 100% that it's more reasonable (and much easier!) to fix the equality behavior than to do the work required to allow subclassing. The discussion of the `function` subclassing were too complicated for me to follow in full; if you feel the use case for subclassing there is weak, I'd take you word on it. Overall, it seems that your comments seem to translate directly into the two reasons we've identified so far: "break something" <=> "singleton pattern"; "not useful" <=> "insufficient interest". So I begin to feel that we're close to consensus. – max Apr 11 '12 at 19:30
  • @max I guess I was trying to say I think it's really insufficient _need_ rather than insufficient interest. The lack of interest is a consequence of the lack of need. It looks to me like the best resolution to this question would be for you to move your update into an answer, since I don't think we're going to get any closer than that. – agf Apr 11 '12 at 19:32
  • @agf: I see. Your view is stronger than just "insufficient interest"; more like no real use case at all. I have to agree based on what I've seen so far. – max Apr 11 '12 at 19:33
  • 1
    Python 2 `xrange` aka Python 2 `range` aka `six.moves.xrange` cannot be subclassed. – Bob Stein Jun 09 '17 at 20:13
  • `memoryview` is not an acceptable base type – AJNeufeld Jul 04 '18 at 21:17
  • `range` built-in function is an other important example,and it is discussed on this post : https://stackoverflow.com/q/30362799/8844500 – FraSchelle Jan 05 '21 at 10:51
  • TypeError: type '_frida.Process' is not an acceptable base type – CS QGB Oct 09 '22 at 09:34

2 Answers2

41

There seems to be two reasons for a class to be "final" in Python.

1. Violation of Class Invariant

Classes that follow Singleton pattern have an invariant that there's a limited (pre-determined) number of instances. Any violation of this invariant in a subclass will be inconsistent with the class' intent, and would not work correctly. Examples:

  • bool: True, False; see Guido's comments
  • NoneType: None
  • NotImplementedType: NotImplemented
  • ellipsis: Ellipsis

There may be cases other than the Singleton pattern in this category but I'm not aware of any.

2. No Persuasive Use Case

A class implemented in C requires additional work to allow subclassing (at least in CPython). Doing such work without a convincing use case is not very attractive, so volunteers are less likely to come forward. Examples:

Note 1:

I originally thought there were valid use cases, but simply insufficient interest, in subclassing of function and operator.itemgetter. Thanks to @agf for pointing out that the use cases offered here and here are not convincing (see @agf comments to the question).

Note 2:

My concern is that another Python implementation might accidentally allow subclassing a class that's final in CPython. This may result in non-portable code (a use case may be weak, but someone might still write code that subclasses function if their Python supports it). This can be resolved by marking in Python documentation all built-in and standard library classes that cannot be subclassed, and requiring that all implementations follow CPython behavior in that respect.

Note 3:

The message produced by CPython in all the above cases is:

TypeError: type 'bool' is not an acceptable base type

It is quite cryptic, as numerous questions on this subject show. I'll submit a suggestion to add a paragraph to the documentation that explains final classes, and maybe even change the error message to:

TypeError: type 'bool' is final (non-extensible)
Community
  • 1
  • 1
max
  • 49,282
  • 56
  • 208
  • 355
  • 5
    Of course if you were writing a simulation of Schrodinger's Cat, you might want to subclass `bool` to include `unknown` :-P just kidding – Endophage Apr 11 '12 at 21:47
  • 10
    @Endophage: a Qbit would certainly be a useful type, but it's not a sublcass of `bool`, in fact, quite the opposite, every single boolean value is a kind of qbit! one which happens to be observed. I'd probably handle that case with `__subclasscheck__` and friends, though. – SingleNegationElimination Apr 11 '12 at 21:51
  • 5
    @TokenMacGuy excellent observation! So why don't we have a Qbit class in Python that bool is a subclass of! I demand it should be so! :-P – Endophage Apr 12 '12 at 01:10
  • But the value of a Qbit will always be False or True when observed/tested. Seems you just need a subclass of bool that does its "calculation" only on observation.... (`LazyBool`) – DylanYoung Jan 24 '20 at 15:27
  • 2
    @SingleNegationElimination @Endophage Maybe that's the reason why `bool` cannot be an `Enum` base class, Guido does not play dice – Nuno André Mar 21 '21 at 23:54
2

if we check for the builtins (excluding errors, dunder methods, warnings; one could include them also if required)

import keyword, re
x = sorted([i for i in list(keyword.__builtins__) if not re.search('.*Error|Warning|__', i)], key=len)

and then run,

l1 = []
l2 = []
for i in x:
  try:
    A = type('A', (eval(i),), {})
    l1.append(i)
  except TypeError:
    l2.append(i)

then,

l1

gives,

['int', 'map', 'set', 'str', 'zip', 'dict', 'list', 'type', 'bytes', 'float',
 'super', 'tuple', 'filter', 'object', 'complex', 'property', 'reversed',
 'bytearray', 'enumerate', 'frozenset', 'Exception', 'SystemExit',
 'classmethod', 'staticmethod', 'BaseException', 'StopIteration',
 'GeneratorExit', 'KeyboardInterrupt', 'StopAsyncIteration']

while,

l2

gives,

['id', 'abs', 'all', 'any', 'bin', 'chr', 'dir', 'hex', 'len', 'max', 'min',
 'oct', 'ord', 'pow', 'sum', 'eval', 'exec', 'hash', 'iter', 'next', 'repr',
 'vars', 'None', 'True', 'bool', 'open', 'help', 'ascii', 'input', 'print',
 'round', 'False', 'range', 'slice', 'divmod', 'format', 'locals', 'sorted',
 'compile', 'delattr', 'getattr', 'globals', 'hasattr', 'setattr', 'credits',
 'license', 'display', 'runfile', 'dreload', 'callable', 'Ellipsis', 'execfile',
 'copyright', 'breakpoint', 'isinstance', 'issubclass', 'memoryview',
 'get_ipython', 'NotImplemented']

the list l1 contains builtins which could act as a base class.

apostofes
  • 2,959
  • 5
  • 16
  • 31
  • `l2` also includes objects that are not types, like `id`, which is a function, and `False`. – wjandrea Mar 08 '22 at 20:45
  • Why are you using `keyword.__builtins__` instead of [`builtins`](https://docs.python.org/3/library/builtins.html)? – wjandrea Mar 08 '22 at 20:46
  • `get_ipython` shouldn't be in there. That's only provided by IPython. – wjandrea Mar 08 '22 at 20:50
  • 1
    yes, I changed it, `get_ipython` is because I use Ipython notebook. `import builtins` also gives the same result. – apostofes Mar 08 '22 at 21:06
  • Could I suggest a better solution? [Here's a gist](https://gist.github.com/wjandrea/8333862d56e4ca2abe6c7c1fc8d0a1a8). I added a check that the builtin is a type, as well as using `builtins`, try-else, clearer names, and a few other minor improvements. – wjandrea Mar 08 '22 at 21:24
  • should I make those changes to my answer – apostofes Mar 09 '22 at 04:56
  • Yeah, by all means! That's why I commented :) You can copy my code verbatim if you want, just give me attribution please. (Name and link to user profile. And if you want to get formal, I'll license it under CC BY-SA 4.0.) – wjandrea Mar 09 '22 at 19:35