Implementation of NoneType, Reasons and Details

Question

I recently read somewhere that the special value None in python is a singleton object of its own class, specifically NoneType. This explained a lot, since most errors involving None in python produce AttributeErrors instead of some special "NoneError" or something.

Since all of these AttributeErrors reflected the attributes that NoneType lacked, I became intrigued by what attributes NoneType did have, if any.

I decided to look into this NoneType and learn more about it. I've always found the best way to learn about a new language feature is to use it, so I tried instantiating NoneType in IDLE:

>>> n = NoneType()

This produced an error:

Traceback (most recent call last):
File "<pyshell#0>", line 1, in <module>
    n = NoneType()
NameError: name 'NoneType' is not defined

Confused, I inspected None to see if I'd gotten the type name correct. Sure enough,

>>> type(None)
<class 'NoneType'>

Now very confused, I did a quick google search. This revealed that for some reason NoneType was somehow removed in Python 3.

Well I though, ha ha! I can work around this by storing the type of None in a variable, since classes are objects in python. This seemed to work:

>>> NoneType = type(None)
>>> n = NoneType()

And when I printed n, I got pretty much what I was expecting:

>>> print(n)
None

But then this happened:

>>> n is None
True

And:

>>> id(n)
506768776
>>> id(None)
506768776

My variable n IS None. Not only the same type as None. It IS None. This is not what I expected.

I tried using dis to get more info on NoneType, but when I called

>>> dis.dis(type(None))

It produced no output.

I then then tried investigating the __new__ method, which several users had mentioned in the comments:

dis.dis(type(None).__new__)
Traceback (most recent call last):
  File "<pyshell#4>", line 1, in <module>
    dis.dis(type(None).__new__)
  File "C:\Python33\lib\dis.py", line 59, in dis
    type(x).__name__)
TypeError: don't know how to disassemble builtin_function_or_method objects
>>> 

More errors.

Here are my questions:

• Why is n the exact same Object as None?
• Why was the language designed such that n is the exact same Object as None?
• How would one even implement this behavior in python?

Answer 1

Other answers describe how to use __new__ to implement a singleton, but that's not how None is actually implemented (in cPython at least, I haven't looked into other implementations).

Trying to create an instance of None through type(None)() is special cased, and ends up calling the following C function:

static PyObject *
none_new(PyTypeObject *type, PyObject *args, PyObject *kwargs)
{
    if (PyTuple_GET_SIZE(args) || (kwargs && PyDict_Size(kwargs))) {
        PyErr_SetString(PyExc_TypeError, "NoneType takes no arguments");
        return NULL;
    }
    Py_RETURN_NONE;
}

And Py_RETURN_NONE is defined here:

/*
_Py_NoneStruct is an object of undefined type which can be used in contexts
where NULL (nil) is not suitable (since NULL often means 'error').

Don't forget to apply Py_INCREF() when returning this value!!!
*/
PyAPI_DATA(PyObject) _Py_NoneStruct; /* Don't use this directly */
#define Py_None (&_Py_NoneStruct)

/* Macro for returning Py_None from a function */
#define Py_RETURN_NONE return Py_INCREF(Py_None), Py_None

Contrast this with the function that creates a normal python object:

PyObject *
_PyObject_New(PyTypeObject *tp)
{
    PyObject *op;
    op = (PyObject *) PyObject_MALLOC(_PyObject_SIZE(tp));
    if (op == NULL)
        return PyErr_NoMemory();
    return PyObject_INIT(op, tp);
}

When you create a normal object, memory for the object is allocated and initialized. When you try to create a new instance of None, all you get is a reference to the already existing _Py_NoneStruct. That's why, no matter what you do, every reference to None will be the exact same object.

Answer 2

Why is n the exact same Object as None?

The C implementation keeps a singleton instance. NoneType.__new__ is returning the singleton instance.

Why was the language designed such that n is the exact same Object as None?

If there was not a singleton instance, then you could not rely on the check x is None since the is operator is based on identity. Although None == None is also True, it's possible to have x == None be True when x is not actually None. See this answer for an example.

How would one even implement this behavior in python?

You can implement this pattern by overridding __new__. Here's a basic example:

class Singleton(object):
  _instance = None
  def __new__(cls, *args, **kwargs):
    if Singleton._instance is None:
      Singleton._instance = object.__new__(cls, *args, **kwargs)
    return Singleton._instance

if __name__ == '__main__':
  s1 = Singleton()
  s2 = Singleton()
  print 's1 is s2:', s1 is s2
  print 'id(s1):', id(s1)
  print 'id(s2):', id(s2)

Output:

s1 is s2: True
id(s1): 4506243152
id(s2): 4506243152

Of course this simple example doesn't make it impossible to create a second instance.

Answer 3

Why is n the exact same Object as None?

Many immutable objects in Python are interned including None, smaller ints, and many strings.

Demo:

>>> s1='abc'
>>> s2='def'
>>> s3='abc'
>>> id(s1)
4540177408
>>> id(s3)
4540177408    # Note: same as s1
>>> x=1
>>> y=2
>>> z=1
>>> id(x)
4538711696
>>> id(z)
4538711696    # Note: same as x

Why was the language designed such that n is the exact same Object as None?

See above -- speed, efficiency, lack of ambiguity and memory usage among other reasons to intern immutable objects.

How would one even implement this behavior in python?

Among other ways, you can override __new__ to return the same object:

class Singleton(object):
    _instance = None
    def __new__(cls, *args, **kwargs):
        if not cls._instance:
            cls._instance = super(Singleton, cls).__new__(
                                cls, *args, **kwargs)
        return cls._instance

For strings, you can call intern on Python 2 or sys.intern on Python 3

Answer 4

1. The NoneType overrides __new__ which always return the same singleton. The code is actually written in C so dis cannot help, but conceptually it's just like this.

2. Having only one None instance is easier to deal with. They are all equal anyway.

3. By overriding __new__... e.g.

   class MyNoneType(object):
       _common_none = 0
       def __new__(cls):
           return cls._common_none
   
   MyNoneType._common_none = object.__new__(MyNoneType)
   
   m1 = MyNoneType()
   m2 = MyNoneType()
   print(m1 is m2)