4

I have the following code:

import gc
import resource
import time

gc.set_debug(gc.DEBUG_SAVEALL)

while True:
    class A():
        pass

    print(resource.getrusage(resource.RUSAGE_SELF).ru_maxrss)
    print("Garbage: ", len(gc.garbage))
    print("Collected: ", gc.collect())
    time.sleep(1)

and when i run it for a few iterations, i get the following:

9916
Garbage:  0
Collected:  0
9916
Garbage:  0
Collected:  5
9916
Garbage:  5
Collected:  5
9916
Garbage:  10
Collected:  5
9916
Garbage:  15
Collected:  5

As we can see, the garbage in memory starts to grow and the garbage collector can't collect the allocated memory. As a result, we have just created a memory leak.

My question is why this memory leak happens?

Does it have to do with the way class definitions are stored in the heap?

pakallis
  • 181
  • 6
  • 1
    Interestingly enough, I'm not able to reproduce this in Python 2.7 – mwarrior Apr 14 '20 at 15:25
  • I don't believe class definitions have a reference counter for object instances. They are probably never fully cleared so that any remaining object instance can still find its definition. Given that this is definition, not data, I don't see a big issue with it. Scenarios where you would dynamically redefine classes (let alone in a loop) aren't your typical use case. – Alain T. Apr 14 '20 at 15:29
  • Its interesting to add `for x in gc.garbage: print(id(x), type(x), x)` – tdelaney Apr 14 '20 at 15:37
  • @AlainT. - object instances reference their class with `self.__class__` and that bumps the refcount. – tdelaney Apr 14 '20 at 15:50
  • 1
    Interesting post - https://stackoverflow.com/questions/36787603/what-exactly-is-weakref-in-python. It seems like python is keeping a weakref to the class each time it is rebuilt in the loop. Its looking like a memory leak to me and worth a bug report. If class definitions within a function are allowed, they should operate sanely. – tdelaney Apr 14 '20 at 15:59
  • Each time you define `A` you create a new type and types don't go away. – tdelaney Apr 14 '20 at 16:04
  • I filed a bug report in https://bugs.python.org/issue40292. The ticket was closed as it is expected behavior of gc.set_debug – pakallis Apr 15 '20 at 15:33

0 Answers0