How to clear RAM after deleteng all variables from python

Question

I have Django project where I need to get data from postgres DB and then manipulate with it (1GB in postgres). After that when I executed all postgres data what I needed my ram increase, after another operation like this RAM also increase and so on...

If I have 64GB RAM and after execution postgres data my RAM increase ~1GB then if I run that function 10 times my ram will be ~10GB

I tried to delete all variables from Python with del or gc, but after that I also have full RAM.

Only option to clear RAM is to close Django project.

Is there option to clear RAM after I execute data from postgres?

This is an example how I get data from postgres. I use psycopg2 library. This is Django view, and after I go to this it runs this peace of code.

from django.http import HttpResponse
import psycopg2

def index(request):

    con = psycopg2.connect(host="localhost", database="database", user="postgres", password="postgres")
    cur = con.cursor()

    text = "select * from table"
    cur.execute(text)

    text = cur.fetchall()

    cur.close()
    con.close()

    test_len = len(text)
    del text

    return HttpResponse(test_len)

Every time I refresh page I get all data from table And RAM increases

One more example:

when I run python3 mnanage.py runserver my ram is 2GB, after I refresh page it increases to 13GB and then drops to 7.7GB, and after that if I resresh again and again I still get RAM on 7.7GB, but after I am getting data I delete this variable. why I still get 7.7GB RAM?

CPython returns memory to the OS in rare conditions only, but it reuses the freed memory internally. — Klaus D., Apr 18 '22 at 13:29
Maybe there are builds other than CPython or PyPy that handle memory differently, but otherwise, I think Klaus's answer is the most relevant. In fact, `del` and manual garbage collection won't help you, they just give the memory back to python, but python doesn't give it back to the os. Maybe you want to limit the resources the process can use if you need to use it for other processes on the same machine. Otherwise, what's the harm in letting python manage it? — theherk, Apr 18 '22 at 13:45
Relevant [1](https://stackoverflow.com/q/15455048/5320906), [2](https://stackoverflow.com/q/1316767/5320906), some of https://stackoverflow.com/questions/tagged/python%2bmemory-management?tab=Votes — snakecharmerb, Apr 18 '22 at 15:17
Maybe that’s beside the point but all you seem to be achieving is a `select count(*) from table` … — JL Peyret, Apr 18 '22 at 17:05
@KlausD. even so shouldn’t the second and subsequent passes use the RAM reclaimed from the first pass? — JL Peyret, Apr 18 '22 at 17:08
@JLPeyret memory management is an inexact science. Sometimes the freed blocks of memory don't match the size of memory being allocated later. Sometimes there will be a non-freed block between two free blocks that keep them from being consolidated. This is all normal and expected. — Mark Ransom, Jul 30 '22 at 02:34

score 0 · Answer 1 · edited Jun 05 '23 at 13:42

0

You should use .iterator() after query.

You can read about it here

edited Jun 05 '23 at 13:42

lemon

14,875
6
18
38

answered Jun 05 '23 at 11:50

Sergey Ivanov

3
2

How to clear RAM after deleteng all variables from python

1 Answers1