Django: ThreadPool and TooManyConnections

Question

Context: Django(3) commands, MySQL

from django.db import connections

...
def func1(a):
    with connections['second_db'].cursor() as cursor:
        cursor.execute(statement)

def func2(b):
    with connections['second_db'].cursor() as cursor:
        cursor.execute(statement)

for ...:
    pool = ThreadPool(processes=2)
    result1_proc = pool.apply_async(func1, args=(a))
    result2_proc = pool.apply_async(func2, args=(b))
    pool.close()
    pool.join()

At some point i will get "Too many connections" error from server. If i add "connections['second_db'].close()" at the end of each function, i get the "django.db.utils.InterfaceError: (0, '')".
What is wrong?

score 0 · Answer 1 · answered Jun 30 '20 at 09:15

0

Too many connections is a MySQL error, not a django error. You can upadte the param in :

# vi /etc/my.cnf
max_connections = 512

ref: https://www.thegeekdiary.com/mysql-error-too-many-connections-and-how-to-resolve-it/

answered Jun 30 '20 at 09:15

mansuetus

782
5
17

I want to solve the problem, not the symptoms.Shouldn't the connections be closed at the threads death? – Stefan Weiss Jun 30 '20 at 09:39
I saw the "for" in your sample, indeed, the root cause is to be analysed. I saw this (you probably have too) https://stackoverflow.com/questions/5504340/python-mysqldb-connection-close-vs-cursor-close : closing cursor seems to be required... did you try "cursor.close()" after each cursor.execute() ? – mansuetus Jun 30 '20 at 09:43

Django: ThreadPool and TooManyConnections

1 Answers1