Why can I create more threads than "free memory" / "thread stack size"?

Question

In Linux the maximum number of threads is defined as max_threads = mempages / (8 * THREAD_SIZE / PAGE_SIZE);, and can be retreived by calling cat /proc/sys/kernel/threads-max. This returns around 14,000 for my raspberry Pi 3. However, when I just create threads in a loop with pthread_create(),(which are empty), I can create only 250, before I getENOMEM (Cannot allocate memory).

Now I looked at the default stack that is allocated to a process or thread, and that is 8192k. So at around 250 threads I would be using 2GB memory. However, in my mind this also does not add up, because calling free -m shows I got total of 1GB memory.

Since I have 1GB of ram, I expected to be able to only create 125 threads at maximum, not 250, and not 14000.

Why can I create 250 threads?

this could point you in the right direction https://stackoverflow.com/questions/344203/maximum-number-of-threads-per-process-in-linux — haronaut, Apr 10 '19 at 17:21
@harandk I read this, and it did not answer the question to my satisfaction — Willy Wonka, Apr 10 '19 at 17:21

score 2 · Answer 1 · answered Apr 10 '19 at 17:48

2

By default, Linux performs memory overcommit. This means that you can allocate more anonymous, writable memory than there is physical memory.

You can turn off memory overcommit using:

# sysctl vm.overcommit_memory=2

This will cause some workloads to fail which work perfectly fine in vm.overcommit_memory=0 mode. Some details can be found in the overcommit accounting documentation.

answered Apr 10 '19 at 17:48

Florian Weimer

32,022
3
48
92

This is a very interesting answer! However, this did not change anything. Do I just input the command in console? – Willy Wonka Apr 10 '19 at 18:38
Yes, it's a command that has to run as root. How much swap space do you have? Try disabling swap as well. – Florian Weimer Apr 11 '19 at 06:41

Why can I create more threads than "free memory" / "thread stack size"?

1 Answers1