Shared memory "Too many open files" but ipcs doesn't show many allocations

Question

I'm writing unit tests for code which creates shared memory.

I only have a couple of tests. I make 4 allocations of shared memory and then it fails on the fifth.

After calling shmat() perror() says Too many open files:

template <typename T> 
bool Attach(T** ptr, const key_type& key)
{
    // shmemId was 262151
    int32_t shmemId = shmget( key.key( ), ( size_t )0, 0644 );

    if (shmemId < 0)
    {
        perror("Error: ");
        return false;
    }

    *ptr = ( T * ) shmat(shmemId, 0, 0 );

    if ( ( int64_t ) * ptr < 0 )
    {
        // Problem is here. perror() says 'Too many open files'
        perror( "Error: ");
        return false;
    }

    return true;
}

However, when I check ipcs -m -p I only have a couple of shared memory allocations.

T     ID     KEY        MODE       OWNER    GROUP  CPID  LPID
Shared Memory:
m 262151 0x0000a028 --rw-r--r--                          3229      0
m 262152 0x0000a029 --rw-r--r--                          3229      0

In addition, when I check my OS shared memory limits sysctl -A | grep shm I get:

kern.sysv.shmall: 1024
kern.sysv.shmmax: 4194304
kern.sysv.shmmin: 1
kern.sysv.shmmni: 32
kern.sysv.shmseg: 8
security.mac.posixshm_enforce: 1
security.mac.sysvshm_enforce: 1

Are these variables large enough/are they the cause/what values should I have?

I'm sure I edited the file to increase them and restarted machine but perhaps it hasn't accepted (this is on Mac/OSX).

`if ( ( int64_t ) * ptr < 0 )` should be `if ( *ptr == reinterpret_cast(-1) )` if I read the documentation correctly. Since it's C++, you could make `ptr` a `T*&` instead to not have to dereference `ptr` everywhere. — Ted Lyngmo, Feb 04 '22 at 15:11
@TedLyngmo The code isn't mine so I'm hesitant to change that. Do you think it's causing the problem or just it's not as good as it could be? — user997112, Feb 04 '22 at 15:14
Answer to the problem: remove the tag C. As for an answer to the question, I have assumptions, but no [mcve] was posted for getting a correct answer. — 273K, Feb 04 '22 at 15:16
@user997112 I don't think the cast will matter with regards to the number of open files. Changing to a `*&` instead of a `**` only simplifies the usage so that you can write `ptr` instead of `*ptr` inside the function. Please make a [mre] as suggested and it'll be easier to see if you leak resources. — Ted Lyngmo, Feb 04 '22 at 15:18
“Too many open file” can not be checked with ipcs. Use lsof command (LiSt Open Files) — Ptit Xav, Feb 04 '22 at 15:36

Craig Estey · Answer 1 · 2022-02-04T19:39:46.553

Your problem may be elsewhere.

Edit: This may be a shmmni limit of macOS. See below.

When I run your [simplified] code on my system (linux), the shmget fails.

You didn't specify IPC_CREAT to the third argument. If another process has created the segment, this may be okay.

But, it doesn't/shouldn't like a size of 0. The [linux] man page states that it returns an error (errno set to EINVAL) if the size is less than SHMMIN (which is 1).

That is what happened on my system. So, I adjusted the code to use a size of 1.

This was done [as I mentioned] on linux.

macOS may allow a size of 0, even if that doesn't make practical sense. (e.g.) It may round it up to a page size.

For shmat, it returns (void *) -1.

But, some systems can have valid addresses that have the high bit set. (e.g.) 0xFFE0000000000000 is a valid address, but would fail your if test because casting that to int64_t will test negative.

Better to do:

if ((int64_t) *ptr == (int64_t) -1)

Or [possibly better]:

if ((void *) *ptr == (void *) -1)

Note that errno is not set/changed if the call succeeds.

To verify this, do: errno = 0 before the shmat call. If perror says "Success", then the shmat is okay. And, your current test needs to be adjusted as above--I'd do that change regardless.

You could also do (e.g):

printf("ptr=%p\n",*ptr);

Normally, errno starts as 0.

Note that there are some differences between macOS and linux.

So, if errno is ever set to "too many open files", this can be because the process has too many open files (EMFILE).

It might be because the system-wide limit is reached (ENFILE) but that is "file table overflow".

Note that under linux shmat can not generate EMFILE. However, it appears that under macOS it can.

However, if the number of calls to shmat is limited [as you mention], the shmat should succeed.

The macOS man page is a little vague as to what the limit is based on. However, I checked the FreeBSD man page for shmat and that says it is limited by the sysctl parameter: kern.ipc.shmseg. Your grep should have caught that [if applicable].

It is possible some other syscall elsewhere in the code is opening too many files. And, that syscall is not checking the error return.

Again, I realize you're running macOS.

But, if available, you may want to try your program under linux. For example, it has much larger limits from the sysctl:

kernel.shm_next_id = -1
kernel.shm_rmid_forced = 0
kernel.shmall = 18446744073692774399
kernel.shmmax = 18446744073692774399
kernel.shmmni = 4096
vm.hugetlb_shm_group = 0

Note that shmmni is the system-wide maximum number of shared memory segments.

Note that for macOS, shmmni is 32 (vs. 4096 for linux)!?!?

That means that the entire system can only have 32 open shared memory segments for any/all processes???

That seems very low. You can probably set this to a larger number and see if that helps.

Linux has the strace program and you could use it to monitor the syscalls.

But, macOS has dtruss: How to trace system calls of a program in Mac OS X?

*Better to do: `if ((int64_t) *ptr == (int64_t) -1)`*? The [POSIX documentation for `shmat()`](https://pubs.opengroup.org/onlinepubs/9699919799.2018edition/functions/shmat.html) says the return value is `(void *)-1` on failure, so the comparison should be `if ((void *) *ptr == (void *) -1)` — Andrew Henle, Feb 04 '22 at 19:19

Shared memory "Too many open files" but ipcs doesn't show many allocations

1 Answers1