On Linux CentOS 7 I am using named fifo pipes created with mkfifo to handle data passing between pairs of processes running on a multiprocessor disk server (Dell R730xd with 132GB RAM and 20TB RAID disk).
In order to have some data buffer between the two processes connected to a pipe, when I open the pipe I also set its size to some large value (say 128MB).
To allow a large pipe size I had to set:
/proc/sys/fs/pipe-max-size = 134217728
/proc/sys/fs/pipe-user-pages-soft = 0 (i.e. no limit)
In my code I then use (example from a Perl script)
fcntl(\*FD, F_SETPIPE_SZ, int($size))
Once in a while the operation of setting the pipe size fails with the message "Cannot allocate memory". I noticed that this happens when the disk activity on the server is very high. Looking into /proc/meminfo I see that the Cached memory occupies most of the memory of my server, leaving only a few MB as MemFree. Once the disk activity subsides, the Cached memory slowly decreases and after a while the pipe size operation starts working again.
My questions are:
- why does the pipe size clash with the Cached memory? Shouldn't cached memory be released as soon as someone needs it (e.g. I have no problems in starting new processes)?
- How do I limit the maximum amount of memory which can be dedicated to disk caching? I guess that limiting it to 100GB instead of 132GB should not have a big impact on disk performance and this would leave plenty of memory for my pipes...
Thanks in advance for any help.
