79

Currently ulimit -n shows 10000. I want to increase it to 40000. I've edited "/etc/sysctl.conf" and put fs.file-max=40000. I've also edited /etc/security/limits.conf and updated hard and soft values. But still ulimit shows 10000. After making all these changes I rebooted my laptop. I've access to root password.

usr_name@usr_name-lap:/etc$ /sbin/sysctl fs.file-max
fs.file-max = 500000

Added following lines in /etc/security/limits.conf -

*     soft    nofile          40000
*     hard    nofile          40000

I also added following line in /etc/pam.d/su-

session    required   pam_limits.so

I've tried every possible way as given on other forums, but I can reach up to a maximum limit of 10000, not beyond that. What can be the issue?

I'm making this change because neo4j throws maximum open file limits reached error.

Cyril Graze
  • 3,881
  • 2
  • 21
  • 27
theharshest
  • 7,767
  • 11
  • 41
  • 51

10 Answers10

83

What you are doing will not work for root user. Maybe you are running your services as root and hence you don't get to see the change.

To increase the ulimit for root user you should replace the * by root. * does not apply for root user. Rest is the same as you did. I will re-quote it here.

Add the following lines to the file: /etc/security/limits.conf

root soft  nofile 40000

root hard  nofile 40000

And then add following line in the file: /etc/pam.d/common-session

session required pam_limits.so

This will update the ulimit for root user. As mentioned in comments, you may don't even have to reboot to see the change.

Warren Seine
  • 2,311
  • 2
  • 25
  • 38
Sambhav Sharma
  • 5,741
  • 9
  • 53
  • 95
  • 2
    You actually don't need to reboot for the new limit to take effect. The point of editing the /etc/pam.d/common-session file is so that the creation of any new session will cause the limits files to be processed. – Christopher Smith Nov 20 '14 at 17:06
  • 1
    This procedure works. No reboot is required. Just logout and log back in to see the result. – Bill Butler Mar 25 '15 at 21:52
  • 2
    In the case of neo4j-service or any other service using `start-stop-daemon`, the `common-session-*` trick won't work, because `start-stop-daemon` doesn't leverage PAM at all. There's no other way than adding an explicit `ulimit -n xxx` call somewhere in the init script before the service is started. – David Ammouial Oct 06 '15 at 00:47
  • 3
    Worked beautifully on Ubuntu 16.04.1 LTS – Andrew T Finnell Aug 09 '16 at 20:57
63

1) Check sysctl file-max limit:

$ cat /proc/sys/fs/file-max

If the limit is lower than your desired value, open the sysctl.conf and add this line at the end of file:

fs.file-max = 65536

Finally, apply sysctl limits:

$ sysctl -p

2) Edit /etc/security/limits.conf and add below the mentioned

* soft     nproc          65535    
* hard     nproc          65535   
* soft     nofile         65535   
* hard     nofile         65535

These limits won't apply for root user, if you want to change root limits you have to do that explicitly:

root soft     nofile         65535   
root hard     nofile         65535
...

3) Reboot system or add following line to the end of /etc/pam.d/common-session:

session required pam_limits.so

Logout and login again.

4) Check soft limits:

$ ulimit -a

and hard limits:

$ ulimit -Ha
....

open files                      (-n) 65535

Reference : http://ithubinfo.blogspot.in/2013/07/how-to-increase-ulimit-open-file-and.html

Tombart
  • 30,520
  • 16
  • 123
  • 136
minhas23
  • 9,291
  • 3
  • 58
  • 40
  • 1
    Make Sure you logged out and back in so it can reflect your changes, if you are not checking it with root user, for ubuntu you can do this by `gnome-session-quit` – Mayank Saxena Jun 17 '15 at 09:46
  • /proc/sys/fs/file-max IS NOT THE MAX nofile allowed by the system !!! For example on my system by default it's set to 1528702. Setting it to that value for the soft/hard limits in limits.conf does not allow the user to increase the limits from 1024 at all. I had to set limits.conf to something much more conservative, e.g. 256*1024 = 262144, and finally ulimit -a is showing the same for a regular user. – sabujp Apr 28 '17 at 10:31
8

I am using Debian but this solution should work fine with Ubuntu.
You have to add a line in the neo4j-service script.
Here is what I have done :

nano /etc/init.d/neo4j-service
Add « ulimit –n 40000 » just before the start-stop-daemon line in the do_start section

Note that I am using version 2.0 Enterprise edition. Hope this will help you.

Martin Larivière
  • 264
  • 1
  • 10
  • 13
    FWIW there is no such script on Ubuntu Server 14.04. My solution was found in this question (and lornix's answer): http://askubuntu.com/questions/162229/how-do-i-increase-the-open-files-limit-for-a-non-root-user -- in short, you need to edit `/etc/security/limits.conf` and `/etc/pam.d/common-session*` – Lambart Jun 26 '14 at 18:16
  • 1
    I see that the original questioner mentioned neo4j in passing. It seems like maybe that belongs in the question's subject since it has nothing to do with Ubuntu. – Lambart Jun 26 '14 at 18:18
  • In Ubuntu 14.04, there really is a `/etc/init.d/neo4j-service` file and Martin Larivière is right, inserting `uname -n 40000` solves the issue. I don't know the differences between Ubuntu Server and Ubuntu, so maybe you're right too, I'm adding this just to clarify that what Martin wrote solves the problem with plain Ubuntu 14.04. – Grégoire C Mar 26 '15 at 09:30
  • 3
    @Lambart, in the case of neo4j-service, the `common-session-*` trick won't work, because neo4j-service is based on `start-stop-daemon` and `start-stop-daemon` doesn't leverage PAM at all. The only solution is indeed to add an explicit `ulimit -n xxx` call in the init script. – David Ammouial Oct 05 '15 at 23:11
  • 2
    This is not a solution, this is a bandaid. – Tim Wachter Oct 24 '16 at 09:06
3

I was having the same issue, and got it to work by adding entries to /etc/security/limits.d/90-somefile.conf. Note that in order to see the limits working, I had to log out completely from the ssh session, and then log back in.

I wanted to set the limit for a specific user that runs a service, but it seems that I was getting the limit that was set for the user I was logging in as. Here's an example to show how the ulimit is set based on authenticated user, and not the effective user:

$ sudo cat /etc/security/limits.d/90-nofiles.conf
loginuser    soft    nofile   10240
loginuser    hard    nofile   10240
root         soft    nofile   10241
root         hard    nofile   10241
serviceuser  soft    nofile   10242
serviceuser  hard    nofile   10242

$ whoami
loginuser
$ ulimit -n
10240
$ sudo -i
# ulimit -n
10240    # loginuser's limit
# su - serviceuser
$ ulimit -n
10240    # still loginuser's limit.

You can use an * to specify an increase for all users. If I restart the service as the user I logged in, and add ulimit -n to the init script, I see that the initial login user's limits are in place. I have not had a chance to verify which user's limits are used during a system boot or of determining what the actual nofile limit is of the service I am running (which is started with start-stop-daemon).

There's 2 approaches that are working for now:

  1. add a ulimit adjustment to the init script, just before start-stop-daemon.
  2. wildcard or more extensive ulimit settings in the security file.
Brett
  • 5,690
  • 6
  • 36
  • 63
  • when I do what you did (starting from `whoami`) I first get 1024, then 65536 (for root), then 65536 for user. But the first was the same user. Do you have an explanation for this, I'm baffled – Danielson Oct 23 '15 at 08:56
  • That sounds bizarre. How do you switch from user to root and back to user? Did you log out completely after setting the file? you may be in a cached session initially, and then using a new session depending on how you get back to the user. – Brett Oct 23 '15 at 12:18
  • I keep rebooting, just to be sure. Also tried setting `/etc/sysctl.conf` -> `fs.file-max = 1000000` should fix it globally for all users, also no result. It's like a process is ignoring me. I switch from user to root to user, like you described, from root -> user using `su - my_user_name`. Which indicates the system knows what it should use for that user, but it doesn't... – Danielson Oct 23 '15 at 13:39
2

You could alter the init script for neo4j to do a ulimit -n 40000 before running neo4j.

However, I can't help but feel you are barking up the wrong tree. Does neo4j legitimately need more than 10,000 open file descriptors? This sounds very much like a bug in neo4j or the way you are using it. I would try to address that.

abligh
  • 24,573
  • 4
  • 47
  • 84
2

I have lots of trouble getting this to work.

Using the following allows you to update it regardless of your user permission.

sudo sysctl -w fs.inotify.max_user_watches=100000

Edit

Just saw this from another user also on another stackexchange site (both work, but this version permanently updates the system setting, rather than temporarily):

echo fs.inotify.max_user_watches=100000 | sudo tee -a /etc/sysctl.conf; 
sudo sysctl -p
Layke
  • 51,422
  • 11
  • 85
  • 111
2

Try run this command it will create a *_limits.conf file under /etc/security/limits.d

echo "* soft nofile 102400" > /etc/security/limits.d/*_limits.conf && echo "* hard nofile 102400" >> /etc/security/limits.d/*_limits.conf

Just exit from terminal and login again and verify by ulimit -n it will set for * users

Mahattam
  • 5,405
  • 4
  • 23
  • 33
2

tl;dr set both the soft and hard limits

I'm sure it's working as intended but I'll add it here just in case. For completeness the limit is set here (see below for syntax): /etc/security/limits.conf

some_user       soft    nofile          60000
some_user       hard    nofile          60000

and activated with the following in /etc/pam.d/common-session:

session required pam_limits.so

If you set only the hard limit, ulimit -a will show the default (1024): If you set only the soft the limit ulimit -a will show (4096)

If you set them both ulimit -a will show the soft limit (up to the hard limit of course)

jorfus
  • 2,804
  • 27
  • 23
2

I did it like this

echo "NEO4J_ULIMIT_NOFILE=50000" >> neo4j
mv neo4j /etc/default/
bigkahunaburger
  • 426
  • 5
  • 10
1

ULIMIT configuration:

  1. Login by root
  2. vi security/limits.conf

  3. Make Below entry

    Ulimit configuration start for website user

    website   soft   nofile    8192
website   hard   nofile    8192
website   soft   nproc    4096
website   hard   nproc    8192
website   soft   core    unlimited
website   hard   core    unlimited

  4. Make Below entry for ALL USER

    Ulimit configuration for every user

    *   soft   nofile    8192
*   hard   nofile    8192
*   soft   nproc    4096
*   hard   nproc    8192
*   soft   core    unlimited
*   hard   core    unlimited

  5. After modifying the file, user need to logoff and login again to see the new values.

Sandeep kr gupta
  • 481
  • 4
  • 5