How to find out exactly which process was killed by OOM killer?

Question

OOM killer just killed some process. I can know about it when the value of grep oom_kill /proc/vmstat increases.

How would I know which process exactly was killed (name, pid, owner and so on) and when exactly (timestamp)?

Answer 1

The crude, but easy way to know details on recent OOM-kill is to grep everywhere (proper log path may differ from distrib to distrib):

sudo dmesg -T | grep -Ei 'killed process|oom.killer'
sudo grep -Ei 'killed process|oom.killer' /var/log/messages /var/log/syslog 2>/dev/null

. means "any character" in regexp for grep, so it will find lines containing both "oom-killer" and "oom killer".

2>/dev/null on the end is needed to get rid of the "No such file or directory" error, if there is no such logfiles in your system.

Answer 2

Look for entries from OOM killer events in the logs.

sudo grep -i "oom-killer" /var/log/messages

Result looks something like this:

kernel: [timestamp] Out of memory: Kill process [process_name] ([process_pid]), UID [user_id]/[username], VmSize:[memory_size] kB, VmRSS:[resident_memory_size] kB, MemLimit:[memory_limit] kB