I am running a paperless-ngx docker on my SBC. Because of processing some documents with many images (and thus OCR is required) out of memory errors are happening.
To troubleshoot it and to know the frequency of this error I used dmesg & journalctl tools, however, the amount of output is less for the latter. It seems journalctl skips many instances of error on Mar 03:
Why does journalctl miss multiple events?
ukhan@radxa-zero3:~$ dmesg -T -l err,warn | grep 'out of memory'
[Sun Mar 2 23:09:54 2025] Memory cgroup out of memory: Killed process 6281 ([celeryd: celer) total-vm:3522152kB, anon-rss:754164kB, file-rss:0kB, shmem-rss:0kB, UID:1000 pgtables:3256kB oom_score_adj:0
[Sun Mar 2 23:22:21 2025] Memory cgroup out of memory: Killed process 6119 (gunicorn: worke) total-vm:2893596kB, anon-rss:1000404kB, file-rss:0kB, shmem-rss:0kB, UID:1000 pgtables:3896kB oom_score_adj:0
[Mon Mar 3 00:17:10 2025] Memory cgroup out of memory: Killed process 22837 ([celeryd: celer) total-vm:3172564kB, anon-rss:661080kB, file-rss:0kB, shmem-rss:0kB, UID:1000 pgtables:2572kB oom_score_adj:0
[Mon Mar 3 01:06:41 2025] Memory cgroup out of memory: Killed process 53718 ([celeryd: celer) total-vm:3517492kB, anon-rss:929352kB, file-rss:8kB, shmem-rss:0kB, UID:1000 pgtables:3220kB oom_score_adj:0
[Mon Mar 3 01:23:44 2025] Memory cgroup out of memory: Killed process 29113 (gunicorn: worke) total-vm:2882844kB, anon-rss:966700kB, file-rss:0kB, shmem-rss:0kB, UID:1000 pgtables:3776kB oom_score_adj:0
[Mon Mar 3 02:06:44 2025] Memory cgroup out of memory: Killed process 87661 ([celeryd: celer) total-vm:3197144kB, anon-rss:806156kB, file-rss:0kB, shmem-rss:0kB, UID:1000 pgtables:2620kB oom_score_adj:0
[Mon Mar 3 03:06:00 2025] Memory cgroup out of memory: Killed process 120961 ([celeryd: celer) total-vm:2967012kB, anon-rss:819544kB, file-rss:0kB, shmem-rss:0kB, UID:1000 pgtables:2172kB oom_score_adj:0
[Mon Mar 3 03:25:40 2025] Memory cgroup out of memory: Killed process 67254 (gunicorn: worke) total-vm:3274500kB, anon-rss:870740kB, file-rss:0kB, shmem-rss:0kB, UID:1000 pgtables:4808kB oom_score_adj:0
[Mon Mar 3 04:08:45 2025] Memory cgroup out of memory: Killed process 154808 ([celeryd: celer) total-vm:3685908kB, anon-rss:1004956kB, file-rss:0kB, shmem-rss:0kB, UID:1000 pgtables:3572kB oom_score_adj:0
[Mon Mar 3 05:10:19 2025] Memory cgroup out of memory: Killed process 184890 ([celeryd: celer) total-vm:3682320kB, anon-rss:1002228kB, file-rss:0kB, shmem-rss:0kB, UID:1000 pgtables:3556kB oom_score_adj:0
[Mon Mar 3 06:13:34 2025] Memory cgroup out of memory: Killed process 212008 ([celeryd: celer) total-vm:3684684kB, anon-rss:1008476kB, file-rss:0kB, shmem-rss:0kB, UID:1000 pgtables:3564kB oom_score_adj:0
[Mon Mar 3 07:08:01 2025] Memory cgroup out of memory: Killed process 234857 ([celeryd: celer) total-vm:3685248kB, anon-rss:1020868kB, file-rss:4kB, shmem-rss:0kB, UID:1000 pgtables:3560kB oom_score_adj:0
[Mon Mar 3 08:06:47 2025] Memory cgroup out of memory: Killed process 252453 ([celeryd: celer) total-vm:3636468kB, anon-rss:995568kB, file-rss:0kB, shmem-rss:0kB, UID:1000 pgtables:3472kB oom_score_adj:0
[Mon Mar 3 09:06:54 2025] Memory cgroup out of memory: Killed process 269858 ([celeryd: celer) total-vm:3645884kB, anon-rss:1001132kB, file-rss:0kB, shmem-rss:0kB, UID:1000 pgtables:3488kB oom_score_adj:0
[Mon Mar 3 10:07:08 2025] Memory cgroup out of memory: Killed process 287896 ([celeryd: celer) total-vm:3646636kB, anon-rss:1001308kB, file-rss:0kB, shmem-rss:0kB, UID:1000 pgtables:3488kB oom_score_adj:0
[Mon Mar 3 11:06:40 2025] Memory cgroup out of memory: Killed process 304723 ([celeryd: celer) total-vm:3644656kB, anon-rss:1001416kB, file-rss:0kB, shmem-rss:0kB, UID:1000 pgtables:3484kB oom_score_adj:0
[Mon Mar 3 12:07:22 2025] Memory cgroup out of memory: Killed process 320622 ([celeryd: celer) total-vm:3655608kB, anon-rss:1005540kB, file-rss:0kB, shmem-rss:0kB, UID:1000 pgtables:3508kB oom_score_adj:0
ukhan@radxa-zero3:~$ journalctl -xb -p 3 --since yesterday | grep 'out of memory'
Mar 03 10:06:51 radxa-zero3 kernel: Memory cgroup out of memory: Killed process 287896 ([celeryd: celer) total-vm:3646636kB, anon-rss:1001308kB, file-rss:0kB, shmem-rss:0kB, UID:1000 pgtables:3488kB oom_score_adj:0
Mar 03 11:06:23 radxa-zero3 kernel: Memory cgroup out of memory: Killed process 304723 ([celeryd: celer) total-vm:3644656kB, anon-rss:1001416kB, file-rss:0kB, shmem-rss:0kB, UID:1000 pgtables:3484kB oom_score_adj:0
Mar 03 12:07:05 radxa-zero3 kernel: Memory cgroup out of memory: Killed process 320622 ([celeryd: celer) total-vm:3655608kB, anon-rss:1005540kB, file-rss:0kB, shmem-rss:0kB, UID:1000 pgtables:3508kB oom_score_adj:0
