1

I run a server mostly for off-site backup purposes for my job. But since I have a server, I do run a few applications in it.

The machine is an older Intel i7 with 16gb memory running Ubuntu Server.

One of the tasks I have automates there (via Cron) is a simple rsync the make some of the files available for another application.

rsync -ahvrO --delete --exclude={'Capture','Trash','Selects','.BridgeSort','*.csv','*.txt'} /samba/A4/files/ /redpool/A4/

It runs once a day and usually it does not take to much processing. But when I'm in the middle of a big job, the number of files per day grow a lot and it takes a lot of resources making all the other application become very slow or stop working.

This bottleneck is in the filesystem, I run a ZFS Pool in regular mechanical hard drives. I won't change the structure of the filesystem for now, but I would like to try and limit the amount of resources it takes, but in a "cronifiable" way. It's a lower priority task, so it could take longer and it would not be a huge problem I just want it to not take up all the resources available.

Hastur
  • 19,483
  • 9
  • 55
  • 99
augustoL
  • 13

1 Answers1

0

Usually with nice you should solve this problem. Read man nice:

nice - run a program with modified scheduling priority

You can limit resources via

  1. You can limit the resources (CPU) via nice...

    nice -n 19 rsync -ahvrO --delete --exclude={'Capture','Trash','Selects','.BridgeSort','*.csv','*.txt'} /samba/A4/files/ /redpool/A4/

  2. In rsync you can limit if needed bandwidth with the explicit option --bwlimit=RATE

  3. In rsync you can limit memory with --max-alloc=SIZE.

But really nice should be enough.

Hastur
  • 19,483
  • 9
  • 55
  • 99