How to use sched_getaffinity and sched_setaffinity in Linux from C?

Question

I am trying to:

• Run 16 copies concurrently with processor pinning (2 copies per core)

• Run 8 copies concurrently with processor pinning (2 copies per core) and flipping processor core to the furthest core after certain function say function 1 finishes.

The problem I am facing is how to select the farthest processor.

Some friends suggested to use sched_getaffinity and sched_setaffinity but I count not find any good examples.

Answer 1

To use sched_setaffinity to make the current process run on core 7 you do this:

cpu_set_t my_set;        /* Define your cpu_set bit mask. */
CPU_ZERO(&my_set);       /* Initialize it all to 0, i.e. no CPUs selected. */
CPU_SET(7, &my_set);     /* set the bit that represents core 7. */
sched_setaffinity(0, sizeof(cpu_set_t), &my_set); /* Set affinity of tihs process to */
                                                  /* the defined mask, i.e. only 7. */

See http://linux.die.net/man/2/sched_setaffinity & http://www.gnu.org/software/libc/manual/html_node/CPU-Affinity.html for more info.

Answer 2

Minimal runnable example

In this example, we get the affinity, modify it, and check if it has taken effect with sched_getcpu().

main.c

#define _GNU_SOURCE
#include <assert.h>
#include <sched.h>
#include <stdbool.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>

void print_affinity() {
    cpu_set_t mask;
    long nproc, i;

    if (sched_getaffinity(0, sizeof(cpu_set_t), &mask) == -1) {
        perror("sched_getaffinity");
        assert(false);
    }
    nproc = sysconf(_SC_NPROCESSORS_ONLN);
    printf("sched_getaffinity = ");
    for (i = 0; i < nproc; i++) {
        printf("%d ", CPU_ISSET(i, &mask));
    }
    printf("\n");
}

int main(void) {
    cpu_set_t mask;

    print_affinity();
    printf("sched_getcpu = %d\n", sched_getcpu());
    CPU_ZERO(&mask);
    CPU_SET(0, &mask);
    if (sched_setaffinity(0, sizeof(cpu_set_t), &mask) == -1) {
        perror("sched_setaffinity");
        assert(false);
    }
    print_affinity();
    /* TODO is it guaranteed to have taken effect already? Always worked on my tests. */
    printf("sched_getcpu = %d\n", sched_getcpu());
    return EXIT_SUCCESS;
}

GitHub upstream.

Compile and run:

gcc -ggdb3 -O0 -std=c99 -Wall -Wextra -pedantic -o main.out main.c
./main.out

Sample output:

sched_getaffinity = 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 
sched_getcpu = 9
sched_getaffinity = 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
sched_getcpu = 0

Which means that:

• initially, all of my 16 cores were enabled, and the process was randomly running on core 9 (the 10th one)
• after we set the affinity to only the first core, the process was moved necessarily to core 0 (the first one)

It is also fun to run this program through taskset:

taskset -c 1,3 ./a.out

Which gives output of form:

sched_getaffinity = 0 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 
sched_getcpu = 2
sched_getaffinity = 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
sched_getcpu = 0

and so we see that it limited the affinity from the start.

This works because the affinity is inherited by child processes, which taskset is forking: How to prevent inheriting CPU affinity by child forked process?

nproc respects sched_getaffinity by default as shown at: How to find out the number of CPUs using python

Python: os.sched_getaffinity and os.sched_setaffinity

See: How to find out the number of CPUs using python

Tested in Ubuntu 16.04.

Answer 3

Don't use CPU_SETSIZE as cpusetsize parameter for sched_[set|get]affinity. The names are misleading but this is wrong. The macro CPU_SETSIZE is (quoting man 3 cpu_set) "a value one greater than the maximum CPU number that can be stored in cpu_set_t." You have to use

sched_setaffinity(0, sizeof(cpu_set_t), &my_set);

instead.