1

I read the source code of linux aarch64: https://elixir.bootlin.com/linux/latest/source/arch/arm64/kernel/fpsimd.c#L311

It says that TIF_SVE can control the SVE register state save/restore machanism.

My userspace program is using SVE instructions and runs in an environment where the SVE Vector Length is 256bits. Therefore, the default machanism of SVE state save/restore during interrupts is not enough, as it only saves 128bits of each register. Then my program will fail when an interrupt happens. Thus, I need to set the TIF_SVE flag in my userspace program.

By the way, my userspace program uses OpenMP library.

How can I set the TIF_SVE flag in my userspace program?

I have seen the pthread_attr_t structs in pthread_create() interface. But I'm not sure there is any relationships between pthread_attr_t and TIF_SVE.

And, I searched OpenMP library manual, I've not seen where to set the TIF_SVE flag.

Now, I know that the TIF_SVE flag is set by the linux kernel, other than by a system call interface.

After I add some trace_printk() in fpsimd.c . I find that the do_sve_acc() is not called. The SVE load/restore using fpsimd_load_state() and fpsimd_save_state() functions, other than sve_load_state() and sve_save_state() functions. This finding can explain why after interrupts, the z0 register only remains the low 128bits data.

But, why do_sve_acc() is not called ?

aisv
  • 21
  • 4
  • 1
    Seems like `TIF_SVE` is an internal flag and you aren't supposed to be able to set it from userspace. In the comment you link I can read: *"An attempt by the user task to execute an SVE instruction causes do_sve_acc() to be called, which does some and then sets TIF_SVE."* - so it seems like the kernel does it for you. Do you have a program that uses SVE but is crashing because the kernel only partially saves/restores registers? If so, I would suggest attaching a [mre], that seems like a potential bug to me. – Marco Bonelli Sep 02 '23 at 03:48
  • Reading more into kernel code this comment seems interesting: https://elixir.bootlin.com/linux/v6.5/source/arch/arm64/kernel/fpsimd.c#L1478 ... You could try issuing some dummy SVE instructions at program startup to let the kernel trap and set `TIF_SVE` for you. – Marco Bonelli Sep 02 '23 at 03:51
  • @MarcoBonelli Thank you very much. I'm trying and debuging. – aisv Sep 02 '23 at 06:27
  • I would be quite surprised if a standard Linux kernel isn't already properly saving and restoring the full width of SVE registers. That would be a really obvious and severe bug, indeed a gaping security hole. Is your program actually failing, or were you just assuming from reading docs that it would be necessary to set `TIF_SVE` yourself? If your program is failing, what's your evidence for believing that the failure is due to SVE register corruption by interrupt handlers? – Nate Eldredge Sep 02 '23 at 06:38
  • @NateEldredge I have used gdb to debug my SVE program. This program is simple. It saves a value like '0x1f1f1f1f...' in `z0` register. By using `dup` instruction, `z0` contains `0x1f1f1f1f...`, a 256bits value. I also save the value `0x1f1f1f1f...` in memory. Then I use `ldr` instruction to load it in `z3`, then compare `z0` and `z3` in a loop. After several loops, `z0` becomes `0x1f1f1f...` (128bits) + `0x000000...` (128 bits). – aisv Sep 02 '23 at 07:16
  • @MarcoBonelli I found that the `do_sve_acc()` function is not called. Therefore, the `TIF_SVE` flag is not set. Do you know why the `do_sve_acc()` function is not called ? – aisv Sep 02 '23 at 10:04
  • 1
    @aisv that function should be called when the userspace program traps on the first SVE instruction that it executes, see [`el0t_64_sync_handler` here](https://elixir.bootlin.com/linux/v6.5/source/arch/arm64/kernel/entry-common.c#L677). Are you using QEMU (which version?) or real hardware? What exact CPU and kernel version? What kernel config? You need to add a lot more details to your question otherwise it's kind of impossible to help. The way you are describing it, it looks like a bug. I don't know enough about ARM64 (or Linux for ARM64) to say for sure though. – Marco Bonelli Sep 02 '23 at 13:50

0 Answers0