C++ multithreaded inline asm

Question

I'm looking to run inline asm in a thread but I get a seg fault on certain instructions, such as the below:

#include <thread>

void Foo(int &x) 
{
    int temp;
    asm volatile ("movl $5, %%edx;"
                  "movl $3, %%eax;"
                  "addl %%edx, %%eax;"
                  "movl %%eax, -24(%%rbp);" // seg faults here
                  "movl -24(%%rbp), %0;"
                  : "=r" (temp) : : );
    x=temp;
}

int main() 
{
    int x;
    std::thread t1(Foo, std::ref(x));
    t1.join();
    return 0;
}

(I'm using std::ref to be able to pass a reference to an std::thread, but have to use the temp variable because the extended asm syntax doesn't work with references.)

I've tried clobbering all involved registers and it doesn't help. If I don't pass any arguments to the thread, it seems to work: and it strangely also works if I clobber the %ebx register (which isn't involved). I'm using gcc 4.8.4 on 64-bit Ubuntu 14.04.

What is the best/safest way I can execute inline asm in a thread?

Where does `-24(%%rbp)` come from? Seems to me you are potentially destroying the contents of something on the stack that may be used in the function prologue/epilogue. -24(%%rbp) could potentially be where `%rdi` (first parameter which will effectively be a pointer) gets temporarily stored on the stack. You overwrite it, and then I'd guess that `x=temp` would segfault because because `x` no longer has a valid pointer. But that all really depends on the code that gets generated. — Michael Petch, Dec 24 '15 at 21:16
This is a bit of a contrived example so the -24(%%rbp) is just an example of a normal asm instruction that seg faults the program. My question is whether there's a way to safely execute asm on a thread like this, so that bits of the stack don't get overwritten? I thought this was the point of clobbering but that doesn't seem to help... — jaw, Dec 24 '15 at 21:21
I'm curious what makes you think that `"movl %%eax, -24(%%rbp);" // seg faults here` would be the instruction that segfaults? I'd venture to guess that actually doesn't segfault but the data you potentially destroy on the stack could lead to a segfault on an instruction later on. — Michael Petch, Dec 24 '15 at 21:25
Your contrived example arbitrarily overwrites a place on the stack that may actually be used for information that is used later one. Corrupting the stack can lead to weird and wonderful bugs. Seems like an XY problem. What is it you really are trying to achieve with such code? — Michael Petch, Dec 24 '15 at 21:28
The code is using ATT syntax, and is correct for the first two instruction, but then it attempts to write eax to -24[rbp], so the operands on that move need to be reversed. The other issue, is that assuming this is 64 bit code with a normal entry code, then to read the first parameter, it's movq 16(%%rbp),rax. If frame pointers are disabled via a compiler option, then rbp is not setup. — rcgldr, Dec 24 '15 at 21:29
@rcgldr : which goes back to my comment about `But that all really depends on the code that gets generated.` optimizations could potentially wipe out the need for any stackframe. — Michael Petch, Dec 24 '15 at 21:30
If there's no stack frame, then it probably should be movq 8(%%rsp), rax, if this is 64 bit code, or movl 4(%%esp),eax if this is 32 bit code. For Windows 64 bit mode API (or at least Visual Studio), the parameters are in registers, not the stack. Take a look at [64 bit api](http://msdn.microsoft.com/en-us/library/ms235286.aspx). In this case the pointer to x would be in rcx. — rcgldr, Dec 24 '15 at 21:33
I'm not sure why you didn't just do `"movl %%eax, %0;"` instead of ` `"movl %%eax, -24(%%rbp);"` `"movl -24(%%rbp), %0;"` . And since you destroy _EAX_(_RAX_) and _EDX_(_RDX_) your going to want to add them to your clobber list. — Michael Petch, Dec 24 '15 at 21:37
@rcgldr : OP is on Linux, not Windows. In 64-bit Linux code the first integer value would be placed in register _rdi_ . Per the 64-bit System V ABI. http://www.x86-64.org/documentation/abi.pdf. I also don't recommend inline assembler instructions relying on the behavior of generated code outside its control. If something needs to be passed in or out then the input and output constraints of the assembler template are for that purpose. — Michael Petch, Dec 24 '15 at 21:41
@MichaelPetch - I only mentioned Windows API as an example. I didn't know where to find the Linux 64 bit api. I also had the impression that calling conventions depend the compiler, not the operating system, so is this a GCC convention or a Linux convention or both? — rcgldr, Dec 24 '15 at 21:46
@rcgldr The System V 64-ABI is a convention used on a variety of OSes including BSD and Linux. Calling convention in this case can be OS specific, since the spec also contains the convention to pass parameters to a syscall. So it is a bit of both. GCC on Linux follows the 64-bit System V ABI for userland (unless it can perform a safe optimization). If you want interoprability between shared objects then you'll want to conform to the 64-bit System V ABI on Linux. — Michael Petch, Dec 24 '15 at 21:54
My guess is that clobbering `%ebx` maybe gets gcc to make a stack frame, initializing `%rbp`. Without that, `%rbp` probably has a garbage value that isn't pointing to stack memory. Don't make assumptions about what the compiler does outside your inline asm block. You will have a bad time when your function gets inlined into something else, changing the context. — Peter Cordes, Dec 25 '15 at 00:35
These other examples of sane (I think) inline asm may help: http://stackoverflow.com/a/34450163/224132 http://stackoverflow.com/a/34445757/224132 http://stackoverflow.com/a/34447658/224132 http://stackoverflow.com/a/6671550/224132 — Peter Cordes, Dec 25 '15 at 00:38

Michael Petch · Accepted Answer · 2015-12-24T23:49:57.427

I'm still not completely convinced I understand what your objectives are. I find the comment "but have to use the temp variable because the extended asm syntax doesn't work with references." a bit unusual . I'd expect you can pass a reference into an assembler template since it really is just a pointer under the hood. Since you are using a contrived example, I'll maintain that, but exclude the move of data to an arbitrary location on the stack, and I'll add the registers that get clobbered to the list. You could probably go with something like:

#include <thread>

void Foo(int &x) 
{
    asm volatile ("movl $5, %%edx;" // We clobber EDX
                  "movl $3, %%eax;" // We clobber EAX
                  "addl %%edx, %%eax;" // Result in EAX=3+5=8
                  "movl %%eax, %0;" // Move to variable x
                  : "=r" (x) : : "eax", "edx" );
}

int main() 
{
    int x;
    std::thread t1(Foo, std::ref(x));
    t1.join();
    return 0;
}

I've added "eax" and "edx" to the clobber list since we destroy them in our assembler template (and they don't appear as input or output constraints). You should also notice I don't use a temporary variable. The assembler code can be reduced to a single instruction since the contrived example is the equivalent of movl $8, %0;.

You could also use the memory address (reference) to x like this:

void Foo(int &x) 
{
    asm volatile ("movl $5, %%edx;" // We clobber EDX
                  "movl $3, %%eax;" // We clobber EAX
                  "addl %%edx, %%eax;" // Result in EAX=3+5=8
                  "movl %%eax, %0;" // Move to variable x
                  : "=mr" (x) : : "eax", "edx" );
}

In this case I use =mr (memory operand or register) as an output constraint. This allows us to move a value right to the memory operand without an intermediate register.

I'd use an `"=mr"` constraint, to give the compiler the option of where to put it. So it doesn't force a store to memory when inlining, for example. Or `=r` and use it instead of one of the scratch registers. Also, you can use C variables like `int tmp` as output operands (which you never use in the C code) to let the compiler choose which scratch registers your inline asm will use. I wrote some inline asm answers in the past couple days, will dig up the links later. — Peter Cordes, Dec 24 '15 at 23:31
@PeterCordes I'll agree with `"=mr"`. The problem is that he is using a contrived example (his words) where the real meaning is unknown. As I stated in my question, I'm not attempting to do anything more than what his contrived example did. As I pointed out his contrived example can be reduced to a single instruction without any intermediates. My quote: "The assembler code can be reduced to a single instruction since the contrived example is the equivalent of `movl $8, %0`" . — Michael Petch, Dec 24 '15 at 23:44
@PeterCordes : I'm not unaware of specifying inputs to be used as scratch registers.I deliberately kept them in, given the OP comment about clobbers. If the OP had not provided a contrived example, and we knew what it was they were trying to do, then we could find an efficient means of doing it. It seems this OP has a ways to go in understanding. One of my comments under the question is suggesting this sounds like an XY problem, and I asked for clarification that never came. — Michael Petch, Dec 24 '15 at 23:47

C++ multithreaded inline asm

1 Answers1