2

I want to call a function that returns a struct, this function has the __cdecl calling convention.

How does the __cdecl calling convention returns a struct?

  • 1
    Depends on struct size and environment (e.g. windows or linux/mac). It may fit into register(s) or it may be passed back in memory through a hidden pointer argument. Read the appropriate documentation. You can also just compile some test code. – Jester Jul 21 '17 at 15:33
  • Possible duplicate of [How do C compilers implement functions that return large structures?](https://stackoverflow.com/questions/2155730/how-do-c-compilers-implement-functions-that-return-large-structures) – Johan Jul 21 '17 at 16:13
  • 1
    It depends on the platform. The Microsoft 32-bit ABI returns 64-bit structs by value in EDX:EAX. The Linux ABI doesn't do that; it only returns a struct by value if it will fit in EAX. Otherwise, it is passed by pointer, using an implicit hidden parameter. A complete answer would be too broad unless you can narrow this down a bit. – Cody Gray - on strike Jul 21 '17 at 16:45
  • @Cody Gray So there is no single implementation for the `__cdecl` calling convention, each operating system specifies a different implementation! good to know. –  Jul 21 '17 at 17:02
  • Yes, calling conventions are platform-specific. [The Wikipedia page on this](https://en.wikipedia.org/wiki/X86_calling_conventions#cdecl) is *very* good. Have you read it? It covers exactly what Jester and I were talking about. – Cody Gray - on strike Jul 21 '17 at 17:03
  • this is absolute independent from calling convention. __cdecl or __stdcall, __fastcall or __thiscall - no difference. structure allocated in the stack before call and pass pointer to this struct to function. function init own instance of structure in stack and then copy to caller pointer. all this is very ugly and not effective, need simply never return structs as function result – RbMm Jul 21 '17 at 22:00
  • @RbMm: it is perhaps independent of calling convention, but not independent of the compiler used. Some compilers use EAX or RAX (or even two registers) to return small structs, others don't. Some always use an extra pointer argument. Some make this dependent on the size of the struct. Also, the extra argument can be passed as first or as last argument. This is also implementation dependent. – Rudy Velthuis Jul 21 '17 at 22:40
  • @CodyGray: On Windows, it even depends on the compiler (even if all use __cdecl). Some use registers to return small structs, some don't. And the hidden pointer may be the first or the last argument being passed. Or even a register. – Rudy Velthuis Jul 21 '17 at 22:50
  • @user8315006: Indeed, there is no uniform way, not even if a function is declared as `__cdecl`. That is even so for different compilers on the same platform, i.e. Win32. – Rudy Velthuis Jul 21 '17 at 22:55
  • @RudyVelthuis - yes, compiler use *eax* or *edx:eax* pair on x86 and up to *rdx:rax* (16 bytes) on x64. but in general used hided stack allocation, additional parameter, copy.. and of course this absolute independent from calling convention – RbMm Jul 22 '17 at 06:56
  • @Rudy Interesting that this is compiler-specific. I mean, it isn't *supposed* to be. The ABI is actually defined by the platform, even if that means it is *de facto* defined by the platform's principal complier (MSVC on Windows, GCC on Linux). If you have a compiler that isn't conforming to that ABI, then you have a broken compiler. There is no end of bugs that can arise from that. Still, if they do exist, then it's good to be aware of that. – Cody Gray - on strike Jul 22 '17 at 11:29
  • @CodyGray: the calling conventions rule a lot, but not how structs are returned. That is why compilers do it their way. – Rudy Velthuis Jul 22 '17 at 11:52
  • Yes, I'm blurring together ABIs and calling conventions when I should be more careful to separate them. You're right that the calling convention is more limited in scope and doesn't define this, but the platform ABI absolutely does, and if a compiler is generating code that expects to pass all structs by reference, it will be incompatible with code generated by other compilers that expect to receive small struct by-value. This would make it incompatible with standard platform API functions, which would be a fatal flaw. @Rudy – Cody Gray - on strike Jul 22 '17 at 11:56
  • I have seen different ways that structs (even ones that fir in one or two registers) are returned. One could say that the MS compiler is a de facto standard, but apparently not all compilers do this the same way as MS VC++. I don't think there is a real ABI for Win32 that has rules for this (for Win64, there is no cdecl, so this can only be about Win32). – Rudy Velthuis Jul 22 '17 at 12:04

1 Answers1

7

That depends.

There is no uniform way.

Simple function returns, like ints are returned in registers, but none of the calling conventions, __cdecl, __stdcall, etc. describe how exactly a struct is returned.

  • Some return simple structs that fit in one (EAX, RAX) or two registers (EDX:EAX or RDX:RAX) in these mentioned registers.
  • Others return such structs by reference, i.e. the function result is compiled as a void function taking an extra pointer argument to such a struct.
  • Others apply a mix of these strategies.

I have had big problems with this (DLL functions that return structs). The fact that different compilers use different strategies to return structs, even on the same platform and using the same calling convention, has made me issue the warning that one should never return structs from, say, DLL functions.

More info:

Rudy Velthuis
  • 28,387
  • 5
  • 46
  • 94
  • FWIW, unlike in Win32, the [Win64 documentation](https://learn.microsoft.com/en-us/cpp/build/return-values-cpp) *does* govern return values. This is officially for Microsoft's C++, but it seems to be the de facto ABI. – Rudy Velthuis Jul 23 '17 at 16:41
  • 1
    The updated MS link is https://learn.microsoft.com/en-us/cpp/build/x64-calling-convention, for y'all's information. – Mingye Wang Oct 29 '19 at 13:37