How to limit a 64-bit process address space to less than 4G?

Question

When I execute a 64-bit process and look at /proc/[pid]/maps, the layout show that the shared libraries and stack section are in a larger address; such as the following:

7ffff7ffc000-7ffff7ffd000 r--p 0001d000 08:03 16643      /lib/ld-2.11.2.so
7ffff7ffd000-7ffff7ffe000 rw-p 0001e000 08:03 16643      /lib/ld-2.11.2.so
7ffff7ffe000-7ffff7fff000 rw-p 00000000 00:00 0
7ffffffea000-7ffffffff000 rw-p 00000000 00:00 0         [stack]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall]

Can I limit these sections to less 4G space and it must still be a 64-bit process rather than compile it into a 32-bit process?

If I may ask -- why exactly do you not want to have access to the entire 64-bit address space? — paulcam, Mar 22 '11 at 08:09
@Gabe : I am using llvm jit and forcing it to generate the x86 code in a 64-bit process. When I generate code for a "printf" instruction, it will call the function in libc; but the size of register is become 32-bit, I can't reach to address (The libc address is in 7fffxxxxxxxx.) — Hsiao-Hui Chiu, Mar 22 '11 at 08:16
I just want to examine Does the jited x86 code work correctly. — Hsiao-Hui Chiu, Mar 22 '11 at 08:19
Do you have the requirement that you have to JIT 32-bit code and that the process itself has to be 64-bit? — paulcam, Mar 22 '11 at 08:27
@paulcam :No, I look at the JITed code and they are normal x86 instructions, I think it should work fine in a 64-bit process. — Hsiao-Hui Chiu, Mar 22 '11 at 08:40
Don't do that. Compile the host process as a 32 bit program instead - hey presto, problem solved. — gnud, Mar 22 '11 at 08:58
@Hsiao 32-bit pointers in a 64-bit application screams a rather useless blackbox that cannot interact with anything. — Steve-o, Mar 22 '11 at 09:37
Sounds like you want [the Linux x32 ABI: 32-bit pointers in long mode](https://en.wikipedia.org/wiki/X32_ABI). — Peter Cordes, Jan 22 '19 at 05:21

score 3 · Answer 1 · answered Mar 22 '11 at 08:32

While the prelink(8) tool and concept are widely detested (and probably not shipped on your distribution), you may be able to use it to link libraries into a binary into low memory:

-r --reloc-only=ADDRESS
    Instead of prelinking, just relink given shared libraries
    to the specified base address.

Since the address that libraries will be mapped into the process is determined by ld(1), you might be able to modify your Makefile to invoke ld with different --section-start values:

--section-start SECTION=ADDRESS
                           Set address of named section
-Tbss ADDRESS               Set address of .bss section
-Tdata ADDRESS              Set address of .data section
-Ttext ADDRESS              Set address of .text section
-Ttext-segment ADDRESS      Set address of text segment

I moved the text and bss segments down to lower addresses:

$ gcc -Wl,-Ttext-segment=0x200000 -Wl,-Tbss=0x400000 -o broken broken.c
$ readelf -a broken
ELF Header:
  Magic:   7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00 
  Class:                             ELF64
  Data:                              2's complement, little endian
  Version:                           1 (current)
  OS/ABI:                            UNIX - System V
  ABI Version:                       0
  Type:                              EXEC (Executable file)
  Machine:                           Advanced Micro Devices X86-64
  Version:                           0x1
  Entry point address:               0x200450
...

If you can move all the sections provided in the executable with --section-start and move the libraries down with prelink(8), you might be able to get all the code loaded below 4 gigabytes.

Thanks, this is good for me. But I still do know how it can link shared library (libc) to below 4G. I don't know how to use the -r option. — Hsiao-Hui Chiu, Mar 22 '11 at 11:57

Prof. Falken · Answer 2 · 2011-03-22T08:31:49.537

I don't know any general answer to this, but if you patched libc and the kernel, you could modify them to never create any address higher than 4G. Then your user mode code would not have to changed at all, any generated address through for example malloc() would never be above 4G.

^{It may also be most convenient if the machine you ran the kernel also has at most 4G combined swap and physical memory.}

One thing to try, is to turn off overcommit. According to this page, if you set overcommit to "2" (off), and use say, 512 megabyte physical memory (including swap) then your address space should never grow outside of what a 32 bit address can hold. There may also be some kind of offset added to the 64 bit addresses, but in that case you should be able to find it and remove it.

Yet another idea is very hackish, but might work. Allocate shared memory from a 32 bit process and use this memory in your 64 bit process.

The amount of physical memory in the machine has no relationship to the virtual addresses assigned when `ld.so(8)` dynamically links in libraries. — sarnold, Mar 22 '11 at 08:08
@sarnold, no, but it helps avoiding the case where the kernel has allocated more than be can expressed in 32 bits. But thanks, I added "and combined swap". — Prof. Falken, Mar 22 '11 at 08:18

score 0 · Answer 3 · answered Mar 23 '11 at 00:49

Based on your comment:

I am using llvm jit and forcing it to generate the x86 code in a 64-bit process. When I generate code for a "printf" instruction, it will call the function in libc; but the size of register is become 32-bit, I can't reach to address (The libc address is in 7fffxxxxxxxx.)

...merely limiting libraries to 32 bit addresses is entirely insufficient. For one thing, the calling convention is different in 64 bit mode - the 64 bit printf() expects its first parameter in %rdi, but the 32 bit code will instead push it on the stack.

You will need to have the generated code call a printf() wrapper that sets up the parameters correctly for the call to the real printf(). You can put that wrapper in a MAP_32BIT region.

If I switch back to compatibility mode, Will the calling convention change to be passing through stack? — Hsiao-Hui Chiu, Mar 23 '11 at 11:16
@Hsaio-Hui Chiu: No - the calling convention expected by the library code, where `printf` resides, is "baked-in" to the code. Changing the CPU mode won't change that - in fact, the 64 bit library code won't even run in compatibility mode, because it's full of instruction sequences that aren't valid in that mode. — caf, Mar 23 '11 at 11:21

Chad · Answer 4 · 2011-03-22T11:30:38.647

-1

Seeing I was voted down I went and re-corrected the answer.

For Microsoft Visual Studio C++ if the compiler is a 32bit compiler, the /LARGEADDRESSAWARE flag sets the virtual address space from 2gb to 3gb this is only when running on a 32bit OS.

When running the same 32bit program on a 64bit os, the application will be given access to the full 4gb of virtual address space. For 64bit compilers the /LARGEADDRESSAWARE flag is enabled by default.

Seeing this question is for GCC I guess its not important for your problem.

edited Mar 22 '11 at 11:30

answered Mar 22 '11 at 07:53

Chad

2,938
3
27
38

Probably GCC, as she is using Linux. – Prof. Falken Mar 22 '11 at 07:54
1

Even allowing for the gratuitous discussion of Windows when the question concerns Linux, your facts regarding Windows are all out. /LARGEADDRESAWARE is a PE flag for 32 bit executables. It marks the executable of being able to handle memory addresses with the top bit set, >2GB. It's useful for executables that run under 32 bit Windows with /3GB boot switch, and for 32 bit executables under WOW64. – David Heffernan Mar 22 '11 at 08:10
@David The flag /LARGEADDRESAWARE really works for 64b applications as well, the only difference is it is ON by default when linking (http://msdn.microsoft.com/en-us/library/wz223b1z(v=VS.100).aspx). – Suma Mar 22 '11 at 09:42

score -1 · Answer 5 · edited Aug 17 '12 at 10:16

-1

Maybe this is what you are looking for:

man setrlimit

edited Aug 17 '12 at 10:16

stakx - no longer contributing

83,039
20
168
268

answered Mar 22 '11 at 08:32

Alex Y.

1
1

1

The `RLIMIT_AS` address space limitation only limits the _number_ of pages a process may allocate, not which addresses may be allocated. See `may_expand_vm()` in `mm/mmap.c`. – sarnold Mar 22 '11 at 08:40

How to limit a 64-bit process address space to less than 4G?

5 Answers5

Linked