gdb reverse debugging avx2

Question

So I have a new fancy cpu that supports avx2 instruction set. This is great, but breaks gdb reverse debugging. When compiling with no optimisations code still uses shared libraries, eg calls memset() which then goes and invokes an avx2 optimised version of memset. This is great but avx2 is not supported by gdb record.

process record does not support instruction 0xc5 at address 0x7ffff690dd80.

0xc5 is the vex prefix here.

reverse debugging works great with a cpu that does not support avx2. How do I get libc etc to not use the avx2 optimised versions of library calls so I can use gdb record, step backwards etc?

I've tried

LD_BIND_NOW=1
LD_HWCAP_MASK=0
compiling with -static

And short of debugging on an old machine, I'm out of ideas.

2.23 as shipped on ubuntu 16.04 amd64 arch. cpu is an i7 xeon that supports avx2 — Hal, May 03 '17 at 07:41
Was already asked here: http://stackoverflow.com/q/42451492/72178. — ks1322, May 03 '17 at 09:27
You could try a [pre-5.0 version of virtualbox](http://stackoverflow.com/questions/24543874/avx-inside-a-virtualbox-vm) — Mark Plotnick, May 03 '17 at 21:59
Hello. As required, I just documented my kludge in the: stackoverflow.com/q/42451492 as https://stackoverflow.com/a/44468494 (`jg` bit patched manually into `jle` to disable all avx2 detection in the `ld-linux.so.2`) — osgx, Jun 10 '17 at 01:05
Possible duplicate of [Disable AVX-optimized functions in glibc (LD\_HWCAP\_MASK, /etc/ld.so.nohwcap) for valgrind & gdb record](https://stackoverflow.com/questions/42451492/disable-avx-optimized-functions-in-glibc-ld-hwcap-mask-etc-ld-so-nohwcap-for) — Ciro Santilli OurBigBook.com, Sep 08 '17 at 09:38

score 2 · Accepted Answer · answered Jun 10 '17 at 01:52

For your glibc 2.23 as shipped on ubuntu 16.04 amd64 there is adaptation of my binary kludge (1 bit patch) made exactly for the same reason. Package libc6 (2.23-0ubuntu7) was downloaded from https://packages.ubuntu.com/xenial/amd64/libc6 and file ld-2.23.so was edited (keep copy of the original, or save patched into different path and change INTERP section of your own binary to use different path):

 83 3D 5B C9 20 00 06   cmpl $0x6, smth...
 7E 21                  jle  some_forward_label
 B8 07 00 00 00         mov $0x7, %eax
 31 C9                  xor %ecx,%ecx
 0F A2                  cpuid

There is code of the get_common_indeces: if (cpu_features->max_cpuid >= 7) __cpuid_count (7, 0, ... called from __get_cpu_features. EAX=7 leaf of cpuid has all info needed to detect AVX2 support and enable it, so I just skipped fragment with cpuid eax=0x7,ecx=0 and saving its results into some parts of memory by changing 0x7e 0x21 into 0x7f 0x21.

So, the binary patch is like replacing 83 3D xx xx xx xx 06 7E xx B8 07 00 00 00 31 C9 0F A2 (where xx may be any byte) into 83 3D xx xx xx xx 06 7F xx B8 07 00 00 00 31 C9 0F A2. You may do this with any Hex Editor or by some binary diff. In 2.23-0ubuntu7 this code is at 0x0193B0 - 0x0193B9 is 7e to be changed into 7f.

The patch it dumb and don't use patched file globally if your root fs may be started with CPU without eax=7 cpuid leaf support (pre Intel Core CPU) or in virtual machine emulating such pre Intel Core CPU ("Pentium D 8xx/9xx", Pentium 4, Pentium M - will fail).

You may put patched file at path which name is equal in length or shorter to the original path of /lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 (which symlinks into /lib/x86_64-linux-gnu/ld-2.23.so file). For example as /lib_x86_64-linux-gnu_ld-linux-noAVX2.so.2. Then use same hexeditor to replace the string "/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2" of your program executable (ELF) into "/lib_x86_64-linux-gnu_ld-linux-noAVX2.so.2", or use patchelf tool from the patchelf package:

cp /lib/x86_64-linux-gnu/ld-2.23.so /lib_x86_64-linux-gnu_ld-linux-noAVX2.so.2

bless  /lib_x86_64-linux-gnu_ld-linux-noAVX2.so.2
# or any other hex editor

patchelf --set-interpreter /lib_x86_64-linux-gnu_ld-linux-no-AVX2.so.2 ./my_program

If I could award retrospective bounty on an answer this would qualify. Fantastic content, well written and the solution checked out and was used successfully. Cheers. — Hal, May 16 '18 at 03:11
@Hal, There should be "start a bounty" button below the question with reason of "Reward existing answer - One or more of the answers is exemplary and worthy of an additional bounty". Answer should be checked/updated to 18.04 LTS; and in some linked question there was scripted binary patch (no manual binary editing), probably in https://stackoverflow.com/questions/42451492 with `re.sub` of the python. Another solution: check https://github.com/mozilla/rr RR reverse debugging tool from Mozilla for Firefox (thanks to https://stackoverflow.com/questions/42451492#comment79266853_42451492). — osgx, May 16 '18 at 04:35
For some reason this method didn't work for me on Ubuntu 16.04 as of today (25/04/19). But the Python script in this answer patches something slightly different and did work: https://stackoverflow.com/questions/42451492/disable-avx-optimized-functions-in-glibc-ld-hwcap-mask-etc-ld-so-nohwcap-for?rq=1 — silverscania, Apr 25 '19 at 14:53

gdb reverse debugging avx2

1 Answers1

Linked