What is your favourite anti-debugging trick?

Question

At my previous employer we used a third party component which basically was just a DLL and a header file. That particular module handled printing in Win32. However, the company that made the component went bankcrupt so I couldn't report a bug I'd found.

So I decided to fix the bug myself and launched the debugger. I was surprised to find anti-debugging code almost everywhere, the usual IsDebuggerPresent, but the thing that caught my attention was this:

    ; some twiddling with xor 
    ; and data, result in eax 
    jmp eax 
    mov eax, 0x310fac09 
    ; rest of code here 

At the first glance I just stepped over the routine which was called twice, then things just went bananas. After a while I realized that the bit twiddling result was always the same, i.e. the jmp eax always jumped right into the mov eax, 0x310fac09 instruction. I dissected the bytes and there it was, 0f31, the rdtsc instruction which was used to measure the time spent between some calls in the DLL.

So my question to SO is: What is your favourite anti-debugging trick?

Answer 1

My favorite trick is to write a simple instruction emulator for an obscure microprocessor.

The copy protection and some of the core functionality will then compiled for the microprocessor (GCC is a great help here) and linked into the program as a binary blob.

The idea behind this is, that the copy protection does not exist in ordinary x86 code and as such cannot be disassembled. You cannot remove the entire emulator either because this would remove core functionality from the program.

The only chance to hack the program is to reverse engineer what the microprocessor emulator does.

I've used MIPS32 for emulation because it was so easy to emulate (it took just 500 lines of simple C-code). To make things even more obscure I didn't used the raw MIPS32 opcodes. Instead each opcode was xor'ed with it's own address.

The binary of the copy protection looked like garbage-data.

Highly recommended! It took more than 6 month before a crack came out (it was for a game-project).

Answer 2

I've been a member of many RCE communities and have had my fair share of hacking & cracking. From my time I've realized that such flimsy tricks are usually volatile and rather futile. Most of the generic anti-debugging tricks are OS specific and not 'portable' at all.

In the aforementioned example, you're presumably using inline assembly and a naked function __declspec, both which are not supported by MSVC when compiling on the x64 architecture. There are of course still ways to implement the aforementioned trick but anybody who has been reversing for long enough will be able to spot and defeat that trick in a matter of minutes.

So generally I'd suggest against using anti-debugging tricks outside of utilizing the IsDebuggerPresent API for detection. Instead, I'd suggest you code a stub and/or a virtual machine. I coded my own virtual machine and have been improving on it for many years now and I can honestly say that it has been by far the best decision I've made in regards to protecting my code so far.

Answer 3

Spin off a child process that attaches to parent as a debugger & modifies key variables. Bonus points for keeping the child process resident and using the debugger memory operations as a kind of IPC for certain key operations.

On my system, you can't attach two debuggers to the same process.

Nice thing about this one is unless they try to tamper w/ things nothing breaks.

Answer 4

Reference uninitialized memory! (And other black magic/vodoo...)

This is a very cool read: http://spareclockcycles.org/2012/02/14/stack-necromancy-defeating-debuggers-by-raising-the-dead/

Answer 5

The most modern obfuscation method seems to be the virtual machine.

You basically take some part of your object code, and convert it to your own bytecode format. Then you add a small virtual machine to run this code. Only way to properly debug this code will be to code an emulator or disassembler for your VM's instruction format. Of course you need to think of performance too. Too much bytecode will make your program run slower than native code.

Most old tricks are useless now:

• Isdebuggerpresent : very lame and easy to patch
• Other debugger/breakpoint detections
• Ring0 stuff : users don't like to install drivers, you might actually break something on their system etc.
• Other trivial stuff that everybody knows, or that makes your software unstable. remember that even if a crack makes your program unstable but it still works, this unstability will be blamed on you.

If you really want to code the VM solution yourself (there are good programs for sale), don't use just one instruction format. Make it polymorphic, so that you can have different parts of the code have different format. This way all your code can't be broken by writing just one emulator/disassembler. For example MIPS solution some people offered seems to be easily broken because MIPS instruction format is well documented and analysis tools like IDA can already disassemble the code.

List of instruction formats supported by IDA pro disassembler

Answer 6

I would prefer that people write software that is solid, reliable and does what it is advertised to do. That they also sell it for a reasonable price with a reasonable license.

I know that I have wasted way too much time dealing with vendors that have complicated licensing schemes that only cause problems for the customers and the vendors. It is always my recommendation to avoid those vendors. Working at a nuclear power plant we are forced to use certain vendors products and thus are forced to have to deal with their licensing schemes. I wish there was a way to get back the time that I have personally wasted dealing with their failed attempts to give us a working licensed product. It seems like a small thing to ask, but yet it seems to be a difficult thing for people that get too tricky for their own good.

Answer 7

I second the virtual machine suggestion. I implemented a MIPS I simulator that (now) can execute binaries generated with mipsel-elf-gcc. Add to that code/data encryption capabilities (AES or with any other algorithm of your choice), the ability of self-simulation (so you can have nested simulators) and you have a pretty good code obfuscator.

The nice feature of choosing MIPS I is that 1) it's easy to implement, 2) I can write code in C, debug it on my desktop and just cross-compile it for MIPS when it's done. No need to debug custom opcodes or manually write code for a custom VM..

Answer 8

My personal favourite was on the Amiga, where there is a coprocessor (the Blitter) doing large data transfers independent from the processor; this chip would be instructed to clear all memory, and reset from a timer IRQ.

When you attached an Action Replay cartridge, stopping the CPU would mean that the Blitter would continue clearing the memory.

Answer 9

Calculated jumps in the middle of a legitimate looking but really hiding an actual instruction instructions are my favorite. They are pretty easy to detect for humans anyway, but automated tools often mess it up.

Also replacing a return address on the stack makes a good time waster.

Answer 10

Using nop to remove assembly via the debugger is a useful trick. Of course, putting the code back is a lot harder!!!