1

I currently hava a running Java application, which has a bug. I don't know how to fully reproduce it and it didn't happen for weeks until now. When it occurs one times, I can reproduce it over and over again easily until I restart the application. The bug causes a StackOverflowError because of a recursion and I don't know how this happens. The printed stacktrace of the StackOverflowError isn't helpful because it contains only the repeating part, but not the more insteresting initial part, because the JVM has a limit for stacktrace entries. The -XX:MaxJavaStackTraceDepth=... can be used to set this limit as explained here. The problem is that I think I have to restart my application in order to add this flag. But if I do so, I won't be able to reproduce the bug anymore. Is there any solution how I can get the full stacktrace or set this flag without restarting the application?

stonar96
  • 1,359
  • 2
  • 11
  • 39
  • 2
    You can attach a debugger to the running application to inspect the entire stack. – nanofarad Dec 10 '20 at 17:57
  • What's wrong with the question? Why do people downvote this? – stonar96 Dec 10 '20 at 18:01
  • 1
    I don't know, as I didn't downvote (but I think your comment got routed back to me as a ping because nobody else commented). I think it's a reasonable question, and I gave a suggestion, but I don't have the time to write a full answer to high quality at the moment. – nanofarad Dec 10 '20 at 18:05
  • 1
    Sorry my comment was not targeting you but it was rather a general question what I can improve. I will try to attach a debugger, thanks. – stonar96 Dec 10 '20 at 18:12
  • It seems like this is not possible. According to https://docs.oracle.com/javase/7/docs/technotes/tools/windows/jdb.html a VM that is to be debugged with jdb must be started with the options `-agentlib:jdwp=transport=dt_shmem,server=y,suspend=n` – stonar96 Dec 10 '20 at 18:31
  • Ack - this is part of the reason why I was remiss to write a full answer. – nanofarad Dec 10 '20 at 18:58

1 Answers1

4

I know at least two solutions.

  1. Create HotSpot Serviceability Agent tool to find the address of MaxJavaStackTraceDepth variable in memory, and then update the memory of the process using OS-specific mechanism.

  2. Attach a JVM TI agent that intercepts StackOverflowErrors and prints a stack trace right from the agent.

Here is the code for the first solution (as it is presumably shorter):

import sun.jvm.hotspot.debugger.Address;
import sun.jvm.hotspot.runtime.VM;
import sun.jvm.hotspot.tools.Tool;

import java.io.IOException;
import java.io.RandomAccessFile;

public class ChangeVMFlag extends Tool {
    private static String pid;

    @Override
    public void run() {
        Address addr = VM.getVM().getCommandLineFlag("MaxJavaStackTraceDepth").getAddress();
        long addrValue = VM.getVM().getDebugger().getAddressValue(addr);

        try (RandomAccessFile raf = new RandomAccessFile("/proc/" + pid + "/mem", "rw")) {
            raf.seek(addrValue);
            raf.writeInt(Integer.reverseBytes(1_000_000));
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

    public static void main(String[] args) {
        pid = args[0];
        new ChangeVMFlag().execute(new String[]{pid});
    }
}

This tool changes the value of MaxJavaStackTraceDepth in the target process to 1 million.
Note: it uses Linux-specific /proc API to write into the target process' memory. Other OSes have different interfaces.

How to run

On JDK 8

java -cp .:$JAVA_HOME/lib/sa-jdi.jar ChangeVMFlag <pid>

On JDK 9+

java --add-modules=jdk.hotspot.agent \
     --add-exports jdk.hotspot.agent/sun.jvm.hotspot.tools=ALL-UNNAMED \
     --add-exports jdk.hotspot.agent/sun.jvm.hotspot.runtime=ALL-UNNAMED \
     --add-exports jdk.hotspot.agent/sun.jvm.hotspot.debugger=ALL-UNNAMED \
     ChangeVMFlag <pid>
apangin
  • 92,924
  • 10
  • 193
  • 247