Java - Use Input and OutputStream of ProcessBuilder continuously

Question

I want to use an external tool while extracting some data (loop through lines). For that I first used Runtime.getRuntime().exec() to execute it. But then my extraction got really slow. So I am searching for a possibility to exec the external tool in each instance of the loop, using the same instance of shell.

I found out, that I should use ProcessBuilder. But it's not working yet.

Here is my code to test the execution (with input from the answers here in the forum already):

public class ExecuteShell {
   ProcessBuilder builder;
   Process process = null;
   BufferedWriter process_stdin;
   BufferedReader reader, errReader;

   public ExecuteShell() {
    String command;
    command = getShellCommandForOperatingSystem();
    if(command.equals("")) {
        return; //Fehler!  No error handling yet
    }
    //init shell
    builder = new ProcessBuilder( command);
    builder.redirectErrorStream(true);
    try {
        process = builder.start();
    } catch (IOException e) {
        System.out.println(e);
    }

    //get stdout of shell  
    reader    = new BufferedReader(new InputStreamReader(process.getInputStream()));  
    errReader = new BufferedReader(new InputStreamReader(process.getErrorStream()));

    //get stdin of shell
    process_stdin = new BufferedWriter(new OutputStreamWriter(process.getOutputStream()));
    System.out.println("ExecuteShell: Constructor successfully finished");
   }

   public String executeCommand(String commands) {
    StringBuffer output;
    String line;
    try {
        //single execution
        process_stdin.write(commands);
        process_stdin.newLine();
        process_stdin.flush();
    } catch (IOException e) {
        System.out.println(e);
    }
    output    = new StringBuffer();
    line = ""; 

    try {
        if (!reader.ready()) {
            output.append("Reader empty \n");
            return output.toString();
        }
        while ((line = reader.readLine())!= null) {
            output.append(line + "\n");
            return output.toString();
        }
        if (!reader.ready()) {
            output.append("errReader empty \n");
            return output.toString();
        }
        while ((line = errReader.readLine())!= null) {
            output.append(line + "\n");
        }
    } catch (Exception e) {
        System.out.println("ExecuteShell: error in executeShell2File");
        e.printStackTrace();
        return "";
    }
    return output.toString();
   }


   public int close() {
    // finally close the shell by execution exit command
    try {
        process_stdin.write("exit");
        process_stdin.newLine();
        process_stdin.flush();
    }
    catch (IOException e) {
        System.out.println(e);
        return 1;
    }
    return 0;
   }

   private static String getShellCommandForOperatingSystem() {
    Properties prop = System.getProperties( );
    String os =  prop.getProperty( "os.name" );
    if ( os.startsWith("Windows") ) {
        //System.out.println("WINDOWS!");
        return "C:/cygwin64/bin/bash";
    } else if (os.startsWith("Linux") ) { 
        //System.out.println("Linux!");
        return"/bin/sh";
    }
    return "";      
   }
}

I want to call it in another Class like this Testclass:

public class TestExec{
    public static void main(String[] args) {
        String result = "";
        ExecuteShell es = new ExecuteShell();
        for (int i=0; i<5; i++) {
          // do something
          result = es.executeCommand("date"); //execute some command
          System.out.println("result:\n" + result); //do something with result
          // do something
        }
        es.close();
    }
}

My Problem is, that the output stream is always empty:

ExecuteShell: Constructor successfully finished
result:
Reader empty 

result:
Reader empty 

result:
Reader empty 

result:
Reader empty 

result:
Reader empty 

I read the thread here: Java Process with Input/Output Stream

But the code snippets were not enough to get me going, I am missing something. I have not really worked with different threads much. And I am not sure if/how a Scanner is of any help to me. I would really appreciate some help.

Ultimatively, my goal is to call an external command repeatetly and make it fast.

EDIT: I changed the loop, so that the es.close() is outside. And I wanted to add, that I do not want only this inside the loop.

EDIT: The problem with the time was, that the command I called caused an error. When the command does not cause an error, the time is acceptable.

Thank you for your answers

Answer 1

You are probably experiencing a race condition: after writing the command to the shell, your Java program continues to run, and almost immediately calls reader.ready(). The command you wanted to execute has probably not yet output anything, so the reader has no data available. An alternative explanation would be that the command does not write anything to stdout, but only to stderr (or the shell, maybe it has failed to start the command?). You are however not reading from stderr in practice.

To properly handle output and error streams, you cannot check reader.ready() but need to call readLine() (which waits until data is available) in a loop. With your code, even if the program would come to that point, you would read only exactly one line from the output. If the program would output more than one line, this data would get interpreted as the output of the next command. The typical solution is to read in a loop until readLine() returns null, but this does not work here because this would mean your program would wait in this loop until the shell terminates (which would never happen, so it would just hang infinitely). Fixing this would be pretty much impossible, if you do not know exactly how many lines each command will write to stdout and stderr.

However, your complicated approach of using a shell and sending commands to it is probably completely unnecessary. Starting a command from within your Java program and from within the shell is equally fast, and much easier to write. Similarly, there is no performance difference between Runtime.exec() and ProcessBuilder (the former just calls the latter), you only need ProcessBuilder if you need its advanced features.

If you are experiencing performance problems when calling external programs, you should find out where they are exactly and try to solve them, but not with this approach. For example, normally one starts a thread for reading from both the output and the error stream (if you do not start separate threads and the command produces large output, everything might hang). This could be slow, so you could use a thread pool to avoid repeated spawning of processes.