Why i get an "java.io.IOException: Stream closed" in Linux with ExecutorService when reading the thread's stream?

Question

I have a java program that pings a list of ip addresses using an Executor and a thread pool and it prints the result on the screen.

It works fine in a Windows environment but that when i run it in linux I often get an "java.io.IOException: Stream closed" error. But not for all threads.

Name_49: xxx.xxx.xxx.3: 2.558
Name_50: xxx.xxx.xxx.56: 0.419
Name_44: Endsjava.io.IOException: Stream closed
    at java.io.BufferedInputStream.getBufIfOpen(BufferedInputStream.java:162)
    at java.io.BufferedInputStream.read(BufferedInputStream.java:325)
    at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:283)

Here is my main action:

ArrayList<String> ips= new ArrayList<>();
    elements.add("123.123.123.123");
// etc..

 ExecutorService executor = (ExecutorService) Executors.newCachedThreadPool();
    List<Task> taskList = new ArrayList<>();
    for (int i = 0; i < ips.size(); i++) {
        Task task = new Task("Name_"+ i, elements.get(i));
        taskList.add(task);
    }

    List<Future<Result>> resultList = null;

    try {
        resultList = executor.invokeAll(taskList);
    } catch (InterruptedException e) {
        e.printStackTrace();
    }

    executor.shutdown();

    System.out.println("Main: Printing the results");
    for (int i = 0; i < resultList.size(); i++) {
        Future<Result> future = resultList.get(i);
        try {
            Result result = future.get();
            System.out.println(result.getName() + ": " + result.getIP() + ": " + result.getPing() );
        } catch (InterruptedException | ExecutionException e) {
            e.printStackTrace();
        }
    }

The pinging happens in a callable class with a result object:

public class Task implements Callable<Result> {

private String name;
private String ping;
private String ip;

public Task(String name, String ip) {
    this.name = name;
    this.ip = ip;
}

@Override
public Result call() throws Exception {
    System.out.printf("%s: Starting\n", this.name);

    Result result = new Result();
    String resultstr = linuxPingServer(this.ip);
    //String resultstr = winPingServer(this.ip);

    result.setName(this.name);
    result.setIP(this.ip);
    result.setPing(resultstr);

    System.out.println(this.name + ": Ends");
    return result;
}

private String runCMD(String cmd) {
    Process p = null;
    ReadStream s1 = null;
    ReadStream s2 = null;
    String outerr = "";
    String outstd = "";
    String output = "";

    try {
        p = Runtime.getRuntime().exec(cmd);

        s1 = new ReadStream("stdin", p.getInputStream());
        s2 = new ReadStream("stderr", p.getErrorStream());
        s1.start();
        s2.start();
        p.waitFor();

        //s2.returnOut();
        outstd = s1.returnOut();
        outerr = s2.returnOut();

        if (outstd != null && !outstd.isEmpty()) {
            output = outstd;
        } else //if (outerr != null && !outerr.isEmpty())
        {
            output = outerr;
        }

    } catch (IOException | InterruptedException ex) {
        System.out.println("StreamCheck_Exception - BashExec" + ex.getMessage());
    } finally {
        if (p != null) {
            p.destroy();
        }
    }
    return output;
}

public String linuxPingServer(String ip) {

    // Ping with reduced interval ( -i 0.1secs)
    // ping count = 5
    String pingCMD = "ping -i 0.1 -c5 " + ip;

    String result1 = "";
    String result2 = "";
    String result3 = "";

    result1 = runCMD(pingCMD);

    // split string on end of line
    String[] lines = result1.split(System.getProperty("line.separator"));

    // get statistics ( ussually last line )
    result2 = lines[lines.length - 1];

    // parse result -avg is third value ( fifth "/" )
    lines = result2.split("/");
    result3 = lines[4];

    return result3;

}

and here is my reader class:

public class ReadStream implements Runnable {
String name;
InputStream is;
Thread thread;      
StringBuffer output;

public ReadStream(String name, InputStream is) {
    this.name = name;
    this.is = is;
    this.output = new StringBuffer();
}       

public void start () {
    thread = new Thread (this);
    thread.start ();
}       

public void run () {
    try {
        InputStreamReader isr = new InputStreamReader (is);
        BufferedReader br = new BufferedReader (isr);   
        while (true) {
            String s = br.readLine ();
            if (s == null) break;
            this.output.append(s + "\n");
        }
        is.close ();    
    } catch (Exception ex) {
        System.out.println ("Problem reading stream " + name + "... :" + ex);
        ex.printStackTrace ();
    }
}

I am baffled why is this happening. Can someone please explain to me what is wrong with the operation I am performing?

EDITED - more trace -

java -jar WinThreadPing.jar
Name_0: Starting
Name_52: Starting
Name_50: Starting
Name_51: Starting
Name_53: Starting
Name_48: Starting
Name_47: Starting
//.. more Starting threads
Name_98: Staring
Name_99: Staring
Name_59: Ends
Name_57: Ends
Name_60: Ends
Name_56: Ends
Name_45: Ends
Name_50: Ends
Name_48: Ends
Name_51: Ends
Name_39: Ends
Name_52: Ends
Problem reading stream stdin... :java.io.IOException: Stream closed
Name_43: Ends
java.io.IOException: Stream closedName_53: Ends
Name_42: Ends

Name_38: Ends
        at java.io.BufferedInputStream.getBufIfOpen(BufferedInputStream.java:162)
Name_47: Ends   at java.io.BufferedInputStream.read(BufferedInputStream.java:325)

        at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:283)
Problem reading stream stdin... :java.io.IOException: Stream closed     at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:325)

        at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:177)
        at java.io.InputStreamReader.read(InputStreamReader.java:184)
        at java.io.BufferedReader.fill(BufferedReader.java:154)
        at java.io.BufferedReader.readLine(BufferedReader.java:317)
Name_34: Ends   at java.io.BufferedReader.readLine(BufferedReader.java:382)
        at WinThreadedPing.ReadStream.run(ReadStream.java:40)
        at java.lang.Thread.run(Thread.java:745)

java.io.IOException: Stream closedName_36: Ends

        at java.io.BufferedInputStream.getBufIfOpen(BufferedInputStream.java:162)
        at java.io.BufferedInputStream.read1(BufferedInputStream.java:272)
        at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
        at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:283)
Problem reading stream stdin... :java.io.IOException: Stream closed     at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:325)

        at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:177)
        at java.io.InputStreamReader.read(InputStreamReader.java:184)
        at java.io.BufferedReader.fill(BufferedReader.java:154)
        at java.io.BufferedReader.readLine(BufferedReader.java:317)
Name_44: Ends   at java.io.BufferedReader.readLine(BufferedReader.java:382)
        at WinThreadedPing.ReadStream.run(ReadStream.java:40)
        at java.lang.Thread.run(Thread.java:745)
java.io.IOException: Stream closed
        at java.io.BufferedInputStream.getBufIfOpen(BufferedInputStream.java:162)
        at java.io.BufferedInputStream.read1(BufferedInputStream.java:272)
        at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
        at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:283)
        at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:325)
        at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:177)

Name_46: Ends
        at java.io.InputStreamReader.read(InputStreamReader.java:184)
        at java.io.BufferedReader.fill(BufferedReader.java:154)Name_29: Ends

        at java.io.BufferedReader.readLine(BufferedReader.java:317)
        at java.io.BufferedReader.readLine(BufferedReader.java:382)
        at WinThreadedPing.ReadStream.run(ReadStream.java:40)
        at java.lang.Thread.run(Thread.java:745)
Name_27: Ends
Problem reading stream stdin... :java.io.IOException: Stream closed
Problem reading stream stdin... :java.io.IOException: Stream closed
Name_26: Ends
Problem reading stream stdin... :java.io.IOException: Stream closed
Name_22: Ends
Name_28: Ends
Problem reading stream stdin... :java.io.IOException: Stream closed
java.io.IOException: Stream closed
Problem reading stream stdin... :java.io.IOException: Stream closed
        at java.io.BufferedInputStream.getBufIfOpen(BufferedInputStream.java:162)Problem reading stream stdin... :java.io.IOException: Stream closed

        at java.io.BufferedInputStream.read(BufferedInputStream.java:325)Name_23: Ends
        at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:283)

        at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:325)
        at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:177)Problem reading stream stdin... :java.io.IOException: Stream closed
        at java.io.InputStreamReader.read(InputStreamReader.java:184)

Name_17: Ends
Problem reading stream stdin... :java.io.IOException: Stream closed
Name_15: Ends
Name_18: Ends
Name_16: Ends
        at java.io.BufferedReader.fill(BufferedReader.java:154)
        at java.io.BufferedReader.readLine(BufferedReader.java:317)
Problem reading stream stdin... :java.io.IOException: Stream closed
        at java.io.BufferedReader.readLine(BufferedReader.java:382)
Name_20: Ends   at WinThreadedPing.ReadStream.run(ReadStream.java:40)
Name_14: Ends
Problem reading stream stdin... :java.io.IOException: Stream closed
Problem reading stream stdin... :java.io.IOException: Stream closed

        at java.lang.Thread.run(Thread.java:745)Name_10: Ends

Problem reading stream stdin... :java.io.IOException: Stream closed
Name_19: Endsjava.io.IOException: Stream closed

Name_7: Ends
Name_8: Ends
        at java.io.BufferedInputStream.getBufIfOpen(BufferedInputStream.java:162)
        at java.io.BufferedInputStream.read(BufferedInputStream.java:325)
        at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:283)
        at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:325)
        at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:177)
        at java.io.InputStreamReader.read(InputStreamReader.java:184)
Name_11: Ends
Problem reading stream stdin... :java.io.IOException: Stream closed
Name_4: Ends
Problem reading stream stdin... :java.io.IOException: Stream closed
Name_5: Ends    at java.io.BufferedReader.fill(BufferedReader.java:154)

        at java.io.BufferedReader.readLine(BufferedReader.java:317)
        at java.io.BufferedReader.readLine(BufferedReader.java:382)
        at WinThreadedPing.ReadStream.run(ReadStream.java:40)
        at java.lang.Thread.run(Thread.java:745)
Problem reading stream stdin... :java.io.IOException: Stream closed
Name_62: Ends
//..etc

I didn´t see that you called invokeAll(). With that you do not need to call to awaitTermination() — Victor, Feb 09 '15 at 11:58
Could we see some more of that stack trace? There's not enough information here to connect it to your code. — Kenster, Feb 09 '15 at 12:29

score 2 · Answer 1 · edited May 23 '17 at 12:27

I am answering my own question.

After some experimentation I was able to correct the problem using ProcessBuilder and an answer for ProcessBuilder OpenFiles problem :

ProcessBuilder pb = new ProcessBuilder("bash", "-c", cmd);

        pb.redirectError(new File(this.Filepath + ".err"));
        pb.redirectOutput(new File(this.Filepath));

        Process shell = pb.start();

        //shell.wait();
        int exitVal= shell.waitFor();

        //missing these was causing the mass amounts of open 'files'
        shell.getInputStream().close();
        shell.getOutputStream().close();
        shell.getErrorStream().close();

Why i get an "java.io.IOException: Stream closed" in Linux with ExecutorService when reading the thread's stream?

1 Answers1