How to detect first file in the stream

Question

I have source files in Cp1250 encoding. All of those file are in dirName directory or its subdirectories. I would like to merge them into one utf-8 file by adding their contents. Unfortunately I get empty line at the beginning of result file.

public static void processDir(String dirName, String resultFileName) {
    try {
        File resultFile = new File(resultFileName);
        BufferedWriter bw = new BufferedWriter(new OutputStreamWriter(new FileOutputStream(resultFile), "utf-8"));
        Files.walk(Paths.get(dirName)).filter(Files::isRegularFile).forEach((path) -> {
            try {
                Files.readAllLines(path, Charset.forName("Windows-1250")).stream().forEach((line) -> {
                    try {
                        bw.newLine();
                        bw.write(line);                     
                    } catch (Exception e) {
                        e.printStackTrace();
                    }
                });
            } catch (Exception e) {
                e.printStackTrace();
            }
        });
        bw.close();
    } catch (Exception e) {
        e.printStackTrace();
    }
}

The reason is that I don't know how to detect the first file in my stream.

---

I came up with extremely stupid solution which does not rely on streams so it is unsatisfactory:

public static void processDir(String dirName, String resultFileName) {
        try {
            File resultFile = new File(resultFileName);
            BufferedWriter bw = new BufferedWriter(new OutputStreamWriter(new FileOutputStream(resultFile), "utf-8"));
            Files.walk(Paths.get(dirName)).filter(Files::isRegularFile).forEach((path) -> {
                try {
                    Files.readAllLines(path, Charset.forName("Windows-1250")).stream().forEach((line) -> {
                        try {
                            if(resultFile.length() != 0){
                                bw.newLine();
                            }
                            bw.write(line);
                            if(resultFile.length() == 0){
                                bw.flush();
                            }
                        } catch (Exception e) {
                            e.printStackTrace();
                        }
                    });
                } catch (Exception e) {
                    e.printStackTrace();
                }
            });
            bw.close();
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

Also I could use static boolean but that is total gibberish.

Answer 1

You can use the flatMap to create the stream of all lines of all files, then use flatMap again to interleave it with line separator, then use skip(1) to skip the leading separator like this:

public static void processDir(String dirName, String resultFileName) {
    try(BufferedWriter bw = Files.newBufferedWriter(Paths.get(resultFileName))) {
        Files.walk(Paths.get(dirName)).filter(Files::isRegularFile)
            .flatMap(path -> {
                try {
                    return Files.lines(path, Charset.forName("Windows-1250"));
                } catch (IOException e) {
                    throw new UncheckedIOException(e);
                }
            })
            .flatMap(line -> Stream.of(System.lineSeparator(), line))
            .skip(1)
            .forEach(line -> {
                try {
                    bw.write(line);
                } catch (IOException e) {
                    throw new UncheckedIOException(e);
                }
            });
    } catch (IOException e) {
        throw new UncheckedIOException(e);
    }
}

In general using flatMap+skip combination can help to solve many similar problems.

Also note the Files.newBufferedWriter method which is simpler way to create BufferedWriter. And don't forget about try-with-resources.

Answer 2

Rethink your strategy. If you want to join files and neither, remove nor convert, line terminators, there is no reason to process lines. It seems, the only reason for you to write code processing lines, is, that you have a demand to bail lambda expressions and streams into the solution and the only possibility offered by the current API is to process streams of lines. But obviously, they are not the right tool for the job:

public static void processDir(String dirName, String resultFileName) throws IOException {
    Charset cp1250 = Charset.forName("Windows-1250");
    CharBuffer buffer=CharBuffer.allocate(8192);
    try(BufferedWriter bw
          =Files.newBufferedWriter(Paths.get(resultFileName), CREATE, TRUNCATE_EXISTING)) {
        Files.walkFileTree(Paths.get(dirName), new SimpleFileVisitor<Path>() {
            @Override public FileVisitResult visitFile(
                             Path path, BasicFileAttributes attrs) throws IOException {
                try(BufferedReader r=Files.newBufferedReader(path, cp1250)) {
                    while(r.read(buffer)>0) {
                        bw.write(buffer.array(), buffer.arrayOffset(), buffer.position());
                        buffer.clear();
                    }
                }
                return FileVisitResult.CONTINUE;
            }
        });
        bw.close();
    }
}

Note how this solution solves the problems of your first attempt. You don’t have to deal with line terminators here, this code doesn’t even waste resources in trying to find them in the input. All it does, is performing the charset conversion on chunks of input data and writing them to the target. The performance difference can be significant.

Further, the code isn’t cluttered with catching exceptions, that you can’t handle. If an IOException occurs at any place of the operation, all pending resources are properly closed and the exception is relayed to the caller.

Granted, it just uses a good old inner class instead of a lambda expression. But it doesn’t reduce the readability compared to your attempt. If it still really bothers you that there is no lambda expression involved, you may check this question & answer for a way to bring them in again.