Java file not written to stream with new line characters

Question

We're streaming a CSV file from a web service. It appears that we're losing the new line characters when streaming - the client gets the file all on a single line. Any idea what we're doing wrong?

Code:

 public static void writeFile(OutputStream out, File file) throws IOException {
    BufferedReader input = new BufferedReader(new FileReader(file)); //File input stream 
    String line;
    while ((line = input.readLine()) != null) { //Read file
        out.write(line.getBytes());  //Write to output stream 
        out.flush();
    }
    input.close();
}

BalusC · Accepted Answer · 2009-12-30T18:45:53.793

11

Don't use BufferedReader. You already have an OutputStream at hands, so just get an InputStream of the file and pipe the bytes from input to output it the usual Java IO way. This way you also don't need to worry about newlines being eaten by BufferedReader:

public static void writeFile(OutputStream output, File file) throws IOException {
    InputStream input = null;
    byte[] buffer = new byte[10240]; // 10KB.
    try {
        input = new FileInputStream(file);
        for (int length = 0; (length = input.read(buffer)) > 0;) {
            output.write(buffer, 0, length);
        }
    } finally {
        if (input != null) try { input.close(); } catch (IOException logOrIgnore) {}
    }
}

Using a Reader/Writer would involve character encoding problems if you don't know/specify the encoding beforehand. You actually also don't need to know about them here. So just leave it aside.

To improve performance a bit more, you can always wrap the InputStream and OutputStream in an BufferedInputStream and BufferedOutputStream respectively.

edited Dec 30 '09 at 18:45

answered Dec 30 '09 at 18:11

BalusC

1,082,665
372
3,610
3,555

10KB is kinda big and will inhibit scalability in a popular/much-used usage scenario. 1k-2k is just fine. And, yes. Use a `BufferedOutputStream` for the writing. – Stu Thompson Dec 30 '09 at 18:43
@Stu: BufferedInputStream/BufferedOutputStream uses 8KB under the hoods and are configureable by 2nd constructor argument. I just use 10KB all the time, it's certainly enough with cheap RAM nowadays. – BalusC Dec 30 '09 at 18:47
@BalusC: Is that not JVM implementation and/or Java version dependent? Anyway, yes, RAM is cheap. I guess the fact that my 'baby' *(a heavy IO webapp with many users)* has the need to keep memory down is influencing my perspective. A 5x or 10x improvement in that narrow aspect can be important for some, like myself. **A last thought:** Unless one has a good reason to, and understands what they are doing, using the default is almost always best. – Stu Thompson Jan 02 '10 at 16:45

Nathan Hughes · Answer 2 · 2009-12-30T18:26:38.227

The readline method uses the newline chars to delimit what gets read, so the newlines themselves are not returned by readLine.

Don't use readline, you can use a BufferedInputStream and read the file one byte at a time if you want, or pass your own buffer into OutputStream.write.

Note that, like BalusC and Michael Borgwardt say, Readers and Writers are for text, if you just want to copy the file you should use InputStream and OutputStream, you are only concerned with bytes.

score 4 · Answer 3 · edited May 23 '17 at 10:34

4

Any idea what we're doing wrong?

Yes. This line drops the "new line character"

while ((line = input.readLine()) != null) {

And then you write it without it:

 out.write(line.getBytes());

This this related question.

edited May 23 '17 at 10:34

Community

1
1

answered Dec 30 '09 at 18:24

OscarRyz

196,001
113
385
569

score 3 · Answer 4 · answered Dec 30 '09 at 18:09

There are several things wrong with that code. It may also mutilate any NON-ASCII text since it converts via the platform default encoding twice - and for no good reason at all.

Don't use a Reader to read the file, use a FileInputStream and transfer bytes, avoiding the unnecessary and potentially destructive charset conversions. The line break problem will also be gone.

score 1 · Answer 5 · answered Dec 30 '09 at 18:05

1

BufferedReader.ReadLine() does not preserve the newline. Thus you'll have to add it when writing it out

answered Dec 30 '09 at 18:05

nos

223,662
58
417
506

score 0 · Answer 6 · answered Dec 30 '09 at 18:06

You can use a PrintWriter which offers a prinln() method. This will also save you from converting the string into an array of chars.

public static void writeFile(OutputStream o, File file) throws IOException {
   PrintWriter out = new PrintWriter(new OutputStreamWriter(o));
   BufferedReader input = new BufferedReader(new FileReader(file)); //File input stream 
   String line;
   while ((line = input.readLine()) != null) { //Read file
       out.println(line);  //Write to output stream 
       out.flush();
   }
   input.close();
}

He's converting it to an array of *bytes* (after first converting it from bytes to String) - and the PrintWriter will do the same thing. — Michael Borgwardt, Dec 30 '09 at 18:11

Java file not written to stream with new line characters

6 Answers6

Linked