How to chain write stream, immediately with a read stream in Node.js 0.10?

Question

The following line will download an image file from a specified url variable:

var filename = path.join(__dirname, url.replace(/^.*[\\\/]/, ''));
request(url).pipe(fs.createWriteStream(filename));

And these lines will take that image and save to MongoDB GridFS:

 var gfs = Grid(mongoose.connection.db, mongoose.mongo);
 var writestream = gfs.createWriteStream({ filename: filename });
 fs.createReadStream(filename).pipe(writestream);

Chaining pipe like this throws Error: 500 Cannot Pipe. Not Pipeable.

request(url).pipe(fs.createWriteStream(filename)).pipe(writestream);

This happens because the image file is not ready to be read yet, right? What should I do to get around this problem?Error: 500 Cannot Pipe. Not Pipeable.

Using the following: Node.js 0.10.10, mongoose, request and gridfs-stream libraries.

score 14 · Accepted Answer · answered Jun 14 '13 at 03:56

14

request(url).pipe(fs.createWriteStream(filename)).pipe(writestream);

is the same as this:

var fileStream = fs.createWriteStream(filename);
request(url).pipe(fileStream);
fileStream.pipe(writestream);

So the issue is that you are attempting to .pipe one WriteStream into another WriteStream.

answered Jun 14 '13 at 03:56

loganfsmyth

156,129
30
331
251

2

Yes, you can only read from readable or duplex streams not writable. See https://github.com/joyent/node/pull/4843. To chain pipes the stream needs to be duplex i.e. both readable and writable – user568109 Jun 14 '13 at 18:59

score 9 · Answer 2 · edited Jul 19 '16 at 12:02

9

// create 'fs' module variable
var fs = require("fs");

// open the streams
var readerStream = fs.createReadStream('inputfile.txt');
var writerStream = fs.createWriteStream('outputfile.txt');

// pipe the read and write operations
// read input file and write data to output file
readerStream.pipe(writerStream);

edited Jul 19 '16 at 12:02

YakovL

7,557
12
62
102

answered Jul 19 '16 at 10:52

BIJENDER KHATANA

107
1
1

1

It was actually the opposite: pipe a readable stream to a writeable stream. – loretoparisi Sep 19 '17 at 13:43

mechanicious · Answer 3 · 2016-07-31T23:18:54.323

4

I think the confusion in chaining the pipes is caused by the fact that the pipe method implicitly "makes choices" on it's own on what to return. That is:

readableStream.pipe(writableStream) // Returns writable stream
readableStream.pipe(duplexStream) // Returns readable stream

But the general rule says that "You can only pipe a Writable Stream to a Readable Stream." In other words only Readable Streams have the pipe() method.

edited Jul 31 '16 at 23:18

answered Mar 30 '14 at 14:59

mechanicious

1,576
2
15
32

1

Then whats going on here? In `npm axios` documentation it says; `response.data.pipe(fs.createWriteStream('path-to-file))` As we know that request is readable stream and response is writable stream, and you said that only Readable streams have `pipe()` method but here we are pipping writable stream to new writestream directly using pipe no read stream involve. This may suggest that pipe method is available to writable streams also. – Ammar Bayg May 18 '18 at 13:17
@AmmarBayg I have written this post 2 years ago relating to npm version 0.10. Have you checked it on that version also? – mechanicious May 18 '18 at 16:52

score 0 · Answer 4 · answered Nov 18 '21 at 15:21

You cannot chain the ReadStream to the WriteStream because the latter is not duplex, therefore you would do - for a gzipped archive

request.get(url, {
        gzip: true,
        encoding: null
    })
    .pipe(fs.createWriteStream(tmpPath))
    .on('close', function() {
        console.info("downloaded %s", tmpPath);
        fs.createReadStream(tmpPath)
            .pipe(gunzip)
            .pipe(fs.createWriteStream(destPath))
            .on('close', function() {
                console.info("unarchived %s", destPath);
            })
            .on('error', (error) => {
                console.warn("gzip error:%@", error);
            })
    })
    .on('error', (error) => {
        console.warn("download error:%@", error);
    })

How to chain write stream, immediately with a read stream in Node.js 0.10?

4 Answers4

Linked