callback to handle completion of pipe

Question

I am using the following node.js code to download documents from some url and save it in the disk. I want to be informed about when the document is downloaded. i have not seen any callback with pipe.Or, Is there any 'end' event that can be captured on completion of download ?

request(some_url_doc).pipe(fs.createWriteStream('xyz.doc'));

score 331 · Accepted Answer · edited Dec 15 '15 at 00:39

331

Streams are EventEmitters so you can listen to certain events. As you said there is a finish event for request (previously end).

 var stream = request(...).pipe(...);
 stream.on('finish', function () { ... });

For more information about which events are available you can check the stream documentation page.

edited Dec 15 '15 at 00:39

steampowered

11,809
12
78
98

answered Jul 12 '12 at 08:59

Pickels

33,902
26
118
178

17

`var r = request(...).on("end",function(){/* CALLBACK */}).pipe(...);` – Denys Vitali Aug 31 '13 at 21:43
7

for me the event 'close', rather than 'end' works `r.on('close'), function () {...})` – Judson Dec 16 '13 at 20:20
10

The 'end' event is now 'finish' Pipe events: http://nodejs.org/api/stream.html#stream_event_finish – Pier-Luc Gendreau Apr 16 '14 at 14:44
15

The 'end' event still exists and is used on Readable streams. Writable streams use 'finish'. This is because the Transform stream is a mix of both and needs to differentiate between the events. – noderman Jan 29 '16 at 18:31
1

I've tried this but only the `'complete'` event worked. – GGG May 22 '17 at 04:28
25

This thread is a good summary of how working with node streams looks like. – pomber Aug 10 '17 at 19:53
3

@Pickels this answer looks cool, but have a Q: the "var stream" is the source stream or dest stream, the one you request from or pipe into ? – user3552178 Mar 25 '18 at 01:25
Does `finish` fire in case of `error`, or should I add a separate callback to handle interruptions? – Klesun Nov 12 '20 at 15:51

score 45 · Answer 2 · edited Oct 13 '14 at 08:50

45

Based nodejs document, http://nodejs.org/api/stream.html#stream_event_finish, it should handle writableStream's finish event.

var writable = getWriteable();
var readable = getReadable();
readable.pipe(writable);
writable.on('finish', function(){ ... });

edited Oct 13 '14 at 08:50

answered Oct 13 '14 at 08:47

vr3C

1,734
19
16

The accepted answer doesn't work for me with node v0.12.7, but your solution does. – Luca Steeb Aug 30 '15 at 14:13
Doesn't do anything for me: https://stackoverflow.com/questions/76262508/node-js-writable-finish-not-emitted-after-pushing-null-into-pipe – sezanzeb May 16 '23 at 12:20

score 9 · Answer 3 · answered Sep 11 '15 at 15:45

9

Code snippet for piping content from web via http(s) to filesystem. As @starbeamrainbowlabs noticed event finish does job

var tmpFile = "/tmp/somefilename.doc";

var ws = fs.createWriteStream(tmpFile);
ws.on('finish', function() {
  // pipe done here, do something with file
});

var client = url.slice(0, 5) === 'https' ? https : http;
client.get(url, function(response) {
  return response.pipe(ws);
});

answered Sep 11 '15 at 15:45

ruX

7,224
3
39
33

The on 'finish' works for me and 'end' does not. Thank you! – shaosh Sep 20 '15 at 05:04
1

in one particular case, finish seems to be firing before all the expected bytes arrive – Michael Oct 06 '16 at 06:02

Abu Shumon · Answer 4 · 2019-09-26T13:04:08.907

I found an a bit different solution of my problem regarding this context. Thought worth sharing.

Most of the example create readStreams from file. But in my case readStream has to be created from JSON string coming from a message pool.

var jsonStream = through2.obj(function(chunk, encoding, callback) {
                    this.push(JSON.stringify(chunk, null, 4) + '\n');
                    callback();
                });
// message.value --> value/text to write in write.txt 
jsonStream.write(JSON.parse(message.value));
var writeStream = sftp.createWriteStream("/path/to/write/write.txt");

//"close" event didn't work for me!
writeStream.on( 'close', function () {
    console.log( "- done!" );
    sftp.end();
    }
);

//"finish" event didn't work for me either!
writeStream.on( 'close', function () {
    console.log( "- done!"
        sftp.end();
        }
);

// finally this worked for me!
jsonStream.on('data', function(data) {
    var toString = Object.prototype.toString.call(data);
    console.log('type of data:', toString);
    console.log( "- file transferred" );
});

jsonStream.pipe( writeStream );

Instead of listening for "finish" you have two handlers for "close", maybe that was the reason. The "finish" event works for me. — jowo, May 24 '17 at 15:37

score 4 · Answer 5 · answered Jan 18 '18 at 21:41

Here's a solution that handles errors in requests and calls a callback after the file is written:

request(opts)
    .on('error', function(err){ return callback(err)})
    .pipe(fs.createWriteStream(filename))
    .on('finish', function (err) {
        return callback(err);
    });

callback to handle completion of pipe

5 Answers5

Linked

Related