How to pipe to a Node.js stream that's a buffer, not a file?

Question

I've been searching everywhere, and all examples lead to fs.createWriteStream which is not what I want.

I'm using the archiver package, and would like to use archive.pipe() to pipe to a writeable stream that is not a file, but rather a buffer that I can use to send to s3.putObject to write to an S3 bucket. How do I set up a buffer that I can pipe to?

When I run the code below, I get "Error: not implemented".

const stream = require('stream');
const archiver = require('archiver');

const archive = archiver('zip');
const outputStream = new stream.Writable();
outputStream.on('close', () => {
  console.log('done');
});
outputStream.on('error', err => {
  console.error(err);
});
archive.pipe(outputStream);
archive.append('Testing 1 2 3', { name: 'file1.txt' });
archive.finalize();

See also ["How do I read the contents of a Node.js stream into a string variable?"](https://stackoverflow.com/a/63361543/704244) — bartolo-otrit, Jan 18 '21 at 14:45

score 4 · Accepted Answer · answered Jul 21 '17 at 18:46

Got it!

const bl = require('bl');
const AWS = require('aws-sdk');
const archiver = require('archiver');

const s3 = new AWS.S3();
const archive = archiver('zip');
let buf;

archive.pipe(
  bl((err, data) => {
    buf = data;
  })
);
archive.file('whoa.wav', { name: 'file1.wav' });
archive.finalize().then(x => {
  console.log('finalized', buf);
  s3.putObject(
    {
      Bucket: 'xyz',
      Key: 'whatever.zip',
      Body: buf,
    },
    err => {
      console.log('s3 result:', err);
    }
  );
});

How to pipe to a Node.js stream that's a buffer, not a file?

1 Answers1