Exec : display stdout "live"

Question

I have this simple script :

var exec = require('child_process').exec;

exec('coffee -cw my_file.coffee', function(error, stdout, stderr) {
    console.log(stdout);
});

where I simply execute a command to compile a coffee-script file. But stdout never get displayed in the console, because the command never ends (because of the -w option of coffee). If I execute the command directly from the console I get message like this :

18:05:59 - compiled my_file.coffee

My question is : is it possible to display these messages with the node.js exec ? If yes how ? !

Answer 1

Don't use exec. Use spawn which is an EventEmmiter object. Then you can listen to stdout/stderr events (spawn.stdout.on('data',callback..)) as they happen.

From NodeJS documentation:

var spawn = require('child_process').spawn,
    ls    = spawn('ls', ['-lh', '/usr']);

ls.stdout.on('data', function (data) {
  console.log('stdout: ' + data.toString());
});

ls.stderr.on('data', function (data) {
  console.log('stderr: ' + data.toString());
});

ls.on('exit', function (code) {
  console.log('child process exited with code ' + code.toString());
});

exec buffers the output and usually returns it when the command has finished executing.

Answer 2

exec will also return a ChildProcess object that is an EventEmitter.

var exec = require('child_process').exec;
var coffeeProcess = exec('coffee -cw my_file.coffee');

coffeeProcess.stdout.on('data', function(data) {
    console.log(data); 
});

OR pipe the child process's stdout to the main stdout.

coffeeProcess.stdout.pipe(process.stdout);

OR inherit stdio using spawn

spawn('coffee -cw my_file.coffee', { stdio: 'inherit' });

Answer 3

There are already several answers however none of them mention the best (and easiest) way to do this, which is using spawn and the { stdio: 'inherit' } option. It seems to produce the most accurate output, for example when displaying the progress information from a git clone.

Simply do this:

var spawn = require('child_process').spawn;

spawn('coffee', ['-cw', 'my_file.coffee'], { stdio: 'inherit' });

Credit to @MorganTouvereyQuilling for pointing this out in this comment.

Answer 4

Inspired by Nathanael Smith's answer and Eric Freese's comment, it could be as simple as:

var exec = require('child_process').exec;
exec('coffee -cw my_file.coffee').stdout.pipe(process.stdout);

Answer 5

I'd just like to add that one small issue with outputting the buffer strings from a spawned process with console.log() is that it adds newlines, which can spread your spawned process output over additional lines. If you output stdout or stderr with process.stdout.write() instead of console.log(), then you'll get the console output from the spawned process 'as is'.

I saw that solution here: Node.js: printing to console without a trailing newline?

Hope that helps someone using the solution above (which is a great one for live output, even if it is from the documentation).

Answer 6

I have found it helpful to add a custom exec script to my utilities that do this.

utilities.js

const { exec } = require('child_process')

module.exports.exec = (command) => {
  const process = exec(command)
  
  process.stdout.on('data', (data) => {
    console.log('stdout: ' + data.toString())
  })
  
  process.stderr.on('data', (data) => {
    console.log('stderr: ' + data.toString())
  })
  
  process.on('exit', (code) => {
    console.log('child process exited with code ' + code.toString())
  })
}

app.js

const { exec } = require('./utilities.js')
    
exec('coffee -cw my_file.coffee')

Answer 7

After reviewing all the other answers, I ended up with this:

function oldSchoolMakeBuild(cb) {
    var makeProcess = exec('make -C ./oldSchoolMakeBuild',
         function (error, stdout, stderr) {
             stderr && console.error(stderr);
             cb(error);
        });
    makeProcess.stdout.on('data', function(data) {
        process.stdout.write('oldSchoolMakeBuild: '+ data);
    });
}

Sometimes data will be multiple lines, so the oldSchoolMakeBuild header will appear once for multiple lines. But this didn't bother me enough to change it.

Answer 8

child_process.spawn returns an object with stdout and stderr streams. You can tap on the stdout stream to read data that the child process sends back to Node. stdout being a stream has the "data", "end", and other events that streams have. spawn is best used to when you want the child process to return a large amount of data to Node - image processing, reading binary data etc.

so you can solve your problem using child_process.spawn as used below.

var spawn = require('child_process').spawn,
ls = spawn('coffee -cw my_file.coffee');

ls.stdout.on('data', function (data) {
  console.log('stdout: ' + data.toString());
});

ls.stderr.on('data', function (data) {
  console.log('stderr: ' + data.toString());
});

ls.on('exit', function (code) {
  console.log('code ' + code.toString());
});

Answer 9

Here is an async helper function written in typescript that seems to do the trick for me. I guess this will not work for long-lived processes but still might be handy for someone?

import * as child_process from "child_process";

private async spawn(command: string, args: string[]): Promise<{code: number | null, result: string}> {
    return new Promise((resolve, reject) => {
        const spawn = child_process.spawn(command, args)
        let result: string
        spawn.stdout.on('data', (data: any) => {
            if (result) {
                reject(Error('Helper function does not work for long lived proccess'))
            }
            result = data.toString()
        })
        spawn.stderr.on('data', (error: any) => {
            reject(Error(error.toString()))
        })
        spawn.on('exit', code => {
            resolve({code, result})
        })
    })
}

Answer 10

I found another way to do it:

const { spawn } = require('child_process');

export const execCommand = async (command) => {
  return new Promise((resolve, reject) => {
    const [cmd, ...args] = command.split(' ');
    const childProcess = spawn(cmd, args);
    childProcess.stdout.on('data', (data) => {
      process.stdout.write(data.toString());
    });
    childProcess.stderr.on('data', (data) => {
      process.stderr.write(data.toString());
    });
    childProcess.on('error', (error) => {
      reject(error);
    });
    childProcess.on('exit', (code) => {
      if (code === 0) {
        resolve();
      } else {
        reject(new Error(`Command exited with code ${code}.`));
      }
    });
  });
};

This code gives and ability to get real-time output from executed command and redirect all stdout and stderr to parent process. It also allows using the command the same way you use it in bash/sh (single string input). Here I use process.stdout.write for more accurate output instead of console.log that is used in other answers.

Usage:

await execCommand('sudo apt-get update');
await execCommand('sudo apt-get install -y docker.io docker-compose');

Note: compared to exec it does not support execution of multiple commands using &&. So each single command should be executed with a single execCommand statement.

And here is a simplified version that supports both realtime streaming and shell execution:

const { spawn } = require('child_process');

export const execCommand = async (command) => {
  return new Promise((resolve, reject) => {
    const childProcess = spawn(command, { 
      stdio: 'inherit',
      shell: true
    });
    childProcess.on('error', (error) => {
      reject(error);
    });
    childProcess.on('exit', (code) => {
      if (code === 0) {
        resolve();
      } else {
        reject(new Error(`Command exited with code ${code}.`));
      }
    });
  });
};

Usage:

await execCommand('sudo apt-get update && sudo apt-get install -y docker.io docker-compose');