#StackBounty: #javascript #node.js How to get stdout and stderr unbuffered or at least in order from child_process

Bounty: 250

Here’s a simple node program

const log = msg => console.log(`log: ${msg}`);
const error = msg => console.error(`error: ${msg}`);

log('line1');
error('line2');
log('line3');
error('line4');

If I run it I get this output

log: line1
error: line2
log: line3
error: line4

Now I want the same output from child_process.execFile. The problem is the lines are buffered so they don’t arrive to me in the same order

const child_process = require('child_process');
const lines = [];
const ch = child_process.execFile(process.argv[0], ['main.js'], () => {
  console.log(lines.join(''));
});
ch.stdout.on('data', (data) => { lines.push(data); });
ch.stderr.on('data', (data) => { lines.push(data); });

produces

log: line1
error: line2
error: line4
log: line3

How can I get the lines in the same order and also know which are from stdout and which are from stderr?

note, the docs says some conflicting things about buffering. On the one hand they say

By default, pipes for stdin, stdout, and stderr are established between the parent Node.js process and the spawned child. These pipes have limited (and platform-specific) capacity. If the child process writes to stdout in excess of that limit without the output being captured, the child process will block waiting for the pipe buffer to accept more data

But later they mention a setting, maxBuffer which says

maxBuffer Largest amount of data in bytes allowed on stdout or stderr. If exceeded, the child process is terminated and any output is truncated

Those 2 explanations seem in conflict. In any case though it’s not clear how to unbuffer these streams.

spawn itself has an example in the docs but trying it

const child_process = require('child_process');
const lines = [];
const ch = child_process.spawn(process.argv[0], ['main.js']);

ch.stdout.on('data', (data) => {
  console.log(`stdout: ${data}`);
});

ch.stderr.on('data', (data) => {
  console.error(`stderr: ${data}`);
});

ch.on('close', (code) => {
  console.log(`child process exited with code ${code}`);
});

leads to the same results

stdout: log: line1

stderr: error: line2
error: line4

stdout: log: line3

child process exited with code 0

I suspect it has to do with making a Writeable stream and passing it in with spawn but it’s not clear how to do that. I tried creating a Writeable and passing it in but node complained

internal/child_process.js:996
      throw new ERR_INVALID_OPT_VALUE('stdio', inspect(stdio));
      ^

TypeError [ERR_INVALID_OPT_VALUE]: The value "Writable {
  _writableState: WritableState {
    objectMode: false,
    highWaterMark: 0,
    finalCalled: false,
    needDrain: false,
    ending: false,
    ended: false,
    finished: false,
    destroyed: false,
    decodeStrings: true,
    defaultEncoding: 'utf8',
    length: 0,
    writing: false,
    corked: 0,
    sync: true,
    bufferProcessing: false,
    onwrite: [Function: bound onwrite],
    writecb: null,
    writelen: 0,
    afterWriteTickInfo: null,
    bufferedRequest: null,
    lastBufferedRequest: null,
    pendingcb: 0,
    prefinished: false,
    errorEmitted: false,
    emitClose: true,
    autoDestroy: false,
    bufferedRequestCount: 0,
    corkedRequestsFree: {
      next: null,
      entry: null,
      finish: [Function: bound onCorkedFinish]
    }
  },
  writable: true,
  _write: [Function: write],
  _events: [Object: null prototype] {},
  _eventsCount: 0,
  _maxListeners: undefined,
  [Symbol(kCapture)]: false
}" is invalid for option "stdio"

Any idea how to make a synchronous or at least correctly interleaved capture of of stdout and stderr when spawning a child process in node?


Get this bounty!!!

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.