0

Relatively new with Node. Trying to work with streams.

I am looking to read from and sort a large file containing numbers (newline separated). I have sorted my file as per the the method here (method #5, referenced in this thread actually), and am trying to tweak it to return the first X lines from the sorted data (var request = 5 in the code below just for testing). I would like to display these this after writing the data. 

var fs = require('fs'),
util = require('util'),
spawn = require('child_process').spawn,
sort = spawn('sort', ['-nr','../data/data.txt']),
writer = fs.createWriteStream('output.txt');

var start = Date.now();
var output = 'output.txt';

var request = 5;
var lineCount = 0;
var numbers;
var selection = [];

sort.stdout.on('data', function(data) {
    lineCount ++
    numbers = data.toString()
    // data.toString()
        .split('\n')
        .sort((a, b) => b-a)

    selection = numbers.slice(0, request)
        // 
        // .forEach(record => writer.write(record + '\n'))
    },
)

sort.on('exit', function(code) {
    if (code) {
        // handle error
        console.log(code);
    }
    // writer.end()    
    // console.log(lineCount);
    let array = numbers.slice(0, request);
    console.log(selection);
    var closetime = Date.now();
    console.log('Time. ', (closetime -start) /1000, ' secs');
});

sort.stderr.on('data', function(data) {
    // handle error...
    console.log('stderr: ' + data);
});

I can't get it to work regardless of whether I put my code in the .on('data' section, or in the .on('exit' section. Any suggestions?

I did reference a large number of threads here on Stack Overflow, including this one, this one, this one on reading a specific line via nodejs, reading the first two lines via Nodejs.

This last one sounds like it should be what i need, but I can't get it to work within my code.

Mugshep
  • 788
  • 1
  • 9
  • 18
  • Do you need only the 5 rows in the output file, or do you need everything and just show the sorted ones? – Michał Karpacki Feb 27 '19 at 14:20
  • this was just a sample request, using a placeholder (ie: `request`) for a number of lines to return just to confirm it was sorting. It could be any number. – Mugshep Feb 28 '19 at 12:28
  • I see... Well `sort` will definitely sort your data, but it won't be split by lines so you'd need to pass it through `readline` module or use a framework - I'll write the answer for you, but first let me know if do need the data to show up **before** you start writing (and why so?). If you just want to show the first lines, while the file is written, it'll be much simpler. – Michał Karpacki Mar 01 '19 at 11:53
  • @MichałKapracki - very delayed on this, but I would still be interested in seeing how you do this. Does not need to display the data before. – Mugshep Sep 04 '19 at 20:32

0 Answers0