Write objects into file with Node.js

Question

I've searched all over stackoverflow / google for this, but can't seem to figure it out.

I'm scraping social media links of a given URL page, and the function returns an object with a list of URLs.

When I try to write this data into a different file, it outputs to the file as [object Object] instead of the expected: [ 'https://twitter.com/#!/101Cookbooks', 'http://www.facebook.com/101cookbooks'] as it does when I console.log() the results.

This is my sad attempt to read and write a file in Node, trying to read each line(the url) and input through a function call request(line, gotHTML):

fs.readFileSync('./urls.txt').toString().split('\n').forEach(function (line){
    console.log(line); 
    var obj = request(line, gotHTML); 
    console.log(obj); 
    fs.writeFileSync('./data.json', obj , 'utf-8'); 
});

for reference -- the gotHTML function:

function gotHTML(err, resp, html){ 
    var social_ids = []; 

    if(err){
        return console.log(err); 
    } else if (resp.statusCode === 200){ 
        var parsedHTML = $.load(html); 

        parsedHTML('a').map(function(i, link){
            var href = $(link).attr('href');
            for(var i=0; i<socialurls.length; i++){
                if(socialurls[i].test(href) && social_ids.indexOf(href) < 0 ) {
                    social_ids.push(href); 
                }; 
            }; 
        })
    };

    return social_ids;
};

`[object Object]` is an object `toString`. If you want the representation of the object, use `JSON.stringify`. — elclanrs, Feb 24 '14 at 00:12
Be careful with `JSON.stringify`. With arrays, you're safe, but when objects have circular references it will fail ([see this topic](http://stackoverflow.com/a/11616993/151445)). The `util` module handles circular references. — Joseph Yaduvanshi, Feb 24 '14 at 15:02

score 126 · Answer 1 · edited Sep 09 '19 at 18:11

126

Building on what deb2fast said I would also pass in a couple of extra parameters to JSON.stringify() to get it to pretty format:

fs.writeFileSync('./data.json', JSON.stringify(obj, null, 2) , 'utf-8');

The second param is an optional replacer function which you don't need in this case so null works.

The third param is the number of spaces to use for indentation. 2 and 4 seem to be popular choices.

edited Sep 09 '19 at 18:11

Florian Neumann

5,587
1
39
48

answered Jul 10 '15 at 21:06

Guy

65,082
97
254
325

1

it is going to cause circular dependency problem. two-way dependencies could not be converted to JSON. – Ozan Ertürk Dec 12 '17 at 19:32
The written text file is missing the newline character at the end of the file. Any way to fix this? – friederbluemle Sep 13 '19 at 21:27
2

@friederbluemle - You could `+ '\n'` to that stringify() call to add that newline. – Guy Oct 31 '19 at 21:42
1

This one worked best for me cause using the accepted answer gave me the wrong result. – Anshuman Kumar Dec 15 '20 at 04:51

score 85 · Accepted Answer · answered Feb 24 '14 at 00:23

obj is an array in your example.

fs.writeFileSync(filename, data, [options]) requires either String or Buffer in the data parameter. see docs.

Try to write the array in a string format:

// writes 'https://twitter.com/#!/101Cookbooks', 'http://www.facebook.com/101cookbooks'
fs.writeFileSync('./data.json', obj.join(',') , 'utf-8');

Or:

// writes ['https://twitter.com/#!/101Cookbooks', 'http://www.facebook.com/101cookbooks']
var util = require('util');
fs.writeFileSync('./data.json', util.inspect(obj) , 'utf-8');

edit: The reason you see the array in your example is because node's implementation of console.log doesn't just call toString, it calls util.format see console.js source

If your object contains large arrays, this will not work as the inspect method will output something like "45000 more...]" — Coxer, Feb 16 '17 at 16:26

score 23 · Answer 3 · edited Oct 02 '14 at 14:38

23

If you're geting [object object] then use JSON.stringify

fs.writeFile('./data.json', JSON.stringify(obj) , 'utf-8');

It worked for me.

edited Oct 02 '14 at 14:38

Fatih Acet

28,690
9
51
58

answered Sep 04 '14 at 04:19

deb2fast

990
11
18

score 12 · Answer 4 · answered Aug 12 '16 at 17:19

In my experience JSON.stringify is slightly faster than util.inspect. I had to save the result object of a DB2 query as a json file, The query returned an object of 92k rows, the conversion took very long to complete with util.inspect, so I did the following test by writing the same 1000 record object to a file with both methods.

JSON.Stringify

fs.writeFile('./data.json', JSON.stringify(obj, null, 2));

Time: 3:57 (3 min 57 sec)

Result's format:

[
  {
    "PROB": "00001",
    "BO": "AXZ",
    "CNTRY": "649"
   },
  ...
]

util.inspect

var util = require('util');
fs.writeFile('./data.json', util.inspect(obj, false, 2, false));

Time: 4:12 (4 min 12 sec)

Result's format:

[ { PROB: '00001',
    BO: 'AXZ',
    CNTRY: '649' },
    ...
]

score 4 · Answer 5 · edited Aug 19 '22 at 00:44

4

Could you try doing JSON.stringify(obj);

Like this:

var stringify = JSON.stringify(obj);
fs.writeFileSync('./data.json', stringify, 'utf-8');

edited Aug 19 '22 at 00:44

bdkopen

494
1
6
16

answered Sep 28 '18 at 14:37

Navvye Anand

41
2

score 2 · Answer 6 · answered Aug 22 '19 at 06:42

2

Just incase anyone else stumbles across this, I use the fs-extra library in node and write javascript objects to a file like this:

const fse = require('fs-extra');
fse.outputJsonSync('path/to/output/file.json', objectToWriteToFile);

answered Aug 22 '19 at 06:42

tljw86

31
4

score 0 · Answer 7 · answered May 31 '21 at 09:19

Further to @Jim Schubert's and @deb2fast's answers:

To be able to write out large objects of order which are than ~100 MB, you'll need to use for...of as shown below and match to your requirements.

const fsPromises = require('fs').promises;

const sampleData = {firstName:"John", lastName:"Doe", age:50, eyeColor:"blue"};

const writeToFile = async () => {
  for (const dataObject of Object.keys(sampleData)) {
      console.log(sampleData[dataObject]);
      await fsPromises.appendFile( "out.json" , dataObject +": "+ JSON.stringify(sampleData[dataObject]));
  }
}

writeToFile();

Refer https://stackoverflow.com/a/67699911/3152654 for full reference for node.js limits

Write objects into file with Node.js

7 Answers7

Linked

Related