1

I'm currently using zlib node library (node version 14.19.1) using fs.createReadStream and then fs.createWriteStream. However I need a text file (all the .gz files contain exactly a single .csv text file) to continue working further downstream.

Here's the code I have so far (I'm using MeteorJS, hence the import statement instead of require):

import ZLib from 'zlib';
import fs from 'fs';
import stream from 'stream';

const gunzip = async function (fileInfo) {
    // unzip .gz file
    const source = fs.createReadStream(fileInfo.filePath);
    const destination = fs.createWriteStream(fileInfo.filePath.replace(".gz", ""));

    stream.pipeline(source, ZLib.createGunzip(), destination, (err) => {
        if (err) {
            console.error('An error occurred:', err);
            process.exitCode = 1;
        }
    });
};

I've tried several npm libraries that wrap around zlib (eg. 'node-gzip') however I get problems with them as they don't allow to set the optional windowBits value to 16 or beyond and hence I get an invalid header error - as found in many other SO posts).

Thanks in advance for your help!

a4xrbj1
  • 445
  • 3
  • 21
  • So... what is the problem exactly? – Thomas Nov 16 '22 at 08:52
  • I don't know how I can get a text file out of a compressed .gz file. In the most effective way, avoiding any unnecessary reading/writing to file and rather handle it in memory. – a4xrbj1 Nov 16 '22 at 10:14
  • But you are showing code that is supposed to do exactly that. What's wrong with that code? What unexpected behaviour are you seeing? – Thomas Nov 16 '22 at 10:31
  • Appreciate your help, Thomas. The code write a physical file to a random folder. I want to avoid that step and converting that file with eg ```fs.readFileSync(txtFilename, 'utf8').split(",");``` - is there no way that this read and write can be solved in memory only? – a4xrbj1 Nov 16 '22 at 12:59
  • If you want in memory only, why are you using "fs.createWriteStream"? Implement a custom writable stream that does exactly what you want. TBH: I still dont get 100% what you try to achieve. You need the CSV "parsed" only in memory? Post the csv file. – Marc Nov 16 '22 at 14:03
  • So you don't want a text _file_, you just want to unzip the contents into a string? See here: https://stackoverflow.com/questions/10623798/how-do-i-read-the-contents-of-a-node-js-stream-into-a-string-variable – Thomas Nov 16 '22 at 14:06
  • Marc, I'm using `fs.createWriteStream` as I'm currently re-reading that uncompressed file again to convert it into a text file as the final step. – a4xrbj1 Nov 17 '22 at 02:55

0 Answers0