Write csv data to directly to azure blob in node js

Question

I want to create a CSV file using csv-writer , and upload that csv to azure blob, I am able to create the csv, store it on local system, and then read from local and upload to blob using azure-storage npm. But I dont want to create/store the CSV on local filesystem (because of some issues that I am running into on Prod), is there any way to create the CSV and directly feed to azure blob storage, without writing the csv to local file system.

Some code for reference

const csvWriter = createCsvWriter({
        path: `__dirname/${blobName}`,
        header: [
          { id: "id", title: "name" },
        ],
      });

      await csvWriter
        .writeRecords(csvData)
        .then(() => console.log("file successfully written")); 

And once this csv is created on local, read it from there using fs module, and upload to blob using "blobService.createBlockBlobFromStream" function.

Can you please suggest how can I directly give path of azure blob storage to csvWriter? or is there any other way to achieve this?

Answer 1

Please try the code below.

const {BlobServiceClient, StorageSharedKeyCredential} = require('@azure/storage-blob');
const createCsvStringifier = require('csv-writer').createObjectCsvStringifier;
const accountName = 'account-name';
const accountKey = 'account-key';
const container = 'container-name';
const blobName = 'text.csv';

const csvStringifier = createCsvStringifier({
    header: [
        {id: 'name', title: 'NAME'},
        {id: 'lang', title: 'LANGUAGE'}
    ]
});
const records = [
    {name: 'Bob',  lang: 'French, English'},
    {name: 'Mary', lang: 'English'}
];
const headers = csvStringifier.getHeaderString();
const data = csvStringifier.stringifyRecords(records);
const blobData = `${headers}${data}`;
const credentials = new StorageSharedKeyCredential(accountName, accountKey);
const blobServiceClient = new BlobServiceClient(`https://${accountName}.blob.core.windows.net`, credentials);
const containerClient = blobServiceClient.getContainerClient(container);
const blockBlobClient = containerClient.getBlockBlobClient(blobName);
const options = {
    blobHTTPHeaders: {
        blobContentType: 'text/csv'
    }
};
blockBlobClient.uploadData(Buffer.from(blobData), options)
.then((result) => {
    console.log('blob uploaded successfully!');
    console.log(result);
})
.catch((error) => {
    console.log('failed to upload blob');
    console.log(error);
});

Two things essentially in this code:

1. Use createObjectCsvStringifier if you don't want to write the data to disk.

2. Use @azure/storage-blob node package instead of azure-storage package as former is the newer one and the latter is being deprecated.

---

Update

Here's the code using azure-storage package.

const azure = require('azure-storage');
const createCsvStringifier = require('csv-writer').createObjectCsvStringifier;
const accountName = 'account-name';
const accountKey = 'account-key';
const container = 'container-name';
const blobName = 'text.csv';

const csvStringifier = createCsvStringifier({
    header: [
        {id: 'name', title: 'NAME'},
        {id: 'lang', title: 'LANGUAGE'}
    ]
});
const records = [
    {name: 'Bob',  lang: 'French, English'},
    {name: 'Mary', lang: 'English'}
];
const headers = csvStringifier.getHeaderString();
const data = csvStringifier.stringifyRecords(records);
const blobData = `${headers}${data}`;

const blobService = azure.createBlobService(accountName, accountKey);
const options = {
    contentSettings: {
        contentType: 'text/csv'
    }
}
blobService.createBlockBlobFromText(container, blobName, blobData, options, (error, response, result) => {
    if (error) {
        console.log('failed to upload blob');
        console.log(error);
    } else {
        console.log('blob uploaded successfully!');
        console.log(result);
    }
});