Observables in nestjs - Reading a file asynchronously

Question

I am trying a use case of reading a json file asynchronously and sending it out as a response (as a rxjs observable data). Here is the service that I use

 import { logger } from './../../shared/utils/logger';
import { Injectable } from '@nestjs/common';
import * as fs from 'fs';
import * as path from 'path';
import { BehaviorSubject, Observable, pipe, of, from, throwError, merge} from 'rxjs';
import { map, filter, scan, take, debounce, switchMap, retry, catchError, mergeMap, delay, zip, tap, mapTo } from 'rxjs/operators';
import { HttpResponseModel } from '../model/config.model';
import { isNullOrUndefined } from 'util';
@Injectable()
export class NewProviderService {
    serviceSubject: BehaviorSubject<HttpResponseModel[]>;
    filePath: string;
    httpResponseObjectArray: HttpResponseModel[];
    constructor() {
        this.serviceSubject = new BehaviorSubject<HttpResponseModel[]>([]);
        this.filePath = path.resolve(__dirname, './../../shared/assets/httpTest.json');
        this.setSubject();
    }


 readFileFromJSON() {
      this.readFileFromJsonSync();
      fs.exists(this.filePath.toString(), exists => {
        if (exists) {
           fs.readFile(this.filePath.toString(), 'utf-8', (err, data) => {
                logger.info('file read without parsin', data);
                this.httpResponseObjectArray = JSON.parse(data).HttpTestResponse;
                logger.info('array obj is:', this.httpResponseObjectArray);
                logger.info('file read after parsing', JSON.parse(data));
                return this.httpResponseObjectArray;
            });
        } else {
            return null;
        }

    });
}


getObservable(): Observable<HttpResponseModel[]> {
       // create an observable
        // return Observable.create(observer => {
        //     observer.next(this.readFileFromJSON());
        // });

        return of(this.readFileFromJsonSync()).pipe(map(data => {
            logger.info('inside obs methid', data);
            return data;
        }));

    }

    setSubject() {
        this.getObservable().subscribe(data => {
            logger.info('data before setting in sub', data);
            this.serviceSubject.next(data);
        });
    }
}

So I wanted to subscribe to this emitted observable in the controller, but the values are getting read after I have subscribed and read the subject (BehaviorSubject). I understand that I am kind of doing something wrong with the subscription and emitting of data, but couldn't understand where I am doing wrong. Every time the controller prints 'data subscribed undefined' and then continues to read the file and emit the observable

This is the controller data

@Get('/getJsonData')
  public async getJsonData(@Req() requestAnimationFrame, @Res() res) {
    this.newService.serviceSubject.subscribe(data => {
      logger.info('data subscribed', data);
      res.status(HttpStatus.OK).send(data);
    });

  }

It works well if I read the file synchronously

replace readFileFromJSON() with the following method and it works well

readFileFromJsonSync(): HttpResponseModel[] {
        const objRead = JSON.parse(fs.readFileSync(this.filePath.toString(), {encoding: 'utf-8'}));
        logger.info('object read is', objRead.HttpTestResponse);
        return objRead.HttpTestResponse;

    }

So I am missing something while reading the file async. I am not sure what am I doing wrong. Could someone please help?

score 3 · Accepted Answer · answered Feb 19 '19 at 16:25

The problem is that you don't actually return anything in readFileFromJSON. It will asynchronously run fs.exists and fs.readFile and the corresponding callbacks but the result from the callbacks is ignored.

You should use Promises instead. You can either create a Promise yourself or use a library like bluebird that transforms fs from a callback based API to a Promise based API. For more information see this thread.

return new Promise(function(resolve, reject) {
    fs.readFile(this.filePath.toString(), 'utf-8', (err, data) => {
        if (err) {
            reject(err); 
        } else {
            const httpResponseObjectArray = JSON.parse(data).HttpTestResponse;
            resolve(httpResponseObjectArray);
        }
    });
});

Thank you. I created a promise API just like above and converted the observable method to read from the promise using "from" operator. It worked !!!. — vijayakumarpsg587, Feb 20 '19 at 11:39

Observables in nestjs - Reading a file asynchronously

1 Answers1

Linked