Post request to wait a function

Question

I know the question is asked really often, and I might get downvoted for that. But I really struggle to understand how I can wait for a function to process data before returning a value.

I had a look on many popular posts (such as here), but I cannot achieve what I want.

Here is my code:

   app.post("/video_url", function (req, res) {
       videoProcessed = videoScraper(req.body.videoURL.videoURL);
       res.send(videoProcessed);
   });

It does not wait for this function to process the data:

function videoScraper(url) {
console.log("URL to Scraper: " + url);
const options = {
    uri: `${url}`,
    transform: function(body) {
        return cheerio.load(body);
    }
};

var videoProcessed;

rp(options)
    .then(($) => {
        videoProcessed = $("body").find("iframe").attr("src");
        return videoProcessed;
    })
    .catch((err) => {
        console.log(err);
    });
}

I tried using callbacks but it gets really messy, and I don't know were to put the promise (if any) in my code.

@Jérémy You're already using promise with `rp`. It returns a promise (can be seen from the use of `.then` & `.catch`) — coyotte508, Sep 15 '18 at 16:03

score 3 · Accepted Answer · answered Sep 15 '18 at 16:02

Add await and async (if you have node 8+):

app.post("/video_url", async function (req, res) {
    const videoProcessed = await videoScraper(req.body.videoURL.videoURL);
    res.send(videoProcessed);
});

And in your videoScraper function, you need to return rp! :

function videoScraper(url) {
    console.log("URL to Scraper: " + url);
    const options = {
        uri: `${url}`,
        transform: function(body) {
            return cheerio.load(body);
        }
    };

    return rp(options)
    .then($ => $("body").find("iframe").attr("src"))
    .catch((err) => {
        console.error(err);
    });
}

That would depend on the videoScrapper working fine, I've no idea what rp is so I can't tell.

Don't forget to handle videoProcessed === undefined (error case) in the first code snippet. It can also be abstracted using express-promise-router that will even catch async errors... That's further down the road.

Don't hesitate to read up on await & async, it's really wonderful to write asynchronous code in the same manner as synchronous code.

wow ok, as simple as that! Thank you very much it works like a charm! — Jerlam, Sep 15 '18 at 16:08

score 1 · Answer 2 · answered Sep 15 '18 at 16:27

Use async/await

app.post("/video_url", async (req, res)=> {
try{
    let videoProcessed = await videoScraper(req.body.videoURL.videoURL);
    res.send(videoProcessed);
}
catch(ex){
// handle the exception
}
});

const videoScraper = async fuction(url) {
console.log("URL to Scraper: " + url);
let options = {
    uri: `${url}`,
    transform: function(body) {
        return cheerio.load(body);
    }
};
try{
let temp = await rp(options);
let videoProcessed = $("body").find("iframe").attr("src");// you can return it directly
return videoProcessed;
}
catch(ex){
// handle the exception
}
}

if you your node is < 8 then use promises (bluebird module)

const bluebird = require('bluebird');
function videoScraper(url){
return new bluebird(function(resolve,reject){
let options = {
    uri: `${url}`,
    transform: function(body) {
        return cheerio.load(body);
    }
};
rp(options)
.then(($)=>{
resolve($("body").find("iframe").attr("src"));
})
.catch(err=>{
return err;
})
})
}
app.post("/video_url", (req, res)=> {
videoScraper(req.body.videoURL.videoURL)
.then(result=>{
res.send(result)
})
.catch(err=>{
// handle the error
})
});

Do not use const for variable declaration unless its value is constant, and usually use let instead of var

score 1 · Answer 3 · answered Sep 15 '18 at 16:32

You can try the following:

app.post("/video_url", function (req, res) {
    videoScraper(req.body.videoURL.videoURL)
        .then(videoProcessed => {
            res.send(videoProcessed);
        })
        .catch(err => {
            // render proper response with error message
        })
});

And change the function to something as below, so as to return a promise from the same:

function videoScraper(url) {
    console.log("URL to Scraper: " + url);
    const options = {
        uri: `${url}`,
        transform: function(body) {
            return cheerio.load(body);
        }
    };

    return rp(options)
        .then(($) => {
            videoProcessed = $("body").find("iframe").attr("src");
            return videoProcessed;
        });
}

Post request to wait a function

3 Answers3