Get POST request headers after click() and wait() in Nightmare JS

Question

I'm crawling a page where I type() something in an input field, click() a button and then wait() for some element to appear.

I would like to get all the headers for the POST request that is fired after click().

I see that goto() returns the headers, but I don't think I can use it in my case because after click() I stay on the same page.

Here is an example of my code on another test page (I would like to get the POST request headers after wait()):

var Nightmare = require('nightmare');
var nightmare = Nightmare({ show: true });

nightmare
    .goto('https://jigsaw.w3.org/HTTP/300/')
    .click('tbody tr:nth-child(5) td form input[type=submit]')
    .wait('body p:nth-of-type(3)')
    .evaluate(function () {
        return document.querySelector('body p:nth-of-type(3)').innerText;
    })
    .end()
    .then(function (result) {
        console.log(result);
    })
    .catch(function (error) {
        console.error('Search failed:', error);
    });

Note: I use Nightmare 2.10.0 and Node.js 8.1.3.

Answer 1

There's no simple way of doing this that I'm aware of.

You can, however, get that information.

Via .on('did-get-response-details', callback)

If you call .on('did-get-response-details') its callback will have the following information:

• event Event
• status Boolean
• newURL String
• originalURL String
• httpResponseCode Integer
• requestMethod String
• referrer String
• headers Object
• resourceType String

You have to call .on(<event>) prior to calling .goto().

A list of available events is located here: https://electron.atom.io/docs/api/web-contents/#instance-events

Via .evaluate

You could go directly to the POST endpoint via ajax (instead of submitting the form via a form submission).

Ajax example stolen from here: Send POST data using XMLHttpRequest

nightmare.evaluate(() => {
    return new Promise((resolve, reject) => {
        var http = new XMLHttpRequest();
        var url = "get_data.php";
        var params = "lorem=ipsum&name=binny";
        http.open("POST", url, true);
        
        http.setRequestHeader("Content-type", "application/x-www-form-urlencoded");
        
        http.onreadystatechange = function() {
            if(http.readyState == 4 && http.status == 200) {
                resolve(http); // <- Get some data here
            }
        }
        http.send(params);
    });
});

How these two ideas would specifically address your use case would depend on some things.