Missing requests from phantomjs network monitoring results

Question

I have a web application which is built with AngularJS. I am using phantomjs network monitoring for sniffing all requests triggered from the website on a page load. I get the following list of requests:

 "https:.../assets/application-bf61473a35661d960c262995b314b0af.css" 
 "https:.../assets/lib/modernizr-c569e351684c78953f5f6298b4e1e485.js" 
 "https:.../assets/application-04936fc61dbebda777c3f816afa39726.js" 
 "https://www.google-analytics.com/analytics.js" 
 "https://ssl.google-analytics.com/ga.js"  
 "https:.../assets/app_install_page_header-a4b182016c1769bad626d1409b6c95f1.png"
 "https:.../assets/app_install_page_landing_text-14a162dca43a9a84b9fe0a7a77472fad.png"

The problem is that the list doesn't include any dynamic requests such as:

request with data for google analytics;

"https://www.google-analytics.com/collect?v=1&vb=..."
request for pictures witch are requested from a backend;
...

I used a waitFor method in order to give phantomjs time to wait for the delayed requests but it didn't help.

I used this documentation http://phantomjs.org/network-monitoring.html.

Code:

var page = require('webpage').create();

page.onConsoleMessage = function(msg, lineNum, sourceId) {
    console.log('CONSOLE: ' + msg + ' (from line #' + lineNum + ' in "' + sourceId + '")');
};

page.onError = function(msg, trace) {
    var msgStack = ['ERROR: ' + msg + trace];
    console.error(msgStack.join('\n'));
};

page.onResourceRequested = function(request) {
    url = request.url
    console.log(url);
};

page.onRecourseReceived = function(response) {
    console.log('received: ' + JSON.stringify(response, undefined, 4));
};

page.onLoadFinished = function() {
    page.render("on_finish.png");
};

page.open(address, function(status){

    setTimeout(function(){
        phantom.exit();
    }, 15000);
});

`waitFor` is useful when you are waiting for something specific. What is it you're waiting for (please show your code). In your case a `setTimeout(function(){phantom.exit();}, 5000);` should be sufficient. — Artjom B., Oct 28 '14 at 13:29
Please register to the [`onConsoleMessage`](http://phantomjs.org/api/webpage/handler/on-console-message.html) and [`onError`](http://phantomjs.org/api/webpage/handler/on-error.html) events. Maybe there are errors. — Artjom B., Oct 28 '14 at 13:30
I couldn't catch any errors with the methods, however I checked in a browser console and there are some errors for third party libs but not for google-analytics for example. — Yuliya Brnzk, Oct 28 '14 at 14:54
Yes, it helped. Thanks a lot Artjom. Gould you please write an answer why this matters? — Yuliya Brnzk, Oct 28 '14 at 15:20
Yes, but first I try to find a site that uses google-analytics, but I can't find one. — Artjom B., Oct 28 '14 at 15:22
It's impossible to find a site that itself is http, but uses https analytics. It seems you have one. Do you see some errors when you register http://phantomjs.org/api/webpage/handler/on-resource-error.html without `--ssl-protocol=tlsv1`? I will start writing the answer. — Artjom B., Oct 28 '14 at 16:07
When I registered on resource error I have got the following errors. Running without the flag specified. Unable to load resource (#4URL:https://.../assets/application-04936fc61dbebda777c3f816afa39726.js) Error code: 6. Description: SSL handshake failed — Yuliya Brnzk, Oct 29 '14 at 10:03

score 3 · Accepted Answer · edited May 23 '17 at 12:17

It seems that you have an http site which itself uses https analytics.
Recently the POODLE vulnerability forced website owners to disable SSLv3. Since PhantomJS < v1.9.8 uses SSLv3 by default, the analytics and additional scripts could not be loaded due to handshake failure. Therefore the then following requests could not run, because the scripts didn't even reach the browser.

Since PhantomJS 1.9.8 the default protocol is set to TLSv1, but it can set manually for earlier versions by passing --ssl-protocol=tlsv1 as command line option. See this answer for more.

This can be checked by registering a onResourceError event handler. The error message will contain something like SSL handshake failed.

Missing requests from phantomjs network monitoring results

1 Answers1