CasperJS' getElementsByXpath returns null with valid XPath

Question

I m scraping soccer scores data from website. All scores are in a table, every <tr> has "block home matches 17" and some unique stuff after it.

I tested my xpath in the Chrome dev tools, it recognizes only the table rows I need.

var utils = require('utils');
var casper = require('casper').create();
var xpath = require('casper').selectXPath;
var result = [];

function getScores(){
    console.log("getting scores");
    result = __utils__.getElementsByXPath("//tr[contains(@id,'block_home_matches_17')");
}

casper.start('http://int.soccerway.com/', function() {
    console.log("casper start....");       
    var l = getScores();
    utils.dump(l);
});

casper.run();

The code returns [] as utils.dump! Why? my xpath is valid!

Related? http://stackoverflow.com/questions/3655549/xpath-containstext-some-string-doesnt-work-when-used-with-node-with-more — Marvin Smit, Jan 03 '16 at 16:25

score 4 · Answer 1 · answered Jan 03 '16 at 16:45

You have three problems:

You can only access DOM nodes in the sandboxed page context through evaluate(), but
You can't pass DOM nodes to the outside context (read the PhantomJS documentation).
The clientutils (__utils__) module is only available in the page context.

You can retrieve a representation of your targeted DOM nodes either through CasperJS functions:

casper.start('http://int.soccerway.com/', function() {
    utils.dump(this.getElementsInfo(xpath("//tr[contains(@id,'block_home_matches_17')")));
});

or by directly working on the elements in the page context:

casper.start('http://int.soccerway.com/', function() {
    utils.dump(this.evaluate(function(){
        return __utils__.getElementsByXPath("//tr[contains(@id,'block_home_matches_17')").map(function(el){
            return {} // TODO: produce your own representation
        });
    }));
});

Additionally, you're not returning anything from `getScores()` so `l` will always be null. — Artjom B., Jan 03 '16 at 16:46

CasperJS' getElementsByXpath returns null with valid XPath

1 Answers1