How can I get table 'td' values with CasperJS?

The HTML source looks like than this:

<table id="my_table">
  <tr id='header'>
    <th>sth_head_name</th>
    <th>ath_head_name</th>
    <th>sth_head_name</th>
    <th>sth_head_name</th>
    <th>sth_head_name</th>
  </tr>
  <tr>
    <td>sth_value</td>
    <td>sth_value</td>
    <td>sth_value</td>
    <td>sth_value</td>
    <td>sth_value</td>
  </tr>
  <tr>
    <td>sth_value</td>
    <td>sth_value</td>
    <td>sth_value</td>
    <td>sth_value</td>
    <td>sth_value</td>
  </tr>
  <tr>
    <td>sth_value</td>
    <td>sth_value</td>
    <td>sth_value</td>
    <td>sth_value</td>
    <td>sth_value</td>
  </tr>
</table>

I'd want to get table values using CasperJS. Firstly, I need to select the rows of table; and then I want to get 'td' values. How can I solve this?

I tried a lot of ways, but those didn't work. My solution would look like something similar that you can see below. Its important, that firstly select 'table_rows'; and then select that's td value inside the for cycle.

var table_rows = casper.getElementsByXpath("//table[@id='my_table']/tr[not(@id='header')]");

for (var i = 0; i < table_rows.length; i++) {
  var firstRequiredCell_query = table_rows[j].getElementByXpath("//td[position()=2]");
  var secondRequiredCell_query = table_rows[j].getElementByXpath("//td[position()=4]");
  var firstRequiredCell = firstRequiredCell_query.text;
  var secondRequiredCell = secondRequiredCell_query.text;
}

How can I get table 'td' values with CasperJS?

The HTML source looks like than this:

<table id="my_table">
  <tr id='header'>
    <th>sth_head_name</th>
    <th>ath_head_name</th>
    <th>sth_head_name</th>
    <th>sth_head_name</th>
    <th>sth_head_name</th>
  </tr>
  <tr>
    <td>sth_value</td>
    <td>sth_value</td>
    <td>sth_value</td>
    <td>sth_value</td>
    <td>sth_value</td>
  </tr>
  <tr>
    <td>sth_value</td>
    <td>sth_value</td>
    <td>sth_value</td>
    <td>sth_value</td>
    <td>sth_value</td>
  </tr>
  <tr>
    <td>sth_value</td>
    <td>sth_value</td>
    <td>sth_value</td>
    <td>sth_value</td>
    <td>sth_value</td>
  </tr>
</table>

I'd want to get table values using CasperJS. Firstly, I need to select the rows of table; and then I want to get 'td' values. How can I solve this?

I tried a lot of ways, but those didn't work. My solution would look like something similar that you can see below. Its important, that firstly select 'table_rows'; and then select that's td value inside the for cycle.

var table_rows = casper.getElementsByXpath("//table[@id='my_table']/tr[not(@id='header')]");

for (var i = 0; i < table_rows.length; i++) {
  var firstRequiredCell_query = table_rows[j].getElementByXpath("//td[position()=2]");
  var secondRequiredCell_query = table_rows[j].getElementByXpath("//td[position()=4]");
  var firstRequiredCell = firstRequiredCell_query.text;
  var secondRequiredCell = secondRequiredCell_query.text;
}

• javascript
• html
• web-scraping
• casperjs

Share Improve this question Follow edited Dec 12, 2015 at 11:22 Artjom B. 62k2626 gold badges135135 silver badges230230 bronze badges asked Dec 12, 2015 at 11:11 Erik BrauneErik Braune 2311 gold badge11 silver badge44 bronze badges

Add a ment  | 

1 Answer 1

Sorted by: Reset to default 0

CasperJS has two contexts. You can only access the DOM directly only from the page context which you get access to inside of casper.evaluate()¹. It is sandboxed and therefore variables defined outside are not available in evaluate().

__utils__.getElementsByXpath() and __utils__.getElementByXpath() are only available in the page context where casper is not available. Those two functions return DOM nodes directly, so those nodes itself don't have the getElementByXpath() function on them.

But you don't need that at all:

casper.then(function(){
    var info = this.evaluate(function(){
        var table_rows = __utils__.getElementsByXpath("//table[@id='my_table']/tr[not(@id='header')]");

        return table_rows.map(function(tr){
            return {
                a: tr.children[1].textContent,
                b: tr.children[3].textContent
            };
        });
    });
    this.echo(JSON.stringify(info, undefined, 4));
});

You can use all of the ways to traverse the DOM like children, querySelector() or document.evaluate().

¹ Please also read the PhantomJS documentation of the same function.