Cucumber load data tables dynamically

Question

I am currently trying to use cucumber together with capybara for some integration tests of a web-app.

There is one test where I just want to click through all (or most of) the pages of the web app and see if no error is returned. I want to be able to see afterwards which pages are not working.

I think that Scenario outlines would be the best approach so I started in that way:

Scenario Outline: Checking all pages pages

   When I go on the page <page>
   Then the page has no HTTP error response

    Examples:
      | page                        |
      | "/resource1"                |
      | "/resource2"                |
      ...

I currently have 82 pages and that works fine.

However I find this approach is not maintable as there may new resources and resources that will be deleted.

A better approach would be to load the data from the table from somewhere (parsing HTML of an index page, the database etc...).

But I did not figure out how to do that.

I came across an article about table transformation but I could not figure out how to use this transformation in an scenario outline.

Are there any suggestions?

OK since there is some confusion. If you have a look at the example above. All I want to do is change it so that the table is almost empty:

Scenario Outline: Checking all pages pages

  When I go on the page <page>
  Then the page has no HTTP error response

  Examples:
    | page                        |
    | "will be generated"         |

Then I want to add a transformation that looks something like this:

Transform /^table:page$/ do
  all_my_pages.each do |page|
    table.hashes << {:page => page}
  end 
  table.hashes
end

I specified the transformation in the same file, but it is not executed, so I was assuming that the transformations don't work with Scenario outlines.

Answer 1

Cucumber is really the wrong tool for that task, you should describe functionality in terms of features. If you want to describe behavior programmatically you should use something like rspec or test-unit.

Also your scenario steps should be descriptive and specialized like a written text and not abstract phrases like used in a programming language. They should not include "incidental details" like the exact url of a ressource or it's id.

Please read http://blog.carbonfive.com/2011/11/07/modern-cucumber-and-rails-no-more-training-wheels/ and watch http://skillsmatter.com/podcast/home/refuctoring-your-cukes

Concerning your question about "inserting into tables", yes it is possible if you mean adding additional rows to it, infact you could do anything you like with it. The result of the Transform block completely replaces the original table.

Transform /^table:Name,Posts$/ do
  # transform the table into a list of hashes
  results = table.hashes.map do |row|
    user = User.create! :name => row["Name"]
    posts = (1..row["Posts"]).map { |i| Post.create! :title => "Nr #{i}" }
    { :user => user, :posts => posts }
  end
  # append another hash to the results (e.g. a User "Tim" with 2 Posts)
  tim = User.create! :name => "Tim"
  tims_posts = [Post.create! :title => "First", Post.create! :title => "Second"]
  results << { :user => tim, :posts => tims_posts }
  results
end

Given /^I have Posts of the following Users:$/ do |transformation_results|
  transformation_results.each do |row|
    # assing Posts to the corresponding User
    row[:user].posts = row[:posts]
  end
end

You could combine this with Scenario Outlines like this:

Scenario Outline: Paginate the post list of an user at 10
  Given I have Posts of the following Users:
    | Name | Posts |
    | Max  | 7     |
    | Tom  | 11    |
  When I visit the post list of <name>
  Then I should see <count> posts
Examples:
  | name | count |
  | Max  |     7 |
  | Tom  |    10 |
  | Tim  |     2 |

This should demonstarte why "adding" rows to a table, might not be best practice.

Please note that it is impossible to expand example tags inside of a table:

Scenario Outline: Paginate the post list of an user at 10
  Given I have Posts of the following Users:
    | Name   | Posts      |
    | <name> | <existing> | # won't work
  When I visit the post list of <name>
  Then I should see <displayed> posts
Examples:
  | name | existing | displayed |
  | Max  |     7    |         7 |
  | Tom  |    11    |        10 |
  | Tim  |     2    |         2 |

Answer 2

For the specific case of loading data dynamically, here's a suggestion:

1. A class, let's say PageSets, with methods, e.g. all_pages_in_the_sitemap_errorcount, developing_countries_errorcount.

2. a step that reads something like

   Given I am on the "Check Stuff" page
   Then there are 0 errors in the "developing countries" pages
   

or

    Then there are 0 errors in "all pages in the sitemap"

The Then step converts the string "developing countries" into a method name developing_countries_errorcountand attempts to call it on class PageSets. The step expects all _errorcount methods to return an integer in this case. Returning data structures like maps gives you many possibilities for writing succinct dynamic steps.

For more static data we have found YAML very useful for making our tests self-documenting and self-validating, and for helping us remove hard-to-maintain literals like "5382739" that we've all forgotten the meaning of three weeks later.

The YAML format is easy to read and can be commented if necessary (it usually isn't.)

Rather than write:

Given I am logged in as "jackrobinson@gmail.com"
And I select the "History" tab
Then I can see 5 or more "rows of history"

We can write instead:

Given I am logged in as "a user with at least 5 items of history"
When I select the "History" tab
Then I can see 5 or more "rows of history"

In file logins.yaml....

a member with at least 5 items of history:
     username: jackrobinson@gmail.com
     password: WalRus

We use YAML to hold sets of data relating to all sorts of entities like members, providers, policies, ... the list is growing all the time:

In file test_data.yaml...

a member who has direct debit set up:
    username: jackrobinson@gmail.com
    password: WalRus
    policyId: 5382739
    first name: Jack
    last name: Robinson
    partner's first name: Sally
    partner's last name: Fredericks

It's also worth looking at YAML's multi-line text facilities if you need to verify text. Although that's not usual for automation tests, it can sometimes be useful.

Answer 3

A quick hack is to change the Examples collector code, and using eval of ruby to run your customized ruby function to overwrite the default collected examples data, here is the code: generate-dynamic-examples-for-cucumber

drawback: need change the scenario_outline.rb file.

Answer 4

I think that the better approach would be using different tool, just for crawling your site and checking if no error is returned. Assuming you're using Rails

The tool you might consider is: Tarantula.

https://github.com/relevance/tarantula

I hope that helps :)