5

I have a rails application that runs parallel_test with rspec inside circleci

Looking around on Internet I added up this to the very beginnging of my spec_helper.rb file:

if ENV['COVERAGE']
  require 'simplecov'
  # on circleci change the output dir to the artifacts
  if ENV['CIRCLE_ARTIFACTS']
    dir = File.join("..", "..", "..", ENV['CIRCLE_ARTIFACTS'], "coverage")
    SimpleCov.coverage_dir(dir)
    SimpleCov.merge_timeout 3600
    SimpleCov.command_name "rspec_#{Process.pid.to_s}#{ENV['TEST_ENV_NUMBER']}"
  end
  SimpleCov.start 'rails'
end

The problem is that as a result I get different folders one for each circleci instance:

enter image description here

What am I doing wrong ?

fabrizioM
  • 46,639
  • 15
  • 102
  • 119

3 Answers3

5

I work at CircleCI. Unfortunately this isn't going to work - we don't collect the artifacts directories from different builds until after all builds have finished running, so tools that try to merge them together during the build won't work. We have talked about adding capabilities to do this, but it's not currently on our feature roadmap, sorry!

tvachon
  • 413
  • 2
  • 5
  • Is this still the case? I am just about to implement @frank-c-eckert's solution via scp... – GDorn Aug 20 '15 at 23:13
3

For anyone who still looking for a solution to this problem, there is a new possibility: using ssh between containers to manually sync and merge reports, see the docs. This is, however, not a turnkey solution, you will have to write the necessary scripts yourself.

Otherwise, you can also use an external coverage service (we use coveralls codecov) together with CircleCI's notification webhook.

Edit

You can add the webhook like this in your circle.yml (thanks Ian): notify: webhooks: - url: https://coveralls.io/webhook?repo_token=(your repo token)

Frank C. Schuetz
  • 684
  • 5
  • 7
  • 1
    can you give a little more detail in how you configure coveralls to work with circle's webhooks? – Nader Aug 28 '15 at 18:08
  • Found this info at https://coveralls.zendesk.com/hc/en-us/articles/203484329-Parallel-Build-Webhook – Ian E Dec 17 '15 at 01:22
1

To expound upon what Frank Eckert said, Coveralls can do this. However, that documentation is slightly off.

Add to you Gemfile:

gem 'coveralls', require: false

Add to your spec/spec_helper.rb:

if ENV['CIRCLECI']
  # If running in CircleCI, run with Coveralls too
  require 'coveralls'
  Coveralls.wear!('rails')
end

Add to your circle.yml (not coveralls.yml as the article states):

notify:
  webhooks:
    - url: https://coveralls.io/webhook?repo_token=your-repo-token

Add to (or create) a .coveralls.yml in the root folder, and add:

repo_token: your-repo-token

Finally, go into the CircleCI project configuration and add an environment variable: COVERALLS_PARALLEL=true

This worked for us, however we find errors in the proper line count (reported to Coveralls) but it is working and gives us a good idea about our coverage changing over time.

Ian E
  • 304
  • 1
  • 7