simplecov-ruby / simplecov Goto Github PK

I notice that when I have autotest running, my coverage results fluctuate as autotest runs.

For example, I run autotest for the first time and it runs my full test suite. I view the coverage results, and say the total is 80%. Then I add some new tests for some model, and autotest runs only the test file for that model, and now when I view the coverage results, the percent covered is lower, say 70%, because not all of the tests were run (I'm guessing). If I make autotest run the entire test suite again, now I see the coverage go up, to reflect the new tests I've written.

Is this normal behavior? Is there any way to configure simplecov so that the coverage doesn't go down like that?

Hi,

Running simplecov along with parallel_tests. Added the following SimpleCov block in both spec/spec_helper.rb
and features/support/env.rb

SimpleCov.start 'rails' do
merge_timeout 7200
add_filter "/vendor/"
end

Coverage results are overwritten which I understand as each Rspec or cucumber test generates
a coverage report.

Is it possible to run simplecov with parallel_tests and merge the coverage of each of
them ?

If there is an empty resultset.yml file (maybe from an aborted run) you get the error:
lib/simplecov/result_merger.rb:56:in store_result': undefined method[]=' for false:FalseClass (NoMethodError)

I don't know if this needs a fix but it might be useful to have a troubleshooting note.

require 'simplecov'
SimpleCov.start
describe 'exit status' do
  it "should exit with a non-zero exit status when assertion fails" do
    1.should == 2
  end
end

Put the above code in a file, run the file.

echo $?
=> 0

it should return 1

Because I have a model in my rails app called Coverage when running tests I get the error:

Coverage is not a class (TypeError)

This is obviously a naming conflict. Do you think it is worth name spacing the simplecov internal classes so they do not conflict with the app being tested?

Thanks!

irb(main):001:0> require 'git'
=> true
irb(main):002:0> require 'simplecov'
=> true
irb(main):003:0> require 'thin'
LoadError: no such file to load -- thin
from (irb):3:in require' from (irb):3 from /usr/bin/irb:12:in

I'm using ruby 1.9.2-p180, simplecov gem 0.4.2.

Though, i've written test for my model methods, simple coverage html file shows all of them as missed lines. This has happened only in few of my models and not all.

Nothing work for me anymore... =/

    /Users/diegorv/.rvm/gems/ruby-1.9.2-p136@rails3/gems/simplecov-html-0.4.0/lib/simplecov-html.rb:113:in `require': no such file to load -- simplecov-html/version (LoadError)
from /Users/diegorv/.rvm/gems/ruby-1.9.2-p136@rails3/gems/simplecov-html-0.4.0/lib/simplecov-html.rb:113:in `<top (required)>'
from <internal:lib/rubygems/custom_require>:29:in `require'
from <internal:lib/rubygems/custom_require>:29:in `require'
from /Users/diegorv/.rvm/gems/ruby-1.9.2-p136@rails3/gems/simplecov-0.4.0/lib/simplecov.rb:122:in `<top (required)>'
from <internal:lib/rubygems/custom_require>:33:in `require'
from <internal:lib/rubygems/custom_require>:33:in `rescue in require'
from <internal:lib/rubygems/custom_require>:29:in `require'
from /Users/diegorv/Documents/rails/work/phenix/spec/spec_helper.rb:5:in `block in <top (required)>'
from /Users/diegorv/.rvm/gems/ruby-1.9.2-p136@rails3/gems/spork-0.8.4/lib/spork.rb:23:in `prefork'
from /Users/diegorv/Documents/rails/work/phenix/spec/spec_helper.rb:4:in `<top (required)>'
from <internal:lib/rubygems/custom_require>:29:in `require'
from <internal:lib/rubygems/custom_require>:29:in `require'
from /Users/diegorv/Documents/rails/work/phenix/spec/helpers/admin/admin_helper_spec.rb:3:in `<top (required)>'
from /Users/diegorv/.rvm/gems/ruby-1.9.2-p136@rails3/gems/rspec-core-2.5.1/lib/rspec/core/configuration.rb:386:in `load'
from /Users/diegorv/.rvm/gems/ruby-1.9.2-p136@rails3/gems/rspec-core-2.5.1/lib/rspec/core/configuration.rb:386:in `block in load_spec_files'
from /Users/diegorv/.rvm/gems/ruby-1.9.2-p136@rails3/gems/rspec-core-2.5.1/lib/rspec/core/configuration.rb:386:in `map'
from /Users/diegorv/.rvm/gems/ruby-1.9.2-p136@rails3/gems/rspec-core-2.5.1/lib/rspec/core/configuration.rb:386:in `load_spec_files'
from /Users/diegorv/.rvm/gems/ruby-1.9.2-p136@rails3/gems/rspec-core-2.5.1/lib/rspec/core/command_line.rb:18:in `run'
from /Users/diegorv/.rvm/gems/ruby-1.9.2-p136@rails3/gems/rspec-core-2.5.1/lib/rspec/core/runner.rb:55:in `run_in_process'
from /Users/diegorv/.rvm/gems/ruby-1.9.2-p136@rails3/gems/rspec-core-2.5.1/lib/rspec/core/runner.rb:46:in `run'
from /Users/diegorv/.rvm/gems/ruby-1.9.2-p136@rails3/gems/rspec-core-2.5.1/lib/rspec/core/runner.rb:10:in `block in autorun'

Rails 3 application. Simple tests (no rspec, no cucumber).

If I add this to my gemfile :

require 'test-unit', '2.3.0'

then merging doesn't work as unit, functional, and integration test results overwrite each-other.

Thing is, I need this for the performance tests to work.

This problem is demonstrated by putting the following code in spec/spec_helper.rb of a rails project.

require 'simplecov'
SimpleCov.start 'rails'
raise "some exception in the class loading before the tests start"

When rspec is invoked via rake, the above exception is logged to stderr and no tests are run, but the rake process exits 0 (i.e. success), where it should exit non-zero (failure) as it does when the SimpleCov.start is omitted.

Context:
This is a simplified version of the problem we encountered. We had a CI box that didn't receive a new config file required by a codebase change. When rake (rspec) ran, our app initialization code correctly raised an exception to protest but because our test suite was running under simplecov the rake (rspec) task exited 0 (success) indicating a passing build.

Environment:
ruby 1.9.2-p180,
simplecov gem 0.4.2
rspec gem 2.5.0
rubygems 1.6.2

Extra information:
I note that if I register an at_exit handler which does NOT call SimpleCov.result.format!, the exit code is propogated correctly (though of course no report will be produced). E.g. SimpleCov.at_exit { puts "simplecov at_exit handler - doing nothing" }

Thanks for any help, and for your work on this gem!

This test doesn't work on the first or second day of the month, because Time.mktime is called with:

Time.mktime(Time.now.year, Time.now.month, Time.now.day-2)

This can result in the month day being zero or negative, which is obviously not valid. A better way of calculating the timestamp two days (since activesupport isn't available) is something like:

Time.now - 172800 # two days ago

The files content isn't escaped. So if there's HTML in it (like in dynamic_form for example), the html fields are rendered in the page unescaped.

I'm using cucumber-rails 1.0.2 and cucumber 1.0,0, simplecov 0.4.2, ruby 1.9.2 and rails 3.0.9

After running the specs I'm getting

simplecov-0.4.2/lib/simplecov/merge_helpers.rb:16:in block in merge_resultset': undefined method+' for #Syck::PrivateType:0x00000109058a48 (NoMethodError)
from /Users/felix/.rvm/gems/ruby-1.9.2-p180/gems/simplecov-0.4.2/lib/simplecov/merge_helpers.rb:10:in each' from /Users/felix/.rvm/gems/ruby-1.9.2-p180/gems/simplecov-0.4.2/lib/simplecov/merge_helpers.rb:10:ineach_with_index'
from /Users/felix/.rvm/gems/ruby-1.9.2-p180/gems/simplecov-0.4.2/lib/simplecov/merge_helpers.rb:10:in merge_resultset' from /Users/felix/.rvm/gems/ruby-1.9.2-p180/gems/simplecov-0.4.2/lib/simplecov/merge_helpers.rb:32:inblock in merge_resultset'
from /Users/felix/.rvm/gems/ruby-1.9.2-p180/gems/simplecov-0.4.2/lib/simplecov/merge_helpers.rb:31:in each' from /Users/felix/.rvm/gems/ruby-1.9.2-p180/gems/simplecov-0.4.2/lib/simplecov/merge_helpers.rb:31:inmerge_resultset'
from /Users/felix/.rvm/gems/ruby-1.9.2-p180/gems/simplecov-0.4.2/lib/simplecov/result_merger.rb:44:in block in merged_result' from /Users/felix/.rvm/gems/ruby-1.9.2-p180/gems/simplecov-0.4.2/lib/simplecov/result_merger.rb:43:ineach'
from /Users/felix/.rvm/gems/ruby-1.9.2-p180/gems/simplecov-0.4.2/lib/simplecov/result_merger.rb:43:in merged_result' from /Users/felix/.rvm/gems/ruby-1.9.2-p180/gems/simplecov-0.4.2/lib/simplecov.rb:51:inresult'
from /Users/felix/.rvm/gems/ruby-1.9.2-p180/gems/simplecov-0.4.2/lib/simplecov/configuration.rb:120:in block in at_exit' from /Users/felix/.rvm/gems/ruby-1.9.2-p180/gems/simplecov-0.4.2/lib/simplecov.rb:134:incall'
from /Users/felix/.rvm/gems/ruby-1.9.2-p180/gems/simplecov-0.4.2/lib/simplecov.rb:134:in `block in <top (required)>'

When I run my test suite (with rake which runs rspec, then cucumber, then stats). I added
require 'simplecov'
SimpleCov.start 'rails'
to the very top of the spec_helper file. The output after rspecs are run show:
75.92% covered.
Then after cucumbers and stats run I see:
75.92% covered.
But in the HTML index page, it says
All Files (85.42%)
If I open console right after running the test suite, I try:
>> require 'simplecov'
=> ["SimpleCov"]
>> SimpleCov.result.covered_percent
=> 75.91836734693878

Earlier I tried adding the require 'simplecov'; SimpleCov.start 'rails' to the top of env.rb (for cucumber), but then I got a third, different number AND the coverage percentage in the HTML index file went down after I did that (which would seem incorrect, because merging your code coverage %'s should actually only cause them to go UP, rather than be a simple average).

So what gives? I assume the higher number is correct. I added a final task to my rake task that looks like this:
desc 'check coverage'
task :coverage do
raise "Code coverage too low!" if SimpleCov.result.covered_percent < 85
end
This seems to exit with status 0, so it seems that in that context, the covered_percent is 85. I just wish I knew what exactly was going on, and all the variance in the percentage definitely shakes my confidence in the accuracy of the tool.

This is the error message I get on winXP sp2:

• ruby 1.9.1 (mingw32)
• simplecov (0.3.2)
• simplecov-html (0.3.8)
• cucumber (0.8.5)

c:/ruby/lib/ruby/gems/1.9.1/gems/simplecov-0.3.2/lib/simplecov/result_merger.rb:
57:in initialize': No such file or directory - c:/repositories/calculator/coverage/resultset.yml (Errno::ENOENT) from c:/ruby/lib/ruby/gems/1.9.1/gems/simplecov-0.3.2/lib/simplecov/result_merger.rb:57:inopen'
from c:/ruby/lib/ruby/gems/1.9.1/gems/simplecov-0.3.2/lib/simplecov/result_merger.rb:57:in store_result' from c:/ruby/lib/ruby/gems/1.9.1/gems/simplecov-0.3.2/lib/simplecov.rb:53:inresult'
from c:/ruby/lib/ruby/gems/1.9.1/gems/simplecov-0.3.2/lib/simplecov/configuration.rb:118:in block in at_exit' from c:/ruby/lib/ruby/gems/1.9.1/gems/simplecov-0.3.2/lib/simplecov.rb:119:incall'
from c:/ruby/lib/ruby/gems/1.9.1/gems/simplecov-0.3.2/lib/simplecov.rb:119:in `block in <top (required)>'

Once the folder is created, simplecov works fine.

Christian

Run via autotest -> rspec, with the following in my spec_helper.rb:

require 'simplecov'

SimpleCov.start do
add_filter "spec"
add_filter "binstubs" #Prevents binstubs folder from appearing in code coverage
merge_timeout 3600
end

root = File.expand_path(File.dirname(File.dirname(FILE)))
$LOAD_PATH.unshift(
File.join(root,'lib'),
File.join(root,'config'),
File.join(root,'lib','models')
)

/Users/aaronstewart/.rvm/gems/ruby-1.9.2-p180/gems/simplecov-0.4.2/lib/simplecov/merge_helpers.rb:16:in `+': Hash can't be coerced into Fixnum (TypeError)
from /Users/aaronstewart/.rvm/gems/ruby-1.9.2-p180/gems/simplecov-0.4.2/lib/simplecov/merge_helpers.rb:16:in `block in merge_resultset'
from /Users/aaronstewart/.rvm/gems/ruby-1.9.2-p180/gems/simplecov-0.4.2/lib/simplecov/merge_helpers.rb:10:in `each'
from /Users/aaronstewart/.rvm/gems/ruby-1.9.2-p180/gems/simplecov-0.4.2/lib/simplecov/merge_helpers.rb:10:in `each_with_index'
from /Users/aaronstewart/.rvm/gems/ruby-1.9.2-p180/gems/simplecov-0.4.2/lib/simplecov/merge_helpers.rb:10:in `merge_resultset'
from /Users/aaronstewart/.rvm/gems/ruby-1.9.2-p180/gems/simplecov-0.4.2/lib/simplecov/merge_helpers.rb:32:in `block in merge_resultset'
from /Users/aaronstewart/.rvm/gems/ruby-1.9.2-p180/gems/simplecov-0.4.2/lib/simplecov/merge_helpers.rb:31:in `each'
from /Users/aaronstewart/.rvm/gems/ruby-1.9.2-p180/gems/simplecov-0.4.2/lib/simplecov/merge_helpers.rb:31:in `merge_resultset'
from /Users/aaronstewart/.rvm/gems/ruby-1.9.2-p180/gems/simplecov-0.4.2/lib/simplecov/result_merger.rb:44:in `block in merged_result'
from /Users/aaronstewart/.rvm/gems/ruby-1.9.2-p180/gems/simplecov-0.4.2/lib/simplecov/result_merger.rb:43:in `each'
from /Users/aaronstewart/.rvm/gems/ruby-1.9.2-p180/gems/simplecov-0.4.2/lib/simplecov/result_merger.rb:43:in `merged_result'
from /Users/aaronstewart/.rvm/gems/ruby-1.9.2-p180/gems/simplecov-0.4.2/lib/simplecov.rb:51:in `result'
from /Users/aaronstewart/.rvm/gems/ruby-1.9.2-p180/gems/simplecov-0.4.2/lib/simplecov/configuration.rb:120:in `block in at_exit'
from /Users/aaronstewart/.rvm/gems/ruby-1.9.2-p180/gems/simplecov-0.4.2/lib/simplecov.rb:134:in `call'
from /Users/aaronstewart/.rvm/gems/ruby-1.9.2-p180/gems/simplecov-0.4.2/lib/simplecov.rb:134:in `block in <top (required)>'

If I run rspec directly (no autotest) the error message changes slightly:

/Users/aaronstewart/.rvm/gems/ruby-1.9.2-p180/gems/simplecov-0.4.2/lib/simplecov/merge_helpers.rb:16:in `block in merge_resultset': undefined method `+' for {"str"=>"", :@taguri=>"x-private:null"}:Hash (NoMethodError)
from /Users/aaronstewart/.rvm/gems/ruby-1.9.2-p180/gems/simplecov-0.4.2/lib/simplecov/merge_helpers.rb:10:in `each'
from /Users/aaronstewart/.rvm/gems/ruby-1.9.2-p180/gems/simplecov-0.4.2/lib/simplecov/merge_helpers.rb:10:in `each_with_index'
from /Users/aaronstewart/.rvm/gems/ruby-1.9.2-p180/gems/simplecov-0.4.2/lib/simplecov/merge_helpers.rb:10:in `merge_resultset'
from /Users/aaronstewart/.rvm/gems/ruby-1.9.2-p180/gems/simplecov-0.4.2/lib/simplecov/merge_helpers.rb:32:in `block in merge_resultset'
from /Users/aaronstewart/.rvm/gems/ruby-1.9.2-p180/gems/simplecov-0.4.2/lib/simplecov/merge_helpers.rb:31:in `each'
from /Users/aaronstewart/.rvm/gems/ruby-1.9.2-p180/gems/simplecov-0.4.2/lib/simplecov/merge_helpers.rb:31:in `merge_resultset'
from /Users/aaronstewart/.rvm/gems/ruby-1.9.2-p180/gems/simplecov-0.4.2/lib/simplecov/result_merger.rb:44:in `block in merged_result'
from /Users/aaronstewart/.rvm/gems/ruby-1.9.2-p180/gems/simplecov-0.4.2/lib/simplecov/result_merger.rb:43:in `each'
from /Users/aaronstewart/.rvm/gems/ruby-1.9.2-p180/gems/simplecov-0.4.2/lib/simplecov/result_merger.rb:43:in `merged_result'
from /Users/aaronstewart/.rvm/gems/ruby-1.9.2-p180/gems/simplecov-0.4.2/lib/simplecov.rb:51:in `result'
from /Users/aaronstewart/.rvm/gems/ruby-1.9.2-p180/gems/simplecov-0.4.2/lib/simplecov/configuration.rb:120:in `block in at_exit'
from /Users/aaronstewart/.rvm/gems/ruby-1.9.2-p180/gems/simplecov-0.4.2/lib/simplecov.rb:134:in `call'
from /Users/aaronstewart/.rvm/gems/ruby-1.9.2-p180/gems/simplecov-0.4.2/lib/simplecov.rb:134:in `block in <top (required)>'

Hi,

I'm getting 0 coverage when I try to configure simplecov in the Rakefile using rspec 2.6.0 and ruby 1.9.2. It only seems to work in the spec_helper.rb. The reason I'd like it to work in the Rakefile so I can have the option of just running tests without coverage, for example "rake spec" would run just the tests and "rake coverage" would run the tests as well as coverage.

Thanks,
Jay

Is there a way to put SimpleCov in a rake task? I would love to have it only run the coverage when I do rake coverage and then open the results with Launchy. I tried this in a Rails app:

desc "Run RSpec with code coverage"
task :coverage do
  require "simplecov"
  SimpleCov.start "rails"
  Rake::Task["spec"].execute
end

But I got this exception when running it.

/Users/rbates/.rvm/gems/ruby-1.9.2-p136@railscasts/gems/simplecov-0.4.2/lib/simplecov/source_file.rb:25:in `initialize': Only String accepted for source (ArgumentError)
    from /Users/rbates/.rvm/gems/ruby-1.9.2-p136@railscasts/gems/simplecov-0.4.2/lib/simplecov/source_file.rb:66:in `new'
    from /Users/rbates/.rvm/gems/ruby-1.9.2-p136@railscasts/gems/simplecov-0.4.2/lib/simplecov/source_file.rb:66:in `block in lines'
    from /Users/rbates/.rvm/gems/ruby-1.9.2-p136@railscasts/gems/simplecov-0.4.2/lib/simplecov/source_file.rb:65:in `each'
    from /Users/rbates/.rvm/gems/ruby-1.9.2-p136@railscasts/gems/simplecov-0.4.2/lib/simplecov/source_file.rb:65:in `each_with_index'
    from /Users/rbates/.rvm/gems/ruby-1.9.2-p136@railscasts/gems/simplecov-0.4.2/lib/simplecov/source_file.rb:65:in `lines'
    from /Users/rbates/.rvm/gems/ruby-1.9.2-p136@railscasts/gems/simplecov-0.4.2/lib/simplecov/source_file.rb:91:in `missed_lines'
    from /Users/rbates/.rvm/gems/ruby-1.9.2-p136@railscasts/gems/simplecov-html-0.4.4/lib/simplecov-html.rb:82:in `block in lines_missed'
    from /Users/rbates/.rvm/gems/ruby-1.9.2-p136@railscasts/gems/simplecov-html-0.4.4/lib/simplecov-html.rb:82:in `map'
    from /Users/rbates/.rvm/gems/ruby-1.9.2-p136@railscasts/gems/simplecov-html-0.4.4/lib/simplecov-html.rb:82:in `lines_missed'
    from /Users/rbates/.rvm/gems/ruby-1.9.2-p136@railscasts/gems/simplecov-html-0.4.4/lib/simplecov-html.rb:72:in `lines_of_code'
    from /Users/rbates/.rvm/gems/ruby-1.9.2-p136@railscasts/gems/simplecov-html-0.4.4/lib/simplecov-html.rb:65:in `coverage'
    from (erb):4:in `formatted_file_list'
    from /Users/rbates/.rvm/rubies/ruby-1.9.2-p136/lib/ruby/1.9.1/erb.rb:753:in `eval'
    from /Users/rbates/.rvm/rubies/ruby-1.9.2-p136/lib/ruby/1.9.1/erb.rb:753:in `result'
    from /Users/rbates/.rvm/gems/ruby-1.9.2-p136@railscasts/gems/simplecov-html-0.4.4/lib/simplecov-html.rb:60:in `formatted_file_list'
    from (erb):28:in `block in format'
    from /Users/rbates/.rvm/rubies/ruby-1.9.2-p136/lib/ruby/1.9.1/erb.rb:753:in `eval'
    from /Users/rbates/.rvm/rubies/ruby-1.9.2-p136/lib/ruby/1.9.1/erb.rb:753:in `result'
    from /Users/rbates/.rvm/gems/ruby-1.9.2-p136@railscasts/gems/simplecov-html-0.4.4/lib/simplecov-html.rb:26:in `block in format'
    from /Users/rbates/.rvm/gems/ruby-1.9.2-p136@railscasts/gems/simplecov-html-0.4.4/lib/simplecov-html.rb:25:in `open'
    from /Users/rbates/.rvm/gems/ruby-1.9.2-p136@railscasts/gems/simplecov-html-0.4.4/lib/simplecov-html.rb:25:in `format'
    from /Users/rbates/.rvm/gems/ruby-1.9.2-p136@railscasts/gems/simplecov-0.4.2/lib/simplecov/result.rb:77:in `format!'
    from /Users/rbates/.rvm/gems/ruby-1.9.2-p136@railscasts/gems/simplecov-0.4.2/lib/simplecov/configuration.rb:120:in `block in at_exit'
    from /Users/rbates/.rvm/gems/ruby-1.9.2-p136@railscasts/gems/simplecov-0.4.2/lib/simplecov.rb:134:in `call'
    from /Users/rbates/.rvm/gems/ruby-1.9.2-p136@railscasts/gems/simplecov-0.4.2/lib/simplecov.rb:134:in `block in <top (required)>'

Is there a good way to do this?

Hi,

sometimes my code is too big to fit in the default pop-up window for line coverage hit analysis.

Would it be possible to add a 'maximize' button ?

Christian

Trace: https://gist.github.com/dda22fa6e6c8c9b3974a

SimpleCov should throw a warning and skip coverage analysis in Ruby 1.8.6 (as in 1.8.7).

MRI 1.8.6 is still widely used in development since it shipped with Mac OS X 10.5.

Hi,

after installing simplecov and adding the two lines to the top of spec_helper.rb, I get this message after running rake spec:

/Users/sebastian/.rvm/rubies/ruby-1.9.2-p136/lib/ruby/1.9.1/psych/tree_builder.rb:24: stack level too deep (SystemStackError)

rake aborted!

The resultset.yml is empty after rake spec finishes...

Regards,
sewid

A few months ago @lsegal released ripper, a rubygem that ports the ripper parser to 1.8:

http://gnuu.org/2011/01/12/announcing-ripper-for-ruby-1-8-x/

I was wondering if it were possible to get simplecov to work with that gem on 1.8, or are there other dependencies on 1.9 besides ripper?

I could see about testing this out but I wanted to check with you first in case there are constraints I don't know about.

When a source file is deleted from the repo, simplecov will still try to read it's source lines when importing the resultset.yml cache, which obviously will fail.

I've run into this twice now, and I'm sorry, but I don't know what's tripping it up.

I'm using guard to run my tests / simplecov.

If it runs with this code:

def following_all
  ...
  all.map! do |a|
    # Return array
    if a.class == MyClass
      ... do something
    else
      ... do something else
    end
  end
  all.flatten!
end

It will be fine. But if I take out that the line with the comment (the comment doesn't matter, the extra line does) then it will crash.

/Users/ryanjm/.rvm/gems/ruby-1.9.2-head/gems/simplecov-0.4.2/lib/simplecov/source_file.rb:25:in `initialize': Only String accepted for source (ArgumentError)
    from /Users/ryanjm/.rvm/gems/ruby-1.9.2-head/gems/simplecov-0.4.2/lib/simplecov/source_file.rb:66:in `new'
    from /Users/ryanjm/.rvm/gems/ruby-1.9.2-head/gems/simplecov-0.4.2/lib/simplecov/source_file.rb:66:in `block in lines'
    from /Users/ryanjm/.rvm/gems/ruby-1.9.2-head/gems/simplecov-0.4.2/lib/simplecov/source_file.rb:65:in `each'
    from /Users/ryanjm/.rvm/gems/ruby-1.9.2-head/gems/simplecov-0.4.2/lib/simplecov/source_file.rb:65:in `each_with_index'
    from /Users/ryanjm/.rvm/gems/ruby-1.9.2-head/gems/simplecov-0.4.2/lib/simplecov/source_file.rb:65:in `lines'
    from /Users/ryanjm/.rvm/gems/ruby-1.9.2-head/gems/simplecov-0.4.2/lib/simplecov/source_file.rb:91:in `missed_lines'
    from /Users/ryanjm/.rvm/gems/ruby-1.9.2-head/gems/simplecov-html-0.4.5/lib/simplecov-html.rb:80:in `block in lines_missed'
    from /Users/ryanjm/.rvm/gems/ruby-1.9.2-head/gems/simplecov-html-0.4.5/lib/simplecov-html.rb:80:in `map'
    from /Users/ryanjm/.rvm/gems/ruby-1.9.2-head/gems/simplecov-html-0.4.5/lib/simplecov-html.rb:80:in `lines_missed'
    from /Users/ryanjm/.rvm/gems/ruby-1.9.2-head/gems/simplecov-html-0.4.5/lib/simplecov-html.rb:70:in `lines_of_code'
    from /Users/ryanjm/.rvm/gems/ruby-1.9.2-head/gems/simplecov-html-0.4.5/lib/simplecov-html.rb:63:in `coverage'
    from (erb):4:in `formatted_file_list'
    ....

Any idea what is tripping this issue?

https://github.com/colszowka/simplecov/issues/closed#issue/15

Okay, so I was seeing this for a while. It was almost reproduce-able, in that it was (I could add a pending to a test, then uncomment the pending and it would pass and fail deterministically.) So I took my code, and prepared to present it here to help find the problem ... and while doing so ... it started passing every time. So, the error definitely appears to be non-deterministic.

Anyway, here's the code
https://github.com/softwaregravy/base_project (first checkin and first change)
And here's the change. I just kept doing and undoing this and saw the error
softwaregravy/base_project@896d527

I was also installing and uninstalling annotate-models 1.0.4 which was the last gem I had installed before this happened (but no, I have no idea how that might impact it, just a reaction like "oh no, what did I change last"). First observed while running autotest.

It does not now appear to be reproduce-able. Weird. Anything pulling timestamps in the code?

I was in the process of factoring out my simplecov config (so there's some code there, probably in the wrong place, but it was not in use).

So, in conclusion, I observed the failure, was able to reproduce for a short while, but not cannot. Not vary satisfying, I know.

Hi,

maybe this is more a help than a issue, but I'm blind at this moment to got what is going on.

Everything works fine in a Rails project, but after some test rans I saw that nothing is changing reports. I can comments all my tests and the coverage still the same.

Is anyone suffering from that?
Any clue?

See http://twitter.com/#!/hayafirst/status/28368786944

And my responses:
Check out custom exit behaviour: http://bit.ly/953iOR - could raise an exception after result change. See http://bit.ly/9PSaOt
Basically, cache the result.covered_percent somewhere, then later compare it with previous and cause an exit with status > 0

This could be an (optional) built-in functionality.
The SimpleCov.result.covered_percent should be stored in coverage_path/last_coverage_percent, then compared with the current run.

Configure with:

SimpleCov.start(:rails) do
  check_coverage_change 
end

Hi there,

the following setup worked fine until Rails 3.0.1.

I got the following custom adapter:

SimpleCov.adapters.define 'rspec' do
  add_filter 'test/'
  add_filter 'config/'
  add_filter 'spec/'
  add_filter 'features/'
  add_filter 'vendor/'
  add_filter 'lib/'

  add_filter 'app/uploaders'
  add_filter 'app/controllers'
  add_filter 'app/behaviours'

  add_group 'Models', 'app/models'
  add_group 'Modules', 'app/modules'
  add_group 'Helpers', 'app/helpers'
  add_group 'Mailers', 'app/mailers'
end

SimpleCov.adapters.define 'cucumber' do
  add_filter 'test/'
  add_filter 'config/'
  add_filter 'spec/'
  add_filter 'features/'
  add_filter 'vendor/'
  add_filter 'lib/'

  add_group 'Models', 'app/models'
  add_group 'Modules', 'app/modules'
  add_group 'Mailers', 'app/mailers'
  add_group 'Helpers', 'app/helpers'
  add_group 'Uploaders', 'app/uploaders'
  add_group 'Controllers', 'app/controllers'
  add_group 'Behaviours', 'app/behaviours'
end

Added the following for Rspec to spec/spec_helper.rb:

require 'simplecov'
require File.dirname(__FILE__) + '/../lib/simplecov_custom_adapters'
SimpleCov.start 'rspec'

And the following for Cucumber to the beginning of features/support/env.rb:

require 'simplecov'
require File.dirname(__FILE__) + '/../../lib/simplecov_custom_adapters'
SimpleCov.start 'cucumber'

This throws (at least with Rails 3.0.1 and Ruby 1.9.2) the following error:

/path_to/gems/activesupport-3.0.1/lib/active_support/core_ext/float/rounding.rb:16:in 'round': NaN (FloatDomainError)
from /path_to/gems/activesupport-3.0.1/lib/active_support/core_ext/float/rounding.rb:16:in 'round'
from /path_to/gems/activesupport-3.0.1/lib/active_support/core_ext/float/rounding.rb:14:in 'round'
from (erb):4:in 'formatted_file_list'
from /path_to_ruby/lib/ruby/1.9.1/erb.rb:753:in 'eval'
from /path_to_ruby/lib/ruby/1.9.1/erb.rb:753:in 'result'
from /path_to/gems/simplecov-html-0.3.8/lib/simplecov-html.rb:58:in 'formatted_file_list'
from (erb):31:in 'block (2 levels) in format'
from (erb):30:in 'each'
from (erb):30:in 'block in format'
from /path_to_ruby/lib/ruby/1.9.1/erb.rb:753:in 'eval'
from /path_to_ruby/lib/ruby/1.9.1/erb.rb:753:in 'result'
from /path_to/gems/simplecov-html-0.3.8/lib/simplecov-html.rb:24:in 'block in format'
from /path_to/gems/simplecov-html-0.3.8/lib/simplecov-html.rb:23:in 'open'
from /path_to/gems/simplecov-html-0.3.8/lib/simplecov-html.rb:23:in 'format'
from /path_to/gems/simplecov-0.3.6/lib/simplecov/result.rb:47:in 'format!'
from /path_to/gems/simplecov-0.3.6/lib/simplecov/configuration.rb:119:in 'block in at_exit'
from /path_to/gems/simplecov-0.3.6/lib/simplecov.rb:122:in 'call'
from /path_to/gems/simplecov-0.3.6/lib/simplecov.rb:122:in 'block in '

Using the build-in Adapter 'rails' works fine. So I guess something changed in the output formatter for custom Adapters or I missed some syntax change in SimpleCov.adapters.define.

Thanks for any hint & congrats to a very good looking and useful gem,
Fred

Hi,

I can't seem to get coverage results when running tests through textmate but when running rake spec on the command line it works just fine. I'd like to have coverage generated when I do a full run of the tests but can't seem to see what the problem is. The coverage is running I just don't get any results. This is what I have in my spec_helper.sh

require 'simplecov'
SimpleCov.start

In case you want to use the: SimpleCov::Formatter::HTMLFormatter for humans and another formatter for machines.. SimpleCov::Formatter::CSVFormatter for example.

So the SimpleCov process will generate both formats.

As the title says... how do I include test coverage for my routes?

Using the following code setup:

http://pastie.org/private/dj86cs3ynpdkk6gjj2jo3g

And it would appear that no matter what I do, simplecov just doesn't want to mark that line as covered. I'm not exactly sure why, because I don't know how to cover it any more in my current test, so I thought perhaps simplecov isn't playing well with shoulda (or the other way around if you prefer :) ).

Is there anyway to make simplecov work with spork? I can only seem to get test coverage results when I don't have spork running...

https://github.com/timcharper/spork

I was thinking it might be cool to have a "bonus" score that goes over and above the percentage coverage. Basically it would just be the average multiple of times the lines were covered.

For simplicity sake lets say a program has only one relevant line of code. And lets say is was covered. Great so that's 100% coverage. But say that line was run not just once, but 3 times. So it would have a Bonus Multiple of 3.

I think that would add some extra "strength" to the coverage scores. What do you think?

tl;dr:

Summary of results:

I'm on an existing Rails project using Mongoid, Devise, RSpec, and Spork (0.9.0.rc5). Our spec/spec_helper.rb looks like this:

require 'spork'

ENV["RAILS_ENV"] ||= 'test'

Spork.prefork do
  require 'simplecov'
  SimpleCov.start 'rails'

  # trap mongoid
  require 'rails/mongoid'
  Spork.trap_class_method(Rails::Mongoid, :load_models)

  # trap devise
  require 'rails/application'
  Spork.trap_method(Rails::Application, :reload_routes!)

  require File.expand_path("../../config/environment", __FILE__)
  require 'rspec/rails'
  Dir[Rails.root.join("spec/support/**/*.rb")].each {|f| require f}

  DatabaseCleaner.strategy = :truncation

  RSpec.configure do |config|
    config.mock_with :rspec

    config.before(:each) do
      DatabaseCleaner.clean
    end
  end
end

Spork.each_run do
  # reload factories
  Factory.definition_file_paths = [
    File.join(Rails.root, 'spec', 'factories')
  ]
  Factory.find_definitions
end

This configuration will run SimpleCov when Spork is not running, and it produces the expected result. When Spork is running, no coverage message is shown.

If I move the require 'simplecov'; SimpleCov.start 'rails' lines to the beginning of the Spork.each_run block, as SimpleCov's readme suggests, then when Spork is not running, I get this:

Coverage report generated for RSpec to /home/johnf/projects/<elided>/coverage. 0 / 0 LOC (0.0%) covered.

If Spork is running, then instead I get no coverage message, as before.

I have this rails project in which I try simplecov. I had 100% coverage. then I created a dummy class in lib, and put a dummy method into it. ran spec again, the coverage was still 100%. Now I moved the same file into app/model. the coverage changed to 98%. Why didn't the files under lib get counted?

After config simplecov in my Gemfile and spec_helper.rb, i ran my specs with rake spec. i got the coverage report in my project, but there is no items in it.

Env: rails3 + ruby 1.9.2-p0

I believe I can give you 1.8 compatibility, and in fact compatibility with any ruby engine that supports #set_trace_func.

A long time ago I wrote a tool called TracePoint. It should be pretty trivial to provide a result from it that exactly mimics Coverage.result. Would that do the trick? If so I will write it for you.

I just accidentally closed #23 which I opened this morning. ... I don't know how to reopen it. It's a real issue. Please read it.

https://github.com/colszowka/simplecov/issues/closed/#issue/23

Just started using simplecov and as rcov replacement after upgrading our Rails 3 project to Ruby 1.9.2. A coverage report is generated after rake test:units but it only covers several (not all) classes under /lib and none under /models. Running a full rake added one lib class to the report and that's it, still no model or controller coverage.

Unfortunately, I can't upload my app code since its all proprietary company code. I'm using .3.7 with a pretty standard rails structure. I'm running simplecov like so:

SimpleCov.start 'rails' do
#remove chef cookbooks from reports
add_filter "/cookbooks/"
root(File.dirname(FILE) + "/..")
coverage_dir("coverage")
end

SimpleCov.at_exit do
puts "Coverage is #{SimpleCov.result.covered_percent}"
puts "Your coverage has dipped bellow 90%" if SimpleCov.result.covered_percent < 90
SimpleCov.result.format!
end

Both the html report and covered_percent are finding 99 files. The html report is showing 88.33 with no one module under 80.88. The command line is showing 80.44. I did find it a little odd that plugins in the html report is showing up at 100% with no files but I don't think that would affect the numbers like I'm seeing. Could this have something to do with lines vs relevant lines?

How does one exclude a subdir from a group?

add_group "Foo", "lib/foo" - "lib/foo/bar"

so to speak.

We've found that running even non-test related rake tasks seem to generate/modify code coverage. Is there any way to combat this?

Example:

task :foo do
  p "foo"
end

! rake foo --trace
(in /Users/ip/Dev/health-teams)
** Invoke foo (first_time)
** Execute foo
"foo"
Coverage report generated for ...

Hello,

I'm getting empty reports (no files listed; all sections 100% coverage) when running my Rails 3.0.7 app's test suite in parallel with test-loop and SimpleCov 0.4.2. This problem was also described previously in issue #30.

I tried the following ways of invoking SimpleCov in my test suite.

Starting SimpleCov at the top of my test/test_helper.rb in the master process:

require 'simplecov'
SimpleCov.start 'rails'

ENV["RAILS_ENV"] = "test"
require File.expand_path('../../config/environment', __FILE__)
require 'rails/test_help'

Test::Loop.before_each_test.push lambda {
  |test_file, log_file, test_names|

  SimpleCov.root File.dirname(log_file)
  SimpleCov.coverage_dir File.basename(test_file) + '.cov'
}

Starting SimpleCov in a worker process just before it loads and runs a test/**_test.rb file from my Rails app test suite:

ENV["RAILS_ENV"] = "test"
require File.expand_path('../../config/environment', __FILE__)
require 'rails/test_help'

Test::Loop.before_each_test.push lambda {
  |test_file, log_file, test_names|

  require 'simplecov'
  SimpleCov.start 'rails'
  SimpleCov.root File.dirname(log_file)
  SimpleCov.coverage_dir File.basename(test_file) + '.cov'
}

Starting SimpleCov (after configuring its output locations) in a worker process just before it loads and runs a test/**_test.rb file from my Rails app test suite:

ENV["RAILS_ENV"] = "test"
require File.expand_path('../../config/environment', __FILE__)
require 'rails/test_help'

Test::Loop.before_each_test.push lambda {
  |test_file, log_file, test_names|

  require 'simplecov'
  SimpleCov.root File.dirname(log_file)
  SimpleCov.coverage_dir File.basename(test_file) + '.cov'
  SimpleCov.start 'rails'
}

None of these worked; I always get empty reports. Any ideas?

Thanks for your consideration.

Great work on this gem. Is it possible to check code coverage in ERB templates? In a Rails app there might be an "if" condition in a view which is not covered. It would be great if it were somehow possible to check this.

I haven't used this gem, so I may be mistaken, but it looks it's saying you have to add code mentioning simplecov to your code base.

Is it practical to create a simplecov executable and do

simplecov test/my_test.rb

without having to add any code?

Hi,

I have seen that when you run rake test with SimpleCov activated there are two SimpleCov messages:

Coverage report generated for Unit Tests to /Users/fguillen/Develop/Rails/SponsorPay/OfferWall/coverage

and

Coverage report generated for Unit Tests, Functional Tests to /Users/fguillen/Develop/Rails/SponsorPay/OfferWall/coverage

Is one of them overwriting the other one? or what is happening?

Thanks

f.

Files without any coverage are not shown at all. It is important for me to be able to view these files.

As this is not detected by the built-in Coverage module, perhaps an option could be added to search for files in the current folder?

I'll try to create a proof-of-concept for this.

When I run

rake spec cucumber

and check the coverage I'm finding that the cucumber results overwrite the rspec results. This is easy to confirm because I have a couple of classes with 100% rspec coverage but only 50% cucumber coverage. I've tried setting

SimpleCov.use_merging true

but no luck. My understanding is that they should be merged by default, is this correct? If not how's it done?

I'm using ruby 1.9.2p136, rails 3.0.5 and simplecov 0.4.1

Hi,
I don't think this is technically a bug but I have no idea how to change the behavior, I organize my files like this:

project_root
  specs
    spec_helper.rb
      sevices
        m1_spec.rb
        m2_spec.rb

I execute my test with a simple:

bacon specs/**/*_spec.rb

and common.rb contains the simplecov start code, my problem is that like that simplecov (or the coverage module, no idea) will consider the specs folder as the root folder and display every path relative to it.

Is there a way to tells it what is the real root folder ?
I already have a root configuration directive in the simplecov block:

root( File.expand_path('../../', __FILE__) )

which sets the root to the top folder (all the requires in my code uses absolute expanded path).

All my path are shown like that in the report:

./specs/../lib/migration_helpers.rb
./specs/../services/dhcp_link_svc.rb