Bug-free code is hard to come by, but we strive to do the best we can. The most important part of this is automated testing, and code coverage metrics are a commonly used indicator of the quality of tests. However, code coverage is a misleading metric: Having “Good” code coverge in your tests isn’t really an indicator of good testing, but having “Bad” code coverage definitely means that your tests are insufficient.

In Ruby (and most other common languages), it’s possible to get “C0” code coverage metrics. All this means is that a particular line of code was executed. It seems like this would be the holy grail of testing, but it’s anything but that. Heres a simple psuedo-ruby example of something that would not be adequately checked even with 100% C0 coverage:

 def simple_check?   true end def my_great_function   simple_check? or (foo? and bar? or (baz? not bat?)) end  test "my_great_function should work" do   assert_equal true, my_great_function end

The foo/bar/baz/bat logic never gets looked at in the test. While a contrived example, situations like this come up a lot in code and just because one line gets executed, there could be all sorts of bugs.

A more appropriate use of C0 metrics is to identify code that is “definitely” undertested. If you’re tasked with getting some legacy code that is new to you and isn’t very well tested into shape, running C0 metrics will give you the places you should start looking first.

Other types of coverage include:

• C1 – branch coverage – each branch of each line of code. Some languages support this, but it’s a little tricky. “Good” tests should have ~50% or higher C1 coverage. A 100% C1 metric would mean that the example given above would be properly tested. There aren’t any C1 tools for Ruby yet.

• C2 – path coverage – each path through each method is covered. A very hard metric to collect, and one that requires lots and lots of tests. It’s not recommended to try and improve this coverage metric as it will lead to an un-maintainable mess of test code that is at least an order of magnitude larger than your application code.

To get C0 metrics, we used and loved a little Gem called rails_code_qa with our Ruby 1.8 applications. It runs a handful of tools including rcov which provides C0 code coverage information, but due to changes in Ruby rcov no longer works with Ruby 1.9.

Enter cover_me. This gem gives you the same C0 sort of metrics that rcov does, but it works with Ruby 1.9. Setting it up is very straightforward, and the end result is a simple report (that automatically opens in your browser!) that provides C0 information both in aggregate and for each of your files. cover_me works with all the common Ruby test frameworks, and there isn’t a reason to not include it in all of your projects.

If your C0 coverage is around 100%, your code to test ratio is above 1:1 (We try for 1:1.4, you can get this number for Ruby on Rails projects by running rake stats), and you’ve been doing Test Driven Development all along, chances are pretty good that your code might well tested.

You are writing automated tests for all your code, right?

Highgroove does, and if you’d like us to take on a project for you, we’ll deliver uncommonly bug-free code.