I'm a hockey fan, have been for a while. In being a hockey fan, I've had a number of statistical discussions around the sport. Second only to baseball fans in statsville, hockey fans love their numbers. One major discussion took place around shots on goal. That semi-subjective stat that tells a team how many times the goalie got between them and a goal. I've seen teams put 40 shots on net and lose to a team that got 18 because 3 of the 18 went in and only 1 of the 40. After the game, you lost, but hey...you put 40 shots on that guy! You certainly didn't lose for a lack of trying.
So, over in the software world, I'm now wondering if Code Coverage is our shots on goal stat. You get over beers and start talking with peers and hear...
Bob: "I've got 40% code coverage!"
Terry: "Bah, that's nothing, we hit 55% this afternoon!"
James: "You both suck, we've got 70% coverage and are almost through UAT!"
Here's what's missing in that discussion...Bob is 60% uncovered, Terry is 45% uncovered, and James has 30% of his code waiting to spring something on him in the next few days of UAT. In essence, they've each got a number of shots on goal, but how many are in the net? Are we focused on the wrong side of the equation?
The goal of 100% code coverage is a tough one, unless you're Joe O'Brien. (See: Testing Mandatory, CodeMash 08) In my life in ASP.Net, getting 1% coverage on a code behind file would be worth a round of beers.
But, are we focused on the wrong goal? Clearly, increasing your code coverage is decreasing uncovered code. But, if your goal is 75% coverage and you reach that goal, do you stop? Maybe shifting our focus to code uncoverage will close that gap.
Don't be happy with those 40 shots on goal...
2 comments:
Code coverage is a worthwhile metric, but only if your tests are valid and if your tests are passing. Those two things must be true in order for code coverage to mean anything.
And I firmly believe 100% code coverage is not a worthwhile goal. If you can achieve it, congratulations. But my experience has shown that trying to go from 90% - 100% coverage can take as much time (or more) as getting to 90% in the first place. So the value of going that extra 10% is greatly reduced in most cases.
On our project, we shoot for 100% code coverage on parts of the application (specifically, business logic and validation). Beyond that, we don't dictate whether or not you should have 100% code coverage and we count on people to use their own discretion.
Code coverage is great, but you have to look at the return on that investment. If you unit test a UI class and no other classes are dependent on that UI class and the logic in the UI class isn't that complicated, is it really worth the time to unit test it if there it's unlikely that that code will break?
Post a Comment