On the eradication of software defects

I loved Andy Glover's hip comparison of mousetraps to testing tools and mice to defects.

I live in a country that had no mammals, other than a few bats, and no software defects, until man arrived around 1000 years ago. The introduced mammals have had a devastating effect on the native biodiversity. A desire to protect the remaining species has led New Zealand to be world-renowned in the removal of introduced species. I wish I could say the same for software defects.

K.P. Brown and G.H. Sherley, in their paper describing the eradication of possums on Kapiti Island, show the following success rates for different phases of the trapping programme.



The first phase, Commercial Trapping, resulted in a 24% success rate of trapping possums. Once trapping stopped being a commercial proposition, 4 trappers were paid to set up to 1500 traps per night and had a success rate of 0.7%. The final Eradication phase introduced dog teams in addition to the trapping program. At this stage, the trapping success rate was down to 0.007%, and dogs caught the remaining 32 trap-shy possums.

I suspect that similar success rates would be encountered in finding software defects. A good unit testing program, at a cost of 3-4 lines of test code for each line of code under test, might yield a success rate similar to Commercial Trapping. Some of the remaining bugs would be detected in intensive system testing. Many escape undetected into production.

Most development shops don't even make it through the Commercial Trapping phase.

The true cost of the $1 mouse trap is in the time, and cheese, taken to set it.

Agitar changed the equation. At the press of a button, Agitator would set the traps for me hundreds if not thousands of times. This would drive my unit testing past the Commercial Trapping into Intensive Control territory. With a bit more tuning, I could even start dreaming of bug eradication.

My hat goes off to those hardy souls who rid Kapiti Island of possums (and mice, rats, stoats etc.) and to the folks at Agitar who wanted to make it easier for us to eradicate those damn electronic vermin.