{ by david linsin }

February 02, 2009

Do You Write Unit Tests For Yourself?

A couple of days ago I listened to the stackoverflow podcast with Jeff Atwood and Joel Spolsky. They were talking about when it is appropriate to write unit tests for your code.

"Changing a parameter in some function call from 37 to a 9 is not really gonna fail or introduce a bug"
"It's great if you use TDD for your encryption library or compiler, but that's a very finite number of apps that are like that"
"For a team of good programmers the prescriptions are probably gonna be different than from a team of mediocre programmers"

"With stackoverflow there are very few cases where I feel like: okay I'm gonna sit down and write a unit test and this is gonna result in a measurably better experience."

Of course I cannot quote the whole discussion here, so this might sound very harsh on testing for you. Jeff kind of put their statements into perspective by saying that he is pro testing, but just doesn't do it on stackoverflow. However, I still have this feeling that they turned their backs on unit testing too easily.

First off, let me tell you that I love unit tests. I write a lot of them and I care about code coverage quite a bit. You might even say that I am a little religious or dogmatic about it. I only have good experience with unit testing, so I tend to believe that I am not that far off. On my current project we are about 8 people working on our code base, so you can imagine there is a lot going on. It's easy to break something and believe me we are breaking stuff all the time. Although each developer has his domain, you are constantly working with code that was not written by yourself.

Fortunately our test coverage is pretty good. In case a test fails and something seems to be broken, you are being notified by the continuous integration server. Personally that gives me the confidence to do refactorings and changes without the fear of introducing major bugs. The biggest downside, which was also mentioned by Joel, is the "bang for the bucks". You need a big investment to get to this degree of test coverage. You really need to gauge, whether it's worth the time invested to write a unit test for a piece of code.

As mentioned before there is a lot of stuff breaking during development of new features. This is mainly due to not knowing all of the code base. However, I think working with other developers, you are bound to have unit tests. At least for the parts of your code that might be extended or maintained by others. It is worth to invest in a unit test, if you want to make life easier for your fellow developers. At least they can change your code with confidence. You are basically giving them a guideline, which helps to get their work done faster and more reliable. In the long run, I think that will increase the quality of your application and hopefully leaves an impression of your code on your fellow team members.

So what if you are working on a small team or almost alone on your project? Do you still need unit tests? That's when I want to come back to the discussion Joel and Jeff had. The stackoverflow team is quite small. I think Jeff probably thinks, that if you are working alone on a project it might not be worth the investment to write unit tests.

I kind of disagree, because I've been there myself. As the only developer of an application and working on various features at the same time, I found myself breaking my own features over and over again. I started writing unit tests and setup a continuous integration server, which helped me automate my builds and improved quality dramatically. By the time a second developer started working on the project the test coverage was reasonable enough, so he could change my code without the fear of breaking anything.

Maybe I'm just not that good of a developer and I need to write unit tests for myself. However, I do think it's worth the investment, because sooner or later another developer will have to work with your code. Go and make his life a little easier and provide some unit tests.


grandfatha said...

I have just turned to writing unit tests and caring about code coverage. Not only does the testing force me to think about the design, it also brings up nasty corner-cases up front that I would not have thought about, had I just written the feature.

My second experience is that when a project reaches a certain set of size/non-trivialism, you tend to spend a lot of time using a debugger. That is time one could have avoided, if the quality assurance was automated.

But the biggest advantage I get from testing is the confidence in my code. Seeing this green bar just gives me a satisfying feeling of having written solid code that solves the project problem and delivers value.

Producing quality code has become a source of happiness and therefore motivation.

Developer Dude said...

Yes, I find unit tests very valuable. I often write them for non-trivial code (not for POJOs), or to test risky code/changes, or to understand somebody else's code.

I don't write tests before I write the code under test though (TDD), but when I get to a certain point, usually when the code has firmed up a little, I will write unit/integration tests to find the bugs and to use during refactoring or bug fixing.

Almost every time I have written unit tests it has been worthwhile - paying back the ROI.

I have seen unit tests written by other devs and most of the time their tests suck; usually just a smoke test that calls a method or two with expected params that they know already work. It is not often that I see unit tests that actually test the code as it should be tested - they test for success when they should be testing for failure, trying to find bugs. But then most developers don't really understand how to test properly.

You don't have to test everything. Any good test/QA person knows you can't test everything. The best ROI is when you write a test in such a way that it will find a bug.

Write a unit test in a way that it will break the code under test and you will be happy when you find your bugs before others do.

David Linsin said...

Thanks for the feedback guys! I always appreciate "meeting" like-minded developers!

Ollie said...

I can only second that. I listened to the podcast, too, and very quickly realized that the parameter example they gave was really quite a bad one. Of course TDD or the habit to feel a strong need for tests does not tell you to test EVERY single line of code. This would even be as dogmatic as saying you need no tests at all.

Daniel mentioned a very important fact on writing unit tests - the design. By writing unit tests you easily get a feel for how usable in terms of APIs your code is. If you have to write complicated tests theres probably something wrong with the code under test. But what "complex" exactly means is certainly up to each of us. So once again it is a tradeof...

Code coverage: I think coverage in an isolated view does not give information as it might feel. o me it is important to see WHAT code has WHAT coverage. Usually I don't care if there is a getter that was not called by my testing code. Nevertheless I like to use tools like Sonar, that generate code metrics regarding complexity, stability (in the Bob Martin's sense). Thus you find out the classes that are most critical. Either if they abstract a certain complexity or are used by many other classes or even both. Combine this information with the coverage measurement and you really get an overview of where you can gain quality by writing tests, as you should keep the coverage on complex and stable classes as high as possible.

Usually I tend to think that for coverage Pareto applies, too. ;)

David Linsin said...

Ollie > By writing unit tests you easily get a feel for how usable in terms of APIs your code is. If you have to write complicated tests theres probably something wrong with the code under test.

That's one of the reasons I love to write test cases! You have to eat your own dog food. If writing test cases for other people's code you implicitly do an API review.

Ollie > Usually I tend to think that for coverage Pareto applies, too. ;)

The reason why I care about code coverage is confidence! It's giving me somehow quantification that my code has been run before it is being deployed. However, IMHO it is a byproduct of proper unit testing anyways and you shouldn't strive for more code coverage than unit testing gives you.

Blogger said...

BlueHost is ultimately one of the best hosting provider with plans for all of your hosting requirements.


  • mail(dlinsin@gmail.com)
  • jabber(dlinsin@gmail.com)
  • skype(dlinsin)