Tim Barcz

I would tend to agree that tests are better to start with.  However, if the developers on the team are new to testing then adding CI would be a faster win.

I agree that unit testing is more important, but to piggyback on Derik's point, the developers may be more accepting to a 'controversial' practice after seeing an easy win provided another agile practice like CI.

Also, in my experience, one of the greatest barriers to getting a team to adopt unit testing is not only getting them to write them but also getting them to actually run them regularly and keep them up to date.  Having a CI server in place that could automate this process forces the devs to pay attention to broken tests and respond to them immediately when they fail, instead of every few days or whenever they remember them.

Nice post,

Jeremy

I would tend to pick CI, because it's (near) instant gratification, which is helpful when trying to sell new methods.  UT is ALWAYS hard to convince people to do, but a continuous build (with a couple UT) is a nice warm fuzzy for managers.  Also, it doesn't take much time or energy to get it up and running, and then you can start building your UT portfolio, which just looks even better in the CI environment.

If I had to choose 'twixt the two, then yes, I'd go with unit testing. But when setting up an environment, I'd add an automated build and CI to the mix first.

It's the first step in creating a sense of collective code ownership. And it is key in breaking developers of the habit of checking in coding just because it "works on my machine".

That said, adding unit testing to the build is the first check-in I do after CI is set up.

CI gives you visual feedback to testing efforts. That can often be used to apply social or political pressure to ensure that testing gets done. Sure, it's not the ideal way but if you can lead a horse to water and he doesn't want to drink, you have to try ulterior methods to encourage the behavior.

I agree that tests are of more value, I'd say only marginally.  Getting multiple developers to work seamlessly without breaking each others changes is difficult.  Having a continuous build is highly valuable even if no tests are in place.

If you're in a shop that doesn't yet practice unit testing, CI is definitely the first step.  Once you have it the value of having unit tests increases and becomes too large to ignore (to the rest of the team).

Testing might be more important, but CI is probably an easier battle to win.  You can get it in place relatively quickly and you get an immediate payoff.  In my situation it was a great momentum builder to start getting other changes made.  Unit tests require more work, more setup and the benefits are not usually as immediate.  Plus, as was mentioned, having the CI server in place allows tests to be added in gradually and run automatically.

Also integrating unit testing into an existing project assumes that the existing code is either in a testable state already or that refactoring for testability would not be way too disruptive.

I think everyone here is saying what I was trying to say.  

I think CI adds value indirectly because build process and unit testing is more visible to developers, and it puts those ideas in their heads.  which in the end, will add more value to the customer than independent developers possibly writing UT on their own.

maybe?  :)

Pingback from  The Inquisitive Coder - Davy Brion’s Blog  » Blog Archive   » The CI Build should be a given

Which one first and which one period are two different questions.  I'd agree that unit testing is more critical if you can only have one.  But, as previously mentioned, they are very complimentary and the goal should be to get both into place.

Good question.  I think it depends on the size of the team, the LENGTH of the build, and assuming we are talking "brownfield" development, how fast you think  you can get things under test.

Assuming it's brownfield, with a LONG build, I'd go "CI", just on the "it's going to save the team N hours/week, right a way", and we can use those N hours to get testing started...

CI first. Always. Never tests first.

Why? Unit tests are pointless if they aren't part of a CI process.  I've seen at least 2 teams with a whole mess of failing tests, but nobody knows because they aren't being run.  Tests have no value unless they are executed.  Unit tests not part of an automated process will eventually stop being executed because it is too much friction otherwise.

Automated build is such a low hanging fruit, we always do that first when coaching.  It's a great introduction into the Agile values such as feedback.  CI is a team enabler, and opens the door to other practices.

@Jimmy Good point!

When this person is coming from what sounds like a fairly mature project (or set of projects) I would say CI first, without a doubt. What use are tests if you can't even provide a build to the customer. Tests provide customer value, yes. But even having full test coverage doesn't have any customer value if you can't even provide a build for them to use (in a timely fashion).

Personally, I'd make sure that I had a build process in place where I could, on demand, provide a deployable instance of the application, and then start adding the tests. If you can't deliver, you don't have any value.

Without a doubt, CI.

CI provides benefits to the entire team immediately (Well, once you get it running anyway).  Even if you have lazy bastards on the team who won't try to fix the build when I break it (you looking at me?), you know the state of the build at all times.

Unit testing requires conscious effort and buy in from the team.  

This isn't to say CI provides more value in the long run than unit tests (glad you didn't tie it to TDD, makes it easier), but you did ask which is better to have first.

I've worked on teams that had really weak unit testing but had CI.  CI's value was strong.

Did I mention I think CI is more important to have first?

"If you remove automated tests from a build server all you have is a dummy machine that compiles code."

Oh, absolutely not.  Imagine a VS solution that has over 100 projects, and is being worked on by a team of over 75 people.  Knowing that you can compile the individual changes in the central build, and know it as soon as possible is of *immense* value.

"If you remove automated tests from a build server all you have is a dummy machine that compiles code."

Oh, absolutely not.  Imagine a VS solution that has over 100 projects, and is being worked on by a team of over 75 people.  Knowing that you can compile the individual changes in the central build, and know it as soon as possible is of *immense* value.

I agree with Jimmy - CI is the first step.  I'm in a large-ish shop where we're trying to introduce responsible design and implementation practices.  In my experience there, unit tests in a code base being worked on my more than one developer are absolutely worthless - without enforcement as a failable build step, their value will quickly erode.

One of the first things I did when I started at this job was to write the following article arguing for CI.

creedcultcode.blogspot.com/.../continuous-integration-and-you.html

I want to agree with Tim, b/c I am at his previous place of employment, and we did have a lot of broken stuff, but...

Since shops generally don't have just one project that could use CI, you only have to set up the CI server once (with config for each project), and you get that benefit across the organization.  

With different projects, you might be setting up unit tests for each.  Granted, you could set up libraries to use, but then that just means that it would take longer to have your unit tests up and running.

CI

It is fast to setup, provides instant visibility to all, provides a sense of ownership ... and without it, you will never know if your tests are working or not anyway without direct developer interaction

That and it takes less than an hour to get CI fully up and running from having a blank Windows Server to a fully building and deploying project

Tim - It really doesn't HAVE to be that hard. You do not need NAnt to run a continuous integration build. If using Cruise Control, you can simply have it run MSBuild on your .sln file.

Or you can use CI Factory.

Or, if you have TFS, you click through a build wizard and check a box to trigger builds on checkins.

Even if it does take more than an hour, it is still going to be a faster by orders of magnitude compared with training a team to write automated unit tests (and testable code).

Pingback from  Dew Drop - August 20, 2008 | Alvin Ashcraft's Morning Dew

Unit testing shouldn't even be open to discussion. When you are doing TDD, you can not have any code without having tests in place.

As a second side, for a true CI you will need to run unit tests as part of your built anyway.

So I concur with you that unit tests are more important even if there is a resistance about it in the team.

CI takes less than an hour ... if it doesn't then:

1) Stop using CC.NET, use TeamCity

2) Don't start with complex build files or NAnt, use the Build Solution task from MSBuild, or even use a batch file for that matter.

Yes I agree CC.Net could well take a few hours... writing complex NAnt scipts can add a few hours ... but even then, half a day spent will pay off within the first week.

That isn't to say don't do unit tests, but if you think CI is hard to get setup and accepted ... unit tests are another world of pain to get people up to speed with.

@Sidar

>>As a second side, for a true CI you will need to run unit tests as part of your built anyway.<<

Totally incorrect ...

CI does not require unit tests, but unit tests almost certainly do require CI

A build process that merges my changes with all other developers, checks the latest version out of source control, ensures it compiles, and deploys it to a UAT server has massive value, regardless of whether you have unit tests.

A unit test suite that is comprehensive cannot be shown, nor proven, to be unless it directly follows the above process. So you are going to waste huge amounts of time every check in by repeating pull updates from SCM, build, manually run all tests, check in - when it could all be done for you ...  and even if you do it manually, where is the visibility to management that your test are running, where is the history of their runs, where are the coverage reports being generated? Where is your guarantee on consistency that all developers will follow the above process?

>> If you work on a small team with a lot of one-off project, of which you might be the only eveloper and you're deploying lots of bugs, then I'd go with unit-tests<<<

You said you spoke to another developer - hence your project has 2 people - hence you should have CI ... if you are writing code at home, sure forget CI, you don't ever merge code anyway so no loss.

The rest of us work in teams where we annoyingly screw up other people's code all the time .... hence CI is the important part - it is also the only part the customer sees.

Unit tests are for the developer - CI is for the business to see what the developers are doing.

But regardless, as it takes an hour (or half a day) to setup - just do it - you spent longer justifying to the developer why tests were more important than to show him how to setup CI ...

@Casey

Read the sentence with "true" in the beginning again. CI does not have to have unit tests, but do you call it a good CI in place then ?

How do you ensure your certain unit is working properly without unit tests ? If you don't ensure, don't you break "make your builds auto validate themselves" rule of CI ?

CI is about integration, building, and deployment for testing ... It has value in a project where no unit tests are present... Adding tests to the build adds more value, but there is no "true" version of CI that must have tests ...

Currently I am busy recovering a project gone wrong ... The first thing was to put in a build server.

From that alone we now have developers encouragedto check in frequently, to make sure they always have a working build, to be able to verify their changes on the automatically deployed UAT server, and we have the business users now checking the project progress every morning, or whenever we say "hey check the site now, we made that change you wanted", and we have the manager and PM able to see changes and progress live as it happens.

We have no unit tests at all.

One gives massive value, is zero cost, and has increased confidence of everyone on the team ... If I had started with unit testing and TDD we still wouldn't have delivered any functionality, as unit testing WELL is something that takes months to return on time invested... TDD takes even longer.

In the long term unit tests pay off, as does training people on how to write them ... In the short term they do not if you have an inexperienced team

CI pays off from hour one