jutopia

Cedric makes a fumbling attempt at discrediting agile by associating it with TDD and then trying to snipe TDD.

It was a long time since I grew tired of this debate so I’m very happy that my ex-workmate Marcus Ahnve did a good job defending agile.

I have done exclusively agile projects for almost 5 years now, before that I did a mix of RUP, waterfall and “cowboy style” projects for about as long. These 5 years have taught me the following about agile: Sometimes it works, sometimes it doesn’t. Though it does seem to work a bit more often than other methodologies. Most importantly, it is a way of working that I enjoy much more. Most people I meet at customers where we introduce agile have a much more fulfilling experience, regardless whether they are customers, end-users, testers, developers, project managers and so on.

You know what? This may be the only reason it works a bit better.

All these hierarchies and documents and gates and GANTT diagrams seem to get in the way of the most basic and important thing: people caring about what they do. When people start caring things start to change. Quite dramatically so.

Of course, there needs to be some structure and planning to a project and I guess that is what agile is to me: a set of tools and practices that doesn’t get in the way of people caring about their work.

Well, I’m definitely going. Aren’t you?

Just a great quote I got sent through an internal ThoughtWorks mailing list I thought I would like to share.

The constraints on when agile approaches can be used are primarily
organizational and cultural, not project types. Some organizations
and some contexts are incompatible with agile/lean thinking. When
these organizations eventually come up against a strong competitive
threat, they will fail to meet it unless they change their values and
mindsets. Lean/Agile is at it’s foundation, the fourth industrial
paradigm, the first being Craft Production, Factory Production with
machine tooling, Automation and Taylorism. These come along every
hundred years or so and take a few decades to work through. Each
paradigm includes the preceding one and makes it dramatically more
productive.

There is no need to sell agile except to organizations that want to
survive long term. If they don’t see the threat/opportunity they
cannot succeed with agile or lean nor can they sustain economic
viability in the long run.

Tom Poppendiek the guy behind “Lean Software Development”

Just something to think about.

Humane interfaces and YAGNI does at least superficially seem to be in conflict, but it’s of course all down to how you understand and apply YAGNI.

I think YAGNI needs to be qualified to what context it is applied. In XP “do the simplest thing” simply means that you only write the code that is needed up to that point in time. You do your requirements analysis, divide it up into releases, iterations and stories. For each story you code and design only what is strictly required to implement that story and nothing else. The context in which you do this is usually application development.

XP cannot be fully applied to designing and implementing a class library but you can do something similar. If you’re requirements on that class library is to have a “humane interface” then that is what should control how you implement that class library. So you implement each feature of the “humane interface” in a YAGNI style. YAGNI doesn’t dictate what features you implement rather it dictates how you implement these features. The principle of “humane interfaces” is what dictates what to implement.

The most common way class libraries are implemented in XP projects are through extraction or harvested frameworks in Martin Fowlers terminology. In this case the class library is the result of the refactorings you apply to it while implementing stories. If you follow the principle of “humane interfaces” while doing these refactorings you will end up with a library with a humane interface.

One of the strengths of a process is how well understood it’s applicability is. XP “out of the box” is excellent for doing small to mid-sized application development. It’s less applicable to large scale application development and development (or extraction) of frameworks or class libraries.

At ThoughtWorks we do a lot of large scale development and we have learned the hard way that XP doesn’t scale “out of the box” to these sizes. XP is all about inventing slogans and pronouncable acronyms so clearly we need another one: “Enough Design Up Front” or in short “EDUF”. This is what we try to do at ThoughtWorks, we get a relatively complete understanding of the requirements then we drill down and design an initial architecture for the application. This way we can scale the project relatively fast without too much pain.

The trick is of course to know when to stop, before EDUF becomes BDUF.

Metrics are great. They provide insight into such a complicated thing as a software development project. They can give you input to make decisions and how to tune your processes. But remember…

They are just a simplification. A simple model of something incredibly complicated.

When you measure story points per iteration and hope they should go up they probably will. Unfortunately so will the number of bugs as well as developers focus on delivering functionality rather than quality.

If you measure number of issues in your issue tracker and hope they go down, they probably will. And productivity with them.

If you measure both productivity and bugs then testers might stop using the issue tracker and talk directly to the developers. Of course, this will result in issues being missed and forgotten and making it all the way into production. The number of production issues goes up.

So when you use metrics, be careful. Always understand that behind these numbers are not just individuals. Behind them are a group of individuals with incredibly complicated dynamics and interactions. Choose any number of metrics, it doesn’t matter. You will never be able to get a complete picture of a project using metrics alone.

So be careful what you measure…

…and what you hope for.

Context

You have a very big system with thousands and thousands of tests. Your builds are slowing down more and more and is starting to become a burden for the team. You’re working on new functionality and a large portion of the tests is testing stuff you haven’t touched for months. While you want the safety of not regressing existing functionality you’re actually spending hours every day testing something that’s less likely to break.

Solution

Create two builds: the fast build and the full build. Keep a list of excluded tests and exclude the majority of the tests from the fast build and make it run really fast, around 10 minutes. Only leave in things that are very likely to break, things you’re working on currently and things that are sensitive. Don’t bother too much about classifying tests, just think about whether it’s likely that they will be broken or not. The full build runs all the tests.

Run the fast build as part of every commit, before the commit and as your continuous integration commit-verification build.

Run the full build more seldom. Nightly, a couple of times per day, or on a separate machine running continuously (but much less frequent as it’s obviously much slower). When you do larger refactorings in shared components you should probably also run the full build. Also, the changeset associated with the full build should be all the changes since the last successful full build.

Additionally keep a list of included tests for the fast build, as a test fails in the full build it should be automatically added to this list and start running in the next fast build.

Motivation

Continuous integration is all about feedback. The more rapid the feedback the faster you can solve a problem and move on. The productivity gains of a 10 minute build is in the order of magnitude higher than a one hour build.

When you have a large and slow build you’ve essentially lost the rapid feedback already. To get back some level of rapid feedback you introduce a fast build that will pick up a large portion of the possible failures. You’re not sacrificing any safety at all because the full build is still running and protecting you against regressions.

This safety is further enhanced by automatically adding a recently failed test to the fast build as soon as it fails in the full build. If it has recently failed it can obviously fail again. This also ensures developers fix the test as they shouldn’t check in with a broken fast build.

Extensions

This pattern can be extended to include several tiers of builds. For example, you can have tiers with fast build, full build and then the entire functional test suite. Functional tests with the entire backend systems plugged in and active can often be extremely slow, it’s not uncommon to have suites that run in more than 24 hours.

Joel is writing about how much he likes BDUF and in doing so reveals exactly how little he understands of XP and what XP-practitioners refer to as Big Up Front Design or in short BDUF.

Downloading and reading through the linked PDF which is the specification for his Project Aardvark or Copilot reveals a functional specification which is a perfectly sound document to have in a “true” XP project. At ThoughtWorks very few projects start without a document of at least the same magnitude (disregarding maybe from some smaller details). We split it up into stories, estimate and rate them and run it through some clever Excel spreadsheet employing various statistical analysis. Out comes a pretty good estimate of how long the projects going to take and what it’s going to cost. We call it inception and it’s a perfectly “acceptable” thing to do at an XP project.

It is the next step after that in a typical waterfall project that we skip and go straight to development. This step is what we refer to as BDUF. Here fancy “designers” and “architects” go through the functional specifications and using various CASE tools start designing every single component in the system. Down to each class and method. This is what we call Big Design Up Front and Joel, honestly, not even you would be so crazy to do that.

We wouldn’t say you’re doing BDUF Joel, you’re doing EDUF, Enough Design Up Front. We XPers like to do that too.

I love automatic functional tests. They are the most valuable type of automated tests you can do. They give me confidence to do refactorings without regressions. They give the business confidence to release the software when they feel enough functionality is there. They make it possible to release updates to production systems without having to do a long and expensive manual test cycle.

In short, it allows me to be agile.

So functional tests are the most valuable tests you can have. But they are also the most expensive. They are expensive to write, they are expensive to maintain, they are slow to run. When they fail they are very hard to debug and figure out how to fix them. When you change functionality you need to go through your test suite and determine what tests needs to be changed and what tests needs to be removed. As the system grows this becomes more and more tedious and hard work.

Something I’m struggling very hard with now is how to manage test data for functional tests. In order to write a simple test for, for example, a paging list I need to set up all these different objects in the domain to the right state. When some constraints or rules are changed in the domain I need to go through all this test data setup code to make everything obey the new situation.

As the system grows the problem becomes bigger and bigger. It is very hard to get reuse of the test data setup between tests as each test has special needs on the data it uses. I can’t use something like mock objects to just mock out where the data is read in because this is a functional tests and I want to test as much as possible with each test. The more I test, the more valuable the test is.

I can use something like the Object Mother pattern but it only provides for individual objects. I need something higher level than this, something that can set up scenarios of objects that can be reused between tests and still allow the tests to tune these scenarios to fit their special needs.

I just don’t know how to solve this, it seems to be a significant non-trivial problem of very large functional test suites.

What have other people done out there? What has worked (that I can try) and what has not worked (that I don’t have to try)? I could use some help!