I found a good starting point at geokit stubbing for faster tests. I also wanted a way to stub a geocoding failure, and a way to prevent any unit tests from using the real geocoding web service.

Here’s how I did it.

In test/test_helper.rb, add this:

class ActiveSupport::TestCase
  def setup
    GeoKit::Geocoders::MultiGeocoder.stubs(:geocode).raises(RuntimeError,
      'Use mock_geocoding_success! or mock_geocoding_failure! in your test')
  end

  def mock_geocoding_success!
    geocode_payload = GeoKit::GeoLoc.new(:lat => 123.456, :lng => 123.456)
    geocode_payload.success = true
    GeoKit::Geocoders::MultiGeocoder.expects(:geocode).returns(geocode_payload)
  end

  def mock_geocoding_failure!
    geocode_payload = GeoKit::GeoLoc.new
    geocode_payload.success = false
    GeoKit::Geocoders::MultiGeocoder.expects(:geocode).returns(geocode_payload)
  end
end

What this does is to force you to choose either one of those mock_geocoding methods before you call the geocode method. To me this seems like a good idea since the integration tests that exercise the full application stack should probably be written using Cucumber and Webrat (which is what I’m using).

You will probably want to merge my one-line setup method into your existing setup code in test_helper, if any. Also note that this uses Mocha for mocking.

Pair Programming is intense, mentally and physically. You need to take breaks, stretch, walk around, and hopefully go outside for sunshine and fresh air. Even so, 8 hours of solid pair programming is a very tiring day. That much pairing time may be appropriate now and then, but it isn’t physically sustainable.

Also, although a side benefit of working in a pair is that it helps you stay focused on work instead of goofing off, there are other things you need to do while you’re at work. You may need to make personal calls, shop for a keyboard or a bike, read about some new technology that you might use on the project, or hack up a technology spike. Those activities require some personal space, and most of them require exclusive solo use of a computer. Without a certain amount of privacy and a small amount of storage for personal items, developers can feel dehumanized and unattached to the workplace, like a guest at their own job.

One option is to expect every developer to bring a personal laptop to work with them. If you do this, you will need to provide some physically secure place for them to put their laptop when they are not using it, such as a small locker. A bit of extra desk space to the side of the pairing station is required if a small separate desk is not possible.

Another option is to provide a small number of inexpensive, low-end computers (such as a 3 or 4 year old, semi-obsolete computer that used to be somebody’s main work computer) to act as shared “personal time” workstations. These can be laptops (in which case they need at minimum a bit of available desk space next to the pairing station) or desktops (in which case a separate desk is required).

In either case, it’s necessary to have a private area that can act as a phone booth, and some long term storage available for personal items. A locking drawer unit on casters is good enough for a developer to have a sense of personal space. A tiny conference room or storage room that can accomodate a single occupant can provide enough privacy for personal calls.

There is a fundamental tension in software development between delivering something quickly now, and being able to deliver something quickly later. Over time, quick and dirty hacks pile up, and code becomes difficult to work with.

Even if you practice test-first coding, your tests may be badly written, and the code can be badly written, even if they produce the correct answer.

Examples include:
- The test was copied and pasted from an existing test and then hacked, but you didn’t bother to DRY up the duplication.
- The API doesn’t lend itself to pre-test setup because you wrote the code before the tests or failed to refactor the API when you found you had to write ugly test code in order to test it, and so your tests incorporate large chunks of setup code.
- The setup for a test is slow because you have to load a lot of data or do a lengthy computation before you can check to see if the result is correct.

All of these mean that at some point you cut corners on refactoring. Ugly test setup code isn’t The Simplest Thing That Could Possibly Work.

Having a fast unit test suite is a technical requirement that you may have forgotten to build into the test suite. Your unit test suite should fail if all of the individual tests pass but it takes more than 30 seconds total. If it takes longer, you don’t get to see if your recent changes worked, you rapidly exit the Flow state, and productivity plummets.

(Note: The integration test suite, which should be running on a separate continuous integration server, can take a lot longer since it’s not in the innermost loop of the Test Driven Development process. Just don’t mix slow-running full-stack integration and acceptance tests in with your fast, isolated unit tests.)

Keep in mind that it’s okay to refactor code that satisfies customer requirements, but is hard to test with a reasonable amount of setup, or takes too long to test.

Accumulated technical debt has a three-pronged attack on team productivity:
- Features take several times longer to implement than they would with well designed, clean code.
- It becomes harder to estimate how long a seemingly simple feature will take to implement.
- Good programmers get stressed out and discouraged because they know they are performing far below their ability, and they quit. Bad programmers who are used to producing slow, buggy code while missing deadlines will hang around.

…and all of these are true even if you are doing test-first development!

My own mistake in this area has been to allow too much input into the ebb and flow of source code entropy by non-programmers. Managers percieve code quality as if it were a programmer luxury like a foosball table or a team party, that can be taken away when the schedule is tight. It isn’t.

In short, just don’t offer to do shoddy work now in exchange for a repair period later. You’re obligated to keep the code (including tests!) in excellent condition, as a matter of professional competence. You achieve this not by Gold Plating (satisfying your ego by trying to predict features that will be needed and building them in now; see YAGNI), but by keeping the code simple and clear.

Maybe there’s an exceptional case where there’s a super important demo deadline to a Gold Owner, and you need to hack something in as fast as possible by Tuesday. That must be followed by a symmetrically urgent cleanup effort starting on Wednesday. Be firm. The longer you wait to clean it up, the harder it will be: it’s a lot easier to stand firm and insist on a day of cleanup than a month of cleanup.

Non-programmers will never be in a position to realize that this week we should just stop implementing new features and Sharpen the Saw. So, don’t let a week of cleanup work pile up. Just explain that the feature is barely working and demoable, but that it isn’t finished yet. Then clean it up, and declare it done when it really is done. If you have to start a half dozen user stories and then leave them all “in progress” while you do the demo, and then clean all of them up because they’re all in desperate need of additional tests and refactoring, then do that. Whatever you do, don’t create a pile of Engineering Tasks for “go back and write tests for [feature]” and “go back and refactor [feature]“. Those tasks will never get done.

If the demand for emergency demo hackery happens often, there’s something seriously wrong with your scheduling and estimation process: why are you expected to drastically outperform your effort estimate? If the feature was so important, why wasn’t it prioritized high enough to be ready for the demo without the desperate hackery?

Remember, the biggest broken promise that developers hear is “we’ll make time to clean this up later.” And usually it’s a promise you’ve made to yourself.

This is a serious problem. Your project exists to serve somebody, and your success is directly proportional to understanding what they want, so that you can build it. You need as much communication bandwidth between the programmers and those people as possible.

If you have one customer and they can’t come to you, consider going to them. Can you co-locate on their premises? Can a fragment of your development team visit the customer for a few days of intensive collaboration on requirements? The quality of information you get from a customer will probably be better if the developers go to where the customer is, due to “knowledge accidents”. The customer may be distracted in their own office by their day to day work, but to the extent that the developers can witness and understand the customer’s daily life, this can actually be a good thing.

However, many software companies have more than one customer, or are trying to build something for multiple prospective customers. They may not consider your product important enough to dedicate someone to be your onsite customer, nor to let you colocate your team on their site.

One option is to choose a single customer and treat them as though their requirements are typical of others. There is a risk that their requirements are atypical, but in exchange you avoid the risk of building something that doesn’t satisfy anyone’s requirements enough for them to buy it.

In any case, when you’re hoping to sell to a large number of customers, you still have to focus on a small number of customers (a particular type of company, a vertical market, etc.) in order to be able to prioritize your efforts.

In the case where no customer is willing to co-locate with you or allow you to co-locate with them, the key is to reduce the number of layers of communication between the customer and development team. The usual substitute for an actual customer is an internal employee acting as a Customer Proxy, who talks to the real customer very often, preferrably in person, preferrably while demoing the current software.

The thing to avoid is an arrangement where multiple layers of opinion and guesswork sit between the customer and the developers. Perhaps sales and marketing folks have gathered wishes from prospective customers, and customer account reps have gathered feedback and wishes from existing customers, and they filter and reword and interpret everything before giving this information to the product manager. Then the product manager synthesizes this into a requirements document and hands this down to the development team, and acts as the Customer, answering requirements questions without consulting a real customer. This is broken. The fix is to have the product manager act as a true Customer Proxy rather than a Mock Customer: the synthesized requirements and their prioritization must be validated with multiple customers to ensure their correctness, and questions about requirements must be relayed to real customers in order to obtain a real answer. And if you can get a customer to come in and visit once in a while (or if you can visit them), by all means do so.

You Aren’t Gonna Need It can lead to some silly situations if you interpret it too pessimistically. The pessimistic (and wrong) interpretation is that you should pretend that the user story you’re implementing is the only one you know about. This equates to doing zero up-front design, because you’re only concerned with whether the design satisfies the user story you’re currently implementing. The price of making this mistake is design churn: each new requirement may incur a large rewrite of existing code. This is obviously not a recipe for productivity.

Since you have a Release Plan which contains a list of User Stories that you’re going to need, you can tell that You Are Gonna Need It. YAGNI defends against programmer-initiated imaginary user requirements that divert attention away from actual documented user requirements.

If you’re coming up with a design for a user story and see that another story coming up soon could influence your design decision, you are in the most informed position compared to whoever estimated the stories when they were being prioritized. You may be able to avoid a lot of wasted effort by adopting a design now that will support both requirements. This is why XP Does Design before writing the tests and implementation.

However, user stories that are far in the future (> 6 months away) are likely to change, and their existence at all is probably a sign of premature planning. Looking even 3 months ahead is probably excessive. Your customer will learn a lot between now and then about what they really need, and may reprioritize or rewrite that future story. They’ve already prioritized it as being less important than the next 3 months of user stories, so you’re taking a risk when you decide to accommodate it in advance.

Note that I’m not talking about reprioritizing user stories. That is not something that engineers are allowed to do outside of the current iteration (only the customer may do that). I’m talking about extracting an Engineering Task from the future story and pulling it into the present iteration, so that the tests you write in the current iteration force you to use a particular design that will better accommodate the future story when you eventually write it.

This does not mean “do the least work possible”. Doing the least possible is not XP, it is hackery, which is the methodology that leads to a Big Ball of Mud.

You do want to do the least work possible in between the Red and Green steps, perhaps even by following the Pair Programming Ping Pong Pattern, a.k.a. The Pair Programming TDD Game.

But once you reach Green, you still need to Refactor Mercilessly. That step obligates you to eliminate duplication (Don’t Repeat Yourself / Once and Only Once / the Zero One Infinity Rule) and to make sure that the code Does The Simplest Thing That Could Possibly Work.

This is very important, because this is how you avoid AddingEpicycles. That is, if you skip the Refactor stage and just keep on doing Red-Green over and over, the code and tests quickly become very hard to work with, because you’re just hacking on top of a design that no longer fits the requirements. Pretty soon you will find that your feature velocity has dropped dramatically, because it’s so darn hard to make a simple change without breaking pre-existing tests.

Over the last six months I’ve had the privilege of working in a software development team that was practicing the closest thing to a full implmentation of XP that I’ve ever seen.

This team was incubated in Pivotal Labs, who are experts at the Ruby on Rails software framework and the XP methodology.

I’ve read a lot about XP over the last ten years, and tried some of the practices when it seemed to make sense for the project I was working on at the time. I’ve tended to feel a mix of curious optimism and curmudgeonly skepticism toward some of the practices: could these ideas really work?

Well, I have tried to think critically in my analysis of fuzzy topics like how much design to do, so that I could figure out not whether something was a universally Great Idea or not, but rather when it would work well and when it wouldn’t. The only way to find out for sure is to try them.

Well, now I’ve worked in an XP shop and I’ve learned a lot about how to make XP work, and about the challenges that a team following an agile methodology faces. Here are the most important things I’ve learned, in the next few posts (broken up by topic to help keep the comments focused).

The coder is the one on the hook for long nights, weekends, and stress-related health problems if the estimates suck. It’s in your interest to exert as much control over estimation and scheduling as you can. If you’re not making the estimate, someone is making it for you.

If you think about it, you’re opting out of an ongoing salary negotiation process. You’re going to work more for the same amount of money, and worse yet, you’re going to have to squeeze that work into the same amount of time, under stressful conditions. Why would you do that?

McConnell’s “Software Estimation” book addresses this problem of wildly inaccurate early estimates quite well. I highly recommend it. In a nutshell, estimates need to carry error bars, so early estimates would be given but with a disclaimer of +/- 100% (or more!), and later estimates get closer to being correct.

You will have to manage expectations, though. Human nature leads to the shortest end of the date range becoming the remembered end date, the guess being remembered as a promise, and the latest feature set being remembered as if it were the original feature set.

This is a big reason to hire a good project manager: they’re there to facilitate the change control process, or planning game, or whatever you call it in your organization when the paying folks want more stuff, and the PM reminds them that need to pay more and wait longer in order to get more out of the dev team. A good project manager can make a huge difference to the stress level and likelihood of success of a project.

It’s psychologically icky for developers to have to participate in the estimation and scheduling process — “eww, it’s a non deterministic domain with lots of ill-defined inputs!” — and I’ve seen some managers play head games like “Oh, you need more time? I thought you were really good at programming… I guess not…” to try and manipulate developers into working nights and weekends out of a sense of guilt or wounded pride. But you need to suck it up and keep providing status updates and reminding people how the process works, because you-the-programmer are the expert on the nature of programming.

Yes, it’s more like Lewis and Clark or the Apollo project than like building a house, and there are just going to be a lot of surprises, but you can still provide a framework and exert a tremendous amount of control over how the team will operate. Things like whether you plan to build a technology proof-of-concept to make sure your architecture doesn’t suck, or whether you intend to spend lots of time writing automated tests and doing daily code reviews in order to control that horrible “bug fixing” phase toward the end are in your hands if you participate in the estimation process. You’re being offered an opportunity to make sure that all those important tasks like performance testing and cross-browser testing and writing an installation guide and an operations manual for the sysadmin get done. It doesn’t matter if the estimates for those tasks are completely wrong at first – what matters is that they’re on the list.

And really, the largest battle you’ll probably have to fight is the one over scope creep: we can’t launch until it has all the original features plus all these additional changes and new features perfectly working, but we need to launch tomorrow. That’s also human nature, and you need to train people to release early and often. Agile methodologies can help here, if you can get your software working and stable from a very early point in the project, and keep it that way. Then you probably have a launchable product very early, and you can just push highly-desired features into post-1.0 releases.

In any case, a list with a bunch of crappy numbers and low confidence factors is a lot better than no list, or a one-item list with a crappy number and an implied 100% confidence factor.

With a DVCS, everybody can be a lone ranger, unless you build a process on top that puts back these features (i.e. a remote repository that everybody pushes to). This is something that many of the introductory documents I’ve seen regarding Git try to downplay — you don’t need a central repository, yippee! Just pass changes around from developer to developer — but while that may thrill the solo coder hacking on his laptop on a long flight, it isn’t music to the ears of a CIO.

In Git’s case, there is no central list of version numbers, and the history can be altered using git-rebase and git-commit --amend. Do you really need to see the precise history of every change ever made to the code? Perhaps you think you do because it gets screwed up so easily in some version control tools, and you need to at least be able to backtrack and manually unsnarl a really big screw-up. In that case, you probably don’t need it, as long as the version control tool can be trusted.

Or maybe you’re guarding against human error, and what you really want is granular remote backups. Just as you can use svnadmin dump and svnadmin load to intentionally truncate the history that the repository knows about, you could use git rebase to squash a bunch of commits together into a single big change, and thereby lose the ability to go back to one of those intermediate steps. So you probably should think about archiving the full-granularity repository before you consolidate the old historical steps for developer convenience reasons. And even if you’re a lone ranger, you should think about using rsync or duplicity to regularly put your local .git repositories someplace safe, or push to a remote git repository that no one else has access to, or both.

One solution to make Git more enterprise-friendly git-svn, which gives you the tool integration and centralness of Subversion, with the local branch/merge and offline commit benefits of Git (which are very very nice). I haven’t used it but I am told that it works very well. It’s also a very low-risk way to try out Git, since you can just check your changes in and go back to using Subversion at any time.

Git is challenging to learn, partly because the command language is not very intuitive, and partly because the folks documenting it so far are obsessed with explaining how radically new and different it is rather than how you probably will want to use it. And for folks reading this documentation, the “look how different your process could be” examples are not necessarily very appealing, since the benefits gained by using a centralized VCS are not addressed.

Still, I think Git has enough hype around it that in a year or two there will be some very slick, usable tools that conceal the complex inner workings that you currently have to understand well in order to be productive. They may also include features that make backups or the setup and shared use of a remote repository easy. So if you’re not convinced that it’s time to migrate Git now, just wait. It will probably come to where you are eventually.