The coder is the one on the hook for long nights, weekends, and stress-related health problems if the estimates suck. It’s in your interest to exert as much control over estimation and scheduling as you can. If you’re not making the estimate, someone is making it for you.

If you think about it, you’re opting out of an ongoing salary negotiation process. You’re going to work more for the same amount of money, and worse yet, you’re going to have to squeeze that work into the same amount of time, under stressful conditions. Why would you do that?

McConnell’s “Software Estimation” book addresses this problem of wildly inaccurate early estimates quite well. I highly recommend it. In a nutshell, estimates need to carry error bars, so early estimates would be given but with a disclaimer of +/- 100% (or more!), and later estimates get closer to being correct.

You will have to manage expectations, though. Human nature leads to the shortest end of the date range becoming the remembered end date, the guess being remembered as a promise, and the latest feature set being remembered as if it were the original feature set.

This is a big reason to hire a good project manager: they’re there to facilitate the change control process, or planning game, or whatever you call it in your organization when the paying folks want more stuff, and the PM reminds them that need to pay more and wait longer in order to get more out of the dev team. A good project manager can make a huge difference to the stress level and likelihood of success of a project.

It’s psychologically icky for developers to have to participate in the estimation and scheduling process — “eww, it’s a non deterministic domain with lots of ill-defined inputs!” — and I’ve seen some managers play head games like “Oh, you need more time? I thought you were really good at programming… I guess not…” to try and manipulate developers into working nights and weekends out of a sense of guilt or wounded pride. But you need to suck it up and keep providing status updates and reminding people how the process works, because you-the-programmer are the expert on the nature of programming.

Yes, it’s more like Lewis and Clark or the Apollo project than like building a house, and there are just going to be a lot of surprises, but you can still provide a framework and exert a tremendous amount of control over how the team will operate. Things like whether you plan to build a technology proof-of-concept to make sure your architecture doesn’t suck, or whether you intend to spend lots of time writing automated tests and doing daily code reviews in order to control that horrible “bug fixing” phase toward the end are in your hands if you participate in the estimation process. You’re being offered an opportunity to make sure that all those important tasks like performance testing and cross-browser testing and writing an installation guide and an operations manual for the sysadmin get done. It doesn’t matter if the estimates for those tasks are completely wrong at first – what matters is that they’re on the list.

And really, the largest battle you’ll probably have to fight is the one over scope creep: we can’t launch until it has all the original features plus all these additional changes and new features perfectly working, but we need to launch tomorrow. That’s also human nature, and you need to train people to release early and often. Agile methodologies can help here, if you can get your software working and stable from a very early point in the project, and keep it that way. Then you probably have a launchable product very early, and you can just push highly-desired features into post-1.0 releases.

In any case, a list with a bunch of crappy numbers and low confidence factors is a lot better than no list, or a one-item list with a crappy number and an implied 100% confidence factor.

With a DVCS, everybody can be a lone ranger, unless you build a process on top that puts back these features (i.e. a remote repository that everybody pushes to). This is something that many of the introductory documents I’ve seen regarding Git try to downplay — you don’t need a central repository, yippee! Just pass changes around from developer to developer — but while that may thrill the solo coder hacking on his laptop on a long flight, it isn’t music to the ears of a CIO.

In Git’s case, there is no central list of version numbers, and the history can be altered using git-rebase and git-commit --amend. Do you really need to see the precise history of every change ever made to the code? Perhaps you think you do because it gets screwed up so easily in some version control tools, and you need to at least be able to backtrack and manually unsnarl a really big screw-up. In that case, you probably don’t need it, as long as the version control tool can be trusted.

Or maybe you’re guarding against human error, and what you really want is granular remote backups. Just as you can use svnadmin dump and svnadmin load to intentionally truncate the history that the repository knows about, you could use git rebase to squash a bunch of commits together into a single big change, and thereby lose the ability to go back to one of those intermediate steps. So you probably should think about archiving the full-granularity repository before you consolidate the old historical steps for developer convenience reasons. And even if you’re a lone ranger, you should think about using rsync or duplicity to regularly put your local .git repositories someplace safe, or push to a remote git repository that no one else has access to, or both.

One solution to make Git more enterprise-friendly git-svn, which gives you the tool integration and centralness of Subversion, with the local branch/merge and offline commit benefits of Git (which are very very nice). I haven’t used it but I am told that it works very well. It’s also a very low-risk way to try out Git, since you can just check your changes in and go back to using Subversion at any time.

Git is challenging to learn, partly because the command language is not very intuitive, and partly because the folks documenting it so far are obsessed with explaining how radically new and different it is rather than how you probably will want to use it. And for folks reading this documentation, the “look how different your process could be” examples are not necessarily very appealing, since the benefits gained by using a centralized VCS are not addressed.

Still, I think Git has enough hype around it that in a year or two there will be some very slick, usable tools that conceal the complex inner workings that you currently have to understand well in order to be productive. They may also include features that make backups or the setup and shared use of a remote repository easy. So if you’re not convinced that it’s time to migrate Git now, just wait. It will probably come to where you are eventually.

Check out the 4th message in the thread, with the big code sample.

• Table based HTML layout, and no CSS at all? Check. Heck, the table width % values don’t even add up to 100%.
• SQL in the JSP? Check.
• Making a new JDBC connection for each page view, instead of using a connection pool? Check.
• Unescaped strings in the SQL? Check. (Not strings coming from the browser in this particular JSP page, but you don’t know where those strings originate. Why wouldn’t you escape it just in case?)
• Failing to use a prepared statement? Check. (That would also solve the escaping problem.)
• Using a string literal in a SQL or command line context, so you can’t log it beforehand? Check. Even better, the code makes a query string first, prints it (commented out), and then uses a different string in the actual query. Nice!
• Using the JdbcOdbc driver? Check. (From Sun’s JDBC Basics: The JDBC-ODBC Bridge driver provided with JDBC is recommended only for development and testing, or when no other alternative is available.) I’m guessing that the use of ODBC here is the only reason why the database username and password aren’t embedded in the code sample and posted for all to see.
• Empty exception catch block? Check.

I’d have a hard time coming up with a fake example of bad code that was worse.

But wait, what else has this person asked about?

No way. Yes! Error in Socket and File Writing!
The post includes the router’s username and password, and its configuration including:

• Its IP address, and all of the routes it contains, and all of its interfaces and where they go
• A couple of other passwords stored in the router
• A crypto key that appears to be to a VPN (looks like a pre-shared key, meaning not a public key but one that must be kept secret)!

I’m not gonna say “you get what you pay for” since open source software has served me very well, but I will say that you get what you bargain for. If your bargain includes not looking at the work product of the people you hire, which is to say, hiring the cheapest people available and not supervising them, you’re not going to be happy with what you get.

Of course, this could have been written by a U.S. citizen who works in a cube on-site and makes $200/hour. Point is, hire carefully, and supervise your workers. It seems simple when put that way, but it’s amazing how often companies are willing to hire software subcontractors carelessly (solely on price?) and then pay little or no attention to the resulting work, when the arrangement involves offshore outsourcing.

By the way, the IP addresses in the original post (with the awful code sample) are listed next to the name “Areva”, implying that this code is part of a project for Areva. Who is Areva? They make nuclear power plants. Sweet dreams!

Josh’s Rules (of Database Contracting)

Having worked for a few companies that sold themselves as “a magic box that you put millions of dollars into, and great code comes out a few weeks later”, I know that this is a serious misconception. Subcontracting is fraught with peril. Offshore subcontracting is fraught with more peril, but it costs less per hour. In both cases, the peril is avoidable, but avoiding it requires that you manage the relationship carefully.

I’ve worked with offshore teams on a couple of occasions, and in one case I was fortunate enough to get sent overseas to work with the team in their own offices. I think I probably have more direct experience with offshoring than most developers or technical project managers, and I’ve seen how offshored projects can go awry, so I thought I’d share some tips for those of you considering offshoring a software project, or those involved with such a situation already.

One thing to think about first is whether you’ve really looked for folks locally. Pretty much everywhere I’ve worked, the management has been vehemently opposed to posting a public job listing. Simply spending $75 or less on Craigslist was too much to spend for someone who’d probably be paid about $75/hr. Seriously. I’ve basically been limited to finding people I know, from past work or a mailing list or LinkedIn, and if nobody is available, panic! We need more people but there’s noooobody out there! Really? Have you tried?

A second question to ask is whether you could use a part-time (fixed # of hours/wk) or freelance person, and if they have to work in the office or not. If you’re about to hire someone thousands of miles away in a different time zone, you might want to make sure there’s not someone in town who just doesn’t feel like commuting but who’ll answer the phone during your business hours, or if there’s someone with a kid to take care of, who would be happy to work their butt off 5 hours a day in the office, but who needs the rest of their day to spend with the kid. Remember, you’re considering adopting a serious amount of communication overhead in order to get someone to work for you; you might do better to just find a 1/2 time person who is in the next room for a big chunk of each day.

Anyway, let’s assume you’ve exhausted those options and you’re going to definitely hire some folks in a foreign country far away.

Here are some questions to ask about the specific situation you’re evaluating, some of which are similar to things you’d want to ask about non-offshored situations.

Can you fire the developer without firing the project manager? Or are they a team that you have to fire as a unit, and lose all institutional knowledge? This gets even worse as the team gets bigger. Can you afford to can the whole development organization if you don’t like the way things are going? In other words, are you clear on whether you’re hiring a PM and having them hire developers in turn, vs. hiring a company that will act as a subcontractor?

Will the project manager fire the developer and find someone better, if the PM determines that the developer is not working out? Or will they try to cover up for it, or not worry about it since productivity is not their incentive, but keeping billable hours high is? Bear in mind that you may be paying the PM’s company one rate and that the developers may be paid 1/2 of that, and both you and the developers may see that as a really desirable hourly wage. If you think the PM works for you, then the developers should be paid by you, not by the PM, or at least you and the developers should have agreed on what the PM will pay them. Skimming contractor wages is common practice in the US, too, and depending on how extreme it is, it can create serious incentives for the PM to find new and inventive ways to cheat you.

Will the team’s primary language be the primary language used in your source code, and if so, will that limit your options if you decide to fire them and hire someone else? This isn’t a big issue for India, China, and Russia, but for some languages, you might have a very hard time finding another team that can read and understand the source code. You might also find that it’s impossible to integrate a local developer into your team because they only speak your language, not the language of the offshore team.

Are you sure they’re only working on your project, full time, and billing you their actual hours? Again, 150% of a low wage for you may not be a lot of money, but to them it might be livin’ large, and worth the risk of getting caught padding hours. As far as my moderate research can tell, there’s no systematic, reliable, direct way to measure programmer productivity (not that people haven’t been trying for decades to come up with a way to do this), so you pretty much have to have someone on your end working closely with the team who can make a subjective judgement as to whether they really need 12 weeks to do something, or if 8 people are really working on your project all day every day on their end, or if they’re just scamming you.

Do you know that the developer you thought you hired is the same person who is coding for you? This is a textbook US consulting company scam: bring in the rockstars with the stellar resumes for the sales pitch, and then gradually roll them off onto the next new project, backfilling with clueless newbies, at the same billable rate as the rockstars.

Of course, falsified resumes are a problem too, just as they are here. If the company is in another country, they may do things like making up impressive-sounding universities and companies to pad resumes, or just copying someone else’s resume and putting their name on it. I have had a resume sent to me by a colleague who said it sounded strangely familiar. It belonged to a person that we had worked with, and was copied verbatim, with the name changed. I was on a 2 person team with the actual owner of this resume; I’m pretty darn sure the third guy wasn’t sitting in the little office in Mountain View with us unnoticed. In another case, a large offshoring company introduced a team from India and even flew some of them over to meet us. Two of them were supposedly the project technical leads, but in 3 months they never responded to my emails, never checked in code, and never were available for conference calls, and the PM made up excuse after excuse for why they were missing. We received invoices with their hours on them, and I couldn’t get our folks to stop paying for them. I’m convinced that they never worked on our project.

Is the professional culture in the offshore country the same as in yours? For instance, do they work exactly 8 hours a day and then bail, even if they promised they’d get something done today and it’s not done? Will they stay late, work weekends, etc. if you say something is an emergency? Most cultures in the rest of the world consider U.S. and U.K. working hours and work culture to be too long and too intense (and they’re right, but that’s a different issue). A colleague told me a story about an offshore team that was working on fixing a serious production bug which the U.S. team was freaking out about and clearly communicating the level of urgency of, but which didn’t stop the offshore team from quitting at the usual time on Friday and not working again until Monday even though the problem wasn’t resolved. ASAP doesn’t necessarily mean what you think it does, if the person on the other end is not on the same wavelength about work culture as you are. Getting someone to work all night or all weekend may not be negotiable, or it might just cost time and a half, or double, or something like that. Just make sure you have that spelled out explicitly, and if that means you need a “B team” in another time zone, or a local developer resource, so be it.

My favorite model for offshoring basically eliminates the subcontracting agreement, making the PM and developers (and anybody else you want to hire offshore) part of a wholly owned subsidiary company in their country. Instead of treating them as a unit, treat them as individuals working for a unit that you own, so you can give raises individually, replace the manager if necessary, and generally so that you can keep working with your subsidiary continuously no matter what happens. This means that you need to be pretty hands-on with managing them, but you have to do that no matter what.

Hope this helps!