Category: tools

Techniques for Exhaustively Testing a Rails App

I subscribe to dozens of tech blogs (including but not limited to Ruby and Rails), and although I’ve seen quite a lot of commentary about TDD and BDD, I’m not sold on either of them yet. TDD is interesting, but BDD seems like a waste of time. But I am completely sold on automated testing in general. I write lots and lots of tests and make sure using rcov that my tests cover all of the code.

Getting 100% coverage isn’t easy. In general it means you definitely can’t just write a single test case per method and declare victory when it passes. When the number of possible combinations of inputs (method arguments and/or mock objects) and expected outputs (return values, exceptions, and side effects) becomes large, then the potential for copy-and-paste errors in your test code becomes large, and legibility becomes an issue. This is the point at which I find the recent fascination with writing tests in a near-natural-language DSL to be a distraction. It’s orthogonal to the problem I’m dealing with, which is how to comprehensively test the code. In other words, the problem is not making a small number of tests more readable, but concisely expressing a large number of input/output combinations and a large number of different tests, and making it all readable.

I assume that there are others working on this problem too, so I’ll describe some of the things I’ve come up with, and hopefully if you have any good ideas you’ll post them in the comments section.

Continue reading “Techniques for Exhaustively Testing a Rails App”

Rakefile snippet to run Rails’ rake:stats in a plain old gem

I’ve been moving a bunch of utility code out of a medium sized project (>10KLOC) to make it easier to test. I started by trying to make a set of plugins, but inter-plugin dependency management is basically nonexistent, and now that Rails supports explicit Gem dependencies, I decided to make them gems.

I’m happy I chose to make them gems, but I miss some of the stuff you get in a Rails project. In particular I wanted ‘rake:stats’, so I can update my estimation spreadsheet which is now almost 9 months old. I need the stats for each gem in addition to the main Rails project, in order to compare this to prior figures from the Rails project before I split it up.

So, here is the Rakefile snippet that I added, which adds the rake:stats task into a regular gem. If you don’t have the Rails gem installed, it will fail gracefully, without breaking your whole gem. So you need not make your teeny little gem depend on all of Rails being installed on every machine where your gem needs to go. Just install Rails whereever you want to run stats, which is probably already the case on your development machine.
Continue reading “Rakefile snippet to run Rails’ rake:stats in a plain old gem”

Making Rcov measure your whole Rails app, even if tests miss entire source files

I’ve seen a few Rake tasks for Rcov that work OK, but which fail in an interesting way (if you care about coverage): they give your coverage metrics an unexpected boost if you have 0% coverage in one or more source files.

Huh? Exactly. If you have 500 source files, and your test suite only requires one of them, then you get a free ride on those 499 files that have 0% coverage. Theoretically you could get 100% coverage in your report even though 499 source files are not touched at all. D’oh!
Continue reading “Making Rcov measure your whole Rails app, even if tests miss entire source files”

Thoughts about using Git for closed source projects

Git is getting a lot of press in the open source world lately, but hasn’t got much traction in the closed source corporate development world. There’s a reason for this, and it’s more than conservatism on the part of the corporate developers. Git (or any DVCS, really) embodies a development culture that isn’t very enterprise-y.
Continue reading “Thoughts about using Git for closed source projects”