`method_missing': No sham defined for name

It’s discussed in the machinist Google Group as well.

It’s because of a wacky hack that’s part of Machinist that overrides Module.name so you can do Sham.name, but ZenTest expects Module.name to do what it does normally.

I have a fix for this.

I sent notahat a pull request via GitHub, so hopefully it’ll be merged by notahat into the official machinist gem, but until that happens you can try mine. I published the gem as JamieFlournoy-machinist to rubygems.org.

Try ‘gem install JamieFlournoy-machinist -v 1.0.6′ and see if it works for you. It does for me with ZenTest 4.2.1, autotest-rails 4.1.0, and a Rails 2.3.5 project.

(my machinist fork on github)

Getting 100% coverage isn’t easy. In general it means you definitely can’t just write a single test case per method and declare victory when it passes. When the number of possible combinations of inputs (method arguments and/or mock objects) and expected outputs (return values, exceptions, and side effects) becomes large, then the potential for copy-and-paste errors in your test code becomes large, and legibility becomes an issue. This is the point at which I find the recent fascination with writing tests in a near-natural-language DSL to be a distraction. It’s orthogonal to the problem I’m dealing with, which is how to comprehensively test the code. In other words, the problem is not making a small number of tests more readable, but concisely expressing a large number of input/output combinations and a large number of different tests, and making it all readable.

I assume that there are others working on this problem too, so I’ll describe some of the things I’ve come up with, and hopefully if you have any good ideas you’ll post them in the comments section.

Test Data Iterators

If you express the set of inputs and outputs as a data structure, such as a Hash of input => expected ouput pairs, or an Array of [input, expected output] Arrays, you can easily do test_data.each do |input,output| ... end and drive tests that way.

(I considered pulling the data itself out of the test code and loading it from a YAML or CSV file or even including a data-only Ruby script, but that seemed like pointless added complexity. If I were working with a professional tester who didn’t want to touch Ruby code, I might change that to make it easier to dump from an Excel spreadsheet into a text file that the tests would use.)

One problem with this approach is that you may have a test_frobnicate method that invokes frobnicate dozens of times. This means that this one test can take a while to run if the tested method takes any amount of time, which is problematic if you just want to re-test that one input/output combo until it works. Your edit/test/debug loop should be ultra fast, and re-testing successful pairs gets in the way of testing the pair you’re not sure about.

More importantly, trying all those combinations in one test method means that if an assertion fails, the failure masks the outcome of the remaining combinations. If there are 25 combinations and combination #2 fails, then a Test::Unit::AssertionFailedError is thrown, and the other 23 of them go unchecked. That’s not ideal, since it can be useful in debugging if you can see that 24/25 failed vs. 1/25 failed.

To solve this, I changed the code so that it iterates over the test data inside the class definition, generating a new test method for each input/output pair:

class_eval do
  test_data_hash.each do |input, output|
    define_method("test_frobnicator_#{input.hash}") do
      check_frobnicator(input, output)
    end
  end
end

That gives you a bunch of test methods that each run the test once with different input and expected output values. Autotest integrates well with this technique, since it will not re-run the test methods which previously passed until after you have fixed all of the ones which failed.

Test Setup Blocks

I also use a with_x_y_z do...end idiom for mock setup when possible. So a test might contain:

def check_index(inputs, expected_outputs)
  if inputs[:logged_in]
    with_fake_login{|user| get :index, inputs[:params] }
  else
    get :index, inputs[:params]
  end

  # ... test outputs against expected output
end

Since this idiom uses blocks, you can nest them to create complicated setup scenarios, pass arguments to the with_blah method to control the behavior of the mock objects, and so on. (You could do this with regular set_up_blah methods but I think this is easier to read.)

Calculated Outputs

Despite their shortcomings, I still use Fixtures (with the PreloadFixtures plugin) and I write unit tests for my models. That means that I can then assume that the models’ behavior is correct in other tests (provided that those unit tests pass). So I try not to put any data that’s already expressed in fixture files into my test method inputs or expected outputs. Instead, I ask for the fixture by its label (widgets(:doodad)) and then use its properties as needed in assertions within the current test. This reduces the size of the input/output data, which makes it easier to read and reduces the amount of effort required to add more combinations later.

Testing Views, Model Associations, and Model Validation

As Bruce Eckel says, If it’s not tested, it’s broken. Or as Peter Drucker said, “What gets measured gets managed.”

I test all of these. I’ve read the arguments of those who say that none of these are necessary, and I disagree. The arugment that testing associations and validations is tantamount to testing Rails itself is just incorrect. Rails’ own unit tests cannot possibly test the validations and associations expressed in every Rails application’s model classes, so Rails application developers still have to do this.

Associations

I can’t tell you how many times I’ve gotten has_one and belongs_to backwards and only found out when ActiveRecord told me that the column didn’t exist, because I told it to look at the table on the wrong side of the association. I feel stupid when I see that I made that mistake, but writing that test takes seconds and once I see the failure it’s easy to fix. Better to catch it and fix it now rather than have a user find it, right?

Validations

The same goes for model validations. Yes, I know that validates_format_of has been tested, but my regexes for username format and email addresses need testing too.

Views

Views should be tested only if you care what your users see.

Somewhere there is probably an organization with no users, where inputs matter but outputs can be anything. Maybe the goal of the project is to burn cash rather than to ship something good. They don’t need to test their views.

I agree that views should be dumbed down as much as possible using helpers, but that’s not enough to assure that they work. If you’re not testing views with automated tests, then how do you test them? With a browser? Every time you change anything, you’re going to exhaustively check every combination of inputs and outputs to make sure that all the dynamic goodies are in the right place? I doubt it. Obviously you do have to check views in a browser to make sure that purely visual aspects are correct, but there are things you can check automatically. You can do quite a lot with assert_select, and for client-side tests, Selenium can do pretty much anything. (It’s probably not worth the effort to have Selenium use the DOM to see that the browser applied CSS in exactly the way that you wanted it to, but if you wanted to, you could.)

I don’t know how to do something similar to coverage testing for views, but a good start is to just check for the effects of everything that the controller stored in instance variables. Maybe those should be displayed as-is in the resulting document, or maybe they should be truncated, or processed by a helper into a different representation.

In any case, you don’t have to test with the same set of inputs as the functional test for the controller; just do white box testing for the minimal set of combinations that should exercise all of the permutations of the view code (which should be a very small number). Testing the helpers separately can cut down on the number of permutations of inputs and outputs also. So if you have a helper that creates a navigation breadcrumb visual element from an array of strings, you could just test that in isolation with 0, 1, 2, and 3 elements, and then check for one version of breadcrumbs when looking for it in a view.

I’m particularly inspired by the claim that Sebastian Delmont of StreetEasy made in the May 31, 2006 episode of the Ruby on Rails Podcast about having no human approval step between a successful test suite run and deployment to production. That’s pretty darn bold, but if you stop and think about it, it is feasible with today’s tools to automate any “white glove test” that a real person would do before approving the release for deployment to production. I’m not that brave yet, but I like the idea of a condition that checks if the changed code has anything to do with a view (ERB, JavaScript, CSS, etc.), and if not, would just rubberstamp it and deploy it. Clearly a change that could require a manual test in multiple browsers wouldn’t be a wise thing to autodeploy, but something like fixing a bug deep in the code shouldn’t require a human to approve it, especially if the autodeploy condition depends on that chunk of code having 100% test coverage following the bug fix.

Your Tips?

I’m always looking for ways to test code with less effort on my part. Please let me know if you have ideas or suggestions.

I’m happy I chose to make them gems, but I miss some of the stuff you get in a Rails project. In particular I wanted ‘rake:stats’, so I can update my estimation spreadsheet which is now almost 9 months old. I need the stats for each gem in addition to the main Rails project, in order to compare this to prior figures from the Rails project before I split it up.

So, here is the Rakefile snippet that I added, which adds the rake:stats task into a regular gem. If you don’t have the Rails gem installed, it will fail gracefully, without breaking your whole gem. So you need not make your teeny little gem depend on all of Rails being installed on every machine where your gem needs to go. Just install Rails whereever you want to run stats, which is probably already the case on your development machine.

Here is a Pastie link, since my blog theme truncates the too-wide text: this same code on Pastie

# stats
begin
  gem 'rails'
  require 'code_statistics'
  namespace :spec do
    desc "Use Rails's rake:stats task for a gem"
    task :statsetup do
      class CodeStatistics
        def calculate_statistics
          @pairs.inject({}) do |stats, pair|
            if 3 == pair.size
              stats[pair.first] = calculate_directory_statistics(pair[1], pair[2]); stats
            else
              stats[pair.first] = calculate_directory_statistics(pair.last); stats
            end
          end
        end
      end
      ::STATS_DIRECTORIES = [['Libraries',   'lib',  /.(sql|rhtml|erb|rb|yml)$/],
                   ['Tests',     'test', /.(sql|rhtml|erb|rb|yml)$/]]
      ::CodeStatistics::TEST_TYPES << "Tests"
    end
  end
  desc "Report code statistics (KLOCs, etc) from the application"
  task :stats => “spec:statsetup” do
    CodeStatistics.new(*STATS_DIRECTORIES).to_s
  end
rescue Gem::LoadError => le
  task :stats do
    raise RuntimeError, “‘rails’ gem not found - you must install it in order to use this task.n”
  end
end

BTW I intend to replace this with a sloccount-based task later. If and when I do that I’ll publish it here.

Autotest.add_hook :initialize do |autotest|
    autotest.sleep=3
    autotest.add_exception(/\\/\.git\\//)
end

require '/Users/jamieflournoy/autotest/sound/sound'
Autotest::Sound.sound_app = '/opt/local/bin/mpg321'
Autotest::Sound.sound_path = "/Users/jamieflournoy/autotest/sound/sound_fx/"

Note that the sound plugin needs a patch:

29c29
<   [:run, :red, :green, :quit, :run_command, :ran_command].each do |hook|
---
>   [:initialize, :red, :green, :quit, :run_command, :ran_command].each do |hook|

You’ll also need to make sure your sound set has an initialize.mp3 instead of a run.mp3.

Huh? Exactly. If you have 500 source files, and your test suite only requires one of them, then you get a free ride on those 499 files that have 0% coverage. Theoretically you could get 100% coverage in your report even though 499 source files are not touched at all. D’oh!

The reason for this is that rcov isn’t responsible for finding all of your source files. It just measures what portion of the files which you loaded were executed. The Rake tasks that people have written just kick off the test code, which also have no need to load all of your application’s source files. They just load whatever they need, and so that’s what rcov is aware of. But that’s not answering the question you thought you were asking, which is “how much of my application is being tested?”

You have to explicitly say that you want to see coverage numbers for all the files your tests need, plus all of the files that your tests did not touch. Put this in your test/test_helper.rb:

# require the entire app if we're running under coverage testing,
# so we measure 0% covered files in the report
coverage_testing_active = defined?(Rcov)
if coverage_testing_active
    all_app_files = Dir.glob('{app,lib}/**/*.rb').grep(
        /^(?!lib\\/(scheduled_tasks|template_optimizer)\\/)/)
    all_app_files.unshift('app/controllers/application.rb')
    all_app_files.each{|rb| require rb}
end

You’ll probably want to customize that regexp to weed out any .rb files that you don’t want to load during the test suite (or which aren’t your own code).

Because of all this require-ing, you might want to apply the snippet in my prior post, which eliminates duplicate required source files if they’re under your RAILS_ROOT. Otherwise you may see evidence of repeated loading of the same source files.

With this snippet in place, you’ll still never see 0% coverage in a file. If there are zero lines of executable code in the file, it will not be listed. But you’ll see really low figures (4% etc.) for files that are loaded but not executed. Hopefully that will help you see where you’ve completely overlooked some code in your tests that users might still be able to get to.

With a DVCS, everybody can be a lone ranger, unless you build a process on top that puts back these features (i.e. a remote repository that everybody pushes to). This is something that many of the introductory documents I’ve seen regarding Git try to downplay — you don’t need a central repository, yippee! Just pass changes around from developer to developer — but while that may thrill the solo coder hacking on his laptop on a long flight, it isn’t music to the ears of a CIO.

In Git’s case, there is no central list of version numbers, and the history can be altered using git-rebase and git-commit --amend. Do you really need to see the precise history of every change ever made to the code? Perhaps you think you do because it gets screwed up so easily in some version control tools, and you need to at least be able to backtrack and manually unsnarl a really big screw-up. In that case, you probably don’t need it, as long as the version control tool can be trusted.

Or maybe you’re guarding against human error, and what you really want is granular remote backups. Just as you can use svnadmin dump and svnadmin load to intentionally truncate the history that the repository knows about, you could use git rebase to squash a bunch of commits together into a single big change, and thereby lose the ability to go back to one of those intermediate steps. So you probably should think about archiving the full-granularity repository before you consolidate the old historical steps for developer convenience reasons. And even if you’re a lone ranger, you should think about using rsync or duplicity to regularly put your local .git repositories someplace safe, or push to a remote git repository that no one else has access to, or both.

One solution to make Git more enterprise-friendly git-svn, which gives you the tool integration and centralness of Subversion, with the local branch/merge and offline commit benefits of Git (which are very very nice). I haven’t used it but I am told that it works very well. It’s also a very low-risk way to try out Git, since you can just check your changes in and go back to using Subversion at any time.

Git is challenging to learn, partly because the command language is not very intuitive, and partly because the folks documenting it so far are obsessed with explaining how radically new and different it is rather than how you probably will want to use it. And for folks reading this documentation, the “look how different your process could be” examples are not necessarily very appealing, since the benefits gained by using a centralized VCS are not addressed.

Still, I think Git has enough hype around it that in a year or two there will be some very slick, usable tools that conceal the complex inner workings that you currently have to understand well in order to be productive. They may also include features that make backups or the setup and shared use of a remote repository easy. So if you’re not convinced that it’s time to migrate Git now, just wait. It will probably come to where you are eventually.

GFS, ZFS:
I’m also interested in distributed filesystems like GFS. I was under the impression until a few months ago that gross ugly NFS was still the state of the art, but apparently somebody has finally addressed that. Correction: GFS is a SAN filesystem, not a distributed filesystem, so it isn’t really comparable with NFS.
ZFS is also interesting, because RAID has turned out to be complicated and expensive, mainly because it has to provide a transparent abstraction of a simple low-level block device. In classic time-sharing situations where many users run many applications on one volume that made more sense than it does now, where many users share one clustered application that has one dominant usage pattern across a large number of servers, each of which has several disks. The smaller set of requirements means that you can customize the clustered storage behavior and get major cost and performance savings, and ZFS seems to be a really good halfway point between application-level 100% custom clustered storage code, and kernel or hardware-level 100% generic RAID. The goal here would be to have dozens of cheap servers with several cheap drives each connected to cheap (motherboard-based) controllers, with HA and parallelization provided in software.

PostgreSQL:
Also I’m becoming a total PostgreSQL weenie and I want to see how they’re presenting themselves, and who’s making closed-source enterprise add-ons, esp. since Sun is increasingly investing in PostgreSQL (in a way that benefits the PostgreSQL community, not just Sun and Sun’s customers). Sun’s wagon is no longer hitched to Oracle on Solaris on SPARC, and the open stack of PostgreSQL on Linux (or Solaris) on x86 is proving to be extremely cost effective without sacrificing features, data integrity, or SMP scalability (as has been the case with MySQL). I think the “who’s the big fast fancy database of choice” tipping point from Oracle->PostgreSQL is already here and the world just hasn’t fully recognized that yet. The only thing left is built-in multimaster clustering (there’s PGCluster but it doesn’t look like a healthy project, and it’s not built in) but I think that’s just a matter of time now, since it’s such a desirable feature for the kind of customers Sun is courting.

Anyway, EMMA is very conservative about coverage estimates. Consider this single line of Java code:

int x = 1; if (x > 1){throw new RuntimeException();}

When you run it, the assignment will execute, as will the comparison, but the block containing throw will never be reached. So, the line is not fully covered. EMMA looks at bytecodes and maps those back to line numbers, and will mark this line as partially covered.

It’s maddening sometimes, but it’s correct; it’s your problem to figure out how to force all those darned IOExceptions that you know can’t ever happen, or to give up and let some apocalyptic error handling code not be covered. I used to shoot for 85-90% coverage, and just live with some uncovered wacky code when I couldn’t find a reasonable way to trigger hideous errors.

(There’s a little voice in my head that says, “Come on, you slacker, you should make it possible to dynamically break the database configuration and remount the root filesystem as read-only during the test suite so that the error handlers can all be 100% covered!” But that little voice never ships anything, he just sits around and writes more and more paranoid code — no Ariane 5 disaster on my watch! — and gets asymptotically closer to 100% coverage.)

Okay, now the Ruby version:

x = 1; raise RuntimeError if x > 1

Same story, prettier code (cuz it’s Ruby; duh, of course it’s prettier): assignment gets executed, condition is executed, no exception is raised. Rcov says it’s 100% covered.

The difference is C0 coverage (Rcov) vs. C1 coverage (EMMA). EMMA only counts a line as 100% covered if all of the bytecodes compiled from it were executed. Rcov counts a line as 100% covered if execution visited that line at all.

(EMMA uses “basic blocks” instead of bytecodes, so in obscure failure cases the reported line coverage figure is even lower, but that doesn’t affect the conditions for what 100% line coverage requires.)

I’m not sure if it’s feasible to get C1 coverage on the regular Ruby VM, but it would be nice to have. In the meantime, just be aware that 100% C0 coverage is easy to attain with rcov, but doesn’t mean you’re done testing everything.

Of course, 100% C1 coverage doesn’t mean you’re done testing everything either, but it’s closer to meaning that than 100% C0 coverage is.

In fact, it’s good to over-test code that’s already covered once. On any given project in the real world, there is code that will succeed with some values and fail with others. (print x/y, for example.) Just testing with one set of values isn’t enough. I like to create an array of inputs and expected outputs and use Array.each to iterate over them and pass them into a block that calls the code under test and then compares the actual outputs to the expected outputs with assert_equal. It means that I’m following DRY (rather than cutting and pasting several lines of test code and then editing a teeny bit of each pasted chunk… eww) but still testing many different ways.

But, even with a C0 coverage tool, at least you know what hasn’t been touched at all, and you can use your big smart programmer brain to think of a few ways to torture that code. Just remember that you probably should be aiming for an unmeasurable way-over-100% imaginary coverage target, beating the heck out of your fanciest code with many, many different inputs and then moving on to the next chunk of uncovered code only when the last chunk is thoroughly proven to not suck.

Good testing!

Well, they’re fixing it. I dunno if it’s gonna be perfect, but at least they’re addressing the problem and seem to understand what the needs of the masses are. So I’m optimistic.

And yeah, at some point I’ll probably check out Git. But not until Subversion irritates me more than it does today. :)

The short version:
- Ruby is fun to program in, as you’ve probably heard
- Rails is over-hyped, but it’s still quite good (definitely not perfect)
- I like the productivity of Ruby on Rails but I wouldn’t call it a silver bullet by any means
- Ruby performance was bad and is getting less bad, and can even be good if you do what the experts say
- The real gem (har har) in the Ruby and Rails space is the community itself

Now for the longer version:

It’s hard for me to be objective since I only started programming in Ruby in February and started using Rails in April, but the language and framework don’t strike me as tedious or kludgy at all. I’m definitely enjoying Ruby and making use of its fancier language features, all of which seem to actually work as advertised without any hideous gotchas when you actually try to do something slick.

I particularly enjoy the way Ruby’s collections allow you to write in the style of a functional language; have a look at the Array, Enumerable, and Hash classes to see what I mean. It’s possible to write extremely compact code that morphs a collection into something very different, but without the inscrutability of Perl.

Compared to my experience with mainstream Java and J2EE, I certainly don’t miss Tomcat, Ant, Struts, or JSP Expression Language. (Instead I’m using Mongrel, Rake, Rails, and ERB.) That’s not Java’s fault, and could be alleviated by just picking better tools and frameworks at the start of a project. I don’t miss Hibernate but I don’t have any gripes about it either.

There are features in Java 1.5 that are helpful in alleviating some of the annoyances of programming in Java, but because these are new features, the vast majority of library code in the wild doesn’t use them (similar to the problems with OO being bolted onto Perl and PHP). Also there are political and business deployment roadblocks to using the latest technology: just because you can download an official release of something and it works perfectly for your code, that doesn’t mean that you necessarily have permission to deploy that technology stack onto production servers. Big organizations take years to officially support new releases, so you may be stuck developing against 3 year old products and APIs.

I’m not trying to troll for Java vs Ruby comment flames here. My point is just that Ruby has had features for a relatively long time that Java is just recently adopting (or simulating), and that’s important for more than just petty bragging rights; a feature that’s unavailable to you for your own org’s political or financial reasons might as well not be there at all. (Or maybe it’s worse, if it makes you bitter about your own work environment: “I have to write all this ugly stupid code because Bob is afraid to upgrade the server… grr.”)

I can remember quite a few times when I painted myself into a corner in Java and had to write lots of type casting / checking / converting code, or had to try and work around single inheritance using composition and lots of boilerplate proxy methods, or really wished java.lang.String weren’t final so I could subclass it, etc. Ruby feels a lot more like Perl or JavaScript than Java when it comes to being able to do whatever you want when you’re sure you know what you’re doing, and being able to write weird fancy code that magically makes everything that calls it become dead simple.

I used to be horrified at the idea of messing with somebody else’s classes (as is done by Hash.slice from will_paginate, which I swiped all by itself since I don’t really need will_paginate), but I’m finding that the apocalypse I feared just isn’t there. If you do something wacky, like add a new method to String, the world doesn’t collapse. Or maybe it does, and you decide not to do it that particular way; just do something else. ActiveSupport adds all kinds of stuff to base classes like String, and it’s OK. Example: ActiveSupport::CoreExtensions::String::Inflections.

I’m still adjusting to using a dynamic language: not uber-checking arguments and state all the way down a method call stack (oh no! is it nil?), not locking down everything so it can only work the way I planned, etc. In most cases, the Ruby equivalent of a NullPointerException is fine. I think this is because the built-in error message includes the type and method you were trying to call, so you can usually just read that and see what you did wrong. The only time you’d want to prevent that is if you bury that error deep in your own code, so that the caller would have no idea of what they did that led to it. I’m finding that I’m OK with writing less paranoid code, because the failure will still happen, and be suitably clear in its cause, without my explicitly checking for it.

An issue with Ruby and Rails is that the official documentation is limited. It’s imperative to have 2 or 3 books on hand at all times, in addition to the RDoc Rails documentation. I own three of the Pragmatic Programmer books: Programming Ruby (the “pickaxe book”), Agile Web Development with Rails, 2nd Edition, and Rails Recipes. Just looking at generated API documentation won’t tell you how you’re supposed to use libraries.

The reverse way of looking at that need for documentation is symbol lookup. Rails does some pre-including of library and framework files before loading the application file you’re working on, so unlike Java, you don’t have a mandatory list of imports at the top that tell you where every class you’re working with came from. There are some symbols and functions you can call from instances of particular classes, and they came from somewhere but it’s not obvious where, at least when you’re starting. A good IDE with “find declaration” could fix this, but initially it’s daunting: “what are *all* the options for this method?” “Where is this method defined, even?” etc. This is where the books are less useful and API documentation can help.

I’ve also subscribed to a couple dozen RSS feeds that help me keep up with what the really slick Ruby and Rails people are using, writing, struggling with, etc. Chief among them is probably Err the Blog. The Ruby on Rails podcast is helpful also.

As for productivity, here are 4 specific time-savers I’ve found:

1) TextMate, which is very slick. There are Windows equivalents, though from what I’ve heard, they’re not 100% as good. Point is, a text editor that understands Ruby is critical, and if it helps you jump around in a big tree of files, that’s very helpful.

I’m an Emacs devotee, and TextMate is far better for Ruby and Rails. It’s hard to explain, but let’s just say that TextMate is much deeper and more featureful than it appears at first glance.

I looked at NetBeans which is pretty close to being a very smart Ruby & Rails IDE, but it was still suffering an identity crisis (am I a Java IDE? or a multi-language thing?) that cluttered the UI badly. This is a hot area of innovation right now so any week now TextMate might be eclipsed (or Eclipsed, har har) by something more full-featured. There are a few other options but I didn’t see anything that led me to believe a multi-hour or multi-day eval would be worth it.

2) Locomotive. It’s like MAMP or WAMP or any of those all-in-one application stack installers, except it gives you a working Rails environment. I set up PostgreSQL separately thru MacPorts (a cousin of the FreeBSD ports project), which was somewhat tedious but decently documented.

Getting all the Ruby and Rails stuff compiled from source is no picnic, and frankly unnecessary. On Ubuntu there’s a gotcha: the Ubuntu people and the RubyGems people can’t agree on a file layout, so Ubuntu’s rubygems package sucks. It’s better to install Ruby using apt-get and then install gems manually from the tarball.

3) Rails Plugins. There’s a lot of stuff being built on top of Rails and then improved day by day, and svn-over-WebDAV is the popular sync protocol. There’s a tool called Piston that helps you manage this. Anyway there’s a lot of stuff that is hard to do and commonly needed which is available in plugin form.

4) ZenTest, and more specifically, Autotest. It watches your source tree and runs only the tests in your test suite that need to be run based on what has changed; it’s the dynamic-language version of the incremental compilation that’s present in any good Java IDE.