But because the concept of database migrations is such a powerful one, it’s tempting to jam any old change that affects the database into a new migration and run rake db:migrate to make it happen. I’ve been guilty of a bit of this in the past, and I’ve joined some projects that did other ugly things in migrations. In the process I’ve learned the hard way that there are some things you must never do in a migration or they will come back to haunt you later. Here they are.

Antipattern: Require the Database to Exist Already

In other words, the antipattern is for the first migration to depend on some tables and maybe even some data already being in the database.

I know that the original Rails blog video shows DHH using a MySQL admin tool to create the blog database interactively, but really you should be using migrations to create the schema programmatically from scratch.

If you’re already working on a project that didn’t do that, you can run rake db:schema:dump and look at db/schema.rb; it contains code that you can insert into a new migration to create the same schema in your development environment. If you’re using DB features that the design philosophy of ActiveRecord doesn’t agree with, such as triggers, and the schema.rb dump doesn’t include them (or if you just think the migration DSL is ugly and you like SQL DDL better), you can do a mysqldump / pg_dump / whateverdump and wrap a migration around the loading of that SQL file.

If you have a hybrid (you have to start with an old db dump and then migrate it so it becomes current), that’s gross, and you have a couple of options which are both pretty ugly. But they will work, and when you’re done the ugliness will be gone.

You could fight your way back to the oldest schema version by debugging the self.down methods and running rake db:rollback repeatedly until you can create a 00001_starting_db_schema.rb migration, or you could just blow away all the migrations and use the highest schema version for a new migration that contains the output of a current rake db:schema:dump. It depends on how many copies of the database are out there with old schemas that would need to be brought up to date. Clearing out db/migrate and replacing it all with a single migration is cleaner, but if your production database is 5 migrations out of date you obviously can’t do that. But you could collapse it down to the one big-bang migration (as the oldest), plus the 5 pending schema changes. If you do it right, you can just deploy the new code and run rake db:migrate and everything will be fine. If not, well, you were testing it on a backup of the production database, right? :)

Antipattern: Only Work Correctly With the Production Data

What’s wrong with developers just making dumps of the production database and loading them locally?

First of all, it means that all schema changes have to start at the production database and work backwards to developers’ sandboxed development environments. Hopefully this strikes you as a very stupid workflow.

Secondly, maybe your users don’t all want to get a message that says “test message foo bar sdfasdfasd bloopity bloop” when you’re testing your new alert system. Should you really be putting their data (passwords, contact info, etc.) at the mercy of your crummy new code?

You should be able to immediately generate an empty, clean database for development. rake db:drop; rake db:create; rake db:migrate should do this; rake db:reset should have the same result but should be faster since it doesn’t bother with each migration in sequence.

You should also be able to immediately generate any essential base data such as the initial admin user. The SeedFu plugin does a good job here.

If you need some additional fake data to fiddle around with in your development environment, the Populator gem is handy for mass-inserting a bunch of faux data, especially in conjunction with Faker.

Note that the migrations should neither depend on nor contain actual data. They should just change the data model.

Antipattern: Clean Up That Only Works on Production Data

This is really a subset of the previous item but it’s worth considering as a special case.

If you want to fix some data that got slightly corrupted by some bad code that has been replaced, migrations aren’t a terrible way to accomplish that.

It’s not really what migrations are for, and a one-off rake task can do it just as well, but if you really want to, you can get away with it under one condition: you have to make your cleanup migration code succeed even if the database is empty (such as when a developer has just run rake db:reset; rake db:migrate).

Antipattern: Load Data

The populator gem is good for initial, mandatory data. The machinist gem is good for synthetic test data. Delete db/fixtures and everything in it. Fixtures are evil.

Wrap a rake task around the “get my development database ready” concept. This task should start with the “get my empty production database ready” task (or some subset of that which is appropriate for developer use).

If you need to load arbitrary data now and then, write an importer. Do this as a rake task, or a web UI to a bulk data importer feature. Better yet, make a web UI in your admin area which is just a wrapper around the rake task that bulk imports data. Then delegate the bulk importing to your customers so your admins can do real admin work. But don’t load data in a migration.

Antipattern: Use Rails Models in the Migration

Models evolve, but old migrations don’t change (nor should they). So when you wrote a migration that used a model, it used the old version of the model code. Then a year later the model has evolved, and the new validations on first_name and last_name fail because it used to be full_name, and that old migration that hasn’t changed has stopped working. It depended on something that did change, incompatibly.

For rockstar points, in your continuous integration environment you should run rake db:drop; rake db:create; rake db:migrate to make sure that this can never happen.

But if it has already happened, rip out the model code and replace it with Rails DSL code, with execute statements containing raw SQL code, or (if you feel like a Ruby rockstar) declare new, stripped down model classes inside your migration class that will act as stand-ins for the limited needs of the migration. See Migrating with Models for more on how to do this last trick.

Conclusion

You should always be able to do this in every Rails environment that your application has: rake db:drop; rake db:create; rake db:migrate; rake db:reset

At this point you should then be able to run rake db:test:prepare and then rake spec or rake test or whatever and have it work.

If any part of that process fails, you are missing out on the benefits of using Rails migrations.

“HI. You’re using libxml2 version 2.6.16 which is over 4 years old and has plenty of bugs. We suggest that for maximum HTML/XML parsing pleasure, you upgrade your version of libxml2 and re-install nokogiri. If you like using libxml2 version 2.6.16, but don’t like this warning, please define the constant I_KNOW_I_AM_USING_AN_OLD_AND_BUGGY_VERSION_OF_LIBXML2 before requring nokogiri.”

Aaron Kalin figured out how to fix this if you’re installing nokogiri as a system gem, but I want to use Bundler and keep my system gems down to the bare minimum. I figured out how to do this under Bundler 0.8.5.

The gems I have installed via plain old “gem install xxx” are bundler08 -v0.8.5, rake -v0.8.7, and ZenTest -v4.2.1. That’s it. Everything else is under vendor/bundler. I’m using RVM and REE 1.8.7 2009.10 as described in my previous post.

Here is a quick summary of Aaron’s blog post; this will update your libxml and libxslt libraries to reasonably recent versions. (Be my guest if you want to use libxml 2.7.7 and libxslt 1.1.26, which are current as of today, but I didn’t test those myself.)

mkdir -p ~/Downloads/XML
cd ~/Downloads/XML
curl -O ftp://xmlsoft.org/libxml2/libxml2-2.7.3.tar.gz
curl -O ftp://xmlsoft.org/libxml2/libxslt-1.1.24.tar.gz
tar xzf libxml2-2.7.3.tar.gz
tar xzf libxslt-1.1.24.tar.gz
cd ~/Downloads/XML/libxml2-2.7.3
./configure
nice make
sudo make install
cd ~/Downloads/XML/libxslt-1.1.24
./configure --with-libxml-prefix=/usr/local --with-libxml-include-prefix=/usr/local/include --with-libxml-libs-prefix=/usr/local/lib
nice make
sudo make install

Now make this YAML file and call it bundler_build_options_macosx_leopard.yml and put it in your RAILS_ROOT:

nokogiri:
  xml2-include: /usr/local/include/libxml2
  xml2-lib: /usr/local/lib

Then, to build nokogiri so that it uses the new libs you just installed:

gem bundle --build-options bundler_build_options_macosx_leopard.yml

You should now have nokogiri installed under vendor/bundler/ruby/1.8/gems, and when you run your app it should not complain about the version of libxml2 anymore.

Igor P’s solution is correct (just install REE 1.8.7-2009.10), but it took a little fiddling to figure out how to get RVM to use the older version of REE. Here’s how to do it:

cd ~/.rvm/archives
wget -q http://rubyforge.org/frs/download.php/66162/ruby-enterprise-1.8.7-2009.10.tar.gz
rvm install ree-1.8.7-2009.10

I tried a lot of stuff and finally solved it. My solution is on the Ubuntu Forum, here: One or more of the mounts listed in /etc/fstab/ cannot yet be mounted (Karmic).

I had to re-customize these files since I’m not running with 100% default configuration:

/etc/monit/monit
/etc/monit/monitrc
/etc/dovecot/dovecot.conf
/etc/apache2/apache2.conf
/etc/php/apache2/php.ini

I basically did a manual diff side by side in Emacs and copied my changes over into the new config files. Reboot, no problems. Nice.

Put this in config/initializers/respond_with_http_errors.rb :

class ActionController::Base

  protected

  # Display a "400 Bad Request" response based on a determination made by application logic.
  def respond_bad_request(message = "Your request was invalid.", options = {})
    respond_with_error(400, 'Bad Request', message, options)
  end

  # Display a "403 Forbidden" response based on a determination made by application logic.
  def respond_forbidden(message = "You are not authorized to access that document.", options = {})
    respond_with_error(403, 'Forbidden', message, options)
  end

  # Display a "404 Not Found" response based on a determination made by application logic.
  def respond_not_found(message = "The document you requested doesn't exist.", options = {})
    respond_with_error(404, 'Not Found', message, options)
  end

  def respond_with_error(status_code, status_name, message, options)
    flash[:error] = message
    defaults = { :status => status_code }
    respond_to do |format|
      format.html { defaults.merge!(:inline => "<h1>#{status_name}</h1><p><%=h flash[:error] %></p>", :layout => true) }
      format.json { defaults.merge!(:json => "#{status_name}: #{message}".to_json) }
    end
    render defaults.merge(options)
  end

end

No, it’s not. Falling back on 500 Server Error for everything outside of the “happy path” through your code is sloppy coding.

I say this for three reasons:

The 5xx response codes are for server-side errors.

If the code is working properly but the request is wrong (no such document, an invalid request, or insufficient user privileges for the requested operation), the 4xx response codes are more appropriate. You should only return a 5xx series error message when your application is broken.

A generic “oops” page hides the cause of the error from the user.

Maybe the user followed a bad link, that goes to a document that has been deleted. They should see a 404 Not Found response, not a 500 Server Error. This is especially important if the user agent is a web service client actively under development. And remember that you can still customize the response body, so a 404 can show a search or site map to help them find the content that was Not Found, and a 403 might suggest that the user upgrade their subscription so they can see the Forbidden content.

Failing to trap bogus input is a recipe for a security vulnerability.

Rails will detect ridiculous input in many cases, but it doesn’t know about your business rules.

For example, what if someone fires up Firebug and figures out your API, and deletes her ex-boyfriend’s account because you forgot to check that the ID being deleted matches the currently logged in user? Oops.

You know who else got pwn3d because they put their security rules in the client? AOL. Don’t be like AOL. They got pwn3d big time. Make sure your controller handles the “unhappy path” too, and responds appropriately.

You make sure your controller handles these cases by writing tests, and checking that the response is correct. To do this, you need unambiguous errors that clearly explain what the controller didn’t like about the request.

Your controller tests should never expect an exception. Exceptions belong inside the application (such as between a model and a controller), not at the boundary.

Instead, you should be using something like assert_response :bad_request, and providing a useful textual message in the page that will help the user understand what they did wrong.

A nice bonus of using the right HTTP error responses is that your tests are more readable. And since controller tests tend to be really verbose already (mine tend to be ~10x as many LOC as the controller itself), the extra readability is really nice.

Here’s the Cucumber step implementation I wrote.

Put this in a new file called features/step_definitions/ym4r_gm_steps.rb :

# Steps for use with the ym4r_gm plugin

Then /^I should see a Google Map$/i do
  within 'head' do
    ['http://maps.google.com/maps', '/javascripts/ym4r-gm.js'].each do |js_url|
      assert_have_xpath "//script[starts-with(@src, '#{js_url}')]"
    end
  end
  assert_select '#map_div'
end

This assumes that you let the plugin installer put the .js file in the default location, and that you have your Google Map inside #map_div.

I found a good starting point at geokit stubbing for faster tests. I also wanted a way to stub a geocoding failure, and a way to prevent any unit tests from using the real geocoding web service.

Here’s how I did it.

In test/test_helper.rb, add this:

class ActiveSupport::TestCase
  def setup
    GeoKit::Geocoders::MultiGeocoder.stubs(:geocode).raises(RuntimeError,
      'Use mock_geocoding_success! or mock_geocoding_failure! in your test')
  end

  def mock_geocoding_success!
    geocode_payload = GeoKit::GeoLoc.new(:lat => 123.456, :lng => 123.456)
    geocode_payload.success = true
    GeoKit::Geocoders::MultiGeocoder.expects(:geocode).returns(geocode_payload)
  end

  def mock_geocoding_failure!
    geocode_payload = GeoKit::GeoLoc.new
    geocode_payload.success = false
    GeoKit::Geocoders::MultiGeocoder.expects(:geocode).returns(geocode_payload)
  end
end

What this does is to force you to choose either one of those mock_geocoding methods before you call the geocode method. To me this seems like a good idea since the integration tests that exercise the full application stack should probably be written using Cucumber and Webrat (which is what I’m using).

You will probably want to merge my one-line setup method into your existing setup code in test_helper, if any. Also note that this uses Mocha for mocking.

I was on “hardy”, a.k.a. the Hardy Heron release, a.k.a. Ubuntu 8.04 LTS.

I had not bothered to install Ubuntu 8.10 / “Intrepid Ibex” because I didn’t have a reason to when it was release. I now wanted to upgrade to Ubuntu 9.04 “Jaunty Jackalope” which has WordPress 2.7.1, the current release (as of today).

The way to upgrade from 8.04 to 9.04 is to upgrade to 8.10 first. So I did that:

Intrepid Upgrades: Network Upgrade for Ubuntu Servers worked really well. I had to do a little bit of manual file merging as usual (I still don’t understand why dpkg can’t merge changes from the old file into a new file) but that was it. Easy!

When I rebooted the VPS, it kernel panicked: can’t mount the root filesystem. Oh crap. /dev/xvda1 is missing? Really? I told the VPS to hard reboot and it came up fine. But that’s a little scary. (I think this is something more related to my VPS hosting provider than Ubuntu, but I haven’t yet upgraded my laptop VMWare Ubuntu VPS’s yet so I’m not sure.)

The second stage didn’t go so well. I did the same sort of simple upgrade: the Jaunty Network Upgrade for Ubuntu Servers instructions are the same as the ones for Intrepid. Upgrade, edit a couple of config files, reboot. Kernel panic again, same reason, reboot. Should work, right?

It booted, but had no network access. I was able to log in via my VPS hosting provider’s SSH remote console feature, so I was able to see that /etc/init.d/networking was failing to start. It was the same problem that’s described in Ubuntu 9.04 in an OpenVZ VE. Adding that one line to /etc/init.d/networking fixed the problem. Reboot, all better.

So if you’re doing this upgrade on a VPS, make sure you’ve added that little 1-line hack after you do the Jaunty upgrade and before you reboot.