So now instead of making a YAML file and referring to it, pass the options like this:

bundle config build.nokogiri \
--with-xml2-include=/usr/local/include/libxml2 \
--with-xml2-lib=/usr/local/lib

User story: “As a princess I want to confirm my royalty so I get bruised after sleeping on 40 mattresses over a pea”. Just tell real stories”

That sounded so much like a Cucumber feature that I decided to write it as one:

Feature: Physical Sensitivity
  In order to confirm my royalty
  As a princess
  I want to be very delicate

  Scenario: 40 mattresses on a pea
  Given there is a pea on the bed
  And there is a stack of 20 mattresses on the pea
  And there is a stack of 20 featherbeds on the mattresses
  When I try to sleep on top of the stack of featherbeds
  Then I should not be able to sleep

`method_missing': No sham defined for name

It’s discussed in the machinist Google Group as well.

It’s because of a wacky hack that’s part of Machinist that overrides Module.name so you can do Sham.name, but ZenTest expects Module.name to do what it does normally.

I have a fix for this.

I sent notahat a pull request via GitHub, so hopefully it’ll be merged by notahat into the official machinist gem, but until that happens you can try mine. I published the gem as JamieFlournoy-machinist to rubygems.org.

Try ‘gem install JamieFlournoy-machinist -v 1.0.6′ and see if it works for you. It does for me with ZenTest 4.2.1, autotest-rails 4.1.0, and a Rails 2.3.5 project.

(my machinist fork on github)

But because the concept of database migrations is such a powerful one, it’s tempting to jam any old change that affects the database into a new migration and run rake db:migrate to make it happen. I’ve been guilty of a bit of this in the past, and I’ve joined some projects that did other ugly things in migrations. In the process I’ve learned the hard way that there are some things you must never do in a migration or they will come back to haunt you later. Here they are.

Antipattern: Require the Database to Exist Already

In other words, the antipattern is for the first migration to depend on some tables and maybe even some data already being in the database.

I know that the original Rails blog video shows DHH using a MySQL admin tool to create the blog database interactively, but really you should be using migrations to create the schema programmatically from scratch.

If you’re already working on a project that didn’t do that, you can run rake db:schema:dump and look at db/schema.rb; it contains code that you can insert into a new migration to create the same schema in your development environment. If you’re using DB features that the design philosophy of ActiveRecord doesn’t agree with, such as triggers, and the schema.rb dump doesn’t include them (or if you just think the migration DSL is ugly and you like SQL DDL better), you can do a mysqldump / pg_dump / whateverdump and wrap a migration around the loading of that SQL file.

If you have a hybrid (you have to start with an old db dump and then migrate it so it becomes current), that’s gross, and you have a couple of options which are both pretty ugly. But they will work, and when you’re done the ugliness will be gone.

You could fight your way back to the oldest schema version by debugging the self.down methods and running rake db:rollback repeatedly until you can create a 00001_starting_db_schema.rb migration, or you could just blow away all the migrations and use the highest schema version for a new migration that contains the output of a current rake db:schema:dump. It depends on how many copies of the database are out there with old schemas that would need to be brought up to date. Clearing out db/migrate and replacing it all with a single migration is cleaner, but if your production database is 5 migrations out of date you obviously can’t do that. But you could collapse it down to the one big-bang migration (as the oldest), plus the 5 pending schema changes. If you do it right, you can just deploy the new code and run rake db:migrate and everything will be fine. If not, well, you were testing it on a backup of the production database, right? :)

Antipattern: Only Work Correctly With the Production Data

What’s wrong with developers just making dumps of the production database and loading them locally?

First of all, it means that all schema changes have to start at the production database and work backwards to developers’ sandboxed development environments. Hopefully this strikes you as a very stupid workflow.

Secondly, maybe your users don’t all want to get a message that says “test message foo bar sdfasdfasd bloopity bloop” when you’re testing your new alert system. Should you really be putting their data (passwords, contact info, etc.) at the mercy of your crummy new code?

You should be able to immediately generate an empty, clean database for development. rake db:drop; rake db:create; rake db:migrate should do this; rake db:reset should have the same result but should be faster since it doesn’t bother with each migration in sequence.

You should also be able to immediately generate any essential base data such as the initial admin user. The SeedFu plugin does a good job here.

If you need some additional fake data to fiddle around with in your development environment, the Populator gem is handy for mass-inserting a bunch of faux data, especially in conjunction with Faker.

Note that the migrations should neither depend on nor contain actual data. They should just change the data model.

Antipattern: Clean Up That Only Works on Production Data

This is really a subset of the previous item but it’s worth considering as a special case.

If you want to fix some data that got slightly corrupted by some bad code that has been replaced, migrations aren’t a terrible way to accomplish that.

It’s not really what migrations are for, and a one-off rake task can do it just as well, but if you really want to, you can get away with it under one condition: you have to make your cleanup migration code succeed even if the database is empty (such as when a developer has just run rake db:reset; rake db:migrate).

Antipattern: Load Data

The populator gem is good for initial, mandatory data. The machinist gem is good for synthetic test data. Delete db/fixtures and everything in it. Fixtures are evil.

Wrap a rake task around the “get my development database ready” concept. This task should start with the “get my empty production database ready” task (or some subset of that which is appropriate for developer use).

If you need to load arbitrary data now and then, write an importer. Do this as a rake task, or a web UI to a bulk data importer feature. Better yet, make a web UI in your admin area which is just a wrapper around the rake task that bulk imports data. Then delegate the bulk importing to your customers so your admins can do real admin work. But don’t load data in a migration.

Antipattern: Use Rails Models in the Migration

Models evolve, but old migrations don’t change (nor should they). So when you wrote a migration that used a model, it used the old version of the model code. Then a year later the model has evolved, and the new validations on first_name and last_name fail because it used to be full_name, and that old migration that hasn’t changed has stopped working. It depended on something that did change, incompatibly.

For rockstar points, in your continuous integration environment you should run rake db:drop; rake db:create; rake db:migrate to make sure that this can never happen.

But if it has already happened, rip out the model code and replace it with Rails DSL code, with execute statements containing raw SQL code, or (if you feel like a Ruby rockstar) declare new, stripped down model classes inside your migration class that will act as stand-ins for the limited needs of the migration. See Migrating with Models for more on how to do this last trick.

Conclusion

You should always be able to do this in every Rails environment that your application has: rake db:drop; rake db:create; rake db:migrate; rake db:reset

At this point you should then be able to run rake db:test:prepare and then rake spec or rake test or whatever and have it work.

If any part of that process fails, you are missing out on the benefits of using Rails migrations.

“HI. You’re using libxml2 version 2.6.16 which is over 4 years old and has plenty of bugs. We suggest that for maximum HTML/XML parsing pleasure, you upgrade your version of libxml2 and re-install nokogiri. If you like using libxml2 version 2.6.16, but don’t like this warning, please define the constant I_KNOW_I_AM_USING_AN_OLD_AND_BUGGY_VERSION_OF_LIBXML2 before requring nokogiri.”

Aaron Kalin figured out how to fix this if you’re installing nokogiri as a system gem, but I want to use Bundler and keep my system gems down to the bare minimum. I figured out how to do this under Bundler 0.8.5.

The gems I have installed via plain old “gem install xxx” are bundler08 -v0.8.5, rake -v0.8.7, and ZenTest -v4.2.1. That’s it. Everything else is under vendor/bundler. I’m using RVM and REE 1.8.7 2009.10 as described in my previous post.

Here is a quick summary of Aaron’s blog post; this will update your libxml and libxslt libraries to reasonably recent versions. (Be my guest if you want to use libxml 2.7.7 and libxslt 1.1.26, which are current as of today, but I didn’t test those myself.)

mkdir -p ~/Downloads/XML
cd ~/Downloads/XML
curl -O ftp://xmlsoft.org/libxml2/libxml2-2.7.3.tar.gz
curl -O ftp://xmlsoft.org/libxml2/libxslt-1.1.24.tar.gz
tar xzf libxml2-2.7.3.tar.gz
tar xzf libxslt-1.1.24.tar.gz
cd ~/Downloads/XML/libxml2-2.7.3
./configure
nice make
sudo make install
cd ~/Downloads/XML/libxslt-1.1.24
./configure --with-libxml-prefix=/usr/local --with-libxml-include-prefix=/usr/local/include --with-libxml-libs-prefix=/usr/local/lib
nice make
sudo make install

Now make this YAML file and call it bundler_build_options_macosx_leopard.yml and put it in your RAILS_ROOT:

nokogiri:
  xml2-include: /usr/local/include/libxml2
  xml2-lib: /usr/local/lib

Then, to build nokogiri so that it uses the new libs you just installed:

gem bundle --build-options bundler_build_options_macosx_leopard.yml

You should now have nokogiri installed under vendor/bundler/ruby/1.8/gems, and when you run your app it should not complain about the version of libxml2 anymore.

Update: Bundler 1.0.0.beta.10 allows this too. See my other post that shows how to do it.

Igor P’s solution is correct (just install REE 1.8.7-2009.10), but it took a little fiddling to figure out how to get RVM to use the older version of REE. Here’s how to do it:

cd ~/.rvm/archives
wget -q http://rubyforge.org/frs/download.php/66162/ruby-enterprise-1.8.7-2009.10.tar.gz
rvm install ree-1.8.7-2009.10

I found a good starting point at geokit stubbing for faster tests. I also wanted a way to stub a geocoding failure, and a way to prevent any unit tests from using the real geocoding web service.

Here’s how I did it.

In test/test_helper.rb, add this:

class ActiveSupport::TestCase
  def setup
    GeoKit::Geocoders::MultiGeocoder.stubs(:geocode).raises(RuntimeError,
      'Use mock_geocoding_success! or mock_geocoding_failure! in your test')
  end

  def mock_geocoding_success!
    geocode_payload = GeoKit::GeoLoc.new(:lat => 123.456, :lng => 123.456)
    geocode_payload.success = true
    GeoKit::Geocoders::MultiGeocoder.expects(:geocode).returns(geocode_payload)
  end

  def mock_geocoding_failure!
    geocode_payload = GeoKit::GeoLoc.new
    geocode_payload.success = false
    GeoKit::Geocoders::MultiGeocoder.expects(:geocode).returns(geocode_payload)
  end
end

What this does is to force you to choose either one of those mock_geocoding methods before you call the geocode method. To me this seems like a good idea since the integration tests that exercise the full application stack should probably be written using Cucumber and Webrat (which is what I’m using).

You will probably want to merge my one-line setup method into your existing setup code in test_helper, if any. Also note that this uses Mocha for mocking.

However, if you’re upgrading an existing project and you’re sticking with official releases (as opposed to edge Rails) like I am, your code probably needs some tweaking to work with Rails 2.2.2. Mine certainly did. Although there were a lot of failed tests with ugly stacktraces, there were only a few API changes in Rails that needed to be accomodated to fix them all. Here’s a list of the changes that broke my app, and what I had to do to get it working again.

TextHelper.truncate no longer allows you to supply the length parameter as a second argument. You have to use truncate(somestring, :length => 20) instead of truncate(somestring, 20).

I was getting an error that looked like this:

ActionView::TemplateError: wrong number of arguments (3 for 1)

The line in question is:

<%= render(:partial => 'layouts/header') %>

The problem is that the simply_helpful plugin’s behavior is now built-in; just delete the whole vendor/plugins/simply_helpful directory if you were using it, and any existing code that depends on simply_helpful’s API should work again.

I was using the tztime and tzinfo_timezone plugins, which are no longer needed now that time zone support is available via Time.zone in Rails 2.2. Delete the plugins, upgrade the tzinfo gem to 0.3.12 or uninstall any older version of it, and search for TzTime instances in your code and replace any of the occurences of Time.at(some_tztime_variable) that you had been using. TzInfo is still useful in cases such as this:

TZInfo::Timezone.get('UTC')

But since you can now embed the time zone information in a Time instance, you don’t need TzTime objects anymore.

I was using a nice hack from Technoweenie, to allow me to validate forms using the familiar ActiveRecord model validations API even if those forms were not backed by a database table (sometimes the UI just doesn’t look anything like the data model, y’know?). It broke, and I was unable to find any solutions from anyone else, so I wrote my own: EphemeralModel, for Rails 2.2.2 form validation without a DB table.

I was using ActiveRecord::Errors.default_error_messages[:invalid] in a few model validators, which is now deprecated because it’s not internationalized. The new way to ask for the same string in the currently configured language is: I18n.t(:invalid, :scope => 'activerecord.errors.messages')

Finally, the app wouldn’t start because it wants a :secret option to config.action_controller.session in config/environment.rb. I came up with something sufficiently random and now it’s fine.

I’d just like to make one last comment about this: having thorough automated tests is a huge time and sanity saver during this process. If you’re not close to 100% test coverage on your current Rails project, then your house is on fire and you need to address that immediately. Even as I write this in late 2008, I’m talking to prospective clients about contracting work, and some of them don’t do automated testing at all(!). Unless you’re in a mad dash for a demo that will save the company from financial collapse, having automated tests that verify that your code works should be priority #1. Upgrades can wait.

# Usage: in the faux-model class, include EphemeralModel and ActiveRecord::Validations
# Then just say "column :foo" to set up a property called 'foo' for validation
module EphemeralModel
    # Stubbing methods from ActiveRecord::Base that ActiveRecord::Validations may call
    def EphemeralModel.included(mod)
        def mod.self_and_descendents_from_active_record; [self]; end
        def mod.human_attribute_name(arg); arg.to_s; end
        def mod.human_name; self.name; end
        def mod.column(attr_name)
            attr_name = attr_name.to_s
            column_names.push(attr_name) unless column_names.include?(attr_name)
            attr_accessor attr_name
        end
        mod.module_eval do
            def self.column_names
                cn = self.instance_variable_get(:@column_names)
                if cn.nil?
                    cn = self.instance_variable_set(:@column_names, [])
                end
                cn
            end
        end
    end
    def save; end
    def save!; end
    def update_attribute; end
    def new_record?; end

    def initialize(initial_attributes = nil)
        initial_attributes.each{|k,v| instance_variable_set("@#{k}".to_sym, v)} if initial_attributes
    end

    def attributes
        @attributes = {}
        self.class.column_names.each do |colname|
            @attributes[colname]= instance_variable_get("@#{colname}".to_sym)
        end
        @attributes
    end

    def attribute_names
        self.class.column_names
    end

    def [](name); attributes[name.to_s]; end

    def []=(name, value)
        instance_variable_set("@#{name}".to_sym, value)
        # print "set #{name} to " + instance_variable_get("@#{name}".to_sym) + ".\n" # DEBUG
    end
end

I’m happy I chose to make them gems, but I miss some of the stuff you get in a Rails project. In particular I wanted ‘rake:stats’, so I can update my estimation spreadsheet which is now almost 9 months old. I need the stats for each gem in addition to the main Rails project, in order to compare this to prior figures from the Rails project before I split it up.

So, here is the Rakefile snippet that I added, which adds the rake:stats task into a regular gem. If you don’t have the Rails gem installed, it will fail gracefully, without breaking your whole gem. So you need not make your teeny little gem depend on all of Rails being installed on every machine where your gem needs to go. Just install Rails whereever you want to run stats, which is probably already the case on your development machine.

Here is a Pastie link, since my blog theme truncates the too-wide text: this same code on Pastie

# stats
begin
  gem 'rails'
  require 'code_statistics'
  namespace :spec do
    desc "Use Rails's rake:stats task for a gem"
    task :statsetup do
      class CodeStatistics
        def calculate_statistics
          @pairs.inject({}) do |stats, pair|
            if 3 == pair.size
              stats[pair.first] = calculate_directory_statistics(pair[1], pair[2]); stats
            else
              stats[pair.first] = calculate_directory_statistics(pair.last); stats
            end
          end
        end
      end
      ::STATS_DIRECTORIES = [['Libraries',   'lib',  /.(sql|rhtml|erb|rb|yml)$/],
                   ['Tests',     'test', /.(sql|rhtml|erb|rb|yml)$/]]
      ::CodeStatistics::TEST_TYPES << "Tests"
    end
  end
  desc "Report code statistics (KLOCs, etc) from the application"
  task :stats => “spec:statsetup” do
    CodeStatistics.new(*STATS_DIRECTORIES).to_s
  end
rescue Gem::LoadError => le
  task :stats do
    raise RuntimeError, “‘rails’ gem not found - you must install it in order to use this task.n”
  end
end

BTW I intend to replace this with a sloccount-based task later. If and when I do that I’ll publish it here.