Chef: Why are resources in an "include_recipe" step being skipped?

Question

Chef seems to be processing resources in a strange order, causing my build to fail. My main recipe (mytardis-chef/site-cookbooks/recipes/default.rb) starts like this:

include_recipe "build-essential"
include_recipe "mytardis::deps"
include_recipe "mytardis::nginx"
include_recipe "mytardis::postgresql"

mytardis-chef/cookbooks/build-essential/recipes/default.rb looks like this:

case node['platform']
when "ubuntu","debian"
  %w{build-essential binutils-doc}.each do |pkg|
    package pkg do
      action :install
    end
  end
when "centos","redhat","fedora","scientific"
  %w{gcc gcc-c++ kernel-devel make}.each do |pkg|
    package pkg do
      action :install
    end
  end
end

... (it's an older version of https://github.com/opscode-cookbooks/build-essential/blob/master/recipes/default.rb)

At runtime, for reasons I don't understand, this build-essential recipe gets loaded but not executed:

[default] Waiting for VM to boot. This can take a few minutes.
[default] VM booted and ready for use!
[default] Mounting shared folders...
[default] -- v-root: /vagrant
[default] -- v-csr-3: /tmp/vagrant-chef-1/chef-solo-3/roles
[default] -- v-csc-2: /tmp/vagrant-chef-1/chef-solo-2/cookbooks
[default] -- v-csc-1: /tmp/vagrant-chef-1/chef-solo-1/cookbooks
[default] -- v-csdb-4: /tmp/vagrant-chef-1/chef-solo-4/data_bags
[default] Running provisioner: Vagrant::Provisioners::ChefSolo...
[default] Generating chef JSON and uploading...
[default] Running chef-solo...
[Sun, 08 Jul 2012 05:14:32 +0200] INFO: *** Chef 10.12.0 ***
[Sun, 08 Jul 2012 05:14:32 +0200] DEBUG: Building node object for lucid32
[Sun, 08 Jul 2012 05:14:32 +0200] DEBUG: Extracting run list from JSON attributes provided on command line
[Sun, 08 Jul 2012 05:14:32 +0200] INFO: Setting the run_list to ["recipe[mytardis]"] from JSON
[Sun, 08 Jul 2012 05:14:32 +0200] DEBUG: Applying attributes from json file
[Sun, 08 Jul 2012 05:14:32 +0200] DEBUG: Platform is ubuntu version 10.04
[Sun, 08 Jul 2012 05:14:32 +0200] INFO: Run List is [recipe[mytardis]]
[Sun, 08 Jul 2012 05:14:32 +0200] INFO: Run List expands to [mytardis]
[Sun, 08 Jul 2012 05:14:32 +0200] INFO: Starting Chef Run for lucid32
[Sun, 08 Jul 2012 05:14:32 +0200] INFO: Running start handlers
[Sun, 08 Jul 2012 05:14:32 +0200] INFO: Start handlers complete.
...
[Sun, 08 Jul 2012 05:14:33 +0200] DEBUG: Loading Recipe mytardis via include_recipe
[Sun, 08 Jul 2012 05:14:33 +0200] DEBUG: Found recipe default in cookbook mytardis
[Sun, 08 Jul 2012 05:14:33 +0200] DEBUG: Loading Recipe build-essential via include_recipe
[Sun, 08 Jul 2012 05:14:33 +0200] DEBUG: Found recipe default in cookbook build-essential
[Sun, 08 Jul 2012 05:14:33 +0200] DEBUG: Loading Recipe mytardis::deps via include_recipe
[Sun, 08 Jul 2012 05:14:33 +0200] DEBUG: Found recipe deps in cookbook mytardis
[Sun, 08 Jul 2012 05:14:33 +0200] DEBUG: Loading Recipe mytardis::nginx via include_recipe
[Sun, 08 Jul 2012 05:14:33 +0200] DEBUG: Found recipe nginx in cookbook mytardis
[Sun, 08 Jul 2012 05:14:33 +0200] DEBUG: Loading Recipe iptables via include_recipe
[Sun, 08 Jul 2012 05:14:33 +0200] DEBUG: Found recipe default in cookbook iptables
[Sun, 08 Jul 2012 05:14:33 +0200] DEBUG: Loading Recipe mytardis::postgresql via include_recipe
[Sun, 08 Jul 2012 05:14:33 +0200] DEBUG: Found recipe postgresql in cookbook mytardis
[Sun, 08 Jul 2012 05:14:33 +0200] DEBUG: Loading Recipe postgresql::server via include_recipe
[Sun, 08 Jul 2012 05:14:33 +0200] DEBUG: Found recipe server in cookbook postgresql
[Sun, 08 Jul 2012 05:14:33 +0200] DEBUG: Loading Recipe postgresql::client via include_recipe
[Sun, 08 Jul 2012 05:14:33 +0200] DEBUG: Found recipe client in cookbook postgresql
[Sun, 08 Jul 2012 05:14:33 +0200] INFO: Processing package[postgresql-client] action install (postgresql::client line 37)
[Sun, 08 Jul 2012 05:14:33 +0200] DEBUG: package[postgresql-client] checking package status for postgresql-client
....
[Sun, 08 Jul 2012 05:14:45 +0200] ERROR: gem_package[pg] (postgresql::client line 42) has had an error
.
make
sh: make: not found

That is, the build-essential recipe is "found" and "loaded", but it's the postgres recipe that gets "processed" first. And since build-essential (which installs the C compiler) wasn't run, the latter fails.

The relevant part of my Vagrantfile looks like this:

  config.vm.provision :chef_solo do |chef|
     chef.log_level = :debug
     chef.cookbooks_path = ["mytardis-chef/site-cookbooks", "mytardis-chef/cookbooks"]
     chef.roles_path = "mytardis-chef/roles"
     chef.data_bags_path = "mytardis-chef/data_bags"
     chef.add_recipe "mytardis"    
  end

I was previously using a slightly earlier version of Chef (perhaps 10.10.0?) In that version, build-essential was also not being run, but mytardis::deps was. Now using Chef 10.12.0. Physical machine is OSX, VM is Ubuntu Lucid.

So, a few questions:

1. Why is build-essential not being "processed"?
2. What's the right way to do this? (I didn't write these recipes, and I understand they do or have worked for their author.)
3. Should the site-cookbooks and cookbooks 'shadowing' feature still work? It's supposedly deprecated: http://tickets.opscode.com/browse/CHEF-2308 (I tried making a symbolic link under from site-cookbooks/mytardis/recipes/build-essential but no joy).

Answer 1

It turns out this is a pretty normal (but under-documented) part of how Chef works: it compiles everything, and then starts running. Except some recipes (like Postgres) jump the queue to actually install at compile-time, using code like this:

  action :nothing
end.run_action(:run)

The solution is that anything that needs to run before Postgres also needs to do this. Fortunately, newer versions of Build-essentials allow this. You set attributes on a role as follows:

name "myapp"
run_list(
  "recipe[build-essential]",
  "recipe[myapp]"
)
default_attributes(
  "build_essential" => {
    "compiletime" => true
  }
)