FactoryGirl build_stubbed strategy with a has_many association

Question

Given a standard has_many relationship between two objects. For a simple example, let's go with:

class Order < ActiveRecord::Base
  has_many :line_items
end

class LineItem < ActiveRecord::Base
  belongs_to :order
end

What I'd like to do is generate a stubbed order with a list of stubbed line items.

FactoryGirl.define do
  factory :line_item do
    name 'An Item'
    quantity 1
  end
end

FactoryGirl.define do
  factory :order do
    ignore do
      line_items_count 1
    end

    after(:stub) do |order, evaluator|
      order.line_items = build_stubbed_list(:line_item, evaluator.line_items_count, :order => order)
    end
  end
end

The above code does not work because Rails wants to call save on the order when line_items is assigned and FactoryGirl raises an exception: RuntimeError: stubbed models are not allowed to access the database

So how do you (or is it possible) to generate an stubbed object where it's has_may collection is also stubbed?

Answer 1

TL;DR

FactoryGirl tries to be helpful by making a very large assumption when it creates it's "stub" objects. Namely, that: you have an id, which means you are not a new record, and thus are already persisted!

Unfortunately, ActiveRecord uses this to decide if it should keep persistence up to date. So the stubbed model attempts to persist the records to the database.

Please do not try to shim RSpec stubs / mocks into FactoryGirl factories. Doing so mixes two different stubbing philosophies on the same object. Pick one or the other.

RSpec mocks are only supposed to be used during certain parts of the spec life cycle. Moving them into the factory sets up an environment which will hide the violation of the design. Errors which result from this will be confusing and difficult to track down.

If you look at the documentation for including RSpec into say test/unit, you can see that it provides methods for ensuring that the mocks are properly setup and torn down between the tests. Putting the mocks into the factories provides no such guarantee that this will take place.

There are several options here:

• Don't use FactoryGirl for creating your stubs; use a stubbing library (rspec-mocks, minitest/mocks, mocha, flexmock, rr, or etc)

  If you want to keep your model attribute logic in FactoryGirl that's fine. Use it for that purpose and create the stub elsewhere:

  stub_data = attributes_for(:order)
  stub_data[:line_items] = Array.new(5){
    double(LineItem, attributes_for(:line_item))
  }
  order_stub = double(Order, stub_data)
  

  Yes, you do have to manually create the associations. This is not a bad thing, see below for further discussion.

• Clear the id field

  after(:stub) do |order, evaluator|
    order.id = nil
    order.line_items = build_stubbed_list(
      :line_item,
      evaluator.line_items_count,
      order: order
    )
  end
  

• Create your own definition of new_record?

  factory :order do
    ignore do
      line_items_count 1
      new_record true
    end
  
    after(:stub) do |order, evaluator|
      order.define_singleton_method(:new_record?) do
        evaluator.new_record
      end
      order.line_items = build_stubbed_list(
        :line_item,
        evaluator.line_items_count,
        order: order
      )
    end
  end
  

What's Going On Here?

IMO, it's generally not a good idea to attempt to create a "stubbed" has_many association with FactoryGirl. This tends to lead to more tightly coupled code and potentially many nested objects being needlessly created.

To understand this position, and what is going on with FactoryGirl, we need to take a look at a few things:

• The database persistence layer / gem (i.e. ActiveRecord, Mongoid, DataMapper, ROM, etc)
• Any stubbing / mocking libraries (mintest/mocks, rspec, mocha, etc)
• The purpose mocks / stubs serve

The Database Persistence Layer

Each database persistence layer behaves differently. In fact, many behave differently between major versions. FactoryGirl tries to not make assumptions about how that layer is setup. This gives them the most flexibility over the long haul.

Assumption: I'm guessing you are using ActiveRecord for the remainder of this discussion.

As of my writing this, the current GA version of ActiveRecord is 4.1.0. When you setup a has_many association on it, there's a lot that goes on.

This is also slightly different in older AR versions. It's very different in Mongoid, etc. It's not reasonable to expect FactoryGirl to understand the intricacies of all of these gems, nor differences between versions. It just so happens that the has_many association's writer attempts to keep persistence up to date.

You may be thinking: "but I can set the inverse with a stub"

FactoryGirl.define do
  factory :line_item do
    association :order, factory: :order, strategy: :stub
  end
end

li = build_stubbed(:line_item)

Yep, that's true. Though it's simply because AR decided not to persist. It turns out this behavior is a good thing. Otherwise, it would be very difficult to setup temp objects without hitting the database frequently. Additionally, it allows for multiple objects to be saved in a single transaction, rolling back the whole transaction if there was a problem.

Now, you may be thinking: "I totally can add objects to a has_many without hitting the database"

order = Order.new
li = order.line_items.build(name: 'test')
puts LineItem.count                   # => 0
puts Order.count                      # => 0
puts order.line_items.size            # => 1

li = LineItem.new(name: 'bar')
order.line_items << li
puts LineItem.count                   # => 0
puts Order.count                      # => 0
puts order.line_items.size            # => 2

li = LineItem.new(name: 'foo')
order.line_items.concat(li)
puts LineItem.count                   # => 0
puts Order.count                      # => 0
puts order.line_items.size            # => 3

order = Order.new
order.line_items = Array.new(5){ |n| LineItem.new(name: "test#{n}") }
puts LineItem.count                   # => 0
puts Order.count                      # => 0
puts order.line_items.size            # => 5

Yep, but here order.line_items is really an ActiveRecord::Associations::CollectionProxy. It defines it's own build, #<<, and #concat methods. Of, course these really all delegate back to the association defined, which for has_many are the equivalent methods: ActiveRecord::Associations::CollectionAssocation#build and ActiveRecord::Associations::CollectionAssocation#concat. These take into account the current state of the base model instance in order to decide whether to persist now or later.

All FactoryGirl can really do here is let the behavior of the underlying class define what should happen. In fact, this lets you use FactoryGirl to generate any class, not just database models.

FactoryGirl does attempt to help a little with saving objects. This is mostly on the create side of the factories. Per their wiki page on interaction with ActiveRecord:

...[a factory] saves associations first so that foreign keys will be properly set on dependent models. To create an instance, it calls new without any arguments, assigns each attribute (including associations), and then calls save!. factory_girl doesn’t do anything special to create ActiveRecord instances. It doesn’t interact with the database or extend ActiveRecord or your models in any way.

Wait! You may have noticed, in the example above I slipped the following:

order = Order.new
order.line_items = Array.new(5){ |n| LineItem.new(name: "test#{n}") }
puts LineItem.count                   # => 0
puts Order.count                      # => 0
puts order.line_items.size            # => 5

Yep, that's right. We can set order.line_items= to an array and it isn't persisted! So what gives?

The Stubbing / Mocking Libraries

There are many different types and FactoryGirl works with them all. Why? Because FactoryGirl doesn't do anything with any of them. It's completely unaware of which library you have.

Remember, you add the FactoryGirl syntax to your test library of choice. You don't add your library to FactoryGirl.

So if FactoryGirl isn't using your preferred library, what is it doing?

The Purpose Mocks / Stubs Serve

Before we get to the under the hood details, we need to define what a "stub" is and its intended purpose:

Stubs provide canned answers to calls made during the test, usually not responding at all to anything outside what's programmed in for the test. Stubs may also record information about calls, such as an email gateway stub that remembers the messages it 'sent', or maybe only how many messages it 'sent'.

this is subtly different from a "mock":

Mocks...: objects pre-programmed with expectations which form a specification of the calls they are expected to receive.

Stubs serve as a way to setup collaborators with canned responses. Sticking to only the collaborators public API which you touch for the specific test keeps stubs lightweight and small.

Without any "stubbing" library, you can easily create your own stubs:

stubbed_object = Object.new
stubbed_object.define_singleton_method(:name) { 'Stubbly' }
stubbed_object.define_singleton_method(:quantity) { 123 }

stubbed_object.name       # => 'Stubbly'
stubbed_object.quantity   # => 123

Since FactoryGirl is completely library agnostic when it comes to their "stubs", this is the approach they take.

Looking at the FactoryGirl v.4.4.0 implementation, we can see that the following methods are all stubbed when you build_stubbed:

• persisted?
• new_record?
• save
• destroy
• connection
• reload
• update_attribute
• update_column
• created_at

These are all very ActiveRecord-y. However, as you have seen with has_many, it is a fairly leaky abstraction. The ActiveRecord public API surface area is very large. It's not exactly reasonable to expect a library to fully cover it.

Why does the has_many association not work with the FactoryGirl stub?

As noted above, ActiveRecord checks it's state to decide if it should keep persistence up to date. Due to the stubbed definition of new_record? setting any has_many will trigger a database action.

def new_record?
  id.nil?
end

Before I throw out some fixes, I want to go back to the definition of a stub:

Stubs provide canned answers to calls made during the test, usually not responding at all to anything outside what's programmed in for the test. Stubs may also record information about calls, such as an email gateway stub that remembers the messages it 'sent', or maybe only how many messages it 'sent'.

The FactoryGirl implementation of a stub violates this tenet. Since it has no idea what you are going to be doing in your test/spec, it simply tries to prevent database access.

Fix #1: Do Not Use FactoryGirl to Create Stubs

If you wish to create / use stubs, use a library dedicated to that task. Since it seems you are already using RSpec, use it's double feature (and the new verifying instance_double, class_double, as well as object_double in RSpec 3). Or use Mocha, Flexmock, RR, or anything else.

You can even roll your own super simple stub factory (yes there are issues with this, it's simply an example of an easy way to make an object with canned responses):

require 'ostruct'
def create_stub(stubbed_attributes)
  OpenStruct.new(stubbed_attributes)
end

FactoryGirl makes it very easy to create 100 model objects when really you needed 1. Sure, this is a responsible usage issue; as always great power comes create responsibility. It's just very easy to overlook deeply nested associations, which don't really belong in a stub.

Additionally, as you have noticed, FactoryGirl's "stub" abstraction is a bit leaky forcing you to understand both its implementation and your database persistence layer's internals. Using a stubbing lib should completely free you from having this dependency.

If you want to keep your model attribute logic in FactoryGirl that's fine. Use it for that purpose and create the stub elsewhere:

stub_data = attributes_for(:order)
stub_data[:line_items] = Array.new(5){
  double(LineItem, attributes_for(:line_item))
}
order_stub = double(Order, stub_data)

Yes, you do have to manually setup the associations. Though you only setup those associations which you need for the test/spec. You don't get the 5 other ones that you do not need.

This is one thing that having a real stubbing lib helps make explicitly clear. This is your tests/specs giving you feedback on your design choices. With a setup like this, a reader of the spec can ask the question: "Why do we need 5 line items?" If it's important to the spec, great it's right there up front and obvious. Otherwise, it shouldn't be there.

The same thing goes for those a long chain of methods called a single object, or a chain of methods on subsequent objects, it's probably time to stop. The law of demeter is there to help you, not hinder you.

Fix #2: Clear the id field

This is more of a hack. We know that the default stub sets an id. Thus, we simply remove it.

after(:stub) do |order, evaluator|
  order.id = nil
  order.line_items = build_stubbed_list(
    :line_item,
    evaluator.line_items_count,
    order: order
  )
end

We can never have a stub which returns an id AND sets up a has_many association. The definition of new_record? that FactoryGirl setup completely prevents this.

Fix #3: Create your own definition of new_record?

Here, we separate the concept of an id from where the stub is a new_record?. We push this into a module so we can re-use it in other places.

module SettableNewRecord
  def new_record?
    @new_record
  end

  def new_record=(state)
    @new_record = !!state
  end
end

factory :order do
  ignore do
    line_items_count 1
    new_record true
  end

  after(:stub) do |order, evaluator|
    order.singleton_class.prepend(SettableNewRecord)
    order.new_record = evaluator.new_record
    order.line_items = build_stubbed_list(
      :line_item,
      evaluator.line_items_count,
      order: order
    )
  end
end

We still have to manually add it for each model.