The Shape at the Bottom of All Things

I've been teaching a fair amount, which means I've been revisiting my 'class problems' regularly.  When I chose the problems, I thought that I understood them completely (hubris, I know) but now that I've worked them repeatedly I'm seeing new and surprising things.

These new things have to do with the shape of code.  Code can be written, or shaped, in many ways, and I've always believed that for any given problem many different code shapes gave equally 'good' solutions.  (I think of programming as an art and am willing to give artists a fair amount of expressive leeway.)

But I'm having a change of heart.  These days it feels like all shapes are not equally 'good', that some code shapes are actually better than others.  Some shapes expose information that others conceal.  This blog post illustrates the transition I'm undergoing.

Example 1 below is a slightly modified version of the code [1] used in my previous blog post Getting It Right by Betting on Wrong about the Open/Closed principle.  The House class contains code to produce the tale 'The House that Jack Built' [2].  The Controller class invokes House#line in its #play_house method on line 33.  Line 37 invokes the controller.  The output is on line 40.

###Example 1: The House that Jack Built

class House
  DATA = [
    'the horse and the hound and the horn that belonged to',
    'the farmer sowing his corn that kept',
    'the rooster that crowed in the morn that woke',
    'the priest all shaven and shorn that married',
    'the man all tattered and torn that kissed',
    'the maiden all forlorn that milked',
    'the cow with the crumpled horn that tossed',
    'the dog that worried',
    'the cat that killed',
    'the rat that ate',
    'the malt that lay in',
    'the house that Jack built',
  ]

  def recite
    (1..DATA.length).map {|i| line(i)}.join("\n")
  end

  def line(number)
    "This is #{phrase(number)}.\n"
  end

  private
  def phrase(number)
    DATA.last(number).join(" ")
  end
end

class Controller
  def play_house
    House.new.line(12)
  end
end

puts "\n----\n" + Controller.new.play_house

# ----
# This is the horse and the hound and the horn that belonged to the farmer sowing his corn that kept the rooster that crowed in the morn that woke the priest all shaven and shorn that married the man all tattered and torn that kissed the maiden all forlorn that milked the cow with the crumpled horn that tossed the dog that worried the cat that killed the rat that ate the malt that lay in the house that Jack built.

Example 1 works fine but let's imagine that requirements change.  Our customer tells us that they like House and they want it to continue to work as is, but they'd also like a variant that randomizes the data before producing the tale.  

Example 2 meets this new requirement.  House#initialize now takes random, a boolean.  If random is false, House behaves normally, if true, House randomizes and caches the data before producing the tale.

###Example 2

class House
  # ...
  def initialize(random)
    @pieces = DATA.shuffle if random
  end

  def recite
    (1..pieces.length).map {|i| line(i)}.join("\n")
  end

  def line(number)
    "This is #{phrase(number)}.\n"
  end

  private
  def phrase(number)
    pieces.last(number).join(" ")
  end

  def pieces
    @pieces ||= DATA
  end
end

class Controller
  def play_house(random = false)
    House.new(random).line(12)
  end
end

puts "\n--random? false--\n" + Controller.new.play_house(false)
puts "\n--random? true --\n" + Controller.new.play_house(true)

# --random? false--
# This is the horse and the hound and the horn that belonged to the farmer sowing his corn that kept the rooster that crowed in the morn that woke the priest all shaven and shorn that married the man all tattered and torn that kissed the maiden all forlorn that milked the cow with the crumpled horn that tossed the dog that worried the cat that killed the rat that ate the malt that lay in the house that Jack built.

# --random? true --
# This is the rat that ate the malt that lay in the priest all shaven and shorn that married the farmer sowing his corn that kept the cat that killed the house that Jack built the horse and the hound and the horn that belonged to the man all tattered and torn that kissed the cow with the crumpled horn that tossed the maiden all forlorn that milked the dog that worried the rooster that crowed in the morn that woke.

Example 2 now contains conditionals on line 4 and 21.  These conditionals collaborate to meet the 'random' requirement but the way the code is shaped makes it hard to see that these two conditionals are about the same concept.  Not only are they far apart in the code but one is expressed as a trailing if (which checks the value of random) and the other as ||= (which checks the value of @pieces).

Changing the requirements again will bring the underlying issue more sharply into focus.  Our customer, when shown the output, decides they'd like a third variant.  The current 'randomized' version can end in very unsatisfying ways (for example, with 'the rat that ate').  Our customer would like a 'mostly random' version which randomizes all pieces except the last.  This 'mostly random' version should always end with 'the house that Jack built'.

Example 3 shows the interesting new bits of code.  

###Example 3

class House
  # ...
  attr_reader :pieces

  def initialize(order)
    @pieces = initialize_pieces(order)
  end

  # ...
  def initialize_pieces(order)
    case order
    when :random
      DATA.shuffle
    when :mostly_random
      DATA[0...-1].shuffle << DATA.last
    else
      DATA
    end
  end
end

class Controller
  def play_house(choice = nil)
    House.new(choice).line(12)
  end
end

puts "\n----\n"               + Controller.new.play_house
puts "\n--:random--\n"        + Controller.new.play_house(:random)
puts "\n--:mostly_random--\n" + Controller.new.play_house(:mostly_random)

# ----
# This is the horse and the hound and the horn that belonged to the farmer sowing his corn that kept the rooster that crowed in the morn that woke the priest all shaven and shorn that married the man all tattered and torn that kissed the maiden all forlorn that milked the cow with the crumpled horn that tossed the dog that worried the cat that killed the rat that ate the malt that lay in the house that Jack built.

# --:random--
# This is the dog that worried the house that Jack built the malt that lay in the rat that ate the maiden all forlorn that milked the cat that killed the rooster that crowed in the morn that woke the horse and the hound and the horn that belonged to the man all tattered and torn that kissed the farmer sowing his corn that kept the priest all shaven and shorn that married the cow with the crumpled horn that tossed.

# --:mostly_random--
# This is the man all tattered and torn that kissed the cow with the crumpled horn that tossed the maiden all forlorn that milked the horse and the hound and the horn that belonged to the dog that worried the malt that lay in the rooster that crowed in the morn that woke the rat that ate the cat that killed the farmer sowing his corn that kept the priest all shaven and shorn that married the house that Jack built.

Now that we have three different ordering requirements it's no longer sufficient to pass a boolean.  Therefore, House's initialize method takes a symbol (:random, :mostly_random or anything else) and sets the value of @pieces to the result of calling initialize_pieces on that symbol (line 6 above).  #initialize_pieces contains a case statement (lines 11-18) that arranges the data in the correct order and returns it.

Example 3 does two new things.  First, it adds the new 'mostly random' variant, and second, it moves all of the code related to the concept of 'data order' into a single case statement.  

While we certainly need to do the first we could easily have gotten by without the second.  We could instead have kept the existing #pieces method and omitted the else branch from the new case statement, like so:

def initialize_pieces(order)
    case order
    when :random
      DATA.shuffle
    when :mostly_random
      DATA[0...-1].shuffle << DATA.last
    end
  end

  def pieces
    @pieces ||= DATA
  end

This works, but the code doesn't feel natural.  Once the number of variants forces us to change to a case statement it feels more 'right' to expect that case statement to deal with all of the ordering, including the default.  The code above separates the default from the variants while Example 3 treats the default as a variant. Example 3 line 17 replaces Example 2 line 21 and groups all of the code that controls the concept of 'order' in one place.

The key idea here is that 'not changing the order' is a real thing, as real as 'randomizing' or 'mostly randomizing' it.  It's not as if :random and :mostly_random represent one concept and 'doing nothing' represents another.  There's one concept, 'order', and a number of different possibilities.  One way to order something is to leave its current order unchanged; this is an algorithm as valid as any other.

Now that we're treating every order as a real thing let's do a thought exercise.  Imagine that each branch of the case statement contained many lines of code, so much that you felt obliged to extract them into methods of their own.  How would you name these extracted methods?

Example 3a illustrates one possibility.

###Example 3a

class House
  # ...
  def initialize_pieces(order)
    case order
    when :random
      random_order
    when :mostly_random
      mostly_random_order
    else
      default_order
    end
  end

  def random_order
    DATA.shuffle
  end

  def mostly_random_order
    DATA[0...-1].shuffle << DATA.last
  end

  def default_order
    DATA
  end
  # ...

These xxx_order methods above represent 'order' variants.  Unsurprisingly, most of the method names reflect the symbols that we used in the case statement.  Symbol :random becomes method #random_order, :mostly_random becomes #mostly_random__order and the else branch becomes #default_order.  The fact that we can imagine a method named #default_order supports the notion that the else branch represents the same kind of thing as the other branches.  Ordering something as 'unchanged' is as valid as ordering it 'random'; to insist otherwise judges some algorithms as not as 'real' than others.

Now that we've explicitly named the methods we can see that the names have a repeating suffix.  When methods have a repeating prefix or suffix it's a sign that you have untapped objects hidden within your code.  Going through the exercise of giving the branches of the case statement explicit names helps identify these missing objects.  Instead of forcing House to know both 1) the values of order upon which it should switch and 2) what to do in every case, we can disperse the 'what to do' logic into other objects.  

Example 4 creates a new class for each kind of order.

###Example 4

class House
  # ...
  def initialize_pieces(order)
    case order
    when :random
      Random.new.order(DATA)
    when :mostly_random
      MostlyRandom.new.order(DATA)
    else
      Default.new.order(DATA)
    end
  end
end

class Default
  def order(data)
    data
  end
end

class Random
  def order(data)
    data.shuffle
  end
end

class MostlyRandom
  def order(data)
    data[0...-1].shuffle << data.last
  end
end

Example 4 creates three new classes, each of which plays the 'orderer' role.  Each 'orderer' implements #order to take a list and return it in the correct order.

These classes will be a delight to test. :-)

Example 4a slightly rearranges the case statement (and likely offends some Rubyists, but that's for another day) to make its purpose more obvious.

###Example 4a

class House
  # ...
  def initialize_pieces(order)
    case order
    when :random
      Random
    when :mostly_random
      MostlyRandom
    else
      Default
    end.new.order(DATA)
  end
    # ...
end

If the syntax above is new to you remember that the case...end statement returns an object to which you can send a message.  This case statement returns a class; line 11 sends new.order(DATA) to that class.  Thus, the case statement's responsibility is to return a class that plays the role of 'orderer'; actual ordering is a separate task that happens afterwards.

Example 4a reveals a curious thing.  House is initialized on the order symbol, which it immediately converts into a different object.  You can think of House as being injected with a behaviorally impaired kind of 'orderer' (the symbol) which it is then forced to convert into a more robust kind of 'orderer' (an instance of Random, MostlyRandom or Default).  In the above implementation House depends on (knows about) many things.  It knows the names of all possible symbols, the names all of the 'orderer' classes and the mapping between the two.  Many distant changes might force changes to House; it would be more flexible if it knew less.

We could spare House many of these dependencies if we inject the object it actually wants, and Example 5 does exactly that.  Here, Controller has assumed responsibility for creating 'orderer's and injecting them into House (line 13).

###Example 5

class House
  # ...
  attr_reader :pieces

  def initialize(orderer)
    @pieces = orderer.order(DATA)
  end
  # ...
end

class Controller
  def play_house(choice = nil)
    House.new(orderer_for(choice)).line(12)
  end

  def orderer_for(choice)
    case choice
    when :random
      Random
    when :mostly_random
      MostlyRandom
    else
      Default
    end.new
  end
end

The responsibility for converting feeble 'orderer' objects into more robust ones belongs no more in Controller than it did in House, but this new code is an improvement.  It's best to do these kinds of conversions at the first opportunity and at least now we're pushing the conversion back up the stack, searching for its natural home.

With this change House becomes open/closed to new 'orderers'; you can inject any object you like as long as it implements #order.  House also has fewer dependencies; it can collaborate with new 'orderers' without being forced to change.

The Controller#orderer_for method, however, is not yet open/closed; it must change if you add new 'orderers'.  If you're willing to commit to a naming convention and do a bit of metaprogramming (as in Example 6), this is easily remedied.

###Example 6

class Controller
  # ...
  def orderer_for(choice)
    Object.const_get(
      (choice || 'default').to_s.split('_').map(&:capitalize).join
      ).new
  end
end

As long as you follow the naming convention this code will convert any symbol to an instance of the corresponding class.

Controller's #orderer_for method was uncomfortable when it was merely in the wrong place but now that we've complicated the code in the name of making it open/closed it feels increasingly important to figure out where the method belongs.  We have a number of things that revolve around the concept of 'order' (three classes and this factory method) and this code would be easier to understand if they all lived together.  Example 7 creates an Order module to hold them.

###Example 7

module Order
  def self.new(choice)
    const_get(
      (choice || 'default').to_s.split('_').map(&:capitalize).join
      ).new
  end

  class Default
    def order(data)
      data
    end
  end

  class Random
    def order(data)
      data.shuffle
    end
  end

  class MostlyRandom
    def order(data)
      data[0...-1].shuffle << data.last
    end
  end
end

Moving the factory method to the Order module makes it natural to change its name from #orderer_for (as in Example 6 line 3) to #new (above, line 2).  The #new method of Order takes a symbol for an argument and returns the right 'orderer'.  You need not care about the class of the returned object; the thing you get back responds to #order and that's good enough.

Here's a complete listing of the current code.

###Example: Complete

class House
  DATA = [
    'the horse and the hound and the horn that belonged to',
    'the farmer sowing his corn that kept',
    'the rooster that crowed in the morn that woke',
    'the priest all shaven and shorn that married',
    'the man all tattered and torn that kissed',
    'the maiden all forlorn that milked',
    'the cow with the crumpled horn that tossed',
    'the dog that worried',
    'the cat that killed',
    'the rat that ate',
    'the malt that lay in',
    'the house that Jack built',
  ]

  attr_reader :pieces

  def initialize(orderer)
    @pieces = orderer.order(DATA)
  end

  def recite
    (1..pieces.length).map {|i| line(i)}.join("\n")
  end

  def line(number)
    "This is #{phrase(number)}.\n"
  end

  private
  def phrase(number)
    pieces.last(number).join(" ")
  end
end

module Order
  def self.new(choice)
    const_get(
      (choice || 'default').to_s.split('_').map(&:capitalize).join
      ).new
  end

  class Default
    def order(data)
      data
    end
  end

  class Random
    def order(data)
      data.shuffle
    end
  end

  class MostlyRandom
    def order(data)
      data[0...-1].shuffle << data.last
    end
  end
end

class Controller
  def play_house(choice = nil)
    House.new(Order.new(choice)).line(12)
  end
end

puts "\n----\n"               + Controller.new.play_house
puts "\n--:random--\n"        + Controller.new.play_house(:random)
puts "\n--:mostly_random--\n" + Controller.new.play_house(:mostly_random)

This refactoring is complete, and I have just one final thought before we return to the original problem of 'code shapes'.  

I totally understand that this is a small example and that these techniques can feel like overkill for a problem of this size.  Perhaps they are; I wouldn't resist if you insisted it were so.  However, there are bigger problems for which these techniques are the perfect solution and I rely on your ability to see the larger abstraction.  You can't choose whether to use these techniques unless you know them and it's much easier practice on a small example like this.  

###Example 2: Reprise And now, back to the idea that some code shapes are better than others.  Here's a reminder of Example 2, the code that was written to meet the first new requirement.

class House
  # ...
  def initialize(random)
    @pieces = DATA.shuffle if random
  end

  def recite
    (1..pieces.length).map {|i| line(i)}.join("\n")
  end

  def line(number)
    "This is #{phrase(number)}.\n"
  end

  private
  def phrase(number)
    pieces.last(number).join(" ")
  end

  def pieces
    @pieces ||= DATA
  end
end

At first glance this code seems fine but its shape hides objects that we found during the refactoring.  Line 4 hides Order::Random and line 21, Order::Default.

We can easily expose these objects by rewriting the code in a more explicit, straightforward way.  The code below replaces the #pieces method with an else branch in the if statement and adds an attr_reader for @pieces.

class House
  # ...
  attr_reader :pieces
  def initialize(random = false)
    @pieces =
      if random
        DATA.shuffle
      else
        DATA
      end
  end

  def recite
    (1..pieces.length).map {|i| line(i)}.join("\n")
  end

  def line(number)
    "This is #{phrase(number)}.\n"
  end

  private
  def phrase(number)
    pieces.last(number).join(" ")
  end
end

Once the if statement on line 6 is written this way we can see that it uses the value of the boolean random to choose the algorithm to apply to DATA.  This is a form of primitive obsession.   The booleans true and false should be replaced by more robust 'orderer' objects that provide these algorithms and which are injected into House in their stead.

The arrangement of the code in the original Example 2 hides these objects, the code above reveals them.

###Summary

Code shape matters, especially when it comes to conditionals.  Dividing a conditional into multiple parts and placing those parts far apart makes it hard to see underlying objects.  The opposite is also true; clarity can be achieved by hunting down all the parts of a conditional and putting them back together.  

When conditionals are shaped correctly it's easy to see and extract missing objects.  Once extracted, these more robust objects can be re-injected in place of the original primitives.  When House was injected with an 'orderer' it became both more consistent and more flexible.  The likelihood that it will be forced to change went down and its ability to collaborate with objects it knows little about went up.

And finally, the 'default' is often just another kind of specialization.  Negative space is as valid as positive; in the Rubin Vase image the vase and the face are equally real.  Recognizing that the default case is in the same category as all the other specializations allows you to inject an object that does the right thing, and objects that can be trusted to do the right thing make everything easier.


This exercise was extracted from my Practical Object-Oriented Design course, which is chock full of stuff like this.

_ Schedule a private course._

_ Sign up for my newsletter, which contains random thoughts that never make it into blog posts._

###Notes [1] This code is on github.

[2] This Is the House That Jack Built is a cumulative tale. Cumulative tales are like cumulative songs which in turn are one wikipedia hop from the complexity of songs which in link to the article on computational complexity theory. Tales and songs are great as examples because they let us practice dealing with complexity without requiring that we learn about revolving bank loans or shipping containers.  They provide surprisingly complex problems within simple, well-known domains.

News:

(updated July 7, 2017)

Upcoming Public POOD course in beautiful North Carolina

I'm pleased to announce that I'll be teaching a public Practical Object-Oriented Design course in Durham, NC on Oct 25-27, 2017, which we're fondly referring to as POODNC. Join me and a group of your like-minded peers for three days of collaboration, discussion, and instruction.

The leaves will be turning. Minds will be blown. You can't go wrong.

Tickets are on sale now!

99 Bottles of OOP Book

In case you missed the official announcement, 99 Bottles of OOP is complete, and version 1.0 now available. The book is co-authorized by Katrina Owen, and was years in the painful and painstaking making. Learn more about it, read an extended sample, peruse independent reviews, or buy it now.

Posted on September 9, 2014 .