Hugo Ribeira

C’mon Ruby People

2016-11-08T00:00:00+00:00

The Ruby community is one of the best I’ve ever had the chance to participate in. We’ve got strong values on code quality, tests and best practices but sometimes it feels like a religion.

By now, most of you have heard DHH preaching about this N+1 being a feature. If not, just read this and come back after.

The gist of the issue is:

When a page (a blog index for instance) starts to slow down, the first thing
to look for is N+1 queries. We solve that using :include to include every
relation that is referenced on our views.

If after all of our N+1 queries are solved our app is still slower than
what we would like, we then reach out for view caching. But our views are
cached in our .html.erb files, meaning our :include statement will still
run and if we're including lots of relations this will still slow down our
request even if we hit the cache.

DHH basically says, remove the :include statement and let your uncached
requests hit the N+1 queries.

To me this sounds a little worrying. I mean, we all have done hacks like this to save our asses in production and that’s ok. Software development is all about the trade-off of what you can build in the time frame that it actually is useful.

The problem with this trade-off in particular is that the worst case-scenario (stale cache) might be extremely slow.

On one case at Nourish, the difference between a request with and without including the referenced models would drive the response time from around 600 ms to 45 seconds.

Let’s put that in perspective:

No cache, referenced models NOT included: 45 s (god, that’s slow)
No cache, including referenced models: 600 ms (ok’ish, but slow)
Cache, including referenced models: 500 ms (not much faster)
Cache, not including referenced models: 80 ms (best case) / 45 seconds (worst case)

This can’t be something that we preach as a feature or the right way to do caching. And what worries me the most is that as a community we’ve mostly embraced this sort of hacks under the umbrella of the “Rails way” without asking if it really is something we should be doing.

Ok, enough ranting. Let’s take a look at what might be wrong and a possible solution to fix it without the original workaround drawbacks.

Smell #1 - Why U No Faster?

The first thing that starts to stink with this approach is the fact that the cache layer is not helping at all. This happens because for this particular request the most costly operation is instantiating all ActiveRecord objects and their associations, not rendering the view.

Caching is about saving the results of an expensive computation that is pure (given the same input, it will always produce the same output with no side-effects).

If the cached version is not faster than the initial version, then we’re probably caching at the wrong level or the computation is not expensive at all.

Smell #2 - But you said I should include all the things

Removing the included associations might mean a substantial slow down of the request for the case where we’re not hitting the cache (remember 600 ms is faster than 45 s).

If your application has frequent writes, hitting a stale cache might not be rare at all and be frustrating a big number of your users.

Smell #3 - Cache is logic

Caching mechanisms are part of your application logic, they are conditionals. The same way you don’t want logic in your views, you probably don’t want cache either.

This doesn’t mean you should never have cache in your views though. Be pragmatic. The same way an if statement on your view won’t render you application a nightmare to maintain, a cache block won’t either.

Just keep in mind that there are other alternatives, specially when what you want to cache might not be the view at all but the code that is producing the input for your view layer (an expensive query for instance).

Ok. How do we solve it then?

I know I started this with a rant, but it might just be the case that for you, introducing cache and removing preloaded associations is good enough. If your worst case scenario is fast enough, than there’s no good reason to introduce extra complexity (but in that case, why did you need cache anyway?).

When in doubt, measure.

Putting the cache in its place

Since in this case the most expensive thing we’re doing is query’ing and instantiating records, we need to move the cache to the point where this operation is being computed. Which in this case would be the controller.

The problem with that approach is that we would be stuck with caching a single level (no russian doll cache for you).

So what I like to do instead, is to create small components for this. Bare with me, I’m not going all React on you.

These components will preload associations if needed and contain your view logic as well, essentially keeping all logic out of your views. You get to keep your view logic tidy, nestable components and a great place to cache expensive work as well.

Hmmm. Components?

Let’s imagine a simple blog index page and build a component for the posts list.

A component is simply a small PORO that will render a given view:

class BlogPostsComponent
  attr_reader :blog_posts

  def initialize(blog_posts)
    @blog_posts = blog_posts
  end

  def render
    @blog_posts = blog_posts.includes(:author)
    render_to_string 'components/blog_posts.html.erb', locals: { component: self }
  end
end

As you can see, there’s nothing scary or complex here. We’re just bringing the concept of a partial to our code.

Our partial view will stay exactly the same:

<% component.blog_posts.each do |post| %>
  <h1><%= post.title %></h1>
  <div><%= post.body %></div>
  <div><%= post.author %></div>
<% end %>

Your controller won’t change dramatically:

class BlogPostsController < ApplicationController
  # BEFORE - The Rails way
  def index
    @posts = BlogPost.all.includes(:author)
  end

  # AFTER - With components
  def index
    posts = BlogPost.all
    @posts_component = BlogPostsComponent.new(posts)
  end
end

And your view will still be much like rendering a partial:

<!-- BEFORE - The Rails way -->
<h1>All Posts</h1>
<%= render @posts %>

<!-- AFTER - With components -->
<h1>All Posts</h1>
<%= @posts_component.render %>

With components you get a place for you logic view that you can easily test and maintain. Your views will be as dumb as possible and as you will see in a moment you’ll get a place where you can use your cache without introducing N+1 problems.

Caching done right

Now that we’ve got our component set up, caching our posts is as simples as:

class BlogPostsComponent
  def render
    Rails.cache.fetch(@blog_posts) do
      @blog_posts = blog_posts.includes(:author)
      render_to_string 'components/blog_posts.html.erb', locals: { component: self }
    end
  end
end

As you can see, we’re including the associations inside the cache block. This means that if we hit the cache no association preloading will take place, but if we miss it we will still preload the authors.

So basically, you keep the cake and eat it too.

Further improvements

Now that we’ve got our components working, we can clean this up a bit by introducing an ApplicationComponent:

class ApplicationComponent
  include ActionView::Rendering

  protected

  def cache(key, &block)
    Rails.cache.fetch(key, &block)
  end

  def render_template(template)
    render_to_string(template, locals: { component: self })
  end
end

So that our BlogPostsComponent becomes:

class BlogPostsComponent < ApplicationComponent
  attr_reader :blog_posts

  def initialize(blog_posts)
    @blog_posts = blog_posts
  end

  def render
    cache(blog_posts) do
      @blog_posts = blog_posts.includes(:author)
      render_template 'components/blog_posts.html.erb'
    end
  end
end

If our individual blog entries start to grow in complexity we might want to create a component for them as well:

class BlogPostComponent < ApplicationComponent
  attr_reader :blog_post

  LENGTH_CAP = 255

  def initialize(blog_post)
    @blog_post = blog_post
  end

  def title
    blog_post.title.titleize
  end

  def body
    return blog_post.body if blog_post.body.length < LENGTH_CAP
    "#{blog_post.body[0..max_length - 3]}..."
  end

  def author_name
    blog_post.author.name
  end

  def render
    render_template 'components/blog_post.html.erb'
  end
end

This way we’re able to get rid of all the logic in our views, which will improve both maintainability and testability.

Summing it up

The Rails Way is great and provides great agility in the beginning of a project, but as our code base grows in size and complexity we need a different set of abstractions to cope with that.

In the case of views, components are a great way to structure your code from the controller to the view level:

Components are a good pattern if your views are growing larger and more complex. They are the best place to cache parts of your pages and to put your logic view.
Preloading associations is usually tightly coupled with your views, so moving them from the controller to the component (basically a presenter that knows how to render its view), means you’re grouping two bits of logic that usually change at the same time - this is a good indicator that your classes are following the SRP.
Cache in your views is NOT the root of all evil. But keep an eye open for smells such as degraded performance for worst case scenarios.

Refactoring Rails models with Database Views

2016-03-11T00:00:00+00:00

For the past couple of weeks at Nourish we’ve been tackling shifts and rota planning.

Taking the chance to over simplify the problem, rotas are all about allocating people to shifts on a given date. That means that in terms of data structures you basically have a table for shifts that contains time (but not date) information and allocations that belong to shifts and have date (but not time) information.

The data model

In terms of Rails models you end up with something like this:

class Shift < ActiveRecord::Base
  has_many :allocations, dependent: :destroy

  # DB Fields:
  #
  # start  => String (format: 'HH:MM')
  # finish => String (format: 'HH:MM')
end

class Allocation < ActiveRecord::Base
  belongs_to :shift

  has_many :user_allocations, dependent: :destroy
  has_many :users, through: :user_allocations

  # DB Fields
  #
  # date => Date
end

class User < ActiveRecord::Base
  has_many :user_allocations, dependent: :destroy
  has_many :allocations, through: :user_allocations
end

Even though the data model is quite simple it presents a few challenges when we actually start querying for data.

Imagine you want all allocations for the 1st of January active at 10:00 am:

1st_jan = Date.parse('01-01-2016')
Allocation
  .joins(:shift)
  .where(date: 1st_jan)
  .where("shifts.start <= :time && :time < shifts.finish", time: '10:00')

Simple enough right? That’s until a new requirement comes: We need to support shifts that don’t start and finish in the same day (imagine a night shift starting at 22:00 and ending at 06:00 of the following day).

Data-wise that’s easy to do. We remove the limitation where shift.start <= shift.finish and assume that if the finish time is before the start time then the shift spawns midnight and roll’s over to the next day.

While it is still possible to query this data using raw SQL and ActiveRecord, you’ll start accumulating big chunks of SQL everytime you have to join allocations on shifts time.

To prove my point, it would look like something like this:

1st_jan = Date.parse('01-01-2016')
Allocation
  .joins(:shift)
  .where(
    <<-SQL
      (
        allocations.date = :date
          AND shifts.start <= :time
          AND shifts.finish > :time && :time < shifts.finish
      )
      OR
      (
        allocations.date = :date
          AND shifts.start > shifts.finish
          AND shifts.start <= :time AND
          AND ...
      )
    SQL
    , # I won't even bother trying to write the rest here.
    date: 1st_jan,
    time: '10:00'
  )

This is error prone, unfriendly and just painful. And I haven’t even tryed to query intervals that fit into a certain shift allocation. This is just plain wrong.

If only we had time intervals in the same table rather then one table with dates and another one with time, if only…

Views to the rescue

Database views let you simplify data access through virtual tables that can be composed of any data you can query. Turns out they’re a great way to simplify complex queries and fortunately they’re completly transparent to ActiveRecord as they look just like a normal table (at least for reads, not writes).

Database views are not a shiny new thing. They’ve been around for decades but for some reason Rails developers seem to have forgotten about them.

Bare with me and take a look at this patch of SQL:

SELECT
  allocations.id,
  allocations.date,
  allocations.created_at,
  allocations.updated_at,
  allocations.shift_id,
  (allocations.date || ' ' || shifts.start)::timestamp as start_at,
  (
    CASE WHEN (shifts.start > shifts.finish) THEN
      (allocations.date || ' ' || shifts.finish)::timestamp + interval '1' day
    ELSE
      (allocations.date || ' ' || shifts.finish)::timestamp
    END
  ) as finish_at
FROM
  shift_allocations
INNER JOIN shifts ON shifts.id = shift_allocations.shift_id

This select statement will compose a new database view that we’ll call AllocationsWithTime. As you can see, instead of leaking business logic of rolling the day over when the shift start is greater than the shift finish time we encapsulate that in a view.

To actually instantiate your View take a look at scenic from thoughbot, it as simple as creating a migration:

class CreateAllocationsWithTime < ActiveRecord::Migration
  def change
    create_view :allocations_with_time
  end
end

and your model would look like:

class AllocationWithTime < ActiveRecord::Base
  belongs_to :shift

  scope :for, -> (datetime) do
    where(
      <<-SQL
        allocations_with_time.start_at <= :datetime
          AND allocations_with_time.finish_at > :datetime
      SQL
      ,
      datetime: datetime
    )
  end

  private

  def read_only?
    # Prevent trying to write to the view as that
    # would result in a postgres error anyway.
    true
  end
end

The best part about this is that we can now use ActiveRecord to query our allocations in a simple way. Because we have datetime’s instead of split date and time information, interval query is simple and we don’t need to worry about the midnight crossing:

1st_of_jan_at_10_am = Time.zone.parse('01/01/2016 10:00')
AllocationWithTime.for(1st_of_jan_at_10_am)

No SQL trains, no repeated business logic across queries just simple ActiveRecord queries.

Sometimes all you need to simplify your SQL is a bit of SQL. If you enjoyed this and want to get in touch give me a shout at @eidgeare.

DRY’ing Elixir Tests with Macros

2016-02-15T00:00:00+00:00

I’ve been experimenting with Elixir/Phoenix for a while now and I must say it has been an excellent journey. Phoenix brings Rails happiness and productivity without imposing to much of its ideology upon you. Which to be honest is kind of refreshing.

This week I set up user authentication using Guardian and, as an avid RSpec user, I wondered what the shared_examples_for [1] would like in Elixir.

If you don’t know RSpec, shared_examples_for let you share test cases between different test collections. One common use for such would be making sure your endpoints are being authenticated correctly.

It looks mostly like this:

# Shared Code
module AuthenticatedEndpoint
  shared_examples_for ‘AuthenticatedEndpoint’ do |method, api_path|
    it “returns unauthorized when no authentication token is given” do
      send :method, api_path, nil, nil # body, headers
      expect(response.status_code).to eq 401
    end
  end
end

# Test Case
describe SomeEndpointController do
  # setup
  behaves_like “AuthenticatedEndpoint”, :get, some_path
  # the rest of your test code
end

Elixir

A lifetime with Ruby has grown in me a distaste for DSLs, albeit good looking, when used too often they tend to obscure simple concepts.

With Elixir, I didn’t want to add another DSL just for sharing tests across units. A spice of metaprogramming should be all that’s needed to enable dynamic test generation at compile time, plus, I’ve been wanting to mess with Elixir’s macro system for a while now and this seemed like the perfect excuse for it.

Let’s start with a simple, vanilla exUnit test:

defmodule MacroTests.BiciclesControllerTest do
  use MacroTests.ConnCase

  alias MacroTests.Bicycle

  test “creating a bicycle requires authentication”, %{conn: conn} do
    assert conn |> post(conversation_path(conn, :create)) |> json_response(401)
  end

  test “listing bicycles requires authentication”, %{conn: conn} do
    assert conn |> get(conversation_path(conn, :index)) |> json_response(401)
  end

  # ...

end

While this works, my OCD can’t bare the amount of repetition that’s going around. And this is only one endpoint that we’re talking about… What happens if I decide to return a 404 instead of a 401? How many files will I have to go through to make my tests reflect that?

It just doesn’t seem right.

What I would prefer is to have that authentication test logic abstracted away somewhere… Something that I could use like this:

defmodule MacroTests.BiciclesControllerTest do
  use MacroTests.ConnCase

  test_authentication_required_for(:post, :bicycle_path, :create)
  test_authentication_required_for(:get, :bicycle_path, :index)

  # ...

end

To have ExUnit run dinamically generated tests, we need to turn test_authentication_required_for(:post, :bicycle_path, :create) into test "test doc string", do: test_stuff clause at compile time.

Luckily enough, Elixir provides us both ways of dinamically evaluating functions and writing macros that are expanded at compile time for you. The following paragraphs will cover how to write such a dinamically generated test using Elixir’s metaprogramming features.

Dynamic function calling

The first step we need to accomplish to generate our test cases is to take the path_name argument and dinamically invoke it from Router.Helpers at run-time.

Elixir gives you the ability to call a function dynamically using Kernel.apply/3. This function takes a module, a function name and a list of arguments the function will be invoked with.

To evaluate a path at runtime from its name all we need to do is:

path_action = :get
path_name = :bicycle_path

path = apply(MacroTests.Router.Helpers, path_name, [ConversationApi.Endpoint, path_action])

Knowing the path, we’ll have to make the actual request.

To simulate a request going through a plug in a test, Phoenix provides you with get, post, put, delete, (...) methods. These are just macros that call Phoenix.ConnTest.dispatch\5 with the given method name as an argument, a bit like so:

  defmacro get(path, options), do: dispatch(conn, @endpoint, :get, path, options)

Then to dinamically trigger a request all we would need to do is:

def make_request(path_name, path_action) do
  path = apply(MacroTests.Router.Helpers, path_name, [ConversationApi.Endpoint, path_action])
  dispatch(conn, @endpoint, path_action)
end

Putting it all together

Now that we can retrive the paths and make requests dinamically, all that’s left is generating test cases at compile time and Elixir macros let you do just that.

defmodule MacroTests.AuthConnTest do
  alias MacroTests.Router

  import Plug.Conn, only: [delete_req_header: 2]
  import Phoenix.ConnTest, only: [dispatch: 5, json_response: 2]

  defmacro test_authentication_required_for(method, path_name, action) do
    quote do
      test "requires authentication", %{conn: conn} do
        method    = unquote(method)
        path_name = unquote(path_name)
        action    = unquote(action)
        assert make_unauthenticated_request(conn, @endpoint, method, path_name, action)
      end
    end
  end

  def make_unauthenticated_request(conn, endpoint, method, path_name, action) do
    path = apply(Router.Helpers, path_name, [conn, action])
    conn = conn |> delete_req_header("authorization")
    dispatch(conn, endpoint, method, path, nil) |> json_response(401)
  end
end

The quote method takes your Elixir code and converts it into an AST. While unquote let’s you access variables defined outside of the quote block.

The best I’ve seen it explained (by Chris Mccord) is to think of quote like a string for code where unquote lets you do string (in this case code) interpolation. This way you can access computed properties inside your block of otherwise static code.

Explanations aside, it’s a case of importing MacroTests.AuthConnTest on your controller tests and you should be good to go.

defmodule MacroTests.BiciclesControllerTest do
  use MacroTests.ConnCase

  import MacroTests.AuthConnTest

  test_authentication_required_for(:post, :bicycle_path, :create)
  test_authentication_required_for(:get, :bicycle_path, :index)

  # ...

end

Happy DRY testing! :)

If you want to know more about Elixir and Metaprogramming Chris Mccord Metaprogramming Elixir Talk is an excellent start. It covers the basics and some real use case scenarios (such as Ecto and Logging utilities).

If you liked what you read and want to get in touch, give me a shout at @eidgeare.

Building an API Facade with Elixir

2015-12-19T00:00:00+00:00

Coming from Ruby and growing a bit tired of resorting to caching every time things get a bit more complicated than a simple CRUD application, I’ve been meaning to try Elixir and its web framework Phoenix for a while.

Elixir is the new cool kid in the block. It’s built on top of the Erlang and uses most of its concurrency and failure handling models.

Needless to say, it’s blazingly fast!

Building API’s with phoenix

With the market revolving more and more around smartphone and multi-client applications, I’ve been writing API based applications for a while now.

Building an API with Phoenix is quite straightforward and, if you came from Rails, you’ll find many of the same concepts and ideas with a fresh touch of functional programming on top of it.

This post will cover the serialization of your objects into beautifully crafted JSON (or any other protocol if that matters).

Phoenix has the concept of views whose responsibility is to expose variables and methods to be used when rendering templates.

defmodule HelloPhoenix.UserView do
  use HelloPhoenix.Web, :view

  def render("index.json", %{users: users}) do
    Enum.map(users, fn user -> %{id: user.id, name: user.name} end)
  end
end

While this works fine for HTML templates, it felt awkward for serializing objects into JSON. I felt like I shouldn’t worry about mapping over an array of objects, or wrapping the response with a root key every time I wrote a new serializer. Also, reusing these views for including nested objects wasn’t easy either.

That being said, I ended up writing a small library to deal with this.

Building an API Facade Library

When writing Ruby, I usually resort to either ActiveModel Serializers or Grape Entities, but this time around I wanted a simpler, more lightweight approach to go with the functional style Elixir has to offer.

I started by defining the way I wanted to write my serializers knowing that I wanted them to be format agnostic (they shouldn’t really care whether we’re rendering JSON or XML) and that a single method should handle both a single resource or a list of resources.

Writing and using a serializer should look like something like this:

defmodule HelloPhoenix.UserSerializer do
  alias HelloPhoenix.User

  def root, do: “user”
  def root_plural, do: “users”

  def map(resources)
    # rendering logic
  end

  def serialize(user) do
    %{ id: user.id, name: name(user) }
  end

  defp name(user) do
    “#{user.first_name} #{user.last_name}”
  end
end

> user = %User(id: 1, first_name: “Hugo”, last_name: “Ribeira”)
> UserSerializer.map(user)
%{client: %{id: 1, name: “Hugo Ribeira”}}
> UserSer ializer.map([user, user])
%{clients: [%{id: 1, name: “Hugo Ribeira”}, %{id: 1, name: “Hugo Ribeira”}]}

Once defined the interface, I went on with the implementation:

defmodule HelloPhoenix.UserSerializer do
  # (...)
  def map(list), when: is_list(list) do
     list
       |> Enum.map(fn item -> serialize item end)
       |> Serializer.wrap_with_root(__MODULE__)
  end
  def map(item) do
    Serializer.wrap_with_root serialize(item), __MODULE__
  end

  defp wrap_with_root(list, serializer), when: is_list(list) do # when non empty list
    Map.put(%{}, UserSerializer.root_plural, list)
  end
  defp wrap_with_root(item, serializer) do # when single item
    Map.put(%{}, UserSerializer.root, item)
  end

  # (...)
end

Elixir really shines here. Pattern matching and guard clauses give you some type safety at compile time and avoid gigantic if clauses (that will be evaluated at runtime) and the functional style provides a simple and idiomatic way to describe the serialization process as a pipeline of transformations from your model to the final API response.

We’ve got our implementation and the interface we defined is respected, but we still have the problem of repeating common logic across our serializers. Gladly, Elixir let’s us DRY up our implementation quite easily.

The first step is to remove this duplicated logic into its own module.

defmodule HelloPhoenix.Serializer do
  def map(list, serializer), when: is_list(list) do
    list
      |> Enum.map(fn item -> serializer.serialize item end)
      |> wrap_with_root(serializer)
  end
  def map(item, serializer) do
    wrap_with_root serializer.serialize(item), serializer
  end

  defp wrap_with_root(list, serializer), when: is_list(list) do
    Map.put(%{}, serializer.root_plural, list)
  end
  defp wrap_with_root(item, serializer) do
    Map.put(%{}, serializer.root, item)
  end
end

defmodule HelloPhoenix.UserSerializer do
  def root, do: 'user'
  def root_plural, do: 'users'

  def serialize(user) do
    %{ id: user.id, name: name(user) }
  end

  defp name(user) do
    “#{user.first_name} #{user.last_name}”
  end
end

> Serializer.map(%User{id: 1, first_name: "Jonh"}, UserSerializer)
%{user: %{id: 1, name: "Jonh"}}

That’s better!

We decoupled the logic responsible for root wrapping and mapping from the serialization logic itself but we ended up changing the interface to support this refactor.

Hmmm, that’s not nice!

Glady, Elixir has got us covered its powerful macro system that let’s us inject functionality from a module into another. Next we’ll use these macros to refactor our serializer, maintaining logic decoupled and our interface as described initially.

defmodule HelloPheonix.Serializer do
  defmacro __using__(_) do
    quote do # inject this code when :use macro is used inside a module
      @behaviour HelloPheonix.Serializer

      alias HelloPheonix.Serializer

      def map(list), when: is_list(list) do
        list
          |> Enum.map(fn item -> serialize item end)
          |> Serializer.wrap_with_root(__MODULE__) # Pass extended module to :wrap_with_root
      end

      def map(item) do
        Serializer.wrap_with_root serialize(item), __MODULE__
      end
    end
  end

  @callback map(Struct.t) :: Struct.t
  @callback serialize(Struct.t) :: Struct.t
  @callback root() :: String.t
  @callback root_plural() :: String.t

  def wrap_with_root(list, serializer), when: is_list(list) do
    Map.put(%{}, serializer.root_plural, list)
  end
  def wrap_with_root(item, serializer) do
    Map.put(%{}, serializer.root, item)
  end
end

defmodule HelloPhoenix.UserSerializer do
  use HelloPhoenix.Serializer # Import functionality from Serializer

  def root, do: 'user'
  def root_plural, do: 'users'

  def serialize(user) do
    %{ id: user.id, name: name(user) }
  end

  defp name(user) do
    “#{user.first_name} #{user.last_name}”
  end
end

> UserSerializer.map(%User{id: 1, first_name: "Jonh"})
%{user: %{id: 1, name: "Jonh"}}

The @behaviour and @callback macros are elixir’s ways of defining an interface.

Elixir is dynamically typed, meaning that explicit interfaces are not needed but defining them allows the compiler to assert at compile time that our serializers abide the serialization contract. This avoids weird bugs caused by unexpected return types or necessary but undefined methods.

Wrapping Up

I must say I’m quite impressed with Elixir. Not only it is great to see your endpoints return in microseconds rather then hundreds of milliseconds, as the functional paradigm seems to fit perfectly around the data transformation and request life cycle we work with when developing web applications.

When working with dynamic languages I sometimes miss the ability to explicitly define interfaces, as implicit interfaces do come at the cost of needing more documentation and/or extra runtime checks. Elixir really shines here, as besides supporting implicit interfaces, the mix between pattern matching and explicit interface definition allows for type safety when needed without too much runtime overhead.

Elixir is really fun and I’ll probably end up writing a few more posts about it, if you enjoyed this, do consider following me on twitter @eidgeare!

Getting validations out your models

2015-12-13T00:00:00+00:00

Validations are the Rails way of validating user input and data integrity. They’re usually written inside your model and constrain which states are saveable to the database.

class Human < ActiveRecord::Base
  attr_accessor :name
  validates :name, presence: true
end

# > human = Human.new
# > human.valid?
# => false

# > human.name = 'jonh'
# > human.valid?
# => true

They’re a simple but effective tool to manage your data validation but shouldn’t be abused as a data integrity mechanism. In this post I’m making a case for getting some of your validations out of your model and closer to the user input you’re validating.

Data integrity checks belong in the database

If you want to guarantee a given state (say name presence), than that validation belongs in the database and should be replicated in the application layer for user feedback.

But that’s not DRY, is it?

It isn’t. But your model validations can be bypassed (think Human.save(validate: false)) or there might be a few services using your database directly and therefore bypassing your application data checks. Therefore, the only way to guarantee data integrity is to rely on your database to enforce it.

Data validation belongs in the application

Data integrity and data validation are complementary but two different things.

Say you only want to allow users called “Jonh” to sign up, that’s the kind of validation that belongs in your application code. It is business logic and therefore your database has got nothing to do with it.

A simple misleading example

With rails, that’s quite an easy requirement:

class User < ActiveRecord::Base
  validate :user_is_named_jonh
  
  def jonh?
    first_name.downcase == 'jonh'
  end
  
  private
  
  def user_is_named_jonh
    errors.add(:first_name, :not_jonh) unless jonh?
  end
end

The code above seems like the right way but the truth is that it isn’t.

What happens if you decide you have too many “Jonh”s on your platform and from now on you only want to accept “Paul”s signing up?

That’s simple enough, you say:

class User < ActiveRecord::Base
  validate :user_is_named_paul
  
  def paul?
    first_name.downcase == 'paul'
  end
  
  private
  
  def user_is_named_paul
    errors.add(:first_name, :not_paul) unless paul?
  end
end

Done! Except… you just broke the app for every existing user named “Jonh” as they won’t be able to update their user information unless they change their name to Paul.

The problem here, is that you don’t really want to validate your model data. You want to validate user input upon registration and your model is not a good place to do it. Model validations should be used to ensure your models are in a state your application can handle and in this case, the application code doesn’t really care what the first name of the user is.

The bad solution

class User < ActiveRecord::Base
  validate :user_is_named_paul, if: :new?
  # ...
end

This definitely works! But once you start collecting a few validations like this it starts to be difficult to argue which validations run and when.

If your validations are coupled to a specific action (say user registration) then why not couple it directly to that action instead of the model?

A better solution: Form objects

Form objects have been an essential tool in my day to day with Rails. Not only it makes dealing with complicated forms cleaner (think accepts__nested__attributes, or json fields) it also helps declutter your models and controllers.

Form objects are great for cases like the one we’ve been going over:

class User < ActiveRecord::Base
  validates :first_name, :email, :password_hash, presence: true
end

class NewUserForm < ActiveRecord::Model
  attr_accessor :first_name, :email, :password
  
  validates :first_name, :email, :password, presence: true
  validate :user_is_named_paul
  
  def paul?
    first_name.downcase == 'paul'
  end
  
  def user
    User.new(first_name: first_name, email: email, password: password)
  end
  
  def valid?
    super && user.valid?
  end
  
  def save
    valid? && user.save!
  end
  
  private
  
  def user_is_named_paul
    errors.add(:first_name, :not_paul) unless paul?
  end
end

Now we keep validations that ensure our data is in a state our application can handle in the model and validations that are tied to user registration in a object that sole purpose is validating user input for that specific action (users have name, email and password_hash validations belong in the database as well).

If the requirements change which users are allowed to sign up, only one class needs to be changed. Because the changes are self contained you don’t have to worry about making sure your new validations don’t break in other cases. They will only run on user registration.

Summing up

When writing a validation, try to figure out which layer you’re validating. Are you guaranteing data integrity and therefore you should also write a database constraint? Are you guaranteing you application data is in valid, computable state? Or are you enforcing business logic as data?
Form objects are a good tool to declutter your models and controllers from parameter validation, complex associations forms and to move your validations closer to the user input.
When in doubt, prefer to write your validations in the model and refactor them out to form objects once you’re sure they do not belong there. If you end up instantiating form objects outside of controllers and without user input, you’re doing something wrong.