Some Abstract Talk

Applicatives are too restrictive, breaking Applicatives and introducing Functional Builders

This post aims to document a practical design implementation we came up with when designing some APIs in Scala. Of course the concept is not Scala specific and applies to other functional languages.

If you don’t want to go into the full introduction, this post talks about how Applicatives are too restrictive and breaking them into two independent components can be interesting for Contravariant and Invariant Functors. Jump to implementation attached.

We are taking JSON serialization/deserialization as a motivating example.

JsValue is the name of the type that represents the JSON AST. What we need is to be able to read a JsValue and produce an A:

trait Reads[A]{

  def reads(json:JsValue):JsResult[A]

}

Implementing Reads interface provides a way to deserialize a JSON value into an A (and here giving you an opportunity to fail with JsError which is a valid JsResult).

Writing JSON we have its dual:

trait Writes[A]{

  def writes(a:A):JsValue

}

Implementing Writes provides a way to write JSON for a type A.

We have also a type that represents both, being able to read and write a JSON for an A type:

trait Format[A] extends Reads[A] with Writes[A]

Reads happens to be a monad, which is interesting but we are not going to talk about this much more. What we are interested in is having a simple way of combining JSON serializers, meaning if we have

val readAge: Reads[Int]
val readName: Reads[String]

it could be very interesting to have a Reads[(Int,String)] or even better, a Reads[User] having for instance case class User(age:Int, name:String)

This is actually quite easy to achieve with Applicative Functors. To simplify, Applicative Functors are Functors (having the map or fmap function implemented) together with the Applicative part which adds the power we need to achieve the combination we want to do. Explaining Applicative Functors is out of scope and there is a lot of material on the web explaining these.

val readUser: Reads[User] = readAge.and(readName).apply(User)

// or using symbolic methods:

val readUser: Reads[User] = (readAge ~ readName).apply(User)

This composition simply means that if we have two readers of two different types, they could read from the same JSON and provide two values which I can pack into another type (User here).

Now that we solved our problem of combining the Reads with Applicative Functors, wouldn’t it be even more interesting if we could combine similarly the Writes? Having:

val writeAge: Writes[Int]
val writeName: Writes[String]

then we could have a Writes[User], or can we?

Almost. If we have two writers (capable of writing two different JsValue from two different types, Int and String here) then we end up with two JsValues, and there is no way of merging these into one, except if they are both JsObjects.

Let’s do a new type that only writes JsObject

trait OWrites[A] extends Writes[A]{

  def writes(a:A):JsObject

}

Now it makes sense to have one writer out of our two writes above ( writeAge, writeName) by simply merging the resulting objects. So logically it makes sense, except that we can’t use our Applicative based builder since an OWrites is not a Functor at all!

Actually our OWrite is, naturally, the dual of a Functor, a CoFunctor (or Contravariant). Put simply, this means that to change the A type to B in OWrites we should provide a function backwards from B => A. This makes a lot of sense since if we know how to write an A and we know how to transform a B to A then we can write a B. Instead of fmap we got a contramap

def contramap[B](B => A):OWrite[B]

So or status here, we know how to transform the type of OWrites, we know how to merge two OWrites but all of that doesn’t help us to use the builder which based on an Applicative Functor.

What if we split our problem into two components. Let’s introduce a new type that we call FunctionalCanBuild:

trait FunctionalCanBuild[M[_]]{

  def apply[A,B](ma:M[A],mb:M[B]):M[(A,B)]

}

Meaning for a given container M, we can some how compose two instances into one with the tuple of the type parameters. So for Reads it will be:

val fpReads = new FunctionalCanBuild[Reads]{

  // read the same value using the two readers and return a tuple
  def apply[A,B](ra:Reads[A],rb:Reads[B]):Reads[(A,B)] = ...

}

and for the OWrites it will make:

val fpOwrites = new FunctionalCanBuild[OWrites]{

  /* write two objects using the two writers
  and merge the two resulting JsObjects */
  def apply[A,B](oa:OWrites[A],ob:OWrites[B]):OWrites[(A,B)] = ...

}

But that gets us only half way through what we want to achieve. What we want is actually a Reads and OWrites of User and not tuple of Int, String.

For the Reads[(Int,String)] we can solve this problem easily using the fmap function of Functor. But even for OWrites[(Int,String)] it is easy, OWrites is a CoFunctor (or ContraVariant) and we can use contramap to get back from (Int,String) to User.

So it seems that separating the problem into two sub problems is working for us. But what about format? Format is not a Functor and a Contra, it is actually Invariant. This means that it need functions in both directions, A => B and B => A to transform the A type to B.

Like the OWrites we will use OFormat:

trait OFormat[A] extends OWrites[A] with Reads[A]

Now we can implement FunctionalCanBuild[OFormat] :

val fbf: FunctionalCanBuild[OFormat] = new FunctionalCanBuild[OFormat]{

  // merge reads and writes
  def apply[A,B](fa: OFormat[A], fb: OFormat[B]): OFormat[(A,B)] = ...

}

and once we have the OFormat[(Int,String)] we can transform it to a User by using inmap and passing (Int,String) => User and User => (Int,String).

This led us to an API that is capable of building compositions for Functors (Reads), CoFunctors (Writes) and Invariant (Format). Using the FunctionBuilder type class and the correct Variance we can offer a nice unified API for composing different abstractions.

Bottom Line: By breaking Applicatives into two independent components (FunctionalCanBuild and Variance we could use them for more than Functors, ie Covariant and Invariant Functors.

ps: FunctionalCanBuild[M[_]] may look like a monoid but it is not exactly one.

ps2: This is a work I did together with @mandubian and other Zenexity guys

ps3: The Format,Reads and Writes approach is copied and adapted from @debasishg on sjson

ps4: This work is integrated into Play’s included JSON library

ps5: A potentially more readable version as a gist