Leveraging Scala Macros for Better

Validation Tomer Gabel, Wix

March 2015

I Have a Dream •  Definition:

case  class  Person(      firstName:  String,      lastName:  String    )    

implicit  val  personValidator  =        validator[Person]  {  p  ⇒          p.firstName  is  notEmpty          p.lastName  is  notEmpty      }  

I Have a Dream •  Usage:

validate(Person("Wernher",  "von  Braun"))        ==  Success    

validate(Person("",  "No  First  Name"))        ==  Failure(Set(RuleViolation(                value              =  "",                constraint    =  "must  not  be  empty",                description  =  "firstName"            )))

ENTER: ACCORD.

Basic Architecture

API

Combinator Library

DSL

Macro Transformation

The Accord API

•  Validation can succeed or fail •  A failure comprises one or more violations

sealed  trait  Result  case  object  Success  extends  Result  case  class  Failure(violations:  Set[Violation])      extends  Result

•  The validator typeclass:

trait  Validator[-­‐T]  extends  (T  ⇒  Result)

Why Macros?

•  Quick refresher: implicit  val  personValidator  =      validator[Person]  {  p  ⇒          p.firstName  is  notEmpty          p.lastName  is  notEmpty      }  

Implicit “and”

Automatic description generation

Full Disclosure

Macros are (sort of) experimental

Macros are hard

I will gloss over a lot of details

… and simplify many things

Abstract Syntax Trees

•  An intermediate representation of code

– Structure (semantics)

– Metadata (e.g. types) – optional!

•  Provided by the reflection API

•  Alas, mutable

– Until scala.meta comes along…?

Abstract Syntax Trees def  method(param:  String)  =  param.toUpperCase    

Abstract Syntax Trees def  method(param:  String)  =  param.toUpperCase      

Apply(      Select(          Ident(newTermName("param")),          newTermName("toUpperCase")      ),      List()  )  

Abstract Syntax Trees def  method(param:  String)  =  param.toUpperCase      

ValDef(      Modifiers(PARAM),        newTermName("param"),        Select(          Ident(scala.Predef),          newTypeName("String")      ),      EmptyTree            //  Value  )

Abstract Syntax Trees def  method(param:  String)  =  param.toUpperCase      

DefDef(      Modifiers(),        newTermName("method"),        List(),                  //  Type  parameters      List(                      //  Parameter  lists          List(parameter)      ),        TypeTree(),          //  Return  type      implementation  )

Def Macro 101

•  Looks and acts like a normal function def  radix(s:  String,  base:  Int):  Long  val  result  =  radix("2710",  16)  //  result  ==  10000L  

•  Two fundamental differences: –  Invoked at compile time instead of runtime – Operates on ASTs instead of values

Def Macro 101

•  Needs a signature & implementation def  radix(s:  String,  base:  Int):  Long  =      macro  radixImpl    def  radixImpl      (c:  Context)      (s:  c.Expr[String],  base:  c.Expr[Int]):      c.Expr[Long]

Values

ASTs

Def Macro 101

•  What’s in a context?

–  Enclosures (position)

–  Error handling

–  Logging

–  Infrastructure

Basic Architecture

API

Combinator Library

DSL

Macro Transformation

Overview implicit  val  personValidator  =      validator[Person]  {  p  ⇒          p.firstName  is  notEmpty          p.lastName  is  notEmpty      }    

•  The validator macro: – Rewrites each rule by addition a description – Aggregates rules with an and combinator

Macro Application

Validation Rules

Signature def  validator[T](v:  T  ⇒  Unit):  Validator[T]  =      macro  ValidationTransform.apply[T]              def  apply[T  :  c.WeakTypeTag]      (c:  Context)      (v:  c.Expr[T  ⇒  Unit]):      c.Expr[Validator[T]]

Brace yourselves

Here be dragons

Walkthrough

Search for rule

Process rule

Generate description

Rewrite rule

Walkthrough

Search for rule

Process rule

Generate description

Rewrite rule

Search for Rule

•  A rule is an expression of type Validator[_]

•  We search by:

– Recursively pattern matching over an AST

– On match, apply a function on the subtree

– Encoded as a partial function from Tree to R

Search for Rule def  collectFromPattern[R]              (tree:  Tree)              (pattern:  PartialFunction[Tree,  R]):  List[R]  =  {      var  found:  Vector[R]  =  Vector.empty      new  Traverser  {          override  def  traverse(subtree:  Tree)  {              if  (pattern  isDefinedAt  subtree)                  found  =  found  :+  pattern(subtree)              else                  super.traverse(subtree)          }      }.traverse(tree)      found.toList  }

Search for Rule

•  Putting it together: case  class  Rule(ouv:  Tree,  validation:  Tree)    

def  processRule(subtree:  Tree):  Rule  =  ???    

def  findRules(body:  Tree):  Seq[Rule]  =  {      val  validatorType  =  typeOf[Validator[_]]    

   collectFromPattern(body)  {          case  subtree  if  subtree.tpe  <:<  validatorType  ⇒              processRule(subtree)      }  }  

Walkthrough

Search for rule

Process rule

Generate description

Rewrite rule

Process Rule

•  The user writes: p.firstName  is  notEmpty

•  The compiler emits: Contextualizer(p.firstName).is(notEmpty)

Object Under Validation (OUV) Validation

Type: Validator[_]

Process Rule Contextualizer(p.firstName).is(notEmpty)

•  This is effectively an Apply AST node

•  The left-hand side is the OUV

•  The right-hand side is the validation

– But we can use the entire expression!

•  Contextualizer is our entry point

Process Rule Contextualizer(p.firstName).is(notEmpty)

Apply Select

Apply

TypeApply Contextualizer

String

Select Ident(“p”)

firstName is

notEmpty

Process Rule Contextualizer(p.firstName).is(notEmpty)

Apply Select

Apply

TypeApply Contextualizer

String

Select Ident(“p”)

firstName is

notEmpty

Process Rule

Apply

TypeApply Contextualizer

String

Select Ident(“p”)

firstName

Process Rule

Apply

TypeApply Contextualizer

Φ

Select Ident(“p”)

firstName

Process Rule

Apply

TypeApply Contextualizer

Φ

Select Ident(“p”)

firstName

Process Rule

case  Apply(TypeApply(Select(_,  `term`),  _),  ouv  ::  Nil)  ⇒  

Apply

TypeApply Contextualizer

Φ

OUV Φ

Φ

Process Rule

• Putting it together: val  term  =  newTermName("Contextualizer")    

def  processRule(subtree:  Tree):  Rule  =      extractFromPattern(subtree)  {          case  Apply(TypeApply(Select(_,  `term`),  _),  ouv  ::  Nil)  ⇒              Rule(ouv,  subtree)      }  getOrElse  abort(subtree.pos,  "Not  a  valid  rule")  

Walkthrough

Search for rule

Process rule

Generate description

Rewrite rule

Generate Description Contextualizer(p.firstName).is(notEmpty)

•  Consider the object under validation •  In this example, it is a field accessor •  The function prototype is the entry point

Select Ident(“p”)

firstName

validator[Person]  {  p  ⇒      ...  }

Generate Description •  How to get at the prototype? •  The macro signature includes the rule block:  

def  apply[T  :  c.WeakTypeTag]      (c:  Context)      (v:  c.Expr[T  ⇒  Unit]):      c.Expr[Validator[T]]  

•  To extract the prototype: val  Function(prototype  ::  Nil,  body)  =        v.tree          //  prototype:  ValDef

Generate Description  

•  Putting it all together:

def  describeRule(rule:  ValidationRule)  =  {      val  para  =  prototype.name      val  Select(Ident(`para`),  description)  =          rule.ouv      description.toString  }

 

Walkthrough

Search for rule

Process rule

Generate description

Rewrite rule

Rewrite Rule

•  We’re constructing a Validator[Person]

•  A rule is itself a Validator[T]. For example: Contextualizer(p.firstName).is(notEmpty)  

•  We need to: – Lift the rule to validate the enclosing type – Apply the description to the result

Quasiquotes

•  Provide an easy way to construct ASTs:

Apply(      Select(          Ident(newTermName"x"),          newTermName("$plus")      ),        List(          Ident(newTermName("y"))      )  )        

q"x  +  y"    

Quasiquotes

•  Quasiquotes also let you splice trees:

def  greeting(whom:  c.Expr[String])  =      q"Hello  \"$whom\"!"

•  And can be used in pattern matching:

val  q"$x  +  $y"  =  tree

Rewrite Rule Contextualizer(p.firstName).is(notEmpty)   new  Validator[Person]  {      def  apply(p:  Person)  =  {          val  validation  =              Contextualizer(p.firstName).is(notEmpty)          validation(p.firstName)  withDescription  "firstName"      }  }

Rewrite Rule

•  Putting it all together:

def  rewriteRule(rule:  ValidationRule)  =  {      val  desc  =  describeRule(rule)      val  tree  =  Literal(Constant(desc))      q"""      new  com.wix.accord.Validator[${weakTypeOf[T]}]  {          def  apply($prototype)  =  {              val  validation  =  ${rule.validation}              validation(${rule.ouv})  withDescription  $tree          }      }      """  }  

The Last Mile

Epilogue •  The finishing touch: and combinator def  apply[T  :  c.WeakTypeTag]      (c:  Context)      (v:  c.Expr[T  ⇒  Unit]):  c.Expr[Validator[T]]  =  {        val  Function(prototype  ::  Nil,  body)  =  v.tree      //  ...  all  the  stuff  we  just  discussed        val  rules  =  findRules(body)  map  rewriteRule      val  result  =          q"new  com.wix.accord.combinators.And(..$rules)"      c.Expr[Validator[T]](result)  }

WE’RE DONE HERE!

Thank you for listening

tomer@tomergabel.com

@tomerg

http://il.linkedin.com/in/tomergabel

Check out Accord at: http://github.com/wix/accord