Introducing Scala to your Java/Ruby Shop

Manish PanditSilicon Valley Code Camp ‘12

Oct 7th, 2012

My experiences at IGN

About me

Manish PanditDirector of Engineering, IGN

@lobster1234

linkedin.com/in/mpandit

About IGNWe are a leading online media and services company obsessed with gaming and entertainment.

56MM Monthly Unique Views737MM Monthly Page Views20MM Monthly Video Views

My Objective

Is:To share my story of introducing Scala to IGN, and (hopefully) show you the path to enlightenment.

Isn't:To incite a functional vs. imperative, or Scala vs. ____ debate.

The IGN Tech Stack

Best tool for the job

• Front End : JS, CSS3, HTML5, backbone, Coffeescript, jQuery

• Middle Tier : ZF2 (PHP), Rails• APIs : Scala, some Java• Persistence : MongoDB, MySQL, Redis• Search : ElasticSearch• Caching : Memcached, Varnish

APIs at IGN : Numbers

• ~5 Billion requests a month• Average Response time of under

20ms on cache misses• 17 APIs between Social, OAuth,

and Content• Hardest hit APIs doing about

25K RPM at peak

The Kitchen Sink

• CMS – JSPs talking to Oracle DB and some Java services

• V1 API : Java Services using Oracle DB with Memcached

• V2 API : Rails based API layer using MongoDB and a Java/Memcached front cache

• Lots of duct tape like messaging and cron jobs for simple tasks

• Too diverse of a stack in the API infrastructure

The API Evolution : V3 [2012]

• A fresh look at the APIs and IGN’s direction• First attempt to – Integrate all content types– Open up the APIs to external consumers– Evolve the CMS, traditionally a monolithic system– ElasticSearch– Varnish Response Caching

• A learning opportunity for traditional Java stack engineers

Roadblocks

• Resistance to change– Culture– Willingness and commitment

• Too many choices – making the right pick– Involve every one– Do not decide by committee, decide by learning– Time box and measure everything– Be accountable, as you’re looked upon as the

expert

The hunt for…

• A runtime as fast, if not faster than Java • Concise, yet expressive• Less boilerplate• Smooth learning curve and ramp up• Re-use ecosystem of libraries • Growing adoption and developer velocity• Built in support for concurrency• Fast and fun development• Next level of programming – disrupt!

Why Scala : Strategic Reasons

• Evolve the talent brand along side evolving the platform

• Establish an API platform by replacing “rewrite” with learning instead of porting

• Start thinking functional• Align with the leaders in the APIs• Get involved with a technology as it is evolving

(vs. established). Helps with influence.

Why Scala : Tactical Reasons

• Performance tied to cores than processor speed, i.e. concurrency

• Ideal for API development – (relatively) simple to handle concurrency and immutability

• JVM based = Performance• Re-use Java toolkits and libraries• Concise, yet expressive code • Actor model makes the difficult parts easy

Why Scala : Tactical Reasons

• Richer Collections API• Traits enabling Mix-Ins and behavior re-use• Statically Typed w/Type inference• Functional, but not alien (it can be!)• REPL

Yes, Virginia, Scala is hard : DPPSo, how can you figure out if Scala will be "easy" or "hard" for your organization:•Your company has speakers at JavaOne, OSCON, Strangle Loop, QCon: Scala will be easy•Lunch-time discussions involve the criteria for moving from a developer to a senior developer: Scala will be hard•Your developers can write code in NotePad if they have to: Easy•Your developers stare blankly or say 3 "Hail Marys" when they hear the name "Zed Shaw": Scala == Hard•Developers all follow Dean Wampler on Twitter: Scala Easy•Your developers come in at 9:15 and leave before 6 and don't check work email at night: Hard

http://blog.goodstuff.im/yes-virginia-scala-is-hard

Picking up Scala : Week 1

• Scala for Java Developers fast-tracked the learning

• Syntax Familiarity• Java without semicolons?

def total(nums: List[Int]) :Int = { var sum:Int=0 for(num<-nums) { sum+=num } sum}

Picking up Scala : Week 4

• Reduce the lines of code, embrace immutability, use the functional paradigms built into the language

• Unmystrify the implicits magic

nums.foldLeft(0)((n,c) => n+c)

Picking up Scala : Today

• Focus on immutability• Think functional – with the knowledge gained

from using the functional aspects of the language, build control abstractions

• Explore and implement ScalaZ• Keep learning, and applying• Move all the Actor processing to Akka

Less Boilerplatepublic class HelloWorld{

public static void main(String... args){System.out.println("Hello, World!");

}}

object HelloWorld extends App{println("Hello, World!")

}

scala> println("Hello, World!")

Expressive, yet conciseval x = if(n%2==0) "Even" else "Odd"

val x = for(i <- 1 to 10) yield i*2

val x = for(i <- 1 to 10; if(i%2==0)) yield i*2

val y = 20 match{case p if(p%2==0) => "Even"case _ => "Odd"

}

val largest = List(1,5,2,6).foldLeft(0)((a,b)=> if(a>b) a else b)

Less pain points

No more dreaded null or throwing unwanted exceptions : Use Option[T]

def thisMayReturnNull[T](x:T):Option[T]={if(true) Some(x) else None

}

No more creating classes only to return more than 1 value : Use Tuples

def doublePair(x:Int)= (x,x*2)

Try-catch messtry{

//Construct a URL which can throw a MalformedURLException

//do something with a URL which can throw IOException//do something with string encoding which can throw //

UnsupportedEncodingException

} catch{ case mal:MalformedURLException => //something case ioe: IOException => //something case e: Exception => //something }

Type Inferencescala> val x = "Sunday"x: java.lang.String = Sunday

scala> def x(n:Int) = n*3x: (n: Int)Int

scala> 1 to 5res2: scala.collection.immutable.Range.Inclusive = Range(1, 2, 3, 4, 5)

scala> def x = if(true) Left("Yes!") else Right(new Exception("Didn’t work out!"))x: Product with Serializable with Either[java.lang.String,java.lang.Exception]

Dynamic Mix-insclass Home(bedrooms:Int=3,bathrooms:Double=2.5)

trait Garage{ def park(n:Int) = println("Parking " + n + "cars")}

val someHome = new Home

val someHomeWithGarage = new Home with Garage someHomeWithGarage.park(2)

Concurrency

• Actors make concurrency so much easier– Messaging vs. Blocking– Lightweight, 300-500 bytes per instance– Isolated

Scala API Components

• Scalatra• Lift-MongoRecord for MongoDB • Casbah for MongoDB• PlayFramework 2• Actors for tasks like syndication• sbt and Maven for Builds• MongoDB for persistence

Other API Components

• Varnish Cache • Elasticsearch• Yammer Metrics (ping, healthcheck)• Swagger (self-documenting RESTful APIs)• 3Scale for Partner APIs• IntelliJ IDEA and Eclipse with ScalaIDE• PlayFramework 2.0 (New!)• ScalaTest for Testing

Challenges with Scala*

• Steepness of the Learning curve depends entirely on your org culture

• Scala is what you make of it• http://scalaz.github.com/scalaz/scalaz-2.9.1-6.0.4/doc.sxr/scalaz/

BKTree.scala.html

• Middle Ground between Simplicity and, well..elegance

* None of these challenges outweigh the benefits. They’re just something to be aware of.

Challenges with Scala

• Slow compilation based on the source• No (binary) compatibility of dependencies for

the major releases (2.8, 2.9, 2.10(?))• Tooling not as rich as Java – Changing (rapidly) with Typesafe investing in

ScalaIDE– ScalaIDE comes with a Scala Worksheet, which is

like REPL but richer

Lessons Learned : Culture

• Culture of Meetups, Hack Nights and Code Jams

• Being able to connect with the contributors and language creators

• Willingness to invest in learning, evolving vs. maintaining

Lessons Learned : Team

• Small team (~5 engineers) • Java background helped a lot• 10% time set aside for skill building• Learn and share• It is a commitment• Communication!

Lessons Learned : Material

• Books that have exercises, so you can quantify your learning.

• Online tutorials• StackOverflow • Coursera : A course on Functional

Programming by Martin Odersky

Resources

• Books– Programming in Scala by Odersky– Scala for the Impatient by Horstmann

• Interwebz– http://debasishg.blogspot.com– http://www.reddit.com/r/scala– http://stackoverflow.com/tags/scala/info– http://twitter.github.com/scala_school– https://www.coursera.org/course/progfun

scala> case class Me(name:String="Manish Pandit",twitter:String="@lobster1234")defined class Me

scala> val me = new Meme: Me = Me(Manish Pandit,@lobster1234)

scala> println(“Questions!”)Questions!