Repeating History…on purpose…

with Elixir

Barry Jones

Who am I?

• Developer since 98• Used to run Brightball– Contract programming business using PHP– 2008 – 2011• R.I.P. Code one out for my homies

• PHP, Java, Perl, Python, Ruby, Groovy, Go– Elixir

• And lots of databases #postgresql

So…another language…? Rules for learning a language– Must solve a problem– Problem not addressed by

current stack– “Fast” is not a purpose• Lots of things are fast

– Should be the best solution• If it’s not, I’d rather learn the

one that is…

I hate new languages

Yeah. Surprise.

So why the other languages?• PHP

– Needed to build a site– Clemson’s options were PHP / Perl– Perl was badly supported

• Java– Clemson CPSC– Also…jobs are good

• Groovy– Java environment– Wanted to be productive

• Go– Great concurrency– Relatively simple– Fast compilation– Portability– Solves some bloat problems

• Perl– Sysadmin work– Installed on all *nix derivatives

• including very old– Best at text parsing

• Python– Employer required it

• Ruby– Incredible productivity– Library ecosystem– Monkey patching– Rails– Focus on dev efficiency

SO WHY ELIXIR?I wondered the same thing

"Those who cannot remember the past are condemned to repeat it."

- George Santayana

"What has been is what will be, and what has been done is what will be done, and there is

nothing new under the sun." - Solomon

Elixir isn’t new

• It’s more productive Erlang• Compiles down to the BEAM (Erlang VM)• Erlang was born in 1986 (Linux was 1991)• Erlang/OTP is on version 19• Erlang libraries work directly in Elixir– And vice versa

Processor POWER!!!

• Industry focus used to be on Mhz/Ghz• Growth was steady…but then stalled

• Industry shifted to multicore / concurrency and trying to put it in their languages

• Lookup “Beowulf Cluster” to see how foreign “parallel” was at the time

Funny thing…

“Any sufficiently complicated concurrent program in another language contains an ad hoc informally-specified bug-ridden slow implementation of half of Erlang.”- Robert Virding

Erlang view of the World• Everything is a process.• Processes are strongly isolated.• Process creation and destruction is a lightweight operation.• Message passing is the only way for processes to interact.• Processes have unique names.• If you know the name of a process you can send it a

message.• Processes share no resources.• Error handling is non-local.• Processes do what they are supposed to do or fail.

Message Passing

• Process isolation and message passing• Passing between threads, cores or machines is

transparent• Microservice concept…but everywhere and

not terrible

“If Java is the right one to run anywhere, then Erlang is the right one to run forever.” – Joe Armstrong

What’s the big deal with Elixir?• Ruby-like focus on developer productivity• Functional programming made simple• Embedded database• Compiles down to code to run on the BEAM Virtual Machine• BEAM/OTP is what Erlang runs on• Erlang/BEAM is the best existing language for concurrency, consistency and fault

tolerance, hot code swapping– Single Processor– Multi Processor– Distributed Multi Machine Cluster

• Erlang does not focus on developer productivity– Writing Erlang kinda sucks...

• #1 Problem in Ruby is concurrency model– Cannot be fixed. It’s the way the language works. – Enables great things but also causes limitations

So what does that mean?

Standard Web App OTP

What’s the big deal?

• Facebook paid $22 billion for WhatsApp• WhatsApp had $10 million in revenue• What was the big deal?– Erlang/OTP– 2 million users / server– No central relay point– Scales horizontally– Deploys w/o disconnect

AND HOW DOES ALL THIS WORK?Why should you care?

Other languages

• Boot up• Memory is shared– Where leaks come from– Changing shared memory requires a mutex lock

• Garbage collector periodically runs– Pause entire stack

• Requests run in threads in the same process– Threads are cooperatively scheduled

• Deployment means shutting down current code, starting new code

Erlang/Elixir/OTP

• No memory is shared• Data structures are immutable• Each Erlang process (basically a light thread) has

its own HEAP– Reclaimed on completion

• Code can be hot deployed– New code runs next time it’s accessed (existing code

keeps running)• Processes are prescheduled

Sound familiar?

• Difference is size of the allocations– An Erlang process is 0.5 kb– A Go goroutine is 2 kb (version 1.4)– A Java thread is 1024 kb on 64 bit VM– PHP request varies by how much is loaded• Laravel averages 7-12mb / request

Programming Elixir, Chapter 15Laptop w/ 4 cores and 4gb of RAM counting concurrently1,000,000 processes = • 0.48 gb in Elixir• 1.91 gb in Golang (go routines)• 977 gb in Java (threads)• 6,836 gb in PHP (Laravel requests)

LET’S GET STARTED

wait.…we haven’t started yet?

Quick History• Linux was created in 1991

• Erlang was created in 1987 by Ericson– Powers about half of global telecom– Needed distributed, fault tolerant system– Deploy updates without interrupting existing calls– OTP = Open Telecom Protocol– Erlang/OTP 19.0 release in June 2016

• Elixir was created in 2012 by Jose Valim– Former Rails Core team member– Elixir 1.3 released in June 2016

The HighlightsIt’s huge…we don’t have all night

Immutable Data

• There’s no passing pointers• Add something to a list, get a new list• Everything is “message passing”– Avoids mutex locks– Enables per-process garbage collection– Makes calling a function locally, in another process

or on another machine transparent

3 Databases Built In• ETS – Erlang Term Storage

– In memory table storage for a node

• DETS – Disk-based Erlang Term Storage– Disk table storage for a node

• Mnesia - #awesome– A relational/object hybrid data model that is suitable for telecommunications applications. – A DBMS query language, Query List Comprehension (QLC) as an add-on library. – Persistence. Tables can be coherently kept on disc and in the main memory. – Replication. Tables can be replicated at several nodes. – Atomic transactions. A series of table manipulation operations can be grouped into a single

atomic transaction. – Location transparency. Programs can be written without knowledge of the actual data location. – Extremely fast real-time data searches. – Schema manipulation routines. The DBMS can be reconfigured at runtime without stopping the

system.

https://blog.codeship.com/elixir-ets-vs-redis/

Preemptive Scheduling

• Context switching among running tasks and has the power to preempt (interrupt) tasks and resume them at a later time without the cooperation of the preempted tasks.

• Cooperative: Running tasks voluntarily release control

What does that mean?

• Response time consistency• A tight loop or resource heavy process can’t

cannibalize resources• Critical for real time systems• Running a database inside your code would be

unreliable otherwise

Pattern Matching

=iex> list = [1, 2, [ 3, 4, 5 ] ] [1, 2, [3, 4, 5]]iex> [a, b, c ] = list[1, 2, [3, 4, 5]] iex> a 1iex> b 2 iex> c [3, 4, 5]

Examples from Programming Elixir 1.3

Works if left can be matched to rightiex> list = [1, 2, 3][1, 2, 3]

iex> [a, 1, b ] = list** (MatchError) no match of right hand side value: [1, 2, 3]

Pattern Matching Functionsdefmodule Factorial do def of(0), do: 1 def of(n), do: n * of(n-1) end

defmodule PrintStuff do def print({:error, stuff}) do IO.puts “ERROR! #{stuff}” end def print({:ok, stuff}), do: IO.puts stuffend PrintStuff.print({:ok, stuff})

Loops?

• How do you have a for loop with an immutable increment?– Recursion. Lots of recursion.

Stack Overflow

Who knows what a stack overflow is?

Tail Call Optimizationdefmodule TailRecursive do def factorial(n), do: _fact(n, 1) defp _fact(0, acc), do: acc defp _fact(n, acc), do: _fact(n-1, acc*n)

end

# defp is private

If the last function called is itself, the stack doesn’t grow.

Concurrencypid = spawn(Object, :method, [vars])# Creates a process# returns the ID of the process

pid = spawn_link(Object, :method, [vars])# Creates a process# returns the ID of the process# If the process dies, creator should too

Quick Example (from book)defmodule Link2 do import :timer, only: [ sleep: 1 ] def sad_function do sleep 500 exit(:boom) end

def run do spawn_link(Link2, :sad_function, []) receive do msg -> IO.puts "MESSAGE RECEIVED: #{inspect msg}" after 1000 -> IO.puts "Nothing happened as far as I am concerned" end end End

Link2.run # The runtime reports the abnormal termination: $ elixir -r link2.exs** (EXIT from #PID<0.35.0>) :boom

That’s where we start

• Building blocks for best practice patterns– GenServer– Task (async/await)– Agent (async / await + state)– Supervisor / Worker

Fault Tolerance / Supervisors

• Applications operate as a Supervisor tree• Process is created with another process

dedicated to monitoring it• Worker process dies, it’s immediately

restarted in original state– This is how Erlang applications can get

99.9999999% uptime (yes, 9 nines)

HANDLING ERRORS IS CODE SMELLThis is just fun to say

Error that could kill process?

• If you have an error that could kill a process…– Make sure the process knows how to restart in a

desirable state

• Very different way of thinking about problems

Simple Supervision Example> Math.Calculate.divide(10,2)5.0:ok

> Math.Calculate.divide(34,3)11.333333333333334:ok

> Math.Calculate.divide(34,0)A BIG UGLY ERROR MESSAGE... BUT LITTLE DID YOU KNOW THE PROCESS WAS RESTARTED AND LIVES!

> Math.Calculate.divide(34,2)17.0:OK

https://github.com/kblake/simple-supervision

“Where you’d previously think Object you’ll begin to think Process“

- Confucius

Dialyzer

• Operators are not overridden• + is always math– What’s on either side of it is always a number

• Allows dynamic typing WITH compiler checks• Best of both worlds, problems of neither

THIS ALL SEEMS REALLY COMPLICATED

I know right? I’m just here for the web stuff

Phoenix Framework

• Significantly lowers barrier to entry– Familiar Routes / Controllers– Flexible Middleware

• Plug– Excellent web sockets

• Channels– Excellent / flexible / replaceable database layer

• Ecto– Best view layer ever– Pretty dang fast

About that….

Rebuilt brightball.com

• Read about ithttp://www.brightball.com/articles/insanity-with-elixir-phoenix-postgresql

• Summary– As a dynamic site it’s as fast as my static nginx site– This is considered normal (and awesome)

Phoenix Channels vs Rails ActionCable

• https://dockyard.com/blog/2016/08/09/phoenix-channels-vs-rails-action-cable

• Results– Rails: 50 rooms, 2500 users - .05s avg– Rails: 75 rooms, 3750 users – 8s avg, degrading– Phoenix: 1100 rooms, 55,000 users - .25s avg• (maxed 55,000 client connections)

Terraform

• Use in your current environment incrementally– Put Phoenix in front of your app– Route specific requests to Phoenix– Pass the rest through to your current app– Bundled Cowboy webserver used by Heroku, AWS

Cloudfront, Incapsula, etc…it good• https://medium.com/@sugarpirate/rise-from-t

he-ashes-incremental-apis-with-phoenix-b08cd66bd142#.e1iojykq9

GenStage

• http://elixir-lang.org/blog/2016/07/14/announcing-genstage/

• Streaming, auto-scaling data ingestion

Nerves

• http://nerves-project.org/• ElixirConf was about Phoenix and Nerves• Nerves is for embedded software• Fault tolerance, reliability, consistency seem

important for something like that…?

Functional Programming

• What are the gains?– Readability / Maintainability– No side effects code– Simplified testing– Simplified personnel turnover– No object inheritance nightmares– Clear separation of concerns

Objects…so dumb • Data tied to Functions– Breeds repetition– Weird nesting– Breeds repetition– Modify up the tree– Breeds repetition

• Separate the two– Data structures– Functions– #mindblown

"Object oriented programs are offered as alternatives to correct ones” - Edsger W. Dijkstra

http://www.yegor256.com/2016/08/15/what-is-wrong-object-oriented-programming.html

Why is it the future?• Nothing with a shared memory model can match it for distribution…ever

– You can’t pass a memory reference to another machine transparently• Ruby like productivity

– Minus an eventual full rewrite expectation• Extremely fast, small footprint, embeddable, consistent, reliable• Low process overhead ideal for holding connections

• You can easily start SMALL, knowing you have all of the tools to grow/get crazy when you need them– Refactoring is just rearranging stuff (yes, really)– Naturally avoids bloat

• Compiler tells you when stuff isn’t used to make cleanup easy• But isn’t so strict that it forces you to change them

– Stop worrying about development time OR code performance tradeoffs

Which makes it ideal for…

• Server applications that talk to a lot of things…• Like web sockets• Internet of Things devices• Real time communications• Low latency applications• Geographic distribution– Cluster across data centers…yes, really.

• Avoiding bottlenecks

Resources

• Hack Greenville Slack #elixir-phoenix• elixir-lang.org– Books, chat, links, resources, groups

• phoenixframework.org• Elixir Conf sessions on YouTube– So much good stuff– Using Phoenix w/ Riak Core https

://www.youtube.com/watch?v=sYYOLaJ-VDQ&start=2&autoplay=1

More Resources

• Programming Elixir• Programming Phoenix• RedFour (fake company training)• ElixirForums• Assorted newsletters

Elixir Tank Game

http://tanx.verse15.net/Go there now, fight to the death

THANKS!Questions?