HELLO ELIXIR (AND OTP)Abel Muiño (@amuino) & Rok Biderman (@RokBiderman)
ULTRA SHORT INTROS
Abel Muiño Lead developer at Cabify. Works with ruby for a living.
[email protected] / @amuino
Rok Biderman Senior Go developer at Cabify. Has an interesting past position. Go ask him.
[email protected] / @RokBiderman
WE ARE HIRINGRuby, Go, Javascript, Android, iOS
(Just not for Elixir, yet)
HELLO ELIXIR (AND OTP)Abel Muiño (@amuino) & Rok Biderman (@RokBiderman)
GOALS
➤ Show some code, this is a programming meet up
➤ Share our Elixir learning path
➤ Learn something from feedback and criticism
➤ Hopefully at least one other person will learn one thing
“This is not production code
-Abel Muiño
BUILDING AN OCR MODULE
Extracting quotes from memes
TL;DR http://github.com/amuino/ocr
MIX NEW
$ mix new ocr* creating README.md* creating .gitignore* creating mix.exs* creating config* creating config/config.exs* creating lib* creating lib/ocr.ex* creating test* creating test/test_helper.exs* creating test/ocr_test.exs
Your Mix project was created successfully.You can use "mix" to compile it, test it, and more:
cd ocr mix test
Run "mix help" for more commands.
MIX NEW
$ mix new ocr* creating README.md* creating .gitignore* creating mix.exs* creating config* creating config/config.exs* creating lib* creating lib/ocr.ex* creating test* creating test/test_helper.exs* creating test/ocr_test.exs
Your Mix project was created successfully.You can use "mix" to compile it, test it, and more:
cd ocr mix test
Run "mix help" for more commands.
Project Definition
MIX NEW
$ mix new ocr* creating README.md* creating .gitignore* creating mix.exs* creating config* creating config/config.exs* creating lib* creating lib/ocr.ex* creating test* creating test/test_helper.exs* creating test/ocr_test.exs
Your Mix project was created successfully.You can use "mix" to compile it, test it, and more:
cd ocr mix test
Run "mix help" for more commands.
Project Definition
App config
MIX NEW
$ mix new ocr* creating README.md* creating .gitignore* creating mix.exs* creating config* creating config/config.exs* creating lib* creating lib/ocr.ex* creating test* creating test/test_helper.exs* creating test/ocr_test.exs
Your Mix project was created successfully.You can use "mix" to compile it, test it, and more:
cd ocr mix test
Run "mix help" for more commands.
Project Definition
App config
Main module
MIX NEW
$ mix new ocr* creating README.md* creating .gitignore* creating mix.exs* creating config* creating config/config.exs* creating lib* creating lib/ocr.ex* creating test* creating test/test_helper.exs* creating test/ocr_test.exs
Your Mix project was created successfully.You can use "mix" to compile it, test it, and more:
cd ocr mix test
Run "mix help" for more commands.
Project Definition
App config
Main module
😓Not talking about tests today
THE INTERACTIVE SHELL
$ iex -S mixErlang/OTP 18 [erts-7.3] [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]
Compiled lib/ocr.exGenerated ocr appConsolidated List.CharsConsolidated CollectableConsolidated String.CharsConsolidated EnumerableConsolidated IEx.InfoConsolidated InspectInteractive Elixir (1.2.5) - press Ctrl+C to exit (type h() ENTER for help)iex(1)> OcrOcr
THE INTERACTIVE SHELL
$ iex -S mixErlang/OTP 18 [erts-7.3] [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]
Compiled lib/ocr.exGenerated ocr appConsolidated List.CharsConsolidated CollectableConsolidated String.CharsConsolidated EnumerableConsolidated IEx.InfoConsolidated InspectInteractive Elixir (1.2.5) - press Ctrl+C to exit (type h() ENTER for help)iex(1)> OcrOcr
Automatically compiles new files
THE INTERACTIVE SHELL
$ iex -S mixErlang/OTP 18 [erts-7.3] [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]
Compiled lib/ocr.exGenerated ocr appConsolidated List.CharsConsolidated CollectableConsolidated String.CharsConsolidated EnumerableConsolidated IEx.InfoConsolidated InspectInteractive Elixir (1.2.5) - press Ctrl+C to exit (type h() ENTER for help)iex(1)> OcrOcr
Automatically compiles new files
Lots of first-run noise
THE INTERACTIVE SHELL
$ iex -S mixErlang/OTP 18 [erts-7.3] [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]
Compiled lib/ocr.exGenerated ocr appConsolidated List.CharsConsolidated CollectableConsolidated String.CharsConsolidated EnumerableConsolidated IEx.InfoConsolidated InspectInteractive Elixir (1.2.5) - press Ctrl+C to exit (type h() ENTER for help)iex(1)> OcrOcr
Automatically compiles new files
Lots of first-run noise
SUCCESS! Our module exists
💡 TIP: TRAINING WHEELS
➤ We are learning, so getting quick feedback is useful
➤ Credo is a static code analysis tool for the Elixir language with a focus on teaching and code consistency. https://github.com/rrrene/credo
➤ Credo installs as a project dependency
➤ Adds a new task to mix to analyse our code
➤ Excellent, very detailed, feedback
ADD A DEPENDENCY
Find the dependency info in hex.pmEdit mix.exs defp deps do [ {:credo, "~> 0.3", only: [:dev]} ] end
Install it locally$ mix geps.get
ADD A DEPENDENCY
Find the dependency info in hex.pmEdit mix.exs defp deps do [ {:credo, "~> 0.3", only: [:dev]} ] end
Install it locally$ mix geps.get
Dependencies are an array of tuples.
ADD A DEPENDENCY
Find the dependency info in hex.pmEdit mix.exs defp deps do [ {:credo, "~> 0.3", only: [:dev]} ] end
Install it locally$ mix geps.get
Dependencies are an array of tuples.
Only installs the dependency in the :dev
environment
TRY IT
$ mix credo… Code Readability [R] ! Modules should have a @moduledoc tag. lib/ocr.ex:1:11 (Ocr)
$ mix credo lib/ocr.ex:1:11
TRY IT
$ mix credo… Code Readability [R] ! Modules should have a @moduledoc tag. lib/ocr.ex:1:11 (Ocr)
$ mix credo lib/ocr.ex:1:11
OMG!
TRY IT
$ mix credo… Code Readability [R] ! Modules should have a @moduledoc tag. lib/ocr.ex:1:11 (Ocr)
$ mix credo lib/ocr.ex:1:11
OMG!
Detailed explanation on the error, how to suppress it,
etc…
NOW WHAT?
➤ Use Google Vision API to perform the actual OCR
➤ Has no client in hex.pm
➤ It is a REST API → {:httpoison, "~> 0.8.3"}
➤ Returns JSON → {:poison, "~> 2.1.0"}
➤ Needs authentication → {:goth, "~> 0.1.2”}
➤ Build a nice façade
MIX.EXS
def application do [applications: [:logger, :httpoison, :goth]] end
defp deps do [ {:httpoison, "~> 0.8.3"}, {:poison, "~> 2.1.0"}, {:goth, "~> 0.1.2"}, {:credo, "~> 0.3", only: [:dev]} ] end
MIX.EXS
def application do [applications: [:logger, :httpoison, :goth]] end
defp deps do [ {:httpoison, "~> 0.8.3"}, {:poison, "~> 2.1.0"}, {:goth, "~> 0.1.2"}, {:credo, "~> 0.3", only: [:dev]} ] end
Some deps also need their app to be started
CONFIG/CONFIG.EXS
use Mix.Config
config :goth, json: "config/google-creds.json" |> File.read!
CONFIG/CONFIG.EXS
use Mix.Config
config :goth, json: "config/google-creds.json" |> File.read!
Some deps also have their own config
NOW WHAT?
➤ We will write 2 modules:
➤ Ocr.GoogleVision for the API client.
➤ Ocr for our façade
💡 TIP: MODULE NAMES
➤ Convention: ➤ Ocr ! lib/ocr.ex➤ Ocr.GoogleVision ! lib/ocr/google_vision.ex
➤ Modules names are just names. Dots in the name do not represent any parent/child relationship.
LIB/OCR/GOOGLE_VISION.EX
defmodule Ocr.GoogleVision do def extract_text(image64) do image64 |> make_request |> read_body end # MAGIC!end
LIB/OCR/GOOGLE_VISION.EX
defmodule Ocr.GoogleVision do def extract_text(image64) do image64 |> make_request |> read_body end # MAGIC!end
base64 encoded image
LIB/OCR/GOOGLE_VISION.EX
defmodule Ocr.GoogleVision do def extract_text(image64) do image64 |> make_request |> read_body end # MAGIC!end
base64 encoded image
send to Google
LIB/OCR/GOOGLE_VISION.EX
defmodule Ocr.GoogleVision do def extract_text(image64) do image64 |> make_request |> read_body end # MAGIC!end
base64 encoded image
send to Google
get the text from the response
LIB/OCR/GOOGLE_VISION.EX
@url "https://vision.googleapis.com/v1/images:annotate"@feature_text_detection "TEXT_DETECTION"@auth_scope "https://www.googleapis.com/auth/cloud-platform"
def make_request(image64) do HTTPoison.post!(@url, payload(image64), headers)end
defp payload(image64) do %{requests: [ %{image: %{content: image64}, features: [%{type: @feature_text_detection}]} ] } |> Poison.encode!end
defp headers do {:ok, token} = Goth.Token.for_scope(@auth_scope) [{"Authorization", "#{token.type} #{token.token}"}]end
LIB/OCR/GOOGLE_VISION.EX
@url "https://vision.googleapis.com/v1/images:annotate"@feature_text_detection "TEXT_DETECTION"@auth_scope "https://www.googleapis.com/auth/cloud-platform"
def make_request(image64) do HTTPoison.post!(@url, payload(image64), headers)end
defp payload(image64) do %{requests: [ %{image: %{content: image64}, features: [%{type: @feature_text_detection}]} ] } |> Poison.encode!end
defp headers do {:ok, token} = Goth.Token.for_scope(@auth_scope) [{"Authorization", "#{token.type} #{token.token}"}]end
Module attributes (used as a constants)
LIB/OCR/GOOGLE_VISION.EX
@url "https://vision.googleapis.com/v1/images:annotate"@feature_text_detection "TEXT_DETECTION"@auth_scope "https://www.googleapis.com/auth/cloud-platform"
def make_request(image64) do HTTPoison.post!(@url, payload(image64), headers)end
defp payload(image64) do %{requests: [ %{image: %{content: image64}, features: [%{type: @feature_text_detection}]} ] } |> Poison.encode!end
defp headers do {:ok, token} = Goth.Token.for_scope(@auth_scope) [{"Authorization", "#{token.type} #{token.token}"}]end
Module attributes (used as a constants)
HTTP POST some JSON to some URL with some Headers
LIB/OCR/GOOGLE_VISION.EX
@url "https://vision.googleapis.com/v1/images:annotate"@feature_text_detection "TEXT_DETECTION"@auth_scope "https://www.googleapis.com/auth/cloud-platform"
def make_request(image64) do HTTPoison.post!(@url, payload(image64), headers)end
defp payload(image64) do %{requests: [ %{image: %{content: image64}, features: [%{type: @feature_text_detection}]} ] } |> Poison.encode!end
defp headers do {:ok, token} = Goth.Token.for_scope(@auth_scope) [{"Authorization", "#{token.type} #{token.token}"}]end
Module attributes (used as a constants)
HTTP POST some JSON to some URL with some Headers
The JSON Google wants
LIB/OCR/GOOGLE_VISION.EX
@url "https://vision.googleapis.com/v1/images:annotate"@feature_text_detection "TEXT_DETECTION"@auth_scope "https://www.googleapis.com/auth/cloud-platform"
def make_request(image64) do HTTPoison.post!(@url, payload(image64), headers)end
defp payload(image64) do %{requests: [ %{image: %{content: image64}, features: [%{type: @feature_text_detection}]} ] } |> Poison.encode!end
defp headers do {:ok, token} = Goth.Token.for_scope(@auth_scope) [{"Authorization", "#{token.type} #{token.token}"}]end
Module attributes (used as a constants)
HTTP POST some JSON to some URL with some Headers
The JSON Google wants
Get a token
LIB/OCR/GOOGLE_VISION.EX
@url "https://vision.googleapis.com/v1/images:annotate"@feature_text_detection "TEXT_DETECTION"@auth_scope "https://www.googleapis.com/auth/cloud-platform"
def make_request(image64) do HTTPoison.post!(@url, payload(image64), headers)end
defp payload(image64) do %{requests: [ %{image: %{content: image64}, features: [%{type: @feature_text_detection}]} ] } |> Poison.encode!end
defp headers do {:ok, token} = Goth.Token.for_scope(@auth_scope) [{"Authorization", "#{token.type} #{token.token}"}]end
Module attributes (used as a constants)
HTTP POST some JSON to some URL with some Headers
The JSON Google wants
Get a token
Put the token on the request headers
LIB/OCR/GOOGLE_VISION.EX
def read_body(%HTTPoison.Response{body: body, status_code: 200}) do body |> Poison.decode! |> get_in(["responses", &first/3, "textAnnotations", &first/3, "description"])end
defp first(:get, nil, _), do: nildefp first(:get, data, next) do data |> List.first |> next.()end
LIB/OCR/GOOGLE_VISION.EX
def read_body(%HTTPoison.Response{body: body, status_code: 200}) do body |> Poison.decode! |> get_in(["responses", &first/3, "textAnnotations", &first/3, "description"])end
defp first(:get, nil, _), do: nildefp first(:get, data, next) do data |> List.first |> next.()end
Only care about body and success http status
LIB/OCR/GOOGLE_VISION.EX
def read_body(%HTTPoison.Response{body: body, status_code: 200}) do body |> Poison.decode! |> get_in(["responses", &first/3, "textAnnotations", &first/3, "description"])end
defp first(:get, nil, _), do: nildefp first(:get, data, next) do data |> List.first |> next.()end
Only care about body and success http status
Parse JSON response
LIB/OCR/GOOGLE_VISION.EX
def read_body(%HTTPoison.Response{body: body, status_code: 200}) do body |> Poison.decode! |> get_in(["responses", &first/3, "textAnnotations", &first/3, "description"])end
defp first(:get, nil, _), do: nildefp first(:get, data, next) do data |> List.first |> next.()end
Only care about body and success http status
Parse JSON response
Extract the text
LIB/OCR/GOOGLE_VISION.EX
def read_body(%HTTPoison.Response{body: body, status_code: 200}) do body |> Poison.decode! |> get_in(["responses", &first/3, "textAnnotations", &first/3, "description"])end
defp first(:get, nil, _), do: nildefp first(:get, data, next) do data |> List.first |> next.()end
Only care about body and success http status
Parse JSON response
Extract the text
Custom lookup functions
LIB/OCR/GOOGLE_VISION.EX
def read_body(%HTTPoison.Response{body: body, status_code: 200}) do body |> Poison.decode! |> get_in(["responses", &first/3, "textAnnotations", &first/3, "description"])end
defp first(:get, nil, _), do: nildefp first(:get, data, next) do data |> List.first |> next.()end
Only care about body and success http status
Parse JSON response
Extract the text
Custom lookup functions
funky syntax to invoke an anonymous function
💡 TIP: GET_IN IS AWESOME
➤ Navigates nested structures (maps)
iex> users = %{"john" => %{age: 27}, "meg" => %{age: 23}}iex> get_in(users, ["john", :age])27➤ Returns nil on missing keys
iex> get_in(users, ["unknown", :age])nil➤ Accepts functions for navigating other types
➤ Elixir 1.3 will have common functions predefined for tuples and lists
➤ Access.at(0) will replace my custom first (PR #4719)
➤ Also check get_and_update_in, put_in, update_in
LIB/OCR.EX
defmodule Ocr do def from_base64(b64), do: Ocr.GoogleVision.extract_text(b64)
def from_image(image_data) do image_data |> Base.encode64 |> from_base64 end
def from_path(path), do: path |> File.read! |> from_image
def from_url(url), do: HTTPoison.get!(url).body |> from_imageend
LIB/OCR.EX
defmodule Ocr do def from_base64(b64), do: Ocr.GoogleVision.extract_text(b64)
def from_image(image_data) do image_data |> Base.encode64 |> from_base64 end
def from_path(path), do: path |> File.read! |> from_image
def from_url(url), do: HTTPoison.get!(url).body |> from_imageend
LIB/OCR.EX
defmodule Ocr do def from_base64(b64), do: Ocr.GoogleVision.extract_text(b64)
def from_image(image_data) do image_data |> Base.encode64 |> from_base64 end
def from_path(path), do: path |> File.read! |> from_image
def from_url(url), do: HTTPoison.get!(url).body |> from_imageend
LIB/OCR.EX
defmodule Ocr do def from_base64(b64), do: Ocr.GoogleVision.extract_text(b64)
def from_image(image_data) do image_data |> Base.encode64 |> from_base64 end
def from_path(path), do: path |> File.read! |> from_image
def from_url(url), do: HTTPoison.get!(url).body |> from_imageend
FUN!
$ iex -S mixErlang/OTP 18 [erts-7.3] [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]
A new Hex version is available (0.12.0), please update with `mix local.hex`Interactive Elixir (1.2.5) - press Ctrl+C to exit (type h() ENTER for help)iex(1)> meme_url = "http://ih0.redbubble.net/image.16611809.2383/fc,550x550,black.jpg""http://ih0.redbubble.net/image.16611809.2383/fc,550x550,black.jpg"iex(2)> IO.puts Ocr.from_url meme_urlGETS ELIKIR PR ACCEPTEDI SAID WHO WANTS TOFUCKING TOUCH ME?Suranyami
:ok
FUN!
$ iex -S mixErlang/OTP 18 [erts-7.3] [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]
A new Hex version is available (0.12.0), please update with `mix local.hex`Interactive Elixir (1.2.5) - press Ctrl+C to exit (type h() ENTER for help)iex(1)> meme_url = "http://ih0.redbubble.net/image.16611809.2383/fc,550x550,black.jpg""http://ih0.redbubble.net/image.16611809.2383/fc,550x550,black.jpg"iex(2)> IO.puts Ocr.from_url meme_urlGETS ELIKIR PR ACCEPTEDI SAID WHO WANTS TOFUCKING TOUCH ME?Suranyami
:ok
STATEFUL? STATELESS?Cast your vote!
ANSWER: STATEFUL
➤ Auth tokens are not requested every time
➤ Requested on first use
➤ Refreshed on the background when about to expire
➤ Goth.TokenStore is a GenServer➤ Just one of the predefined behaviours to make easier to
work with processes
➤ Starts a process with some state
➤ Receives messages and updates the state
➤ There is more state (like Goth.Config)
DEPS/GOTH/LIB/GOTH/TOKEN_STORE.EX
defmodule Goth.TokenStore do use Xenserver alias Goth.Token
def start_link do GenServer.start_link(__MODULE__, %{}, [name: __MODULE__]) end
def handle_call({:store, scope, token}, _from, state) do pid_or_timer = Token.queue_for_refresh(token) {:reply, pid_or_timer, Map.put(state, scope, token)} end
def handle_call({:find, scope}, _from, state) do {:reply, Map.fetch(state, scope), state} endend
DEPS/GOTH/LIB/GOTH/TOKEN_STORE.EX
defmodule Goth.TokenStore do use Xenserver alias Goth.Token
def start_link do GenServer.start_link(__MODULE__, %{}, [name: __MODULE__]) end
def handle_call({:store, scope, token}, _from, state) do pid_or_timer = Token.queue_for_refresh(token) {:reply, pid_or_timer, Map.put(state, scope, token)} end
def handle_call({:find, scope}, _from, state) do {:reply, Map.fetch(state, scope), state} endend
Start the process,
DEPS/GOTH/LIB/GOTH/TOKEN_STORE.EX
defmodule Goth.TokenStore do use Xenserver alias Goth.Token
def start_link do GenServer.start_link(__MODULE__, %{}, [name: __MODULE__]) end
def handle_call({:store, scope, token}, _from, state) do pid_or_timer = Token.queue_for_refresh(token) {:reply, pid_or_timer, Map.put(state, scope, token)} end
def handle_call({:find, scope}, _from, state) do {:reply, Map.fetch(state, scope), state} endend
Start the process, Initial state is an empty map
DEPS/GOTH/LIB/GOTH/TOKEN_STORE.EX
defmodule Goth.TokenStore do use Xenserver alias Goth.Token
def start_link do GenServer.start_link(__MODULE__, %{}, [name: __MODULE__]) end
def handle_call({:store, scope, token}, _from, state) do pid_or_timer = Token.queue_for_refresh(token) {:reply, pid_or_timer, Map.put(state, scope, token)} end
def handle_call({:find, scope}, _from, state) do {:reply, Map.fetch(state, scope), state} endend
Start the process, Initial state is an empty map
The process has a name
DEPS/GOTH/LIB/GOTH/TOKEN_STORE.EX
defmodule Goth.TokenStore do use Xenserver alias Goth.Token
def start_link do GenServer.start_link(__MODULE__, %{}, [name: __MODULE__]) end
def handle_call({:store, scope, token}, _from, state) do pid_or_timer = Token.queue_for_refresh(token) {:reply, pid_or_timer, Map.put(state, scope, token)} end
def handle_call({:find, scope}, _from, state) do {:reply, Map.fetch(state, scope), state} endend
Start the process, Initial state is an empty map
The process has a name
Handle 2 types of messages, returning something
FUN WITH TOKEN STORE
$ iex -S mixiex(1)> token = %Goth.Token{token: "FAKE", expires: :os.system_time + 10_000_000}%Goth.Token{expires: 1464647952951907000, scope: nil, token: "FAKE", type: nil}
iex(2)> GenServer.call Goth.TokenStore, {:find, "Elixir"} :error
iex(3)> GenServer.call Goth.TokenStore, {:store, "Elixir", token}{:ok, {1464647950910798703208727, #Reference<0.0.7.228>}}
iex(4)> GenServer.call Goth.TokenStore, {:find, "Elixir"}{:ok, %Goth.Token{expires: 1464647952951907000, scope: nil, token: “FAKE", type: nil}}
FUN WITH TOKEN STORE
$ iex -S mixiex(1)> token = %Goth.Token{token: "FAKE", expires: :os.system_time + 10_000_000}%Goth.Token{expires: 1464647952951907000, scope: nil, token: "FAKE", type: nil}
iex(2)> GenServer.call Goth.TokenStore, {:find, "Elixir"} :error
iex(3)> GenServer.call Goth.TokenStore, {:store, "Elixir", token}{:ok, {1464647950910798703208727, #Reference<0.0.7.228>}}
iex(4)> GenServer.call Goth.TokenStore, {:find, "Elixir"}{:ok, %Goth.Token{expires: 1464647952951907000, scope: nil, token: “FAKE", type: nil}}
The process name
FUN WITH TOKEN STORE
$ iex -S mixiex(1)> token = %Goth.Token{token: "FAKE", expires: :os.system_time + 10_000_000}%Goth.Token{expires: 1464647952951907000, scope: nil, token: "FAKE", type: nil}
iex(2)> GenServer.call Goth.TokenStore, {:find, "Elixir"} :error
iex(3)> GenServer.call Goth.TokenStore, {:store, "Elixir", token}{:ok, {1464647950910798703208727, #Reference<0.0.7.228>}}
iex(4)> GenServer.call Goth.TokenStore, {:find, "Elixir"}{:ok, %Goth.Token{expires: 1464647952951907000, scope: nil, token: “FAKE", type: nil}}
The process name
The message
DEPS/GOTH/LIB/GOTH/TOKEN_STORE.EX
defmodule Goth.TokenStore do
def store(%Token{}=token), do: store(token.scope, token) def store(scopes, %Token{} = token) do GenServer.call(__MODULE__, {:store, scopes, token}) end
def find(scope) do GenServer.call(__MODULE__, {:find, scope}) endend
DEPS/GOTH/LIB/GOTH/TOKEN_STORE.EX
defmodule Goth.TokenStore do
def store(%Token{}=token), do: store(token.scope, token) def store(scopes, %Token{} = token) do GenServer.call(__MODULE__, {:store, scopes, token}) end
def find(scope) do GenServer.call(__MODULE__, {:find, scope}) endend
Provide a client API (usually in the same module) with nicer methods hiding the use of
Genserver.call
LET’S TALK ABOUT OTPProcesses, state, concurrency, supervisors,… oh my!
RECAP
➤ Erlang is:
➤ general-purpose
➤ concurrent
➤ garbage-collected
➤ programming language
➤ and runtime system
➤ Elixir:
➤ Builds on top of all that
START A PROCESS
➤ Basic concurrency primitive
➤ Simplest way to create, use spawn with a function defmodule BasicMessagePassing.Call do def concat(a, b) do IO.puts("#{a} #{b}") endend
iex(2)> BasicMessagePassing.Call.concat "Elixir", "Madrid"Elixir Madrid:okiex(3)> spawn BasicMessagePassing.Call, :concat, ["Elixir", "Madrid"]Elixir Madrid#PID<0.69.0>
START A PROCESS
➤ Basic concurrency primitive
➤ Simplest way to create, use spawn with a function defmodule BasicMessagePassing.Call do def concat(a, b) do IO.puts("#{a} #{b}") endend
iex(2)> BasicMessagePassing.Call.concat "Elixir", "Madrid"Elixir Madrid:okiex(3)> spawn BasicMessagePassing.Call, :concat, ["Elixir", "Madrid"]Elixir Madrid#PID<0.69.0>
Same process
START A PROCESS
➤ Basic concurrency primitive
➤ Simplest way to create, use spawn with a function defmodule BasicMessagePassing.Call do def concat(a, b) do IO.puts("#{a} #{b}") endend
iex(2)> BasicMessagePassing.Call.concat "Elixir", "Madrid"Elixir Madrid:okiex(3)> spawn BasicMessagePassing.Call, :concat, ["Elixir", "Madrid"]Elixir Madrid#PID<0.69.0>
Same process
Spawned process id
LISTEN FOR MESSAGES
defmodule BasicMessagePassing.Listen do def listen do receive do {:ok, input} -> IO.puts "#{input} Madrid" end end end
iex(5)> pid = spawn(BasicMessagePassing.Listen, :listen, [])#PID<0.82.0>iex(6)> send pid, {:ok, "Elixir"}Elixir Madrid{:ok, "Elixir"}iex(8)> Process.alive? pidfalse
FIBONACCI TIME!defmodule FibSerial do def calculate(ns) do ns |> Enum.map(&(calc(&1))) |> inspect |> IO.puts end
def calc(n) do calc(n, 1, 0) end
defp calc(0, _, _) do 0 end
defp calc(1, a, b) do a + b end
defp calc(n, a, b) do calc(n - 1, b, a + b) end end FibSerial.calculate(Enum.to_list(1..10000))
FIBONACCI TIME!defmodule FibSerial do def calculate(ns) do ns |> Enum.map(&(calc(&1))) |> inspect |> IO.puts end
def calc(n) do calc(n, 1, 0) end
defp calc(0, _, _) do 0 end
defp calc(1, a, b) do a + b end
defp calc(n, a, b) do calc(n - 1, b, a + b) end end FibSerial.calculate(Enum.to_list(1..10000)) About 6 seconds
PARALLEL FIBONACCI TIME!defmodule FibParallel do def calculate(ns) do ns |> Enum.with_index |> Enum.map(fn(ni) -> spawn FibParallel, :send_calc, [self, ni] end) listen(length(ns), []) end
def send_calc(pid, {n, i}) do send pid, {calc(n), i} end
defp listen(lns, result) do receive do fib -> result = [fib | result] if lns == 1 do result |> Enum.sort(fn({_, a}, {_, b}) -> a < b end) |> Enum.map(fn({f, _}) -> f end) |> inspect |> IO.puts else listen(lns - 1, result) end end end end FibSerial.calculate(Enum.to_list(1..10000))
PARALLEL FIBONACCI TIME!defmodule FibParallel do def calculate(ns) do ns |> Enum.with_index |> Enum.map(fn(ni) -> spawn FibParallel, :send_calc, [self, ni] end) listen(length(ns), []) end
def send_calc(pid, {n, i}) do send pid, {calc(n), i} end
defp listen(lns, result) do receive do fib -> result = [fib | result] if lns == 1 do result |> Enum.sort(fn({_, a}, {_, b}) -> a < b end) |> Enum.map(fn({f, _}) -> f end) |> inspect |> IO.puts else listen(lns - 1, result) end end end end FibSerial.calculate(Enum.to_list(1..10000)) About 2 seconds (4 cores)
LINKING PROCESSES
➤ If child dies, parent dies defmodule BasicMessagePassing.Linking do def exit, do: exit(:crash)
def start do spawn_link(BasicMessagePassing.Linking, :exit, []) receive do {:done} -> IO.puts "no more waiting" end end end
iex(13)> BasicMessagePassing.Linking.start** (EXIT from #PID<0.57.0>) :crash
Interactive Elixir (1.2.5) - press Ctrl+C to exit (type h() ENTER for help)iex(1)>
LINKING PROCESSES
➤ If child dies, parent dies defmodule BasicMessagePassing.Linking do def exit, do: exit(:crash)
def start do spawn_link(BasicMessagePassing.Linking, :exit, []) receive do {:done} -> IO.puts "no more waiting" end end end
iex(13)> BasicMessagePassing.Linking.start** (EXIT from #PID<0.57.0>) :crash
Interactive Elixir (1.2.5) - press Ctrl+C to exit (type h() ENTER for help)iex(1)>
iex died! and was restarted
LINKING PROCESSES
➤ If child dies, parent dies… unless it handles the dead defmodule BasicMessagePassing.Linking do def exit, do: exit(:crash)
def start do Process.flag(:trap_exit, true) spawn_link(BasicMessagePassing.Linking, :exit, []) receive do {:EXIT, from_pid, reason} -> IO.puts "#{inspect(self)} is aware #{inspect(from_pid)} exited because of #{reason}" end end end
iex(24)> BasicMessagePassing.Linking.start#PID<0.57.0> is aware #PID<0.157.0> exited because of crash:ok
LINKING PROCESSES
➤ If child dies, parent dies… unless it handles the dead defmodule BasicMessagePassing.Linking do def exit, do: exit(:crash)
def start do Process.flag(:trap_exit, true) spawn_link(BasicMessagePassing.Linking, :exit, []) receive do {:EXIT, from_pid, reason} -> IO.puts "#{inspect(self)} is aware #{inspect(from_pid)} exited because of #{reason}" end end end
iex(24)> BasicMessagePassing.Linking.start#PID<0.57.0> is aware #PID<0.157.0> exited because of crash:ok
survive children
LINKING PROCESSES
➤ If child dies, parent dies… unless it handles the dead defmodule BasicMessagePassing.Linking do def exit, do: exit(:crash)
def start do Process.flag(:trap_exit, true) spawn_link(BasicMessagePassing.Linking, :exit, []) receive do {:EXIT, from_pid, reason} -> IO.puts "#{inspect(self)} is aware #{inspect(from_pid)} exited because of #{reason}" end end end
iex(24)> BasicMessagePassing.Linking.start#PID<0.57.0> is aware #PID<0.157.0> exited because of crash:ok
survive children
handle deads
GENSERVER
➤ Simplifies all this stuff
➤ Is a process like any other Elixir process
➤ Standard set of interface functions, tracing and error reporting
➤ call: request with response
➤ cast: request without response
A STACK
defmodule Stack do use GenServer
def start_link(state, opts \\ []) do GenServer.start_link(__MODULE__, state, opts) end
def handle_call(:pop, _from, [h|t]) do {:reply, h, t} end
def handle_cast({:push, h}, t) do {:noreply, [h|t]} endend
A SUPERVISED STACKiex(7)> import Supervisor.Specniliex(8)> children = [...(8)> worker(Stack, [[:first], [name: :stack_name]])...(8)> ][{Stack, {Stack, :start_link, [[:first], [name: :stack_name]]}, :permanent, 5000, :worker, [Stack]}]iex(9)> {:ok, pid} = Supervisor.start_link(children, strategy: :one_for_one){:ok, #PID<0.247.0>}iex(10)> GenServer.call(:stack_name, :pop):firstiex(11)> GenServer.call(:stack_name, :pop)18:04:35.012 [error] GenServer :stack_name terminating** (FunctionClauseError) no function clause matching in Stack.handle_call/3 iex:13: Stack.handle_call(:pop, {#PID<0.135.0>, #Reference<0.0.1.317>}, [])[..]Last message: :popState: [] (elixir) lib/gen_server.ex:564: GenServer.call/3iex(11)> GenServer.call(:stack_name, :pop):first
SUPERVISOR FLAVORS
➤ one_for_one: dead worker is replaced by another one
➤ rest_for_all: after one dies, all others have to be restarted
➤ rest_for_one: all workers started after this one will be restarted
➤ simple_one_for_one: for dynamically attached children, Supervisor is required to contain only one child
QUESTIONS?
THANKS!Abel Muiño (@amuino) & Rok Biderman (@RokBiderman)