What To Leave Implicit

What to Leave Implicit

Martin Odersky

Scala Days Chicago

April 2017

Con-textual

what comes with the text,

but is not in the text

Context is all around us

- the current configuration

- the current scope

- the meaning of “<” on this type

- the user on behalf of which the

operation is performed

- the security level in effect

…

- globals

rigid if immutable,

unsafe if mutable

- monkey patching

- dependency injection

at runtime (Spring, Guice)

or with macros (MacWire)

- cake pattern

close coupling + recursion

Traditional ways to express context

“Parameterize all the things”

The Functional Way

- no side effects

- type-safe

- fine-grained control

Functional is Good

- sea of parameters

- most of which hardly ever change

- repetitive, boring, prone to mistakes

But sometimes it’s too much of a

good thing …

If passing a lot of parameters gets tedious,

leave some of them implicit.

A more direct approach

• If there’s one feature that makes Scala “Scala”,

I would pick implicits.

• There’s hardly an API without them.

• They enable advanced and elegant

architectural designs.

• They are also misused way too often.

Implicits

• takes you through the most common uses of

implicits,

• gives recommendations of use patterns,

• goes through a set of proposed language

changes that will make implicits even more

powerful and safer to use.

This Talk

t

• If you do not give an argument to an implicit

parameter, one will be provided for you.

• Eligible are all implicit values that are visible at

the point of call.

• If there are more than one eligible candidate,

the most specific one is chosen.

• If there’s no unique most specific candidate, an

ambiguity error Is reported.

Ground Rules

• They are a cousin of implicit parameters.

• If the type A of an expression does not match

the expected type B …

Implicit Conversions

• They are a cousin of implicit parameters.

• If the type A of an expression does not match

the expected type B …

… the compiler tries to find an implicit

conversion method from A to B.

• Same rules as for implicit parameters apply.


• Shorthand for defining a new class and an

implicit conversion into it.

implicit class C(x: T) { … }

expands to

class C(x: T) { … }implicit def C(x: T) = new C(x)

Implicit Classes

• Implicits leverage what the compiler knows

about your code.

• They remove repetition and boilerplate.

• But taken too far, they can hurt readability.

When To Use Implicits?

Applicability:

Where are you allowed to elide implied information?

How do you find out this is happening?

Power:

What influence does the elided info have?

Can it change radically behavior or types?

Scope:

How much of the rest of the code do you need to know

to find out what is implied?

Is there always a clear place to look?

* Adapted from Rust’s Language Ergonomics Initiative

Reasoning Footprint of Implicitness*

Patterns of Implicit Conversions

Extension Methods

Discoverability: medium

Power: low

Scope: large, but IDEs help

Extension Methods

Discoverability: medium

Power: low


Late Trait Implementation

Make existing classes implement new traits

without changing their code.

This was the original reason for implicits in Scala.

Discoverability: low to medium

Power: low to medium


They also have some use cases, e.g.

• cached implicit classes

• context-dependent views

What about simple conversions?

• Conversions that go both ways

• Conversions that change semantics

E.g. collection.convert.WrapAs{Java,Scala}

Better: Use Converters

collection.convert.DecorateAs{Java,Scala}

Anti Patterns

Conversions that undermine type safety

Anti Patterns

Conversions between pre-existing types

Discoverability: low

Power: high

Scope: very large

Anti Patterns

Discoverability: high

Power: low to high

Scope: large, but can be explored in IntelliJ

Implicit Parameters

Implicit parameters can

• prove theorems

• establish context

• set configurations

• inject dependencies

• model capabilities

• implement type classes

Implicit Parameters - Use Cases

Curry Howard isomorphism:

Types = Theorems

Programs = Proofs

C.f. Kennedy & Russo: “Generalized Type Constraints”, OOPSLA 2004

Prove Theorems

Establish Context

Example: conference management system.

Reviewers should only see (directly or indirectly) the

scores of papers where they have no conflict with an

author.

Establish Context

Example: conference management system.

Context is usually stable, can change at specific points.

Configuration &

Dependency Management

are special cases of context passing.

see also: Dick Wall: The parfait pattern

Implement Type Classes

Example: Ordering

How can we make implicits better?

What will change:

1. Tighten the rules for implicit conversions

2. Lazy implicits

3. Multiple implicit parameter lists

4. Coherence(?)

5. Implicit function types

1. Tighten Rules for


Question: What does this print?

Answer: java.lang.IndexOutOfBoundsException: 42

Hint: List[String] <: Function[Int, String]

In the future: Only implicit methods are eligible as

conversions.

A new class ImplicitConverter allows to abstract over

implicit conversions.

Converters are turned into conversions like this:

1. Tighten Rules for


Implementation Status

2. Lazy Implicits

Problem: When synthesizing code for recursive data

structures, we get divergent implicit searches.

E.g.

will diverge if A expands recursively to Sum[A, B]

2. Lazy Implicits

Solution: Delay the implicit search and tie the recursive

knot with a lazy val if a parameter is call-by-name:

This change, proposed by Miles Sabin, is a more robust

solution than the current “Lazy” type in shapeless.


3. Multiple Implicit Parameter Lists

Problem: Implicit parameters are currently a bit irregular

compared to normal parameters:

• there can be only one implicit parameter section

• and it must come last.

This leads to some awkward workarounds (c.f. Aux

pattern).

Related problem: It’s sometimes confusing when a

parameter is implicit or explicit.

3. Multiple Implicit Parameter Lists

Proposal:

• Allow multiple implicit parameter lists

• Implicit and explicit parameter lists can be mixed freely.

• Explicit application of an implicit parameter must be

marked with a new “magic” method, explicitly.

• Implementation status: Proposal. Main challenge is

migration from current Scala.

4. Coherence

Difference between Scala’s implicits and Haskell’s type

classes: The latter are required to be coherent:

A type can implement a type class in one way only

(globally).

This is very restrictive, rules out most of the implicit use

cases we have seen.

But it also provides some benefits.

Coherence Rules Out Ambiguities

Say you have a capability system dealing with driver

licences:

If you can drive a truck and a cab, you should be able to

drive a car. But Scala would give you an ambiguity error.

Coherence Rules Out Ambiguities

Proposal:

• Allow type classes to declare themselves coherent.

• Have the compiler check that coherent traits have only

one implementation per type.

This is quite tricky. See github.com/lampepfl/dotty/issues/2047

• Drop all ambiguity checks for coherent implicits.

Parametricity

• It turns out that a necessary condition to ensure

coherence is to disallow operations like equals,

hashCode, isInstanceOf on coherent types.

• This restriction is useful in other contexts as well

because it gives us “theorems for free”.

• Proposal: Change Scala’s

top types to:

• AnyObj has all of current

Any’s methods. Any has only

the escape-hatch method

asInstanceOf.


5. Implicit Function Types

Have another look at the conference management system:

In a large system, it gets tedious to declare all these

implicit Viewers parameters.

Can we do better?

Having to write

a couple of times does not look so bad.

But in the dotty compiler there are > 2600 occurrences of

the parameter

Would it not be nice to get rid of them?

Towards a solution

Let’s massage the definition of viewRankings a bit:

Towards a solution


Towards a solution


What is its type?

So far: Viewers => List[Paper]

From now on: implicit Viewers=> List[Paper]

or, desugared: ImplicitFunction1[Viewers, List[Paper]]

Inside ImplicitFunction1

ImplicitFunction1 can be thought of being defined as

follows:

Analogously for all other arities.

Two Rules for Typing

1. Implicit functions get implicit arguments just like implicit

methods. Given:

val f: implicit A => B

implicit val a: A

f expands to f(a).

2. Implicit functions get created on demand. If the

expected type of b is implicit A => B, then

b expands to implicit (_: A) => b

Revised Example

Assume:

Then reformulate:

Efficiency

Implicit function result types can be optimized

Instead of creating a closure like this:

we can simply create a curried function like this:

This brings the cost of implicit functions down to simple

implicit parameters.


• The reader monad is a somewhat popular method to

pass context.

• Essentially, it wraps the implicit reading in a monad.

• One advantage: The reading is abstracted in a type.

• But I believe this is shooting sparrows with cannons.

• Monads are about sequencing, they have have nothing

to do with passing context.

The Reader Monad

• allow the same conciseness as the reader

monad,

• don’t force you into monadic style with explicit

sequencing,

• are fully composable,

• are more than 7x faster than the reader monad.

Implicit function types

Neat way to define structure-building DSLs, like this:

Natively supported in Groovy and in Kotlin via

“receiver functions”.

An encore: The Builder Pattern

Scala Implementation

• Any situation where an entity is implicitly

understood can be expressed with implicit

function types.

• We have seen:

“The current set of viewers”

“The structure to which current code should be added”

• Other possibilities:

“The current configuration”

“The currently running transaction”

“The capabilities needed to run this code”

"The effects this code has on the outside world”

…

Conjecture

Find out more on scala-lang.org/blog

• Implicit function types are a neat way to

abstract over contexts.

• It’s a very powerful feature, because it allows

one to inject implicit values in a scope simply

by defining a type.

• I expect it will fundamentally affect the kind of

code we will write in the future.

Summary (1)

• Implicit parameters are a fundamental and

powerful language construct.

• They are “just” parameterization, but remove

the boilerplate.

• One construct multifaceted use cases

• Abstractable using implicit function types.

• Implicit conversions are also very convenient,

but should be used with care.

Summary (2)

When Can I Expect This?

Scala 2.12

Scala 2.13

Scala 3.0TASTY,

middle end?

stdlib

collections

dotty MVP

dotty 0.x releases

2016backend, classpath handling

Scala 2.14

2017

2018

This is open source work, depends on community’s contributions.

Roadmap is tentative, no promises:

“MVP” = minimal

viable

prototype

Dotty:

Dmitry Petrashko Nicolas Stucki

Guillaume Martres Sebastien Douraene

Felix Mulder Ondrej Lhotak

Liu Fengyun Enno Runne

Close collaboration with scalac team @ Lightbend:

Adriaan Moors Seth Tisue

Jason Zaugg Stefan Zeiger

Lukas Rytz

Credits

Thank You

Technology

What To Leave Implicit