DSL Construction with Ruby - Masterclass Series

goals •  DSL Fundamentals •  Some Ruby Basics •  DSL construction with Ruby •  Advanced Topics - A step further •  Summary •  Q & A

problem?

Problem

Solution

reduce the gap

Customer

Programmer

01000001

evolution

expr

essi

ve

?

Object Oriented

Procedural

Assembly

100101010

Emergence of language

C

C++ Java

Problem

GA

P

Solution

•  Arm Ball •  Around the wicket •  Cow Corner •  Duck •  Fly Slip •  Googly

http://en.wikipedia.org/wiki/List_of_cricket_terms - an long list of cricket terms

the power of language

evolution

expr

essi

ve

DSL

Object Oriented

Procedural

Assembly

100101010

Emergence of language

C

C++ Java

Problem

GA

P

Solution

example

Monit – automatic management and monitoring - http://mmonit.com/

definition a computer programming language of limited

expressiveness focused on a particular domain

- Martin Fowler

Martin Fowlers book on DSLs - http://martinfowler.com/dslwip/

definition a computer programming language of

limited expressiveness focused on a particular domain

- Martin Fowler




- Martin Fowler




- Martin Fowler


Domain Specific Languages vs.

General Purpose Languages

what DSLs bring to the table •  Quality •  Productivity •  Reliability •  Maintainability •  Reusability


•  Social impact


•  Social impact •  Validation at the domain level

no silver bullet!

no silver bullet! •  Learning curve

no silver bullet! •  Learning curve •  Good language design is hard

no silver bullet! •  Learning curve •  Good language design is hard •  Cost of building

no silver bullet! •  Learning curve •  Good language design is hard •  Cost of building •  Limited applicability

no silver bullet! •  Learning curve •  Good language design is hard •  Cost of building •  Limited applicability •  Maintenance

no silver bullet! •  Learning curve •  Good language design is hard •  Cost of building •  Limited applicability •  Maintenance •  Could be overused or abused

Types of DSL

external DSL

Need to build a parser to process the custom syntax

sql, make files, xml config files, regular expressions

advantages •  Free to use any syntax

advantages •  Free to use any syntax •  Run time evaluation

disadvantages •  Starts simple, can get ugly and complex

disadvantages •  Starts simple, can get ugly and complex •  Building parsers is difficult

disadvantages •  Starts simple, can get ugly and complex

•  Building parsers is difficult •  Lack of tooling support

internal DSL

Extends the host language

advantages •  Don't have to write and debug a new

language


language

•  Full power of base language is available


language

•  Full power of base language is available

•  Tooling support available

disadvantages •  constrained by the host language.

Ruby based DSLs are internal

DSLs - not special to Ruby

Ruby is special

why is ruby special •  minimally intrusive Syntax to allow for more

concise code


concise code •  Ruby culture - values expressiveness in

code


concise code •  Ruby culture - values expressiveness in

code •  Dynamic and Reflective

* Working code writing using RSpec, a testing frame work

DSL goodness

OPTIONAL PUNCTUATION concise code

vs.

SYMBOLS less noisy than strings

or

BLOCKS delayed evaluation of code

OPEN CLASSES build your language

some code here

METAPROGRAMMING expand your mind

define_method eval

module_eval class_eval

instance_eval

alias_method

‘whenever’ a DSL for cron jobs

30 4 1,3,5,7,9,11,13,15,17,19,21,23,25,27,29,31 * * reboot

http://github.com/javan/whenever

language constructs

hour

day month

year

reboot weekend

weekday

sunday monday

a.m p.m

expressive

30 4 1,3,5,7,9,11,13,15,17,19,21,23,25,27,29,31 * * reboot

http://github.com/javan/whenever

The fascinating thing is that, in my experience, most well-written Ruby

programs are already a DSL, just by nature of Ruby’s syntax.”

- Jamis Buck, 37signals

THE PROCESS Writing DSLs in Ruby

Application Programming Interface

try { Socket client = new Socket(“www.google.com”,80); } catch(IOException e) { System.out.println(e); }

Example 1



Customer.find :all, :condition => [ :age >= 25 ]

Example 1

Example 2




•  Libraries give a sense of domain-specificity because of vocabulary

•  Nouns / verbs / adverbs / adjectives

Example 1

Example 2




•  Libraries give a sense of domain-specificity because of vocabulary

•  Nouns / verbs / adverbs / adjectives •  They are all internal DSLs

Example 1

Example 2

DSL

Abstractions

Interfaces

Patterns

“But, I believe a DSL is a healthy bi-product of a good object-oriented design.”

Blaine Buxton (Smalltalk developer)

Capture vocabulary and processes of the domain

Discover desired syntax

Define DSL interface (API)

Define classes and abstractions

Implement validations

Internal DSLs – A coarse process

Capture vocabulary •  Task scheduling is the domain of ‘cron’

•  Tasks •  Timing •  Frequency


•  Tasks (e.g. ‘reboot’, ‘send mail’, ‘alert’, etc) •  Timing (e.g. ‘5 pm’, ‘4.30 am’, etc) •  Frequency (e.g. ‘yearly’, ‘weekend’, etc)


•  Tasks (e.g. ‘reboot’, ‘send mail’, ‘alert’, etc) •  Timing (e.g. ‘5 pm’, ‘4.30.am’, etc) •  Frequency (e.g. ‘yearly’, ‘weekend’, etc)

•  Discover keywords •  Special words (‘yearly’, ‘reboot’, etc) •  Domain values (‘5 pm’, etc) •  A good design would decide ownership of these keywords

monday

hourly

day month

year

reboot weekend

weekday

sunday

a.m. p.m.

runner

at

every

annually

Discover syntax

Discuss with

DSL user

Experiment around

keywords

Design a syntax

Define constructs

Define ownership

of keywords

Write extended methods

(e.g. 2.days)

Design considerations

•  “Follow good design principles” –  Entities as classes

•  As nouns

–  Operations as methods •  Typically as verbs •  Adaptive interfaces (wrappers) to instantiate aggregations •  Accept hash as argument to simulate ‘call by name’

Using Ruby features to realize DSL constructs

Purpose (what)

Getter/setter

Pattern matching

Alternative interfaces

Context

Code generation

Execution

Arbitrary interfaces/attributes

Ruby feature (how)

Function calls w/o parentheses

Regular expressions

‘alias_method’

Closure/block

Strings

‘load’, ‘require’, ‘eval’

‘method_missing’

Writing ‘Whenever’

every 2.days, :at => '4:30 am‘ do runner “/usr/bin/reboot” end


every 2.days, :at => '4:30 am‘ do runner “/usr/bin/reboot” end

every(2.days(),{:at => '4:30 am’}) do runner(“/usr/bin/reboot”) end


Class JobList def every(frequency, option={}) … yield #handles block end

def runner(task, options={}) … end end

2.days()

{ :at => ‘4.30.am ‘ }

A TALE OF TWO DSLS Real examples

EXAMPLE 1: DSL FOR GMRT

Giant Metrewave Radio Telescope System

30 Antennae

http://gmrt.ncra.tifr.res.in

GMRT Prototype

•  Objective – Re-engineering ‘ABC’ and ‘Teleset’ – Collaboration among TCS, NCRA and CoEP

•  Challenges – Scientists need a simple, extensible interface to

• Monitor and control antennae •  Schedule experiments

•  Approach –  ABC as collection of Rails web services –  Teleset re-designed as a Ruby DSL

‘Teleset’ as DSL: Version 1.0

a1 = AntennaClient.new (“http://antenna1”) a1.reboot a1.monitor 2.mhz

Single antenna

‘Teleset’ as DSL : Version 1.1

a1 = AntennaClient.new (“http://antenna1”) a1.reboot a1.monitor 2.mhz

Simultaneously, for antennae a1 and a2

engine.register_participant :antenna do | antenna | reboot monitor 2.mhz end

concurrent_iterator on_value => [:a1,:a2], to_variable => ‘antenna’ do participant :antenna => ’${antenna}’

end #Using openwferu engine

Complex !!!

Single antenna

http://openwferu.rubyforge.org - Using OpenWFEru Workflow engine

‘Teleset’ as DSL : Version 2.0

with antenna :a1, :a2 do reboot monitor 2.mhz

end Much

simpler !

engine.register_participant :antenna do | antenna | reboot monitor 2.mhz end

concurrent_iterator on_value => [:a1,:a2], to_variable => ‘antenna’ do participant :antenna => ’${antenna}’

end

Suggested prototype

EXAMPLE 2: DSL FOR VISUALIZATION

DSL for visualization

•  Objective –  A specification-driven dashboard

•  Visualization metaphors (charts, data grids, etc) •  Organization using layouts (window, tab, portal, etc) •  Navigation (page flows)

•  Challenge –  Consistent API –  Integration with other components and environment

•  Ruby was chosen

application ‘ThoughtWorks MCS Demo’ do

add grid ‘Actors list’ do

data_source table do table_name ‘actor’ end # data_source

end # grid end # application

application ‘Thoughtworks MCS Demo’ do add grid ‘Actors list’ do data_source table do table_name ‘actor’ end # data_source

add view ‘Show movies’ do | actor | add grid “Movies for actor #{actor}” do data_source query do text “SELECT … WHERE actor_id=#{actor.actor_id}” end # data_source end # grid end # view end # grid end # application

A STEP FURTHER Advanced topics

Evolution of a DSL •  Generalization versus specialization •  “expressiveness x scope = constant” [3]

DSL

GPL

External DSL

Internal DSLs

ASM GPL

scope scope

expressiveness

expressiveness

Three aspects

Domain aspect

Specification aspect

Language aspect

Domain-specific Language

Three aspects

Domain •  CRUD of

•  Domain entities •  Relationships •  Processes •  Constraints

Specification •  Conditionality (if/

switch) •  Automation

(loops) •  Reusability

(function/classes) •  Data structures •  Error handling

Language •  Natural •  Syntactic noise •  Semantic noise

Three aspects








Three aspects








Evolution of a DSL

Entities (numbers + strings)

Entities (+ relationships)

Conditions and loops

Reusability

Specialization (inheritance)

Crossroads and crosswords

•  “No domain is an island” •  Interoperability in DSLs

•  DSLs need to talk one-another •  Achilles’ Heel for external DSLs •  Parallel development of different DSLs needs early

standardization •  Chicken-egg problem

Future of DSLs •  UML and DSL

•  DSL as front-end to UML (or alternative)[5]

Book “MDA Distilled” (page 16)



•  High assurance systems •  Minimal code, relevant code



•  High assurance systems •  Minimal code, relevant code

•  Multi-core revolution •  Multi-threading •  Message passing

The Free Lunch Is Over – Herb Sutter’s article on Multi-core Revolution

References and resources 1.  Interview of Bjarne Stroustrup 2.  Presentation by Martin Fowler (at InfoQ.com) 3.  Domain-Specific Languages in Perspective 4.  A Picture is Worth a 1000 Words? 5.  Book “MDA Distilled” (page 16) 6.  The Free Lunch Is Over

Language Design is

Library Design

Library Design is

Language Design

Bjarne Stroustrup [1]

Contacts

[email protected] [email protected]

Software

DSL Construction with Ruby - Masterclass Series