Upload
odalys-wisdom
View
216
Download
0
Tags:
Embed Size (px)
Citation preview
[ 1 ]Claus Brabrand, ITU BANANA ALGEBRA March 28, 2009ETAPS/LDTA, York
Banana Algebra:Banana Algebra:
Jacob Andersen[ [email protected] ]
Aarhus University
Claus Brabrand[ [email protected] ]
IT University of Copenhagen
Syntactic Language Extension via an Algebra of Languages and Transformations
[ 2 ]Claus Brabrand, ITU BANANA ALGEBRA March 28, 2009ETAPS/LDTA, York
AbstractAbstract
We propose an algebra of languages and transformations as a means for extending languages syntactically. The algebra provides a layer of high-level abstractions built on top of languages (captured by CFGs) and transformations (captured by constructive catamorphisms).
The algebra is self-contained in that any term of the algebra specifying a transformation can be reduced to a constant catamorphism, before the transformation is run. Thus, the algebra comes "for free" without sacrificing the strong safety and efficiency properties of constructive catamorphisms.
The entire algebra as presented in the paper is implemented as the Banana Algebra Tool which may be used to syntactically extend languages in an incremental and modular fashion via algebraic composition of previously defined languages and transformations. We demonstrate and evaluate the tool via several kinds of extensions.
[ 3 ]Claus Brabrand, ITU BANANA ALGEBRA March 28, 2009ETAPS/LDTA, York
Introduction: "What is a Banana?"
Bananas for Language Transformation
Language Extension Pattern
Banana Algebra
Examples
Implementation
Related Work
Conclusion
OutlineOutline
[ 4 ]Claus Brabrand, ITU BANANA ALGEBRA March 28, 2009ETAPS/LDTA, York
What is a 'Banana' ?What is a 'Banana' ?
Datatype; "list":
Banana ("sum-of-list"):
Separation of recursion and evaluation
Implicit recursion on input structure
bottom-up re-combination of intermediate results
list = Num N | Cons N * list
[Num n] = n[Cons n l] = n + [l]
list N
(aka. "Catamorphism" )
(| n.n , (n,l ).n+l |)
[ 5 ]Claus Brabrand, ITU BANANA ALGEBRA March 28, 2009ETAPS/LDTA, York
Language TransformationLanguage Transformation
Bananas (statically typed):
Source language: 'LS'
Target language: 'LT'
Nonterminal-typing: ' '
Reconstructors: 'c '
list = Num N | Cons N * list
tree = Nil | Leaf N | Node N * tree * tree
[Num n] = Leaf n[Cons n l] = Node n (Nil) [l]
[list -> tree]
(| LS -> LT [] c |)
Type-check'able!
LS -> LT
[ 6 ]Claus Brabrand, ITU BANANA ALGEBRA March 28, 2009ETAPS/LDTA, York
Banana properties:Simple (corresponds to: “simple recursion”)Safe (syntactically safe + always terminate)Efficient (linear time in size of input + output)(Expressive) (…enough for interesting extensions)
Banana Algebra “for free” (16 banana ops):ModularIncrementalSimpleSafeEfficient(Expressive)
Statically reduce: Banana Algebra (term) Banana (term)
Banana PropertiesBanana Properties"The metafront System: Safe and Extensible Parsing and Transformation" [ Claus Brabrand | Michael Schwartzbach ] ( LDTA 2003, SCP J. 2007 )
"Growing Languages with Metamorphic Syntax Macros" [ Claus Brabrand | Michael Schwartzbach ] ( PEPM 2002 )
[ 7 ]Claus Brabrand, ITU BANANA ALGEBRA March 28, 2009ETAPS/LDTA, York
Introduction: "What is a Banana?"
Bananas for Language Transformation
Language Extension Pattern
Banana Algebra
Examples
Implementation
Related Work
Conclusion
OutlineOutline
[ 8 ]Claus Brabrand, ITU BANANA ALGEBRA March 28, 2009ETAPS/LDTA, York
Language Extension PatternLanguage Extension Pattern
[var V] = var V
[lam V E] = lam V [E ]
[app E1 E2] = app [E1] [E2]
[zero] = lam z (var z)
[succ E] = lam s [E ]
[pred E] = app [E ] (lam z (var z))
Numeral extension: Lambda-Calculus:
Nonterminal typing:
Reconstructors:
[Exp -> Exp]
Exp : var Id : lam Id * Exp : app Exp * Exp : zero : succ Exp : pred Exp
Exp : var Id : lam Id * Exp : app Exp * Exp
'LT''LS'
' '
'c '
(| LS -> LT [] c |)Catamorphism:
Using very simple numeral encoding
[ 9 ]Claus Brabrand, ITU BANANA ALGEBRA March 28, 2009ETAPS/LDTA, York
Algebraic SolutionAlgebraic Solution
Exp : var Id : lam Id * Exp : app Exp * Exp
Exp : zero : succ Exp : pred Exp
(| ln -> l [Exp -> Exp]
[zero] = lam z (var z) [succ E] = lam s [E ] [pred E] = app [E ] ...|)
idx
+
lnl
ln l
ln+l l
l l
[ 10 ]Claus Brabrand, ITU BANANA ALGEBRA March 28, 2009ETAPS/LDTA, York
Banana AlgebraBanana Algebra
Languages (L):
lv
L \ L
L + L
src ( X )
tgt ( X )
let v = L in L
letx w = X in L
Transformations (X):
xw
X \ L
X + X
X X
idx ( L )
let v = L in X
letx w = X in X
(| L -> L [] c |){ CFG }
[ 11 ]Claus Brabrand, ITU BANANA ALGEBRA March 28, 2009ETAPS/LDTA, York
Algebraic LawsAlgebraic Laws
Idempotency of '+':
Commutativity of '+':
Associativity of '+':
Source-identity:
…
L L + L
L1 + L2 L2 + L1
L1 + (L2 + L3) (L1 + L2) + L3
L src(idx(L)) L tgt(idx(L))Target-identity:
[ 12 ]Claus Brabrand, ITU BANANA ALGEBRA March 28, 2009ETAPS/LDTA, York
Introduction: "What is a Banana?"
Bananas for Language Transformation
Language Extension Pattern
Banana Algebra
Examples
Implementation
Related Work
Conclusion
OutlineOutline
[ 13 ]Claus Brabrand, ITU BANANA ALGEBRA March 28, 2009ETAPS/LDTA, York
Example RevisitedExample Revisited
{ Id = [a-z] [a-z0-9]* ;
Exp.var : Id ; Exp.lam : "\\" Id "." Exp ; Exp.app : "(" Exp Exp ")" ;}
{ Exp.zero : "zero" ; Exp.succ : "succ" "(" Exp ")" ; Exp.pred : "pred" "(" Exp ")" ;}
--- "l.l" --- --- "ln.l" ---
let l = "l.l"in let ln = "ln.l"in idx(l) + (| ln -> l [Exp -> Exp] Exp.zero = '\z.z' ; Exp.succ = '\s.$1' ; Exp.pred = '($1 \z.z)' ; |)
--- "ln2l.x" ---
[ 14 ]Claus Brabrand, ITU BANANA ALGEBRA March 28, 2009ETAPS/LDTA, York
Numerals + BooleansNumerals + Booleans
l
lb l idx
+lb+l l
l l
lb l
ln l idx
+ln+l l
ln
+l+ln+lb l
…with Nums …with Bools
…with Nums & Bools?
[ 15 ]Claus Brabrand, ITU BANANA ALGEBRA March 28, 2009ETAPS/LDTA, York
Java + RepeatJava + Repeat
let java = "java.l"in let repeat = "repeat.l"in idx(java) + (| repeat -> java [Exp -> Exp, Stm -> Stm]
Stm.repeat = 'do $1 while (!($2));' ; |)
{ Stm.repeat : "repeat" Stm "until" "(" Exp ")" ";" ; }
7 lines !
{ Java ... "try" Stm "catch" ... Name.id : Id ; }
575 lines--- "java.l" ---
--- "repeat.l" ---
--- "repeat2java.x" ---
[ 16 ]Claus Brabrand, ITU BANANA ALGEBRA March 28, 2009ETAPS/LDTA, York
Concrete vs. Abstract SyntaxConcrete vs. Abstract Syntax
Exp.or : Exp1 "||" Exp ; .exp1 : Exp1 ;Exp1.and : Exp2 "&&" Exp1 ; .exp2 : Exp2 ;Exp2.add : Exp3 "+" Exp2 ; .exp3 : Exp3 ; Exp7.neg : "!" Exp8 ; .exp8 : Exp8 ;Exp8.par : "(" Exp ")" ; .var : Id ; .num : IntConst ;
Exp (with explicit assoc./prec.):Stm.repeat = Stm.do(<1>, Exp.exp1( Exp1.exp2( Exp2.exp3( Exp3.exp4( Exp4.exp5( Exp5.exp6( Exp6.exp7( Exp7.neg( Exp8.par(<2>) ))))))))) ;
Abstract syntax:
Stm.repeat = 'do $1 while (!($2));' ;
Concrete syntax:
NB: Tool supports BOTH ! (unambiguous: concrete abstract)
[ 17 ]Claus Brabrand, ITU BANANA ALGEBRA March 28, 2009ETAPS/LDTA, York
"FUN" Example"FUN" Example
Lambda Calculus
Nums→
Unsigned arithmetic + booleans + definitions + pairs
Bools→ Defs→ Pairs→+ + +
Fun
Literals
Literals→Nums
Fun grammar transform
The "FUN" Language: used for Teaching Functional Programming
Basically The Lambda Calculus with…: numerals, booleans, arithmetic, boolean logic, local definitions, pairs, literals, lists, signs, comparisons, dynamic types, fixed-point combinators, …
(at Aarhus University)
[ 18 ]Claus Brabrand, ITU BANANA ALGEBRA March 28, 2009ETAPS/LDTA, York
Literals
Literals→Nums
"FUN" Example"FUN" Example
Lambda Calculus
Nums→
Unsigned arithmetic + booleans + definitions + pairs
Bools→ Defs→ Pairs→+ + +
Fun
Fun grammar transform
Signed arith→Nums
Literals→Nums
Fun grammar transform FunSigned GT+
Fun FunSigned+Component re-use
[ 19 ]Claus Brabrand, ITU BANANA ALGEBRA March 28, 2009ETAPS/LDTA, York
"FUN" Example"FUN" Example
Lambda Calculus
Nums→
Unsigned arithmetic + booleans + definitions + pairs
Bools→ Defs→ Pairs→+ + +
Fun GT FunSigned GT+
Fun FunSigned+ FunCompare FunTypesafe+ +
FunCompare GT+ FunTypesafe GT+
245x Banana Algebra ops 4 MB Banana !
[ 20 ]Claus Brabrand, ITU BANANA ALGEBRA March 28, 2009ETAPS/LDTA, York
"FUN" Usage Statistics"FUN" Usage Statistics
Usage statistics (245x operators) in "FUN":58x { …cfg… } Constant languages51x "file.l" Language inclusions28x L + L Language additions23x v Language variables17x (|L L [] c|) Constant transformations17x X + X Transformation additions14x "file.x" Transformation inclusions10x let-in Local definitions 9x idx(L) Identity transformations 8x X X Compositions 4x L \ L Language restriction 4x w Transformation variables 2x src(X) Source extractions
[ 21 ]Claus Brabrand, ITU BANANA ALGEBRA March 28, 2009ETAPS/LDTA, York
Other ExamplesOther Examples
Self-Application (The tool on itself!):
SQL embedding (in <bigwig>):
My-Java (endless variations):
[L1 << L2] = '(L1 \ L2) + L2' [X1 << X2] = '(X1 \ src(X2)) + X2'
Stm.select = 'factor (<2>) { if (<3>) return ( # \+ (<1>) ); }'
java( + sql) ( \ loops) o syntaxe_francais
[ 22 ]Claus Brabrand, ITU BANANA ALGEBRA March 28, 2009ETAPS/LDTA, York
ImplementationImplementation
[ http://www.itu.dk/people/brabrand/banana-algebra/ ]
The 'Banana Algebra' Tool:(3,600 lines of O'Caml)
Uses (underlying technologies): 'dk.brics.grammar': for parsing, unparsing, and ambiguity analysis ! 'XSugar': for transformation: "concrete syntax abstract XML syntax" 'XSLT': for transformation: "XML XML"
[ 23 ]Claus Brabrand, ITU BANANA ALGEBRA March 28, 2009ETAPS/LDTA, York
Introduction: "What is a Banana?"
Bananas for Language Transformation
Language Extension Pattern
Banana Algebra
Examples
Implementation
Related Work
Conclusion
OutlineOutline
[ 24 ]Claus Brabrand, ITU BANANA ALGEBRA March 28, 2009ETAPS/LDTA, York
Related Work (I/III)Related Work (I/III)
Macro Systems:
"The metafront System: Safe and Extensible Parsing and Transformation" [ Claus Brabrand | Michael Schwartzbach ] ( LDTA 2003 , SCP J. 2007 )
"Growing Languages with Metamorphic Syntax Macros" [ Claus Brabrand | Michael Schwartzbach ] ( PEPM 2002 )
[ 25 ]Claus Brabrand, ITU BANANA ALGEBRA March 28, 2009ETAPS/LDTA, York
Related Work (II/III)Related Work (II/III)
Attribute Grammars:Language transformation (and extension)…
…via computation on AST's
(using "inherited" or "synthesized" or … attributes)
E.g., Eli, JastAdd, Silver, …
Rewrite Systems:Language transformation (and extension)…
…via syntactic rewriting, using encodings…:
gradually rewrite "S-syntax" to "T-syntax"
gradually rewrite "S-syntax" to "T-syntax"
E.g., Elan, TXL, ASF+SDF, Stratego/XT, …
S T
S T
Both; compared to bananas: More ambitious (expressivity) No termination guarantees (safety) Transformation "indirect" (simplicity)
[ 26 ]Claus Brabrand, ITU BANANA ALGEBRA March 28, 2009ETAPS/LDTA, York
Related Work (III/III)Related Work (III/III)
Functional Programming:Catas mimicked by "disciplined style" of fun. programming
…aided by:Traversal functions (auto-synthesized from datatypes)
Combinator libraries
"Shortcut fusion" (to eliminate ' ' at compile-time)
Category Theory:A lot of this work can be viewed as Category Theory:
Basically yeolde issue:GPL vs. DSL
[ 27 ]Claus Brabrand, ITU BANANA ALGEBRA March 28, 2009ETAPS/LDTA, York
ConclusionConclusion
IF bananas are sufficiently:(Expressive)
THEN you get…:
Banana Algebra “for free” (16 banana ops):IncrementalModularSimpleSafeEfficient
"Niche"
Statically reduce: Banana Algebra (term) Banana (term)
[ 28 ]Claus Brabrand, ITU BANANA ALGEBRA March 28, 2009ETAPS/LDTA, York
BONUS SLIDESBONUS SLIDES
- Reduction Semantics -
"Syntactic Language Extension via an Algebra of Languages and Transformations" [ Jacob Andersen | Claus Brabrand ] ( ITU Technical Report, Dec. 2008 )
If you want all the details:
[ 29 ]Claus Brabrand, ITU BANANA ALGEBRA March 28, 2009ETAPS/LDTA, York
Reduction SemanticsReduction Semantics
Environments:
Reduction relations:
Abbreviations: ...as a short-hand for:
...as a short-hand for:
ENVL = VARL EXPL
ENVL ENVX EXPL EXPL 'L'
ENVX = VARX EXPX
ENVL ENVX EXPX EXPX 'X'
, |- L L l (,,L,l) 'L'
(,,X,x) 'X', |- X X x
environment of languages
environment of transformations
[ 30 ]Claus Brabrand, ITU BANANA ALGEBRA March 28, 2009ETAPS/LDTA, York
Semantics (L)Semantics (L)
, l L l[CONL] , v L (v)
[VARL]
, L \ L' L l l'[RESL]
, L' L l', L L l
, L + L' L l l'[ADDL]
, L' L l', L L l
l
ll ~ l'l
lwfl
[ 31 ]Claus Brabrand, ITU BANANA ALGEBRA March 28, 2009ETAPS/LDTA, York
Semantics (L)Semantics (L)
, src (X) L lS
[SRCL], X X (| lS -> lT [] c |)
, tgt (X) L lT
[TGTL], X X (| lS -> lT [] c |)
, let v=L in L' L l'[LETL]
[v=l], L' L l', L L l
[ 32 ]Claus Brabrand, ITU BANANA ALGEBRA March 28, 2009ETAPS/LDTA, York
Semantics (X)Semantics (X)
, X \ L X x l[RESX]
, L L l, X X x
, X + X' X x x'[ADDX]
, X' X x', X X x
x
xx ~ x'x
, (| LS -> LT [] c |) X (| lS -> lT [] c |)[CONX]
, LS L lS , LT L lT(| lS -> lT [] c |)wfx
, w X (w)[VARX]
[ 33 ]Claus Brabrand, ITU BANANA ALGEBRA March 28, 2009ETAPS/LDTA, York
Semantics (X)Semantics (X)
, X' X X (| lS -> lT' [' ] c' c |)[COMPX]
, X' X (| lS' -> lT' ['] c' |)
, idx (L) X (| l -> l [id(l)] idc(l) |)[IDXX]
, letx w=X in X' X x'[LETL]
,[w=x] X' X x', X X x
, X X (| lS -> lT [ ] c |)
, L L l
lT lS'l
[ 34 ]Claus Brabrand, ITU BANANA ALGEBRA March 28, 2009ETAPS/LDTA, York
BONUS SLIDESBONUS SLIDES
- More Examples -
[ 35 ]Claus Brabrand, ITU BANANA ALGEBRA March 28, 2009ETAPS/LDTA, York
Numeral & Boolean ExtensionNumeral & Boolean Extension
Numeral Extension (catamorphism):
Boolean Extension (catamorphism):
[var V] = var [V ]
[lam V E] = lam [V ] [E ]
[app E1 E2] = app [E1] [E2]
[zero] = lam z (var z)
[succ E] = lam s [E ]
[pred E] = app [E ] (lam z (var z))
[var V] = var [V ] [lam V E] = lam [V ] [E ] [app E1 E2] = app [E1] [E2] [true] = lam a (lam b (var a)) [false] = lam a (lam b (var b))[if E1 E2 E3] = app (app [E1] [E2]) [E3]
Exp : var Id : lam Id * Exp : app Exp * Exp : zero : succ Exp : pred Exp
Exp : var Id : lam Id * Exp : app Exp * Exp
Exp : var Id : lam Id * Exp : app Exp * Exp : true : false : if Exp Exp Exp
Exp : var Id : lam Id * Exp : app Exp * Exp
[ 36 ]Claus Brabrand, ITU BANANA ALGEBRA March 28, 2009ETAPS/LDTA, York
Lambda with BooleansLambda with Booleans
Exp : true : false : if Exp Exp Exp
(| lb -> l [Exp -> Exp]
[true] = '\a.\b . a' [false] = '\a.\b . b'[if E1 E2 E3] = '(([E1] [E2]) [E3])'
|)
idx
+
lbl
lb l
lb+l l
l l
Exp : var Id : lam Id * Exp : app Exp * Exp
[ 37 ]Claus Brabrand, ITU BANANA ALGEBRA March 28, 2009ETAPS/LDTA, York
Incremental DevelopmentIncremental Development
let l = "l.l" in idx(l) + (| "ln.l" -> l [Exp -> Exp] Exp.zero : '\z.z' ; Exp.succ : '\x.$1' ; Exp.pred : '($1 \z.z)' ; |)
let l = "l.l" in idx(l) + (| "li.l" -> l [Exp -> Exp] Exp.id : '\z.z' ; |)
{ Exp.zero : "zero" ; Exp.succ : "succ" Exp ; Exp.pred : "pred" Exp ; }
{ Id = [a-z] [a-z0-9]* ; Exp.var : Id ; Exp.lam : "\\" Id "." Exp ; Exp.app : "(" Exp Exp ")" ; }
--- "ln.l" ---
--- "ln2li.x" ---
--- "l.l" ---
--- "ln2l.x" ---let l = "l.l" in idx(l) + (| ln -> l+"li.l" [Exp -> Exp] Exp.zero : 'id' ; Exp.succ : '\x.$1' ; Exp.pred : '($1 id)' ; |)
{ Exp.id : "id" ; }--- "li.l" ---
--- "li2l.x" ---
"li2l.x" o "ln2li.x"
--- "ln2l.x" ---
[ 38 ]Claus Brabrand, ITU BANANA ALGEBRA March 28, 2009ETAPS/LDTA, York
Example cont'dExample cont'd
Both statically reduce to same catamorphism:(|
Exp.app : Exp.app($1, $2) ; Exp.lam : Exp.lam($1, $2) ; Exp.pred : Exp.app($1, Exp.lam(Id("z"), Exp.var(Id("z")))) ; Exp.succ : Exp.lam(Id("x"), $1) ; Exp.var : Exp.var($1) ; Exp.zero : Exp.lam(Id("z"), Exp.var(Id("z"))) ;|)
{ Id = [a-z] [0-9a-z]* ;
Exp.app : "(" Exp Exp ")" ; Exp.lam : "\" Id "." Exp ; Exp.var : Id ; }
{ Id = [a-z] [0-9a-z]* ;
Exp.app : "(" Exp Exp ")" ; Exp.lam : "\" Id "." Exp ; Exp.pred : "pred" Exp ; Exp.succ : "succ" Exp ; Exp.var : Id ; Exp.zero : "zero" ; }
->
[Exp -> Exp, Id->Id]
[ 40 ]Claus Brabrand, ITU BANANA ALGEBRA March 28, 2009ETAPS/LDTA, York
Usage ScenariosUsage Scenarios
Programmers:May extend existing languages (~ syntax macros)
Developers:May embed DSLs into host languages (SQL in Java)
Developers (and teachers):May incrementally specify multi-layered languages
Compiler writers:May rely on tool and implement only a small core
(and then specify the rest externally as extensions)
[ 41 ]Claus Brabrand, ITU BANANA ALGEBRA March 28, 2009ETAPS/LDTA, York
BONUS SLIDESBONUS SLIDES
- Parsing & Error Reporting -
[ 42 ]Claus Brabrand, ITU BANANA ALGEBRA March 28, 2009ETAPS/LDTA, York
ParsingParsing
Parsing (XSugar):Variant of Earley's algorithm: O ( ||3 )
Can parse any context-free grammar
Closed under union of languages
Support for production priority
Tool easily adapts to other parsing algorithms
[ 43 ]Claus Brabrand, ITU BANANA ALGEBRA March 28, 2009ETAPS/LDTA, York
Unparsing:Canonical whitespace
Ambiguity: parsingunparsingAmbiguity: parsingunparsing
ASTL / ~L
.
L
. Parsing:Grammar ambiguity
ASTL / ~L
.
L
.
.
[ 44 ]Claus Brabrand, ITU BANANA ALGEBRA March 28, 2009ETAPS/LDTA, York
Ambiguity AnalysisAmbiguity Analysis
Ambiguity Analysis:
Using implementation ( ) on:Source language;
Target language; and/or
…all intermediate languages (somewhat expensive)
(Note: Ambiguity analysis comes with XSugar tool)
"Analyzing Ambiguity of Context-Free Grammars" [ Claus Brabrand | Robert Giegerich | Anders Møller ] ( CIAA 2007 )
"dk.brics.grammar"[ by Anders Møller ]
[ 45 ]Claus Brabrand, ITU BANANA ALGEBRA March 28, 2009ETAPS/LDTA, York
Error ReportingError Reporting
Error reporting:Static parse-error (O'Caml-lex):
Static transformation error (XSugar):
(is actually a parse-error in a cata reconstructor)
Dynamic parse-error (XSugar):
Dynamic transformation error:impossible :-)
*** In ln2l.x (4,4)-(4,7): Parse error at "Exp"
*** Parse error at character 6 (line 1, column 7) in /tmp/shape84e645.txt
*** Parse error at character 23 (line 1, column 24) in /dev/stdin
Could beimproved
Prototype