MUp Early Preview V6

A Functional Logic Programming Mashup Language

M ( em-’up )

PreviewEarly !

1Monday, March 2, 2009

Agenda• What is MUp?

• Features

• How it works

• Console & Tools

• MUp architecture

• MU language

• Examples

• Validation scenarios

• Comparison with similar products

• Results

• RoadMap

• Project Resources


What is MUp (1)

• MUp stay for Mash Up.

• MUp is a framework for creating mashup of web contents.

• MUp is based on a specific high level language called MU.

• MUp is written in Java.

• MUp defines two runtime profiles : JRE and Javascript.


What is MUp (2)

• MUp is compiled in Javascript with GWT (Google Web Toolkit).

• MUp Javascript profile runs entirely client side (inside a browser).

• MUp can be used client side to define Javascript widgets embedded in Web pages.

• MUp can be used server side to integrate heterogeneous contents.


MUp Features• Simplifies data aggregation and transformation.

• Provides support for unification of JSON objects over JSON models.

• Provides Graph manipulation native support.

• Provides Type Morphing support.

• Provides (forward) XPath native support.

• Provides JSONPath native support.

• Provides UI Induction capabilities.


How it works (1)MUp supports a special language to perform data mashing, this language is called MU.

MU is both a logic and functional language, it provides also procedural constructs.

All predicates provided by MU can be grouped in functional sets.

The main functional sets are:

Source Operations: allow to retrieve web resources.

Inspect Operations: allow to decompose data retrieved from Web.

Model Operations: allow to create JSON models from raw data.

Renderization Operations: allow to renderize JSON models.


How it works (2)MU functional sets:

String Operations

RegexSubstring

Source Operations

Get Post

Model Operations

Jsonize

Renderization Operations

Renderize

Flow Control

if for

Data operations

Add Put

Remove

Context operations

Context PContext

StackTrace

Type Management

As<Type>Type

Math Operations

+-

*

/

%

Conditional / Logical Operations

== <

!= ANDOR

Inspect Operations

JPathXPath

Type

Apply

Unify


A simple processing flow

Source Operations

Get Post

Inspect Operations

PathType

Model Operations

Jsonize

Renderization Operations

Renderize

Source() : Get('http://www.mysource.org/rss');

Entries() : Path('/rss/item/description', Source() );

Model() : Jsonize('{ entry : "_" }', Entries() );

GUI() : Renderize( Model() );

ab

c

d

ab

c

d

[a,b,c,d]

d

a

b

c


A complex processing flow

Source Operations

Get Post

Inspect Operations

PathType

Model Operations

Jsonize

Renderization

Operations

Renderize

Source Operations

Get Post

Inspect Operations

PathType

ab

cd

ab

cd

[ [a,b,c,d] , [e,f,g,h] ]

ef

gh

ef

gh

d

a

b

c

h

e

f

g


MUp Console (1)

MUp provides a Web console that can be executed inside any browser supported by GWT (Firefox, Safari, IE).

MUp Console provides a text area to launch commands and an operation’s history.


MUp Console (2)

Console Tab

Executed command

Command result

command input area


MUp Console: tools (1)

JSON Inspector

Editable

Tree viewInspector Panel

Editable

Text view


MUp Console: tools (2)

Context Inspector

Native predicates

Inspector Panel

Predicate description

panel

User defined

predicates


MUp Architecture (1)

MU Core

Native

JRE Native Impl GWT Native Impl

Parser Processor DigesterCast

ManagerGUI

ToolsConsole



Console: the MU console.

Tools: the tools included in the console.

MU Core : provides core functionalities for the MUp framework.

Parser: parser for MU language.

GUI: GUI abstraction able to renderize JSON objects in widgets.



Processor: abstract processor able to interpret compiled MU operations.

Cast Manager: provides support for type morphing.

Digester: XML/HTML generic path digester.

Native: abstraction layer of native implementation.

JRE Native : implementation of Native layer in Java Runtime Environment.

GWT Native : implementation of Native layer on GWT Javascript.


MU Language (1)MU is based on an hybrid programming paradigm based on functional, procedural and logic constructs.

An example of functional expression is:

Function(a,b) : sum( F1(a), F2(b) );

An example of logic expression is:

Predicate(a,b) : P1(a) & P2(b) | P3(a,b);

An example of procedural expression is:

Procedure(a,b) : r1=Process(a), r2=Process(b), if( Gt(r1,r2), Process(r1), Process(r2) );


MU Language (2)

BooleanValue NumericValue StringValue ListValue MapValue GraphValue JSONValue

MU native types

boolean integer / float string array object object object

JSON counterpart


MU Language (3)MU language types are polymorphic: every native type can be alwayscasted to another native type.Casting among primitive types is managed by Cast Manager.

Boolean

Numeric

String

List

Map

Non primitive

JSON

Graph

DevelopmentUnder !


Predicate overloading is based on unification: given a set of predicates with the same name, it will be invoked the first predicate in order of definition which signature unifies with given arguments. Unlike Erlang/Prolog, MU implements unification over JSON data structures.

MU supports Erlang/Prolog language features like predicate overloading.

Factorial(0) : 1;Factorial(n) : Mult( n , Factorial( Minus(n,1) ) );

MU Advanced Language (1)


MU Advanced Language (2)

Unification samples:

# Basic unification #P1(a, b, c) =::= (List(1,2,3)) ==> P1(a:=1,b:=2,c:=3)

# List unification. #P1([a,b|c]) =::= (List(1,2,3,4,5)) ==> P1(a:=1,b:=2,c:=[3,4,5])

# Object unification. #P1( { “k1” : a, “k2” : b | o} ) =::= ({“k1” : 1 , “k2” : “2”, “k3” : “3”}) ==> P1(a:=1,b:=”2”,c:={ “k3” : “3” })


# This is a comment. #

PI() : 3.14; # Declaring a const. #

Const() : "This string is const.";

# Functional programming. #TriangleArea(l1, l2) : Div( Mult(l1,l2), 2 );MinorOf(a,b) : if( Lt(a,b), a, b );

# Iterative constructs.PrintRange(start, stop) : for( Range(start,stop,1), x, Print(x) );

MU Language Examples (1)


# Procedural programming. #

P1(v1, v2) : v3=P2(v1, k1), v4=P3(v3,k2), P4(v4);

P1(v1, v2); # Invoking a predicate. #

v1=P1(v2, v3), P2(v1, v4); # Complex invocation. #

# Predicate Overloading #OL([a,b,c] , v) : Print('overload1');OL([a,b,c,d], v) : Print('overload2');OL({"k1" : v1, "k2" : v2} , v) : Print('overload3');OL({"k1" : v1, "k2" : v2, "k3" : v3} , v) : Print('overload4');

# Invocation #OL(List(1,2,3) , 4); # prints: overload1 #OL(List(1,2,3,4), 5); # prints: overload2 #OL(Map('k1',1,'k2',2) , 4); # prints: overload3 #OL(Map('k1',1,'k2',2,'k3',3), 5); # prints: overload4 #



# JSON Path integration #

var = DoSomething(), var.p1.p2.p3[i];

# XPath integration #

var = GetFeed(), var//title;

DevelopmentUnder !



MU UI Model

UIComponent

UIContainer

UILabel

UIButton

UIPanel

UITextArea

UIList

UIWindow


GUI Induction (1)

The Renderize() predicate provides the hidden processing flow shown on the right.

The GUI Induction is the ability of the language of automatically generate a GUI for a JSON data.

The Modelize() operation generates a JSON UI Model by applying a set of default transformations.

The JSON UI Model can be customized by using the Apply() operation.

DevelopmentUnder ! Renderize()

Modelize()

JSON data

Concretize()

Out

In

GUI native model

JSON UI Model

Apply()

To customize the GUI generation it is possible to use the couple Modelize() / Concretize() instead of the Renderize() operation.

The Concretize() operation transforms a JSON UI Model in a GUI Native Model .


GUI Induction (2)Examples of GUI Inductions based on the unification are:

# Change Background color. #ChangeBG( { “background” : bg | o } ) : Add(o, Map(“background”, “red”) );

DevelopmentUnder !

# Change panel orientation if size exceed 10. #ChangePO( { o } ) : if( Gt( Size(o), 10 ) , Add(o, Map(“orientation”, “vertical”) ) );


Validation scenarios

Validation scenarios are applications of MUp on real and synthetic data meant to verify the applicability of all the implemented features.


Synthetic validation scenario: description

“We have a Web service S1 returning a list of restaurants in a specified city, providing for each restaurant the street in which it is located.

We've also a Web service S2 returning a list of hotels near a given street.

We want to combine these sources to obtain a list of restaurants in a given city with a sublist of hotels near each restaurant.”


Synthetic validation scenario: formalization

#01# S1(city) : PGet('http://findrestaurants.com/find?city=_', city);#02# RawRomeRestaurants() : S1('Rome');#03# ListOfRomeRestaurants() : Path( 'html/body/table/tr', RawRomeRestaurants() );#04# RestaurantName(row) : Path( 'td[0]' , row );#05# RestaurantStreet(row) : Path( 'td[1]' , row );#06# S2(city,street) : PGet('http://findhotels.com/find?city=_street=_', List(city, street) );#07# HotelsInRomeAtStreet(street) : S2('Rome', street);#08# ListOfStreetHotels(street) : Path( 'html/body/table/tr' , HotelsInRomeAtStreet(street) );#09# HotelName(row) : Path( 'td[0]', row );#10# HotelStars(row) : Path( 'td[1]' , row );#11# JsonHotel(row) : Jsonize( '{ hotel-name : "_", hotel-stars : "_" }', List( HotelName(row), HotelStars(row) ) );#12# HotelsInStreet(street) : for( ListOfStreetHotels(street), hotel, JsonHotel(hotel) );#13# Restaurant(row) : Jsonize( '{ restaurant-name : "_", restaurant-street : "_", hotels : _ }', List(RestaurantName(row), street=RestaurantStreet(row), HotelsInStreet(street)) );#14# RomeRestaurants() : for( ListOfRomeRestaurants(), row, Restaurant(row) );


http://findhotels.com/find?city=_street=_'

http://findhotels.com/find?city=_street=_'

Synthetic validation scenario: result

[ { "restaurant-name" : "RN1Rome",

"restaurant-street" : "RS1Rome", "hotels" : [ { "hotel-name" : "HN1RomeRS1Rome", "hotel-stars" : "SR1RomeRS1Rome" }, { "hotel-name" : "HN2RomeRS1Rome", "hotel-stars" : "SR2RomeRS1Rome" }, { "hotel-name" : "HN3RomeRS1Rome", "hotel-stars" : "SR3RomeRS1Rome" }

] }, { "restaurant-name" : "RN2Rome", "restaurant-street" : "RS2Rome",

"hotels" : [ { "hotel-name" : "HN1RomeRS2Rome", "hotel-stars" : "SR1RomeRS2Rome" }, { "hotel-name" : "HN2RomeRS2Rome", "hotel-stars" : "SR2RomeRS2Rome" }, { "hotel-name" : "HN3RomeRS2Rome", "hotel-stars" : "SR3RomeRS2Rome" } ]

}, { "restaurant-name" : "RN3Rome", "restaurant-street" : "RS3Rome", "hotels" : [

{ "hotel-name" : "HN1RomeRS3Rome", "hotel-stars" : "SR1RomeRS3Rome" }, { "hotel-name" : "HN2RomeRS3Rome", "hotel-stars" : "SR2RomeRS3Rome" }, { "hotel-name" : "HN3RomeRS3Rome", "hotel-stars" : "SR3RomeRS3Rome" } ] }

]


Real validation scenario: description

“Given an RSS feed extract all title elements inside it and show a panel containing them.”


Real validation scenario: formalization

#01# Feed() : PGet("http://www.repubblica.it/rss/homepage/rss2.0.xml", List());

#02# Titles() : Path( 'title', Feed() );

#03# JsonTitle(t) : Jsonize('{ "title" : %s }', List(t));

#04# JsonTitles() : for( Titles(), title, JsonTitle(title) );

#05# Renderize( JsonTitles() );


http://www.repubblica.it/rss/homepage/rss2.0.xml

http://www.repubblica.it/rss/homepage/rss2.0.xml

Real validation scenario: result


Comparison with similar products


Results / Issues

• The GWT profile raises problems when recompiling with performance flags.

• GWT is not enough mature for the complexity of this project, in the following months resources will be invested on the JRE profile.


RoadMap (1)( http://code.google.com/p/em-up/wiki/RoadMap )

Ongoing Tasks

• Implement the GraphValue data type:

1. implement the core type. (2) DONE 2. implement the manipulation operations. (3) DONE 3. implement the triple syntax. (3) 4. add graph search and transformation operators. (3)

• Add support for JSON storaging and querying. (4)

• Empower the JSON Model syntax. (3)


http://code.google.com/p/em-up/wiki/RoadMap

http://code.google.com/p/em-up/wiki/RoadMap

RoadMap (2)

Rewrite the MUp Console Shell with Gwt-EXT. (3)

Complete the User Documentation. (3)

Complete the build system. (2)

Add command-line console. (3)

Complete the XPath support (some operators are missing).


RoadMap (3)

Complete the JSONPath support (some operators are missing).

Add Regexp support for string manipulation. (2)

TODOs reduction in code and documentation. (10)

Add arithmetic expression support in language. (2)

--- MILESTONE ---


Documentation & Code

The MU documentation is hosted in

and can be found here: http://code.google.com/p/em-up/

The MU source code will be released soon under the Apache License, Version 2.0 .


http://code.google.com/p/em-up/

http://code.google.com/p/em-up/

Articles

“MU: an hybrid language for Web Mashups”Davide Palmisano & Michele Mostarda

DevelopmentUnder !

TODO: add Article URL


Michele Mostarda (michele.mostarda)

Credits

michele.mostarda

[email protected]

http://www.asemantics.com

[email protected]


mailto:[email protected]






/* EOF */


Technology

MUp Early Preview V6