Metaprogramming from University to Industry Zoltán Porkoláb [email protected] Dept. of Programming Languages and Compilers, Faculty of

MetaprogrammingMetaprogrammingfrom University to Industryfrom University to Industry

ZoltZoltán Porkolábán Porkolá[email protected]@elte.hu

http://gsd.web.elte.huhttp://gsd.web.elte.hu

Dept. of Programming Languages and Compilers,Dept. of Programming Languages and Compilers,Faculty of InformaticsFaculty of Informatics

Eötvös Loránd University, BudapestEötvös Loránd University, Budapest

22OTS 2007 MariborOTS 2007 Maribor

AgendaAgenda

I always knew C++ templates were the work of the Devil, I always knew C++ templates were the work of the Devil,

and now I'm sure...and now I'm sure... - Cliff Click cited by Todd Veldhuisen - Cliff Click cited by Todd Veldhuisen

Parameterized types Parameterized types

C++ Template C++ Template MetaprogramsMetaprograms

Power of generative metaprogramsPower of generative metaprograms

Sample usagesSample usages

Open questionsOpen questions


Parameterized types (Generics) Parameterized types (Generics)

Widely used in modern programming languages:Widely used in modern programming languages:

ADA genericsADA generics

EiffelEiffel generics generics

C++ templatesC++ templates

Java generics: Pizza, GJ, Java 1.5Java generics: Pizza, GJ, Java 1.5

C#C# generics generics

Clean, Generic Haskell, Clean, Generic Haskell, other functional langs.other functional langs.


Why generics?Why generics?

int max( int a, int b) int max( int a, int b)

{ {

if ( a > b ) return a; else return b; if ( a > b ) return a; else return b;

} }

double max( double a, double b) double max( double a, double b)

{ {

if ( a > b ) return a; else return b; if ( a > b ) return a; else return b;

}}

//…//…

classclass datedate { { //* … */ }* … */ };;

date d = max ( d1, d2);date d = max ( d1, d2);

Conventional techniques are working only for Conventional techniques are working only for complete typescomplete types::


PreprocessorPreprocessor MacroMacro

#define MAX(a,b) a > b ? a : b#define MAX(a,b) a > b ? a : b

Works, because a macro is typeless.Works, because a macro is typeless.

Processed not by the compiler, therefore there are a Processed not by the compiler, therefore there are a

number of "secondary effectnumber of "secondary effectss":":

MAX( x, y)*2 ->MAX( x, y)*2 -> x > y ? x : y*2 x > y ? x : y*2

MAX( ++x, y) -> ++x > y ? ++x : y MAX( ++x, y) -> ++x > y ? ++x : y


Macro – the limitsMacro – the limits

void swap( int& x, int& y) void swap( int& x, int& y)

{{

int temp = x; x = y; y = temp; int temp = x; x = y; y = temp;

}}

Does not work with macro:Does not work with macro: a macro is typeless. a macro is typeless.

We need a facility to use type parameters.We need a facility to use type parameters.


TemplatesTemplates

template <typename T> template <typename T>

void swap( T& x, T& y) void swap( T& x, T& y)

{ {

T temp = x; x = y; y = temp; T temp = x; x = y; y = temp;

}}

AdaAda generics generics: "a form of context-sensitive macro" : "a form of context-sensitive macro"

C++ templates: "a clever kind of macro that obeys the C++ templates: "a clever kind of macro that obeys the scope, naming, and type rules of C++" scope, naming, and type rules of C++"

DDoes not require different types used as arguments to be oes not require different types used as arguments to be explicitly related. In particular, the argument types used as a explicitly related. In particular, the argument types used as a template need not be from a single inheritance hierarchy. template need not be from a single inheritance hierarchy.


C++ Function TemplatesC++ Function Templates

Template is not a single functionTemplate is not a single functionAA schema to instantiate functions on request schema to instantiate functions on request Parameter deductionParameter deduction Template instantiationTemplate instantiation

Compilaton time Compilaton time

template <template <typenametypename T> T max( T a, T b) T> T max( T a, T b) { { if ( a > b ) return a; if ( a > b ) return a; else return b; else return b; }}int i = 3, j = 6, k;int i = 3, j = 6, k;double x = 3.14, y = 4.15, z;double x = 3.14, y = 4.15, z;k = max(i,j); k = max(i,j); z = max(x,y);z = max(x,y);


C++ Function Templates 2.C++ Function Templates 2.

SStrong type system rules are appliedtrong type system rules are applied

int i = 3; double y = 3.14, z;int i = 3; double y = 3.14, z;z = max(i,y);z = max(i,y); // error // error

template <template <typenametypename T T, typename S, typename S> > T max( T a, T max( T a, SS b) b) { { if ( a > b ) return a; if ( a > b ) return a; else return b; else return b; }}

z == 3z == 3.0.0;;

No deduction on return typeNo deduction on return typeNo runtime information could be usedNo runtime information could be used


Explicit specializationExplicit specialization

Explicit specializationExplicit specialization

int i = 3;int i = 3;

double y = 3.14, z;double y = 3.14, z;

template <template <typenametypename R, typename R, typename TT, typename S, typename S> >

RR max( T a, max( T a, SS b) b)

{ {

if ( a > b ) return a; if ( a > b ) return a;

else return b; else return b;

}}

z = max<double>(i,y);z = max<double>(i,y);


User specializationUser specialization

const char *s1 = “Hello”const char *s1 = “Hello”, , *s2 = “world”;*s2 = “world”;

const char *s = max(s1,s2);const char *s = max(s1,s2);

template <template <>>

const char *const char *max(max(const char *aconst char *a, , const char *const char *b) b)

{ {

if ( if ( strcmp(a,b) < 0strcmp(a,b) < 0 ) return a; ) return a;

else return b; else return b;

}}


Template overloadingTemplate overloadingYou can provide overloadd template definitionsYou can provide overloadd template definitionsThe compiler selects the most specific The compiler selects the most specific templatetemplate

cout << max (4,5);cout << max (4,5);cout << max<double>(3.14,’6’);cout << max<double>(3.14,’6’);cout << max (“this”, “greater”);cout << max (“this”, “greater”);

R max(S,T)

char *max(char *,char *)

T max(T, T)


C++ Class Templates 1.C++ Class Templates 1.Similar waySimilar way template classes could be defined template classes could be defined

template <class T> class matrix template <class T> class matrix { { public: public: matrix( int i, int j ); matrix( int i, int j );

matrix& operator=( const matrix &other);matrix& operator=( const matrix &other); T at(int i, int j); T at(int i, int j);

//… //… private:private:

vector<T> v;vector<T> v;};};

Created always with explicit specialisationCreated always with explicit specialisation

matrix<int> m(4,5)matrix<int> m(4,5);;


C++ Class Templates 2.C++ Class Templates 2.User specializationUser specialization

template <> class matrixtemplate <> class matrix<bool><bool> { { public: public: matrix( int i, int j ); matrix( int i, int j ); matrix( const matrix &other)matrix( const matrix &other) bool at(int i, int j);bool at(int i, int j);private:private:

aaBBetteretterRRepresentationepresentation v; v;};};

Used the same wayUsed the same way

matrix<bool> m(4,5);matrix<bool> m(4,5);


C++ TemplatesC++ Templates

The C++ templates were first implementedThe C++ templates were first implemented i in the early ’90sn the early ’90sAccepted as part of the ANSI/ISO in 1994Accepted as part of the ANSI/ISO in 1994Erwin Unruh: 1994Erwin Unruh: 1994

unruh.cpp 30: conversion from enum to D<2> requested unruh.cpp 30: conversion from enum to D<2> requested unruh.cpp 30: conversion from enum to D<3> requested unruh.cpp 30: conversion from enum to D<3> requested unruh.cpp 30: conversion from enum to D<5> requestedunruh.cpp 30: conversion from enum to D<5> requestedunruh.cpp 30: conversion from enum to D<7> requested unruh.cpp 30: conversion from enum to D<7> requested unruh.cpp 30: conversion from enum to D<11> requestedunruh.cpp 30: conversion from enum to D<11> requestedunruh.cpp 30: conversion from enum to D<13> requested unruh.cpp 30: conversion from enum to D<13> requested unruh.cpp 30: conversion from enum to D<17> requested unruh.cpp 30: conversion from enum to D<17> requested unruh.cpp 30: conversion from enum to D<19> requestedunruh.cpp 30: conversion from enum to D<19> requested


Power of C++ templatesPower of C++ templates

The Compiler “executes” template metaprogramsThe Compiler “executes” template metaprograms

The result is a non-templated program The result is a non-templated program

executed inexecuted in “run-time”“run-time”

In 1966 Böhm and Jacopini proved: In 1966 Böhm and Jacopini proved:

Turing machine implementation <==> Turing machine implementation <==> conditional and looping conditional and looping constructionsconstructions

The C++ templates are Turing-completeThe C++ templates are Turing-complete

in compilation timein compilation time


The Factorial exampleThe Factorial example

The run-time solutionThe run-time solution

int factorial( int n)int factorial( int n)

{ {

if ( n == 1 ) return 1; if ( n == 1 ) return 1;

else return n*factorial(n-1)else return n*factorial(n-1);;

}}

int main() int main()

{ {

cout << factorial(cout << factorial(115) << endl; 5) << endl;

return 0; return 0;

} }


The Factorial exampleThe Factorial example

The metaprogram solutionThe metaprogram solution

template <int N> struct Factorial template <int N> struct Factorial { { enum { value = N * Factorial<N-1>::value }; enum { value = N * Factorial<N-1>::value }; }; }; template <> struct Factorial<1> template <> struct Factorial<1> { { enum { value = 1 }; enum { value = 1 }; }; }; int main() int main() { { const int fact15 = Factorial<15>::value; const int fact15 = Factorial<15>::value; std::cout << fact15 << endl; std::cout << fact15 << endl; return 0; return 0; } }


Conditional statementConditional statementtemplate <bool condition, class Then, class Else> template <bool condition, class Then, class Else> struct IF { struct IF { typedef Then RET; typedef Then RET; }; }; template <class Then, class Else> template <class Then, class Else> struct IF<false, Then, Else> struct IF<false, Then, Else> { { typedef Else RET; typedef Else RET; }; };

template <typename T, typename S>template <typename T, typename S>IF< sizeof(IF< sizeof(TT)<sizeof()<sizeof(SS), ), SS, , TT>::RET >::RET max(T t, S s)max(T t, S s){{ if (t > s) return tif (t > s) return t;; else return s;else return s;}}


(Run-time) (Run-time) Programs vs. MetaprogramsPrograms vs. Metaprograms

FunctionFunction

(runtime) Data(runtime) Data

VariableVariable

ConditionCondition

LoopLoop

AssignmentAssignment

ClassClass

Type and constantType and constant

Symbolic namesSymbolic names

Type selectionType selection

RecursionRecursion

No assignmentNo assignment


Data in Template MetaprogramsData in Template Metaprograms

Referential transparency: No assignment Referential transparency: No assignment

Still possible to store, modify, and retrieve dataStill possible to store, modify, and retrieve data

TypelistTypelist

structstruct NullType {} NullType {};;

typedef Typelist< char, Typelist<signed char,typedef Typelist< char, Typelist<signed char,

Typelist<unsigned char, NullType> > > Charlist;Typelist<unsigned char, NullType> > > Charlist;

char signed char NullTypeunsigned char


Data handlingData handling

template <class TList> struct Length;template <class TList> struct Length;

template <>template <>

struct Length<NullType>struct Length<NullType>

{{

enum { value = 0 };enum { value = 0 };

};};

template <class T, class U>template <class T, class U>

struct Length <Typelist<T,U> >struct Length <Typelist<T,U> >

{{

enum { value = 1 + Length<U>::value };enum { value = 1 + Length<U>::value };

};};


MotivationMotivationint main()int main(){{ const unsigned int di = 12; const unsigned int di = 12; const unsigned int oi = 014; const unsigned int oi = 014; const unsigned int hi = 0xc; const unsigned int hi = 0xc; const unsigned int bi0 = binary_value("1101"); const unsigned int bi0 = binary_value("1101"); const unsigned int bi1 = binary<1100>::value; const unsigned int bi1 = binary<1100>::value; }}

template<unsigned long N> struct binarytemplate<unsigned long N> struct binary{{ // prepend higher bits to lowest bit// prepend higher bits to lowest bit static const int value=binary<N/10>::value*2+N%10;static const int value=binary<N/10>::value*2+N%10;}; }; template<> struct binary<0> // specializationtemplate<> struct binary<0> // specialization{{ static unsigned const value = 0;static unsigned const value = 0;};};


MotivationMotivation

Constant expression – array size, case label, etc…Constant expression – array size, case label, etc…

Better code compiledBetter code compiled

Faster in run-timeFaster in run-time

Syntactically checked – the language semantic is Syntactically checked – the language semantic is extendedextended

t

Design time Compilation time Run-time

No change in type systemChange in run-time values

Template metaprograms apply, automatic config

of the program

Decisions on Stategies,

policies


Generative MetaprogramsGenerative Metaprograms

mmetaprogramming: etaprogramming: writing programs that represent and manipulate other writing programs that represent and manipulate other programs or themselves (ie:reflection). Metaprograms programs or themselves (ie:reflection). Metaprograms are programs about programs. are programs about programs. introspection: introspection:

the ability of a program to observe its own statethe ability of a program to observe its own stateintercession: intercession:

the ability to modify its own state the ability to modify its own state

Open compilers: transformations on AST Open compilers: transformations on AST HHygenic macros (Scheme)ygenic macros (Scheme)Two level languages: AspectJ, Template HaskellTwo level languages: AspectJ, Template Haskell


Areas of Template MetaprogrammingAreas of Template Metaprogramming

Expression templates Expression templates Blitz+, PETEBlitz+, PETE

Static interface checkingStatic interface checking Early catch of syntactical/semantical errorsEarly catch of syntactical/semantical errors

Extending the C++ type systemExtending the C++ type system

IntrospectionIntrospection

Code adaption/optimalizationCode adaption/optimalization

Language embeddingLanguage embedding


Expression templatesExpression templatesArray a, b, c, d, e;// Object-oriented way of a = b + c + d + e

double* _t1 = new double[N]; for ( int i=0; i<N; ++i) _t1[i] = b[i] + c[i];

double* _t2 = new double[N]; for ( int i=0; i<N; ++i) _t2[i] = _t1[i] + d[i];

double* _t3 = new double[N*M]; for ( int i=0; i<N; ++i) _t3[i] = _t2[i] + e[i];

for ( int i=0; i<N; ++i) a[i] = _t3[i];

delete [] _t3; delete [] _t2; delete [] _t1;

// Fortran like solution:for ( int i=0; i<N; ++i) a[i] = b[i] + c[i] + d[i] + e[i];



SQL SQL Gil, et.al.: AraRatGil, et.al.: AraRat

XML parsingXML parsing Jarvi, et.al.: type-safe XML libraryJarvi, et.al.: type-safe XML library

Regular expressionsRegular expressions Boost::XpressiveBoost::Xpressive

Compiler embeddingCompiler embedding boost::spirit boost::spirit



SQL exampleSQL example

string s = ”select form ”+tName + ” where 1=1” ;string s = ”select form ”+tName + ” where 1=1” ;if ( cond1 )if ( cond1 )s += ” and fName1 = ” + field1;s += ” and fName1 = ” + field1;

if ( cond2 )if ( cond2 )s += ” and fName2 < ” + field2;s += ” and fName2 < ” + field2;

if ( cond3 )if ( cond3 )s += ” and fName3 = ” + field3;s += ” and fName3 = ” + field3;

Run-time errorsRun-time errorsInjection attacksInjection attacksConversion problemsConversion problems



SQL exampleSQL example

void f()

{

const string s =

(

(tName / (fName1 == field1 &&

fName2 < field2)

)

[ fName1, fName3 ]

).asSQL();

}



Boost::Xpressive exampleBoost::Xpressive example

std::string hello( "hello world!" );std::string hello( "hello world!" );

sregex rex1 = sregex::compile( "(\\w+)(\\w+)!" );sregex rex1 = sregex::compile( "(\\w+)(\\w+)!" );sregex rex2 = (s1= +_w) >> ' ' >> (s2= +_w) >> '!';sregex rex2 = (s1= +_w) >> ' ' >> (s2= +_w) >> '!'; smatch what;smatch what;

if( regex_match( hello, what, rexif( regex_match( hello, what, rex11 ) ) ) ){{ std::cout << what[0] << '\n'; // whole matchstd::cout << what[0] << '\n'; // whole match std::cout << what[1] << '\n'; // first capturestd::cout << what[1] << '\n'; // first capture std::cout << what[2] << '\n'; // second capturestd::cout << what[2] << '\n'; // second capture}}


EmbeddingEmbedding alternatives alternatives

If you are (accidently) not a C++ programmer If you are (accidently) not a C++ programmer

.NET platform, VB, C#: .NET platform, VB, C#: LINQ projectLINQ project

Java platform:Java platform: Eric Van Wijk et.al.: Attribute Grammar-based Language Eric Van Wijk et.al.: Attribute Grammar-based Language

Extensions for JavaExtensions for Java

StrategoStratego TU DelftTU Delft

Charles Simonyi: Intentional programmingCharles Simonyi: Intentional programming


Intentional programmingIntentional programming

Charles SimonyiCharles Simonyi XEROX Palo Alto XEROX Palo Alto BravoBravo MicrosoftMicrosoft Word, ExcelWord, Excel Intentional softwareIntentional software http://www.intentsoft.comhttp://www.intentsoft.com


Open questionsOpen questions

The real expressive powerThe real expressive power

Standard tools Standard tools but: Loki from Andrei Alexandrescubut: Loki from Andrei Alexandrescu boost::mplboost::mpl

GaranteeGarantees (number of instantiations, etc…)s (number of instantiations, etc…)

How to designHow to design??

How to debugHow to debug??

MetaprogrammingMetaprogrammingfrom University to Industryfrom University to Industry

ZoltZoltán Porkolábán Porkolá[email protected]@elte.hu

http://gsd.web.elte.huhttp://gsd.web.elte.hu

Dept. of Programming Languages and Compilers,Dept. of Programming Languages and Compilers,Faculty of InformaticsFaculty of Informatics

Eötvös Loránd University, BudapestEötvös Loránd University, Budapest

Thank you!Questions?

Documents

Metaprogramming from University to Industry Zoltán Porkoláb [email protected] Dept. of Programming Languages and Compilers, Faculty of