Software Developer Journal

7/23/2019 Software Developer Journal

1/112


2/112

A BZ Media EventBig Data TechCon

is a trademark of BZ Media LLC.

www.BigDataTechCon.com

April 26-28, 2015Seaport World Trade Center Hotel

Big Data TechCon is the HOW-TO technical conference

for professionals implementing Big Data solutions

at their company

Choose from 55+ classes and tutorials!

Come to Big Data TechCon to learn the best ways to:

Process and analyze the real-time data pouring into your organization

Learn how to extract better data analytics and predictive analysis

to produce the kind of actionable information and reports your

organization needs.

Come up to speed on the latest Big Data technologies like Yarn, Hadoop,

Apache Spark and Cascading

Understand HOW to leverage Big Data to help your organization today

Big Data Gets Real in Boston!

People are talking aboutBigData TechCon!

Big Data TechCon is a great learningexperience and very intensive.

Huaxia Rui, Assistant Professor,

University of Rochester

Get some sleep beforehand,

and divide and conquer the packed

schedule with colleagues.Paul Reed, Technology Strategy & Innovation, FIS

Worthwhile, technical, and a breath of

fresh air.Julian Gottesman, CIO, DRA Imaging

Big Data TechCon is definitely worth the

investment.Sunil Epari, Solutions Architect, Epari Inc.


3/112

The Best Of

3

Dear Readers!Our Best of SDJ issue is finally released and it is free to download! We worked hard and we hope that yousee that. This issue is dedicated to Web Development mostly. We tried to compare as many frameworksas we can. Our jQuery section starts with Jquery is awesome. So, use it less! by Willian Carvahlo. Author isconvinced that jQuery is a great tool but its sometimes used in a wrong way. This article is a voice ina discussion about proper use of jQuery.

Then youll find jQuery promises by Ryan ONeill. This article is related to the previous one. Author showsin a simple way how you can manage all that you want with jQuery without complications.

Davide Marzioni shows a simple trick with web2py and Tomomichi Onishi presents Tutorial for creatingsimple Hyakunin-Issyu application using Sinatra and Heroku. Manfred Jehle in theoretical way explainshow you can Start developing better applications.

Aimar Rodriguez covers Django subject in the article entitled Developing your own GIS applicationwith GeoDjango and Leaflet.js.

Also look closely to the other articles. You need to read the article on AexolGL and I think you will findthe new 3D graphics engine full of new tools. This issue contains really interesting content and we arehappy to publish that for you!

Were hoping youll enjoy our work.

Ewa & the SDJ Team


4/112

Editor in Chief:Ewa Dudzic

Editorial Advisory Board: David Gillies, Shawn Davis

Special thanks to our Beta testers and Proofreaders who helped

us with this issue. Our magazine would not exist without your

assistance and expertise.

Publisher:Pawe Marciniak

Managing Director:Ewa Dudzic

DTP:Ireneusz Pogroszewski

Marketing Director:Ewa Dudzic

Publisher:Hakin9 Media SK

02-676 Warsaw, PolandPostepu 17D

http://www.sdjournal.org

Whilst every effort has been made to ensure the highest qual-

ity of the magazine, the editors make no warranty, expressed

or implied, concerning the results of the contents usage. All

trademarks presented in the magazine were used for informa-

tive purposes only.

All rights to trademarks presented in the magazine are reserved

by the companies which own them.

DISCLAIMER!

The techniques described in our magazine may be used inprivate, local networks only. The editors hold no respon-sibility for the misuse of the techniques presented or anydata loss.
http://en.sdjournal.org/http://en.sdjournal.org/


5/112

5

The Best Of

Copyright 2015 Hakin9 Media Sp. z o.o. SK

Table of Contents

AexolGL New 3D Graphics Engine ..............................................................................6

Jquery is Awesome. So, Use it Less! .................................................................................9by Willian Carvalho

Jquery Promises .................................................................................................................11by Ryan ONeill

Technical Tip & Tricks For Web2py ................................................................................14by Davide Marzioni

Tutorial for Creating Simple Hyakunin-Issyu ApplicationUsing Sinatra and Heroku .................................................................................................18by Tomomichi Onishi

Start Developing Better Web Applications ......................................................................28by Manfred JEHLE

Developing Your Own GIS Application with GeoDjango and Leaflet.js ......................33by Aimar Rodriguez

Solving Metrics Within Distributed Processes ................................................................44by Dotan Nahum

CreateJS In Brief ...............................................................................................................52by David Roberts

Build Customized Web Apps through Joomla ................................................................67by Randy Carey

What is Drupal? Concepts And Tips for Users and Developers ....................................74by Juan Barba

AngularJS Tool Stack ........................................................................................................80by Zachariah Moreno

Thinking the AngularJS Way ...........................................................................................84by Shyam Seshadri

Reusable UI Components in AngularJS ..........................................................................100by Abraham Polishchuk & Elliot Shiu

Test Automation with Selenium 2 .....................................................................................104by Veena Devi

Grabbing the Elements in Selenium .................................................................................109by Nishant Verma


6/112

The Best Of

6

AexolGL New 3D Graphics Engine

Aexol specialises in creating mobile applications. It was created by

Artur Czemiel, a graduate of the DRIMAGINE 3D Animation & VFXAcademy, who has a lifelong interest in 3D technology. He first startedto realise his passion by working in the film industry. Artur is the co-creator of the special effects in Polish production Weekend andthe short film Hexaemeron, which was awarded Finest Art award

Fokus Festiwal and nominated to best short animated film at fLEXiff2010 Australia. The experience gained by working in the movieindustry and on the mobile applications market was the basis forcreating AexolGL a tool designed to make work easier for Aexol andother programmers around the world.

What is AexolGL?A set of tools for creating visualisations, applications and 3Dgames with little workload. The user doesnt have to worryabout things like differences between OSs or hardware.AexolGL lets you focus on the key elements and appearanceof the end product (application, game) instead of worryingabout technical details.

What was the main objective and the mainincentive to create the engine?We wanted to create a tool for small/medium-sized devel-

oper studios, indie developers, that would let them design3D projects on any platform they want.

Why create two different engines?AexolGL PRO is a tool for creating games and applica-tions natively in C++/Python, for the following platforms:iOS, Android,Windows, Mac and Linux. AexolGL WEB isused to create games and applications for internet browsers(Mozilla, Safari, Chrome) without the need to use pluginsor simple webview apps, games for iOS and Android.

Is AexolGL a tool only for creating games and

mobile applications? Will it nd use in otherelds?AexolGL WEB is a perfect tool for creating visualiza-tions. 3D technology is the modern form of presentation,that works perfectly for visualizing interiors, buildingsand product models (e.g. cars and electronic devices). Aex-olGL takes website product presentation to a whole newlevel.

Will displaying a lot of 3D graphicsin web browser slow the users computer(AexolGL WEB)?

Most certainly not! The web engine handles displaying3D very well, even on machines using integrated graph-ics. Deferred shading technology handles creating compli-cated lighting models without overly taxing the hardware.

Why use Python (AexolGL PRO)?Python is a easily adaptable scripting language. Being inline with the idea behind the engine itself (quick program-ming), it allows rapid prototyping of applications. Pythonsmodule structure allows the addition of many prepared li-

braries, which help make the programmers work easier.

How are different scenes, models etc.imported into the engine?We have integrated the ASSIMP library with our engine,which allows the import of about 20 different formats.

However because it is constantly being expanded thatnumber will increase overtime.

What can you say about the engine structure?One of the main efciency problems that appear whencreating 3D projects are context changes. To minimalizethe number of costly changes, while not forcing the objectsorting order, we created a RenderTree, which makes surethat operations are not repeated and are executed in thecorrect order.

AexolGL team
http://bit.ly/DownloadAexolGL


7/112

The Best Of

7

Does the engine give user the ability toimplement individual solutions, for example,Yes, we let the user create personal solutions, write custom

shaders or effects needed for specialised tasks.

Are there any similar products already onthe market? What makes AexolGL stand out(specically in terms of functionality) in theeld of available solutions?AexolGL, is primarily a tool for small and medium-sizedprojects, that lets you rapidly prototype and previewthem. We do not aim to compete with the big engines.Ours is one of the select few that works on all platformsand has a web counterpart with a similar RenderTreestructure.

Are there any examples available? It seemsthat currently there arent any games or moreimportantly, a tech demo of the engine created

with AexolGL available on the website.We are currently putting nishing touches on our productand the website. Soon gl.aexol.com will host the rst ex-amples showcasing the possibilities of AexolGL WEB aswell as the rst game fro mobile devices created with ourtechnology, called Gravity: Planet Rescue.

Does the engine use optimization algorithms,like occlusion culling? Or others like, forexample, those found in Umbra technology.The engine does have the most popular optimization al-gorithms available. Although not as advanced as Umbras,

Ready for instantation animated sprite object from JSON file (C++)

A simple way of creating objects with assigned materials, shaders, geometry and transformation matrices.InAexolGL the object is ready for display after only 30 lines of code (C++)
http://bit.ly/DownloadAexolGLhttp://bit.ly/DownloadAexolGLhttp://bit.ly/DownloadAexolGLhttp://bit.ly/DownloadAexolGL


8/112

The Best Of

8

they certainly increase the efciency of the application.As we expand the engine we will certainly further im-

prove this system.

What kinds of lighting algorithms areavailable in the engine, does it supportlightmapping or global illumination? Do youplan on including realtime global illuminationshaders?We are constantly working on scene lighting. Ultimatelyit will be one of the advantages of LightRig technologywhich creates a compact lighting model out of the envi-ronment map, giving the illusion of GI. Currently the en-gine is equipped with several types of lighting and sup-

ports shadow-mapping.

How does the engine model terrain, do youplan on using voxels, can you createheightmap based terrain?

Heightmap based terrain creation is already available. Itsactually very convenient and practical tool useful in a ma-

jority of projects. A voxel version might be implementedas well in the future.

To my understanding, the engine providesa joint interface that lets create applications thatwork both under, for example, Windows andAndroid? How does it handle the fundamentaldifference in controls (desktop mouse andkeyboard, mobile devices touchpad)?We give the developer the ability to dene controls on key-

board, joystick, mouse and touchscreen. It is also possibleto dene a virtual joystick on the touchscreen. Howeverhow the application reacts to individual signals is entirelyup to its creator. By default, signals from the mouse andone nger touches are treated the same, however they caneasily be assigned to different actions.

How about the signicant differencein computing power between desktopsand smartphones?Obviously smartphones do have less computing powerthan desktops, however how the application functions onmobile platforms depends primarily on its design. And for

our users, the help of our efcient solutions.

In the currently available version of AexolGLWEB you used the K-3D library licensed byGNU GPL. Why wasnt this fact mentionedon the product page? Are the licensescompatible?The K-3D library is not used in the current version of theengine. The File loading mechanisms employed by K-3Dare obsolete and do not support usemtl.

Is AexolGL only a graphics engine or doesit also handle other aspects of game creation(physics, optimal resource management,AI etc.)?Aside from the graphics engine itself, our framework alsosupports optimal, multithread resource management. Weintroduced a simple system of creating multiple threadsin an application and solved the problem of le loadingon different platforms as well. For mobile platforms we

prepared a suitable small format for saving 3D geometry.Additionally our engine easily integrates with available

physics engines (for example, the popular Bullet Physics).The engine also has an integrated mathematical libraryequipped with the most needed functions for 3D applica-tions: 2D/3D vector math, transformation matrices andquaternions. As well as countless additional instrumentse.g.: color conversions, easing function library, Bezier andCatMull curves and the ability to create simple parameter-ized geometry (cubes, spheres, cylinders).

Similarities and differences between yourproduct and the biggest player, Unity 3D.What is the niche for AexolGL in a marketwith a free Unity 3D?Its difcult for us to compare with Unity. The idea behindour engine is completely different. Were not targeting the

biggest studios with complicated and high-budget projects.Our aim is to let small and medium-sized studios benetfrom a quick and simple tool that will let them begin their

journey into the world of 3d games and applications with-out straining their budget. Obviously we will also contin-

ue to work on our project, extending its capabilities andbroadening its use. Additionally if we take a closer look atthe free version of Unity 3D, we can see that the access tomany useful functions, such as Static Batching, Render-to-Texture Effects, Full-Screen Post-Processing Effects orGPU Skinning, is only available to the paid PRO version.

Does your product benet from the newpossibilities available in OpenGL 4?OpenGL 4 is currently only available on PC. Becausea lot of mobile devices still use OpenGL ES 2.0 our en-gine is compatible mainly with that API version. Althoughthanks to the high exibility of the engine, introducing

OpenGL 4 would not be a problem. Users of the AexolGLLab have the ability to independently adapt the engine toOpenGL 4 thanks to GL abstract.
http://bit.ly/DownloadAexolGL


9/112

The Best Of

9

Jquery is Awesome. So, Use it Less!by Willian Carvalho

One of the greatest features, if not the only one, responsible for making Javascript shine

brighter in the past years is definitely jQuery.Since its birth in 2006, it has become very popular, attracting both programmers and web designers, becauseit made their lives a lot easier.

At that time, server side developers were always paying attention to database and security handling,component layers, message queues, etc and they have never actually been able to focus on client side

programming.

Web designers, on the other hand, focused their efforts on building nice designs for applications, as well ascaring about user experience and on making the best out of HTML/CSS combo, also leaving Javascript behind.

Jquery came to fill this gap between the server and the client tiers.

Enough with cross browser concerns and AJAX handling issues. Enough with lines and lines of code to dosimple and repetitive tasks. It was the beginning of a new era for the web development.

The time has passed and tons of jQuery plugins have been built. Almost everybody uses jQuery now, and ithas become some sort of a common language between developers and web designers.

Jquery became so popular and easy to use that people started using it for nearly everything, from rich pluginsto simple selectors.

This brings us to the whole point on what this article is about: people have forgotten why jquery was built.

Frameworks and APIs are made to solve a lot of problems by encapsulating functionalities, like if they werean utility belt. However, these functionalities come with the cost of being too generic, causing them to beslower than if they were built to solve one single problem.

Jquery is no different from other frameworks. But this also doesnt mean that the additional cost is a badthing. Actually it means that jQuery is doing what it was supposed to do! Its just that we shouldnt be usingour utility belt when our own hands do the job as well (or better) than it.

But, what exactly are we doing wrong with jQuery? When should we be using VanillaJS (http://vanilla-js.com/)instead of it? How can we make sure that we made the right choice?

First of all, let me refresh your mind about what jQuery is for. Heres the description from jquery.com itself:

jQuery is a fast, small, and feature-rich JavaScript library. It makes things like HTML document traversaland manipulation, event handling, animation, and Ajax much simpler with an easy-to-use API that worksacross a multitude of browsers.

This means that jQuery makes easier for us to manipulate DOM elements, bind and unbind event listeners,write less code to some tasks, such as animations, and it handles AJAX requests a lot easier than we wouldhave done on our own. All of this without having to care about browser compatibility!

Besides these awesome features, jQuery also provides us a very useful set of functions for dealing withloops, reading a particular item on an array, css style manipulation and so on.

Although these functions are very tempting to use, some of them dont necessarily make our work easier.

One example is the use of the $.each function (http://api.jquery.com/jQuery.each/) instead of using theregularforstatement.
http://vanilla-js.com/http://api.jquery.com/jQuery.each/http://api.jquery.com/jQuery.each/http://vanilla-js.com/


10/112

The Best Of

10

It might not be of a big difference when iterating a small number of elements, but for larger lists theperformance would be compromised. Because deep inside, jQuery is using the plain old Javascript with ananonymous function for every step of the loop.

Besides, the amount of lines of code required to build a loop is not very different between jQuery and pureJavascript. Lets see:

Listing 1. jQuery sample

var arr = [a, b, c];

$(arr).each(function(index, data) {

console.log(index, data);

});

Listing 2. Javascript sample

JS version

var arr = [a, b, c];len = arr.length;

for(var i=0; i < len; i++) {

console.log(i, arr[i]);

}

As you can see, there is no big difference between them, but keep in mind that pure JS is faster, for sure.

There are other cases where we should be using pure Javascript instead of jQuery. Unfortunately, there is nomagic way to decide which one, other than analyze each situation.

It is very hard to guess which one is faster, and you will want to make sure which one to use.A simple way to compare your code with jQuery (or any other code) is by using JSPerf (http://jsperf.com/).

With this online tool, its possible to create any number of test cases and run them against each other to seehow many operations are done per second.

JSPerf is also able to store the results for every test in every browser and version. This is good, because sometests might be quite similar, but one particular browser might have a very different result than the others,which can help you on your decision.

In conclusion, when you are writing a web application, take a 5 seconds break, think about what you areabout to use jQuery for, and try to use it properly.

We all know how brilliant it is and how much it has helped us for the past years, but its important to alwayskeep in mind why and what it was really built for. Use less jQuery for the best!

About the AuthorWillian is a Senior Javascript Engineer at TOTVS. He worked specificly with Javascript for years and hewants to discuss about jQuery.
http://jsperf.com/http://jsperf.com/


11/112

The Best Of

11

Jquery Promisesby Ryan ONeill

jQuery has made working with asynchronous Javascript incredibly easy. However, callback

functions still get quickly out of control leading to code that is both hard to read and todebug. Usage of the promises pattern via the jQuery Deferred object is a great way to keepyour code clean and maintainable.

This article will give an overview of jQuerys implementation of the promise pattern, how it can be used towrite clean asynchronous jQuery, and an example implementation.

The author has been working with the jQuery library for over five years. He is currently a front-end engineerwith Twitter designing and building single page apps using jQuery and other libraries.

For context, this article assumes that the reader has some general Javascript and jQuery experience and isfamiliar with the asynchronous nature of the language.

Since jQuerys initial release in 2006, it has grown from a simple utility library into the defacto standard forwriting Javascript in the browser. jQuery solved many issues, such as cross-browser incompatibilities andshaky DOM querying, and also introduced features like the $.ajax()function which made it easier than everfor developers to build dynamic pages and applications without the need for full page reloads.

The Status Quo

The $.ajax()function did indeed change how Javascript applications were built. With the power of AJAX soreadily available, it quickly gained traction within the Javascript community. Shortly thereafter developersran into a problem which occurred when they needed to make more than one request to a remote server withthe second request relying on the response of the first. For example, code that looks like this:

Listing 1. A single AJAX request with callback

$.get(/user, function (user) {

$(#user-name).val(user.name);

});

Very soon turns into this as the applications grow in size:


12/112

The Best Of

12

Listing 2. Multiple AJAX requests with nested callbacks

$.ajax(

type: GET,

url: /user,

success: function (user) {


$.ajax(

type: POST

url: /user/login,

data: { userId : user.id },

success: function (loginResult) {

alert(loginResult);

},

error: function (err) {

// Error handling

});

},

error: function (err) {

// Error handling

});

Even in this basic example adding a single nested request and some trivial error handling makes the code muchmore difficult to read (partly due to switching to the longer form $.ajax()for error handling. Note that post()and get()are wrappers for ajax()). In practice, callback and error handling functions are typically much longerand the need for three or four nested requests commonly becomes necessary. This is especially true if theapplication uses more than one web service to function. At this point the code becomes effectively unreadable.

Cleaner Code with Promises

By taking advantage of the fact that the $.ajax()function returns a $.Deferred.promisewe can break this codeinto individual pieces and lay them out much more clearly and without nesting. Well get into the $.Deferredinterface later. First lets look at the above logic written using these promises.

Listing 3. Multiple AJAX requests using promises

var errorHandler = function (err) {

// Handle Error

}

var userRequest = $.get(/user);

var userLoginRequest = function (user) {

return $.post(

/user/login,

{ userId : user.id }

);

}

userRequest

.then(userLoginRequest, errorHandler)

// Any number of then()s can be used here to chain more asynchronous functions together

.done(function (loginResult) { alert(loginResult);

}, errorHandler);


13/112

The Best Of

13

The above code accomplishes the same set of tasks as the code in Figure 2. Through the use of jQuerypromises we are able to chain the asynchronous requests together rather than to rely on messy nestedcallbacks resulting in a script that is both easier to read and more maintainable. A few things to note:

done()and then()appear to be similar. A key difference is that then()will pipe the result of the callback(s)into the next piece of the chain

Note that in the then()block we need to return the result of the second AJAX request rather than to call itstandalone. This is so that the resulting $.Deferredfrom the userLoginRequest gets passed into the done()function, allowing us to make use of its result

We also hoisted a generic error handler for an even clearer solution.

What is $.Deferred?

Under the covers $.Deferredis a stateful callback register that follows the convention of Promises/A (http://wiki.commonjs.org/wiki/Promises/A). The promise has three possible states: pending, resolved, and rejected.

Every $.Deferredobject starts in the pending state and can move into either the resolved state via theresolve()function or the rejected state via the reject()function. In practice, these two methods are calledinternally by the library being used. For instance, the $.ajax()function handles resolving and rejecting the

promise. In fact, the resolve()and reject()functions will not even be available to us. This is because theobject returned from $.ajax()is actually a $.Deferred.promisewhich exposes only the functions to attachcallbacks and hides the functions that control the promise state. You can also take advantage of this if youare writing code that returns a promise that other code will subscribe to.

When a promise is rejected, all callbacks registered to the promise via the fail()function will execute.Similarly, when a promise is resolved the callbacks registered via the then()or the done()method will becalled. If we need a block of code to run when the promise completes regardless of whether it is failed orresolved, we can attach those callbacks using the always()function. This is analogous to a finally statement

in a try/catch block and is generally used for running clean-up code.Listing 4. Using always()

$.get(/user, function (user) {


}).always(function () {

alert(AJAX request complete); // This will always be called

});

Keeping a clean, maintainable, and readable code base requires active effort and diligence from alldevelopers involved. Promises are not a magic bullet and code can still get out of control when usingthis pattern. When used correctly, promises can offer a large improvement in flow control relative to the

traditional callback pattern.

About the AuthorRyan ONeill was born in Washington D.C. in 1986. Since then he has taken residence in Miami, Atlanta,Chicago and San Francisco. He has worked with web technologies for the better part of a decade and iscurrently a senior front-end engineer with Twitter (you can follow him @rynonl).
http://wiki.commonjs.org/wiki/Promises/Ahttp://wiki.commonjs.org/wiki/Promises/Ahttp://wiki.commonjs.org/wiki/Promises/Ahttp://wiki.commonjs.org/wiki/Promises/Ahttp://wiki.commonjs.org/wiki/Promises/Ahttp://wiki.commonjs.org/wiki/Promises/Ahttp://wiki.commonjs.org/wiki/Promises/A


14/112

The Best Of

14

Technical Tip & Tricks For Web2pyby Davide Marzioni

Web2py (http://www.web2py.com) is an amazing framework to develop a fully-featured web-

based application using Python language (http://www.python.org).Web2py inherit all Python feature in term of simplicity, power and flexibility and it appliesthem to a web environment.

You will find a lot of useful tools like a database abstraction layer, authentication form andform-making utilities.

What you will learnIn this article I want to share with you some tips and tricks that could be useful when programming with Web2Py framework.

What you should knowYou should have a basic knowledge about how Web2Py works.

Use Aptana Studio as IDE

The integrated admin tool in Web2py is good enough to develop simple applicationand to do quickly edit,but if you want a stronger tool I suggest you to use Aptana (http://www.aptana.com ). The main advantage touse Aptana as IDE is that you can use the integrated debugger.

Setup Aptana for Web2Py is easy:

Create new PyDev Project

Name it Web2py

Uncheck Use default under Project contents and select the path to your local Web2py folder

Be sure you are using Python 2.5-2.7

To run Web2py you need to create a custom launch configuration:

Select Run menu and then Run congurations...

Right click on Python Run and select New

Name it Web2Py

Select the Web2py project just created

Select web2py.py le as main module

In the arguments tabs use the options:

-i 0.0.0.0 to bind web2py to all network

-a 123456 as dummy administration password

-p if you want to change the default port (8000)

Press Apply button to save the conguration
http://www.web2py.com/http://www.python.org/http://www.python.org/http://www.aptana.com/http://www.aptana.com/http://www.aptana.com/http://www.python.org/http://www.web2py.com/


15/112

The Best Of

15

The only drawback in using Web2py in any IDEs (Aptana included) is that it doesnt understand the context(the gluon module) and therefore autocompletion doesnt work. To solve this issue you can use a trick to addin the models and controllers with the following code:

Listing 1. A trick

if False:

from gluon import *

request = current.request

response = current.response

session = current.session

cache = current.cache

T = current.T

This code isnt executed but it does force the IDE to parse it and understand where the objects in the globalnamespace come from.

Modify a form with form.element

Web2py has some function helpers to define forms (FORM(), SQLFORM(), SQLFORM.factory()). Howeveroftenhappens that youll need to modify a form element after the declaration (styles, default values, etc.). You cando it using .element()and .elements()methods. They are equivalent but elements return a list of item if thereare more corresponding to the criteria you inserted.

Items returned can be modified using the standard notation: undescore + attribute name.

The parameters of these functions are descriptor of what item you want to return. The only positionalparameter is the item type you want to modify (eg. input, textarea, select, etc...). Other parameter are explicitand depend on the attribute you want to filter.

For example we want to change the submit button style classes.Listing 2. Change the submit button style classes

submit_button = form.element(input, _type=submit)

submit_button[_class] = btn btn-large btn-success

To change the selected status of an option item.

Listing 3. Change the selected status of an option item

default_option = form.element(option, _id=option_1)

default_option[_selected] = selected

del default_option[_selected]

To change the text of a textarea.

Listing 4. Change the text of a textarea

textarea = form.element(textarea, _name=description)

textarea[0] = New text

To change the style of all inputs.


16/112

The Best Of

16

Listing 5. Change the style of all inputs

for input_field in form.element(input):

input_field[_style] = input_field[_style] + width: 200px

How to write a custom validator of two linked eld in a

table

Validators are special functions which helps to validate a form or database field. You can insert them usingthe requires keywords. If you want to have a database table where only one of two fields must have a value,there will be trouble to define this with standard validators. This can be resolved by a custom validator.

For example, if you have a website table where, for some reason, you want be filled an IPv4 address ora URL link, only one of them but not both together. In addition you want use the default validator for bothfields. To solve this problem you can define in the model a custom LinkedFieldValidator and use it in therequires values of the fields.

Listing 6. TwoLinkedValidator in a model file.

class TwoLinkedValidator:

def __init__(self, a, b, validator=None,both_filled_message=Enter only one field!, both_

empty_message=Insert at least one field!):

self.a = a

self.b = b

self.v = validator

self.error_filled = both_filled_message

self.error_empty = both_empty_message

def __call__(self, value):

if IS_NOT_EMPTY()(self.a)[1] == None: if IS_NOT_EMPTY()(self.b)[1] == None:

return (value, self.error_filled)

if self.v:

return self.v(value)

return (value, None)

else:

if IS_NOT_EMPTY()(self.b)[1] == None:

return (value, None)

return (value, self.error_empty)

def formatter(self, value):

return value

Init function requires the two field to validate. Parameter a is always the self-referenced field, while b isthe other. Optionally I can pass another validation function in the validator parameter.

The validation functions return a tuple where the first value is the formatted value (no formatting is done inthis case) and the second value is an error message (None if the value is correct).

In our case it looks like that.


17/112

The Best Of

17

Listing 7. A sample code

db.website.weblink.requires = LinkedFieldValidator(request.vars.weblink,

request.vars.ipaddress,

IS_URL(mode=generic, allowed_schemes=[ftp, http, https]))

db.website.ipaddress.requires = LinkedFieldValidator(request.vars.document_file,

request.vars.weblink,

IS_IPV4())

How to solve lazyT issues

Web2py has a great tool to internationalize the content: the T()function. Every string inserted as parameterwill be putted in the translation files.

T()works in default mode as lazy. It means that the real content is established when rendering the view.A lazy T returns an object instead a string, so if you need the translated value string in a function, youll get aan error argument must be string or read-only buffer, not lazyT

Listing 8. Example of error string

timestamp_string = datetime.strftime(datetime.now(), T(%Y-%m-%d %H:%M))

label = DIV(timestamp_string)

Then you have two options:

Use a function that immediately force to cast a lazyT object to a string like .xml().

Disable temporarily lazyness with T.lazy = False,

In the previous example you can.

Listing 9. First solution

timestamp_string = datetime.strftime(datetime.now(), T(%Y-%m-%d %H:%M).xml())


Or you can try that!

Listing 10. Second solution

T.lazy = False

timestamp_string = datetime.strftime(datetime.now(), T(%Y-%m-%d %H:%M))


T.lazy = True

About the AuthorIm Davide Marzioni and I have worked since 2011 as software developer for a small company in Italymainly focused on research and development in automation and electronic fields. I use Web2py in many

projects I can because it bring in easy way your application to a web environment.


18/112

The Best Of

18

Tutorial for Creating Simple Hyakunin-

Issyu Application Using Sinatra and

Herokuby Tomomichi Onishi

Figure 1. Image of Karuta Hyakunin-Issyu based card game (photo credit: aurelio.asiainviaphotopincc)

OverviewIn this tutorial, well see how to create a web application using Sinatra, the light-weight Ruby framework,and how to deploy it on Heroku, the web application hosting service.

We create a simple web application as a sample, using Hyakunin-Issyu, the beautiful anthology of Japaneseancient poems, as a theme.

This app has only two pages; the one shows the list of all the poems and the other one shows the detail ofeach poem.

(Dont worry if you never heard of Hyakunin-Issyu. Youll see a quick guide at the end of this introduction.)

What you will learnThrough this tutorial, youll learn the following.

how to use Sinatra framework

how to deploy your app on Heroku

general understanding of Hyakunin-Issyu

What you should knowThis tutorial expects you to know..

Ruby

Gem management using Bundler Git

Haml
http://www.flickr.com/photos/ionushi/2176677784/http://photopin.com/http://creativecommons.org/licenses/by-nc-nd/2.0/http://creativecommons.org/licenses/by-nc-nd/2.0/http://photopin.com/http://www.flickr.com/photos/ionushi/2176677784/


19/112

The Best Of

19

About Hyakunin-Issyu

Hyakunin-Issyu, or the one hundred poems by one hundred poets, is an anthology of one hundred tanka, aJapanese poem of thirty-one syllables, selected by a famous poet in the medieval period.

http://en.wikipedia.org/wiki/Ogura_Hyakunin_Isshu Wiki page of Hyakunin Isshu

Tanka is made of thirty-one syllables, five-seven-five for the first half of the poem and the seven-seven forthe last half.

As it cant contain very much information on such a limited number of words, its very important to feel theaftertaste of the poem.

Composing a poem with very selected words, describing the delicate feelings and the beautiful scenery ofnature, is a very Zen-like way and this is the culture we Japanese should be proud of.

We often play the Hyakunin-Issyu based card game called Karuta in the New Years holidays in Japan.

The basic idea of the Karuta game is to be able to quickly determine which card out of an array of cards isrequired and then to grab the card before it is grabbed by an opponent.

Chihayafuru, the karuta themed comic, became a big hit in Japan and now this traditional culture has becamepopular again.

Please take a look at this comic if you are interested.

http://www.youtube.com/watch?v=rxebYxY9NXE opening video for Chihayafuru anime

Okay, I think thats enough for the intro.

Now its time to start the tutorial.

Using Sinatra

The first half of this tutorial is to create an simple application with Sinatra.

The very basics of Sinatra

Minimum construction

To start with the smallest possible project, all you need is two files.

Listing 1. The construction of the project files

|-sample

|-main.rb

|-Gemfile

The core parts of the application will be written in main.rb.

At the moment, we only need to add routing for root (/). So any requests for / will be processed here.

In this example, well output a simple hello world.
http://en.wikipedia.org/wiki/Ogura_Hyakunin_Isshuhttp://www.youtube.com/watch?v=rxebYxY9NXEhttp://www.youtube.com/watch?v=rxebYxY9NXEhttp://en.wikipedia.org/wiki/Ogura_Hyakunin_Isshu


20/112

The Best Of

20

Listing 2. The minimum implementation of main.rb

#main.rb

require sinatra

get / do

hello world.

end

Next, make a Gemfilefor gem management. Now you only need a Sinatra gem.

Listing 3. List gems on Gemfile

#Gemfile

source :rubygems

ruby 2.0.0

gem sinatra

From the terminal, run bundle install to install gems to the project.

The project settings are almost done!

Move to the project root and run ruby main.rb from the Terminal.

The application will be run on port:4567 (this may be different on your machine, so be sure to check theoutput in Terminal).

Open your browser and access localhost:4567.

If successful, you should see the words hello world displayed there.

Adding more pages

Okay, now were going to add some more pages to this app (its just too simple, otherwise!).

Edit main.rbto do this:

Listing 4. Adding more page to main.rb

#main.rb

...

get /poem do

this is another page!

end

Well done! Now we have another page with the route /poem.

Restart the project by running ruby main.rband access localhost:4567/poem in your browser.

You should now see this is another page! displayed there.

Auto reloading Sinatra

It can get tiresome to restart the process every time youve changed something in the code.

To make things easier, lets introduce auto-reloading into our app.


21/112

The Best Of

21

Listing 5. Add sinatra-contrib to Gemfile

#Gemfile

...

gem sinatra-contrib

Add this line to Gemfile and run bundle install again.

Then require sinatra/reloader on main.rb.

Listing 6. Require sinatra/reloader in main.rb

#main.rb

require sinatra

require sinatra/reloader

...

Thats all we need. Try restarting main.rb again (its the last time, I promise!), then access localhost:4567in the browser.

Next, change the hello world message on main.rb and refresh the page. If all goes well, youll now see themessage changed without having to restart.

Accept parameters

One last thing for this section is to accept URLs with parameters, like /poem/13, so that the page contentsupdate based on this new value.

Listing 7. Accept parameters in main.rb

#main.rb

get /poem/:id do this page shows the detail of poem-#{params[:id]}

end

Add :idto the get part, and use that param with params[:id].

Now try accessing localhost:4567/poem/13. The content should have changed.

Developing the main parts

Okay, we now have much of the core of the project completed.

I have made aHyakuninIssyu gemwhich allows us to use poem data easily, so lets install it.

(dont worry, the file contains English data, also)

If you want to know how to use the gem, please check it out, here.

https://github.com/Tomomichi/HyakuninIssyu Tomomichi/HyakuninIssyu

Install HyakuninIssyu gem

Add the gem to Gemfile and run bundle install again.
https://github.com/Tomomichi/HyakuninIssyuhttps://github.com/Tomomichi/HyakuninIssyu


22/112

The Best Of

22

Listing 8. Add HyakuninIssyu gem to Gemfile

#Gemfile

gem ...

gem HyakuninIssyu

Youll also need to require it in main.rb.

Listing 9. Require HyakuninIssyu gem in main.rb

#main.rb

require ...

require HyakuninIssyu

With that done, check to make sure it works.

Listing 10. Add sample code to test gem

#main.rb

get / do data = HyakuninIssyu.new

data.poem(1).kanji

end

Add this to main.rb and then access localhost:4567 in your browser.

Have you found the poem of Emperor Tenchi (in Japanese this time)?

Figure 2. The card of Emperor Tenchi

This poem describes a miserable life of farmers, but isnt it strange that the emperor composed a poem likethis? How could he understand the feelings of those people?

Its one of the mysteries of Hyakunin-Issyu.


23/112

The Best Of

23

Index page

Okay, well now finish the index page using this gem.

This page shows the list of all the poems. Use the poems method of the gem:

Listing 11. List all the poems in index page

#main.rb

get / do

data = HyakuninIssyu.new

@poems = data.poems

end

Thats it. We set all the poems data to @poem.

Now its time to finish view files.

Use separate view les

Itll be messy if you write all the html document in main.rb, so we will divide the code and use separateview files.

Listing 12. The construction of the project after adding view files

|-sample

|-...

|-views

|-index.haml

|-poem.haml

Add a views directory and create haml files there.Install haml gem to use haml files.

Listing 13. Adding haml gem to Gemfile

#Gemfile

...

gem haml

And now create the index.hamlfile to show the list of poems.

Listing 14. Index.haml

#views/index.haml

%h1 INDEX

@poems.each do |poem|

unless poem.nil?

%p #{poem.kanji}

%small #{poem.en}

One last thing to do is to declare the use of haml file in main.rb.


24/112

The Best Of

24

Listing 15. Declare the use of haml file

#main.rb

get / do

...

haml :index

end

This simply means that it uses views/index.hamlas a view file.

Now lets access localhost:4567 again to see whether the content of index.hamlis shown there.

Remember that we used @poems in main.rb.

This enables us to pass that variable to the view file.

Now the index page is done. Lets move on to the second page.

Poem detail page

As we enabled the parameter handling already, we use it to get poem data from the gem.

Listing 16. Developing poem detail page

#main.rb

...

get /poem/:id do

id = params[:id].to_i #treat the parameter as an integer

data = HyakuninIssyu.new

@poem = data.poem(id)

@poet = data.poet(id)

haml :poem

end

We set the poem data to @poem and @poet, and declared that we use views/poem.hamlas a view file.

Thepoem.hamlfile should look like this:

Listing 17. The content of poem.haml

#views/poem.haml

%h1 POEM

%div

%h2 Poem Info

%p #{@poem.kanji}

%small #{@poem.en}

%div

%h2 Poet Info

%p #{@poet.name.ja}

%small #{@poet.name.en}

Access localhost:4567/poem/13 in the browser, perhaps with a different poem number, and check thepoem data is shown correctly.

Finish the development

To finish the development of this app, well link these two pages.


25/112

The Best Of

25

Listing 18. Add a link to index.haml

#views/index.haml

%h1 INDEX

@poems.each do |poem|

%p

%a(href=/poem/#{poem.id}) #{poem.kanji}

%small #{poem.en}

And add a very simple back link topoem.haml.

Listing 19. Add a link to poem.haml

#views/poem.haml

...

%a(href=/) Back

Okay, weve now finished developing this very simple Sinatra web application.

It shows the list of all the poems of HyakuninIssyu, and you can see the detail of each poem.

Now lets try to deploy this to Heroku.

Heroku Deployment

The last half of this tutorial is deploying the Sinatra application to Heroku.

Before continuing, please sign up and create your account on Heroku.

https://id.heroku.com/signup Heroku Sign Up

Also youll need the Heroku Toolbelt to use the heroku command.Please download this from the link below:

https://toolbelt.heroku.com/ Heroku Toolbelt

Okay, now lets get started.

Create a Heroku app

First you need to create a Heroku app.

Move to a new project root and run the following comand:

Listing 20. Create a new heroku app

heroku create YOUR-APP-NAME

Thats all. The empty app is created on heroku and its added to your git remote repository.

(You can check this by running the git remotecommand)

Create a system startup file

Before deploying your app, you need the system startup file to run your app on Heroku.
https://id.heroku.com/signuphttps://toolbelt.heroku.com/https://toolbelt.heroku.com/https://id.heroku.com/signup


26/112

The Best Of

26

Create config.rufile as shown below:

Listing 21. Create a config.ru file

#config.ru

require bundler

Bundler.require

require ./main #requiring main.rb

run Sinatra::Application

Introduce a git version management

As we use the git command to deploy the app to Heroku, we need to introduce git and commit the changesso far.

Listing 22. Introduce git version management

git init

git commit -m initial commit

If youre not familiar with git, check the Git Book or other tutorials.

http://git-scm.com/book Git Book

Now were ready for deployment!

Deploy to Heroku

Deploying to Heroku is extremely easy. Just run the following command:

Listing 23. Deploy command to Heroku

git push heroku master

Thats it. After successfully building your app on Heroku, run heroku open or

access APP-NAME.heroku-app.com to see your app.

Is your app working well? If you find some errors, please run heroku logs to see whats wrong.

Okay, thats the end of the tutorial.

The final version of the codes are in my GitHub repository.

If your code doesnt work, please check there and compare it with yours.

And more..

This tutorial covers only the very basics of Sinatra and Heroku to keep it simple.

If you find them interesting, please go further to get to know them better.

The following topics would be your next challenges:
http://git-scm.com/bookhttp://git-scm.com/book


27/112

The Best Of

27

Sinatra

use layout/shared les in view

use Helper

introduce SCSS, CoffeeScript

internationalization of the app

test with Rspec

introduce login management with Sorcery

Heroku

prevent Heroku app from sleeping with Heroku Scheduler

monitor the app performance with NewRelic

use thin server instead of webrick

build the staging app

connect to the database and backup it

use Travis CI for the automatic test and continuous deployment

Hyakunin-Issyu

learn the poems of Hyakunin-Issyu and remember them

read Chihayafuru to know the poems more.

join the Karuta game.

If you have an interest on these topics, Ill write the next article about them.

Please send me a request to let me know what you would like next: [email protected].

About the AuthorThe author of this article is a Japanese web developer interested in Hyakunin-Issyu.

My GitHub account is here: https://github.com/Tomomichi the authors github account.

My past products are:

booklovesmusic: Music recommendation service which matches your favourite books

Hyaku-Ichi: Will help you to remember Hyakunin-Issyu about a month from nowt
mailto:mailto:tomomichi.onishi%40gmail.com?subject=https://github.com/Tomomichihttps://github.com/Tomomichimailto:mailto:tomomichi.onishi%40gmail.com?subject=


28/112

The Best Of

28

Start Developing Better Web Applicationsby Manfred JEHLE

Web applications are a good thing no client installation needed and they usually work with

different browsers and browser versions properly! However, their functionality and look andfeel are currently much different than a desktop application.

That has not to be so! We look step by step at various issues and suggest resolutions to brushthem up. The resolutions also provide additional benefits that make web applications moreuseful than a desktop application.

Avoid that ickering screen

Most web applications are designed as common web pages, frequently by the same developer creating themarketing web pages. On marketing designed web solutions pixel accurate representation of the content isin great demand, whereas page reloads or rebuilds of the whole page are not an issue. On the other hand, in a

web application the permanent reload of the whole page makes the work slow and unattractive. Additionally,the reload means also that a lot of data has to be transferred from page to page.

Solution

To solve the flickering screen use AJAX functionality and you are able to replace any identifiable part on theweb page without reloading the whole page. In other words, you need a real single page application. If youchoose a good AJAX libraries support features such as: changing input element types, for example, text inputto drop-down box depending on the entered value by identifying on server side and providing additionalcontent back to the page.

With AJAX, you can develop user-friendly applications like desktop applications.

Developers corner

Web application architecture contains not only the server side it covers also the client side. To get anefficient client application it is not necessary to hold all the JavaScript code initially time loaded in the single

page. Such designs frequently result in slow, inefficient web applications with too much overhead that sufferfrom lost flexibility and maintainability. Keep an eye on the client side HTML code structure and reloadingand disposing partial JavaScript code.

Web application environment

Some web applications are developed for Silverlight or flash, but these technologies are not usable in themost browsers on mobile devices. Not all browser and operating systems supports flash and SilverLighttechnology. The current heterogeneous environment of devices and device vendors limits the IT departmentin flexibility. Remote desktop orientated solutions like Silverlight or remote desktop solutions are notthe real solution because the usability is not the best for mobile devices. Try to use with the fingers astandard designed desktop application on a mobile device your fingers are mostly too thick to get the bestinteraction on screen.

Solution

Use pure HTML on the client side! The reward for your efforts: approximately 80% compatibility withcommon browsers.


29/112

The Best Of

29

Developers corner

Avoid using hacks to get a nonstandard or incorrectly implemented browser element running! When thebrowser is fixed, your hack will produce mostly side effects so that you have to remove the code previouslyfixing a bug. Use code that will run in all browsers at development time and you will be on the better side!

No Menu

A lot of web applications are not designed with the elements commonly used in desktop applications.The look and feel of desktop applications is given by standard user interfaces like a menu bar with all thecommands needed to handle the application. Web applications are frequently designed against marketing

pages standards, which, as described above, are not the right approach for web applications.

Solution

Consider application processes, keeping the focus on making your web application function like common

desktop applications. Use a common menu element to make all options available in the menu bar and controlthe icons and descriptions only if the function is available in the current context. Use common icons anddomain specific or a general (common) naming for the menu items.

Developers corner

Use a state machine to handle all the combinations of menu items state and the availability of menufunctions. Show and open events are not clear enough because menu parts can be set as inactive or hiddentoo on different content stated like in a common desktop menu.

RibbonThe ribbon user interface element is not often used in web applications. But this element provides of fastaccess to many of the web applications functions and provides handling closer to the common Microsoftdesktop applications.

Solution

If it makes sense add a ribbon to your web application to make more useful functions accessible for users.Dont hide functions in the depth of menu structures!

Developers cornerUse a state machine to handle all the combinations of ribbon items state and the availability of ribbonfunctions. The same applies for any ribbon item as for any menu item applies too to have additional stateslike hidden or inactive.

Undo and Redo

Most web applications have no built in Undo or Redo function, but in desktop applications this functionalityis one of the most used. Providing this functionality in web applications greatly improves user experience.


30/112

The Best Of

30

Solution

Keep the last actions in the background to implement the Undo and Redo functionality. Practically speaking,it is not so simple but try to add it in your next update or new web application.

Developers cornerYou must check on each Redo operation to ensure it makes sense at the current position. The Undo is not a

big problem because the content refers directly to the content part on that a Redo makes sense.

Wizard functionality

In some desktop applications a wizard helps users step by step through entering and editing data. The wizardmakes it easier for users to get enter structured data into the application. Such functionality is also used inonline survey tools. But in many web applications, a wizard would make it easier for the user to enter thedata. Another option is to allow the user to switch between dialog and wizard-based data editing.

Solution

Provide a wizard for the dialog-based data editing and allow switching between the two views.

Developers corner

Use in the edit dialog web parts and make them visible or hide them to get the switch between commondialog content and wizard content.

Push functionIn some desktop applications you can be pushed by other users activities when using the same data or file.The common workaround is notifying a user that you are editing the data which the other user is alreadyviewing Another method is presenting a read-only view until the editing has been completed.This functionality needs information about what you are currently viewing and what other users are doing.Web applications can also provide this functionality but I have not seen many web applications, other thanmy own, implement this functionality.

Solution

It is possible to implement a notification to other users handling the same data, but keep the same in mind,just as with desktop applications use this functionality only when circumstances call for it.

Developers corner

Use a simple JavaScript timer to ask the server who is using the same data as I use currently and holdthe notifications ready for the other user. With a second timer ask for notifications on the server. Withoutany web socket you can provide content like from server pushed. At the moment for web sockets is noimplementation available working on any browser and operating system.


31/112

The Best Of

31

Local devices

In desktop applications it is mostly not a problem to add local devices or other devices in the network to thefunctionality of the solution. In web applications the usage of local devices is mostly a reason why a webapplication is not practical to develop except printing. But that assumption is not true! With a little moreeffort it is possible to use most local devices in a web application.

Solution

Expose devices to web applications accessible by wrapping them in a local service with a web serviceinterface. With this trick it is possible to access local devices through the server, and by proxy theexternal application.

Developers corner

If you build such services it is the easier way to use REST services.

Dump and last actions

In some really professional developed desktop applications when an error occurs you may be able to send anapplication dump, with the last few actions performed, to the software support centre or helpdesk.That allows the support team to rebuild the current state and execute the last action before the error occurred.Similar functionality is currently not common on the client side; on server side it is a common and helpfulsupport feature. It is possible to build in such a dump and last action functionality on the client side.The support team can easily recover the HTML page while getting the real HTML page part.

SolutionAdding such a function to the web application does not take much effort. You have to cover the URL basedactions (GET, POST actions) and store them into a first in last out (FILO) queue. The dump is quite easy toimplement: select the html part, copy the outer HTML and send it through the server to the support team.

Developers corner

Use the standard functions from jQuery to capture the HTML dump. The FILO needs a little bit moreJavaScript effort, but not too much.

Device detectionDevice detection in desktop applications has no significance because it runs mostly in similar environmentsor is designed to run on different operating systems with standard desktop screens. Exemplary webapplications support a wide range of devices such as tablets and phones along with desktop machines.By detecting the device, you are able to deliver device depending content to the target. Common webapplication solutions support the usual boring actions such as zooming and moving screen content, butfrequently suffer from issues such as clicking the wrong link because your fingers are too thick. Mobile users

benefit from device specific content that is intended for fingers.

Most web applications detect devices at CSS level, but deliver every time the full content and hide onlysome parts or enable alternate designs for smart phones and tablets. Such a solution is ineffective because

it wastes bandwidth by forcing the device to download extra, unnecessary content. Only a delivery of theeffectively needed content will be an efficient solution.


32/112

The Best Of

32

Solution

Detect the device in your web application and deliver only device-specific code.

Developers corner

Dont be satisfied with common CSS solutions go a step further by detecting the device and deliveringcontent specified for that device. It is required to get at any time the screen of the smart phone in 100% scalevisible! Overall, a smart phone ready design will involve some effort.

Common URL

In the content of device detection it is mandatory that you have the same URL and sub URLs for all devices.The common URL is needed to store page links in the cloud or in a common link collection. Only with acommon URL base the web application will be handy for users working with different device types.

Developers corner

Dont think about URL switches they dont solve the problem of URLs stored in link collections!

Final statement

Web applications are not dead! In todays multi-device environment, web applications soar to new primeof life. Most web applications are developed for low budgets but they are used as desktop applicationsdeveloped for high levels. This gap and the described missing functions can lift up the web applications toa higher level. Its clear that mostly it needs some effort to achieve the higher state, but finally you get an

actual solution ready to use in several devices.

Another fact

Web applications developed more than thirteen years ago runs without any update of the user interfacewithout problems in the current browsers. How many desktop applications can get such a lifetime with allthe operation system changes in the past?

The discussions about apps reminds me of the late 90s, when the battleground was between operatingsystems. At the moment we have the same kind of solution as we got at the past by JAVA a crutch notreally working perfectly for the app development.

For the foreseeable future, the web application provides a common base for all operating systems anddevices the browser.

About the AuthorManfred is CEO and Chief Architect of several products and customer projects and has more than 17years of experience in web applications and more than 28 years in general information technology.

Contact:[email protected]
mailto:[email protected]:[email protected]


33/112

The Best Of

33

Developing Your Own GIS Application

with GeoDjango and Leaet.jsby Aimar Rodriguez

Geographic information systems seem to have been proliferating lately, both on smartphones and on the web. GeoDjango is a module included in the Django contrib packagewhich allows to easily create geographic web applications. In this article we will build a

simple GIS application using GeoDjango and Leaflet.js.

What you will learnYou will learn to develop a simple geographic application using Django. You will learn to set up a geospatial database using Post-

greSQL and PostGIS, to represent and manipulate the data stored in this database with Django models and GeoDjango extensions

for this models and to present it to the user using the HTML5 map framework Leaet.js.

What you should knowIn order to fully understand this article some knowledge about the basics of the Django web framework are recommended, aswell as knowledge of the Python programming language, even though they are not required. It is also advisable to have some

knowledge about the JavaScript programming language.

A Geographic Information System or GIS is a computer system that enables users to work with spatialdata. Even if this concept was invented around the 60s, it has only taken relevance in the past years, with

powerful applications like Google Maps or OpenStreetMap. The proliferation of this kind of applications hasbeen huge to the point that now event the smallest local transport company uses this technologies. We haveall kinds of projects, from social networks based on routes like Wikiloc to project which attempt to bring aspatial dimension to The Semantic Web, likeLinkedGeoData orGeoSPARQL.

One of the biggest benefits that the developer community have gotten from this phenomena is the

appearance diverse tools and framework for spatial data manipulation, and this is where GeoDjango comesinto play. Django an open source web development framework written in Python, it has a huge communityand a wide amount of tools for the developers. Many of this tools come included in the contribpackage ofthe framework, where we can find the geographic web framework GeoDjango.

What this package offers to the web developers is the following:

The Model API, to store, query and manipulate the geographic data stored in the database using thetypical Django models,

The Database API, to manipulate different spatial database back ends,

The Forms API, which provides some specialized forms and widgets to display and edit the data on a map,

The GeoQuerySet API, which allows using the QuerySet API for spatial lookups,

The GEOS API, a wrapper for the GEOS open source geometry engine, for C++,

The GDAL (Geospatial Data Abstraction Library) API,

The measurement objects, which allow convenient representation of distance and area measure units.

Apart from the aforementioned, several utility functions, commands are included in the package, as well as aspecialized administration site.

We will be developing a very simple GIS application, which allows user to upload routes and to visualize themin maps. We have already seen that we can store and manipulate all this data with GeoDjango, however, westill need some way to present this data adequately to the users of the web page. Fortunately, there are severalchoices for this purpose, however, we will usually find two alternatives, OpenLayersand Leaflet.


34/112

The Best Of

34

Both are JavaScript libraries which allow to create a dynamic map on a web page. Which library to choose isup to each developer, I personally prefer Leaflet.js for its ease of use and learning. However, OpenLayer is amore mature project and promises several improvements in its third version which is yet to come.

With these two tools we can easily create a GIS web application of any kind. However, when developingone of these we will have several concerns, not related with the available technologies, for example, wherecan we get our data from? One approach which is followed by many is to let our users generate the data,however this is not always suitable for our application. It is also quite common to use external informationsources, like available web services. Even if we are not going to explore the possibilities that these webservices offer, I will give the following list of web services with some of the functions they offer.

Nominatim, a tools to search OSM (OpenStreetMaps) data. It allows address lookup and reversegeocoding, among other functions. A guide to this search engine is published on http://wiki.openstreetmap.org/wiki/Nominatim,

The OSM API. OpenStreetMaps offers an XML API whichallows to upload to and download data fromtheirdatabase. You can nd more about it the following address: http://wiki.openstreetmap.org/wiki/API_v0.6,

LinkedGeoData. For those desiring to implement a semantic spatial web application, know that

LinkedGeoDataoffers and API and has developed an ontology. It even has one SPARQL endpoint. Moreinformation on http://linkedgeodata.org/OnlineAccess,

Google Maps API web services. Google maps has its own API (even has a library for map visualization).However it imposes several limitations, so it is not used for more advanced GIS applications. Moreinformationon the google developers webpage: https://developers.google.com/maps/documentation/webservices.

The Web Application

To introduce this two libraries we will develop a very simple web application. It will allow users to uploadroutes in GPX format, which will be processed and stored in a spatial database. Then, the users will be able

to browse all the routes and visualize one of them. We will also perform a small analysis of these routes,using the GEOS API.

The code will be presented along the article, however, the whole project can be found in the followingrepository: https://github.com/aimarrod/SimpleGISApp.

The web application will consist on a simple HTML page, which will contain a form allowing the users tosubmit their GPX files, a map showing a route and a list of all the uploaded routes. On the back end we willhave a small Django project with a PostgreSQL database extended by PostGIS. The choice of database has

been done taking into account that GeoDjango offers some limitations depending on the database, and theleast constraining one is PostGIS. Anyway, there are several choices, for example, MySQL, so feel free touse any of them.

The rst steps

First of all, we have to install a PostgreSQL database and the Django framework (and of course a pythoninterpreter if you dont have it yet). This is usually trivial and if you are working on a Linux distribution youmay find instructions on how to do it in your distros wiki.

Note

From now on I will be assuming that PostgreSQL, Django and Python2.7 are installed. Since I am workingwith an Arch Linux distribution, so some installation steps may vary. Also, I will not be explaining all the

basics of the Django framework, some aspects like the settings file and the urls.py file will be omitted, if

you dont know the framework I encourage you to look up the Django documentation page, which explainseverything very nicely. You can find it in the following address: https://docs.djangoproject.com/en/1.5/.
http://wiki.openstreetmap.org/wiki/Nominatimhttp://wiki.openstreetmap.org/wiki/Nominatimhttp://wiki.openstreetmap.org/wiki/Nominatimhttp://wiki.openstreetmap.org/wiki/Nominatimhttp://wiki.openstreetmap.org/wiki/API_v0.6http://wiki.openstreetmap.org/wiki/API_v0.6http://linkedgeodata.org/OnlineAccesshttp://linkedgeodata.org/OnlineAccesshttps://developers.google.com/maps/documentation/webserviceshttps://developers.google.com/maps/documentation/webserviceshttps://github.com/aimarrod/SimpleGISApphttps://github.com/aimarrod/SimpleGISApphttps://docs.djangoproject.com/en/1.5/https://docs.djangoproject.com/en/1.5/https://docs.djangoproject.com/en/1.5/https://github.com/aimarrod/SimpleGISApphttps://developers.google.com/maps/documentation/webserviceshttp://linkedgeodata.org/OnlineAccesshttp://wiki.openstreetmap.org/wiki/API_v0.6http://wiki.openstreetmap.org/wiki/Nominatimhttp://wiki.openstreetmap.org/wiki/Nominatim


35/112

The Best Of

35

Installing PostGIS will be different depending on the OS you are using. In my case I can obtain it from theofficial repositories of my Linux distribution, however, PostGIS offers some binary installers for Windows,OSX and Linux, plus instructions for downloading and compiling the source code in the following page:http://postgis.net/install.

First, we will create a user for our spatial database, and then we will create a database in which we will loadthe PostGIS spatial types later. We will also need to install the pl/pgSQL language on the database since thisextension need it. Then, we will load the postgres spatial types from the directory in which they reside (inmy case /usr/share/postgresql/contrib/postgis-2.1/ ). Next step is common to make this database a template,so that we can create spatial databases without repeating all these steps.

Listing 1. Sample

$ su simplegisuser

Password:

$ createdb -O simplegisuser template_postgis -E UTF-8

$ createlang plpgsql template_postgis

$ psql -d template_postgis -f /usr/share/postgresql/contrib/postgis-2.1/postgis.sql

$ psql -d template_postgis -f /usr/share/postgresql/contrib/postgis-2.1/spatial_ref_sys.sql $ psql

> UPDATE pg_database SET datistemplate = TRUE WHERE datname = template_postgis;

$ createdb -T template_postgis simplegisdb

Platform specific instructions can be found in the PostGIS homepage and on the GeoDjango documentationpage. https://docs.djangoproject.com/en/dev/ref/contrib/gis/install/#installation.

After all the installation are done we can finally get into creating our project. First we will create a djangoproject. The first thing to do is to access the settings.pyfile in order to add django.contrib.gisto the installedapps. We will also need to edit the database connection setting, in order to match the database we created in

the previous section. The modified parts of the settings.pyfile should look similar to this: DATABASES= {

default: {

ENGINE: django.contrib.gis.db.backends.postgis,

NAME: simplegisdb,

USER: simplegisuser,

}

}

INSTALLED_APPS= (

django.contrib.auth,

django.contrib.contenttypes,

django.contrib.sessions,

django.contrib.sites,

django.contrib.messages,

django.contrib.staticfiles,

django.contrib.gis,

)

Once all the setup steps are done, we can finally start coding our application.
http://postgis.net/installhttp://postgis.net/installhttps://docs.djangoproject.com/en/dev/ref/contrib/gis/install/#installationhttps://docs.djangoproject.com/en/dev/ref/contrib/gis/install/#installationhttps://docs.djangoproject.com/en/dev/ref/contrib/gis/install/#installationhttp://postgis.net/install


36/112

The Best Of

36

The Back End

On the back end of our applications, we will define the models for our routes, we will define a simple formto allow the uploading of files, we will create one view which will show the map to the user and we willimplement a method to parse the uploaded GPX files.

The models

One essential part of most Django applications are the models, and this case is no different. Since we want tostore routes in our web page, we will first create a route model in a models.pyfile, following the convention.If you are familiar with this framework, you should know that the first thing to do is to import the model,however, since we are storing spatial data we wont use the conventional model, but the models defined inGeoDjango. For this, we will import the models from django.contrib.gis.db .

Apart from this little change, we can define our models as usual with the advantage that we now have someadditional fields related to spatial data. Taking advantage of this feature, we will declare the model for ourroutes, which will contain the following fields:

A name for the eld (Django CharField),

The date in which was uploaded (Django DateField),

The geometric representation of the route (GeoDjango MultiLineString).

Here we start seeing the tools that this package offers us. In our route model, we have declared aMultiLineString field, which corresponds to one of the geometry objects specified in the OpenGIS SimpleFeature specification. Simply put, a MultiLineString is formed by a list of LineStrings, which represent a setof points or coordinates. you can find more about the models API in the Django documentation page: https://docs.djangoproject.com/en/dev/ref/contrib/gis/model-api/.

The models.pyfile should look similar to this:

fromdjango.contrib.gis.dbimportmodels

classRoute(models.Model):

name= models.CharField(max_length=255)

creation_date= models.DateField()

representation= models.MultiLineStringField(dim=3)

objects= models.GeoManager()

The reason for dim(dimension) to be 3 is to allow the field to save the altitude. This attribute specifies

the dimension that the geometric field has, which defaults to 2. All geometric fields are composedby points, which must have at least two dimension (latitude and longitude), but can by extended by athird dimension (altitude). The choice on the dimensions of the geometries depends on the applicationto build and on the sources of information, and since the GPX files allow to record altitude, we willinclude the three dimensions.

Of course, it is possible to work a way around to represent this geometrical object without the use of thispackage. We could have defined our own Point model in which we store coordinates as floats and thendefine LineString model and so on, however, this would require us to do extra work and more importantly,we wouldnt have access to all the utilities that the GEOS API offers.

Once the model is defined we can finally synchronize the models with the database, using the following

command: python2 manage.py syncdb.
https://docs.djangoproject.com/en/dev/ref/contrib/gis/model-api/https://docs.djangoproject.com/en/dev/ref/contrib/gis/model-api/https://docs.djangoproject.com/en/dev/ref/contrib/gis/model-api/https://docs.djangoproject.com/en/dev/ref/contrib/gis/model-api/https://docs.djangoproject.com/en/dev/ref/contrib/gis/model-api/https://docs.djangoproject.com/en/dev/ref/contrib/gis/model-api/


37/112

The Best Of

37

The views

The Django views are functions that take a web request and return a web response. For this simple example,we will define a single view which will always return a HTML response. The document we will return willcontain the list of all the uploaded routes and a form which will allow our users to upload files.

As is usual, we will create a forms.pyfile in which the form will be defined. This form will contain twofields, the first one for the name of the route and the second one for the file. We will also perform twovalidations, to see if the name already exists and to check if the uploaded file is a GPX (though at this pointwe can only check it by looking at the extension of the file).

Listing 2. Forms file

fromdjangoimportforms

fromsimplegisapp.modelsimportRoute

classRouteUploadForm(forms.Form):

name= forms.CharField(max_length=255)

file= forms.FileField()

defclean_name(self):

name= self.cleaned_data[name]

ifRoute.objects.filter(name=name).count():

raiseforms.ValidationError(That name is not available)

returnname

defclean_file(self):

f= self.cleaned_data[file]

extension= f.name.split(.)[-1]

ifextensionnotin[gpx]:

raise

forms.ValidationError(Format not supported) returnf

Next we will create a view which will handle the uploading of files and will return the HTML file containingthe map and the form. However, before that, we should take care of the parsing of the documents that will beuploaded to our page. The GPX files we will be parsing follow a structure which is similar to the following:

XXX

For this we will create a file called utils.pyand define a method for parsing the file. This function will createa new LineString for every trksegfound, which will contain all the Points identified in the trkpttags. Whenthe trktag ends, all these LineString will be used to create the MultiLineString which will be stored in thedatabase. There are many ways to do this, so I wont enter into the details of the implementation, you cananyway find the utils.pyfile in the repository. Just one note, I have used the iterator parser from the lxml

python package to parse the file iteratively. This is due to the fact that GPX files may have quite a size (Fortesting purposes I used a file with 33000 lines), so the iterator parser may improve the speed and solve some

recursion problems.


38/112

The Best Of

38

On the view, we will just check if the method of the request is POST or GET. If it is the first case, it meansthat the user has submitted a form, in which case we will check if it is valid and we will parse and store it. In

both cases we will retrieve a list of routes and we will embed it in the HTML file, so the views.pyfile shouldlook more or less like in the following example.

Listing 3. views.py

fromdjango.shortcutsimportrender_to_response

fromformsimportGPXUploadForm

fromutilsimportparse_gpx

fromsimplegisapp.modelsimportRoute

defroute(request):

ifrequest.method== POST:

form= GPXUploadForm(request.POST, request.FILES)

ifform.is_valid():

data= form.cleaned_data

f, name= data[file], data[name]

route= parse_gpx(f=f, name=name)

else: form= GPXUploadForm()

routes= Route.objects.all()

dict= {form:form,routes:routes}

dict.update(csrf(request))

returnrender_to_response(routepage.html, dict)

And with this last step we have a very simple application working. Of course we have to configure thesettings file to point tothe right templates and static files directory, but I will leave that out of the article. Aguide can be found in https://docs.djangoproject.com/en/dev/ref/settings/.

At this point however, we have not used all the power of the GeoDjango package, and we havent developed

any kind of map to show the routes to the users. On the next section we will see some functions of the GEOSAPI, and we will get into the development of the frontend later.

Extending the models

Until now, we have just stored some geometric fields and processed files to obtain that information, butGeoDjangos real power resides in the operations that it allows us to perform with those geometric objects.To show some of these features we will be performing a simple analysis over the routes users have uploadedto the web page.

A very basic operation we can do is to calculate the total length of one route. Usually this would meaniterating over every coordinate and calculating the distance to the next (for example with the Haversine

formula). GEOS makes this trivial providing an attribute on the geometry object, the lengthattribute, whichcalculates the length differently depending on the geometry object. To make it as simple as possible, we willmake a wrapper on the Route model that return the length of it.

Listing 4. The length

deflength(self):

returnself.representation.length

We can, of course, make more complex operations, for example, we will implement a function that givena route, tells us which is the nearest. For this we will be using the distance function, which returns us thedistance between the nearest points on two geometries. We will define the method nearestin the route model.
https://docs.djangoproject.com/en/dev/ref/settings/https://docs.djangoproject.com/en/dev/ref/settings/https://docs.djangoproject.com/en/dev/ref/settings/


39/112

The Best Of

39

Listing 5. nearest method

defnearest(self):

minDist= sys.maxint

rt= self

forrouteinRoute.objects.exclude(pk=self.pk):

dist= self.representation.distance(route.representation)

ifdist< minDist:

minDist= dist

rt= route

returnrt

Finally, we will define another method to get the GeoJSON representation of a route. GeoJSON is a formatdefined to encode simple geographic features in JavaScript Object Notation, which is supported by the JSmapping framework we are using.

Listing 6. geoJSON

defgeoJSON(self):

returnself.representation.json

With this we have seen some of the most simple applications of the GEOS API. However, we have onlyscratched the tip or the iceberg, there is much more it can us, so I encourage anyone to explore this libraryand discover the powerful applications that can be easily developed using it. A complete guide to GeoDjangoand all of its features can be found on the Django documentation pages: https://docs.djangoproject.com/en/dev/ref/contrib/gis/.

The Front End

Now that we have created our models and our views, we can continue to implement the front end of the

application. For that, we will simply create one HTML page, which will contain a form for uploa

Documents

Software Developer Journal