JavaScript on the Serverbouvin/itWoT/2019/slides/JavaScript on the Server.pdf · ./index.js It is considered good coding style to let the central index.js be sparse, and let the actual

JavaScript on the ServerNiels Olof Bouvin

1

Overview

The HTTP protocol The http module Input and output on the Web The Express framework HTML templates with handlebars Data-driven sites

2

Client/server communication

The Web is an example of a client/server architecture clients make requests servers return resources communication is always initiated by the client

In the context of the Web, clients are (usually) Web browsers, though they can be anything that speaks http and https

http and https are the protocols with which Web clients and servers communicate

Server

Client Clientrequest

resource requestresource

3

The Uniform Resource Locator

Combines protocol the method with which computers communicate, e.g., http, https, ftp, rtp, …

with host or server name usually resolved using DNS

resource name could just be a file name, but could also something more general (we’ll return to this)

and fragment identifier used to point into a resource—usually a text-based one

protocolz }| {https ://

serverz }| {users�cs.au.dk/

resourcez }| {bouvin/dBIoTP2PC/2017/exam.html#

fragmentz }| {topics

4

The HyperText Transfer ProtocolAt its simplest, a request and response for a resource

but in reality, it is a bit more complex and far richer: http://127.0.0.1:8080/hello.txt

GET /hello.txt HTTP/1.1 Host: 127.0.0.1:8080 User-Agent: curl/7.54.0 Accept: */*

HTTP/1.1 200 OK server: ecstatic-2.2.1 last-modified: Tue, 02 Jan 2018 11:54:58 GMT etag: "77865272-13-"2018-01-02T11:54:58.000Z"" cache-control: max-age=3600 content-length: 13 content-type: text/plain; charset=UTF-8 Date: Tue, 02 Jan 2018 12:01:52 GMT Connection: keep-alive

Hello World!

request: (from client)

response: (from server)

headers explaining what the client wants and what the server delivers

the actual resource(A blank line)

5

Operating on resources

HTTP supports four main methods:

GETretrieve the state of a resource — don’t modify it

POSTcreate new resource, do not specify identifier

PUTupdate existing resource, or create new resource with an identifier

DELETEremove a resource

6

A selection of HTTP status codes200 OK

Standard response for successful HTTP requests.

201 Created The request has been fulfilled and resulted in a new resource being created

202 Accepted The request has been accepted, but is not yet fulfilled

301 Moved Permanently This and all future requests should be directed to the given URI

404 Not Found The requested resource could not be found

500 Internal Server Error The server has experienced an error, and could not fulfil the request

7

Overview


8

What should a Web server do?

Receive a Request extract information from the Request’s header and body, such as resource identification (‘/min/test.txt’), type of operation (GET, PUT, …), acceptable data formats (‘text/html’), etc.

Create a Response return a suitable resource for the Request of the appropriate data type and with the correct information in the header

This is all

9

Back to Hello World

Now that we are starting server programming, we should begin using proper tools and structure in our code

that means using NPM to handle the basics, such as starting the program, and keeping track of necessary modules, which is done through the package.json file as well as having a systematic approach to folder structure

A new project can be initialised with npm init -fy

10

./package.json

I have filled out a few fields, including start and author

. ├── app │ └── index.js ├── index.js └── package.json

{ "name": "basic-hello-world", "version": "1.0.0", "main": "basic-hello-world.js", "scripts": { "start": "node index.js", "test": "echo \"Error: no test specified\" && exit 1" }, "keywords": [], "author": "Niels Olof Bouvin", "license": "ISC", "description": "A simple hello world server" }

11

./index.js

It is considered good coding style to let the central index.js be sparse, and let the actual functionality reside in appropriately named folders and files


'use strict' require('./app/index')

12

./app/index.js

http.createServer() requires a callback function that gets a request and a response object

the request object is read (quite simply, here) and the response object has data added to it, and is finalised with .end(), which returns the response to the Web browser but first, the server must be set to listen to incoming requests on a specific port


'use strict' const http = require('http')

const port = 3000 let counter = 0

const server = http.createServer((request, response) => { console.log(`request.url=${request.url}`) response.end(`Hello World! times ${counter++}\n`) }) server.listen(port, () => console.log(`Listening on http://localhost:${port}`) )

13

Running the server

14

Taking a closer look

15

Adding a bit of HTML style

This should work, right?




const server = http.createServer((request, response) => { console.log(`request.url=${request.url}`) response.end(`<i>Hello World!</i> times ${counter++}\n`) }) server.listen(port, () => console.log(`Listening on http://localhost:${port}`) )

16

Probably not as intended

17

Correct header info is crucial

If we do not set an explicit Content-Type using the setHeader() method, the browser will assume it is ‘text/plain’, and that is rarely what we want

‘text/html’ is right for HTML documents




const server = http.createServer((request, response) => { console.log(`request.url=${request.url}`) response.setHeader('Content-Type', 'text/html') response.end(`<i>Hello World!</i> times ${counter++}\n`) }) server.listen(port, () => console.log(`Listening on http://localhost:${port}`) )

18

Taking a closer look again

19

And in the Web browser…

20

Overview


21

Input and Output on the Web

We have seen several example of getting data from a server to a Web browser

But how do you get data from a Web page back to the server?

this was solved, in the simplest form, in the early Web around HTML 2.0

There are two basic methods GET POST

22

Input through GET

Very simple: http://…/foo.html?arg1=value&arg2=value&arg3=value (and so on)

Advantage you can make links that contain arguments to the server

Disadvantage there is a space limitation cannot be used for, e.g., file upload you would not want a password to be visible in the browser bar

23

A more specific greeting

The standard Node ‘url’ module can parse the query

'use strict' const http = require('http') const url = require('url')

const port = 3000 let counter = 0 let recipient

const server = http.createServer((request, response) => { console.log(`request.url=${request.url}`) const query = url.parse(request.url, true).query if ('recipient' in query) { recipient = query['recipient'] } else { recipient = 'World' } response.setHeader('Content-Type', 'text/html') response.end(`<i>Hello ${recipient}!</i> times ${counter++}\n`) }) server.listen(port, () => console.log(`Listening on http://localhost:${port}`) )

24

Input through POST

Requires a different HTTP method: POST

Data is transferred in the body of the request rather than the URL

you can send much more data because this data can be split up across several network packages, it is necessary to assemble the data at the server, before it can be processed the standard Node module ‘querystring’ can parse this kind of data

The data must be input through a <form> element on a Web page

25

A HTML form

A form has a method and an action

There are many different kinds of input types

<!DOCTYPE html> <html> <head> <meta charset="utf-8" /> <title>Greetings for you</title> </head> <body> <form method="POST" action="http://localhost:3000/"> <div> <label for="recipient">Recipient:</label> <input type="text" id="recipient" name="recipient"> </div> <div> <label for="message">Message:</label> <textarea id="message" name="message"></textarea> </div> <div> <button type="submit">Submit</button> </div> </form> </body> </html>

26

Handling the POST data'use strict' const http = require('http') const querystring = require('querystring')

const port = 3000 const server = http.createServer((request, response) => { console.log(`request.url=${request.url}`) if (request.method === 'POST') { let body = '' request.on('data', (data) => { // got some data body += data }) request.on('end', () => { // all data received console.log(`request.body=${body}`) const post = querystring.parse(body) const recipient = post['recipient'] || 'World' const message = post['message'] || '' response.setHeader('Content-Type', 'text/html') response.end(`<div>Hello ${recipient}!</div> <div>${message}</div>\n`) }) } }) server.listen(port, () => console.log(`Listening on http://localhost:${port}`) )

27

Filling a form

28

Building a server with the http module

It is completely feasible to build a Web server based on the standard Node.js modules

However, as servers become more sophisticated, it requires a fair amount of manual labour

This is where a Web server framework can come in handy

There are many for Node.js, but the most popular is Express

29

Overview


30

Adding Express to your project

npm install express --save installs the required modules in node_modules/ and updates package.json

add node_modules/ to .gitignore (you can install all the modules with ‘npm install’)31

Hello World in Express

The basic case is not much shorter than with http, but the structure quickly becomes clearer, and we do not have to set the Content-Type header ourselves

'use strict' const express = require('express') const app = express() const port = 3000

app.get('/', (request, response) => { response.send('Hello World!') })

app.listen(port, (err) => { if (err) return console.error(Àn error occurred: ${err}`) console.log(`Listening on http://localhost:${port}/`) })

32

Chaining middleware in Express

Express functions by chaining middleware functions that take (request, response, next) as arguments, and each end by call the next() in line

'use strict' const express = require('express') const app = express() const port = 3000

app.use((request, response, next) => { request.timer = Date.now() next() }) app.use((request, response, next) => { console.log(`request.url=${request.url}`) next() }) app.get('/', (request, response) => { response.send('Hello World!') console.log(`Processing took: ${Date.now() - request.timer} ms`) }) app.listen(port, (err) => { if (err) return console.error(Àn error occurred: ${err}`) console.log(`Listening on http://localhost:${port}/`) })

33

Overview


34

Using a template engine: handlebars

It is extremely tedious and error prone to write HTML directly inside JavaScript

what if the design changes? no tool support for syntax etc

Most Web sites, while having many pages, will have relatively few layouts, wherein the content will vary

it is also tedious to have to repeat the same bits over and over in a static Web site

Template frameworks are made to support this kind of scenario:

define the layout once, mark the places where content should be inserted, configure your application, and everything works

35

The layout: main.hbs

Standard HTML, except things inside {{{}}} will be replaced directly (HTML will not be escaped)

You can develop a Web page until satisfied, and then replace the changing bits with {{{}}}

app └── index.js views ├── hello.hbs └── layouts └── main.hbs

<!DOCTYPE html> <html> <head> <meta charset="utf-8" /> <title>Express handlebars !</title> </head> <body> {{{body}}} </body> </html>

36

The view: hello.hbs

This is what replaces {{{body}}} in main.hbs

Things inside {{}} will have HTML escaped


<h2>Hello {{recipient}}</h2>

37

Configuring Express: index.js'use strict' const express = require('express') const exphbs = require('express-handlebars') const path = require('path') const app = express() const port = 3000

app.engine('.hbs', exphbs({ defaultLayout: 'main', extname: '.hbs', layoutsDir: path.join(__dirname, '../views/layouts') })) app.set('view engine', '.hbs') app.set('views', path.join(__dirname, '../views'))

app.get('/', (request, response) => { response.render('hello', { recipient: ‘World’// remember {{recipient}} in hello.hbs? }) })



38

Handling input with Express

Express handles get input directly request.query.recipient (assuming http://…/somePage?recipient=Johnny)

POST data requires an additional module, such as body-parser (there are quite a few others)

npm install body-parser --save app.use(bodyParser.urlencoded({ extended: false })) // to add it to express

39

The greeting: hello.hbs

Ought to be familiar from earlier

app └── index.js views ├── form.hbs ├── hello.hbs └── layouts └── main.hbs

<h2>Hello {{recipient}}</h2> <p>{{message}}</p>

40

The form: form.hbs

Ought to be familiar from earlier


<form method="POST" action="/"> <div> <label for="recipient">Recipient:</label> <input type="text" id="recipient" name="recipient"> </div> <div> <label for="message">Message:</label> <textarea id="message" name="message"></textarea> </div> <div> <button type="submit">Submit</button> </div> </form>

41

The code: index.js 1/2

All this configures Express with body-parser and handlebars


'use strict' const path = require('path') const express = require('express') const exphbs = require('express-handlebars') const bodyParser = require('body-parser') const app = express() const port = 3000


app.use(bodyParser.urlencoded({ extended: false }))

42

The code: index.js 2/2

This site handles both POST and GET data


app.get('/', (request, response) => { response.render('form') }) app.post('/', (request, response) => { response.render('hello', { recipient: request.body.recipient, message: request.body.message }) }) app.get('/hello', (request, response) => { response.render('hello', { recipient: request.query.recipient, message: request.query.message }) }) app.listen(port, (err) => { if (err) return console.error(Àn error occurred: ${err}`) console.log(`Listening on http://localhost:${port}/`) })

43

In action, handling POST and GET

44

Overview


45

Data drives the Web

In practice, most Websites are database-driven

Layout, stylesheets, images, etc., are typically static resources and handled separately

The actual contents resides in databases, and are added into the Web pages as they are generated by the server

we will later be looking at Web pages that “build themselves” by retrieving data from servers

46

Static resources

Everything found in the specified folder is served as http-server would have served it

the files in public/ are just ordinary HTML and CSS files

You can call app.use(express.static()) multiple times, if you need to serve static resources from different paths

app └── index.js public/ ├── index.html └── style └── main.css

'use strict' const express = require('express') const path = require('path') const app = express() const port = 3000

app.use(express.static(path.join(__dirname, '../public')))


47

Static resources

Note how the css file is down in a folder, but is correctly served nonetheless by express.static

app └── index.js public/ ├── index.html └── style └── main.css

<!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8" /> <title>Static pages!</title> <link rel="stylesheet" type="text/css" media="screen" href="style/main.css" /> </head> <body> <h1>Hello World!</h1> <p>This is just a static page, served up by Express, and referencing a stylesheet, served in a similar manner.</p> </body> </html>

body { background-color: palegreen; }

48

Databases!

I believe you have heard of them?

In a Web context, databases largely come in two kinds: SQL databases, such as MySQL, PostgreSQL, SQLite, DB2, Oracle, … NoSQL databases, such as MongoDB, CouchDB, Cassandra, Redis, …

Given your expertise with SQL, we will stick with SQL…

49

The data driven Greetings site

Let’s create a site, where users can add their own greeting to a list of greetings

50

What should the site support?

GET /form the form with which to create a greeting

POST /greetings create a new greeting

GET /greetings/:id get a specific greeting

GET /greetings get all greetings

DELETE /greeting/:id delete a specific greeting

51

Redirecting

A Web server can instruct a Web browser to another page through the Location: field in the HTTP header

This can be done from Express with the method redirect()

We will use that to automatically direct users from / to /form

52

Let’s start with a simpler version

When starting out, it can be simpler to take things step by step, e.g., starting out with an in-memory array before building the full database-backed version

Once that works, it will provide us with the skeleton for the full version

53

The code: index.js 1/2app └── index.js public └── style └── main.css views ├── form.hbs ├── greeting.hbs └── layouts └── main.hbs

'use strict' const path = require('path') const express = require('express') const exphbs = require('express-handlebars') const bodyParser = require('body-parser') const app = express() const port = 3000 const greetings = []


app.use(bodyParser.urlencoded({ extended: true }))


54

The code: index.js 1/2app └── index.js public └── style └── main.css views ├── form.hbs ├── greeting.hbs └── layouts └── main.hbs

app.get('/', (request, response) => { response.redirect('/form') }) app.get('/form', (request, response) => { response.render('form') }) app.post('/greetings', (request, response) => { const greeting = {recipient: request.body.recipient, message: request.body.message} greetings.push(greeting) response.redirect('/greetings') }) app.get('/greetings', (request, response) => { response.render('greeting', { greetings: greetings }) }) app.get('/greetings/:id', (request, response) => { const id = request.params.id const greeting = greetings[id] response.render('greeting', { greetings: [greeting] }) }) app.listen(port, (err) => { if (err) return console.error(Àn error occurred: ${err}`) console.log(`Listening on http://localhost:${port}/`) })

55

Greetings with Handlebars

With #each, Handlebars can iterate through arrays of objects, accessing the fields within the objects

app └── index.js public └── style └── main.css views ├── form.hbs ├── greeting.hbs └── layouts └── main.hbs

{{#each greetings}} <h2>Hello {{recipient}}</h2> <p>{{message}}</p> {{/each}}

<div><a href="/form">Add a greeting</a></div>

56

The site in action

57

Moving on to a proper database

We need a proper database

You already know MySQL, so let’s try something new!

By far the most widespread SQL database system in the world is SQLite

it is very compact (~700 kB), fast, and implements most of SQL-92 it is not very good with concurrent writes, but concurrent reads are fine it is found almost everywhere, from desktop computers (handles, e.g., Mail on macOS) to Web browsers to mobile phones (your contact list is almost certainly in SQLite) some idiot lecturer forgot to install it on the Raspberry Pi image—see Resources for instructions (and the node-sqlite3 module installer is slow on RPi, sorry about that)

But, once installed it is easy to administer58

SQLite3 in action

59

The database: db.js 1/2

We open the database file, and if our table does not exist, we create it using standard SQL

app └── index.js db ├── db.js └── greetings.sqlite public ├── js │ └── main.js └── style └── main.css views ├── form.hbs ├── greeting.hbs └── layouts └── main.hbs

const sqlite3 = require('sqlite3').verbose() const path = require('path') const dbName = path.join(__dirname, 'greetings.sqlite') const db = new sqlite3.Database(dbName)

db.serialize(() => { const sql = ` CREATE TABLE IF NOT EXISTS greetings (id integer primary key, recipient, message TEXT) ` db.run(sql) })

60

The database: db.js 2/2class Greetings { static all (callback) { db.all('SELECT * FROM greetings', callback) }

static find (id, callback) { db.get('SELECT * FROM greetings WHERE id = ?', id, callback) }

static create (greeting, callback) { const sql = 'INSERT INTO greetings(recipient, message) VALUES (?, ?)' db.run(sql, greeting.recipient, greeting.message, callback) }

static delete (id, callback) { if (!id) { return callback(new Error('Please provide an id')) } db.run('DELETE FROM greetings WHERE id = ?', id, callback) } }

module.exports = db module.exports.Greetings = Greetings


61

The code: index.js 1/2'use strict' const path = require('path') const express = require('express') const exphbs = require('express-handlebars') const bodyParser = require('body-parser') const Greetings = require('../db/db').Greetings const app = express() const port = 3000

app.engine('.hbs', exphbs({ defaultLayout: 'main', extname: '.hbs', layoutsDir: path.join(__dirname, '../views/layouts') })) app.set('view engine', '.hbs') app.set('views', path.join(__dirname, '../views')) app.use(bodyParser.urlencoded({ extended: true }))


app.get('/', (request, response, next) => { response.redirect('/form') })

app.get('/form', (request, response, next) => { response.render('form') })


62

The code: index.js 2/2app.post('/greetings', (request, response, next) => { const greeting = {recipient: request.body.recipient, message: request.body.message} Greetings.create(greeting, (err, greeting) => { if (err) return next(err) response.redirect('/greetings') }) }) app.get('/greetings', (request, response, next) => { Greetings.all((err, greetings) => { if (err) return next(err) response.render('greeting', { greetings: greetings }) }) }) app.get('/greetings/:id', (request, response, next) => { const id = request.params.id Greetings.find(id, (err, greeting) => { if (err) return next(err) response.render('greeting', { greetings: [greeting] }) }) }) app.listen(port, (err) => { if (err) return console.error(Àn error occurred: ${err}`) console.log(`Listening on http://localhost:${port}/`) })


63

The site in action

64

Note on the use of databases

You should always let the module handle data that you have no control over

e.g., something that has been entered into a form and sent over the network

This is correct, because it allows the module to sanitise the input

This is wrong and dangerous, do not do this:

const sql = 'INSERT INTO greetings(recipient, message) VALUES (?, ?)' db.run(sql, greeting.recipient, greeting.message, callback)

const bad = ÌNSERT INTO greetings(recipient, message) VALUES (${greeting.recipient}, ${greeting.message})` db.run(bad)

65

Wrap-up

Node.js comes into its own, once we start server programming

With Express and its associates modules, it is possible to create functional Web sites in relatively little code

Templates alone will save you a lot of time

Proper databases of course makes everything better

66

Documents

JavaScript on the Serverbouvin/itWoT/2019/slides/JavaScript on the Server.pdf · ./index.js It is considered good coding style to let the central index.js be sparse, and let the actual