Revel Gorp and Mysql

7/25/2019 Revel Gorp and Mysql

1/15

This is a compilation of things I've had to piece together from blog

posts, example projects, and experimentation. I hope it saves you a

bunch of time.

Goal:

Build a simple Revel controller that performs CRUD operations against

a MySQL database.

Caveats:

Use RESTful Paths.

Use JSON request bodies and responses.

Use Revel Validation.

This setup should reflect a backend that you would use with a Single

Page Application framework like AngularJS.

Synopsis:

Revel, GORP, and MySQLBuilding a classic 3-tier web application controller in

Golang.

Richard Clayton


2/15

I'm building a simple auction management system for the wife. One of

the model items I have to deal with is a BidItemBidItem , which represents

something you can purchase during the bidding process. I'll

demonstrate how to build the full controller and database bindings for

this one model item.

We will implement the controller with the following steps:

1. Define REST Routes.

2. Define Model.

3. Implement Validation Logic.

4. Set up the GorpControllerGorpController .

5. Register Configuration.

6. Set up the Database.

7. Extend the GorpControllerGorpController .

8. Handle JSON Request Body (for Creation and Updates).

9. Implement Controller CRUD functionality.

What I will not go into is security and proper response handling. Also,

I'm going to be as succinct as possible so I don't waste your time.

1. Define REST Routes.

In /conf/routes/conf/routes , add the following lines:

GET /item/:id BidItemCtrl.GetPOST /item BidItemCtrl.AddPUT /item/:id BidItemCtrl.UpdateDELETE /item/:id BidItemCtrl.DeleteGET /items BidItemCtrl.List

The first two columns are obvious (HTTP Verb and Route). The :id:id is

an example of placeholder syntax. Revel will provides three ways to

get this value, including using the value as a function argument. The

BidItemCtrl.BidItemCtrl. refers to the controller and method that will

handle the route.


3/15

2. Define Model.

I've created a folder called modelsmodels in /app/app . This is where

I define my models. This is my model in file

/app/models/bid-item.go/app/models/bid-item.go :

packagemodels

typeBidItem struct{Id int64 `db:"id" json:"id"`

Name string `db:"name" json:"name"` Category string `db:"category"json:"category"` EstimatedValue float32 `db:"est_value"json:"est_value"` StartBid float32 `db:"start_bid"

json:"start_bid"` BidIncrement float32 `db:"bid_incr"json:"bid_incr"` InstantBuy float32 `db:"inst_buy"json:"inst_buy"`}

Note the meta informationafter each property on the structure. The

db:""db:"" is the desired column-name in the database and thejson:""json:"" is the JSON property name used in

serialization/deserialization. These values are not mandatory, but if

you do not specify them, both GORP and the encoding/jsonencoding/json package

will use the property name. So you will be stuck with columns and

JSON properties in Pascal casing.

3. Implement Validation Logic.

For model validation, we could implement our own custom logic, but

Revel already has a pretty good validation facility (so we'll just use

that). Of course, this means we are going to trade off having a clean

model (unaware of other frameworks) for development expedience.

import("github.com/revel/revel""regexp"

)


4/15

// ...

func(b *BidItem) Validate(v *revel.Validation) {

v.Check(b.Name,revel.ValidRequired(),revel.ValidMaxSize(25))

v.Check(b.Category,revel.ValidRequired(),

revel.ValidMatch(regexp.MustCompile(

"^(travel|leasure|sports|entertainment)$")))

v.Check(b.EstimatedValue,revel.ValidRequired())

v.Check(b.StartBid,revel.ValidRequired())

v.Check(b.BidIncrement,revel.ValidRequired())

}

This implements some of the validation logic by constructing the

ruleset for model. It's certainly not complete and leaves much to be

desired.

4. Set up the GorpControllerGorpController .

If you would like transactions for your database, but don't want to

have to manually set them up in your controller, I would recommend

following this step. Keep in mind, all the code I show you after this will

utilize this functionality.

If you go spelunkingin the Revel sample projects, you will find this

little gem in the Bookingsproject. This is the GorpControllerGorpController , which

is a simple extension to the revel.Controllerrevel.Controller that defines some boiler

plate around wrapping controller methods in database transactions.

I've moved out the database creation code into a separate file (which

we will come back to in step 6), so the file should now look like this:

packagecontrollers

import("github.com/coopernurse/gorp"


5/15

"database/sql""github.com/revel/revel"

)

var(Dbm *gorp.DbMap

)

typeGorpController struct{*revel.Controller

Txn *gorp.Transaction}

func(c *GorpController) Begin() revel.Result {txn, err :=Dbm.Begin()iferr != nil{

panic(err)}c.Txn = txnreturn nil

}

func(c *GorpController) Commit() revel.Result {ifc.Txn == nil{

return nil

}iferr :=c.Txn.Commit(); err != nil &&err !=

sql.ErrTxDone {panic(err)

}c.Txn = nilreturn nil

}

func(c *GorpController) Rollback() revel.Result {ifc.Txn == nil{

return nil

}iferr :=c.Txn.Rollback(); err != nil &&err !=

sql.ErrTxDone {panic(err)

}c.Txn = nilreturn nil

}

Copy this code into a new file under /app/controllers/app/controllers . I

call the file gorp.gogorp.go .

Two important variables to note are the var ( Dbm *gorp.DbMap )var ( Dbm *gorp.DbMap ) and

the GorpControllerGorpController 's property Txn *gorp.TransactionTxn *gorp.Transaction . We will use

the DbmDbm variable to perform database creation and the TxnTxn variable

to execute queries and commands against MySQL in the controller.


6/15

Now in order for us to get the magically wrappedtransactional

functionality with our controller actions, we need to register these

functions with Revel's AOP mechanism.

Create a file called init.goinit.go in /app/controllers/app/controllers . In this

file, we will define an init()init() function which will register the handlers:

packagecontrollers

import "github.com/revel/revel"

funcinit(){revel.InterceptMethod((*GorpController).Begin,

revel.BEFORE)revel.InterceptMethod((*GorpController).Commit,

revel.AFTER)

revel.InterceptMethod((*GorpController).Rollback,revel.FINALLY)}

Now we have transaction support for controller actions.

5. Register Configuration.

We are about ready to start setting up the database. Instead of

throwing in some hard coded values for a connection string, we would

prefer to pull this information from configuration. This is how you do

that.

Open /conf/app.conf/conf/app.conf and add the following lines. Keep in

mind that there are sections for multiple deployment environments.

For now, we're going to throw our configuration values in the [dev][dev]

section:

[dev]

db.user = auctioneerdb.password = passworddb.host = 192.168.24.42

db.port = 3306db.name = auction


7/15

Now these values will be made available to use at runtime via the Revel

API: revel.Config.String(paramName)revel.Config.String(paramName) .

6. Setup the Database.

Next, we need to set up the database. This involves instantiating a

connection to the database and creating the tables for our model if the

table does not already exist.

We are going to go back to our /app/controllers/init.go/app/controllers/init.go

file and add the logic to create a database connection, as well as, the

table for our BidItemBidItem model if it does not exist.

First, I'm going to add two helper functions. The first is a more generic

way for us to pull out configuration values from Revel (including

providing a default value). The second is a helper function for building

the MySQL connection string.

funcgetParamString(param string, defaultValue string)string{

p, found :=revel.Config.String(param)if!found {

ifdefaultValue == ""{revel.ERROR.Fatal("Cound not find parameter:

" +param)} else{

returndefaultValue}

}returnp

}

funcgetConnectionString() string{host :=getParamString("db.host", "")port :=getParamString("db.port", "3306")user :=getParamString("db.user", "")pass :=getParamString("db.password", "")dbname :=getParamString("db.name", "auction")protocol :=getParamString("db.protocol", "tcp")dbargs :=getParamString("dbargs", " ")

ifstrings.Trim(dbargs, " ") != ""{

dbargs = "?" +dbargs} else{

dbargs = "" }

returnfmt.Sprintf("%s:%s@%s([%s]:%s)/%s%s",user, pass, protocol, host, port, dbname, dbargs)

}


8/15

With these two functions, we can construct a connection to MySQL.

Let's create a function that will encapsulate initializing the database

connection and creating the initial databases:

varInitDb func() = func(){

connectionString :=getConnectionString()ifdb, err :=sql.Open("mysql", connectionString);

err != nil{revel.ERROR.Fatal(err)

} else{Dbm = &gorp.DbMap{

Db: db,Dialect: gorp.MySQLDialect{"InnoDB", "UTF8"}}

}// Defines the table for use by GORP

// This is a function we will create soon.

defineBidItemTable(Dbm)iferr :=Dbm.CreateTablesIfNotExists(); err != nil{

revel.ERROR.Fatal(err)}

}

One thing to note in the code above is that we are setting the DbmDbm

variable we defined in the GorpControllerGorpController file gorp.gogorp.go . Using the

connection, DbmDbm , we will define our BidItemBidItem schema and create thetable if it does not exist. We have not yet written the

defineBidItemTable(Dbm)defineBidItemTable(Dbm) function; I will show you that soon.

Before we move on, we need to talk about imports. All of the helper

functions and code in the init.goinit.go file will require the following

libraries:

import("github.com/revel/revel""github.com/coopernurse/gorp""database/sql"

_ "github.com/go-sql-driver/mysql""fmt""strings"

)

Of special note is the import and non-useof the mysqlmysql library:

_ "github.com/go-sql-driver/mysql"_ "github.com/go-sql-driver/mysql" . If you do not include this

import statement, your project will break. The reason is that GORP


9/15

relies on the database/sqldatabase/sql package, which is only a set of interfaces.

The mysqlmysql package implements those interfaces, but you will not see

any direct reference to the library in the code.

Now it's time to implement the defineBidItemTable(Dbm)defineBidItemTable(Dbm) function.

funcdefineBidItemTable(dbm *gorp.DbMap){// set "id" as primary key and autoincrement

t :=dbm.AddTable(models.BidItem{}).SetKeys(true,"id")

// e.g. VARCHAR(25)

t.ColMap("name").SetMaxSize(25)}

Notice how I use the term defineand not create. This is because the

database is not actually created until the call to

Dbm.CreateTablesIfNotExists()Dbm.CreateTablesIfNotExists() .

Finally, the InitDb()InitDb() function needs to be executed when the

application starts. We register the function similar to the way we did

the AOP functions for the GorpControllerGorpController in the init()init() function:

funcinit(){revel.OnAppStart(InitDb)revel.InterceptMethod((*GorpController).Begin,

revel.BEFORE)revel.InterceptMethod((*GorpController).Commit,

revel.AFTER)revel.InterceptMethod((*GorpController).Rollback,

revel.FINALLY)}

When we are done with these steps, you will be able to start the

application and verify the table creation in MySQL. If you are curious

as to how GORP defines the table, here's a screen capture of

mysql > describe BidItem;mysql > describe BidItem; :


10/15

7. Extend the GorpControllerGorpController .

We are ready to start working on the controller. First we need to createa new file for the controller: /app/controllers/bid-item.go/app/controllers/bid-item.go

. Using the GorpControllerGorpController as the prototype, let's define our new

BidItemCtrlBidItemCtrl :

packagecontrollers

import("auction/app/models"

"github.com/revel/revel""encoding/json"

)

typeBidItemCtrl struct{GorpController

}

We will use those imports soon.

8. Handle JSON Request Body.

This is one of those things that is not well documented in Revel, in part

because, Revel doesn't consider it a part of the framework. If you want

to be able to transform JSON requests into your own modelrepresentations you will need to define that functionality.

We are going to add a simple method to the controller that will parse

the request body and return a BidItemBidItem and ErrorError tuple:


11/15

func(c BidItemCtrl) parseBidItem() (models.BidItem,

error) {biditem :=models.BidItem{}err :=json.NewDecoder(c.Request.Body).Decode

(&biditem)returnbiditem, err

}

We now have a way of getting a BidItemBidItem from the Request.BodyRequest.Body .

You will obviously have to know the context in which to use this

method, but that should be obvious since you do that anyway in most

web frameworks (albeit it's a little it's a little less elegant in Revel).

9. Implement Controller CRUD functionality.

Let's finish up the rest of the controller!

Add BidItem.

Inserting the record is pretty easy with GORP:

c.Txn.Insert()c.Txn.Insert() .

func(c BidItemCtrl) Add() revel.Result {ifbiditem, err :=c.parseBidItem(); err != nil{

returnc.RenderText("Unable to parse the BidItemfrom JSON.")

} else{// Validate the model

biditem.Validate(c.Validation)ifc.Validation.HasErrors() {

// Do something better here!returnc.RenderText("You have error in your

BidItem.")} else{

iferr :=c.Txn.Insert(&biditem); err != nil{returnc.RenderText(

"Error inserting record intodatabase!")

} else{returnc.RenderJson(biditem)

}

}}

}


12/15

Get BidItem.

func(c BidItemCtrl) Get(id int64) revel.Result {

biditem := new(models.BidItem)err :=c.Txn.SelectOne(biditem,

`SELECT * FROM BidItem WHERE id = ?`, id)iferr != nil{

returnc.RenderText("Error. Item probably doesn'texist.")

}returnc.RenderJson(biditem)

}

List BidItems (with paging).

For listing BidItemBidItem s, were going to need to implement some simplepaging. Since we have an auto incremented, indexed bigint(20)bigint(20) for

an identifier, we'll keep it simple and use the last id as the start field

and a limit to indicate the amount of records we want. For this, we will

accept two query parameters: lidlid (last id) and limitlimit (number of rows

to return).

Since we don't want a SQL injection attack, we're going to parse these

query options as integers. Here are a couple of functions to help the

parsing (keeping it in /app/controllers/common.go/app/controllers/common.go ):

packagecontrollers

import(

"strconv")

funcparseUintOrDefault(intStr string, _default uint64)uint64{

ifvalue, err :=strconv.ParseUint(intStr, 0, 64);err != nil{

return_default

} else{returnvalue

}}

funcparseIntOrDefault(intStr string, _default int64)int64{

ifvalue, err :=strconv.ParseInt(intStr, 0, 64);

err != nil{return_default


13/15

} else{

returnvalue}

}

The controller function is pretty simple:

func(c BidItemCtrl) List() revel.Result {lastId :=parseIntOrDefault(c.Params.Get("lid"), -1)limit :=parseUintOrDefault(c.Params.Get("limit"),

uint64(25))biditems, err :=c.Txn.Select(models.BidItem{},

`SELECT * FROM BidItem WHERE Id > ? LIMIT ?`,lastId, limit)

iferr != nil{

returnc.RenderText("Error trying to get records from DB.")

}returnc.RenderJson(biditems)

}

Update BidItem.

func(c BidItemCtrl) Update(id int64) revel.Result {

biditem, err :=c.parseBidItem()iferr != nil{returnc.RenderText("Unable to parse the BidItem

from JSON.")}// Ensure the Id is set.

biditem.Id = idsuccess, err :=c.Txn.Update(&biditem)iferr != nil ||success == 0{

returnc.RenderText("Unable to update bid item.")}

returnc.RenderText("Updated %v", id)}

GORP takes the convention of returning a tuple of int64int64 and ErrorError

on Updates and Deletes. The int64int64 is either a 00 or 11 value

signifying whether the action was successful or not.

Delete BidItem.

And the last action for the controller.


14/15

func(c BidItemCtrl) Delete(id int64) revel.Result {

success, err :=c.Txn.Delete(&models.BidItem{Id: id})

iferr != nil ||success == 0{

returnc.RenderText("Failed to remove BidItem")

}

returnc.RenderText("Deleted %v", id)

}

Conclusion.

While a little long, I think that was probably the most complete

walkthrough/example you'll find on implementing a Controller in

Revel with GORP and MySQL. I hope this helps you in your effort of

learning Go, GORP or Revel.

References.

http://revel.github.io/manual/index.html

https://github.com/coopernurse/gorp

https://github.com/revel/revel/tree/master/samples/booking

http://nathanleclaire.com/blog/2013/11/04/want-to-work-

with-databases-in-golang-lets-try-some-gorp/

Plus many other StackOverflow articles and blog posts...

Richard Clayton Unrepentant Thoughts on Software and Management.

July 18, 2014 u 6 Comments

Read Next: Quick MySQL Box Using Vagrantand Ansible


15/15

Blog Home Archive RSS

You can also find me on Twitter and GitHub.

2015 Richard Clayton

Documents

Revel Gorp and Mysql