7/25/2019 Revel Gorp and Mysql
1/15
This is a compilation of things I've had to piece together from blog
posts, example projects, and experimentation. I hope it saves you a
bunch of time.
Goal:
Build a simple Revel controller that performs CRUD operations against
a MySQL database.
Caveats:
Use RESTful Paths.
Use JSON request bodies and responses.
Use Revel Validation.
This setup should reflect a backend that you would use with a Single
Page Application framework like AngularJS.
Synopsis:
Revel, GORP, and MySQLBuilding a classic 3-tier web application controller in
Golang.
Richard Clayton
7/25/2019 Revel Gorp and Mysql
2/15
I'm building a simple auction management system for the wife. One of
the model items I have to deal with is a BidItemBidItem , which represents
something you can purchase during the bidding process. I'll
demonstrate how to build the full controller and database bindings for
this one model item.
We will implement the controller with the following steps:
1. Define REST Routes.
2. Define Model.
3. Implement Validation Logic.
4. Set up the GorpControllerGorpController .
5. Register Configuration.
6. Set up the Database.
7. Extend the GorpControllerGorpController .
8. Handle JSON Request Body (for Creation and Updates).
9. Implement Controller CRUD functionality.
What I will not go into is security and proper response handling. Also,
I'm going to be as succinct as possible so I don't waste your time.
1. Define REST Routes.
In /conf/routes/conf/routes , add the following lines:
GET /item/:id BidItemCtrl.GetPOST /item BidItemCtrl.AddPUT /item/:id BidItemCtrl.UpdateDELETE /item/:id BidItemCtrl.DeleteGET /items BidItemCtrl.List
The first two columns are obvious (HTTP Verb and Route). The :id:id is
an example of placeholder syntax. Revel will provides three ways to
get this value, including using the value as a function argument. The
BidItemCtrl.BidItemCtrl. refers to the controller and method that will
handle the route.
7/25/2019 Revel Gorp and Mysql
3/15
2. Define Model.
I've created a folder called modelsmodels in /app/app . This is where
I define my models. This is my model in file
/app/models/bid-item.go/app/models/bid-item.go :
packagemodels
typeBidItem struct{Id int64 `db:"id" json:"id"`
Name string `db:"name" json:"name"` Category string `db:"category"json:"category"` EstimatedValue float32 `db:"est_value"json:"est_value"` StartBid float32 `db:"start_bid"
json:"start_bid"` BidIncrement float32 `db:"bid_incr"json:"bid_incr"` InstantBuy float32 `db:"inst_buy"json:"inst_buy"`}
Note the meta informationafter each property on the structure. The
db:""db:"" is the desired column-name in the database and thejson:""json:"" is the JSON property name used in
serialization/deserialization. These values are not mandatory, but if
you do not specify them, both GORP and the encoding/jsonencoding/json package
will use the property name. So you will be stuck with columns and
JSON properties in Pascal casing.
3. Implement Validation Logic.
For model validation, we could implement our own custom logic, but
Revel already has a pretty good validation facility (so we'll just use
that). Of course, this means we are going to trade off having a clean
model (unaware of other frameworks) for development expedience.
import("github.com/revel/revel""regexp"
)
7/25/2019 Revel Gorp and Mysql
4/15
// ...
func(b *BidItem) Validate(v *revel.Validation) {
v.Check(b.Name,revel.ValidRequired(),revel.ValidMaxSize(25))
v.Check(b.Category,revel.ValidRequired(),
revel.ValidMatch(regexp.MustCompile(
"^(travel|leasure|sports|entertainment)$")))
v.Check(b.EstimatedValue,revel.ValidRequired())
v.Check(b.StartBid,revel.ValidRequired())
v.Check(b.BidIncrement,revel.ValidRequired())
}
This implements some of the validation logic by constructing the
ruleset for model. It's certainly not complete and leaves much to be
desired.
4. Set up the GorpControllerGorpController .
If you would like transactions for your database, but don't want to
have to manually set them up in your controller, I would recommend
following this step. Keep in mind, all the code I show you after this will
utilize this functionality.
If you go spelunkingin the Revel sample projects, you will find this
little gem in the Bookingsproject. This is the GorpControllerGorpController , which
is a simple extension to the revel.Controllerrevel.Controller that defines some boiler
plate around wrapping controller methods in database transactions.
I've moved out the database creation code into a separate file (which
we will come back to in step 6), so the file should now look like this:
packagecontrollers
import("github.com/coopernurse/gorp"
7/25/2019 Revel Gorp and Mysql
5/15
"database/sql""github.com/revel/revel"
)
var(Dbm *gorp.DbMap
)
typeGorpController struct{*revel.Controller
Txn *gorp.Transaction}
func(c *GorpController) Begin() revel.Result {txn, err :=Dbm.Begin()iferr != nil{
panic(err)}c.Txn = txnreturn nil
}
func(c *GorpController) Commit() revel.Result {ifc.Txn == nil{
return nil
}iferr :=c.Txn.Commit(); err != nil &&err !=
sql.ErrTxDone {panic(err)
}c.Txn = nilreturn nil
}
func(c *GorpController) Rollback() revel.Result {ifc.Txn == nil{
return nil
}iferr :=c.Txn.Rollback(); err != nil &&err !=
sql.ErrTxDone {panic(err)
}c.Txn = nilreturn nil
}
Copy this code into a new file under /app/controllers/app/controllers . I
call the file gorp.gogorp.go .
Two important variables to note are the var ( Dbm *gorp.DbMap )var ( Dbm *gorp.DbMap ) and
the GorpControllerGorpController 's property Txn *gorp.TransactionTxn *gorp.Transaction . We will use
the DbmDbm variable to perform database creation and the TxnTxn variable
to execute queries and commands against MySQL in the controller.
7/25/2019 Revel Gorp and Mysql
6/15
Now in order for us to get the magically wrappedtransactional
functionality with our controller actions, we need to register these
functions with Revel's AOP mechanism.
Create a file called init.goinit.go in /app/controllers/app/controllers . In this
file, we will define an init()init() function which will register the handlers:
packagecontrollers
import "github.com/revel/revel"
funcinit(){revel.InterceptMethod((*GorpController).Begin,
revel.BEFORE)revel.InterceptMethod((*GorpController).Commit,
revel.AFTER)
revel.InterceptMethod((*GorpController).Rollback,revel.FINALLY)}
Now we have transaction support for controller actions.
5. Register Configuration.
We are about ready to start setting up the database. Instead of
throwing in some hard coded values for a connection string, we would
prefer to pull this information from configuration. This is how you do
that.
Open /conf/app.conf/conf/app.conf and add the following lines. Keep in
mind that there are sections for multiple deployment environments.
For now, we're going to throw our configuration values in the [dev][dev]
section:
[dev]
db.user = auctioneerdb.password = passworddb.host = 192.168.24.42
db.port = 3306db.name = auction
7/25/2019 Revel Gorp and Mysql
7/15
Now these values will be made available to use at runtime via the Revel
API: revel.Config.String(paramName)revel.Config.String(paramName) .
6. Setup the Database.
Next, we need to set up the database. This involves instantiating a
connection to the database and creating the tables for our model if the
table does not already exist.
We are going to go back to our /app/controllers/init.go/app/controllers/init.go
file and add the logic to create a database connection, as well as, the
table for our BidItemBidItem model if it does not exist.
First, I'm going to add two helper functions. The first is a more generic
way for us to pull out configuration values from Revel (including
providing a default value). The second is a helper function for building
the MySQL connection string.
funcgetParamString(param string, defaultValue string)string{
p, found :=revel.Config.String(param)if!found {
ifdefaultValue == ""{revel.ERROR.Fatal("Cound not find parameter:
" +param)} else{
returndefaultValue}
}returnp
}
funcgetConnectionString() string{host :=getParamString("db.host", "")port :=getParamString("db.port", "3306")user :=getParamString("db.user", "")pass :=getParamString("db.password", "")dbname :=getParamString("db.name", "auction")protocol :=getParamString("db.protocol", "tcp")dbargs :=getParamString("dbargs", " ")
ifstrings.Trim(dbargs, " ") != ""{
dbargs = "?" +dbargs} else{
dbargs = "" }
returnfmt.Sprintf("%s:%s@%s([%s]:%s)/%s%s",user, pass, protocol, host, port, dbname, dbargs)
}
7/25/2019 Revel Gorp and Mysql
8/15
With these two functions, we can construct a connection to MySQL.
Let's create a function that will encapsulate initializing the database
connection and creating the initial databases:
varInitDb func() = func(){
connectionString :=getConnectionString()ifdb, err :=sql.Open("mysql", connectionString);
err != nil{revel.ERROR.Fatal(err)
} else{Dbm = &gorp.DbMap{
Db: db,Dialect: gorp.MySQLDialect{"InnoDB", "UTF8"}}
}// Defines the table for use by GORP
// This is a function we will create soon.
defineBidItemTable(Dbm)iferr :=Dbm.CreateTablesIfNotExists(); err != nil{
revel.ERROR.Fatal(err)}
}
One thing to note in the code above is that we are setting the DbmDbm
variable we defined in the GorpControllerGorpController file gorp.gogorp.go . Using the
connection, DbmDbm , we will define our BidItemBidItem schema and create thetable if it does not exist. We have not yet written the
defineBidItemTable(Dbm)defineBidItemTable(Dbm) function; I will show you that soon.
Before we move on, we need to talk about imports. All of the helper
functions and code in the init.goinit.go file will require the following
libraries:
import("github.com/revel/revel""github.com/coopernurse/gorp""database/sql"
_ "github.com/go-sql-driver/mysql""fmt""strings"
)
Of special note is the import and non-useof the mysqlmysql library:
_ "github.com/go-sql-driver/mysql"_ "github.com/go-sql-driver/mysql" . If you do not include this
import statement, your project will break. The reason is that GORP
7/25/2019 Revel Gorp and Mysql
9/15
relies on the database/sqldatabase/sql package, which is only a set of interfaces.
The mysqlmysql package implements those interfaces, but you will not see
any direct reference to the library in the code.
Now it's time to implement the defineBidItemTable(Dbm)defineBidItemTable(Dbm) function.
funcdefineBidItemTable(dbm *gorp.DbMap){// set "id" as primary key and autoincrement
t :=dbm.AddTable(models.BidItem{}).SetKeys(true,"id")
// e.g. VARCHAR(25)
t.ColMap("name").SetMaxSize(25)}
Notice how I use the term defineand not create. This is because the
database is not actually created until the call to
Dbm.CreateTablesIfNotExists()Dbm.CreateTablesIfNotExists() .
Finally, the InitDb()InitDb() function needs to be executed when the
application starts. We register the function similar to the way we did
the AOP functions for the GorpControllerGorpController in the init()init() function:
funcinit(){revel.OnAppStart(InitDb)revel.InterceptMethod((*GorpController).Begin,
revel.BEFORE)revel.InterceptMethod((*GorpController).Commit,
revel.AFTER)revel.InterceptMethod((*GorpController).Rollback,
revel.FINALLY)}
When we are done with these steps, you will be able to start the
application and verify the table creation in MySQL. If you are curious
as to how GORP defines the table, here's a screen capture of
mysql > describe BidItem;mysql > describe BidItem; :
7/25/2019 Revel Gorp and Mysql
10/15
7. Extend the GorpControllerGorpController .
We are ready to start working on the controller. First we need to createa new file for the controller: /app/controllers/bid-item.go/app/controllers/bid-item.go
. Using the GorpControllerGorpController as the prototype, let's define our new
BidItemCtrlBidItemCtrl :
packagecontrollers
import("auction/app/models"
"github.com/revel/revel""encoding/json"
)
typeBidItemCtrl struct{GorpController
}
We will use those imports soon.
8. Handle JSON Request Body.
This is one of those things that is not well documented in Revel, in part
because, Revel doesn't consider it a part of the framework. If you want
to be able to transform JSON requests into your own modelrepresentations you will need to define that functionality.
We are going to add a simple method to the controller that will parse
the request body and return a BidItemBidItem and ErrorError tuple:
7/25/2019 Revel Gorp and Mysql
11/15
func(c BidItemCtrl) parseBidItem() (models.BidItem,
error) {biditem :=models.BidItem{}err :=json.NewDecoder(c.Request.Body).Decode
(&biditem)returnbiditem, err
}
We now have a way of getting a BidItemBidItem from the Request.BodyRequest.Body .
You will obviously have to know the context in which to use this
method, but that should be obvious since you do that anyway in most
web frameworks (albeit it's a little it's a little less elegant in Revel).
9. Implement Controller CRUD functionality.
Let's finish up the rest of the controller!
Add BidItem.
Inserting the record is pretty easy with GORP:
c.Txn.Insert()c.Txn.Insert() .
func(c BidItemCtrl) Add() revel.Result {ifbiditem, err :=c.parseBidItem(); err != nil{
returnc.RenderText("Unable to parse the BidItemfrom JSON.")
} else{// Validate the model
biditem.Validate(c.Validation)ifc.Validation.HasErrors() {
// Do something better here!returnc.RenderText("You have error in your
BidItem.")} else{
iferr :=c.Txn.Insert(&biditem); err != nil{returnc.RenderText(
"Error inserting record intodatabase!")
} else{returnc.RenderJson(biditem)
}
}}
}
7/25/2019 Revel Gorp and Mysql
12/15
Get BidItem.
func(c BidItemCtrl) Get(id int64) revel.Result {
biditem := new(models.BidItem)err :=c.Txn.SelectOne(biditem,
`SELECT * FROM BidItem WHERE id = ?`, id)iferr != nil{
returnc.RenderText("Error. Item probably doesn'texist.")
}returnc.RenderJson(biditem)
}
List BidItems (with paging).
For listing BidItemBidItem s, were going to need to implement some simplepaging. Since we have an auto incremented, indexed bigint(20)bigint(20) for
an identifier, we'll keep it simple and use the last id as the start field
and a limit to indicate the amount of records we want. For this, we will
accept two query parameters: lidlid (last id) and limitlimit (number of rows
to return).
Since we don't want a SQL injection attack, we're going to parse these
query options as integers. Here are a couple of functions to help the
parsing (keeping it in /app/controllers/common.go/app/controllers/common.go ):
packagecontrollers
import(
"strconv")
funcparseUintOrDefault(intStr string, _default uint64)uint64{
ifvalue, err :=strconv.ParseUint(intStr, 0, 64);err != nil{
return_default
} else{returnvalue
}}
funcparseIntOrDefault(intStr string, _default int64)int64{
ifvalue, err :=strconv.ParseInt(intStr, 0, 64);
err != nil{return_default
7/25/2019 Revel Gorp and Mysql
13/15
} else{
returnvalue}
}
The controller function is pretty simple:
func(c BidItemCtrl) List() revel.Result {lastId :=parseIntOrDefault(c.Params.Get("lid"), -1)limit :=parseUintOrDefault(c.Params.Get("limit"),
uint64(25))biditems, err :=c.Txn.Select(models.BidItem{},
`SELECT * FROM BidItem WHERE Id > ? LIMIT ?`,lastId, limit)
iferr != nil{
returnc.RenderText("Error trying to get records from DB.")
}returnc.RenderJson(biditems)
}
Update BidItem.
func(c BidItemCtrl) Update(id int64) revel.Result {
biditem, err :=c.parseBidItem()iferr != nil{returnc.RenderText("Unable to parse the BidItem
from JSON.")}// Ensure the Id is set.
biditem.Id = idsuccess, err :=c.Txn.Update(&biditem)iferr != nil ||success == 0{
returnc.RenderText("Unable to update bid item.")}
returnc.RenderText("Updated %v", id)}
GORP takes the convention of returning a tuple of int64int64 and ErrorError
on Updates and Deletes. The int64int64 is either a 00 or 11 value
signifying whether the action was successful or not.
Delete BidItem.
And the last action for the controller.
7/25/2019 Revel Gorp and Mysql
14/15
func(c BidItemCtrl) Delete(id int64) revel.Result {
success, err :=c.Txn.Delete(&models.BidItem{Id: id})
iferr != nil ||success == 0{
returnc.RenderText("Failed to remove BidItem")
}
returnc.RenderText("Deleted %v", id)
}
Conclusion.
While a little long, I think that was probably the most complete
walkthrough/example you'll find on implementing a Controller in
Revel with GORP and MySQL. I hope this helps you in your effort of
learning Go, GORP or Revel.
References.
http://revel.github.io/manual/index.html
https://github.com/coopernurse/gorp
https://github.com/revel/revel/tree/master/samples/booking
http://nathanleclaire.com/blog/2013/11/04/want-to-work-
with-databases-in-golang-lets-try-some-gorp/
Plus many other StackOverflow articles and blog posts...
Richard Clayton Unrepentant Thoughts on Software and Management.
July 18, 2014 u 6 Comments
Read Next: Quick MySQL Box Using Vagrantand Ansible
7/25/2019 Revel Gorp and Mysql
15/15
Blog Home Archive RSS
You can also find me on Twitter and GitHub.
2015 Richard Clayton