Test driven development in Go
Nulab
July 29, 2016
You may have read our recent announcement that we are currently rewriting Cacoo Editor UI to HTML 5, but that’s not all we have in the works; we have also decided to migrate our current API backend written in Java to Go.
Switching from Java to Go is surprisingly easy because the languages have a lot in common. However, without the wealth of online resources typically available for Java, we’ve had to dive into more Go documentation, source code, and open projects to answer some of our questions. We decided to share some things we’ve learned.
This post will focus on how to implement a simple REST API for managing tasks using Test Driven Development (TDD). We will create an API that provides two services: saveTask and getPendingTasks. Then, we will implement our server and API services.
Test Driven Development
TDD is a simple concept: you define a test for a function you haven’t implemented yet and then write the minimal code to achieve the expected behavior. We’ll create examples for these first few steps.
First, create two files in the package store: task.go and task_test.go. Then, create your first test.
package store
import (
"reflect"
"testing"
)
func TestGetPendingTasks(t *testing.T) {
t.Log("GetPendingTasks")
ds := Datastore{
tasks: []Task{
{1, "Do housework", true},
{2, "Buy milk", false},
},
}
want := []Task{ds.tasks[1]}
t.Log("should return the tasks which need to be completed")
if got := ds.GetPendingTasks(); !reflect.DeepEqual(got, want) {
t.Errorf("Got %v wanted %v", got, want)
}
}
The test fails because task and datastore are undefined. We must define our task model, datastore, and function, and our datastore must keep a list of all the tasks. The GetPendingTasks function should return the list of the tasks which are not completed.
package store
// Task job to be done or completed
type Task struct {
ID int // identifier of the task
Title string // Title of the task
Done bool // If task is completed or not
}
// Datastore manages a list of tasks stored in memory
type Datastore struct {
tasks []Task
}
// GetPendingTasks returns all the tasks which need to be done
func (ds *Datastore) GetPendingTasks() []Task {
var pendingTasks []Task
for _, task := range ds.tasks {
if !task.Done {
pendingTasks = append(pendingTasks, task)
}
}
return pendingTasks
}
Run the test again, and this time, it passes.
Next, let’s save a task. The task can be a new task or an existing task. Let’s begin with a test for saving a new task.
func TestSaveNewTask(t *testing.T) {
t.Log("SaveTask")
ds := Datastore{}
task := Task{Title: "Buy milk"}
want := []Task{
{1, "Buy milk", false},
}
t.Log("should save the new task in the store")
ds.SaveTask(task)
if !reflect.DeepEqual(ds.tasks, want) {
t.Errorf("=> Got %v wanted %v", ds.tasks, want)
}
}
The test fails. The SaveTask function must be implemented. Our Datastore will attribute a new ID to each new task, so the last given ID should be stored in a field.
// Datastore manages a list of task in memory
type Datastore struct {
tasks []Task
lastID int // lastID is incremented for each new stored task
}
// SaveTask should store the task in the datastore
func (ds *Datastore) SaveTask(task Task) {
ds.lastID++
task.ID = ds.lastID
ds.tasks = append(ds.tasks, task)
}
Now, all the tests pass.
If a task already exists, it should be updated. Let’s define a test for this feature.
func TestSaveAndUpdateExistingTask(t *testing.T) {
t.Log("SaveTask")
ds := Datastore{
tasks: []Task{
{1, "Buy milk", false},
},
}
want := []Task{
{1, "Buy milk", true},
}
task := Task{1, "Buy milk", true}
t.Log("should update the existing task in the store")
ds.SaveTask(task)
if !reflect.DeepEqual(ds.tasks, want) {
t.Errorf("=> Got %v wanted %v", ds.tasks, want)
}
}
This feature is not yet implemented, so the test fails. We therefore need to modify the SaveTask function, and check if the task to be saved is a new task or an existing task. If the task is already present in our datastore, it will be replaced by its new version.
// SaveTask should store the task in the datastore if the task
// does not exist else update it
func (ds *Datastore) SaveTask(task Task) {
if task.ID == 0 {
ds.lastID++
task.ID = ds.lastID
ds.tasks = append(ds.tasks, task)
return
}
for i, t := range ds.tasks {
if t.ID == task.ID {
ds.tasks[i] = task
return
}
}
}
This time, all tests pass.
Table-Driven Test
You might notice that the tests for saving a new task and updating an existing task are very similar. Both are testing the SaveTask function. How can we avoid code duplication?
We could create a helper function, but another solution exists in Go: refactoring our tests to a Table-Driven Test.
First, create table entries that contain all the data needed for our test cases.
var saveTaskTests = []struct {
name string
ds *Datastore
task Task
want []Task
}{
{
name: "should save the new task in the datastore",
ds: &Datastore{},
task: Task{Title: "Buy milk"},
want: []Task{
{1, "Buy milk", false},
},
},
{
name: "should update the existing task in the store",
ds: &Datastore{
tasks: []Task{
{1, "Buy milk", false},
},
},
task: Task{1, "Buy milk", true},
want: []Task{
{1, "Buy milk", true},
},
},
}
Then, add a new test. The test should iterate through all table entries and perform the test cases.
func TestSaveTask(t *testing.T) {
t.Log("SaveTask")
for _, testcase := range saveTaskTests {
t.Log(testcase.name)
testcase.ds.SaveTask(testcase.task)
if !reflect.DeepEqual(testcase.ds.tasks, testcase.want) {
t.Errorf("=> Got %v wanted %v", testcase.ds.tasks, testcase.want)
}
}
}
Once everything looks good, both TestSaveNewTask and TestSaveAndUpdateTask can be removed. We don’t need them anymore.
One more test case still needs to be tested. If the task ID doesn’t exist an error will be returned. Let’s create a new entry for our table and add an error field to our anonymous struct.
var saveTaskTests = []struct {
name string
ds *Datastore
task Task
want []Task
err error
}{
{
name: "should save the new task in the datastore",
ds: &Datastore{},
task: Task{Title: "Buy milk"},
want: []Task{
{1, "Buy milk", false},
},
},
{
name: "should update the existing task in the store",
ds: &Datastore{
tasks: []Task{
{1, "Buy milk", false},
},
},
task: Task{1, "Buy milk", true},
want: []Task{
{1, "Buy milk", true},
},
},
{
name: "should return an error when task ID does not exist",
ds: &Datastore{},
task: Task{1, "Buy milk", true},
err: ErrTaskNotFound,
},
}
The test fails. The SaveTask function should have returned an error when the task could not be found in the datastore.
import "errors"
// ErrTaskNotFound is returned when a Task ID is not found
var ErrTaskNotFound = errors.New("Task was not found")
// SaveTask should store the task in the datastore if the task
// is new else update it. A Task Not Found error is returned
// when the task ID does not exist
func (ds *Datastore) SaveTask(task Task) error {
if task.ID == 0 {
ds.lastID++
task.ID = ds.lastID
ds.tasks = append(ds.tasks, task)
return nil
}
for i, t := range ds.tasks {
if t.ID == task.ID {
ds.tasks[i] = task
return nil
}
}
return ErrTaskNotFound
}
This time, it works! All tests have passed.
Create a Server
Go comes with a built-in HTTP server and a simple HTTP request multiplexer, so we just need to define a route, a handler, and start the server.
func main() {
http.HandleFunc("/", func(w http.ResponseWriter, req *http.Request) { // HandleFunc from http
w.Write([]byte("Hello World!"))
})
http.ListenAndServe(":8080", nil) // Start the server
}
HTTP Request Handlers
We will develop three HTTP request handlers:
- Get pending tasks,
- Add a task, and
- Update a task.
Get Pending tasks
First, we need to create a test for getting the pending tasks from our datastore and returning them as a JSON response. As we write our test, we find out that we need to access our datastore and insert our test tasks. Let’s define a global variable for our datastore:
var ds = &store.Datastore{}
The tasks slice of our datastore is private. We should change its visibility to public so our testing tasks can be directly inserted:
// Datastore manages a list of task in memory
type Datastore struct {
Tasks []Task
lastID int // lastID is incremented for each new stored task
}
Finally, our test can be completed:
func TestGetPendingTasks(t *testing.T) {
t.Log("GetPendingTasks")
t.Log("should return pending tasks as JSON")
rec := httptest.NewRecorder()
req, _ := http.NewRequest(http.MethodGet, "/tasks/pending", nil)
ds = &store.Datastore{
Tasks: []store.Task{
{1, "Do housework", false},
{2, "Buy milk", false},
},
}
GetPendingTasks(rec, req)
if rec.Code != http.StatusOK {
t.Errorf("KO => Got %d wanted %d", rec.Code, http.StatusOK)
}
want := "[{\"id\":1,\"title\":\"Do housework\",\"done\":false},{\"id\":2,\"title\":\"Buy milk\",\"done\":false}]"
if got := rec.Body.String(); got != want {
t.Errorf("KO => Got %s wanted %s", got, want)
}
}
Our handler retrieves the pending tasks from our datastore, encodes, and returns them in the HTTP response body.
// GetPendingTasks returns pending tasks as a JSON response
func GetPendingTasks(w http.ResponseWriter, r *http.Request) {
t := ds.GetPendingTasks()
j, _ := json.Marshal(t)
w.Header().Set("Content-Type", "application/json")
w.Write(j)
}
Run the test, and you’ll see it fails. Our field names are not in lowercase. We didn’t add field tags to our Task struct, so the marshaller used the same letter case as our struct. Let’s fix it.
// Task job to be done or completed
type Task struct {
ID int `json:"id"` // identifier of the task
Title string `json:"title"` // Title of the task
Done bool `json:"done"` // If task is completed or not
}
Now, the test passes.
Interface
Our Datastore struct should not have been modified for the purpose of testing only. The task slice should remain private and not be accessed directly. Our handler should be tested by isolating it from our data store implementation. At first, let’s revert our Datastore changes.
// Datastore manages a list of task in memory
type Datastore struct {
tasks []Task
lastID int // lastID is incremented for each new stored task
}
A better solution is to create an interface.
// Store defines the datastore services
type Store interface {
GetPendingTasks() []store.Task
}
And force our global variable to implement our Store interface.
var ds Store = &store.Datastore{}
Now our GetPendingTasks can be mocked by creating a new struct that implements our interface.
type mockedStore struct{}
func (ms *mockedStore) GetPendingTasks() []store.Task {
return []store.Task{
{1, "Do housework", false},
{2, "Buy milk", false},
}
}
Finally, our test should be modified and should use our newly created mock.
func TestGetPendingTasks(t *testing.T) {
t.Log("GetPendingTasks")
t.Log("should return pending tasks as JSON")
rec := httptest.NewRecorder()
req, _ := http.NewRequest(http.MethodGet, "/tasks/pending", nil)
// The datastore is restored at the end of the test
defer func() { ds = &store.Datastore{} }()
ds = &mockedStore{}
GetPendingTasks(rec, req)
if rec.Code != http.StatusOK {
t.Errorf("KO => Got %d wanted %d", rec.Code, http.StatusOK)
}
want := "[{\"id\":1,\"title\":\"Do housework\",\"done\":false},{\"id\":2,\"title\":\"Buy milk\",\"done\":false}]"
if got := rec.Body.String(); got != want {
t.Errorf("KO => Got %s wanted %s", got, want)
}
}
Nice! The tests still pass.
Add a task
Now we implement our second handler for adding a task. If the task is added, our REST service should return the HTTP status 201 Created:
func TestAddTask(t *testing.T) {
t.Log("AddTask")
t.Log("should add new task from JSON")
rec := httptest.NewRecorder()
req, _ := http.NewRequest(http.MethodPost, "/tasks", bytes.NewBuffer([]byte(`{"Title":"Buy bread for breakfast."}`)))
defer func() { ds = &store.Datastore{} }()
ds = &mockedStore{}
AddTask(rec, req)
wantCode := http.StatusCreated
if rec.Code != wantCode {
t.Errorf("KO => Got %d wanted %d", rec.Code, wantCode)
}
}
And then create our handler. The JSON in the request body is decoded and the status 201 Created is written to the response header:
// AddTask handles POST requests on /tasks.
// Return 201 if the task could be created
func AddTask(w http.ResponseWriter, r *http.Request) {
var t store.Task
json.NewDecoder(r.Body).Decode(&t)
w.WriteHeader(http.StatusCreated)
}
All tests pass. Some error checks need to be added now. If the JSON cannot be decoded, the status 400 Bad Request should be answered. Previously, we refactored our saveTask test to a Table-Driven Test to avoid code duplication. Let’s do same before implementing our new test case.
var addTaskTests = []struct {
name string
body []byte
want int
}{
{
name: "should add new task from JSON",
body: []byte(`{"Title":"Buy bread for breakfast."}`),
want: http.StatusCreated,
},
}
func TestAddTask(t *testing.T) {
t.Log("AddTask")
for _, testcase := range addTaskTests {
t.Log(testcase.name)
rec := httptest.NewRecorder()
req, _ := http.NewRequest(http.MethodPost, "/tasks", bytes.NewBuffer(testcase.body))
defer func() { ds = &store.Datastore{} }()
ds = &mockedStore{}
AddTask(rec, req)
if rec.Code != testcase.want {
t.Errorf("KO => Got %d wanted %d", rec.Code, testcase.want)
}
}
}
We run the tests again to be sure they still pass, and they do. Next, we add our test for handling decoding errors.
var addTaskTests = []struct {
name string
body []byte
want int
}{
{
name: "should add new task from JSON",
body: []byte(`{"Title":"Buy bread for breakfast."}`),
want: http.StatusCreated,
},
{
name: "should return bad argument when JSON could not be handled",
body: []byte(""),
want: http.StatusBadRequest,
},
}
And add our error check to our implementation.
// AddTask handles requests for adding a new task.
// Return 201 if the task could be created
// Return 400 when JSON could not be decoded into a task
func AddTask(w http.ResponseWriter, r *http.Request) {
var t store.Task
if err := json.NewDecoder(r.Body).Decode(&t); err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
w.WriteHeader(http.StatusCreated)
}
Another check should be added when an error is returned from our datastore. Our service should return the status 400 Bad Request. To be able to test this case, our mock should implement the Datastore SaveTask function. At first, our SaveTask function signature is added to our Store interface.
// Store defines the datastore services
type Store interface {
GetPendingTasks() []store.Task
SaveTask(task store.Task) error
}
Then we implement it in our mock struct. The SaveTask should return an error for our new test case only and not for the previous ones. We add a function field to our mock struct. The test will call this function only if the function was implemented in our mock.
type mockedStore struct {
SaveTaskFunc func(task store.Task) error
}
func (ms *mockedStore) SaveTask(task store.Task) error {
if ms.SaveTaskFunc != nil {
return ms.SaveTaskFunc(task)
}
return nil
}
The saveFunc is added to our anonymous struct table and implemented in our datastore error test case.
var addTaskTests = []struct {
name string
saveFunc func(task store.Task) error
body []byte
want int
}{
{
name: "should add new task from JSON",
body: []byte(`{"Title":"Buy bread for breakfast."}`),
want: http.StatusCreated,
},
{
name: "should response bad argument when JSON could not be handled",
body: []byte(""),
want: http.StatusBadRequest,
},
{
name: "should response bad argument when datastore returns an error",
saveFunc: func(task store.Task) error {
return errors.New("datastore error")
},
body: []byte(`{"Title":"Buy bread for breakfast."}`),
want: http.StatusBadRequest,
},
}
Our mock should use the saveFunc from the test case.
func TestAddTask(t *testing.T) {
t.Log("AddTask")
for _, testcase := range addTaskTests {
t.Log(testcase.name)
rec := httptest.NewRecorder()
req, _ := http.NewRequest(http.MethodPost, "/tasks", bytes.NewBuffer(testcase.body))
defer func() { ds = &store.Datastore{} }()
ds = &mockedStore{
SaveTaskFunc: testcase.saveFunc,
}
AddTask(rec, req)
if rec.Code != testcase.want {
t.Errorf("KO => Got %d wanted %d", rec.Code, testcase.want)
}
}
}
Modify the handler and return a bad request answer when an error is returned from the data store.
// AddTask handles requests for adding a new task.
// Return 201 if the task could be created
// Return 400 when JSON could not be decoded into a task or
// datastore returned an error
func AddTask(w http.ResponseWriter, r *http.Request) {
var t store.Task
if err := json.NewDecoder(r.Body).Decode(&t); err != nil {
http.Error(w, "JSON malformed", http.StatusBadRequest)
return
}
if err := ds.SaveTask(t); err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
w.WriteHeader(http.StatusCreated)
}
Another case needs to be verified: If our the title is empty, a 400 Bad Request should be returned.
var addTaskTests = []struct {
name string
saveFunc func(task store.Task) error
body []byte
want int
}{
{
name: "should add new task from JSON",
body: []byte(`{"Title":"Buy bread for breakfast."}`),
want: http.StatusCreated,
},
{
name: "should response bad argument when JSON could not be handled",
body: []byte(""),
want: http.StatusBadRequest,
},
{
name: "should response bad argument when datastore returns an error",
saveFunc: func(task store.Task) error {
return errors.New("datastore error")
},
body: []byte(`{"Title":"Buy bread for breakfast."}`),
want: http.StatusBadRequest,
},
{
name: "should response bad argument when task title is emtpy",
body: []byte(`{"Title":""}`),
want: http.StatusBadRequest,
},
}
Finally, we add it to our implementation.
// AddTask handles requests for adding a new task.
// Return 201 if the task could be created
// Return 400 when JSON could not be decoded into a task or
// datastore returned an error or task title is empty
func AddTask(w http.ResponseWriter, r *http.Request) {
var t store.Task
if err := json.NewDecoder(r.Body).Decode(&t); err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
if t.Title == "" {
http.Error(w, "Title is missing", http.StatusBadRequest)
return
}
if err := ds.SaveTask(t); err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
w.WriteHeader(http.StatusCreated)
}
Update a task
This handler is quite similar to our AddTask handler, so the code snippets will be provided without explanation.
var updateTaskTests = []struct {
name string
saveFunc func(task store.Task) error
body []byte
want int
}{
{
name: "should response with a status 200 OK when the task was updated",
body: []byte(`{"ID":1, "Title":"Buy bread for breakfast.", "Done":true }`),
want: http.StatusOK,
},
{
name: "should response with a status 400 Bad Request when JSON body could not be handle",
body: []byte(""),
want: http.StatusBadRequest,
},
{
name: "should response with a status 400 Bad Request when the datastore returned an error",
saveFunc: func(task store.Task) error {
return errors.New("datastore error")
},
body: []byte(`{"ID":1, "Title":"Buy bread for breakfast.", "Done":true }`),
want: http.StatusBadRequest,
},
{
name: "should response with a status 400 Bad Request when task title is emtpy",
body: []byte(`{"Title":""}`),
want: http.StatusBadRequest,
},
}
func TestUpdateTask(t *testing.T) {
t.Log("UpdateTask")
for _, testcase := range updateTaskTests {
t.Logf(testcase.name)
rec := httptest.NewRecorder()
req, _ := http.NewRequest(http.MethodPost, "/tasks/1", bytes.NewBuffer(testcase.body))
defer func() { ds = &store.Datastore{} }()
ds = &mockedStore{
SaveTaskFunc: testcase.saveFunc,
}
UpdateTask(rec, req)
if rec.Code != testcase.want {
t.Errorf("KO => Got %d wanted %d", rec.Code, testcase.want)
}
}
}
// UpdateTask handles requests for updating an existing task.
// Return 200 if the task could be modified
// Return 400 when JSON could not be decoded into a task or
// datastore returned an error or task title is empty
func UpdateTask(w http.ResponseWriter, r *http.Request) {
var t store.Task
if err := json.NewDecoder(r.Body).Decode(&t); err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
if err:= validateTask(t); err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
if err := ds.SaveTask(t); err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
w.WriteHeader(http.StatusOK)
}
A small refactor was done. The task title check was extracted to a function, so it can be used in our AddTask and UpdateTask handlers.
func validateTask(t store.Task) error {
if t.Title == "" {
return errors.New("Title is missing")
}
return nil
}
Router
Finally, let’s define our routes:
- /tasks/pending GET Return the pending tasks
- /tasks POST Add a new task
- /tasks/{id} PUT Update an existing task
Unfortunately, the default Server Mux cannot handle dynamic routes such as /tasks/{id}, so we’ll need to implement our own router. First, we create a Table-Driven test with different test cases for our routes.
package router
import (
"testing"
"net/http/httptest"
"net/http"
)
var routeTests = []struct {
name string
routMethod string
routPatn string
reqMethod string
reqURL string
want int
}{
{
name: "should response with a status 200 OK when a route and method match",
routPatn: "/tasks",
routMethod:http.MethodGet,
reqURL: "/tasks",
reqMethod:http.MethodGet,
want:http.StatusOK,
},
{
name: "should response with a status 404 Not Found when HTTP method is different",
routPatn: "/tasks",
routMethod:http.MethodGet,
reqURL: "/tasks",
reqMethod:http.MethodPost,
want: http.StatusNotFound,
},
{
name: "should response with a status 200 OK when a route match regex and method",
routPatn: `/tasks/\d`,
routMethod:http.MethodGet,
reqURL: "/tasks/1",
reqMethod:http.MethodGet,
want: http.StatusOK,
},
{
name: "should response with a status 404 Not Found when route could not be found",
routPatn: `/tasks\d`,
routMethod:http.MethodPost,
reqURL: "/tasks/a",
reqMethod:http.MethodPost,
want: http.StatusNotFound,
},
}
func TestRoute(t *testing.T) {
t.Log("Router")
for _, testcase := range routeTests {
t.Logf(testcase.name)
rec := httptest.NewRecorder()
req, _ := http.NewRequest(testcase.reqMethod, testcase.reqURL, nil)
r := Router{}
r.HandleFunc(testcase.routPatn, testcase.routMethod, func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusOK)
})
r.ServeHTTP(rec, req)
if rec.Code != testcase.want {
t.Errorf("KO => Got %d wanted %d", rec.Code, testcase.want)
}
}
}
Our router contains a slice of routes. Routes are added to the router by passing a regex pattern, a handler, and an HTTP method as parameters to the HandleFunc. When a request is received, the server multiplexer will call the ServeHTTP function. This function loops over the routes slice and checks if a route matches the HTTP method and the regex pattern. If a route is found, the route handler is called else a status 404 Not Found is returned.
package router
import (
"net/http"
"regexp"
)
type Route struct {
Pattern *regexp.Regexp
Handler http.Handler
HTTPMethod string
}
type Router struct {
routes []*Route
}
func (r *Router) HandleFunc(pattern string, httpMethod string, f func(http.ResponseWriter, *http.Request)) {
r.routes = append(r.routes, &Route{
HTTPMethod:httpMethod,
Pattern: regexp.MustCompile(pattern + "$"),
Handler: http.HandlerFunc(f),
})
}
func (rer *Router) ServeHTTP(w http.ResponseWriter, r *http.Request) {
for _, route := range rer.routes {
if route.HTTPMethod == r.Method && route.Pattern.MatchString(r.URL.Path) {
route.Handler.ServeHTTP(w, r)
return
}
}
http.NotFound(w, r)
}
Wrapping up
Finally, we should declare our router, our routes, and start our server in our main function.
func main() {
r := &router.Router{}
r.HandleFunc("/tasks/pending", http.MethodGet, server.GetPendingTasks)
r.HandleFunc("/tasks", http.MethodPost, server.AddTask)
r.HandleFunc(`/tasks/\d`, http.MethodPut, server.UpdateTask)
log.Fatal(http.ListenAndServe(":8080", r))
}
Voilà, the server is started and ready to serve the routes that were defined above. Requests should be sent to localhost:8080.
Final Thoughts
This concludes the development of a simple REST API using Test Driven Development in Go.
Go already includes everything we need to test our functions without using any frameworks. Table-Driven Tests are useful for avoiding code duplication and performing many test cases for a function.
Our router implementation could be more efficient, but it’s good enough for our example. If more complex routing or performance are needed, an advanced router, such as Gorilla Mux or HttpRouter, can be used.
The final source code can be found on GitHub.
The featured image is based on the Go mascot designed by Renee French.
