Parametrising Your BDD Tests In Go

As part of my day-to-day, I keep a collection of end-to-end tests which I use to verify the service I’m responsible for. These tests were written as plain Cucumber tests and are more-or-less what you’d expect from most Behaviour Driven Design tests:

Feature: Sum API

Scenario: Add API should add two numbers together

Given a "GET" request to "https://myservice.com/add"
And the post field "x" is 1
And the post field "y" is 2
When that request is executed
Then the response body should be "{'result':3}"

For a while this worked reasonably well, but recently I found myself needing to run the same set of tests against a different service. You could imagine that simply doing a search-and-replace from myservice.com to adifferentservice.com would not be sustainable.

Fortunately, there’s a way to parametrise these tests so that you can change the hostname without changing the tests themselves.

This technique can be used without adding any steps that imply reading configuration from the environment, thereby keeping the total number of steps down without sacrificing readability. Instead, this technique involves using reflection and Aspect Oriented Programming techniques to “weave” template supports into your step functions, resulting in scenarios that look like this:

Feature: Sum API

Scenario: Add API should add two numbers together

Given a "GET" request to "https://{{.ServiceHost}}/add"
And the post field "x" is 1
And the post field "y" is 2
When that request is executed
Then the response body should be "{'result':3}"

Test Config

For this scenario, I will be focusing on tests written for and executed using godog. This is a Go-based BDD testing framework that works a lot like many of the other Cucumber implementations you’ll find for other languages. It is worth taking a look at if you are interested in writing tests like these.

The first thing to consider is how parameters like ServiceHost will be made available to the tests. After-all, the values need to be stored somewhere.

One method of doing this is to wrap them all in a struct as publicly accessible fields:

type TestConfig {
    ServiceHost string
}

We can read the value of environment variables for each of these fields by using the envconfig package. This can even be used to setup default values so that we don’t loose anything from our existing tests after we parametrise them:

type TestConfig {
    ServiceHost string	`default:"myservice.com" split_words:"service_host"`
}

func ReadTestConfig() (tc TestConfig) {
    err := envconfig.Process("e2e_tests", &tc)
    if err != nil {
        log.Fatal(err.Error())
    }
    return tc
}

Putting this in our InitializeScenario means that we can make this context available to our tests. We’ll see how we can do this a little later. For now, know that we have something to read configuration values from the environment.

Function Builders

First thing to note is that there’s nothing that needs to be done to the step implementations themselves. If you can imagine the following test setup:

func givenARequest(method, url string) error {
    req, err := http.NewRequest(method, url, nil)
    if err != nil {
        return err
    }

    resp, err := http.DefaultClient.Do(req)
    return err
}

func InitializeScenario(ctx *godog.ScenarioContext) {
    ctx.Step(`^a "([^"]*)" request to "([^"]*)"$`, givenARequest)
}

There won’t be any changes made to the givenARequest function itself. Instead we will be “decorating” these step functions so that the arguments pass through to them are treated as templates before the step functions are invoked.

This is relatively easy to do in other languages using techniques like in Aspect Oriented Programming. As far as I can tell, there is no such framework for Go, at least not one that is widely used within the community. However, Go does have a pretty decent reflection API.

One of the functions available to us is the MakeFunc function which allows for the creation of new functions with arbitrary types. This takes a function that accepts, and returns, a slice of reflect.Values corresponding to the arguments and return values.

Using this, we have the ability to wrap functions with modified request parameters and return values without knowing the exact type of the wrapped function beforehand (although you’ll need to use the reflection types to do so). So long as we build our code to handle any arbitrary type, we could use this with any function that we pass it.

To ease into this, the first step would be to build a function wrapper which will simply call the wrapped function without modifying any of the arguments or return values:

func (config TestConfig) decorateFunc(fn interface{}) interface{} {
    fnVal := reflect.ValueOf(fn)
    if fnVal.Kind() != reflect.Func {
        panic("fn must be a function")
    }

    // The 1st argument specifies the signature of the new function.  Since we want
    // it to be the same as the old function, we can simply pass that type as is.
    return reflect.MakeFunc(fnVal.Type(), func(args []reflect.Value) (results []reflect.Value) {
        return fnVal.Call(args)
    }).Interface()
}

Once we have this working, we can start working on our templates.

Rendering Templates

Making the string parameters of the Cucumber tests act as a template means that we’ll need a way to render these templates. How you’d like these templates to work will depend on you and your team. For this example, we will use Go’s built-in template package.

To make integration easier, it’s best to abstract out the template rendering code as a separate function:

func renderTemplate(templateExpr string, data interface{}) (string, error) {
    tmpl, err := template.New("arg").Parse(templateExpr)
    if err != nil {
        return "", err
    }

    result := new(strings.Builder)
    if err := tmpl.Execute(result, data); err != nil {
        return "", err
    }

    return result.String(), nil
}

Calling this function from the TestConfig means that we can use the fields that are configured from the environment available to the tests. Simply passing in the TestConfig to the template renders allows us to do that.

func (config TestConfig) evaluateArgumentTemplate(arg string) (string, error) {
    return renderTemplate(arg, config)
}

We’re now ready to hook up our template rendering method to our function builder. We’re only interested in step parameters that are strings, which we can test for by using the Kind() method on the argument type:

func (config TestConfig) decorateFunc(fn interface{}) interface{} {
    fnVal := reflect.ValueOf(fn)
    if fnVal.Kind() != reflect.Func {
        panic("fn must be a function")
    }

    return reflect.MakeFunc(fnVal.Type(), func(args []reflect.Value) (results []reflect.Value) {
        // Here we make a copy of the new arguments, with the ones corresponding to string types
        // rendered as template.
        newArgs := make([]reflect.Value, len(args))
        for i, a := range args {
            if a.Type().Kind() == reflect.String {
                if templateResult, err := evaluateArgumentTemplate(a.String()); err == nil {
                    newArgs[i] = reflect.ValueOf(templateResult)
                } else {
                    // Step functions in Godog all return an error type.  We can use this
                    // to return any errors from evaluating the template here.
                    return []reflect.Value{reflect.ValueOf(err)}
                }
            } else {
                // Simply use the existing arguments for other types
                newArgs[I] = a
            }
        }

        return fnVal.Call(newArgs)
    }).Interface()
}

Using It With Our Tests

With the function builder done, we’re now ready to use it in our tests. Simply wrap each of your step functions with a call to decorateFunc within InitializeScenario to give it the ability to evaluate string template parameters:

func InitializeScenario(ctx *godog.ScenarioContext) {
    testConfig := ReadTestConfig()

    ctx.Step(`^a "([^"]*)" request to "([^"]*)"$`, testConfig.decorateFunc(givenARequest))
}

After this, you should be able to now use environment variables to configure your tests:

$ godog
.. ServiceHost will be "myservice.com"

$ E2E_TESTS_SERVICE_HOST="adifferentservice.com" godog
.. ServiceHost will be "adifferentservice.com"

As more step functions are added, they can take advantage of this capability so long as they’re wrapped in a call to decorateFunc.

Of course, this is not limited to service hostnames, or even reading things from configuration in general. Since you’re using Go templates, you can extend this to do all sorts of things, like transformations or calculations, to generating data randomly. The capper is that your tests are now relatively independent of the environment and test data that they need to work on, meaning that the only time they need to change is when the functionality does.