Mr. Golang's Crazy Config

The title of this article is a reference to fasterthanlime's I want off Mr. Golang's Wild Ride. I highly recommend it.

For the purpose of clarity, I'm going to refer to the programming language 'go' as 'golang'. Deal with it.

The Problem

Okay, so I'll admit that I messed up. I wasn't thinking the 'golang way' when I originally set out to write some code to read in a config file. I had hubris. I thought that standard golang tools would have the functionality for what I wanted to accomplish. I was wrong.

I had a yaml config file that I wanted my golang program to read. Simple enough I thought, I'll just use yaml.v3 read the config into a struct and call it a day. However, I have this splinter in my mind to avoid writing code that has unexpected behavior. In golang, there are 'zero values' which are the default values for variables of a type. For example:

var s string // an empty string
var w struct{i int, s string} // a struct with i = 0 and s is an empty string
var p *string // nil pointer

Zero values are unexpected behavior landmines. They can easily decieve the programmer into thinking that they have an actual value, when in reality it is value that the golang compiler gave them.

Suppose I have a config file that looks like:

uri: "/abc/123"
count: 2

and wanted to read it into a struct like:

type Config struct {
    Uri string `yaml:"uri"`
    Count int `yaml:"count"`
  }

A problem arises: How do I determine if a field in the config is missing?

If a field in the yaml is not set, golang will provide the zero value in it's stead. Uri would be set to an empty string and Count would be 0. So what? Why does this matter? It matters because of input validation. If Uri and Count must be set, I have no way of knowing whether or not they were set because I get an empty string and 0 respectively, which could be actual vaules.

What's a programmer to do? Ah! I know! I'll just make all fields pointers!

type Config struct {
    Uri *string `yaml:"uri"`
    Count *int `yaml:"count"`
  }

This solves the problem! Now if a field is missing I get a nil and I know whether or not it was set in the config. But wait... Now all fields are pointers... That means I'll be doing:

if config.Uri != nil {
  // ... code here
}

And I'll be doing it everywhere. Now I get that the seasoned golang programmer probably has if x != nil {} burned so deeply into their muscle memory that this doesn't seem like a big deal, but when your config starts to grow to dozens of fields, it's going to get tedious. Additionally, it doesn't make sense to pass around an invalid config just for something else to fail because of it. This fails to properly encapsulate error state.

Solution 1: There must be an setting

At the time of writing, there is no setting. yaml.v3 provides a method on its Decoder with a type signature of func (dec *Decoder) KnownFields(enable bool) that will fail to parse if additional fields are present in the yaml file, but no such luck for a method that fails to parse if fields are missing. I gather from the github issues (the commit message was nondescript) this method was added because yaml.v3 wishes to maintain 1:1 api with encoding/json, which as I discovered also does not have a way to fail if fields are missing.

1:1 api 🤔. Let's see if encoding/json has the KnownFields method. Nope! Instead it has func (dec *Decoder) DisallowUnknownFields() which does the same thing, but has a different name!

Solution 1.1: Just Write Your Own Decoder

No, I refuse. I'm not writing a custom decoder for my config. That's tedious.

Solution 1.2: A Misguided Tale of Two Structs

So here is an even more tedious work around that I breifly tried. If I have two structs:

type Config struct {
    Uri string `yaml:"uri"`
    Count int `yaml:"count"`
  }

type pkgconfig struct {
    Uri *string `yaml:"uri"`
    Count *int `yaml:"count"`
  }

I can unmarshall into pkgconfig and then convert it into Config that way I won't have to deal with those pointers! Oh wait, I still need to deal with those pointers. In the end this was basically writing my own decoder. Damn it!

I could, though, just generate the function to convert between the two structs. Read in the two structs at compile time using go generate and produce the code to nil check all the relevant fields for me. Prior to falling down that rabbit hole I discovered a way better solution.

Solution 2: An Actual Solution

Just use spf13's viper. As it says in its README "Viper is a complete configuration solution for Go applications" and it truly is. It provides a method of UnmarshalExact that allows you to pass options to the config. It's as easy as:

package main

import (
  "bytes"
  "fmt"

  "github.com/mitchellh/mapstructure"
  "github.com/spf13/viper"
)

type Config struct {
  Uri             string  `mapstructure:"uri"`
  Count int     `mapstructure:"count"`
}

func decoderErrorUnset(c *mapstructure.DecoderConfig) {
  c.ErrorUnset = true
}

var conf = []byte(`
uri: abc
count: 2
`)

func main() {
  v := viper.New()
  v.SetConfigType("yaml")
  v.ReadConfig(bytes.NewBuffer(conf))
  c := Config{}
  err := v.UnmarshalExact(&c, decoderErrorUnset)
  if err != nil {
    panic("invalid config")
  }
  fmt.Printf("%x\n", c)
}

That's it. That's the whole thing. Just pass a function to set whether or not we get an error when unmarshaling. No need to if x != nil {} every time a field is read.

Viper even has the ability to set real default values via the SetDefault method and to read from environment variables via BindEnv. It's absolutely amazing.

Conclusion

Don't use encoding/json or yaml.v3 for confguration. They aren't made for that. They do not want to support that. They probably won't support that in the future.

Use viper, it will give you every bell and whistle you could ask for when it comes to configuration parsing.

Don't make the same mistakes I made:

• Not knowing viper existed.
• Thinking standard golang tools would provide neccessary functionality for parsing configs.

I think that tagged unions (Option type) would have been amazing for golang and would have been a better solution than zero values. With Option instead of zero values, variables with no initialized value would be a compile time error but you could still have the ability to express optional values via the None variant. nil would become unneccessary.

However, that ship has long sailed. Even if the golang community wants tagged unions, they fit too poorly into the language as it I doubt the golang team would introduce them.