Why I'm happy to live without generics in go (for now)

I was at GoSF last month (9th May) when Andrew Gerrand gave his talk on go 1.1. At the end of the talk a couple of questions came up on the subject of generics. You can see them on this video at 1:09:20 and 1:14:40. (The questions aren't audible, but the answers are)

If you don't want to watch the video (you should) the summary (and I'm paraphrasing alot here) was essentially that it's not that anyone is trying to keep generics out of go, but rather no one has found a nice way of putting them in. Andrew went on to mention that he didn't miss them at the moment as copying and pasting a few for loops and if statements was an ok alternative.

I nodded in agreement at the time, but last week I proved it to myself in code, so I thought I'd blog about it. I found myself wanting to add up the value of some things in a list, here's a very trivial substitute for the actual code:
It's nothing ground breaking, you could say I'm folding (or reducing) over the list, you could also just say I'm adding up the price of all the things. It would be nice to have a generic fold that I could use to explicitly say that I'm folding over the list, but what advantage would this give me?

I probably wouldn't just call fold by iteself, i'd probably still put fold inside the TotalPriceOfBasket function so that I can state the purpose more explicitly, and the reader would be left with a much simpler looking function, but - and here's my point. fold, map, filter should be recognisable as patterns. You should be able to look at a function and say "Of yeah, a fold, ok"

I'm not making any statements here about how useful generics are or aren't. or how useful a fold/map/filter function would be in go, all I'm saying is that I can live without them for now. Especially as the reason they're not currently in the language is that the core contributors haven't found a good way of implementing them. There's a lot to be said for that approach.

Using go to unmarshal json lists with multiple types

Everyday I seem to be writing go code to parse a json string, and this problem seems to come up often enough for me to write about it. Thanks to adg and asoko on #go-nuts for their suggestions.

The Problem

Given a list of json objects of different types (lets say People and Places). You want to Unmarshal them into two lists. A list of all the people and a list of all the places.

A bit more definition

Let's use this json string

{
    "things": [
        {
            "name": "Alice",
            "age": 37
        },
        {
            "city": "Ipoh",
            "country": "Malaysia"
        },
        {
            "name": "Bob",
            "age": 36
        },
        {
            "city": "Northampton",
            "country": "England"
        }
    ]
}

To help us write some code, let's give ourselves a function, which should be self explanatory:

func solution(jsonString []byte) ([]Person []Place) {}

And some structures

type Person struct {
 Name string
 Age  int
}

type Place struct {
 City    string
 Country string
}

I've got two solutions to this problem. I would love to know of better ways.

SolutionA: map and type assert

If we tell json to unmarshal into a map we can get it to deal with the parts we know about, and the rest of it will go into an interface{}. As we loop over the json structures we use what we do know about the structures to pass the interface{} to some helper functions what will create one of our structs and add it to our list. Because the map we take in is a map[string]interface{} we will need to type assert our values

func solutionA(jsonStr []byte) ([]Person, []Place) {
 persons := []Person{}
 places := []Place{}
 var data map[string][]map[string]interface{}
 err := json.Unmarshal(jsonStr, &data)
 if err != nil {
  fmt.Println(err)
  return persons, places
 }

 for i := range data["things"] {
  item := data["things"][i]
  if item["name"] != nil {
   persons = addPerson(persons, item)
  } else {
   places = addPlace(places, item)
  }

 }
 return persons, places
}

func addPerson(persons []Person, item map[string]interface{}) []Person {
 name, _ := item["name"].(string)
 age, _ := item["age"].(int)
 person := Person{name, age}
 persons = append(persons, person)
 return persons
}

func addPlace(places []Place, item map[string]interface{}) []Place {
 city, _ := item["city"].(string)
 country, _ := item["city"].(string)
 place := Place{city, country}
 places = append(places, place)
 return places
}

SolutionB: Mixed Type struct

This solution involves creating an interim struct which can be used to represent either a person or a place

type Mixed struct {
 Name    string `json:"name"`
 Age     int    `json:"age"`
 City    string `json:"city"`
 Country string `json:"country"`
}

With this struct we can then unmarshal our json string into a list of these mixed types. As we loop over our Mixed structs we just need to examine each one to work out which type it represents, and then build the right struct from it

func solutionB(jsonStr []byte) ([]Person, []Place) {
 persons := []Person{}
 places := []Place{}
 var data map[string][]Mixed
 err := json.Unmarshal(jsonStr, &data)
 if err != nil {
  fmt.Println(err)
  return persons, places
 }

 for i := range data["things"] {
  item := data["things"][i]
  if item.Name != "" {
   persons = append(persons, Person{item.Name, item.Age})
  } else {
   places = append(places, Place{item.City, item.Country})
  }

 }
 return persons, places
}

These are just two ways I've used to solve these problems, I'd love to know how others have done it.

SolutionC: json.RawMessage (Updated 18Jan13)

Thanks to Jordan's comment and zemo on reddit there is another solution. Using the json.RawMessage structure in the json package we can delay unmarshalling the json structures in the list. We can then go through our list and unmarshal each of them into the correct type

func solutionC(jsonStr []byte) ([]Person, []Place) {
 people := []Person{}
 places := []Place{}
 var data map[string][]json.RawMessage
 err := json.Unmarshal(jsonStr, &data)
 if err != nil {
  fmt.Println(err)
  return people, places
 }
 for _, thing := range data["things"] {
            people = addPersonC(thing, people)
            places = addPlaceC(thing, places)
        }
 return people, places
}

func addPersonC(thing json.RawMessage, people []Person) []Person {
    person := Person{}
    if err := json.Unmarshal(thing, &person); err != nil {
        fmt.Println(err)
    } else {
        if person != *new(Person) {
            people = append(people, person)
        }
    }

    return people
}

func addPlaceC(thing json.RawMessage, places []Place) []Place {
    place := Place{}
    if err := json.Unmarshal(thing, &place); err != nil {
        fmt.Println(err)
    } else {
        if place != *new(Place) {
            places = append(places, place)
        }
    }

    return places
}

Here's the full gist:

Using vim's path to speed up your Go project

If you're working on a project in Go, and you're using vim, then you really need to check out path:

:help path

Here's a very quick tour.

Add this to you .vimrc:

set path +=/your/projects/gopath/src/**

** tells path to include all subdirectories. By adding your projects GOPATH entry it opens up access to two rather cool features that are useful in navigating your project.

:find

You can use :find to open a file instead of using :e. But :find will look in all of your project's directories to find the file. So rather than having to do:

:e /package/v10/another-dir/god.go

You just need:

:find god.go

You also get tab-completion for free - just in case you can't remember the file's name.

gf

gf is the useful little command that goes to the file under the cursor. Well providing you've set your path correcly you can navigate your project's imports with ease:

package main

import (
 "code.google.com/p/go.net/websocket"
)

By "gf"ing over the import vim will open the directory, giving you the list of files in that package.

Just don't dare navigate the directory using the arrow keys!

Update: 8th Jan 2012

On the subject of vim and Go it's worth remembering this little gem:

au BufWritePost *.go !gofmt -w %

Whenever a Go file is saved gofmt will be run against it.

go(ing) under

Lots of people like go and this year I began to find it unavoidable - I'm even ending the year with go being the main language that helps pay my bills!

One of the things that go lets you do is pass functions around.
I started the year talking about under from the j language. Which lets you 'bookend' some function with another function (a verb) and a function which undoes that function (obverse).
A great example is Pythagoras' theorem:

1. You perform some 'pre-processing' on each value (squaring)
2. You add all the values up
3. You square root the result

I started the year talking about under in clojure. Here it is implemented in go. Complete with functions being passed around, closures, and functions returning functions. Happy new year!