The new gopher experience

I’m interested in the experiences of recent converts to Go. Specifically, what bugs and problems you ran into when you were learning the language, coming from another programming language.

  1. What were the biggest problems you encountered?
  2. What mistakes have you made (e.g. not realising that ListenAndServe is blocking)?
  3. What is your biggest cause of panics?

I’ll start, even though it’s been quite a while since I learned go:

  1. Groking gopath took a while, at first I didn’t understand why I had to change the project structure for lots of projects, when I wasn’t even completely committed to go.
  2. I made silly mistakes like forgetting ListenAndServe was blocking and wondering why code after it didn’t execute, and the classic of wrapping a goroutine in a for loop or range and getting the same index 10 times.
  3. For the causes of bugs, I think not allocating maps (the zero value is not useful), nil pointer dereference (for various reasons, mostly programmer error), and out of range errors on slices were the biggest causes of crashes in the first programs I wrote.
1 Like

All of those, plus what took me forever to figure out is that returning a nil pointer from a function that returned an interface does not make the interface nil. That confused the heck out of me.

e.g.

type someErr struct {
    msg string
}
func (s *someErr) Error() string { return s.msg }

func doSomething() error {
    var err *someErr
    // do a lot of stuff, maybe set err
    return err
}

The above will always return a non-nil error. Why? Because Go’s conversion to an interface doesn’t care if a pointer is nil… it’ll wrap that nil pointer in the interface just fine. The only way to return a nil error from that function is either to literally return nil, or return a variable that is already the error interface, and happens to be nil, like if it was initialized as var err error

It took me forever to figure out why my function was returning non-nil interfaces when I was returning a nil pointer from the function.

3 Likes

It took me a long time to get comfortable with the enforced directory structure and not end up fighting it.

It also took me quite a while to stop trying to write code as if Go had class inheritance and tying myself in knots that way.

And thirdly, I tried using functions returning closures to work around the lack of generics. That also turned out to be a bad idea from a complexity point of view, worse than the problem I was trying to solve.

Thanks I’d forgotten about this one.

Also calling methods on nil pointers can be unexpected, in C it would fail, in Objective C be ignored, but in Go it calls the method and the method can then decide what to do, which is not necc. intuitive. Often people don’t account for this and when you try to use a nil pointer method bad things happen (e.g. using a nil *sql.DB).

2 Likes

Yes I tried fighting go about GOPATH with symlinks, but go won.

Also my first question about Go on stack overflow when picking it up boiled down to how can I use inheritance in Go? Took a few weeks to leave inheritance behind since every language I’d used previously used it, some excessively (Java and ObjC), some less so but all, it turns out, unnecessarily.

What were you doing with functions returning closures, do you have an example of this approach?

My first encounter with date/time formatting and parsing based on Go’s “unique” layout was (still is) very strange to me. The documentation may be clearer these days, but when I first had to format a date for output, I thought the layout in the docs was simply an example date, not the actual parameters required to format a given date for output.

It still strikes me as odd that the Go team re-invented the wheel on this one, and in a way so unlike anything else, at least that I’ve seen.

4 Likes

I think on dates it’s partly that they chose such an obtuse example format, which makes no sense unless you remember the specific date example involved. In theory in this format it goes 123(but actually 15)4576, so it’s not even in order. No idea why this got chosen, perhaps in some date format it makes sense? It just seems rather arbitrary and I always have to look it up.

Mon Jan 2 15:04:05 -0700 MST 2006

They’d do better just to use the standard specifiers (as they did with sprintf) and define more standard formats for dates within the package. There are a few but not enough, for example there is none for dates in the international date format: 2006-01-02 only the full RFC3339 format. Much clearer to say time.Format(RFC3339) rather than time.Format and than using an arbitrary format string IMO.

2 Likes

Just wanted to say thanks to everyone here for their notes. I’m just dipping my toes into the language and this is incredibly helpful to read/set expectations. Cheers @kennygrant for spinning this up :slight_smile:

1 Like

Interesting topic. I started using Go about 9 months ago. Moved a whole product to go while convincing a dev organization that it’s the right thing to do. So here is my view on some things I was struggling with and some which worked out fine:

Things that were good:

  • $GOPATH - I think this is pretty well explained on the site and the various places where I started out. Not a big deal, but I tend to put all of my code in one folder anyways)
  • The language is very well explained and the Go Playground along with the varios official tutorials on golang.org are just brilliant.
  • The community in general and also amount of code to look at is great

Problems I ran into at the beginning

  • Panics: Maybe I haven’t found the article on how to read them yet. But whenever I get a go panic I seldomly have a clue where it happened or why.
  • Better debugging - I know it’s not a good way to develop software, but especially in the beginning it’s really handy to have the ability to inspect the stack etc via the debugger. I think I still haven’t really come to grips with GDB. Coming from mostly Ruby/Java/C#, having a repl available or at least being able to step through the code via breakpoints makes debugging soo much better.
  • Error handling. This is the biggest hurdle to getting people on board with Go. The syntax, the folder structure - everything else people reacted mostly well to. But the amount of if err != nil all of the basic tutorials contain is immediately offputting to people you show go to. It takes some getting used to, but in actual applications most functions don’t turn long enough for error handling to be as ugly as initial tutorials make it out to be.
  • Packages: I think the main package will trip everyone at some point. Coming from other languages you immediately feel the urge to break that 500 LOC file into a whole armada of go files, putting everything into it’s neat little well named box. And you can get pretty far with that until you end up having to access something that’s in main and you simply can’t. That’s also the point where you realize you’ve been doing it wrong and start adopting a more “sane” approach to packages.
  • Packages can’t satisfy interfaces - this should be written somewhere in bold letters. I thought the design of my package looked neat with it’s init func and everything. Until I had to re-do it completely
  • The naming conventions need to be explained a bit better.
  • The testing utilities in Go are awesome, but the interface of TB should get a lot more attention. The amount of tests I did rewrite to benchmark it is just silly and I could have just used TB from the beginning.
  • The testing.B behavior needs a bit more docs on the official site. I always end up on the blog post by Dave Cheney which is great.
  • It should really be documented that go build will run miuch faster after you did go install. No clue why this is, but when I wrote a program using the kubernetes/client-go library I was really annoyed by how slow the compile was until I accidentally did a go install and it went back to being fast again.

greetings Daniel

1 Like

Thanks Daniel, that’s just the sort of response I was hoping for. Everyone has a few little details that tripped them up at first.

Re naming conventions, I’m not sure they have many formal ones, though some have become more standard than others.

Re main package import, yes that does seem to fox lots of newcomers, that main is a special case and should only ever import, never be imported.

Re go build vs go install this may be because go install keeps the build products whereas go build tends to rebuild each time. You can use go build -i for faster builds with go build, or accept that go install will install to your gopath/bin directory.

Packages can’t satisfy interfaces - this should be written somewhere in bold letters. I thought the design of my package looked neat with it’s init func and everything. Until I had to re-do it completely

Can you expand on this one, I wasn’t sure what you meant here, can you write a quick example, this sounds like an interesting mistake…

I started using empty structs as type parameters so I could avoid reflection:

router.HandleFunc("/api/user/add", makePostHandler(&User{}))
router.HandleFunc("/api/group/add", makePostHandler(&Group{}))

…where makePostHandler took any empty struct of type fulfilling Persistable, and returned a function closure which called methods on the empty struct to perform the work. The Persistable interface was made up of sub-interfaces defining methods to decode an object, validate it, insert it into the database, and so on – the steps the handler needed to take.

Then I thought, well, why not use a similar technique to implement Map operations and other higher-order functions? See this article for how this applies to sorting too. I added some more methods to the interface and implemented them in my data types. Before long I was using Map with a function argument to generate views of objects on the web or dump objects as JSON via the API, again without needing to write the same code multiple times.

The problem is, while I now only had to write my handler functions once, I’d added a whole lot of complexity to every data type ever touched by this technique. All my model data structures gradually had to satisfy more and more interfaces so that they could be passed around the codebase.

What I really wanted was a way to be able to say (of a method argument) that it had to satisfy two interfaces. But an argument could only have one interface type, so I started trying to break things up using interface composition – I’d have a Mappable interface, a Persistable interface, and a MappablePersistable interface for things which had to satisfy both. Next, I needed different PUT/POST behavior depending on whether the API was supposed to INSERT or UPSERT, so I needed PersistableUpserter, PersistableInserter, MappablePersistableUpserter, and so on… and then I realized that this was madness.

So, that was the clever thing to do to save writing half a dozen simple handler functions, that turned out to be a bit of a disaster. I could have started using interface{} all over the place, but I really wanted to keep type safety.

After reading around some more I arrived at a description of using the data mapper pattern in Go. You keep your core objects as simple as possible – just a struct with some fields – and build a UserMapper, GroupMapper and so on to handle HTTP and database stuff. You have to write them, but it’s mostly boilerplate code, and you only ever have to write the ones you’re actually going to use. You also now have to write handlers for PostUser, PostGroup and so on, and it’s a bit repetitive, but again they’re only a few lines long and all standard stuff.

The big win is that the User code doesn’t have to know anything about HTTP methods or anything about databases, and the Handler code doesn’t need to know anything about User objects or databases. The Mapper links the worlds of HTTP and database, and insulates the two areas of concern from each other. If you need to change the INSERT behavior or payload encoding for a particular type of object, it’s all in one file and you can change it without the risk of breaking how other objects behave.

As for higher order functions, I just wrote the code twice for the two specific types I really needed to be able to Map on.

Note that there are totally valid cases where empty structs as type parameters, HTTP handler generation, and higher order functions are the best solutions for the problem. However, I’d categorize them all as clever code, and one thing I’ve learned over the years is that it’s best to avoid clever code unless you really need it, even if it’s shorter.

1 Like

Did you try direnv or virtualgo? These are two approaches to trick Go into using multiple GOPATH’s.

I did not yet get to wrap my head around virtualgo’s concepts but I am using direnv to have a clean Go env when recording course lectures.

I tried doing it by hand (this was a long time ago), but it broke vendoring, so eventually I just gave in and put everything in gopath. I’d be happy if gopath went away and the vendor folder was used instead to install dependencies for a given project - having shared dependencies across projects is saving a little disk space for not much gain, unless you live in a monoculture where there is only one repo for everything and every project you work on will always be in it (i.e. google). I guess gopath was a good solution in that circumstance but for a broad distributed ecosystem it makes life harder.

Sorry it took me so long to respond.
Yes packages can’t satisfy interfaces.I just wanted to insert data into a backend and it got annoying to carry around a struct that contained the method to do so, so I ended up defining it as a regular variable in a package that had one exported function. Quite soon after I changed everything I noticed that I can’t swap the package for a fake implementation in my tests as a package can’t be used with an interface :slight_smile:
Stupid I know, it felt totally obvious afterwards - but while doing it I thought this is smart as I can just swap this out later :slight_smile:

1.Biggest problem
That would be understanding goroutine and channel, because there is a way to use them the blocking way as well as using them the non-bloking way.
And then there is bufferd channel, using select statement to receive channel close signal.
These are really difficult for me.

2.Mistakes
The biggest one would be a memory leak problem happens when I was trying to make a memory cache utility.
Everytime after I fetched data from database, I marshaled them into json, then write them into a file, so that everytime I restart my go process, I can just read from cache file.
It expires after 24 hours, and it’s about 20MB.
After a week, my go process is using like 500MB of memory.
The reason of that is go does not release memory right after it is ok to release, and go does not return the memory to the system unless you tell it to do so.
So I just call debug.FreeOSMemory() after reading that bigass cache file.

3.Biggest panic
That has to be json marshaling.
Especially when that json is not so static typed.
It happens when you trying to map a string to an int, or a string true to bool, can’t really solve this, I just have to add omitempty,string to every single none-string field.
I believe it’s about 1/3 of my panics come from this.

100% agree. The json parser is a bit too picky at times and the workarounds are quite painful. So if you have a int property that might be a “1” instead of a 1 the standard json parser will just die on you…

1 Like

Yep, what is worse is, its error message only tells you are trying to map a wrong type, doesn’t even tell you which field name failed…
That is really painful when dealing with giant json.

Yes I do wonder if struct tags were in fact a mistake, and it wouldn’t have been better to have explicit unmarshalling for json/xml etc in code, because it’d be a lot easier to handle this stuff without using the ad-hoc, underspecified and difficult to understand secondary language encoded in the field tags. I’ve managed to successfully avoid using struct tags while writing quite large apps so it does work fine to do so.

Well, in some situations, I came up with this method:
Mashal the json to a map[string]interface{}, then iterate its values and fields, using reflect to map to a struct.
But then I have to write a lot of extra code to do this, wondering how other languages solve this, maybe there is another way.

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.