Api request lifecycle and some other stuff

[rjeremy]

Hello.

Recently started with Go,
and I’m not fully familiar with it yet.

What is going on under the hood when client hits an api endpoint?

Am I on the right track with the following:
Request → Router → Context →
   Sync pool → Goroutine with handler function →
      Response → Dispose → GC

Can someone point me to the official or other reliable resources
where this topic is covered in detail, as I’m unable to find it?

Also, regarding database connections…
Official tutorial says:

“Making db a global variable simplifies this example.
In production, you’d avoid the global variable,
such as by passing the variable to functions that need it
or by wrapping it in a struct.”

I’ve seen this in most articles I found, but none explained why.

I mean, I’m going to follow official recommendation,
but what is the fundamental difference?
You can wrap it in struct, but both pointers point to the same
connection with internally managed connection pool.

I tested both ways with thousands of requests via Jmeter while monitoring
active and idle connections in database and got the same results.

Was using postgresql with sqlx which is a wrapper of standard db library.
So I’m a bit confused here.
What am I missing?

And lastly…
@hollowaykeanho
I stumbled upon this.
Can you write a word or two about “package decoupling with vertical approach”
you mentioned there, if you don’t mind, of course.
Is that something like vertical slice, maybe?

Much appreciated

[hollowaykeanho (Holloway)]

@rjeremy

Can someone point me to the official or other reliable resources
where this topic is covered in detail, as I’m unable to find it?

Can’t comment on your precise investigation without doing mine but poking at the source codes should help:

1. http package - net/http - Go Packages
2. https://cs.opensource.google/go/go/+/refs/tags/go1.17.2:src/net/http/

The new documentation site has source codes at the bottom of the page. Feel free to explore in the future.

@rjeremy

I mean, I’m going to follow official recommendation,
but what is the fundamental difference?
You can wrap it in struct, but both pointers point to the same
connection with internally managed connection pool.

It’s more on the memory accessibility of the variable; not performance. It should not have any difference because they are both accessing a pointer.

In production system, you want more control over magic. A global variable that is accessible by any functions, inside or outside of package (depending on its access availability) so it can cause unwanted changes by anyone or everyone, especially messing around with a crucial component: your database access. What it meant is instead of:

var DB *sql.DB  // let's make it worse by setting the variable to publicly accessible and func Rogue alters it in its own goroutine
...

You can do something like:

func SaveData(db *sql.DB, ...) {
    ...
}

func main() {
    ...
    db, err = sql.Open("mysql", cfg.FormatDSN())
    ...
    SaveData(db, ...)
}

For the latter, you have tighter control over your database memory object (like which function is responsible to overwrite/access the db memory object at a time). Also, at any given time, you know you can discard db safely (think concurrency as you scale).

Also you need to be aware that this actually works in Go so variable naming can be quite misleading within a function to mess things up:

package main

import (
	"fmt"
)

func counter(fmt int) int {
	return fmt + fmt
}

func main() {
	fmt.Println(counter(5))
}

Playground: Go Playground - The Go Programming Language

@rjeremy

Can you write a word or two about “package decoupling with vertical approach”
you mentioned there, if you don’t mind, of course.
Is that something like vertical slice, maybe?

Definitely not a data type. It’s just a strategy to package your codes in a way of reducing as many error checking as possible. Keep in mind that it’s just an opinion, not a rule.

To do that, you need to:

1. make use of internal/ package for development.
2. Be very good with pointers.
3. Always use struct type to structure package data up.
4. Good at ‘function as a value’.

The trick is:

1. Pass a logger struct object into the intermediate package (place inside your struct as an element, not global variable) for logging noisy error messages instead of returning all the way up to the app-level functions.
2. Make sure the logging function can handle optional logging because not everyone practice this strategy.
3. Make use of pointers as return value (Basically to use nil as a bad output). Then ask yourself: is the error message useful for your higher level function ‘customer’:
   3.1. If ‘yes’, then you return error object (usually ‘no’. See [1])
   3.2. Otherwise, just log it and return nil as value.
4. IMPORTANT: the intermediate package must be reusable for other app development so it will mature out from your internal/ package over time.

[1] - The reason for no is because app-level package (highest level function) has their own dedicated error message to present for the user (See: GitHub - tomarrell/wrapcheck: A Go linter to check that errors from external packages are wrapped). Therefore, these error messages are meant for internal app developers so a runtime logger makes more sense than error escalation, dramatically reduces the error checking all the way up.

[rjeremy]

Thank you for clarification and explanation.
That vertical approach is a pretty clever way of handling errors
and I can see where I can fit it, but on a smaller scale.