API Gateway + Go Microservices

[dhenkes (Dominique Henkes)]

Good morning,

I recently watched a talk about Microservices (I think the guy giving it was from Monzo) and they had an interesting architecture. Now I was wondering if it would be possible to write a basic implementation in Go without using any third party frameworks but I have no idea where to start or how it would work.

What I’m looking for:

There should be an API gateway that takes all requests. It should accept every uri (/users, /tokens, etc.) and forward those uris to the specific microservice. I want to be able to just write a new microservice that accepts the GET, POST and PUT for /test and the API gateway should automatically route everything to that service without restarting.

The basics of how it works in my mind:

API Gateway runs on port :8000, accepting requests. There is a route.json file with all the routes and accepted methods that point to the right port.

I fire up a new endpoint-microservice (port :8001), add the methods and routes to the json and it automatically is being used without me restarting the gateway. Every http request for that endpoint is just being forwarded completely to that microservice.

Now I have a few problems:

If I completely forward the request to the microservice, how do I get the response back to the user? Should I wait until the microservice sends some answer back and send that back to the user?

How do the companies manage authentication and security? How could I theoretically prevent another microservice (not written by me), firing up in the network and intercepting all traffic to an endpoint?

[geosoft1 (George Calianu)]

It’s a long discussion around but in a very few words you can use http package for the gateway. More comfortable is to use gorilla/mux.

[LE] A very basic example of web service with a single route and auth with token:

package main

import (
	"fmt"
	"net/http"
)

func main() {
	// curl -X GET 'http://localhost:8080/api/status' -H "X-AUTH: admin"
	http.HandleFunc("/api/status", func(w http.ResponseWriter, r *http.Request) {
		token := r.Header.Get("X-AUTH")
		if r.Method == "GET" && token == "admin" {
			fmt.Fprintln(w, "ok")
		}
	})
	http.ListenAndServe(":8080", nil)
}

As you can see writing directly to response writer return the response to the client. This should respond to one of your questions. Request allow you to use the methods (see the structure of http request in Go).

Also is a good ideea to config your application with a json file.

For security are a few solutions (basic auth,api keys,custom authentications, jwt). I can suggest you api keys. You can also use a https server in Go but is a little complicated.

Some interesting resources:

https://github.com/arschles/go-in-5-minutes/tree/master/episode1
https://github.com/arschles/go-in-5-minutes/tree/master/episode3
https://github.com/arschles/go-in-5-minutes/tree/master/episode5

[Andjarnic (Kevin)]

Interesting idea. I am just starting out in Go myself. My experience has been Java / Spring and building Rest APIs using RAML (to describe APIs which you should have a look at), and a RAML to java code generator that generates interfaces for the API resource endpoints. It works very well. None the less, in my desire to learn and use Go, I am wondering if I can replace Java completely, building a Go based server rest API framework. Similar to what you propose here… only in my Java experience the entire API implementation is one service application (with an HTTP front end). It is always protected by OAUTH though and I would want my Go implementation to offer the same capability.

I am curious if in Go it is more common to build API resources like you are describing as separate little microservices that run as stand alone applications… or if a larger all in one service implementation with the router and all that makes sense.

I am also asking/interested because I envision all of this being deployed with Docker, so that scalability in the cloud is out of the box. We currently build all our Java stuff using Docker so that we can scale it and it works very well as we can test it on our dev machines and ensure the same exact thing is deployed in production.

If you are looking to model Rest, you might want to consider sync and async requests. You mentioned do you wait for the microservice and send the response back. You have to determine if the service might take some unknown amount of time and thus worth keeping the expensive pipe open to the client. For scale reasons, if you plan to support lots of connections, then it is better to adopt async API endpoints where by the client gets an immediate response back and a resource to a status object… that they can then pull (via GET method) to see the status of the async process. At some point, when it is done, that GET pull from the client will result in a 302 redirect and the Location header would contain the URL to the actual resource. Another way to handle this is to allow the client to offer a callback URL… this may not be possible if the client is a mobile app (unless it watches for SMS messages or something), or a desktop application. However a lot of services are used by other web based services, which have the ability to provide a URL that can be called by your service when a process is done. In this case, the client (service) may make some sort of initialization call with the callback URL for notifications from your service. Then, any async resource you provide can make use of that callback URL to send notification when a process is complete. This is also similar (but not identical) to web hooks. Either could work in this case.

[dhenkes (Dominique Henkes)]

Thank you two for the response.

@Andjarnic As far as I could see it’s more common in Go to write a monolithic application when it comes to REST APIs. I only saw a few projects which used that kind of microservice separation. But take that with a grain of salt as I’ve only seen a handful of projects in the last few months.

What I settled on after some trial and error:

I have my API gateway which subscribes to a REDIS channel. As soon as I spawn a new microservice endpoint, it publishes a message to that channel and the API gateway registers the route (or adds the service to the already registered route). Now I can just publish incoming requests to the specific service channel and one of the correct microservices will pick it up.

The only thing I’m struggling with right now is how to handle the microservice response in the API gateway as I’m not waiting on a response.

Two ideas I had:

• Subscribe to a private channel (UUID based) so the microservice can publish the response to the API gateway.
• Just publish the response to the API Gateway channel with the right uuid.

But I have to try around a bit and see what works best.

[flexd (Kristoffer)]

What you are asking for already exists. For example Kong https://getkong.org/

[dhenkes (Dominique Henkes)]

Thank you, I didn’t know Kong before and will look into it.

I asked originally because I wanted to learn how to do something like this, so it was a learning project not something I was going to use in production.