Is this API attempt breaking any rules?

My goal is to create an API that can serve hundreds of endpoints in the future. I may break several “rules” in my attempts to make this a bit more generic avoiding typing errors and make it easier to maintain.

The first rule I break is to use sqlx when dealing with Postgresql which is reducing the code significant.

The second rule I break is to move the queries to a lookup table. This reduces the number of endpoints. So every API call is calling the database twice. First to get the query and then to execute the query. A benefit of this is that I can maintain the queries on-the-fly without restarting Go.

The third rule I may break is that I am using http.Get because it is simpler to understand.

This is my first attempt to create an API as a newbie. And so far it is only the GET part of the CRUD. And I may over simplify. Here is my code:

The web client
https://play.golang.org/p/4vdPsKHGqBo

The API
https://play.golang.org/p/fEKTIQ13GiD

Live
http://94.237.92.101:6060

My questions:

  1. Should API always deliver JSON (and not map[])?
  2. Do you see any possible danger with this approach?
  3. Which parts should I reconsider?

One suggestion is to replace

fmt.Fprintf(w, "%s", json)

with

w.Write(json)

Regarding the maps, I saw you use it as local variable but generally speaking in concurrent environments you have to be careful to lock them when writing.

About APIs, a good practice is to return JSON structures.

1 Like

Done. But what is the benefits? Other than it is cleaner?

Can you please elaborate? Where do I use local variable and how should I do it in a better way?

TIA

It’s faster than fmt library.

Just saying is better to avoid maps in concurrent environments or use them carefully.

Well, I think keeping queries in the database can lead to security holes or application fail is somebody change a query in the database or somehow that record is altered. I guess that is safer to hardcode the queries in the application.

What is the worst that can happen? Just curious…

The API should just have read only set at role level. Managing and maintaining maybe 1000 endpoints / queries could be even more risky for type errors etc.

I cannot see any risk of altering the lookup database. But there may be other risks that I am not aware of now.

Application could crash (see here an example). But you should only worries if you plan to exchange data through maps in concurrent environment.

1 Like

Thank you! I will investigate this further.

Why would you not use gRPC?

I want to learn gRPC, but I am not able to understand it. There are several “how-to-do” when using traditional REST api on the net, but basically none dealing with common CRUD using gRPC. The complexity of gRPC is higher.

This code is the REST API: http://94.237.92.101:6060/api

How do I “translate” this code into gRPC? Will it be more complicated?

or better still replace

    		json, _ := json.Marshal(data)
		// present result for the web, partner or client
		w.Header().Set("Content-Type", "application/json")
		fmt.Fprintf(w, "%s", json)

with

    json.NewEncoder(w).Encode(data)
2 Likes

Thank you! I worked like a charm! http://94.237.92.101:6060/posts

I also noticed you do a lot of lines like

  page := (path + ".html")

the brackets are unnecessary so

 page := path + ".html"

works just as well…

The endpoint() function is basically a hand written Mux.
That’s fine. But looking at some of the other muxs out there
(including http.ServeMux in the standard library), to see how they usually
behave.

In some places there are more lines of code than you need.

So

// get data and fill template
var data interface{}
data = json2map(url)

if data == nil {
  tpl.ExecuteTemplate(w, page, nil)
} else {
  tpl.ExecuteTemplate(w, page, data)
}

can be replaced by

// get data and fill template
data := json2map(url)
tpl.ExecuteTemplate(w, page, data)

Less code makes it easier for another person to read and see what is happening.

Enjoy Go, and best of luck…

1 Like

Thank you for your advices. Implemented.

I have tried to understand how mux works outside the “hello world”, but never found a example where maybe 1000 endpoints are used. Below I faked an example of how I did understand it. In the long run this will be hard to maintain. Please correct me if I got it wrong.

My calculation of endpoints is based on 100 tables * CRUD = 400 endpoints + some extra endpoints.

r.HandleFunc("/books/{title}", CreateBook).Methods("POST")
r.HandleFunc("/books/{title}", ReadBook).Methods("GET")
r.HandleFunc("/books/{title}", UpdateBook).Methods("PUT")
r.HandleFunc("/books/{title}", DeleteBook).Methods("DELETE")
r.HandleFunc("/books/{title}", CreateBook).Methods("POST")
r.HandleFunc("/books/{title}", ReadBook).Methods("GET")
r.HandleFunc("/books/{title}", UpdateBook).Methods("PUT")
r.HandleFunc("/books/{title}", DeleteBook).Methods("DELETE")
r.HandleFunc("/books/{title}", CreateBook).Methods("POST")
r.HandleFunc("/books/{title}", ReadBook).Methods("GET")
r.HandleFunc("/books/{title}", UpdateBook).Methods("PUT")
r.HandleFunc("/books/{title}", DeleteBook).Methods("DELETE")
r.HandleFunc("/books/{title}", CreateBook).Methods("POST")
r.HandleFunc("/books/{title}", ReadBook).Methods("GET")
r.HandleFunc("/books/{title}", UpdateBook).Methods("PUT")
r.HandleFunc("/books/{title}", DeleteBook).Methods("DELETE")
r.HandleFunc("/books/{title}", CreateBook).Methods("POST")
r.HandleFunc("/books/{title}", ReadBook).Methods("GET")
r.HandleFunc("/books/{title}", UpdateBook).Methods("PUT")
r.HandleFunc("/books/{title}", DeleteBook).Methods("DELETE")
r.HandleFunc("/books/{title}", CreateBook).Methods("POST")
r.HandleFunc("/books/{title}", ReadBook).Methods("GET")
r.HandleFunc("/books/{title}", UpdateBook).Methods("PUT")
r.HandleFunc("/books/{title}", DeleteBook).Methods("DELETE")
r.HandleFunc("/books/{title}", CreateBook).Methods("POST")
r.HandleFunc("/books/{title}", ReadBook).Methods("GET")
r.HandleFunc("/books/{title}", UpdateBook).Methods("PUT")
r.HandleFunc("/books/{title}", DeleteBook).Methods("DELETE")
+ another 1000 of endpoints...

Therefore I am trying to create the mux as dynamic endpoints. Is there any other maintainable way to do this?

Have a look at https://github.com/gorilla/mux
which does exactly what you need…
(look at mux.Vars)
You will only need to define 4 endpoint handlers,
with {title} specified as a var.

I have found no example how to implement this. The closest is this, but I do not get the point. https://opensource.com/article/18/8/http-request-routing-validation-gorillamux

I may have only 4 endpoints, but 1000 handlers? Is there any better example than I found?

Someting like this?

import (
	"net/http"

	"github.com/gorilla/mux"
)

func readBook(w http.ResponseWriter, r *http.Request) {
	vars := mux.Vars(r)
	title  := vars["title"]
	book, err := getBook(title)
	if err != nil {
		http.NotFound(w, r)
		return
	}
    json.NewEncoder(w).Encode(book)
}

func main() {
	r := mux.NewRouter()
	r.HandleFunc("/books/{title}", createBook).Methods("POST")
	r.HandleFunc("/books/{title}", readBook).Methods("GET")
	r.HandleFunc("/books/{title}", updateBook).Methods("PUT")
	r.HandleFunc("/books/{title}", deleteBook).Methods("DELETE")
	http.ListenAndServe(":80", r)
}

And for the other 99 tables? How do I handle them? Not just “books” but “pens”, “glasses”,“flowers” etc. Endpoints and handler for ONE table is crystal clear to me. It is when dealing with 100 SQL tables that I need to understand.

Oh, I see…

How about something like

package main

import (
	"encoding/json"
	"net/http"

	"github.com/gorilla/mux"
)

func readObj(w http.ResponseWriter, r *http.Request) {
	vars := mux.Vars(r)
	objType := vars["objtype"]
	title := vars["title"]
	obj, err := getObj(objType, title)
	if err != nil {
		http.NotFound(w, r)
		return
	}
	json.NewEncoder(w).Encode(obj)
}

func main() {
	r := mux.NewRouter()
	r.HandleFunc("/{objtype}/{title}", createObj).Methods("POST")
	r.HandleFunc("/{objtype}/{title}", readObj).Methods("GET")
	r.HandleFunc("/{objtype}/{title}", updateObj).Methods("PUT")
	r.HandleFunc("/{objtype}/{title}", deleteObj).Methods("DELETE")
	http.ListenAndServe(":80", r)
}

This of course does just move the problem of choosing which table to access to getObj()
But it keeps your http handlers simple and idiomatic…

Thank you! Now it seems a bit clearer. I must work hands on in order to get it fully.

But I think it may differ between Client and API. As I can see there is more need for this in the API. Or am I wrong?

If I understand your code correctly, the REST API and the HTML Client basically do the same thing.
But the former serializes the data into JSON, while the latter renders it into a html template.

So if we extend my toy example, we could just replace the json Encode line with something like
tpl.ExecuteTemplate(w, objType, obj)

My own personal approach is just to do the API, and then instead of writing a html client, write a front end in JS which talks to the API to render pages in the browser. This gives you a nice separation of the
presentation layer from the back end - at the cost of having to mess around with javascript.