Standard REST server is rather simple to understand and get up and running, but a gRPC server that do the same thing seems to be complicated and full of dead ends as soon as you want to communicate with web and postgresql. And the knowledge about gRPC also seems limited. Or at least the knowledge how to inform in a simple way.
I have tried to understand how the pieces work together, but still not get the full picture. Is this correct?
My questions are:
Is it worth the time spent to replace standard REST with gRPC?
gRPC and RPC architecture in general provide more finite access control for micro-services to communicate directly rather than through JSON or the like. You use this with internal service communication not exposed to third parties typically. Whereas, REST services are generally geared towards public consumption by third party devs.
As @CurtGreen suggests, RPC is useful for internal service communication. Whereas REST is more typically used for client to service communication.
So for example, if you consider a basic microservice architecture with two services, a registration service and a email sending service.
The client (the website) would send a request to a REST endpoint of the registration service with an email and password, the registration service would attempt to create the user and if successful, the registration service could use RPC to tell the email service to send an email to the user who signed up.
I noticed recently that some of the more modern Google SDK’s seem to use RPC to speak to Google services.
Many claims that it is possible to use gRPC using gRPC-gateway to convert JSON to protobuf, in order to make the communication faster. But I wonder how many are using gRPC in conjuction with Web to communicate with a SQL database? What is your motivation and what are the benefits?
Many claims that it is possible to use gRPC using gRPC-gateway to convert JSON to protobuf, in order to make the communication faster. But I wonder how many are using gRPC in conjuction with Web to communicate with a SQL database? What is your motivation and what are the benefits?
I suppose it’s like any technology. Most technologies are pretty good and do what they do very well, it all comes down to use case.
You’d have to weigh up the for and against for your use case. If for example, your application only needs endpoints to be able to read and write to an SQL database, it might be that a simple JSON REST API is perfectly fine, without the added complexity of an RPC layer.
To go back to what I mentioned previously, If your application expands and you were to introduce multiple services then gRPC & Protocol Buffers would be a great candidate for internal service communication, but again it would come down to use case.
Protocol Buffers have some really compelling advantages over JSON particularly regarding types and serialisation speed, but having an RPC Gateway that I expect would (don’t quote me on this) deserialise the JSON in any event to then serialise to a Protocol Buffer, might be added overhead to your application where it isn’t necessary.
There are a ton of companies using gRPC at significant scale, to name a couple Google and Netflix. If you have a look around there are some interesting articles breaking down how companies use gRPC in their services.
If it is about binary communication between two services, I should use gRPC without doubt. It seems to be quite obvious. And I find it rather simple.
But my question was about the communication from web to postgresql. How many of you have used gRPC in the real world in this case and can tell me about your experiences?
I have searched in Google for weeks and found very few real world example of web to sql using gRPC.
I recently switched from REST to gRPC for my matchmaking site https://matchtowed.com. It now has an Angular client using TypeScript code to send gRPC-Web requests to an Envoy reverse proxy that passes on the requests to a gRPC Python server and a Redis database. You don’t need to send JSON that gets mapped on the server side, but have protobufs all the way from client to server, so there may not be the overhead you’re talking about. I’m currently re-writing it to use Flutter/Dart clients and a Go server, and switching to gRPC was also a transitional step. Yes, the process isn’t as easy/simple as with REST, but I believe it’s worth it in situations like mine.
Well, I tried to install Envoy, but you need Docker. But basically you need four (4) servers: web server, envoy server, gRPC server and finally the Postgresql server . Have you noticed significant speed enhancement with this setup?
Besides this, there is “analogue” communication from the gRPC-server to the Postgresql server??
Yeah, you’ll have quite a few new moving parts, config files that are super finicky, etc. so it can be pretty frustrating going from the simple client/server setup with REST. BUT, once you have it set up, it is actually liberating because things become modular, easy to develop/fix/publish in parts, etc. You know, we’re talking pros/cons of a microservices setup. E.g., now I’m changing the GRPC server from Python to Go without worrying about the rest of the setup, and I plan to run just the machine learning part (TensorFlow) in a separate Python service, etc. I.e., the whole thing becomes very flexible and like I said, liberating.
WRT performance, my messages between client/server are small, so I’m not sure there’s a tangible performance benefit, though the client does feel more snappy. One side benefit (which I like a lot) is that with GRPC you can no longer make sense of network requests/responses in a browser’s developer console, whereas with REST everything is typically out there in plain view.
Hey Sibert, can you expand in your question please? If I understand correctly, gRPC requires all communicating parties to use protocol buffers. I wasn’t aware you you could define/consume protobuf in dbs!
I am searching for answers how it works as well. Hence my question. IMHO there is not many that are using gRPC to replace a REST server. And if they do, they cannot explain in a simple way that I can understand.
I think Globular is exactly what you need https://github.com/davecourtois/Globular. It give you access to gRpc-web whiout pain. Binaries for Linux and Window are available and sevices like SQL, LDAP, SMTP, Persistence (MongoDB), Storage (levelDB, BigCache) are already available and tested. I made use of it at my work and It run like charm!
I encourage everybody to use it and also share new services!-)
gRpc made use of protobuffer for serialization vs JSON for REST. JSON is simple, easy to read, easy to parse, widely use. But in term of performance it’s far behind protobuffer. There is a lot of optimization in protobuffer for it to take less space by using binary format. Here is a link about JSON vs Protobuf, https://auth0.com/blog/beating-json-performance-with-protobuf.
The way you get your data (sql query) can be a factor in performance to… marshal/umarshaling cycle are very costly! Try to avoid them as much as you can. If you use SQL as backend for your JSON objects then you can consider switching to MongoDB, it is well define for that kind of use…
If you are currious you can try by yourself with Globular, there is a SQL service already there and also an exemple…
The performance increase will be only in the data transfer portion of your application. I think most of the gain can be done at the query optimization level. For example avoid string as key and prefer integer over it. Also avoid try to avoid LIKE operator as mutch as possible. Avoid dynamic query generation to, those kind of query can not be optimize by the database system. Create view instead of making complex query (with a lot of JOIN and CONDITION) that way the database system will be able to optimize your query. Does your user need all the data at once or TOP 1000 for say…
I have some knowledge about query optimization. That was not my question.
My question was how and when the benefits using gRPC arise?
The communication between the Web and the API --OR-- between the API and Postgresql?
Or exact where lies the benefits of using gRPC in a fetching data from a Postgresql server from a webpage? And is it worth the eventual performance increase?
Can you explain this further? Transfer what from and to?
If I look at the schema at top of the page I suppose you want to give access of SQL data to web-application. In both case, JSON/REST and gRPC, you need to connect your server to prosgre via a tcp/ip connection.
Here in case of rest api, the server will be the web-server (ngnix, iis, apache)
[postgre] <----- (tcp/ip) ---- [web server] <---- (Rest/Json) ----- [Browser]
