Efficiently reading a file content


I have a json file that contains millions of records in given form. As you can guess reading the whole content won’t be memory efficient. How do we approach such problem in Go? For a demonstration purposes, I don’t mind seeing a solution where the struct is printed as the file content is read.



  "ID-1": {
    "username": "name-1",
    "password": "password-1"
  "ID-2": {
    "username": "name-2",
    "password": "password-2"
  "ID-3": {
    "username": "name-3",
    "password": "password-3"

user struct (Or some other struct if it simplifies the solution)

type User struct {
	ID       string `json:"id"`
	Username string `json:"username"`
	Password string `json:"password"`

It sounds like you need json.Decoder. You could write a function that decodes the tokens of your User struct and yields the results in a channel or a function.

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.