Returning the correct io.Reader to main function

Getting Help
1

I’m building my own docker client, and I stumble on something very basic, I think. It might be of my own shaky understanding of how GO handles IO streams, or what not… I’m assuming I’m missing something very basic that you guys can help me with:

I’m writting the equivalent of docker load here, where I can handle multiple formats of tarballs (tar.gz, tar, tar.bz2, tar.xz). I’m having issues on how I deal with those multiple formats.

The main “loader” function reads as this (for now) :

func LoadImage(filepaths []string) {
	ctx := context.Background()
	cli := hosts.Connect2daemon(true)

	for _, filepath := range filepaths {
		tarballFile, err := os.Open(filepath)
		if err != nil {
			fmt.Println("Opening tarball file error: ", err)
			os.Exit(-9)
		}
		defer tarballFile.Close()

		tarballFormatReader := getTarballCompressedFormatReader(filepath, tarballFile)
		if tarballFormatReader == nil {
			fmt.Println("Unknown file format, exiting")
			os.Exit(-10)
		}
		defer tarballFormatReader.Close()

		tarReader := tar.NewReader(tarballFormatReader)
		
	}
}

As you see, it calls on getTarballCompressedFormatReader, which reads like:

func getTarballCompressedFormatReader(tbname string, tbfile *os.File) io.Reader {
	if strings.HasSuffix(tbname, ".bz2") {
		return bzip2.NewReader(tbfile)
	}
	if strings.HasSuffix(tbname, ".xz") {
		reader, _ := xz.NewReader(tbfile)
		return reader
	}
	if strings.HasSuffix(tbname, ".gz") {
		reader, _ := gzip.NewReader(tbfile)
		return reader
	}
	if strings.HasSuffix(tbname, ".tar") {
		return tar.NewReader(tbfile)
	}
	return nil
}

The issue is on the on the defer tarballFormatReader.Close() line, where it complains of having no method for Close(). I do not get it, it should, doesn’t it ?

This is my first full-scale software in GO, so I might be missing some basic understanding…

2

That line is missing in your code…

You really should also return an error value…

I guess you are trying to call Close() on the returned value? That doesn’t have Close, that would explain the error.

io.Reader only has Read() IIRC, for close you need the io.ReadCloser type.

3

Thanks, @NobbZ , using io.ReadCloser() is something I do elsewhere in the app, I don’t know why I did not think of it.

I guess I’m stuck somewhere else, then. I’ll have a go at reverse engineer Docker’s code for ImageLoad() as I cannot figure how to seamlessly switch from a plain tarball to tar.gz, tar.bz2, tar.xz, etc, ie pick the right io.Reader at runtime.

I can either take the correct reader, or have a reader that implements Close(), not both, from my limited knowledge.

4

Not sure what you mean, but regular docker has to close all the readers as well, at least the file based ones.

So it might indeed open and close whatever it gets from opening the file, but only passes the reader parts around, and closes it back where it opened the file.

This is actually a good general practice, close stuff where you open it, and close exactly that, not something you get after processing it.

1 Like
5

I agree, and it’s one part of GO that I like very much: the defer statement that allows you to close whatever you’ve opened right under it was opened.

What I’m saying is that I do not find a compression-format-agnostic way to do docker load. I guess I’m biting more than I can chew right now as I obviously do not master io streams yet.

6

It just forwards the input as is to the demon.

1 Like
7

Yes, I went to that same file earlier today.

8

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.