Truncate slice when reading yaml file to unmarshal it


(Pantelis Karamolegkos) #1

I am faced with the following issue:

I am using go-git to checkout a repo in memory.

I am then opening a file using the billy package and reading it in a byte slice (since it implements the io.Reader interface)

file, err := fs.Open("releases/redash.yaml")

	if err != nil {
		log.Fatalf(err.Error())
	}

	data := make([]byte, 10000)
	_, err = file.Read(data)
	if err != nil {
		log.Fatalf("ERROR reading file")
	}
	err = yaml.Unmarshal(data, &v)
	if err != nil {
		log.Fatalf("ERROR Unmarshaling yaml\n", err)
	}
	fmt.Println(string(data))
}

The above fails:

2020/02/06 12:58:45 ERROR Unmarshaling yaml
yaml: control characters are not allowed
exit status 1

From what I understand, this is due to the data buffer having extra / zero-valued characters which yaml tries to incorporate and throws error.

Given that billy.File will not allow me to use ioutil.ReadFile against it, is there a way to address this problem? (say by trimming data to the appropriate length? - I am unable to know in advance the EXACT file length so as to instantiate an appropriately sized buffer)


(Lutz Horn) #2

How do you know that? Which characters are they?


(Pantelis Karamolegkos) #3

If i fmt.Println(data) I get numerous 0 at the end.

I was also hinted towards this direction by this SO post.


(Lutz Horn) #4

file.Read will return the number of bytes read which you ignore. This information should help you to pass the appropriate slice of data to yaml.Unmarshal.


(Pantelis Karamolegkos) #5

Thanks.

Can you provide an example how to use fileRead. The link you posted to io.Reader does not include such a method (unless I am missed it)


(Pantelis Karamolegkos) #6

Nevermind, got it

	data := make([]byte, 10000)
	n, err := file.Read(data)
	data = data[:n]

(Jakob Borg) #7

You might enjoy ioutil.ReadFile or ioutil.ReadAll.