Net/textproto download file from faulty server

Getting Help Code Review
1

Hi there,
I have a project where i need to import/download files from a IOT device.

As simple as this sounds, I can’t download all the files I want to.

First what do I do ?

I use the package net/http and simply download the content. I removed all non necessary code from the example. So straight forward:

import (
	"io"
	"net/http"
	neturl "net/url"
)

func readFile() ([]byte, error)  {
	// build the url, actually they are passed into the function
	var url string
	var filename string
	filename = "filename"
	url = "http://192.168.0.15/?download=" + neturl.QueryEscape(filename)

	// open connection
	res, err := http.Get(url)
	if err != nil {
		// may log  error, return error
		return []byte(""), err
	}
	// remember to close the file reader
	defer func(Body io.ReadCloser) {
		err := Body.Close()
		if err != nil {
			// may log  error
		}
	}(res.Body)
	// read/download the file content
	content, err := io.ReadAll(res.Body)
	if err != nil {
		// may log  error, return error
		return []byte(""), err
	}
	return content, nil
}

So this is actually working fine… better it was.

More info

  • If I build the project with go1.18.10 all works fine.
  • If I build the project with go 1.20.5 I sometimes get an error
  • Reason: the web server of the IOT device

The IOT device sometimes creates broken filenames with non utf8 letters in the name.
The web server on the device puts these names into a http response header, which leads to invalid Content-Disposition headers.

A web browser like Chromium of Firefox will ignore these invalid headers and download the file just fine.

The error and where it comes from

example Error Message:

Get "http://192.168.0.15/download?download=A_123_%7D%7F%FE%EF%EF%FE%E0%FE_0_xyz.txt": net/http: HTTP/1.x transport connection broken: malformed MIME header line: Content-Disposition: attachment; filename=A_123_}������_0_xyz.txt

As you can see in the request url, the filename contains invalid characters. Unfortunately I can not change the software on the IOT device to fix these malformed headers.

On the client side, the information I found is the error comes from the net/textproto package, which was updated with go1.20

So currently I’m looking for a way to download these files even if this header is broken. May someone can point me in the right direction.

Thanks for your help!