Understanding the Limitation of Reusing File-based Request Bodies in Go's HTTP Client

Technical Discussion
1

Hello Go Contributors and Community,

I am working with the Go HTTP client and have encountered a scenario where I need to reuse a file as the body for multiple HTTP requests. Despite attempts to reset the file pointer using the Seek method, it appears that the file cannot be reused for subsequent requests without reopening it. This behavior occurs even when using the GetBody function in the http.Request structure to provide a fresh copy of the request body.

From this observation, I have several questions and am seeking insights:

Design Rationale : Is there a specific reason in Go’s HTTP client design that prevents the reuse of file-based request bodies, even when employing methods like Seek to reset the file’s read pointer?

File I/O and HTTP Client Interaction : How does the interaction between file I/O operations and the HTTP client’s request handling contribute to this limitation? Is it related to how the standard library treats file readers differently from other io.Reader implementations?

2

Go HTML templates (files)?

3

Is there a reason you’re not using http.ServeFile like so?

Also if the rub is you want them in memory you could cache them yourself or just use embed:

4

@Dean_Davidson @Sibert

In a simplified scenario i want to send the content of a file to a server, and if sending file failed, sending it should be retried. in my case file is big enough that can’t be loaded into memory, therefore it should be sent in chunks(transfer-encoding: chunked).

The issue is that, after first try, request.Body is closed by transport is closed and won’t be re-used.

5

You need to add a GetBody function to the Request

	// GetBody defines an optional func to return a new copy of
	// Body. It is used for client requests when a redirect requires
	// reading the body more than once. Use of GetBody still
	// requires setting Body.
	//
	// For server requests, it is unused.
	GetBody func() (io.ReadCloser, error)

From: http package - net/http - Go Packages

6

@ncw

Thanks for your response. As I mentioned in the question, GetBody is only used when request body type are of bytes.Reader or bytes.Buffer or strings.Reader but not file.
I do understand that when an stream is sent out via golang HTTP client, it is handled in chunks and it requires seeking to the beginning of the stream and meantime stream could changes, but i don’t see why this is conflicting with rewinding body?

7

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.