Handling big file uploads

Hello everyone,

I’m having problems handling big file uploads. I have the following code, which fails with an error bad file descriptor.

// Somewhere in the code...
const maxImageUploadSize   = 1 << 10 << 14 // ~16 MB
r.ParseMultipartForm(maxImageUploadSize)

// Somewhere else...
file, header, err := r.FormFile(formName)
if err != nil {
	logger.Error("Failed getting file from form", err)
	return nil, nil, 0, nil // return blank filename, but no error
}
defer file.Close()

fileSize, err := file.Seek(0, 2)
if err != nil {
	logger.Error("Failed getting file size from form", err)
	return nil, nil, 0, err
}

_, err = file.Seek(0, 0) // return the pointer to the beginning
if err != nil {
	logger.Error("Failed getting file size from form", err)
	return nil, nil, 0, err
}

// Then I handle the file, by moving it to a proper location
if err := os.MkdirAll(filepath.Dir(filePath), 0777); err != nil {
	logger.Error("Failed creating directory", err)
	return err
}

out, err := os.Create(filePath)
if err != nil {
	logger.Error("Failed creating destination file", err)
	return err
}
defer out.Close()

if _, err := io.Copy(out, file); err != nil {
	logger.Error("Failed saving file to directory", err)
	return err
}

My code fails on io.Copy (the last couple of lines) with the error: Failed saving file to directory read /tmp/multipart-716526200: bad file descriptor. It does work with files of less than 1 GB, but with bigger images it’ll always fail with this message.

Any clues?

Please build your program with the race detector and see if it reports a problem.

do you have a 32 bits machine ?

I will! I’ll post what I find out. Thanks!

No, it’s actually a 64-bit server with generous amounts of RAM, CPU and others, but running the Go App under a container.

i used io.Copy in a similar code and i transferred a 2GB file without problems. so, i suppose you have some limitations in that container.

I’ve used io.Copy to copy files in a backup utility, including virtual disk images much bigger than 4GB, with no problems.

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.