I was able to reproduce (go version go1.8 darwin/amd64). There are two things going on.
http.ResponseWriter waits for enough data to be written before flushing to the client. This can be triggered manually with http.Flusher.
The first write will trigger ResponseWriter.WriteHeader, which uses the first 512 bytes to figure out the Content-Type.
I have been able to get an immediate response with:
func Hello(w http.ResponseWriter, r *http.Request) {
text := [512]byte{}
for i := range text {
text[i] = 'b'
}
text[511] = 'c'
w.Write(text[:])
w.(http.Flusher).Flush()
out, err := exec.Command("sleep", "10").CombinedOutput()
fmt.Println(string(out))
if err != nil {
fmt.Println(err)
return
}
}
text is designed to be noticeable along with its ending.
The trick seems to be:
Write at least 512 bytes of data (511 does not give immediate display in the browser)
Flush the ResponseWriter after enough data has been written.
I tried circumventing the content type checker by setting the Content-Type header and calling w.WriteHeader() before writing “Hello!!”, but still encountered the delay. This was done both with and without flushing w.
Would the data be flushed to the client before the goroutine exit when the response size is less than 512 byte?
What make me confused is why go runtime didn’t do this before exec?
This is occurring because you are running a command and using combined output which waits for the command to be completed before continuing, so it’s a blocking call.
I haven’t really tested this and it might not be the best thing to do, but maybe you want something similar to this example which pipes any output from the running command and also doesn’t hold up the terminal.
func Hello(w http.ResponseWriter, r *http.Request) {
// Create a new command.
c := exec.Command("ls", "-la")
stdoutPipe, err := c.StdoutPipe()
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
// Start the command.
if err := c.Start(); err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
// Copy all output from the running command to stdout.
go func() {
io.Copy(os.Stdout, stdoutPipe)
if err := c.Wait(); err != nil {
panic(err)
}
}()
// Do stuff while the command is running.
fmt.Println("doing stuff")
}
In that example I’m only using an ls command so you can see how it prints the output, but you can try it with a sleep command and see that it shouldn’t be blocking.
Conclusion:
It’s not about go scheduler, it’s just http server behavior.
Flush will be called after Handler function returned or called by user manually.
After reading your responses and the original post again, I think I misunderstood what you wanted.
You expect running:
time curl http://localhost:8083/hello
to result in:
Hello!!curl http://localhost:8083/hello 0.01s user 0.01s system 39% cpu 0.056 total
but you get:
Hello!!curl http://localhost:8083/hello 0.01s user 0.01s system 0% cpu 10.066 total
The difference being the total time.
This can be accomplished by running the command in a goroutine:
func Hello(w http.ResponseWriter, r *http.Request) {
w.Write([]byte("Hello!!"))
go func() {
out, err := exec.Command("sleep", "10").CombinedOutput()
fmt.Println(string(out))
if err != nil {
fmt.Println(err)
return
}
}()
}
The command is started when the http request is received. Using a goroutine like this runs the command in the background so Hello does not block until the command completes.
I think this is the behavior you desired. Please tell me if I am wrong.