A simple HTTP server crashing Linux network stack?

[Gopher0]

A very simple HTTP server, when packed into a Docker, crashes the Linux network stack a few seconds after launch.

It is the Go HHTP server that does this: when the block is commented, the Docker runs and no hang happens.

Here is the main.go

package main

import (
        "fmt"
        "log"
        "net/http"
)

func main() {
/*      http.HandleFunc("/hello", HelloHandler)
        fmt.Printf("Server running (port=8080), route: http://localhost:8080/hello\n")
        if err := http.ListenAndServe(":8080", nil); err != nil {
                log.Fatal(err)
        }*/
        log.Println("hello")
}

func HelloHandler(w http.ResponseWriter, r *http.Request) {
        fmt.Fprintf(w, "Hello, World!")
}

And here is the Dockerfile:

# ---- Build stage ----
FROM golang:1.23-alpine AS builder
RUN apk add --no-cache build-base
WORKDIR /app
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 GOOS=linux go build -o hello_go_http

# ---- Runtime stage ----
FROM alpine:3.20
# Install shell and any useful tools for debugging
RUN apk add --no-cache bash curl
WORKDIR /
# Copy binary from builder
COPY --from=builder /app/hello_go_http .
EXPOSE 8080
ENTRYPOINT ["/hello_go_http"]

The commands:

docker build -t hello_go .
docker run -p 8080:8080 --memory=256m --cpus=1  hello_go

[Dean_Davidson]

Where are your logs? What is the error?

[Gopher0]

Found almost nothing. Just the SSH connection dropped, after about 20s. It happened systematically.

Only found with a journalctl -u docker:

Handler for POST /v1.42/containers/57193...6ab/kill returned error: Cannot kill container: 57193...6ab: 
Cannot kill container 57193...6ab: 
unknown error after kill: 
runc did not terminate successfully: 
exit status 1: 
unable to signal init: 
permission denied\n: unknown

[Dean_Davidson]

I was able to run it fine locally. The note in the error about the container tells me this might be a docker issue of some kind. Not sure why it’s trying to kill that container.

Can you docker logs <container id>? Also - depending on your environment if you are trying to expose a port or something you could be running in to SELinux or something along those lines. But I think this is pretty clearly a Linux/Docker issue.

[Gopher0]

Yes, running the binary as is does not crash the network, so it is probably related to Docker.

OTOH, when commenting the block, the container runs fine, so it is related to the opening of the HTTP server.

docker logs say nothing, just the message “Server running (port=8080), …” as expected.

SELinux is not running on this Debian.

About exposing the port per se, the hang does not happen when commenting the block, even when exposing the port in the command

docker run -p 8080:8080 ...

(opening the port from the container, but not listening on it from the Go code)

[Dean_Davidson]

What happens if you do the opposite and run it with the web server running but don’t expose the port to the outside world (so remove -p 8080:8080)? Also - you could try this:

docker run -it --entrypoint=sh hello_go

And then manually run ./hello_go_http to see if you can get a better error message that way or otherwise see what is going on.

[Gopher0]

Thank you, you headed me in the right direction! Stripping down the container to a bare Alpine and launching it with your command also crashed.

The solution: as described here, edit /etc/connman/main.conf with the following:

[General]
NetworkInterfaceBlacklist=vmnet,vboxnet,virbr,ifb,docker,veth,eth,wlan

So it wasn’t a Go problem but Docker. Sorry.

=> SOLVED