Main Program to exit gracefully after a goroutine kill

In effort to build a microservice, I have been able to come up this pattern where I am able to gracefully shutdown the server in case of issue using SIGTERM. In the below program, I also have a “testGoroutine” just for explanation. Assume this is important part of application and using context to gracefully exit the goroutines.

I also would like to be able to shutdown the main application gracefully in case of this goroutine produces error - close any open connections in main etc. My understanding is that I should not be executing log fatal inside goroutine

Other option I can think of sending a signal to liveness probe but this would make this example work only from K8s perspective

Any recommendations?

package main

import (
	"fmt"
	"log"
	"net/http"
	"os"
	"os/signal"
	"sync"
	"syscall"
	"time"

	"golang.org/x/net/context"
)

func main() {

	s := &http.Server{
		Addr:    ":8080",
		Handler: router(),
	}

	wg := &sync.WaitGroup{}
	wg.Add(1)

	tc, cancel := context.WithCancel(context.Background())

	go func() {
		if err := s.ListenAndServe(); err != nil && err != http.ErrServerClosed {
			log.Fatal("ListenAndServe:", err)
		}
	}()

	sigChan := make(chan os.Signal)
	signal.Notify(sigChan, os.Interrupt, os.Kill, syscall.SIGTERM, syscall.SIGINT)
	defer signal.Stop(sigChan)

	go func() {
		defer wg.Done()
		testGoroutine(tc)
	}()

	<-sigChan
	log.Println("terminate signal received")

	if err := s.Shutdown(tc); err != nil {
		log.Printf("Server shutdown error: %v", err)
	}
	cancel()
	wg.Wait()
}

func testGoroutine(ctx context.Context) {

	defer fmt.Println("testGroutine was exited")
	for {
		select {
		case <-ctx.Done():
			log.Println("Context has been cancelled")
			return
		default:
			for i := 0; i < 100; i++ {
				time.Sleep(1 * time.Second)
				log.Println("testGoroutine :", i)
				i++
				if i == 8 {
					log.Fatal("error")  // I am aware of that log.Fatal will immediately exit without running any defer
				}
			}
		}
	}
}

I have made a few changes to your code that achieves what you’re after. I’ve commented the key parts & differences, but just let me know if you want me to clarify further.

re: testGoroutine() func, I haven’t changed this as that function wouldn’t run in main. It would be a part of a handler downstream and that’s where the error should be handled.

func main() {

s := &http.Server{
	Addr: "localhost:8080",
}

//Declare vars, we want buffered channels here to immediately handle any errors
var (
	shutdown    = make(chan os.Signal, 1)
	serverError = make(chan error, 1)
)

tc, _ := context.WithCancel(context.Background())

//Launch server in a new routine, but publishing any errors into the serverError chan. If the server fails to launch, the program will exit immediately
go func() {
	log.Printf("http server listening on %v", s.Addr)
	serverError <- s.ListenAndServe()
}()

signal.Notify(shutdown, os.Interrupt, syscall.SIGTERM)

select {
case <-shutdown:
	log.Println("terminate signal received")
	s.Shutdown(tc) //if we have any active connections, s.Shutdown() will wait for them to close before shutting down.
	_ = s.Close()  //need to check this err. s.Close() differs from s.Shutdown() as this will close the listeners. We shouldn't have any connections due to s.Shutdown() stopping any new ones from being created.
case err := <-serverError:
	log.Printf("server error, unable to start: %v", err)
}}

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.