I have an issue with goroutines

[otodorov (Ognyan Todorov)]

Hello guys,
I’m new and still learning the Go language.
I have a script that is used to download multiple git repositories. To speed up this process I’m using a goroutine. I created another script that is short in code to simulate my problem easily.
So here you can see the script that creates multiple folders and subfolders while waiting a random time in each goroutine.

package main

import (
	"fmt"
	"math/rand"
	"os"
	"strconv"
	"sync"
	"time"
)

const (
	xthreads    = 5  // Total number of threads to use, excluding the main() thread
	folderCount = 50 // Maximum number of folders that has to be created
)

func doSomething1(a int) {
	fmt.Println("doSomething1:", a)

	if err := os.Mkdir("testDir"+strconv.Itoa(a), 0755); err != nil {
		fmt.Println(err)
	}
}

func doSomething2(a int) {
	fmt.Println("doSomething2:", a)

	if err := os.Chdir("testDir" + strconv.Itoa(a)); err != nil {
		fmt.Println(err)
	}

	if err := os.Mkdir("anotherDir", 0755); err != nil {
		fmt.Println(err)
	}

	if err := os.Chdir(".."); err != nil {
		fmt.Println(err)
	}
}

func main() {
	var ch = make(chan int)
	var wg sync.WaitGroup
	var dir string
	var err error

	if dir, err = os.MkdirTemp(os.TempDir(), "test-"); err != nil {
		fmt.Println(err)
	}
	if err = os.Chdir(dir); err != nil {
		fmt.Println(err)
	}

	// This starts xthreads number of goroutines that wait for something to do
	wg.Add(xthreads)
	for i := 0; i < xthreads; i++ {
		go func() {
			for {
				a, ok := <-ch
				if !ok { // if there is nothing to do and the channel has been closed then end the goroutine
					wg.Done()
					return
				}

				doSomething1(a)
				doSomething2(a)
				rand.Seed(time.Now().UnixNano())
				randomSleep := time.Duration(rand.Intn(10))
				time.Sleep(randomSleep * time.Second)
				fmt.Printf("Sleep for %d second...\n", randomSleep)
			}
		}()
	}

	// Now the jobs can be added to the channel, which is used as a queue
	for i := 0; i < folderCount; i++ {
		ch <- i // add i to the queue
	}

	close(ch) // This tells the goroutines there's nothing else to do
	wg.Wait() // Wait for the threads to finish
}

What I want to achieve is:

1. For example only 5 goroutines should be executed at the same time xthreads
2. When one of them has finished, another one has to start so I have to have 5 goroutines running at the same time until the folderCount value is reached
3. The folder structure should look like this:

/tmp/test-2325124430
├── testDir0
│   └── anotherDir
├── testDir1
│   └── anotherDir
├── testDir2
│   └── anotherDir
├── testDir3
│   └── anotherDir
└── testDir4
    └── anotherDir
...................
...................
└── testDir50
    └── anotherDir

and so on…

But the problem is that when I start the script it creates the folders like this:

/tmp/test-2325124430
├── anotherDir
├── testDir0
│   └── anotherDir
├── testDir1
├── testDir2
│   └── anotherDir
├── testDir3
│   └── anotherDir
└── testDir4
    └── anotherDir
...................
/tmp
├── testDir16
├── anotherDir
│   └── testDir17
...................
/
├── testDir18
├── testDir19
│   └── testDir20
...................

The console output is:

go run -race test.go
doSomething1: 3
doSomething1: 1
doSomething2: 3
doSomething2: 1
doSomething1: 4
doSomething1: 0
doSomething2: 0
doSomething1: 2
doSomething2: 2
doSomething2: 4
chdir testDir1: no such file or directory
Sleep for 1 second...
doSomething1: 5
doSomething2: 5
Sleep for 0 second...
doSomething1: 6
doSomething2: 6
Sleep for 4 second...
doSomething1: 7
doSomething2: 7
Sleep for 5 second...
doSomething1: 8
doSomething2: 8
Sleep for 7 second...
doSomething1: 9
doSomething2: 9
Sleep for 3 second...
doSomething1: 10
doSomething2: 10
Sleep for 1 second...
doSomething1: 11
doSomething2: 11
Sleep for 9 second...
doSomething1: 12
doSomething2: 12
Sleep for 9 second...
doSomething1: 13
doSomething2: 13
Sleep for 9 second...
doSomething1: 14
doSomething2: 14
Sleep for 7 second...
doSomething1: 15
doSomething2: 15
Sleep for 7 second...
doSomething1: 16
Sleep for 6 second...
doSomething2: 16
doSomething1: 17
doSomething2: 17
chdir testDir17: no such file or directory
Sleep for 8 second...
doSomething1: 18
doSomething2: 18
Sleep for 6 second...
doSomething1: 19
doSomething2: 19
Sleep for 6 second...
Sleep for 7 second...
Sleep for 6 second...
Sleep for 7 second...
Sleep for 8 second...

[skillian (Sean Killian)]

os.Chdir changes the directory of the whole process; it is not goroutine- (or “thread-”) specific.

Instead of changing directories in your goroutines, keep track of the path(s) you want each goroutine to use and use the paths directly; don’t change directory.

[otodorov (Ognyan Todorov)]

Yes, I’m sure that If I change os.Chdir to something tracking the whole path will work for sure. Didn’t know that os.Chdir has that behavior, it isn’t mentioned in the doc.

Thanks