How to make this concurrent

Hi,
Basically I have a slice called a and i’m looking for each of its values it it is reachable or not

this is what I was trying to do:

func check(a []string){

    var subdom []string

	for i := 0; i < len(a); i++ {
		_, error := net.LookupIP(a[i])
    		if error == nil {
        		subdom = append(subdom, a[i])
        		fmt.Println(subdom)
    		} 
	}

}

Is there any way I can make that concurrently? If yes how? I’m newbie in concurrency. Hope you can help,
Thanks in advantage :slight_smile:

You could turn the body of the for loop into a goroutine. But you would have to change it so that such a goroutine does not append to subdom because this could happen concurrently.

Instead, create an output channel in check, pass it to each goroutine started in the loop, write a[i] to this channel if error == nil instead of appending it to subdom, read len(a) strings from the channel in check after the for loop, append each read string to subdom.

Do you get the idea?

Let’s say partially, I have some questions:

  1. If i do not have the for loop how can I use a[i] (if i don’t have for i := 0 etc.)
  2. I didn’t get read len(a) strings from the channel in check after the for loop, append each read string to subdom
  3. should I write 3/4 functions named with different name that do contain and call them in the main as go routines?
_, error := net.LookupIP(a[i])
    		if error == nil {
        		subdom = append(subdom, a[i]) //I know this has to be changed
        		fmt.Println(subdom)
    		} 

Thanks

Let me illustrate my suggestion with a little example:

package main

import (
	"fmt"
)

func main() {
	// The data to process.
	as := []string{"foo", "bar", "baz"}

	// The collected results.
	bs := []string{}

	// The channel to pass results from a goroutine to this main func.
	results := make(chan string)

	// For every piece of data ...
	for _, a := range as {
		// ... start a goroutine with the piece of data and the results channel as argumens.
		go func(a string, results chan string) {
			fmt.Printf("checking %q ... ", a)

			// Let's assume that the check was successful, we write the checked piece of data to the results channel.
			results <- a

			fmt.Println("done")
		}(a, results) // <-- `a` must be an arument!
	}

	// After all goroutines have been started, we wait for each of them to return a result.
	for i := 0; i < len(as); i++ {
		// We read a result from the results channel and append it to the collected results.
		bs = append(bs, <-results)
	}

	fmt.Printf("bs: %q\n", bs)
}

Output:

checking "baz" ... done
checking "bar" ... done
checking "foo" ... done
bs: ["baz" "bar" "foo"]

Note that the order of the checking lines is not fixed!

see https://play.golang.org/p/cnQAUM5NAEq

2 Likes

Thanks a lot man, the order is not important.

1 Like

sorry just last thing since i’m doing if err == nil isn’t the for waiting for something that will not come back?

go func(a string, results chan string) {
			_, error := net.LookupIP(a)
    		if error == nil {
        		results <- a
        		fmt.Println(a)
    		}
		}(a, results)

Good point!

I’ve reworked my example to handle this.

package main

import (
	"fmt"
	"strings"
	"sync"
)

func main() {
	// The data to process.
	as := []string{"foo", "bar", "baz"}

	// The collected results.
	bs := []string{}

	// The channel to pass results from a goroutine to this main func.
	// We give this channel a size so the goroutines can write to it
	// before `main` starts to read from it. Else we would get a deadlock.
	results := make(chan string, len(as))

	// We createa a WaitGroup that will allow us to wait for all
	// started goroutines to finish.
	var wg sync.WaitGroup

	// For every piece of data ...
	for _, a := range as {
		// Increment the WaitGroup since we will start a goroutine.
		wg.Add(1)

		// ... start a goroutine with the piece of data.
		go func(a string) {
			// At the end of this goroutine, decrement the WaitGroup.
			// This will indicate that this goroutine has finished.
			defer wg.Done()

			// We simulate a check that can fail: `a` must start with 'b'.
			// We write the checked piece of data to the results channel.
			if strings.HasPrefix(a, "b") {
				fmt.Printf("OK   %q\n", a)
				results <- a
			} else {
				fmt.Printf("FAIL %q\n", a)
			}
		}(a) // <-- `a` must be an argument!
	}

	// After all goroutines have been started, we wait for all
	// of them to finish.
	wg.Wait()

	// We close the results channels since all goroutines have
	// written their result, if any.
	close(results)

	// We drain the results channel ..
	for a := range results {
		// ... and append each returned piece of data to the collected results.
		bs = append(bs, a)
	}

	fmt.Printf("bs: %q\n", bs)
}

Output:

OK   "baz"
FAIL "foo"
OK   "bar"
bs: ["baz" "bar"]

see https://play.golang.org/p/c5a5BwRrI3r

1 Like