Parallel transactions for Gocolly 'do not work properly

I try to pull the data of a forum with the logic on this page. In this way, I can pull the data in the pages without any problems. But when I create a multi-threading thread for the For loop, the program will exit some time after pulling the data and cannot pull all the data.
Expample : If there are 300 data in total, it will dig 120 data and end the program.

I waiting your solution suggestions. Thank You For Advance

Colly sample:

My Codes Steps :

1- Select urls to be scanned from the database
2- Execute “c.Visit (URL)” in for loop
3 - Scrap with “c.OnHtml”

My Codes Sample:

	counter := 0
	resultsLen := len(results)
	var wg sync.WaitGroup
	for _, result := range results {
		go func(result primitive.M) {
			defer wg.Done()
			mainForumUrl := result["url"].(string)
			fmt.Println("##### Looking Main Forum Url #####",counter,":",mainForumUrl)


This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.