Hey there,
Check my new bot which crawls Bing search engine with dorks and keywords.
It’s multithreaded and you can limit the thread numbers.
it gives output on your localhost.
And it can reach the last page of bing search. ( You don’t have to give the page numbers to be crawled)
And it doesn’t need any external dependency packages
Here it is:
That’s nice, but be sure to build and run with the race detector (“go install -race”) and you will see some problems.
At a quick look, your “total” global variable to calculate the speed needs to be passed safely between goroutines…
Hello @Giulio_Iotti ,
I have tried my best to solve this 
But i couldn’t.
Can you help me?
Thank you!
Sure! You have two main problems to solve:
The first go routine that prints some advancement stats: it must read a channel in a select, together with the time ticker. The channel will receive for example a number (mostly just 1) when a new page is scanned. You can receive this number and sum it to the (local variable) counter of pages scanned. In the ticker case of the select, you print the seconds and total.
You write some files with a “shoot and forget” goroutine. The easy solution is not to use a goroutine. But be sure to check the returned error! Writing to a file often results in errors (mostly permissions…)
If you really want to use a goroutine, you will still have to wait for it to finish, or your software might finish before the write to disk finished, and you will have a half written file. Hint: you might do that with global a sync.WaitGroup…
PS: This stuff is hard. Don’t get frustrated, you’re writing a nice software. When you will master concurrency, you master everything in programming.
May be I get you 70%
Let me deal with it. I will try my best.
Can you please suggest some resources to master in concurrency.
Thanks in advance 
Thanks man!
Finally I have resolved the race conditions!
And I have removed all the unnecessary parts of the bot!
Thank you for your helpful comment!
I’ve learnt something new!