go-pool - is a tiny library goroutine pools conceptually close to the thread pool in other languages.
The common gist is to limit concurrency and execute only a limited number of the tasks at the time.
Goroutines are well done and appropriate for many tasks, but sometimes it is an issue. The run immediately after calling 'ao' might reduce the availability of the other resources like DB for instance.
There is an issue
func doSomething(n int, connectionString string) {
for i := 0; i < n; i++ {
go func() {
db, err := sql.Open(connectionString)
// Something else here ...
defer db.Close()
// Something else here ...
}()
}
}
func main() {
// Your DB has died. May be...
doSomething(100500, "Does not matter")
// ...
}A proposed solution
import (
"context"
"github.com/tdv/go-pool"
)
func doSomething(p pool.Pool, n int, connectionString string) {
for i := 0; i < n; i++ {
p.Go(func() {
db, err := sql.Open(connectionString)
// Something else here ...
defer db.Close()
// Something else here ...
})
}
}
func main() {
ctx := context.Background()
p := pool.New(ctx, 5)
defer p.Stop()
// There is no problem.
// All will go through 5 DB connections.
// DB will be healthy and your mind too.
doSomething(p, 100500, "Does not matter")
// ...
}go get -u github.com/tdv/go-poolexample1
Description
The example demonstrates the normal work of the limited goroutine pool.
package main
import (
"context"
"fmt"
"sync"
"time"
"github.com/tdv/go-pool"
)
func main() {
ctx := context.Background()
ctx, cancel := context.WithTimeout(ctx, time.Second*10)
defer cancel()
p := pool.New(ctx, 5)
defer p.Stop()
wg := sync.WaitGroup{}
for i := 0; i < 20; i++ {
i := i
wg.Add(1)
p.Go(func(context.Context) {
defer wg.Done()
fmt.Println(i)
time.Sleep(time.Second * 2)
})
}
time.Sleep(time.Second)
wg.Wait()
}example2
Description
The example demonstrates the cancellation of the work. In spite of the expectation to see all 20 lines in the terminal there will be only 10 (2 first pieces / portions) and the others will be canceled.
package main
import (
"context"
"fmt"
"time"
"github.com/tdv/go-pool"
)
func main() {
ctx := context.Background()
ctx, cancel := context.WithTimeout(ctx, time.Second*10)
defer cancel()
p := pool.New(ctx, 5)
defer p.Stop()
for i := 0; i < 20; i++ {
i := i
p.Go(func(context.Context) {
fmt.Println(i)
time.Sleep(time.Second * 7)
})
}
time.Sleep(time.Second * 10)
}example3
Description
The example demonstrates how to decline all tasks in the queue manually. In spite of the expectation to see all 20 lines in the terminal there will be only 5 (first 5 tasks have been done before the pool is closed) and the others will be canceled.
package main
import (
"context"
"fmt"
"time"
"github.com/tdv/go-pool"
)
func main() {
ctx := context.Background()
p := pool.New(ctx, 5)
for i := 0; i < 20; i++ {
i := i
p.Go(func(context.Context) {
fmt.Println(i)
time.Sleep(time.Second * 2)
})
}
time.Sleep(time.Second * 1)
p.Stop()
}Goroutines in Go are really useful, but sometimes we need to solve some design issues. The proposed library is one of many solutions.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent