tasq's Introduction

tasq

Tasq is Golang task queue using SQL database for persistence. Currently supports:

PostgreSQL
MySQL

Install

go get -u github.com/greencoda/tasq

Usage Example

To try tasq locally, you'll need a local DB running on your machine. You may use the supplied docker-compose.yml file to start a local instance

docker-compose -f _examples/<example_repo_name>/docker-compose.yml up -d

Afterwards simply run the example.go file

go run _examples/<example_repo_name>/main.go

tasq's People

Contributors

Stargazers

Watchers

tasq's Issues

fatal error: concurrent map writes

Hello @greencoda , I use goroutine worker pool to process tasks concurrently. As follows:

func ConsumeTasks(ctx context.Context, c *tasq.Consumer, conf *config.RouterConfig) {
	logger.Info("Begin Consume Tasks")
	// Create an unbuffered (blocking) pool with a fixed number of workers
	pool := pond.New(10, 0, pond.MinWorkers(10), pond.Context(ctx))

	for {
		job := <-c.Channel()
		if job == nil {
			continue
		}

		pool.Submit(*job)

		select {
		case <-ctx.Done():
			logger.Info("consume tasks stop")
			return
		default:
			continue
		}
	}

	// unreachable code
	// pool.StopAndWait()
}

During operation, the following error will occasionally be reported:

fatal error: concurrent map writes


goroutine 149 [running]:
github.com/greencoda/tasq.(*Consumer).removeFromActiveTasks(...)
    /app/vendor/github.com/greencoda/tasq/consumer.go:338
github.com/greencoda/tasq.(*Consumer).registerTaskSuccess(0xc0004382c0, {0x10d1a88?, 0x1746dc0?}, 0xc0002886e0)
    /app/vendor/github.com/greencoda/tasq/consumer.go:312 +0x91
github.com/greencoda/tasq.(*Consumer).createJobFromTask.(*Consumer).newJob.func1()
    /app/vendor/github.com/greencoda/tasq/consumer.go:491 +0x93
github.com/alitto/pond.(*WorkerPool).executeTask(0xc000000e40, 0x0?, 0x0?)
    /app/vendor/github.com/alitto/pond/pond.go:454 +0x6c
github.com/alitto/pond.worker({0x10d1b30, 0xc0005527d0}, 0x0?, 0x0?, 0xc000403260, 0xc000556c00)
    /app/vendor/github.com/alitto/pond/worker.go:32 +0xc2
created by github.com/alitto/pond.(*WorkerPool).maybeStartWorker in goroutine 105
    /app/vendor/github.com/alitto/pond/pond.go:423 +0x133

Recommend Projects