Code Monkey home page Code Monkey logo

go-async's People

Contributors

rdleal avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar

go-async's Issues

Optimize code to send multiple payload

Hi,

Currently when sending multiple payload in async.Auto mode, for each payload, I need to create async.Auto. So trying to understand if there's any way we could do something like this.

auto, _ := sync.Auth(context.Background(), async.FuncMap{...}, Option...)
result, _ := auto.Process("Argument)
``

I've some comments and suggestions

Thanks for this awesome tool. I was looking for simple workflow tool like this.
I've some questions:

  • What will happen to the data, if one of the function throw error or unable to perform it's operation?
  • Is there any stat feature for total data in function map being processed for each functions?
  • Is individual function async by nature?

I tested this tool with this code and seems the performance is slow when using goroutine

type msg struct {
		Greet, Name string
	}

	fm := async.FuncMap{
		"reader":  strings.NewReader,
		"decoder": async.DependsOn("reader").To(utils.Json.NewDecoder),
		"parser": async.DependsOn("decoder").To(func(d *jsoniter.Decoder) (msg, error) {
			var m msg
			err := d.Decode(&m)
			return m, err
		}),
		"greet": async.DependsOn("parser").To(func(m msg) string {
			return strings.Title(m.Greet)
		}),
		"name": async.DependsOn("parser").To(func(m msg) string {
			return strings.Title(m.Name)
		}),
		"greeter": async.DependsOn("greet", "name").To(func(greet, name string) string {
			return fmt.Sprintf("%s, %s!", greet, name)
		}),
	}
	for i := 0; i < 100; i++ {
		go func(j int) {
			jsonStr := `{"greet": "hello", "name" : "gopher-` + strconv.Itoa(j) + `"}`
			start := time.Now()
			fnc, err := async.Auto(context.Background(), fm, async.WithArgs(jsonStr))
			if err != nil {
				log.Fatal(err)
			}

			for fn := range fnc {
				res, err := fn.Returned()
				if err != nil {
					log.Fatal(err)
				}
				fmt.Println(res[0]) // Hello, Gopher!
			}
			fmt.Println(fmt.Sprintf("%s", time.Since(start)))
		}(i)
	}

Result:

Hello, Gopher-100!
1.239477ms
Hello, Gopher-100!
28.85µs
Hello, Gopher-100!
1.329145ms
Hello, Gopher-100!
1.231549ms
Hello, Gopher-100!
23.32µs
Hello, Gopher-100!
1.287589ms
Hello, Gopher-100!
1.246905ms
Hello, Gopher-100!
29.173µs

Some suggestions:

  • Ability to use cache
  • Make it fault tolerant: Ability to restart from last function where error occurred or unable to process
  • Stats for each functions in a flow

For now I've been using this library to make it happen using Redis and Queue
https://github.com/hibiken/asynq

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.