conduitio / conduit-connector-generator Goto Github PK

Feature description

Currently, if the generator is restarted, it will start generating the data from scratch (e.g. if it's configured to generate 100 records, you restart it after the 30th, then it will start generating from record 1, and the end result will be 130 records).

It would be good to make the generator source resumable, when the number of is defined (since it's also possible to generate records without a limit on the number).

Feature description

This is needed to support ConduitIO/streaming-benchmarks#6.

Feature description

The current approach with read time won't exactly produce the number of messages per second. It will always sleep for the specified time before producing the record, regardless of how much time elapsed since the last time the connector produced a record. Using something like golang.org/x/time/rate would make this more accurate. Another improvement we could do if/when we have time.

-- @lovromazgon

Feature description

Currently, all generated records have the operation set to created.

It would be good if we had the option to specify an operation or if the generator chooses a random operation.