Incremental loading resets last_record field in the config file if no new record is uploaded about embulk-input-mongodb HOT 8 CLOSED

Hi. Thanks for your reporting.

Just wondering if this is a bug, a intended behavior

This is a intended behavior.
Other Embulk plugins like embulk-input-mysql works as same.
As you're saying it might better to remain original last_record in diff.yml when stored data=0.

I think workaround is use something workflow tools.
We're also developing digdag.

from embulk-input-mongodb.

from embulk-input-mongodb.

Hi. I'm engineer of Treasure Data.

1 - if this is the behaviour of the Treasure Data connector powered by
Embulk

Yes, Treasure Data Data Connector is using perfectly same Embulk plugin with OSS one.

I checked other plugin's behavior again.
embulk-input-(s3|mysql) works as you expectes. Plugin will output same last record(=before execution) when there're no diff since previous run.

Then, I think this behavior is a bug and need to be fixed.
I'll create another issue and try to fix it.

Thanks.

from embulk-input-mongodb.

@sakama Hi, any chance this will be fixed in a later release? Thank you!

from embulk-input-mongodb.

@letianw91 Sorry to keep you waiting.
I'll try to fix this issue next week.

from embulk-input-mongodb.

No worries at all! We've been using this plugin heavily and thank you for your work in maintaining this!

from embulk-input-mongodb.

This feature was released at v0.6.1.
I'll ask Treasure Data customer support team to notify you when it's available in TD.

Thanks.

from embulk-input-mongodb.

Hi, @letianw91
As @sakama commented, v0.6.1 was released.
Please reopen or create another issue if something is wrong to you, thank you.

from embulk-input-mongodb.