Question on duplication about monstache HOT 2 CLOSED

The default mapping is as follows:

mongodb database -> elasticsearch index
mongodb collection -> elasticsearch type
mongodb document _id -> elasticsearch document id

Any operation on a mongo document (insert, update, delete) will be recorded in the oplog with it's database, collection, and _id and that individual operation will eventually be folded into a bulk index command to ES (with other operations) where the index, type and _id are generated according to the mapping above. That is how it manages to index, update, or delete the correct ES document.

However, I think you may be correct about the potential for collisions since mongo _ids are unique at the collection level (not guaranteed across the database) while ES ids must be unique at the index level.

According to link, duplicate _ids across the database are unlikely with autogenerated _ids.

Regardless, I should change the default mapping such that

elasticsearch index <- mongodb database . mongodb collection

Thanks for giving monstache a try and reporting this issue.

from monstache.

Awesome, thanks. 👍

from monstache.