Mongo Replicate URL with SSL option Is Not Supported about monstache HOT 12 CLOSED

Can you try without that param? Just make sure you point to a pem file and set the validate pem accordingly.

from monstache.

Go Mongodb driver does not require ssl param. https://github.com/go-mgo/mgo/issues/84 Only custom dial function which will be triggered by pem file.

from monstache.

for official service like the MongoDB atlas， what pem file should be used?

from monstache.

Can you try creating a dummy pem file and then using that? Code currently requires a pem file to connect via TLS. But in case of Atlas a Pem might not be needed.

from monstache.

Hmm, I tried and I can see

INFO 2017/02/21 01:18:22 GET request sent to http://xxxxxxx.us-east-1.aws.found.io:9200/
INFO 2017/02/21 01:18:22 Successfully connected to elasticsearch version 2.3.3
INFO 2017/02/21 01:18:22 Joined cluster data_lake_test
INFO 2017/02/21 01:18:22 Starting work for cluster data_lake_test

but I didn't see the index created, any idea?

from monstache.

Do you know if the user connecting to Mongodb has access to the oplog? For atlas it seems you would need to give permission to access local.

https://docs.atlas.mongodb.com/faq/#does-mongodb-service-expose-the-oplog

Once it can read the oplog you would need to insert new document in Mongodb for index to be created. Or you can use option replay=true to send what is already in oplog.

from monstache.

I tried with replay=true and resume=true, it did create the index and sync some documents, but not all, which is wired.

from monstache.

Do you have multiple shards? I think I may need to make some fixes around this according to https://www.mongodb.com/blog/post/tailing-mongodb-oplog-sharded-clusters.

from monstache.

@rwynn no, just 1 shared.

from monstache.

@skyhit, what percentage of the documents was synced? Do you have workers options set (this could cause only some to sync because in worker mode each worker does a share of the indexing)? With -verbose flag you should see a log of each bulk request. Do you notice any failures?

from monstache.

@skyhit, I tested today on Atlas and did not notice any document loss. Just released a new version of monstache with SSL option so that you don't need to provide dummy PEM to connect to Atlas. Closing this issue. Please open a new one if you have additional problems.

from monstache.

@rwynn thanks, I will try with the new release.

from monstache.