Everything is fine :) about monstache HOT 24 CLOSED

Try removing fail-fast = true since any error will cause the process to exit.

from monstache.

It doesn't make any error, the "commited" and "index" numbers stay at 0 (or 1?), and the "flushed" number keeps growing and growing....

from monstache.

I think the issue is because I used my mongoDB database for a few weeks without having Monstache turned on. But now, I can't make it work anymore. Is there a log file I need to delete?

from monstache.

direct-read-namespaces = ["profiles.Profile", "profiles.Blogs"] is the way to go if you need to backup those entire collections. I'm not sure why monstache would stop syncing changes unless the process is exiting or getting killed. It's not meant to exit unless that fail-fast is on.

from monstache.

It doesn't exit after the backup is done, it just continues, but it doesn't sync if I make changes to the MongoDB database...

from monstache.

to answer your question there is no log file, but monstache keeps the last indexed timestamp in the monstache.monstache collection. It only uses that if resume=true. It goes back to the beginning of the oplog is your run with replay=true.

from monstache.

What happens when you remove those javascript functions? Does the data get sent then? You can use a test.test collection and see what happens.

from monstache.

resume=true and replay=true won't work because I'm using dynamic indexes

from monstache.

let me test that

from monstache.

The 2 don't have anything to do with one another. Resume and replay just tell it when to start tailing the oplog from.

from monstache.

--resume --replay gives me this error: (I thought it's because it cannot drop documents because of dynamic indexes)

INFO 2018/03/10 15:00:00 Successfully connected to MongoDB version 3.6.2
INFO 2018/03/10 15:00:00 Successfully connected to Elasticsearch version 6.2.1
INFO 2018/03/10 15:00:00 Entering event loop
ERROR 2018/03/10 15:00:01 Bulk index request with execution ID 1 has 1 line fail
ure(s)
ERROR 2018/03/10 15:00:01 Failed request line #0 details: {"index":"profiles","
type":"_doc","_id":"5a56a5a9bdaa0edddc9d374c","status":409,"error":{"type":"vers
ion_conflict_engine_exception","reason":"[_doc][5a56a5a9bdaa0edddc9d374c]: versi
on conflict, current version [6531396877726777344] is higher or equal to the one
provided [6522160486132219905]","index":"profiles"}}
STATS 2018/03/10 15:00:30 {"Flushed":30,"Committed":1,"Indexed":1,"Created":0,"U
pdated":0,"Deleted":0,"Succeeded":0,"Failed":1,"Workers":[{"Queued":0,"LastDurat
ion":0},{"Queued":0,"LastDuration":0},{"Queued":0,"LastDuration":0},{"Queued":0,
"LastDuration":0},{"Queued":0,"LastDuration":0},{"Queued":0,"LastDuration":0},{"
Queued":0,"LastDuration":0},{"Queued":0,"LastDuration":0},{"Queued":0,"LastDurat
ion":1000000},{"Queued":0,"LastDuration":0}]}

from monstache.

That is fine. Those errors are meant to happen because ES prevents you from overwriting new data with old data. That doc already exists in your ES.

from monstache.

I'm not sure you get me: The doc exists, but won't sync!

from monstache.

with --resume --replay but without the scripts I get this:

INFO 2018/03/10 15:05:21 Successfully connected to MongoDB version 3.6.2
INFO 2018/03/10 15:05:21 Successfully connected to Elasticsearch version 6.2.1
INFO 2018/03/10 15:05:21 Entering event loop
ERROR 2018/03/10 15:05:22 elastic: bulk processor "monstache" failed but may ret
ry: json: error calling MarshalJSON for type time.Time: Time.MarshalJSON: year o
utside of range [0,9999]
ERROR 2018/03/10 15:05:22 elastic: bulk processor "monstache" failed but may ret
ry: json: error calling MarshalJSON for type time.Time: Time.MarshalJSON: year o
utside of range [0,9999]
ERROR 2018/03/10 15:05:24 elastic: bulk processor "monstache" failed but may ret
ry: json: error calling MarshalJSON for type time.Time: Time.MarshalJSON: year o
utside of range [0,9999]
ERROR 2018/03/10 15:05:27 elastic: bulk processor "monstache" failed but may ret
ry: json: error calling MarshalJSON for type time.Time: Time.MarshalJSON: year o
utside of range [0,9999]
ERROR 2018/03/10 15:05:31 elastic: bulk processor "monstache" failed but may ret
ry: json: error calling MarshalJSON for type time.Time: Time.MarshalJSON: year o
utside of range [0,9999]
ERROR 2018/03/10 15:05:40 elastic: bulk processor "monstache" failed: json: erro
r calling MarshalJSON for type time.Time: Time.MarshalJSON: year outside of rang
e [0,9999]
ERROR 2018/03/10 15:05:40 Bulk index request with execution ID 1 failed: json: e
rror calling MarshalJSON for type time.Time: Time.MarshalJSON: year outside of r
ange [0,9999]
ERROR 2018/03/10 15:05:40 elastic: bulk processor "monstache" failed but may ret
ry: json: error calling MarshalJSON for type time.Time: Time.MarshalJSON: year o
utside of range [0,9999]
ERROR 2018/03/10 15:05:41 elastic: bulk processor "monstache" failed but may ret
ry: json: error calling MarshalJSON for type time.Time: Time.MarshalJSON: year o
utside of range [0,9999]
ERROR 2018/03/10 15:05:42 elastic: bulk processor "monstache" failed but may ret
ry: json: error calling MarshalJSON for type time.Time: Time.MarshalJSON: year o
utside of range [0,9999]
ERROR 2018/03/10 15:05:44 elastic: bulk processor "monstache" failed but may ret
ry: json: error calling MarshalJSON for type time.Time: Time.MarshalJSON: year o
utside of range [0,9999]
INFO 2018/03/10 15:05:45 Shutting down
ERROR 2018/03/10 15:05:48 elastic: bulk processor "monstache" failed but may ret
ry: json: error calling MarshalJSON for type time.Time: Time.MarshalJSON: year o
utside of range [0,9999]

from monstache.

So you are saying that the doc in ES is old (not in sync) but it has a version with a number higher than the most recent timestamp in the MongoDB oplog? When you update the doc in MongoDB the timestamp in the oplog will go up, the version sent to ES will match that, and it should be greater than what is already in ES, thus not fail due to version error.

from monstache.

Now the ES is new, because every time I make a test, I delete the whole elasticsearch namespace, and recreate the mappings.

from monstache.

Right now, my issue is that when I make a change to the MongoDB database, it doesn't get "committed" or "indexed" at the next cycle. And there is no error. All numbers except "flushed" stay the same. What can it be?

from monstache.

Those errors above seem to indicate it’s failing to Marshall the request to Elasticsearch.

from monstache.

yeah, I found out what these errors were, but it doesn't solve my main issue.
These errors come because I have some Date(8640000000000000) in my database (which is the maximum value allowed in Javascript). I don't think monstache likes that.

(But these dates are removed by my dynamic scripts / so that's why the error only appears when I remove the scripts)

from monstache.

I also found out why I was getting this error:

ERROR 2018/03/10 15:00:01 Failed request line #0 details: {"index":"profiles","
type":"_doc","_id":"5a56a5a9bdaa0edddc9d374c","status":409,"error":{"type":"vers
ion_conflict_engine_exception","reason":"[_doc][5a56a5a9bdaa0edddc9d374c]: versi
on conflict, current version [6531396877726777344] is higher or equal to the one
provided [6522160486132219905]","index":"profiles"}}

I found out I get this error every time I send DELETE _all, set the mapping and start monstache too quickly.
I learned that I need to wait at least 30 seconds after DELETE _all before starting monstache

from monstache.

Ok I fixed my main issue!

I still don't know what the issue was. But I fixed it by mongodump and mongorestore my whole database. I also added --master to the mongoDB Windows service path (so that it runs as --master at all time).

If I run the rs.printReplicationInfo() on mongo, I see the following:

configured oplog size:   990MB
log length start to end: 1313secs (0.36hrs)
oplog first event time:  Tue Feb 13 2018 16:38:02 GMT-0500 (Est)
oplog last event time:   Tue Feb 13 2018 16:59:55 GMT-0500 (Est)
now:                     Sat Mar 10 2018 17:46:50 GMT-0500 (Est)

I don't think it is normal that the oplog last event time is Feb 13.

EDIT: I think I found something here.

from monstache.

Glad you figured it out! And useful info if someone else has the same problem. Thanks.

from monstache.

@rwynn if the mongodb have some updates to a existing record then will monstache make those changes happen or do we need to do a collector restart direct-read-namespaces added to the config file?

from monstache.

Monstache syncs all changes it detects while it is running. Just run it without any configuration beyond connection url.

Once you start adding filters then that is where starts ignoring changes if they do not match. You need to restart if you want to change filters.

Does that answer your question?

from monstache.