mongodb to elasticsearch removal strategy about elasticsearch-river-mongodb HOT 11 CLOSED

Hi Martin,

Script filter should work in your scenario. It has not been released yet but it is available from version 1.6.0-SNAPSHOT available here [1].
It has not been clearly documented yet but take alook to my comments to this issue [2].
Also the unit test give you an idea on how to remove a document (see [3]).
I believe ctx.deleted = true should delete the document from the index.

Please let me know if it works or not..

[1] - https://github.com/richardwilly98/elasticsearch-river-mongodb/downloads
[2] - #41
[3] - https://github.com/richardwilly98/elasticsearch-river-mongodb/tree/master/src/test/java/test/elasticsearch/plugin/river/mongodb/script

Thanks,
Richard.

from elasticsearch-river-mongodb.

Hi Richard,

I can see that I can use the script param to delete a document but how can I add in the conditional to check a field which tells me whether it should be deleted or not?

Cheers,
Martin

from elasticsearch-river-mongodb.

Hi Martin,

That's pure javascript.
So for example if your document has an attribute tobedeleted you can use it
to trigger the deletion in ES using the following snippet:
if (ctx.document.tobedeleted == true) { ctx.deleted = true; }

Again I have not tested deletion but it should work.

Thanks,
Richard.

On Thu, Nov 29, 2012 at 10:46 AM, mgcharl [email protected] wrote:

Hi Richard,

I can see that I can use the script param to delete a document but how can
I add in the conditional to check a field which tells me whether it should
be deleted or not?

Cheers,
Martin

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/42#issuecomment-10853028.

from elasticsearch-river-mongodb.

Hi Rich,

I got the elastic javascript plugin installed so the script runs but I get an error:

I am trying to check against a string:
"script": "if( ctx.document.state == 'CLOSED' ) { ctx.deleted = true; }"

but I get an error:

[org.elasticsearch.river.mongodb.MongoDBRiver$Indexer] failed to script process {id=50b79fc33c458821e2000002, operation=u, document={_id=50b79fc33c458821e2000002, title=foobar, state=CLOSED, updatedAt=null}}, ignoring

org.mozilla.javascript.EcmaError: ReferenceError: "CLOSED" is not defined. (Script1.js#1)

Any Ideas? I tried reading elasticsearch scripting docs but they do not give any javascript help.

Cheers,
Martin

from elasticsearch-river-mongodb.

Hi Martin,

The javascript looks good. Can you please provide ES log?
I will take a look tomorrow.

Thanks,
Richard.
Sent via BlackBerry Bold

-----Original Message-----
From: mgcharl [email protected]
Date: Thu, 29 Nov 2012 09:59:58
To: richardwilly98/elasticsearch-river-mongodbelasticsearch-river-mongodb@noreply.github.com
Reply-To: richardwilly98/elasticsearch-river-mongodb [email protected]
Cc: Richard [email protected]
Subject: Re: [elasticsearch-river-mongodb] mongodb to elasticsearch removal
strategy (#42)

Hi Rich,

I got the elastic javascript plugin installed so the script runs but I get an error:

I am trying to check against a string:
"script": "if( ctx.document.state == 'CLOSED' ) { ctx.deleted = true; }"

but I get an error:

[org.elasticsearch.river.mongodb.MongoDBRiver$Indexer] failed to script process {id=50b79fc33c458821e2000002, operation=u, document={_id=50b79fc33c458821e2000002, title=foobar, state=CLOSED, updatedAt=null}}, ignoring

org.mozilla.javascript.EcmaError: ReferenceError: "CLOSED" is not defined. (Script1.js#1)

Any Ideas? I tried reading elasticsearch scripting docs but they do not give any javascript help.

Cheers,
Martin

Reply to this email directly or view it on GitHub:
#42 (comment)

from elasticsearch-river-mongodb.

Thanks Richard, unfortunately I have been tinkering since on my dev box and so the log traces are gone.

To replicate it would simply be trying to check any string value in javascript inside script param.

I'm afraid I don't have to time to reproduce right now but I will definitely come back to the issue soon.

Many Thanks,
Martin

from elasticsearch-river-mongodb.

Hi Richard,

Thanks for the excellent plugin. I had the same need as this issue and see that it works fine. But in my case I have documents as parents- childs and when a parent is deleted in ES how can I cascade that delete to children too.

Thanks
Zafer

from elasticsearch-river-mongodb.

Hi Zafer,

Could you please provide an example (via gists)? I will take a look at the options.

Thanks,
Richard.

from elasticsearch-river-mongodb.

Hi Richard,

When I posted this comment I was not aware that elastic search does not provide a off the shelf feature to delete the children when ever a parent is deleted. There is a issue open for this in ES (elastic/elasticsearch#1954). But there is a work around discussed in elastic/elasticsearch#2705, I am yet to try if that works though. Is it possible to implement this workaround ( if it works ) in the river ?

I'll let you know if the work around works and create a gist as well in the next 2 -3 days.

Thanks

from elasticsearch-river-mongodb.

I extended the scripts you have to test issue #64, updated two files and added a new one. The gist is at https://gist.github.com/mzafer/5542043

So now the question is more of "How do we change the script in the author river (_02_mongodb-river-author.json) to query for the books(with _parentId == author.id ) and delete them when the 'state' of author is changed to 'INACTIVE' ?" if it is possible in the first place. I look for some documentation on the es-javascript plugin but did not find any.

Thanks

from elasticsearch-river-mongodb.

Hi,

It is currently not possible from the script but I will look at the options.

Thanks,
Richard.

from elasticsearch-river-mongodb.