Specifying `ssl => true` leads to exception "URI::InvalidURIError: bad URI(is not URI?)" about logstash-filter-elasticsearch HOT 6 CLOSED

The problem appears to be with this line. It replaces URL strings with ruby hashes.

In my example, before:

{:hosts=>["https://example.us-central1.gcp.cloud.es.io:9243/"]}

and after this line

{:hosts=>[{:host=>"https://example.us-central1.gcp.cloud.es.io:9243/", :scheme=>"https"}]}

from logstash-filter-elasticsearch.

Currently setting ssl => true is completely broken in this plugin. I'm adding tests to describe the desired behaviour. However the full rewrite of the HTTP subsystem we want to do, to bring all 3 ES plugins in line is out of scope for this bug fix.

Here's how I intend to make it behave, after this bug fix.

    it "should accept schema-less hosts"
    it "should accept hosts with a schema"
    it "should add https to schema-less hosts when specifying ssl true"
    it "should log an error and raise, when specifying a schema in at least one host, and setting ssl true"
    it "should log an error and raise, when specifying a schema in at least one host, and setting ssl false"

Note that the last two are a bit more strict than the behaviour of output-elasticsearch. Specifying any schema in the host list, then setting ssl (which is meant to add a schema everywhere for them) is simply a user passing conflicting args.

I don't think we should jump through hoops to try to accomodate this behaviour. The user should either specify schema for all hosts, or not specify them for any. Then if you set ssl => true we add https:// for you (adding ssl => false doesn't need to do anything, ES::Client will default to http).

If someone needs to specify a mixed set of SSL and not SSL, it's just as simple: specify your http and https schemas in the host list and don't touch the ssl attribute.

Of course the error will make it clear that they should use one or the other, not both. Same for the documentation for ssl.

WDYT, @jsvd @robbavey?

from logstash-filter-elasticsearch.

I can confirm I've been getting this as well.

from logstash-filter-elasticsearch.

This bug cost me some hours :(

One suggestion and one please I have:

Suggestion
Could you please remove ssl option from official documentation or better document the problem with it there.

Question
However the full rewrite of the HTTP subsystem we want to do, to bring all 3 ES plugins in line is out of scope for this bug fix.
Can I see new plugin versions already somewhere as an alpha or beta?

Best regards, Nils

from logstash-filter-elasticsearch.

Issue #129 is related to this as well.

from logstash-filter-elasticsearch.

Thank you very much for taking the time to open this issue.

I'm closing it as It seems to be already fixed by this PR #156. Please feel free to reopen if the problem still exists. Thanks!

from logstash-filter-elasticsearch.