Should Logstash create a "message.raw" by default? about logstash-output-elasticsearch HOT 15 CLOSED

-1 on not having error.message.raw available in my kibana
that was one of the first things I wanted to build with my ELK setup
a list of error messages by number

from logstash-output-elasticsearch.

To add to this: we'd like to move to doc-values-by-default on all fields except for analyzed strings fields (which don't support doc values anyway). This not analyzed message field is (almost?) always unique, which is essentially doubling the disk space required, with no benefit.

from logstash-output-elasticsearch.

Talked to @jsvd internally about it.
The 'message' field is usually used in 2 cases

1. Original message field ( people forget to delete / modify it after parsing )
2. people actually modify it with a sensible content. will most likely do free text search on it.

I think because of that it makes sense to have the .raw ( not_analyzed ) version to be dropped.

from logstash-output-elasticsearch.

+1 on omitting 'message.raw'

from logstash-output-elasticsearch.

@jordansissel i think you mean the opposite? "not having message not analyzed"?

from logstash-output-elasticsearch.

@clintongormley oops, typo! I updated my comment to be more accurate.

from logstash-output-elasticsearch.

+1

from logstash-output-elasticsearch.

-1 on not having message analyzed
+1 on not having the message.raw not analyzed

anything else to not vote on?

from logstash-output-elasticsearch.

+1 on not having a message.raw field :)

from logstash-output-elasticsearch.

Closed by #12

from logstash-output-elasticsearch.

-1 on not having error.message.raw available in my kibana

@derEremit This ticket was specifically about the top-level field named message, not error.message, so unless I am not understanding, your error.message field will continue to behave the way you want :)

from logstash-output-elasticsearch.

@jordansissel
okay, I thought this ticket applies to my problem, as I don't get .raw fields on anything ending with message, on my unmodified logstash, elastic 2.3 setup

should I re-check and open a separate issue for that?

from logstash-output-elasticsearch.

Open a new issue

On Tuesday, September 20, 2016, Sebastian Paul [email protected]
wrote:

okay, I thought this ticket applies to my problem, as I don't get .raw
fields on anything ending with message, on my unmodified logstash, elastic
2.3 setup

should I re-check and open a separate issue for that?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#11 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAIC6u0YTEOhzyI77V99KHhVA303GNx-ks5qsF6kgaJpZM4C6F2b
.

from logstash-output-elasticsearch.

@jordansissel btw the way the template is defined at the moment will match any field whose name is message (which includes foo.bar.message). To limit it to just the top-level message field, it should use path_match instead of match

from logstash-output-elasticsearch.

I've opened a PR to do just what you describe @clintongormley

#481

from logstash-output-elasticsearch.