wshihadeh / fluent-plugin-filter-kv-parser Goto Github PK

Hello there,

I am trying to install the latest version 0.0.8 - and in fact I need it because of the new 'keys_prefix' config parameter - but whatever I do, I always keep getting version 0.0.7 installed.

My target system is Ubuntu 18.04
I tried with the original fluentd, and also with the Treasure Data (td-agent4) version.
Both gem installations only install 0.0.7 and not 0.0.8.

Why is that ?

I was testing this plugin with the following configuration:

<filter **>
  @type key_value_parser
  key message
  use_regex true
  remove_key false
</filter>

And for this message:

[MSGRECEIVER] TASKSTATUS receive rate=2 per minute

The value extracted for rate was: 2 per minute, which seems wrong to me, it should have been just 2.

I was using the latest versions of fluentd and this plugin.

I think this is the regex used by the plugin to extract the key and the value:

  "#{line} ".scan(/(?<key>[a-zA-Z_0-9]+)=(?<value>([^=]+|[^ ]+))\s/).to_h

And I wonder if the part ^=]+|[^ ]+ is the problem, perhaps rewriting the regex like this would work better?

  "#{line} ".scan(/(?<key>[a-zA-Z_0-9]+)=(?<value>([^= ]+))\s/).to_h

Hi,
I am trying to extract key value pairs from following log pattern.

Log message : "Received object { Operation1:extract ~~ Time:Thu Oct 06 12:10:47 ~~ Message:text message 123}"

So, I used keys_delimiter as ~~ and kv_delimiter_char as : to seperate key values pairs.

Following given is my full configuration in the filter section.

@type key_value_parser
key content
use_regex true
remove_key false
keys_delimiter ~~
kv_delimiter_char :
filter_out_lines_without_keys false

But seems it is not working. How ever, It extracted some unexpected key values that uses space and = signs as delimeters. It seems that default configurations ( space and = ) are applied here. I tried by wrapping the characters with quatation marks, used forward slashes ( /~~/ and /:/ ) in the configuration. But they also didn't work.

So, What can be the problem here? Appreciate If someone can help on this.