patant / fluent-plugin-loggly Goto Github PK
View Code? Open in Web Editor NEWFluentd plugin for output to loggly
License: Apache License 2.0
Fluentd plugin for output to loggly
License: Apache License 2.0
Hi our work with the Loggly team and we just got this bug report from a customer. We are posting it here because it looks like we need to update a dependency in this library. I pasted his email below. What's the best way to address this? Would you be open to a pull request that we submit?
"I just pushed a new build of our AMI that uses fluentd to log to Loggly,
and things stopped working. Some diagnosis later, I discovered that a new
version of the 'net-http-persistent' has been released (3.0.0 as opposed to
2.9.4, which was working before), and it looks like the 'initialize'
function of that module doesn't like what the loggly module passes to it.
Here's a stacktrace to show what I'm seeing:
2016-10-13 17:27:17 +0000 [error]: unexpected error
error_class=ArgumentError error=#<ArgumentError: wrong number of
arguments (2 for 0)>
2016-10-13 17:27:17 +0000 [error]:
/opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/net-http-persistent-3.0.0/lib/net/http/persistent.rb:505:in
initialize' 2016-10-13 17:27:17 +0000 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluent-plugin-loggly-0.0.8/lib/fluent/plugin/out_loggly_buffered.rb:43:in
new'
2016-10-13 17:27:17 +0000 [error]:
/opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluent-plugin-loggly-0.0.8/lib/fluent/plugin/out_loggly_buffered.rb:43:in
start' 2016-10-13 17:27:17 +0000 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.26/lib/fluent/plugin/out_copy.rb:55:in
block in start'
2016-10-13 17:27:17 +0000 [error]:
/opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.26/lib/fluent/plugin/out_copy.rb:54:in
each' 2016-10-13 17:27:17 +0000 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.26/lib/fluent/plugin/out_copy.rb:54:in
start'
2016-10-13 17:27:17 +0000 [error]:
/opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.26/lib/fluent/agent.rb:70:in
block in start' 2016-10-13 17:27:17 +0000 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.26/lib/fluent/agent.rb:69:in
each'
2016-10-13 17:27:17 +0000 [error]:
/opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.26/lib/fluent/agent.rb:69:in
start' 2016-10-13 17:27:17 +0000 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.26/lib/fluent/root_agent.rb:108:in
start'
2016-10-13 17:27:17 +0000 [error]:
/opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.26/lib/fluent/engine.rb:237:in
start' 2016-10-13 17:27:17 +0000 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.26/lib/fluent/engine.rb:187:in
run'
2016-10-13 17:27:17 +0000 [error]:
/opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.26/lib/fluent/supervisor.rb:570:in
run_engine' 2016-10-13 17:27:17 +0000 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.26/lib/fluent/supervisor.rb:162:in
block in start'
2016-10-13 17:27:17 +0000 [error]:
/opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.26/lib/fluent/supervisor.rb:366:in
call' 2016-10-13 17:27:17 +0000 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.26/lib/fluent/supervisor.rb:366:in
main_process'
2016-10-13 17:27:17 +0000 [error]:
/opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.26/lib/fluent/supervisor.rb:339:in
block in supervise' 2016-10-13 17:27:17 +0000 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.26/lib/fluent/supervisor.rb:338:in
fork'
2016-10-13 17:27:17 +0000 [error]:
/opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.26/lib/fluent/supervisor.rb:338:in
supervise' 2016-10-13 17:27:17 +0000 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.26/lib/fluent/supervisor.rb:156:in
start'
2016-10-13 17:27:17 +0000 [error]:
/opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.26/lib/fluent/command/fluentd.rb:173:in
<top (required)>' 2016-10-13 17:27:17 +0000 [error]: /opt/td-agent/embedded/lib/ruby/site_ruby/2.1.0/rubygems/core_ext/kernel_require.rb:54:in
require'
2016-10-13 17:27:17 +0000 [error]:
/opt/td-agent/embedded/lib/ruby/site_ruby/2.1.0/rubygems/core_ext/kernel_require.rb:54:in
require' 2016-10-13 17:27:17 +0000 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.26/bin/fluentd:5:in
<top (required)>'
2016-10-13 17:27:17 +0000 [error]:
/opt/td-agent/embedded/bin/fluentd:23:in load' 2016-10-13 17:27:17 +0000 [error]: /opt/td-agent/embedded/bin/fluentd:23:in
<top (required)>'
2016-10-13 17:27:17 +0000 [error]: /usr/sbin/td-agent:7:in load' 2016-10-13 17:27:17 +0000 [error]: /usr/sbin/td-agent:7:in
I have a temporary solution - downgrade the net-http-persistent gem in my
environment - and I know this isn't strictly Loggly's problem, but I am
not Ruby-friendly and was wondering if an adjustment could be made to the
Loggly fluentd plugin to account for this change in that module's function
signature."
I'm able to successfully install the fluent-plugin-loggly
but when I actually start td-agent
, I receive the following error:
2015-09-10 00:16:20 +0000 [error]: fluent/supervisor.rb:313:rescue in main_process: config error
file="/etc/td-agent/td-agent.conf" error="Unknown output plugin 'loggly'. Run 'gem search -rd
fluent-plugin' to find plugins"
Here's a full transcript of what I'm doing. Is this a bug in the plugin, or am I doing something wrong?
root@6366f4a521db:/# ruby -v
ruby 1.9.3p484 (2013-11-22 revision 43786) [x86_64-linux]
root@6366f4a521db:/# gem install fluent-plugin-loggly
Successfully installed fluent-plugin-loggly-0.0.6
1 gem installed
Installing ri documentation for fluent-plugin-loggly-0.0.6...
Installing RDoc documentation for fluent-plugin-loggly-0.0.6...
root@6366f4a521db:/# /etc/init.d/td-agent start
* Starting td-agent td-agent
2015-09-10 00:18:49 +0000 [error]: fluent/supervisor.rb:313:rescue in main_process: config error file="/etc/td-agent/td-agent.conf" error="Unknown output plugin 'loggly'. Run 'gem search -rd fluent-plugin' to find plugins"
Here's my td-agent.conf
file:
<source>
type udp
tag mytag
format json
port 5140
</source>
<match **>
type loggly
loggly_url https://logs-01.loggly.com/inputs/TOKEN_CODE/tag/fluentd
</match>
Also, I'm running fluentd in a Docker container, and here's the Dockerfile:
FROM phusion/baseimage:0.9.16
RUN apt-get update && \
apt-get -y upgrade && \
# Install the supported version of fluentd, "td-agent"
curl https://packages.treasuredata.com/GPG-KEY-td-agent | apt-key add - && \
echo "deb [arch=amd64] http://packages.treasuredata.com/2/ubuntu/trusty/ trusty contrib" > /etc/apt/sources.list.d/treasure-data.list && \
apt-get update && \
apt-get install -y --force-yes td-agent && \
apt-get install -y ruby && \
gem install fluent-plugin-loggly && \
apt-get clean && \
rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
ADD files/td-agent.conf /etc/td-agent/td-agent.conf
ADD files/init.d/td-agent /etc/init.d/td-agent
RUN chmod 0700 /etc/init.d/td-agent
EXPOSE 5140
CMD ["/etc/init.d/td-agent", "start"]
The code logs errors but does not raise them upwards, so anything transient like a timeout or internal error at Loggly means the data is thrown away.
If this is an official Loggly repo, then please put it under https://github.com/loggly
Otherwise, please make it very clear that this is not an official repo.
How do I set HTTP proxy server setting ?
I'm sending test records to Fluentd with:
echo "{\"hello\":$(shuf -i1-10 -n1)}" | netcat -q 0 127.0.0.1 5170
These entries are sent to Loggly using the "buffered" plugin:
<match **>
@type loggly_buffered
loggly_url https://logs-01.loggly.com/inputs/<api-key>/tag/test-reto
buffer_type memory
flush_interval 1s
</match>
The fluentd log shows me that the records are sent to Loggly correctly:
2016-04-05 10:05:36 +0000 [debug]: plugin/out_loggly_buffered.rb:60:write: 4 records sent
2016-04-05 10:05:36 +0000 [debug]: plugin/out_loggly_buffered.rb:65:write: HTTP Response code 200
2016-04-05 10:05:37 +0000 [debug]: plugin/out_loggly_buffered.rb:60:write: 5 records sent
2016-04-05 10:05:37 +0000 [debug]: plugin/out_loggly_buffered.rb:65:write: HTTP Response code 200
2016-04-05 10:05:38 +0000 [debug]: plugin/out_loggly_buffered.rb:60:write: 2 records sent
2016-04-05 10:05:38 +0000 [debug]: plugin/out_loggly_buffered.rb:65:write: HTTP Response code 200
2016-04-05 10:05:39 +0000 [debug]: plugin/out_loggly_buffered.rb:60:write: 1 records sent
2016-04-05 10:05:39 +0000 [debug]: plugin/out_loggly_buffered.rb:65:write: HTTP Response code 200
But then in Loggly these records show up combined/aggregated per second:
Am I doing something wrong?
Yajl::Encoder.encode()
raises some errors, such as trying to encode "Infinity" values, as exceptions.
These escape to fluentd, which retries, again and again, until either the retry limit is hit and the whole buffer is dropped, or the buffer space is completely full and all buffers are dropped.
I've been struggling to get this plugin working successfully. I'd like to be able to debug what's going on, but I can't seem to find the plugin's log entries anywhere.
In the code, I see entries like https://github.com/patant/fluent-plugin-loggly/blob/master/lib/fluent/plugin/out_loggly.rb#L30. But in the /var/log/td-agent/td-agent.log
I see none of these entries.
I'm outputting logs both to a local file and to Loggly. The local file works fine but loggly output is not working. Any thoughts on how I can resolve this?
Here's my td-agent.conf
(In my actual file, I have #comments
which I've removed below):
<source>
type forward
port 5140
bind 0.0.0.0
</source>
<match **>
type file
path /tmp/sample.log
</match>
<match **>
type loggly_buffered
loggly_url https://logs-01.loggly.com/bulk/<token>/tag/fluentd
output_include_time true # add 'timestamp' record into log. (default: true)
buffer_type file
buffer_path /tmp/buffer
flush_interval 2s
</match>
I've also confirmed that directly calling loggly's API via curl
works fine from my box. I've also successfully installed the fluent-plugin-loggly
gem by running it in a Docker container with this Dockerfile
:
FROM phusion/baseimage:0.9.16
RUN apt-get update && \
apt-get -y upgrade && \
# Install the supported version of fluentd, "td-agent"
curl https://packages.treasuredata.com/GPG-KEY-td-agent | apt-key add - && \
echo "deb [arch=amd64] http://packages.treasuredata.com/2/ubuntu/trusty/ trusty contrib" > /etc/apt/sources.list.d/treasure-data.list && \
apt-get update && \
apt-get install -y --force-yes td-agent && \
td-agent-gem install fluent-plugin-loggly && \
apt-get clean && \
rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
ADD files/td-agent.conf /etc/td-agent/td-agent.conf
ADD files/init.d/td-agent /etc/init.d/td-agent
RUN chmod 0700 /etc/init.d/td-agent
EXPOSE 5140
CMD ["/etc/init.d/td-agent", "start"]
At this point, I may just use the generic exec plugin, but that seems like a shame given that all of this nice work is done here. Thanks for your input!
Hi,
My environment is: td-agent 4.4.1 fluentd 1.15.2 (c32842297ed2c306f1b841a8f6e55bdd0f1cb27f)
My fluentd config is:
<match **>
@type loggly
loggly_url https://my_app.loggly.com/inputs/my_api_token
</match>
When I start the fluentd, I got this error immediately:
2023-02-02 19:03:40 -0800 [info]: #0 starting fluentd worker pid=137531 ppid=137518 worker=0
2023-02-02 19:03:40 -0800 [error]: #0 unexpected error error_class=ArgumentError error="wrong number of arguments (given 2, expected 0)"
2023-02-02 19:03:40 -0800 [error]: #0 /opt/td-agent/lib/ruby/gems/2.7.0/gems/net-http-persistent-4.0.1/lib/net/http/persistent.rb:471:in `initialize'
2023-02-02 19:03:40 -0800 [error]: #0 /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluent-plugin-loggly-0.0.9/lib/fluent/plugin/out_loggly_buffered.rb:43:in `new'
2023-02-02 19:03:40 -0800 [error]: #0 /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluent-plugin-loggly-0.0.9/lib/fluent/plugin/out_loggly_buffered.rb:43:in `start'
2023-02-02 19:03:40 -0800 [error]: #0 /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.15.2/lib/fluent/compat/call_super_mixin.rb:42:in `start'
2023-02-02 19:03:40 -0800 [error]: #0 /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.15.2/lib/fluent/root_agent.rb:203:in `block in start'
2023-02-02 19:03:40 -0800 [error]: #0 /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.15.2/lib/fluent/root_agent.rb:192:in `block (2 levels) in lifecycle'
2023-02-02 19:03:40 -0800 [error]: #0 /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.15.2/lib/fluent/root_agent.rb:191:in `each'
2023-02-02 19:03:40 -0800 [error]: #0 /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.15.2/lib/fluent/root_agent.rb:191:in `block in lifecycle'
2023-02-02 19:03:40 -0800 [error]: #0 /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.15.2/lib/fluent/root_agent.rb:178:in `each'
2023-02-02 19:03:40 -0800 [error]: #0 /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.15.2/lib/fluent/root_agent.rb:178:in `lifecycle'
2023-02-02 19:03:40 -0800 [error]: #0 /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.15.2/lib/fluent/root_agent.rb:202:in `start'
2023-02-02 19:03:40 -0800 [error]: #0 /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.15.2/lib/fluent/engine.rb:248:in `start'
2023-02-02 19:03:40 -0800 [error]: #0 /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.15.2/lib/fluent/engine.rb:147:in `run'
2023-02-02 19:03:40 -0800 [error]: #0 /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.15.2/lib/fluent/supervisor.rb:764:in `block in run_worker'
2023-02-02 19:03:40 -0800 [error]: #0 /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.15.2/lib/fluent/supervisor.rb:1037:in `main_process'
2023-02-02 19:03:40 -0800 [error]: #0 /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.15.2/lib/fluent/supervisor.rb:755:in `run_worker'
2023-02-02 19:03:40 -0800 [error]: #0 /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.15.2/lib/fluent/command/fluentd.rb:381:in `<top (required)>'
2023-02-02 19:03:40 -0800 [error]: #0 /opt/td-agent/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:83:in `require'
2023-02-02 19:03:40 -0800 [error]: #0 /opt/td-agent/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:83:in `require'
2023-02-02 19:03:40 -0800 [error]: #0 /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.15.2/bin/fluentd:15:in `<top (required)>'
2023-02-02 19:03:40 -0800 [error]: #0 /opt/td-agent/bin/fluentd:23:in `load'
2023-02-02 19:03:40 -0800 [error]: #0 /opt/td-agent/bin/fluentd:23:in `<top (required)>'
2023-02-02 19:03:40 -0800 [error]: #0 /usr/sbin/td-agent:15:in `load'
2023-02-02 19:03:40 -0800 [error]: #0 /usr/sbin/td-agent:15:in `<main>'
2023-02-02 19:03:40 -0800 [error]: #0 unexpected error error_class=ArgumentError error="wrong number of arguments (given 2, expected 0)"
2023-02-02 19:03:40 -0800 [error]: #0 suppressed same stacktrace
2023-02-02 19:03:40 -0800 [error]: Worker 0 finished unexpectedly with status 1
I wonder if anyone encounter the same issue or is it my config wrong?
related to td-agent v3.
When there is no connection to loggly webservice due to networking problems, plugin fails to shutdown and the whole td-agent process is stuck for about a minute.
2016-10-08 01:43:31 +0000 [error]: unexpected error error_class=ArgumentError error=#<ArgumentError: wrong number of arguments (given 2, expected 0)>
2016-10-08 01:43:31 +0000 [error]: /home/fluent/.gem/ruby/2.3.0/gems/net-http-persistent-3.0.0/lib/net/http/persistent.rb:505:in `initialize'
2016-10-08 01:43:31 +0000 [error]: /home/fluent/.gem/ruby/2.3.0/gems/fluent-plugin-loggly-0.0.8/lib/fluent/plugin/out_loggly.rb:37:in `new'
2016-10-08 01:43:31 +0000 [error]: /home/fluent/.gem/ruby/2.3.0/gems/fluent-plugin-loggly-0.0.8/lib/fluent/plugin/out_loggly.rb:37:in `start'
2016-10-08 01:43:31 +0000 [error]: /usr/lib/ruby/gems/2.3.0/gems/fluentd-0.12.29/lib/fluent/agent.rb:71:in `block in start'
2016-10-08 01:43:31 +0000 [error]: /usr/lib/ruby/gems/2.3.0/gems/fluentd-0.12.29/lib/fluent/agent.rb:70:in `each'
2016-10-08 01:43:31 +0000 [error]: /usr/lib/ruby/gems/2.3.0/gems/fluentd-0.12.29/lib/fluent/agent.rb:70:in `start'
2016-10-08 01:43:31 +0000 [error]: /usr/lib/ruby/gems/2.3.0/gems/fluentd-0.12.29/lib/fluent/root_agent.rb:108:in `start'
2016-10-08 01:43:31 +0000 [error]: /usr/lib/ruby/gems/2.3.0/gems/fluentd-0.12.29/lib/fluent/engine.rb:237:in `start'
2016-10-08 01:43:31 +0000 [error]: /usr/lib/ruby/gems/2.3.0/gems/fluentd-0.12.29/lib/fluent/engine.rb:187:in `run'
2016-10-08 01:43:31 +0000 [error]: /usr/lib/ruby/gems/2.3.0/gems/fluentd-0.12.29/lib/fluent/supervisor.rb:570:in `run_engine'
2016-10-08 01:43:31 +0000 [error]: /usr/lib/ruby/gems/2.3.0/gems/fluentd-0.12.29/lib/fluent/supervisor.rb:162:in `block in start'
2016-10-08 01:43:31 +0000 [error]: /usr/lib/ruby/gems/2.3.0/gems/fluentd-0.12.29/lib/fluent/supervisor.rb:366:in `main_process'
2016-10-08 01:43:31 +0000 [error]: /usr/lib/ruby/gems/2.3.0/gems/fluentd-0.12.29/lib/fluent/supervisor.rb:339:in `block in supervise'
2016-10-08 01:43:31 +0000 [error]: /usr/lib/ruby/gems/2.3.0/gems/fluentd-0.12.29/lib/fluent/supervisor.rb:338:in `fork'
2016-10-08 01:43:31 +0000 [error]: /usr/lib/ruby/gems/2.3.0/gems/fluentd-0.12.29/lib/fluent/supervisor.rb:338:in `supervise'
2016-10-08 01:43:31 +0000 [error]: /usr/lib/ruby/gems/2.3.0/gems/fluentd-0.12.29/lib/fluent/supervisor.rb:156:in `start'
2016-10-08 01:43:31 +0000 [error]: /usr/lib/ruby/gems/2.3.0/gems/fluentd-0.12.29/lib/fluent/command/fluentd.rb:173:in `<top (required)>'
2016-10-08 01:43:31 +0000 [error]: /usr/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in `require'
2016-10-08 01:43:31 +0000 [error]: /usr/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in `require'
2016-10-08 01:43:31 +0000 [error]: /usr/lib/ruby/gems/2.3.0/gems/fluentd-0.12.29/bin/fluentd:5:in `<top (required)>'
2016-10-08 01:43:31 +0000 [error]: /usr/bin/fluentd:23:in `load'
2016-10-08 01:43:31 +0000 [error]: /usr/bin/fluentd:23:in `<main>'
My platform is work with ruby2.0.
I can not install fluent-plugin-loggly.
[root@watcher ruby-2.0.0-p353]# ruby -v
ruby 2.0.0p353 (2013-11-22 revision 43784) [x86_64-linux]
[root@watcher ruby-2.0.0-p353]# gem install fluent-plugin-loggly
ERROR: Error installing fluent-plugin-loggly:
fluent-plugin-loggly requires Ruby version ~> 1.9.2.
I hope this plugin upgraded.
Thanks,
Hello,
Currently i'm using fluent-plugin-loggly and i want to use multi proccess worker, but the loggly plugin can't start.
So how can use this plugin with multi process workers ?
Thanks for this great job ๐
<match your_match>
type loggly
loggly_url https://logs-01.loggly.com/inputs/xxx-xxxx-xxxx-xxxxx-xxxxxxxxxx
</match>
it will post messages to http tag on loggly system by default. or it set tag if /tag/[you-tag] appended to loggly_url
Is it possible to post messages to matched tag?
Hi there!
td-agent can't load your pluggin.
$ gem -v
2.4.8
Here are the logs:
2016-10-13 01:29:02 +0300 [info]: starting fluentd-0.12.29
2016-10-13 01:29:02 +0300 [info]: gem 'fluent-mixin-config-placeholders' version '0.4.0'
2016-10-13 01:29:02 +0300 [info]: gem 'fluent-mixin-plaintextformatter' version '0.2.6'
2016-10-13 01:29:02 +0300 [info]: gem 'fluent-plugin-kafka' version '0.3.1'
2016-10-13 01:29:02 +0300 [info]: gem 'fluent-plugin-loggly' version '0.0.8'
2016-10-13 01:29:02 +0300 [info]: gem 'fluent-plugin-mongo' version '0.7.15'
2016-10-13 01:29:02 +0300 [info]: gem 'fluent-plugin-rewrite-tag-filter' version '1.5.5'
2016-10-13 01:29:02 +0300 [info]: gem 'fluent-plugin-s3' version '0.7.1'
2016-10-13 01:29:02 +0300 [info]: gem 'fluent-plugin-scribe' version '0.10.14'
2016-10-13 01:29:02 +0300 [info]: gem 'fluent-plugin-td' version '0.10.29'
2016-10-13 01:29:02 +0300 [info]: gem 'fluent-plugin-td-monitoring' version '0.2.2'
2016-10-13 01:29:02 +0300 [info]: gem 'fluent-plugin-webhdfs' version '0.4.2'
2016-10-13 01:29:02 +0300 [info]: gem 'fluentd' version '0.12.29'
2016-10-13 01:29:02 +0300 [info]: adding match pattern="**" type="stdout"
2016-10-13 01:29:02 +0300 [info]: adding match pattern="**" type="loggly"
2016-10-13 01:29:02 +0300 [info]: adding source type="forward"
2016-10-13 01:29:02 +0300 [info]: using configuration file: <ROOT>
<source>
@type forward
port 24224
bind 0.0.0.0
</source>
<match **>
@type stdout
</match>
<match **>
type loggly
loggly_url https://logs-01.loggly.com/inputs/mytoken
</match>
</ROOT>
2016-10-13 01:29:02 +0300 [error]: unexpected error error_class=ArgumentError error=#<ArgumentError: wrong number of arguments (2 for 0)>
2016-10-13 01:29:02 +0300 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/net-http-persistent-3.0.0/lib/net/http/persistent.rb:505:in `initialize'
2016-10-13 01:29:02 +0300 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluent-plugin-loggly-0.0.8/lib/fluent/plugin/out_loggly.rb:37:in `new'
2016-10-13 01:29:02 +0300 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluent-plugin-loggly-0.0.8/lib/fluent/plugin/out_loggly.rb:37:in `start'
2016-10-13 01:29:02 +0300 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.29/lib/fluent/agent.rb:71:in `block in start'
2016-10-13 01:29:02 +0300 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.29/lib/fluent/agent.rb:70:in `each'
2016-10-13 01:29:02 +0300 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.29/lib/fluent/agent.rb:70:in `start'
2016-10-13 01:29:02 +0300 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.29/lib/fluent/root_agent.rb:108:in `start'
2016-10-13 01:29:02 +0300 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.29/lib/fluent/engine.rb:237:in `start'
2016-10-13 01:29:02 +0300 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.29/lib/fluent/engine.rb:187:in `run'
2016-10-13 01:29:02 +0300 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.29/lib/fluent/supervisor.rb:570:in `run_engine'
2016-10-13 01:29:02 +0300 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.29/lib/fluent/supervisor.rb:162:in `block in start'
2016-10-13 01:29:02 +0300 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.29/lib/fluent/supervisor.rb:366:in `call'
2016-10-13 01:29:02 +0300 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.29/lib/fluent/supervisor.rb:366:in `main_process'
2016-10-13 01:29:02 +0300 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.29/lib/fluent/supervisor.rb:339:in `block in supervise'
2016-10-13 01:29:02 +0300 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.29/lib/fluent/supervisor.rb:338:in `fork'
2016-10-13 01:29:02 +0300 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.29/lib/fluent/supervisor.rb:338:in `supervise'
2016-10-13 01:29:02 +0300 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.29/lib/fluent/supervisor.rb:156:in `start'
2016-10-13 01:29:02 +0300 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.29/lib/fluent/command/fluentd.rb:173:in `<top (required)>'
2016-10-13 01:29:02 +0300 [error]: /opt/td-agent/embedded/lib/ruby/site_ruby/2.1.0/rubygems/core_ext/kernel_require.rb:54:in `require'
2016-10-13 01:29:02 +0300 [error]: /opt/td-agent/embedded/lib/ruby/site_ruby/2.1.0/rubygems/core_ext/kernel_require.rb:54:in `require'
2016-10-13 01:29:02 +0300 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.29/bin/fluentd:5:in `<top (required)>'
2016-10-13 01:29:02 +0300 [error]: /opt/td-agent/embedded/bin/fluentd:23:in `load'
2016-10-13 01:29:02 +0300 [error]: /opt/td-agent/embedded/bin/fluentd:23:in `<top (required)>'
2016-10-13 01:29:02 +0300 [error]: /usr/sbin/td-agent:7:in `load'
2016-10-13 01:29:02 +0300 [error]: /usr/sbin/td-agent:7:in `<main>'
2016-10-13 01:29:02 +0300 [info]: shutting down fluentd
2016-10-13 01:29:02 +0300 [info]: shutting down output type="stdout" plugin_id="object:3ff5058c2bb8"
2016-10-13 01:29:02 +0300 [info]: process finished code=0
2016-10-13 01:29:02 +0300 [warn]: process died within 1 second. exit.
Fluentd 0.14 introduced a new plugin API: http://www.fluentd.org/blog/fluentd-v0.14.0-has-been-released
It would be great if I could use the new "dynamic configurations":
support formatting configurations dynamically (e.g.,
path /my/dest/${tag}/mydata.%Y-%m-%d.log
)
This way I could use my fluentd tags as my Loggly tags by setting the URL to:
loggly_url https://logs-01.loggly.com/bulk/xxxxxxxxxxxx/tag/${tag}
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.