Heavy/Rapid fluentd container logs are logged while the docker daemon logging driver is set to journald when used fluentd-plugin-systemd about fluent-plugin-systemd HOT 4 CLOSED

Hello @ameyrk18; I've run into this problem myself. This is generally caused when fluentd is configured to write out log events to STDOUT, when configured to use systemd as the log-driver, when fluentd is configured to process logs from systemd.

You've basically created a loop, where one container sends a log to STDERR/STDOUT, Docker forwards this to systemd, systemd forwards this to fluentd, fluentd forwards this to STDOUT, which forwards to systemd, which forwards to fluentd, which...

The way I've gotten around this is by configuring the fluentd Docker container to use json-file as its log-driver. When you configure the log-driver in /etc/docker/daemon.json, that sets the default log-driver, but you can still over-ride this on container creation with the --log-driver json-file option being passed to docker container run.

from fluent-plugin-systemd.

Hi @Jitsusama yes your right that happens when the output driver is set stdout. Ahh, yes now I get the loops :) weird. Not sure if this is expected or not.

I bypassed this as you said. Reconfigured my docker logging to journald and forced the container (fluentd) to use json logging.

Many thanks for your inputs.

I leave this topic with the owner to close this if this is not an issue.

Thanks,
Amey

from fluent-plugin-systemd.

Thanks for raising this @ameyrk18

I guess we should document that this can happen ...

Am I correct in thinking @ameyrk18 that when this issue occurs you are reading the full journal without setting matches/filters?

from fluent-plugin-systemd.

@errm . I will document this and update READ.me

Yes your understanding is correct I am reading full journal without setting matches/filters.

Best Regards,
Amey

from fluent-plugin-systemd.