Add in current logfile symlink about fluentd HOT 10 CLOSED

Could you give me a pointer of Scribe's document? Will take a look at it.

from fluentd.

So, Scribe's documentation is notoriously bad, and this is all I could find:
https://github.com/facebook/scribe/wiki/Scribe-Configuration

About halfway down, there is this blurb:

create_symlink: “yes” or anything else; “yes” by default

if true, will maintain a symlink that points to the most recently written file

So, in the log file directory for 'test_log', there is always a 'test_log_current' that points to the log file that is currently being written to. When the file is rotated, you just change the link (and this allows tail -F to work on the 'test_log_current' file).

from fluentd.

OK I understand.

We also might want to have modified version of out_file, just adding recent entries to the files instead of slicing logs by time_slice_format.

from fluentd.

Any update on this?

from fluentd.

Pinging this issue again.

from fluentd.

I'm not sure about the detail behavior of Scribe.

In fluentd, my implementation idea is simple:

1. out_file writes chunk to file.
2. after wrote the file, re-link symlink to written file.

iterates above operations for nomarl file and gz file.
Is it enough?

from fluentd.

Yeah, I think that will totally work. Will you re-link the symlink right after switching what file you are writing to?

For instance, the reason we want this is for displaying logs in our office.

from fluentd.

^ is first implementation.
out_file doesn't provide non buffered mode, so out_file can't implement Scribe like each event tailing.

If you set symlink_path in out_file, out_file plugin creates symlink with suffix to point to latest file.
We can use tail -F to watch written logs.

from fluentd.

Perfect! Thanks :)

from fluentd.

Sorry to reopen this:

I just installed the newest td-agent package, and the way this works seems not that useful.

Right now, the symlink_path file is symlinked to the last fully written file, not the file that's currently being written. For instance, in our nginx access log directory:

last fully written log: access.log.2013080820_1.log
current file being written to: access.log.2013080821.b4e375f0fb757453e

The symlink access.log_current is currently pointing to access.log.2013080820_1.log, when I would expect it to be pointing to access.log.2013080821.b4e375f0fb757453e. Am I missing something?

from fluentd.