new files via wildcard; missing events about logstash-forwarder HOT 4 CLOSED

I have seen this too. Lumberjack seems to miss events if the file does not exists and then its harvested. See my comments in the 5th post of issues #61.

from logstash-forwarder.

This is a bug. Will work on a fix.

from logstash-forwarder.

The issue is the way prospector.go handles unknown files and starts the harvester. There is currently no difference between:

1. a file what was created while lumberjack was running thus its unknown and does not exist in .lumberjack,

2. files that are unknown due to a missing .lumberjack file, the files exist, but lumber has not seem before.

In the first case you would always want to start at the beginning, but in the latter you may not. The current logic always starts at the end. Agree that a flag to start at the beginning could be a nice feature. In fact there is pull request for that, though is at a global level and not a file config item.

from logstash-forwarder.

This is fixed in master. Will appera in 0.4.0

from logstash-forwarder.