How do I parse bog-standard syslog format? about binjr HOT 4 CLOSED

There are maybe a couple of things at play here.

First of all, as you've mentioned, the source data is missing the year information, so binjr has to complete the elements its got with a default starting point in time, as it cannot deal with partial dates.
For better or worse, the default date I have chosen is 1970-01-01 00:00:00.000 UTC, which is 1969-12-31 16:00:00.000 (4pm) in your timezone (UTC-8).

So this explains why the year shown here is 1969: because the year component for the date was not provided, 1969 is used as a substitute while the day and month are overridden with the data parsed from the file.

With that said, even with this does not explain why the time components are not overridden (e.g. it says 4 p.m instead of 12 a.m.). I could not reproduce this with the parsing rules that you shared above, but this is however what you you get with parsing rules that only picked up the month and day, like such:

Is it possible at all that the two screenshots do not actually match (e.g. the log view in the second screenshot was taken with a version of the parsing rules that does not capture the time)?

If not, then there is probably a bug in binjr, but as I said I could not reproduce it.

from binjr.

Created this logfile:

Feb 19 00:00:01 thinkpad systemd[1]: Starting system activity accounting tool...
Feb 19 00:00:01 thinkpad audit: BPF prog-id=739 op=LOAD
Feb 19 00:00:01 thinkpad audit: BPF prog-id=740 op=LOAD
Feb 19 00:00:01 thinkpad systemd[1]: Starting update of the root trust anchor for DNSSEC validation in unbound...

Fresh screenshot of freshly-imported log file in a new worksheet:

I'd argue that the syslog format should have a year on it, but it's been sacrificed in the name of backwards-compatibility.
I would also argue that when there isn't a year, we should try to follow syslog's algorithm and assume the timestamp is for some moment in the past 365.26 days, with an option for users to override that.

If they've got a logfile containing multiple years' worth of entries, it's tougher: assume the file ends in the most recent 365.26-day period, and extrapolate backwards, assuming that there's at least one log entry per month and the timestamps are in monotonically increasing order. The filesystem timestamp of the logfile might be worth considering too: if it's roughly the same as the last log entry in the file (logrotation+compression effect), copy the year from it.

Although at some point, this turns into a data forensics problem. :-)

from binjr.

BTW, I changed the last line in the logfile to be Feb 19 00:00:03 to see if the problem was caused by all the entries having the same timestamp, but it didn't help, the time range at the top of the worksheet was still 1969-02-08 3:59:59pm to 4:00:00pm.

It's odd that it's grabbing Feb 8th, and not Feb 20th or Jan 1st.

BTW, this is Binjr 3.11.0, downloaded yesterday.

from binjr.

I have created two separate issues to track the problems that surfaced in this one:

• #129 - Let users choose the date & time that serves as an anchor to construct timestamps for partial data to track making the default time reference point user-selectable
• 130 - Different behaviour when typing the name of a TemporalCaptureGroup or selecting it from the dropdown list in profile editor to track the bug that was the root cause for some parsing keyword not working (and the source of much confusion)

@PenelopeFudd I'm therefore closing this issue; feel free to reopen it if you feel there's a need.

from binjr.