gemnasium / logrus-graylog-hook Goto Github PK

Hi. first of all thanks for providing graylog hook. it's pretty help for me

i suggest you a feature (this requires little bit change i guess)

currently there is no option to set compression type and i believe the compression is important feature of GELF format specification.

i found that the struct has gelf.Writer as a member that has a interface for setting the compression type but it's not accessible from outside.

type GraylogHook struct {
Facility string
Extra map[string]interface{}
gelfLogger *gelf.Writer
buf chan graylogEntry
wg sync.WaitGroup
mu sync.RWMutex
synchronous bool
}

if you replace the first character 'g' of "gelfLogger" with "G" , all problems are solved cause user can access directly to gelf.writer configuration or maybe you can add some gelf.writer configurable interface in GraylogHook.

please consider it.

thanks.

Since we're sending message in a goroutine, the very last messages are not sent to graylog on program exit, the routine being destroyed before the end of the queue.

When i use two logger instance concurrently then i get "invalid memory address or nil pointer dereference"
Here a sample code to reproduce the error

package main
import (
	graylog "github.com/gemnasium/logrus-graylog-hook/v3"
	"github.com/sirupsen/logrus"
	"sync"
)
func main() {
	var wg sync.WaitGroup

	var gelfHook = graylog.NewGraylogHook("xx.xx.xx.xx:xxx", map[string]interface{}{})

	log1 := logrus.New()
	log2 := logrus.New()

	log1.AddHook(gelfHook)
	log2.AddHook(gelfHook)

	for i := 0; i < 100; i++ {
		wg.Add(1)
		go func() {
			defer wg.Done()
			log1.Info("test log info")
			log2.Info("test log info")
		}()
	}
	wg.Wait()
}

and the error message

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x4d19da]

goroutine 23 [running]:
compress/flate.(*Writer).Close(...)
        /usr/lib/go-1.13/src/compress/flate/deflate.go:729
compress/gzip.(*Writer).Close(0xc0000be210, 0xc0004801e0, 0x97)
        /usr/lib/go-1.13/src/compress/gzip/gzip.go:248 +0x6a
github.com/gemnasium/logrus-graylog-hook/v3.(*Writer).WriteMessage(0xc0000d0070, 0xc00041a100, 0xc000141ac0, 0x0)
        /xxxx/go/pkg/mod/github.com/gemnasium/logrus-graylog-hook/[email protected]/gelf_writer.go:246 +0x2e4
github.com/gemnasium/logrus-graylog-hook/v3.(*GraylogHook).sendEntry(0xc0000ba120, 0xc000119340, 0x0, 0x0, 0x0)
        /xxxx/go/pkg/mod/github.com/gemnasium/logrus-graylog-hook/[email protected]/graylog_hook.go:247 +0xa2b
github.com/gemnasium/logrus-graylog-hook/v3.(*GraylogHook).Fire(0xc0000ba120, 0xc0001192d0, 0x0, 0x0)
        /xxxx/go/pkg/mod/github.com/gemnasium/logrus-graylog-hook/[email protected]/graylog_hook.go:120 +0x294
github.com/sirupsen/logrus.LevelHooks.Fire(0xc00009c750, 0x4, 0xc0001192d0, 0xc0001192d0, 0x40d3a4)
        /xxxx/go/pkg/mod/github.com/sirupsen/[email protected]/hooks.go:28 +0x91
github.com/sirupsen/logrus.(*Entry).fireHooks(0xc0001192d0)
        /xxxx/go/pkg/mod/github.com/sirupsen/[email protected]/entry.go:246 +0xa3
github.com/sirupsen/logrus.Entry.log(0xc0000d0150, 0xc0001104e0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
        /xxxx/go/pkg/mod/github.com/sirupsen/[email protected]/entry.go:224 +0xf4
github.com/sirupsen/logrus.(*Entry).Log(0xc000119260, 0xc000000004, 0xc000141f98, 0x1, 0x1)
        /xxxx/go/pkg/mod/github.com/sirupsen/[email protected]/entry.go:268 +0xeb
github.com/sirupsen/logrus.(*Logger).Log(0xc0000d0150, 0x4, 0xc000141f98, 0x1, 0x1)
        /xxxx/go/pkg/mod/github.com/sirupsen/[email protected]/logger.go:192 +0x7d
github.com/sirupsen/logrus.(*Logger).Info(...)
        /xxxx/go/pkg/mod/github.com/sirupsen/[email protected]/logger.go:206
main.main.func1(0xc0000b4250, 0xc0000d00e0, 0xc0000d0150)
        /xxxx/main.go:25 +0x110
created by main.main
        /xxxx/main.go:22 +0x26e

I think that something will be override because if add this lines

w.mu.Lock()
defer w.mu.Unlock()

to gelf_writer.go line 198 then everything work as expected

TestStackTracer verifies stacktraces with a regex, but this regex assumes all paths start with a /, which is not the case on Windows.

When I enable the report caller in logrus by setting log.SetReportCaller(true), then the fields file and func are visible when in stdout.

In Graylog these fields are not visible at all.

Is this a bug or normal behavior?

https://github.com/gemnasium/logrus-graylog-hook/blob/v3.0.0/.travis.yml need to be updated to reflect changes in v3

I think that #42 needs to be merged first for this

Hello,

Thanks you for this awesome hook!

We are facing a problem with log levels. When we log with logrus.info, the log level in graylog is 4 (warning in syslog) instead of 6.

Can we map the logrus level with syslog level before sending to graylog?

Dependency is broken, due to removal of repository http://github.com/SocialCodeInc/go-gelf/gelf

Hey, the following code produces a race condition in graylog_hook.go line 146 because there is a read access on the Data map outside the Hook interfaces Fire function.

package main

import (
    "time"

    "gopkg.in/gemnasium/logrus-graylog-hook.v1"

    "github.com/Sirupsen/logrus"
)

func main() {
    log := logrus.New()
    hook := graylog.NewAsyncGraylogHook("localhost:12201", "facility", map[string]interface{}{"this": "is logged every time"})
    defer hook.Flush()
    log.Hooks.Add(hook)

    go func() {
        log.WithFields(logrus.Fields{"msg": "test1"}).Info("test1")
    }()

    log.WithFields(logrus.Fields{"msg": "test2"}).Info("test2")

    ch := time.Tick(time.Second * 1)

    <-ch
}

i keep getting

got code 202 Accepted, expected 204

based on gravelog document, 202 should be expected

Both keep-alive and compression are supported via the common HTTP headers. The server will return a 202 Accepted when the message was accepted for processing.

https://go2docs.graylog.org/5-0/getting_in_log_data/ingest_gelf.html

in go version 1.11 I was failed to run go get gopkg.in/gemnasium/logrus-graylog-hook.v2

the error message was

# gopkg.in/gemnasium/logrus-graylog-hook.v2
go/src/gopkg.in/gemnasium/logrus-graylog-hook.v2/error.go:52:15: cannot convert stacktrace[0] (type "github.com/pkg/errors".Frame) to type uintptr

so I fixed this line from

pc := uintptr(stacktrace[0])

into

pc := stacktrace[0].PC

and it seems to be working fine.

so I open this issue waiting it to be fixed soon. what do you think about this particular change?

Hey guys! When defining a level in GraylogHook.Level attribute it will be included also all lower levels. So if I define DebugLevel, it will also include InfoLevel, WarnLevel, ErrorLevel, FatalLevel and PanicLevel. But what if I just want to fire the hook when it's DebugLevel, InfoLevel or WarnLevel?

I think we would need more flexibility from the logic defined here.

Hi, firstly thanks for graylog hook and sorry by my poor english!

I was used to log to graylog using this lib to a shared graylog server that has a innumerous streams, differentiated by facilities.

When you removed facility arg for method NewGraylogHook (because this is deprecated but optional for GELF) and create a new release version, I kept in v1 version by the facilities.

But now, the github.com/SocialCodeInc/go-gelf/gelf is broken and you fix it in v2 but v1 is yet broken.

I understand when you cut off facility from method contract but why not do a optional arg (using pointer arg) or do something like a overload or a NewGraylogHookWithFacility anyway?

Thanks by your attention!

Hey,

I was trying to create a syncronous hook with no compression and it simply not send to graylog, it just work with assyncronous.

obs: I'm using v3.

graylogHook = graylog.NewGraylogHook(graylogURL, extra)
w := graylogHook.Writer();
if writer, ok := w.(*graylog.UDPWriter); ok {
    writer.CompressionType = graylog.NoCompress
    _ = graylogHook.SetWriter(writer)
}

Hello Gravis,

First of all, thanks a lot for this great logrus hook!

I'm using it in a distributed application written in various language. Because of that, I need to be able to trace the logs with millisecond precision otherwise graylog will not display them in order.

I forked this project, hoping I'd be able to do a merge request, but the gelf.Writer library you are using doesn't allow the gelf.Message to be a float64, and I also had to add a modified copy of it to my fork to have this working. Hence opening this bug instead of submitting a merge request.

Here is the project:
https://github.com/RaphYot/logrus-graylog-hook

I'm doing this conversion so that I can get the 3 millisecond numbers after the . separation of the TimeUnix in second:

        TimeUnix: float64(time.Now().UnixNano()/1000000) / 1000.,

Note that I also modified the README page to reflect the new repo, in the example it's still showing the legacy one.

Thanks!
Raph

To send logs encrypted

Not send to graylog if field name = duration
Like this:

logFields := map[string]interface{}{
				"method":       r.Method,
				"url":          r.URL.Path + "?" + r.URL.RawQuery,
				"duration":    fmt.Sprintf("%f seconds", time.Since(startTime).Seconds()),
				"datetime":     startTime,
			}

../go/src/gopkg.in/gemnasium/logrus-graylog-hook.v2/graylog_hook.go:87: too few values in struct initializer

Was working week ago ;-)

In case of unreachable graylog server GraylogHook.gelfLogger is silently initialized to nil: graylog_hook.go#L46 -> graylog_hook.go#L60
Then program crashes with nil pointer dereference on sending entry: graylog_hook.go#L146 -> graylog_hook.go#L208

I believe NewGraylogHook should return error and/or nil *GraylogHook or as a backward-compatible solution *GraylogHook should provide method to get initialization error.

The only way to actual get the v3 is to use the following:

go get github.com/gemnasium/logrus-graylog-hook/v3@84ae10c40

Since the fix for module path was introduced after 3.0.0 in #42 the only way to make this work normally is a new bugfix version 3.0.1

Hi:

any plan to update gopkg.in's version to 1.1.1?

I'd like to use async logger on my env,

thanks.

Why you've not added log level field for hook? I' suffering from the same problem in logstash hook.
I am developing high performance system and it degrade very much if it needs to allocate memory per each log entry, I want just to collect those messages that have severity INFO or higher.

Hi, when using the gralog-hook, the package could not compile with the following message:

case-insensitive import collision: "vendor/github.com/Sirupsen/logrus" and "vendor/github.com/sirupsen/logrus"

The https://github.com/sirupsen/logrus has changed the from "Sirupsen" to "sirupsen".
More details here:

• sirupsen/logrus#451 (comment)
• sirupsen/logrus#384

Note that I've vendored the latest logrus and I had to modify the logrus-gralog-hook's import locally.
Maybe logrus-gralog-hook should update the import statements with lower case "sirupsen"?

I am having problems to import v3.

According to official documentation the /v3 should be included in the module path.

Here is a PR that fix this #42

The code in graylog.Writer.WriteMessage() constructs a new gzip/zlib writer for every message. This slows down the entire application (over 50% of my application time was spent in runtime.gcDrain()).

By only creating a gzip/zlib writer once, this overhead is removed, speeding everything up.

Hello, I'm just opening this issue to let you know, the next release of logrus (v1.2.0) will add a new trace level named Trace below Debug.
Here is the PR for reference sirupsen/logrus#844
You may want to take into account this new level.

when adding an error in the field if we pass the error interface instead of passing err.Error() which is a string, cannot able to print the error message properly it only shows '{}' in the Graylog UI

I'm initializing the Graylog-Hook like this:

hook := graylog.NewGraylogHook(graylogAddr, map[string]interface{}{"project": project, "APP_ENV": appEnv}) logrus.AddHook(hook)

and sent a message directly afterwards:

logrus.WithFields(logrus.Fields{"uuid": startUpID, "event": "startUp"}).Infof("Log messages are now sent to Graylog (udp://%s)", graylogAddr)

in roughly 30% of program starts, no logs at all are published to graylog. Console prints the log lines just fine (did not set NullFormater). I also tried the Async Hook, but no difference. Either it just works after program start, or not at all.
Any idea what this might be caused by? We're using Graylog 3.3.2+ade4779.

I can provide more information if needed.