gemnasium / logrus-graylog-hook Goto Github PK
View Code? Open in Web Editor NEWGraylog hook for logrus
License: MIT License
Graylog hook for logrus
License: MIT License
Hey,
I was trying to create a syncronous hook with no compression and it simply not send to graylog, it just work with assyncronous.
obs: I'm using v3.
graylogHook = graylog.NewGraylogHook(graylogURL, extra)
w := graylogHook.Writer();
if writer, ok := w.(*graylog.UDPWriter); ok {
writer.CompressionType = graylog.NoCompress
_ = graylogHook.SetWriter(writer)
}
Hi:
any plan to update gopkg.in's version to 1.1.1?
I'd like to use async logger on my env,
thanks.
i keep getting
got code 202 Accepted, expected 204
based on gravelog document, 202 should be expected
Both keep-alive and compression are supported via the common HTTP headers. The server will return a
202 Accepted
when the message was accepted for processing.
https://go2docs.graylog.org/5-0/getting_in_log_data/ingest_gelf.html
The code in graylog.Writer.WriteMessage()
constructs a new gzip/zlib writer for every message. This slows down the entire application (over 50% of my application time was spent in runtime.gcDrain()
).
By only creating a gzip/zlib writer once, this overhead is removed, speeding everything up.
Hi, when using the gralog-hook, the package could not compile with the following message:
case-insensitive import collision: "vendor/github.com/Sirupsen/logrus" and "vendor/github.com/sirupsen/logrus"
The https://github.com/sirupsen/logrus has changed the from "Sirupsen" to "sirupsen".
More details here:
Note that I've vendored the latest logrus and I had to modify the logrus-gralog-hook's import locally.
Maybe logrus-gralog-hook should update the import statements with lower case "sirupsen"?
Since we're sending message in a goroutine, the very last messages are not sent to graylog on program exit, the routine being destroyed before the end of the queue.
When I enable the report caller in logrus by setting log.SetReportCaller(true), then the fields file and func are visible when in stdout.
In Graylog these fields are not visible at all.
Is this a bug or normal behavior?
Why you've not added log level field for hook? I' suffering from the same problem in logstash hook.
I am developing high performance system and it degrade very much if it needs to allocate memory per each log entry, I want just to collect those messages that have severity INFO or higher.
when adding an error in the field if we pass the error interface instead of passing err.Error() which is a string, cannot able to print the error message properly it only shows '{}' in the Graylog UI
https://github.com/gemnasium/logrus-graylog-hook/blob/v3.0.0/.travis.yml need to be updated to reflect changes in v3
I think that #42 needs to be merged first for this
../go/src/gopkg.in/gemnasium/logrus-graylog-hook.v2/graylog_hook.go:87: too few values in struct initializer
Was working week ago ;-)
Hello,
Thanks you for this awesome hook!
We are facing a problem with log levels. When we log with logrus.info, the log level in graylog is 4 (warning in syslog) instead of 6.
Can we map the logrus level with syslog level before sending to graylog?
Hello, I'm just opening this issue to let you know, the next release of logrus (v1.2.0) will add a new trace level named Trace below Debug.
Here is the PR for reference sirupsen/logrus#844
You may want to take into account this new level.
TestStackTracer verifies stacktraces with a regex, but this regex assumes all paths start with a /
, which is not the case on Windows.
I am having problems to import v3.
According to official documentation the /v3 should be included in the module path.
Here is a PR that fix this #42
To send logs encrypted
When i use two logger instance concurrently then i get "invalid memory address or nil pointer dereference"
Here a sample code to reproduce the error
package main
import (
graylog "github.com/gemnasium/logrus-graylog-hook/v3"
"github.com/sirupsen/logrus"
"sync"
)
func main() {
var wg sync.WaitGroup
var gelfHook = graylog.NewGraylogHook("xx.xx.xx.xx:xxx", map[string]interface{}{})
log1 := logrus.New()
log2 := logrus.New()
log1.AddHook(gelfHook)
log2.AddHook(gelfHook)
for i := 0; i < 100; i++ {
wg.Add(1)
go func() {
defer wg.Done()
log1.Info("test log info")
log2.Info("test log info")
}()
}
wg.Wait()
}
and the error message
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x4d19da]
goroutine 23 [running]:
compress/flate.(*Writer).Close(...)
/usr/lib/go-1.13/src/compress/flate/deflate.go:729
compress/gzip.(*Writer).Close(0xc0000be210, 0xc0004801e0, 0x97)
/usr/lib/go-1.13/src/compress/gzip/gzip.go:248 +0x6a
github.com/gemnasium/logrus-graylog-hook/v3.(*Writer).WriteMessage(0xc0000d0070, 0xc00041a100, 0xc000141ac0, 0x0)
/xxxx/go/pkg/mod/github.com/gemnasium/logrus-graylog-hook/[email protected]/gelf_writer.go:246 +0x2e4
github.com/gemnasium/logrus-graylog-hook/v3.(*GraylogHook).sendEntry(0xc0000ba120, 0xc000119340, 0x0, 0x0, 0x0)
/xxxx/go/pkg/mod/github.com/gemnasium/logrus-graylog-hook/[email protected]/graylog_hook.go:247 +0xa2b
github.com/gemnasium/logrus-graylog-hook/v3.(*GraylogHook).Fire(0xc0000ba120, 0xc0001192d0, 0x0, 0x0)
/xxxx/go/pkg/mod/github.com/gemnasium/logrus-graylog-hook/[email protected]/graylog_hook.go:120 +0x294
github.com/sirupsen/logrus.LevelHooks.Fire(0xc00009c750, 0x4, 0xc0001192d0, 0xc0001192d0, 0x40d3a4)
/xxxx/go/pkg/mod/github.com/sirupsen/[email protected]/hooks.go:28 +0x91
github.com/sirupsen/logrus.(*Entry).fireHooks(0xc0001192d0)
/xxxx/go/pkg/mod/github.com/sirupsen/[email protected]/entry.go:246 +0xa3
github.com/sirupsen/logrus.Entry.log(0xc0000d0150, 0xc0001104e0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
/xxxx/go/pkg/mod/github.com/sirupsen/[email protected]/entry.go:224 +0xf4
github.com/sirupsen/logrus.(*Entry).Log(0xc000119260, 0xc000000004, 0xc000141f98, 0x1, 0x1)
/xxxx/go/pkg/mod/github.com/sirupsen/[email protected]/entry.go:268 +0xeb
github.com/sirupsen/logrus.(*Logger).Log(0xc0000d0150, 0x4, 0xc000141f98, 0x1, 0x1)
/xxxx/go/pkg/mod/github.com/sirupsen/[email protected]/logger.go:192 +0x7d
github.com/sirupsen/logrus.(*Logger).Info(...)
/xxxx/go/pkg/mod/github.com/sirupsen/[email protected]/logger.go:206
main.main.func1(0xc0000b4250, 0xc0000d00e0, 0xc0000d0150)
/xxxx/main.go:25 +0x110
created by main.main
/xxxx/main.go:22 +0x26e
I think that something will be override because if add this lines
w.mu.Lock()
defer w.mu.Unlock()
to gelf_writer.go line 198 then everything work as expected
Hey guys! When defining a level in GraylogHook.Level
attribute it will be included also all lower levels. So if I define DebugLevel
, it will also include InfoLevel
, WarnLevel
, ErrorLevel
, FatalLevel
and PanicLevel
. But what if I just want to fire the hook when it's DebugLevel
, InfoLevel
or WarnLevel
?
I think we would need more flexibility from the logic defined here.
Hey, the following code produces a race condition in graylog_hook.go line 146 because there is a read access on the Data map outside the Hook interfaces Fire function.
package main
import (
"time"
"gopkg.in/gemnasium/logrus-graylog-hook.v1"
"github.com/Sirupsen/logrus"
)
func main() {
log := logrus.New()
hook := graylog.NewAsyncGraylogHook("localhost:12201", "facility", map[string]interface{}{"this": "is logged every time"})
defer hook.Flush()
log.Hooks.Add(hook)
go func() {
log.WithFields(logrus.Fields{"msg": "test1"}).Info("test1")
}()
log.WithFields(logrus.Fields{"msg": "test2"}).Info("test2")
ch := time.Tick(time.Second * 1)
<-ch
}
Not send to graylog if field name = duration
Like this:
logFields := map[string]interface{}{
"method": r.Method,
"url": r.URL.Path + "?" + r.URL.RawQuery,
"duration": fmt.Sprintf("%f seconds", time.Since(startTime).Seconds()),
"datetime": startTime,
}
I'm initializing the Graylog-Hook like this:
hook := graylog.NewGraylogHook(graylogAddr, map[string]interface{}{"project": project, "APP_ENV": appEnv}) logrus.AddHook(hook)
and sent a message directly afterwards:
logrus.WithFields(logrus.Fields{"uuid": startUpID, "event": "startUp"}).Infof("Log messages are now sent to Graylog (udp://%s)", graylogAddr)
in roughly 30% of program starts, no logs at all are published to graylog. Console prints the log lines just fine (did not set NullFormater). I also tried the Async Hook, but no difference. Either it just works after program start, or not at all.
Any idea what this might be caused by? We're using Graylog 3.3.2+ade4779.
I can provide more information if needed.
Hello Gravis,
First of all, thanks a lot for this great logrus hook!
I'm using it in a distributed application written in various language. Because of that, I need to be able to trace the logs with millisecond precision otherwise graylog will not display them in order.
I forked this project, hoping I'd be able to do a merge request, but the gelf.Writer library you are using doesn't allow the gelf.Message to be a float64, and I also had to add a modified copy of it to my fork to have this working. Hence opening this bug instead of submitting a merge request.
Here is the project:
https://github.com/RaphYot/logrus-graylog-hook
I'm doing this conversion so that I can get the 3 millisecond numbers after the . separation of the TimeUnix in second:
TimeUnix: float64(time.Now().UnixNano()/1000000) / 1000.,
Note that I also modified the README page to reflect the new repo, in the example it's still showing the legacy one.
Thanks!
Raph
Dependency is broken, due to removal of repository http://github.com/SocialCodeInc/go-gelf/gelf
in go version 1.11 I was failed to run go get gopkg.in/gemnasium/logrus-graylog-hook.v2
the error message was
# gopkg.in/gemnasium/logrus-graylog-hook.v2
go/src/gopkg.in/gemnasium/logrus-graylog-hook.v2/error.go:52:15: cannot convert stacktrace[0] (type "github.com/pkg/errors".Frame) to type uintptr
so I fixed this line from
pc := uintptr(stacktrace[0])
into
pc := stacktrace[0].PC
and it seems to be working fine.
so I open this issue waiting it to be fixed soon. what do you think about this particular change?
In case of unreachable graylog server GraylogHook.gelfLogger
is silently initialized to nil: graylog_hook.go#L46 -> graylog_hook.go#L60
Then program crashes with nil pointer dereference on sending entry: graylog_hook.go#L146 -> graylog_hook.go#L208
I believe NewGraylogHook
should return error and/or nil *GraylogHook
or as a backward-compatible solution *GraylogHook
should provide method to get initialization error.
Hi, firstly thanks for graylog hook and sorry by my poor english!
I was used to log to graylog using this lib to a shared graylog server that has a innumerous streams, differentiated by facilities.
When you removed facility arg for method NewGraylogHook
(because this is deprecated but optional for GELF) and create a new release version, I kept in v1 version by the facilities.
But now, the github.com/SocialCodeInc/go-gelf/gelf is broken and you fix it in v2 but v1 is yet broken.
I understand when you cut off facility from method contract but why not do a optional arg (using pointer arg) or do something like a overload or a NewGraylogHookWithFacility
anyway?
Thanks by your attention!
The only way to actual get the v3 is to use the following:
go get github.com/gemnasium/logrus-graylog-hook/v3@84ae10c40
Since the fix for module path was introduced after 3.0.0 in #42 the only way to make this work normally is a new bugfix version 3.0.1
Hi. first of all thanks for providing graylog hook. it's pretty help for me
i suggest you a feature (this requires little bit change i guess)
currently there is no option to set compression type and i believe the compression is important feature of GELF format specification.
i found that the struct has gelf.Writer as a member that has a interface for setting the compression type but it's not accessible from outside.
type GraylogHook struct {
Facility string
Extra map[string]interface{}
gelfLogger *gelf.Writer
buf chan graylogEntry
wg sync.WaitGroup
mu sync.RWMutex
synchronous bool
}
if you replace the first character 'g' of "gelfLogger" with "G" , all problems are solved cause user can access directly to gelf.writer configuration or maybe you can add some gelf.writer configurable interface in GraylogHook.
please consider it.
thanks.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.