Code Monkey home page Code Monkey logo

ipfix's Introduction

Go Reference

Gravwell Open-Source Code

This repository contains open-sourced libraries and commands developed by Gravwell.

There are a selection of Gravwell-specific libraries and tools:

  • ingest/ contains the ingest library, which is used to connect to a Gravwell indexer and upload data.
  • ingesters/ contains the source code for Gravwell ingesters.
  • generators/ is a collection of tools that generate artificial data for testing Gravwell or any other log analytics system.
  • manager/ provides a very simple init command which we use in Docker containers.
  • chancacher/ implements a caching library we use for ingesters.

There are also a few libraries which may be of use outside Gravwell-specific applications:

  • filewatch/ is a library that can monitor files on the filesystem for changes; we use this in the FileFollow ingester.
  • timegrinder/ is a timestamp extraction library we use to extract timestamps from arbitrary data
  • ipexist/ contains a library for efficiently storing and checking for the existence of an IPv4 set with high density sets.
  • winevent/ is a library which can interact with the Windows Event subsystem to extract XML rendered events.

ipfix's People

Contributors

asenci avatar ashnwade avatar available-username avatar calmh avatar floren avatar georgmu avatar john-floren-gravwell avatar kris-watts-gravwell avatar lebauce avatar

Stargazers

avatar avatar avatar

Watchers

avatar avatar avatar

Forkers

traetox floren lyckety david-fritz-gravwell ashnwade

ipfix's Issues

slice bounds check failed in ipfix packet parser

What are the steps to reproduce this issue?

  1. unknown

What happens?

panic in packet parser

panic: runtime error: slice bounds out of range [:516] with capacity 508

goroutine 23 [running]:
github.com/gravwell/ipfix.Message.marshalRecords({{0xa, 0x3cc, 0x0, 0x65860018, 0x1af8, 0xa}, {0xc0001d2000, 0x15, 0x28}, {0xc000160a80, ...}}, ...)
    github.com/gravwell/[email protected]/parser.go:863 +0x55a
github.com/gravwell/ipfix.(*Session).Marshal(0xc0000b62d0, {{0xa, 0x3cc, 0x0, 0x65860018, 0x1af8, 0xa}, {0xc0001d2000, 0x15, 0x28}, ...})
    github.com/gravwell/[email protected]/parser.go:737 +0x1ca
main.(*IpfixHandler).routine(0xc0001235c0, 0x0?)
    github.com/gravwell/gravwell/v3/ingesters/netflow/handlers.go:301 +0xb13
created by main.(*IpfixHandler).Start in goroutine 1
    github.com/gravwell/gravwell/v3/ingesters/netflow/handlers.go:186 +0xfb

What were you expecting to happen?

Not panic

Any logs, error output, etc?

panic: runtime error: slice bounds out of range [:516] with capacity 508

goroutine 23 [running]:
github.com/gravwell/ipfix.Message.marshalRecords({{0xa, 0x3cc, 0x0, 0x65860018, 0x1af8, 0xa}, {0xc0001d2000, 0x15, 0x28}, {0xc000160a80, ...}}, ...)
    github.com/gravwell/[email protected]/parser.go:863 +0x55a
github.com/gravwell/ipfix.(*Session).Marshal(0xc0000b62d0, {{0xa, 0x3cc, 0x0, 0x65860018, 0x1af8, 0xa}, {0xc0001d2000, 0x15, 0x28}, ...})
    github.com/gravwell/[email protected]/parser.go:737 +0x1ca
main.(*IpfixHandler).routine(0xc0001235c0, 0x0?)
    github.com/gravwell/gravwell/v3/ingesters/netflow/handlers.go:301 +0xb13
created by main.(*IpfixHandler).Start in goroutine 1
    github.com/gravwell/gravwell/v3/ingesters/netflow/handlers.go:186 +0xfb

goroutine 1 [chan receive, 81958 minutes]:
github.com/gravwell/gravwell/v3/ingesters/utils.WaitForQuit()
    github.com/gravwell/gravwell/v3/ingesters/utils/signals.go:25 +0x128
main.main()
    github.com/gravwell/gravwell/v3/ingesters/netflow/main.go:142 +0x1845

goroutine 18 [select]:
github.com/gravwell/gravwell/v3/chancacher.(*ChanCacher).cacheHandler(0xc000121170)
    github.com/gravwell/gravwell/v3/chancacher/chancacher.go:292 +0x41b
created by github.com/gravwell/gravwell/v3/chancacher.NewChanCacher in goroutine 1
    github.com/gravwell/gravwell/v3/chancacher/chancacher.go:196 +0xa6d

goroutine 19 [chan receive, 29910 minutes]:
github.com/gravwell/gravwell/v3/chancacher.(*ChanCacher).run(0xc000121170)
    github.com/gravwell/gravwell/v3/chancacher/chancacher.go:207 +0x45
created by github.com/gravwell/gravwell/v3/chancacher.NewChanCacher in goroutine 1
    github.com/gravwell/gravwell/v3/chancacher/chancacher.go:198 +0xaaf

goroutine 20 [select]:
github.com/gravwell/gravwell/v3/chancacher.(*ChanCacher).cacheHandler(0xc000098000)
    github.com/gravwell/gravwell/v3/chancacher/chancacher.go:292 +0x41b
created by github.com/gravwell/gravwell/v3/chancacher.NewChanCacher in goroutine 1
    github.com/gravwell/gravwell/v3/chancacher/chancacher.go:196 +0xa6d

goroutine 21 [chan receive]:
github.com/gravwell/gravwell/v3/chancacher.(*ChanCacher).run(0xc000098000)
    github.com/gravwell/gravwell/v3/chancacher/chancacher.go:207 +0x45
created by github.com/gravwell/gravwell/v3/chancacher.NewChanCacher in goroutine 1
    github.com/gravwell/gravwell/v3/chancacher/chancacher.go:198 +0xaaf

goroutine 5 [chan receive, 29910 minutes]:
github.com/gravwell/gravwell/v3/ingest.(*IngestMuxer).connRoutine(0xc000004300, 0x0)
    github.com/gravwell/gravwell/v3/ingest/muxer.go:1306 +0x3b7
created by github.com/gravwell/gravwell/v3/ingest.(*IngestMuxer).Start in goroutine 1
    github.com/gravwell/gravwell/v3/ingest/muxer.go:483 +0x1a8

goroutine 6 [sleep]:
time.Sleep(0x12a05f200)
    runtime/time.go:195 +0x125
github.com/gravwell/gravwell/v3/ingest.(*IngestMuxer).stateReportRoutine(0xc000004300)
    github.com/gravwell/gravwell/v3/ingest/muxer.go:551 +0x3f
created by github.com/gravwell/gravwell/v3/ingest.(*IngestMuxer).Start in goroutine 1
    github.com/gravwell/gravwell/v3/ingest/muxer.go:488 +0x278

goroutine 22 [select]:
github.com/gravwell/gravwell/v3/ingest.(*IngestMuxer).writeRelayRoutine(0xc000004300, 0xc0001234a0, 0xc0002280e0)
    github.com/gravwell/gravwell/v3/ingest/muxer.go:1122 +0x357
created by github.com/gravwell/gravwell/v3/ingest.(*IngestMuxer).connRoutine in goroutine 5
    github.com/gravwell/gravwell/v3/ingest/muxer.go:1299 +0x1f9

goroutine 24 [IO wait, 81958 minutes]:
internal/poll.runtime_pollWait(0x7fd0e8ae69a8, 0x72)
    runtime/netpoll.go:343 +0x85
internal/poll.(*pollDesc).wait(0xc000215300?, 0xc0001079b8?, 0x0)
    internal/poll/fd_poll_runtime.go:84 +0x27
internal/poll.(*pollDesc).waitRead(...)
    internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).ReadFromInet6(0xc000215300, {0xc0001079b8, 0x5b8, 0x5b8}, 0x0?)
    internal/poll/fd_unix.go:274 +0x22b
net.(*netFD).readFromInet6(0xc000215300, {0xc0001079b8?, 0x0?, 0x0?}, 0x0?)
    net/fd_posix.go:72 +0x25
net.(*UDPConn).readFrom(0x30?, {0xc0001079b8?, 0xc000034d80?, 0x0?}, 0xc000034d80)
    net/udpsock_posix.go:59 +0x79
net.(*UDPConn).readFromUDP(0xc000116658, {0xc0001079b8?, 0x0?, 0x0?}, 0x0?)
    net/udpsock.go:149 +0x30
net.(*UDPConn).ReadFromUDP(...)
    net/udpsock.go:141
main.(*NetflowV5Handler).routine(0xc000123620, 0x2?)
    github.com/gravwell/gravwell/v3/ingesters/netflow/handlers.go:104 +0xec
created by main.(*NetflowV5Handler).Start in goroutine 1
    github.com/gravwell/gravwell/v3/ingesters/netflow/handlers.go:90 +0xfb

goroutine 25 [select]:
main.relay(0xc000123560, 0x0?, {0x0, 0x0, 0x0}, 0x0?)
    github.com/gravwell/gravwell/v3/ingesters/netflow/main.go:186 +0x13b
created by main.main in goroutine 1
    github.com/gravwell/gravwell/v3/ingesters/netflow/main.go:137 +0x1809

goroutine 7 [syscall, 81958 minutes]:
os/signal.signal_recv()
    runtime/sigqueue.go:152 +0x29
os/signal.loop()
    os/signal/signal_unix.go:23 +0x13
created by os/signal.Notify.func1.1 in goroutine 1
    os/signal/signal.go:151 +0x1f

Any other comments?

Seems like a pretty standard lack of a bounds check: https://github.com/gravwell/ipfix/blob/master/parser.go#L863

What versions of software are you using?

5.4.1

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.