i cant subscribed topic about threatbus HOT 11 CLOSED

Hi @mavam

thank you very match. now it worked.

from threatbus.

Hi @wangzhengqing

MISP exposes it's own ZMQ endpoint and the Threat Bus plugin binds to it. You won't need to have a dedicated ZMQ endpoint on your machine. Kafka is a separate application. If you want to use Kafka, you first need to set up a Kafka cluster, then connect MISP and Threat Bus to it.

Note that Threat Bus only can use one channel to communicate with MISP, so you either use MISP ZMQ or a dedicated Kafka endpoint. For ease of use, I recommend starting with the built-in ZMQ endpoint in MISP.

from threatbus.

Hi @0ortmann

when i use threatbus.zeek to start subscrib p2p_topic. but always failed subscrib to p2p_topic. how to establish a connection p2p_topic

start threatbus:
$ threatbus -c config.yml
2020-11-10 09:59:39 INFO [threatbus] Starting plugins...
2020-11-10 09:59:39 INFO [threatbus_inmem.plugin] In-memory backbone started.
2020-11-10 09:59:39 INFO [threatbus_inmem.plugin] Adding subscription to: threatbus/snapshotrequest
2020-11-10 09:59:39 INFO [threatbus_inmem.plugin] Adding subscription to: threatbus/snapshotenvelope
2020-11-10 09:59:40 INFO [threatbus_zeek.plugin] Zeek plugin started
WARNING [api.py:136 - init() ] The version of PyMISP recommended by the MISP instance (2.4.134) is newer than the one you're using now (2.4.133). Please upgrade PyMISP.
2020-11-10 09:59:40 INFO [threatbus_inmem.plugin] Adding subscription to: threatbus/sighting
2020-11-10 09:59:40 INFO [threatbus_misp.plugin] MISP plugin started

start threatbus.zeek:
$ zeek -i ens224 -C threatbus.zeek
listening on ens224

./threatbus.zeek, line 120: subscribing to management topic threatbus/manage
./threatbus.zeek, lines 121-122: reporting noisy intel at 100 matches/sec
./threatbus.zeek, lines 136-137: peering to threatbus at 127.0.0.1:47761/tcp
1604973640.496772 ./threatbus.zeek, line 109: threatbus peered

^C1604974645.390012 received termination signal
1604974645.390012 28001 packets received on interface ens224, 0 dropped
1604974645.390012 ./threatbus.zeek, line 149: unsubscribing from p2p_topic

from threatbus.

Hi @wangzhengqing

can you please tell me a bit more about your setup? I need to know the following:

• What OS do you use to run Threat Bus and Zeek?
• What is the version of Threat Bus and the installed plugins?
• What Zeek version do you use?
• What is the version of the installed zeek/broker bindings?

from threatbus.

Hi, @0ortmann

MISP used vmdk run at EXSI and version 2.4.134
ZEEK OS: centos 7.8
Python：3.8
pip3: 20.2.4
ZEEK: 3.0.8
pip3 list | grep threatbus
threatbus 2020.10.29
threatbus-inmem 2020.10.29
threatbus-misp 2020.10.29
threatbus-zeek 2020.10.29
broker：1.4.0

from threatbus.

@wangzhengqing I think the problem is that Zeek and Threat Bus cannot communicate via broker because of a version mismatch. You use Zeek version 3.0.8 and broker 1.4.0. From the release notes of broker 1.4.0:

Note: Releases in the v1.4.x series are aligned for use in Zeek v3.2.x and do not support Zeek v3.0.x (Broker v1.2.x are meant to be used for that Zeek LTS release branch).

You need to either update Zeek or downgrade broker.

from threatbus.

Hi @0ortmann

now it works. zeek -i en111 -C threatbus.zeek
broker-1.4.0 also worked. my zeek manager listened another port. i changed port, now it worked. but in MISP Sightings 0/0/0 didn't changed.
now my broker version 1.2.8

and i have another question:
when i added to local.zeek, and operate in a cluster.

redef ignore_checksums = T;
@load somepath/threatbus.zeek

and run : zeekctl deploy

tailf reporter.log | grep threatbus
{"ts":1605080660.264733,"level":"Reporter::INFO","message":"subscribing to management topic threatbus/manage","location":"/zeek/spool/installed-scripts-do-not-touch/site/threatbus.zeek, line 120"}
{"ts":1605080660.264733,"level":"Reporter::INFO","message":"reporting noisy intel at 100 matches/sec","location":"/zeek/spool/installed-scripts-do-not-touch/site/threatbus.zeek, lines 121-122"}

it doesn't work again.
Does it have to use alone?

from threatbus.

Hi @wangzhengqing

The option zeek -C already disables checksum checking, so you don't need to set ignore_checksums = T; again.

From the help menu:

zeek -h
zeek version 3.2.0
usage: zeek [options] [file ...]
usage: zeek --test [doctest-options] -- [options] [file ...]
...
    -C|--no-checksums              | ignore checksums
...

However, that issue should be unrelated to Threat Bus.

from threatbus.

@wangzhengqing I'm trying to understand the problem a little better. Is it that Threat Bus runs in local mode but not in cluster mode?

from threatbus.

@mavam
when i run
zeek -i en111 -C threatbus.zeek
it worked

when i add threatbus.zeek to $PREFIX/zeek/share/zeek/site/local.zeek
@load mypath/threatbus.zeek
zeekctl deploy

it doesn't work again

this is my log:

$ cat reporter.log | grep threatbus
{"ts":1605147239.768316,"level":"Reporter::INFO","message":"subscribing to management topic threatbus/manage","location":"/usr/local/zeek/spool/installed-scripts-do-not-touch/site/threatbus.zeek, line 120"}
{"ts":1605147239.768316,"level":"Reporter::INFO","message":"reporting noisy intel at 100 matches/sec","location":"/usr/local/zeek/spool/installed-scripts-do-not-touch/site/threatbus.zeek, lines 121-122"}

from threatbus.

Apologies for the delay, @wangzhengqing.

I have a hunch that this issue is related to the conditional execution of Zeek cluster code. Could I ask you to replace one line of Zeek code and let us know whether it works then?

If you replace line 132 of apps/zeek/threatbus.zeek from

@if ( ! Cluster::is_enabled() )

to

@if ( ! Cluster::is_enabled()
      || Cluster::local_node_type() == Cluster::MANAGER )

does the error still persist?

from threatbus.