Comments (6)
Running with BROKER_DEBUG_LEVEL=trace
is what seems to work for me in Zeek/Broker (didn't check into zeek-agent or other log levels). Examples:
Here's a release Zeek build with no special CAF logging configuration:
$ ./configure --build-type=release && make
...
$ BROKER_DEBUG_LEVEL=trace zeek -r ~/Desktop/get.trace
$ cat broker_78238_1578680818358148000.log
142 caf DEBUG actor0 0x700001db8000 caf.logger log_first_line logger.cpp:613 level = trace, node = 782D1884932E7041B05E3337493D71E11B8BBD00#78238, component-blacklist = []
142 broker INFO actor0 0x11aed2dc0 broker.endpoint endpoint endpoint.cc:195 creating endpoint
142 broker INFO actor0 0x11aed2dc0 ,%20std.vector<topic> subscriber subscriber.cc:156 creating subscriber for topic(s) []
142 broker TRACE actor7 0x700001aa6000 broker operator() core_actor.cc:399 ENTRY filter = []
142 broker TRACE actor7 0x700001aa6000 broker.detail.core_policy add_worker core_policy.cc:365 ENTRY filter = []
...
A standalone Broker debug build (don't think debug matters, just what I happened to build) and again without special CAF logging configuration (looks like it defaults to QUIET logging level):
$ ./configure --build-type=debug && make install
...
$ c++ -std=c++11 -I<broker-install>/include/ listener.cc -L<broker-install>lib/ -lbroker -lcaf_core -lcaf_openssl -lcaf_io -o listener
$ BROKER_DEBUG_LEVEL=trace ./listener
...
$ cat broker_79295_1578681249263050000.log
165 caf DEBUG actor0 0x70000ce22000 caf.logger log_first_line logger.cpp:613 level = trace, node = 782D1884932E7041B05E3337493D71E11B8BBD00#79295, component-blacklist = []
165 broker INFO actor0 0x119d18dc0 broker.endpoint endpoint endpoint.cc:195 creating endpoint
165 broker INFO actor0 0x119d18dc0 ,%20std.vector<topic> subscriber subscriber.cc:156 creating subscriber for topic(s) ["/topic/test"]
165 broker INFO actor0 0x119d18dc0 broker.endpoint listen endpoint.cc:237 listening on :9999 (no SSL)
...
from broker.
I was also testing current master
branches, not sure if it differs from behavior of release versions.
from broker.
The plan is to add the capability to switch on two levels of Broker-side debug output:
- INFO: high-level connections activity (setup, handshake, teardown, any errors or warnings, such as version differences)
- DEBUG: A dump of all messages going over the Broker sessions.
Turning that on will work through an environment variable and through config files, and it will work in both release & debug builds.
Goal is to get this into Zeek 3.1 still.
from broker.
A standalone Broker debug build (don't think debug matters, just what I happened to build) and again without special CAF logging configuration (looks like it defaults to QUIET logging level)
CAF defaults to quiet, but the logger is always available since recent CAF versions.
Turning that on will work through an environment variable and through config files, and it will work in both release & debug builds.
Broker uses its own set of logging macros. They are independent of whatever CAF_LOG_LEVEL
may be. Currently, we always compile Broker with all logging branches. At runtime, Broker has to "pay" one if
-branch per log statement to check whether the log level at runtime permits logging the event. Notably, all BROKER_TRACE
statements currently have to check twice (at function entry and exit).
So far I did not measure how much overhead these extra branches amount to. Also, there's currently no build option for removing them. Maybe we should run some benchmarks to see whether it's worth it to add a BROKER_LOG_LEVEL
in order to remove trace statements from release builds.
warnings, such as version differences
I'm working on this, but as it turns out: CAF doesn't propagate this properly. CAF only signals sec::disconnect_during_handshake
. But this error actually can be any of these things:
- an actual, unexpected TCP disconnect
- an application ID mismatch (e.g.,
broker-v1
vsbroker-v2
) - a CAF version mismatch (e.g. CAF 0.16.x vs CAF 0.17.x)
Broker needs to knows which of the three it is, because only in one case we'd try to reconnect.
Goal is to get this into Zeek 3.1 still.
I'll try to get this fixed on the CAF side with an 0.17.4 bugfix release in time.
from broker.
With the patch to CAF (submitted as PR 1023), I now see this output when trying to connect two incompatible Broker versions:
starting to peer with localhost:8080 retry: 1000000000ns [synchronous]
initiating connection to localhost:8080 (SSL)
remote endpoint unavailable: system_error(incompatible_application_id)
Tested by changing the application ID for one Broker endpoint on the command line. We already log this error in core_actor.cc
as error, so there are no changes to Broker necessary. The CAF change plus PR #80 should make it easier to troubleshoot similar issues.
I'll also do a pass over Broker to improve usefulness of INFO and DEBUG logs, but I'd prefer to do that on the "new" code in topic/neverlord/multi-hop-routing
. Otherwise, we'd have to do the same work twice because the core actor has been redesigned for the ALM feature.
from broker.
Think we've got everything to call this done now: improved configuration for Broker logs and now using CAF 0.17.4 branch to better report version mismatches. Or re-open / correct me if wrong.
from broker.
Related Issues (20)
- Introduce versioning for the Zeek-layer serialization format
- Generic mechanism to add metadata to Zeek events.
- Websocket API: holes in vectors not preseved HOT 3
- docs/websocket: ISO 8601 discrepancies HOT 1
- Broker metrics port setting overriding BROKER_METRICS_PORT env variable HOT 2
- High caf thread utilization with 512 workers and 96 loggers HOT 10
- caf::actor_control_block::enqueue segfault when running supervisor.config-bare-mode test
- Websocket API: messages truncated to 4KB HOT 13
- Allow WebSocket and native clients on the same socket HOT 5
- clang-tidy fails with version 15+ HOT 2
- Metrics scalability performance issue in 6.0 HOT 12
- Resurrect `broker-cluster-benchmark`
- New Broker WebSocket Python bindings
- Set TLS minimum proto version to 1.2 or greater. HOT 6
- Make broker::data_envelope::deserialize() for JSON format accessible HOT 1
- Missing compatibility list for Zeek HOT 2
- Broker include broken HOT 4
- Convert Broker to use prometheus-cpp registry from Zeek HOT 1
- Broker message provenance tracking HOT 2
- Allow tapping into Broker logs
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from broker.