Code Monkey home page Code Monkey logo

trireme-lib's Introduction

Trireme

Build Status codecov Twitter URL Slack URL License Documentation Go Report Card

Welcome to Trireme, an open-source library curated by Aporeto to provide cryptographic isolation for cloud-native applications. Trireme-lib is a Zero-Trust networking library that makes it possible to setup security policies and segment applications by enforcing end-to-end authentication and authorization without the need for complex control planes or IP/port-centric ACLs and east-west firewalls.

Trireme-lib supports both containers and Linux processes as well user-based activation, and it allows security policy enforcement between any of these entities.

TL;DR

Trireme-lib is a library. The following projects use it:

Description

In the Trireme world, a processing unit (PU) end-point can be a container, Kubernetes POD, or a general Linux process. It can also be a user session to a particular server. We will be referring to processing units as PUs throughout this discussion.

The technology behind Trireme is streamlined, elegant, and simple. It is based on the concepts of Zero-Trust networking:

  1. The identity is the set of attributes and metadata that describes the container as key/value pairs. Trireme provides an extensible interface for defining these identities. Users can choose customized methods appropriate to their environment for establishing PU identity. For example, in a Kubernetes environment, the identity can be the set of labels identifying a POD.
  2. There is an authorization policy that defines when PUs with different types of identity attributes can interact or exchange traffic. The authorization policy implements an Attribute-Based Access Control (ABAC) mechanism (https://en.wikipedia.org/wiki/Attribute-Based_Access_Control), where the policy describes relationships between identity attributes.
  3. Every communication between two PUs is controlled through a cryptographic end-to-end authentication and authorization step, by overlaying an authorization function over the TCP negotiation. The authorization steps are performed during the SYN/SYNACK/ACK negotiation.

The result of this approach is the decoupling of network security from the underlying network infrastructure because this approach is centered on workload identity attributes and interactions between workloads. Network security can be achieved simply by managing application identity and authorization policy. Segmentation granularity can be adjusted based on the needs of the platform.

Trireme is a node-centric library. Each node participating in the Trireme cluster must spawn one instance of a process that uses this library to transparently insert the authentication and authorization step. Trireme provides the data path functions but does not implement either the identity management or the policy resolution function. Function implementation depends on the particular operational environment. Users have to provide PolicyLogic (ABAC “rules”) to Trireme for well-defined PUs, such as containers.

Existing implementation using Trireme library

  • This example is a straightforward implementation of the PolicyLogic for a simple use-case.

  • Kubernetes-Integration is a full implementation of PolicyLogic that follows the Kubernetes Network Policies model.

  • Bare-Metal-Integration is an implementation of Trireme for Kubernetes on-prem, with a Cumulus agent that allows you to have a very simple networking model (routes are advertised by Cumulus) together with Trireme for policy enforcement.

Security Model

Trireme is a Zero-Trust networking library. The security model behind Zero-trust networking is:

  • The Network is always untrusted. It doesn't matter if you are inside or outside your enterprise.
  • Every Flow/Connection needs to be authenticated and authorized by the endpoints.
  • The network information (IP/Port) is completely irrelevant to the authorization/authentication.

With Trireme, there is no need to define any security rules with IPs, port numbers, or ACLs. Everything is based on identity attributes; your IP and port allocation scheme is not relevant to Trireme and it is compatible with most underlying networking technologies. The end-to-end authentication and authorization approach is also compatible with NATs and IPv4/IPv6 translations.

A PU is a logical unit of control to which you attach identity and authorization policies. It provides a simple mechanism where the identity is derived out of the Docker manifest; however, other mechanisms are possible for more sophisticated identity definition. For instance, you may want to tag your 3-tier container application as "frontend," "backend," and "database." By associating corresponding labels and containers, these labels become "the identity." A policy for the “backend” containers can simply accept traffic only from “frontend” containers. Alternatively, an orchestration system might define a composite identity for each container and implement more sophisticated policies.

PolicyLogic defines the set of authorization rules as a function of the identity of attributes and loads these rules into Trireme when a container is instantiated. Authorization rules describe the set of identities with which a particular container is allowed to interact. We provide an example of this integration logic with Kubernetes here. Furthermore, we provide an example of a simple policy where two containers can only talk to each other if they have matching labels in this example. Each rule defines a match based on the identity attributes. PolicyLogic assumes a whitelist model where everything is dropped unless explicitly allowed by the authorization policy.

PU identities are cryptographically signed with a node-specific secret and sent as part of a TCP connection setup negotiation. Trireme supports both mutual and receiver-only authorization. Moreover, it supports two authentication and signing modes: (1) A pre-shared key and (2) a PKI mechanism based on ECDSA. In the case of ECDSA, public keys are either transmitted on the wire or pre-populated through an out-of-band mechanism to improve efficiency. Trireme also supports two identity encoding mechanisms: (1) A signed JSON Web Token (JWT) and (2) a custom binary mapping mechanism.

With these mechanisms, the Trireme run-time on each node will only allow communication after an end-to-end authentication and authorization step is performed between the containers.

Trireme Architecture

Trireme-lib is built as a set of modules (Go packages) that provide a default implementation for each component. It is simple to swap the default implementation of each of those modules with custom-built ones for more complex and specific features.

Conceptually, Trireme acts on PU events. In the default implementation, the PU is a Docker container. Trireme can be easily extended to other PUs such as processes, files, sockets, and so forth. Trireme consists of two main packages:

  • The Monitor listens to a well-defined PU creation module. The built-in monitor listens to Docker events and generates a standard Trireme PU runtime representation. Additional monitors provided can listen to events on creation of Linux processes or user sessions from the Linux PAM module. The Monitor hands over the PU runtime to an external Resolver.
    • The Resolver is implemented outside of Trireme and is not part of the library. The Resolver depends on the orchestration system used for managing identity and policy. If you plan to implement your own policy with Trireme, you will essentially need to implement a Resolver.
  • The Controller receives instructions from the Resolver and enforces the policy by analyzing the redirected packets and enforcing the identity and policy rules.

Defining Your Own Policy

Trireme allows you to define any type of identity attribute or policy to associate with the PUs. In order to define your own policies and identities, you need to implement a Resolver interface that will receive policy requests from Trireme whenever a policy resolution is required.

Resolver Implementation

// A Resolver must be implemented by a policy engine that receives monitor events.
type Resolver interface {

	// HandlePUEvent is called by all monitors when a PU event is generated. The implementer
	// is responsible to update all components by explicitly adding a new PU.
	HandlePUEvent(ctx context.Context, puID string, event common.Event, runtime RuntimeReader) error
}

Each container event generates a call to HandlePUEvent

The Resolver can then issue explicit calls to the Controller in order to implement the policy decision. The Controller interface is consumed by the Resolver and it is described below:

// TriremeController is the main API of the Trireme controller
type TriremeController interface {
	// Run initializes and runs the controller.
	Run(ctx context.Context) error

	// CleanUp cleans all the supervisors and ACLs for a clean exit
	CleanUp() error

	// Enforce asks the controller to enforce policy on a processing unit
	Enforce(ctx context.Context, puID string, policy *policy.PUPolicy, runtime *policy.PURuntime) (err error)

	// UnEnforce asks the controller to un-enforce policy on a processing unit
	UnEnforce(ctx context.Context, puID string, policy *policy.PUPolicy, runtime *policy.PURuntime) (err error)

	// UpdatePolicy updates the policy of the isolator for a container.
	UpdatePolicy(ctx context.Context, puID string, policy *policy.PUPolicy, runtime *policy.PURuntime) error

	// UpdateSecrets updates the secrets of running enforcers managed by trireme. Remote enforcers will get the secret updates with the next policy push
	UpdateSecrets(secrets secrets.Secrets) error

	// UpdateConfiguration updates the configuration of the controller. Only specific configuration
	// parameters can be updated during run time.
	UpdateConfiguration(networks []string) error
}

Prerequisites

  • Trireme-lib requires IPTables with access to the Mangle module.
  • Trireme-lib requires access to the Docker event API socket (/var/run/docker.sock by default).
  • Trireme-lib requires privileged access.
  • Trireme-lib requires to run in the Host PID namespace.

Analytics

trireme-lib's People

Contributors

0xmchadha avatar abhijitherekar avatar amitlimaye avatar brianonn avatar bvandewalle avatar chris-serafin avatar dcarastan avatar dogild avatar dstiliadis avatar gae123 avatar jodydadescott avatar kevincantu avatar leecalcote avatar mheese avatar philipatl avatar primalmotion avatar ruvnet avatar satyamsi avatar sharifam avatar sibicramesh avatar t00f avatar tillery-aporeto avatar varks avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

trireme-lib's Issues

[REFACTORING] linter is currently deactivated for some unit tests

Right now, the gometalinter is deactivated for some unittests.

The method deactivated are marked with the following comments :

// TODO: remove nolint
// nolint
func (p *Packet) AddTCPLayer(srcPort layers.TCPPort, dstPort layers.TCPPort) error {

Deactivated methods are in the packages : packetgen and cgnetcls.
Reactivated the test will probably needs some refactoring

About using your simple example trireme-example controller to instantiate the problem. . .

Controller instantiation in trireme-example/triremecli/cli.go: line 102
// Initialize the controllers
ctrl := controller.New("ExampleNode", controllerOptions...)

But the new lib library new is not the same as the latest lib library config structure field.

before:
d704b98b591b02097a3ada68d9b64db
now:
02b53b84539257f31eb73f76f724789

Why is this field necessary? I think controller.New() --> newTrireme(c) --> t.newEnforcers() -> enforcer.New() --> nfqdatapath.New(***) --> d.SetTargetNetworks(cfg) --> cfg.TCPTargetNetworks This throws a pannic. Because cfg is nil. This is when the config structure is instantiated, no assignment is made.

unable to compile the code on rhel7/go1.4.2

make build

CGO_ENABLED=1 go build -a -installsuffix cgo
/local/golang/src/github.com/docker/docker/client/hijack.go:15:2: no buildable Go source files in /local/golang/src/github.co
make: *** [build] Error 1

code seems not to compile either on 1.5 + due to circular import issues..

Logs: Received application ACK packet in the wrong state! 2

Looking at the logs, I can find a lot of those:

{"level":"debug","ts":1497986446.561756,"caller":"enforcer/datapath_tcp.go:192","msg":"Dropping packet ","flow":"10.8.1.19:10.8.1.18:80:59642","Flags":".AP...","error":"Received application ACK packet in the wrong state! 2"}

Long lived existing connections are disrupted by deploying Trireme-Lib

Actual behavior

Upon deploying Trireme-lib, it seems that existing connections sometimes fail

Expected behavior

They should be allowed to continue, or be closed depending on A policy

Steps to reproduce

Deploy Trireme-lib for a container that got a long-lived connection already setup.

trireme openshift origin v1.4.0

% oc version
oc v1.4.1+3f9807a
kubernetes v1.4.0+776c994
features: Basic-Auth

openshift v1.4.0-rc1+b4e0954
kubernetes v1.4.0+776c994

Trying to run trireme on openshift using DaemonSets. It looks like to solution we are desperately in need of. Using the DaemonSet PSK, Ive created the trireme secret but get the following error in my pods when they spin up:

% oc logs po/trireme-2dybv
I0203 14:28:32.609910 6 main.go:27] Config used: &{KubeEnv:true AuthType:PSK KubeNodeName:node01 NodeAnnotationKey: PKIDirectory: KubeConfigLocation: TriremePSK:XXXXXXXXXX== TriremeNets:[10.0.0.0/8] ExistingContainerSync:true}
W0203 14:28:32.616409 6 client_config.go:481] Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work.
I0203 14:28:32.617117 6 main.go:48] Starting Trireme PSK
I0203 14:28:32.644971 6 iptables.go:608] Can't clear PREROUTING iptables command
I0203 14:28:32.647501 6 iptables.go:608] Can't clear PREROUTING iptables command
I0203 14:28:32.648981 6 iptables.go:608] Can't clear PREROUTING iptables command
I0203 14:28:32.651453 6 iptables.go:608] Can't clear POSTROUTING iptables command
E0203 14:28:32.651664 6 datapath.go:224] Error unbinding existing NFQ handler from AfInet protocol family: operation not permitted

pod monitor

Hello, I saw your txt file under monitor/internal/kubernetes saying that the kubetnetes monitor has been deprecated, replacing it with the pod monitor, then I tried using the pod monitor and found the error, because The monitor configuration is much less. You didn't provide some default configuration of the pod monitor, but wrote a few interfaces. Is it necessary for us to implement it ourselves? Will you provide it in the future?

深度截图_选择区域_20191125113420
深度截图_选择区域_20191125113820

深度截图_选择区域_20191125113842

orchestration agnostic policy definition

Network policy definition should be orchestration agnostic. Though trireme supports k8s policy definition which makes it simpler to define policies. should be usefull to have simpler independent policy definition to support k8s,swarm etc..

Validate length of the ContextID// PUID in Trireme

As a general rule, the ContextID is coming straight from the Monitor//Extractor.

The Supervisor and Enforcers are crashing out random errors if the Length of the ContextID is "Too Long".

Keeping this as a placeHolder to refactor the way we handle the ContextID.

Failed to set rmem/wmem

Using version f735be9

We are failing to set net.core.rmem_max=63553920 and net.core.wmem_max=63553920 for each PU that is being created.

{"level":"info","ts":1497550983.161677,"caller":"policy/engine.go:149","msg":"Processing unit created","ID":"5942d087291ebb77c5713b7c","nativeID":"","name":"nginx3_a1961413-5e61-4563-8545-d5b809254a20_2_0"}
{"level":"info","ts":1497550983.5033548,"caller":"policy/engine.go:370","msg":"Processing unit started","ID":"5942d087291ebb77c5713b7c","nativeID":"df1ad23aa40d","name":"nginx3_a1961413-5e61-4563-8545-d5b809254a20_2_0"}
[df1ad23aa40d]:{"level":"info","ts":1497550983.6095226,"caller":"remoteenforcer/remoteenforcer_linux.go:181","msg":"Using PKI Compact Secrets"}
[df1ad23aa40d]:{"level":"error","ts":1497550983.6360884,"caller":"enforcer/datapath.go:129","msg":"Failed to set rmem","error":"exit status 255"}
[df1ad23aa40d]:{"level":"error","ts":1497550983.6386921,"caller":"enforcer/datapath.go:134","msg":"Failed to set wmem","error":"exit status 255"}

Happens for every container launched thus far.

remoteEnforcer exec

remoteEnforcerTempBuildPath = "/var/run/aporeto/tmp/bin/"

在linux里,这个目录下执行文件是不允许的???报出权限被拒绝。.
image

encrypt policy rule

When I configure the first policy p1, the action is allow. The second policy p2, the action is allow and encrypted, then the question is, why is there this comment when searching for the policy, don’t overwrite allow? If so, then the policy id of the encrypted policy p2 will not be matched , But the policy id of the first p1
image

ContextStore Upgrade

Actual behavior

Description of the actual behavior.

Expected behavior

Description of the expected behavior.

Steps to reproduce

Description of the various steps required to reproduce the error.

Solution proposal

Description of what you thingk would need to be done.
The context store today will not be able to handle format changes between restarts or upgrade or trireme.
If we add fields to the context store and restart we will not be able to recover PU's.
Need to add this capability to the contextstore

Installation type

  • console.aporeto.com
  • on-prem
  • dev

Version: ?
Customer: ?
ETA: ?

Index out of range on delete namespace in Trireme

Got this error on a remote with Trireme when an ongoing delete was happening for that container:
(Triggered with Trireme-Kubernetes)

[0fd74fad21ab]:panic: runtime error: index out of range
[0fd74fad21ab]:
[0fd74fad21ab]:goroutine 23 [running]:
[0fd74fad21ab]:github.com/aporeto-inc/trireme-kubernetes/vendor/github.com/aporeto-inc/trireme/enforcer.(*Datapath).startApplicationInterceptor.func1(0xc4201b06e0, 0xc420531000, 0x10, 0x10, 0xc420c049ec, 0xc4216a0000)
[0fd74fad21ab]:	/home/travis/gopath/src/github.com/aporeto-inc/trireme-kubernetes/vendor/github.com/aporeto-inc/trireme/enforcer/nfq_linux.go:89 +0xaf
[0fd74fad21ab]:created by github.com/aporeto-inc/trireme-kubernetes/vendor/github.com/aporeto-inc/trireme/enforcer.(*Datapath).startApplicationInterceptor
[0fd74fad21ab]:	/home/travis/gopath/src/github.com/aporeto-inc/trireme-kubernetes/vendor/github.com/aporeto-inc/trireme/enforcer/nfq_linux.go:93 +0x424

Compilation error in trireme-example

I checked out trireme-example on Fedora 28 using git as below -
'git clone https://github.com/aporeto-inc/trireme-example.git'

Then following all the steps mentioned in README.MD - https://github.com/aporeto-inc/trireme-example/blob/master/README.md , I am trying to compile using the specified command ' make build'.

I am getting the following compilation error -

(69/69) Wrote github.com/docker/[email protected]

github.com/aporeto-inc/trireme-example/vendor/go.aporeto.io/netlink-go/nflog

vendor/go.aporeto.io/netlink-go/nflog/nflog.go:287:31: too many arguments in call to packet.New
have (number, []byte, string, bool)
want (uint64, []byte, string)
vendor/go.aporeto.io/netlink-go/nflog/nflog.go:293:37: cannot call non-function ipPacket.SourceAddress (type net.IP)
vendor/go.aporeto.io/netlink-go/nflog/nflog.go:294:42: cannot call non-function ipPacket.DestinationAddress (type net.IP)
vendor/go.aporeto.io/netlink-go/nflog/nflog.go:296:34: cannot call non-function ipPacket.IPProto (type uint8)
vendor/go.aporeto.io/netlink-go/nflog/nflog.go:297:24: ipPacket.IPTotalLen undefined (type *packet.Packet has no field or method IPTotalLen)
vendor/go.aporeto.io/netlink-go/nflog/nflog.go:298:36: cannot call non-function ipPacket.SourcePort (type uint16)
vendor/go.aporeto.io/netlink-go/nflog/nflog.go:299:25: ipPacket.DestPort undefined (type *packet.Packet has no field or method DestPort)
make: *** [Makefile:23: build] Error 2

Please help to resolve this issue, as we are checking the feasibility of integrating trireme solution into security enhancements for cloud applications in our organisation

data race happening randomly

[46c8cb939ace]:time="2017-04-01T00:27:27Z" level=info msg="Supervise status remote_enforcer " error="exit status 2: iptables v1.6.0: host/network `' not found
[46c8cb939ace]:Try `iptables -h' or 'iptables --help' for more information.
[46c8cb939ace]:" method=Supervise package="remote_enforcer" 
[46c8cb939ace]:==================
[46c8cb939ace]:WARNING: DATA RACE
[46c8cb939ace]:Read at 0x00c4202a2910 by goroutine 30:
[46c8cb939ace]:  github.com/aporeto-inc/trireme/enforcer.(*datapathEnforcer).processApplicationPacketsFromNFQ()
[46c8cb939ace]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/trireme/enforcer/datapath.go:442 +0x28a
[46c8cb939ace]:  github.com/aporeto-inc/trireme/enforcer.(*datapathEnforcer).StartApplicationInterceptor.func1()
[46c8cb939ace]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/trireme/enforcer/datapath.go:366 +0xcd
[46c8cb939ace]:
[46c8cb939ace]:Previous write at 0x00c4202a2910 by goroutine 26:
[46c8cb939ace]:  github.com/aporeto-inc/trireme/enforcer.(*datapathEnforcer).processApplicationPacketsFromNFQ()
[46c8cb939ace]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/trireme/enforcer/datapath.go:442 +0x2a4
[46c8cb939ace]:  github.com/aporeto-inc/trireme/enforcer.(*datapathEnforcer).StartApplicationInterceptor.func1()
[46c8cb939ace]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/trireme/enforcer/datapath.go:366 +0xcd
[46c8cb939ace]:
[46c8cb939ace]:Goroutine 30 (running) created at:
[46c8cb939ace]:  github.com/aporeto-inc/trireme/enforcer.(*datapathEnforcer).StartApplicationInterceptor()
[46c8cb939ace]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/trireme/enforcer/datapath.go:369 +0x29c
[46c8cb939ace]:  github.com/aporeto-inc/trireme/enforcer.(*datapathEnforcer).Start()
[46c8cb939ace]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/trireme/enforcer/datapath.go:299 +0x2b2
[46c8cb939ace]:  github.com/aporeto-inc/trireme/cmd/remoteenforcer.(*Server).InitEnforcer()
[46c8cb939ace]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/trireme/cmd/remoteenforcer/remoteenforcer_linux.go:112 +0x5f3
[46c8cb939ace]:  runtime.call128()
[46c8cb939ace]:      /usr/local/go/src/runtime/asm_amd64.s:516 +0x65
[46c8cb939ace]:  reflect.Value.Call()
[46c8cb939ace]:      /usr/local/go/src/reflect/value.go:302 +0xc0
[46c8cb939ace]:  net/rpc.(*service).call()
[46c8cb939ace]:      /usr/local/go/src/net/rpc/server.go:387 +0x257
[46c8cb939ace]:
[46c8cb939ace]:Goroutine 26 (running) created at:
[46c8cb939ace]:  github.com/aporeto-inc/trireme/enforcer.(*datapathEnforcer).StartApplicationInterceptor()
[46c8cb939ace]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/trireme/enforcer/datapath.go:369 +0x29c
[46c8cb939ace]:  github.com/aporeto-inc/trireme/enforcer.(*datapathEnforcer).Start()
[46c8cb939ace]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/trireme/enforcer/datapath.go:299 +0x2b2
[46c8cb939ace]:  github.com/aporeto-inc/trireme/cmd/remoteenforcer.(*Server).InitEnforcer()
[46c8cb939ace]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/trireme/cmd/remoteenforcer/remoteenforcer_linux.go:112 +0x5f3
[46c8cb939ace]:  runtime.call128()
[46c8cb939ace]:      /usr/local/go/src/runtime/asm_amd64.s:516 +0x65
[46c8cb939ace]:  reflect.Value.Call()
[46c8cb939ace]:      /usr/local/go/src/reflect/value.go:302 +0xc0
[46c8cb939ace]:  net/rpc.(*service).call()
[46c8cb939ace]:      /usr/local/go/src/net/rpc/server.go:387 +0x257
[46c8cb939ace]:==================

Question about your go-ipset library reference log library

About your library: go-ipset/ipset/ipset.go go: line 28
You quoted an uppercase log library "github.com/Sirupsen/logrus", but the author later changed the library name to lowercase, which led us to throw some fatal errors all the time!

And your other library: oxy/forward/fwd.go go: line 23 is a lowercase name that references the log library: log "github.com/sirupsen/logrus"

This will lead to conflict. I hope that you can synchronize the two libraries.

pu runtime

Hello, may I ask: how to get the pid field in the puruntime structure returned by the metadata extractor in the pod monitor?

Decoding 9 bytes of bad CBOR data can exhaust memory (BinaryJWTClaims in controller/pkg/tokens/binaryjwt.go)

Summary

Attempting to decode 9-10 bytes of malformed CBOR data (BinaryJWTClaims) causes
"fatal error: out of memory" or "runtime: out of memory" errors.

Only 1 decode attempt is needed to exhaust memory and cause these errors.

cc @sibicramesh @dstiliadis

Relevant code

trireme-lib/controller/pkg/tokens/binaryjwt.go

import (
...
"github.com/ugorji/go/codec"
...
)
...
func decode(buf []byte) (*BinaryJWTClaims, error) {
	// Decode the token into a structure.
	binaryClaims := &BinaryJWTClaims{}
	var h codec.Handle = new(codec.CborHandle)

	dec := codec.NewDecoderBytes(buf, h)
	if err := dec.Decode(binaryClaims); err != nil {

Actual behavior

Using the above decode function to decode 9-10 bytes of malformed CBOR data (BinaryJWTClaims) causes "fatal error: out of memory" or "runtime: out of memory" errors.

The malformed CBOR data simply needs to "lure a decoder into allocating very big data items (strings, arrays, maps) ..." as warned in RFC 7049 Section 8.

Expected behavior

One attempt to decode 9 bytes of malformed CBOR data (BinaryJWTClaims) should not be able to exhaust memory.

Steps to reproduce

Try to decode 9 bytes of malformed CBOR data described in RFC 7049 Section 8 (CBOR Security Considerations).

Examples of malformed CBOR data that can exhaust memory can be found on GitHub since September 2019 (possibly a lot earlier if you look beyond Go projects).

Solution proposal

trireme-lib should switch to a more secure CBOR library, which is what other projects did to fix this.

alt text

For more comparisons with ugorji/go, see fxamacker/cbor.

Background and Other Projects That Fixed This

In October 2013, RFC 7049 Section 8 (CBOR Security Considerations) warned that malformed CBOR data can be used to exhaust system resources.

Resource exhaustion attacks might attempt to lure a decoder into
allocating very big data items (strings, arrays, maps) or exhaust the
stack depth by setting up deeply nested items. Decoders need to have
appropriate resource management to mitigate these attacks.

In Sept 2019, oasislabs/oasis-core discovered tiny malformed CBOR data can exhaust memory and they traced the problem to the same CBOR library being used by trireme-lib (ugorji/go). They fixed it by switching to a more secure CBOR library.

In Feb 2020, smartcontractkit/chainlink had a CBOR security issue which was fixed by
a GitHub PR titled "Switch to more secure CBOR library". They were also using the same CBOR library as trireme-lib.

For info about CBOR and security, see Section 8 of RFC 7049 (Security Considerations).

HandlePUEvent for the Die Event does not provide the actual PURuntime

When receiving a HandlePuEvent from the Docker Monitor, the PURuntime provided is a dummy deafult one as can be seen in this code from the docker monitor.

//handleDie event is called when a container dies. It generates a "Stop" event.
func (d *DockerMonitor) handleDieEvent(ctx context.Context, event *events.Message) error {

	puID, err := puIDFromDockerID(event.ID)
	if err != nil {
		return err
	}

	return d.config.Policy.HandlePUEvent(ctx, puID, tevents.EventStop, policy.NewPURuntimeWithDefaults())
}

As such, it makes it more difficult to extract data from the resolver without keeping a cache.

protocol == 6

	for _, proto := range rule.Protocols {
		// TODO: 有疑惑。设置受trireme保护的网络作用域会调用此方法,而且协议等于6,但是生成puContext也会调用此方法,协议不可能等于6。这个单一判断无法同时满足。。
		//if strings.ToLower(proto) != constants.TCPProtoNum {
		//	continue
		//}
		for _, address := range rule.Addresses {
			for _, port := range rule.Ports {
				if err := addCache(address, port); err != nil {
					return err
				}
			}
		}
	}

data races happening 100% of time when stopping Trireme

c8cb939ace]:==================
[46c8cb939ace]:WARNING: DATA RACE
[a274b1e1e157]:  github.com/aporeto-inc/trireme/supervisor.NewSupervisor()
[a274b1e1e157]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/trireme/supervisor/supervisor.go:77 +0x7a6
[46c8cb939ace]:Read at 0x00c4203abf92 by main goroutine:
[a274b1e1e157]:  github.com/aporeto-inc/trireme/cmd/remoteenforcer.(*Server).InitSupervisor()
[a274b1e1e157]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/trireme/cmd/remoteenforcer/remoteenforcer_linux.go:142 +0x15e
[46c8cb939ace]:  github.com/coreos/go-iptables/iptables.(*IPTables).runWithOutput()
[46c8cb939ace]:      /home/ubuntu/workspace/code/go/src/github.com/coreos/go-iptables/iptables/iptables.go:228 +0x16f
[46c8cb939ace]:  github.com/coreos/go-iptables/iptables.(*IPTables).run()
[46c8cb939ace]:      /home/ubuntu/workspace/code/go/src/github.com/coreos/go-iptables/iptables/iptables.go:221 +0x6c
[46c8cb939ace]:  github.com/coreos/go-iptables/iptables.(*IPTables).NewChain()
[46c8cb939ace]:      /home/ubuntu/workspace/code/go/src/github.com/coreos/go-iptables/iptables/iptables.go:187 +0x129
[46c8cb939ace]:  github.com/coreos/go-iptables/iptables.(*IPTables).ClearChain()
[46c8cb939ace]:      /home/ubuntu/workspace/code/go/src/github.com/coreos/go-iptables/iptables/iptables.go:193 +0x84
[a274b1e1e157]:  runtime.call128()
[46c8cb939ace]:  github.com/aporeto-inc/trireme/supervisor/iptablesctrl.(*Instance).cleanACLSection()
[46c8cb939ace]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/trireme/supervisor/iptablesctrl/acls.go:751 +0xb0
[a274b1e1e157]:      /usr/local/go/src/runtime/asm_amd64.s:516 +0x65
[46c8cb939ace]:  github.com/aporeto-inc/trireme/supervisor/iptablesctrl.(*Instance).cleanACLs()
[a274b1e1e157]:  reflect.Value.Call()
[46c8cb939ace]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/trireme/supervisor/iptablesctrl/acls.go:741 +0xf9
[46c8cb939ace]:  github.com/aporeto-inc/trireme/supervisor/iptablesctrl.(*Instance).Stop()
[46c8cb939ace]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/trireme/supervisor/iptablesctrl/iptables.go:299 +0x201
[46c8cb939ace]:  github.com/aporeto-inc/trireme/supervisor.(*Config).Stop()
[46c8cb939ace]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/trireme/supervisor/supervisor.go:142 +0x53
[46c8cb939ace]:  github.com/aporeto-inc/trireme/cmd/remoteenforcer.(*Server).EnforcerExit()
[46c8cb939ace]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/trireme/cmd/remoteenforcer/remoteenforcer_linux.go:280 +0x60
[46c8cb939ace]:  main.EnforceOnly()
[46c8cb939ace]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/enforcerd/main.go:543 +0x7a4
[46c8cb939ace]:  main.main()
[46c8cb939ace]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/enforcerd/main.go:637 +0xb7
[46c8cb939ace]:
[46c8cb939ace]:Previous write at 0x00c4203abf90 by goroutine 42:
[46c8cb939ace]:  github.com/coreos/go-iptables/iptables.NewWithProtocol()
[46c8cb939ace]:      /home/ubuntu/workspace/code/go/src/github.com/coreos/go-iptables/iptables/iptables.go:74 +0x1f1
[46c8cb939ace]:  github.com/coreos/go-iptables/iptables.New()
[46c8cb939ace]:      /home/ubuntu/workspace/code/go/src/github.com/coreos/go-iptables/iptables/iptables.go:60 +0x33
[46c8cb939ace]:  github.com/aporeto-inc/trireme/supervisor/provider.NewGoIPTablesProvider()
[a274b1e1e157]:      /usr/local/go/src/reflect/value.go:302 +0xc0
[46c8cb939ace]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/trireme/supervisor/provider/iptablesprovider.go:20 +0x2f
[46c8cb939ace]:  github.com/aporeto-inc/trireme/supervisor/iptablesctrl.NewInstance()
[46c8cb939ace]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/trireme/supervisor/iptablesctrl/iptables.go:40 +0x33
[46c8cb939ace]:  github.com/aporeto-inc/trireme/supervisor.NewSupervisor()
[46c8cb939ace]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/trireme/supervisor/supervisor.go:77 +0x7a6
[a274b1e1e157]:  net/rpc.(*service).call()
[46c8cb939ace]:  github.com/aporeto-inc/trireme/cmd/remoteenforcer.(*Server).InitSupervisor()
[46c8cb939ace]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/trireme/cmd/remoteenforcer/remoteenforcer_linux.go:142 +0x15e
[46c8cb939ace]:  runtime.call128()
[46c8cb939ace]:      /usr/local/go/src/runtime/asm_amd64.s:516 +0x65
[a274b1e1e157]:      /usr/local/go/src/net/rpc/server.go:387 +0x257
[a274b1e1e157]:
[a274b1e1e157]:Goroutine 42 (finished) created at:
[a274b1e1e157]:  net/rpc.(*Server).ServeCodec()
[a274b1e1e157]:      /usr/local/go/src/net/rpc/server.go:481 +0x496
[a274b1e1e157]:  net/rpc.(*Server).ServeConn()
[46c8cb939ace]:  reflect.Value.Call()
[a274b1e1e157]:      /usr/local/go/src/net/rpc/server.go:458 +0x62b
[a274b1e1e157]:  net/rpc.(*Server).ServeHTTP()
[46c8cb939ace]:      /usr/local/go/src/reflect/value.go:302 +0xc0
[46c8cb939ace]:  net/rpc.(*service).call()
[46c8cb939ace]:      /usr/local/go/src/net/rpc/server.go:387 +0x257
[46c8cb939ace]:
[46c8cb939ace]:Goroutine 42 (finished) created at:
[46c8cb939ace]:  net/rpc.(*Server).ServeCodec()
[46c8cb939ace]:      /usr/local/go/src/net/rpc/server.go:481 +0x496
[46c8cb939ace]:  net/rpc.(*Server).ServeConn()
[46c8cb939ace]:      /usr/local/go/src/net/rpc/server.go:458 +0x62b
[46c8cb939ace]:  net/rpc.(*Server).ServeHTTP()
[46c8cb939ace]:      /usr/local/go/src/net/rpc/server.go:700 +0x506
[46c8cb939ace]:  net/http.(*ServeMux).ServeHTTP()
[46c8cb939ace]:      /usr/local/go/src/net/http/server.go:2238 +0xa2
[46c8cb939ace]:  net/http.serverHandler.ServeHTTP()
[46c8cb939ace]:      /usr/local/go/src/net/http/server.go:2568 +0xbc
[46c8cb939ace]:  net/http.(*conn).serve()
[46c8cb939ace]:      /usr/local/go/src/net/http/server.go:1825 +0x71a
[46c8cb939ace]:==================
[a274b1e1e157]:      /usr/local/go/src/net/rpc/server.go:700 +0x506
[a274b1e1e157]:  net/http.(*ServeMux).ServeHTTP()
[a274b1e1e157]:      /usr/local/go/src/net/http/server.go:2238 +0xa2
[a274b1e1e157]:  net/http.serverHandler.ServeHTTP()
[a274b1e1e157]:      /usr/local/go/src/net/http/server.go:2568 +0xbc
[a274b1e1e157]:  net/http.(*conn).serve()
[a274b1e1e157]:      /usr/local/go/src/net/http/server.go:1825 +0x71a
[a274b1e1e157]:==================
[ca41dd3ab346]:==================
[ca41dd3ab346]:WARNING: DATA RACE
[ca41dd3ab346]:Read at 0x00c4202787d2 by main goroutine:
[ca41dd3ab346]:  github.com/coreos/go-iptables/iptables.(*IPTables).runWithOutput()
[ca41dd3ab346]:      /home/ubuntu/workspace/code/go/src/github.com/coreos/go-iptables/iptables/iptables.go:228 +0x16f
[ca41dd3ab346]:  github.com/coreos/go-iptables/iptables.(*IPTables).run()
[ca41dd3ab346]:      /home/ubuntu/workspace/code/go/src/github.com/coreos/go-iptables/iptables/iptables.go:221 +0x6c
[ca41dd3ab346]:  github.com/coreos/go-iptables/iptables.(*IPTables).NewChain()
[ca41dd3ab346]:      /home/ubuntu/workspace/code/go/src/github.com/coreos/go-iptables/iptables/iptables.go:187 +0x129
[ca41dd3ab346]:  github.com/coreos/go-iptables/iptables.(*IPTables).ClearChain()
[ca41dd3ab346]:      /home/ubuntu/workspace/code/go/src/github.com/coreos/go-iptables/iptables/iptables.go:193 +0x84
[ca41dd3ab346]:  github.com/aporeto-inc/trireme/supervisor/iptablesctrl.(*Instance).cleanACLSection()
[ca41dd3ab346]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/trireme/supervisor/iptablesctrl/acls.go:751 +0xb0
[ca41dd3ab346]:  github.com/aporeto-inc/trireme/supervisor/iptablesctrl.(*Instance).cleanACLs()
[ca41dd3ab346]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/trireme/supervisor/iptablesctrl/acls.go:741 +0xf9
[ca41dd3ab346]:  github.com/aporeto-inc/trireme/supervisor/iptablesctrl.(*Instance).Stop()
[ca41dd3ab346]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/trireme/supervisor/iptablesctrl/iptables.go:299 +0x201
[ca41dd3ab346]:  github.com/aporeto-inc/trireme/supervisor.(*Config).Stop()
[ca41dd3ab346]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/trireme/supervisor/supervisor.go:142 +0x53
[ca41dd3ab346]:  github.com/aporeto-inc/trireme/cmd/remoteenforcer.(*Server).EnforcerExit()
[ca41dd3ab346]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/trireme/cmd/remoteenforcer/remoteenforcer_linux.go:280 +0x60
[ca41dd3ab346]:  main.EnforceOnly()
[ca41dd3ab346]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/enforcerd/main.go:543 +0x7a4
[ca41dd3ab346]:  main.main()
[ca41dd3ab346]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/enforcerd/main.go:637 +0xb7
[ca41dd3ab346]:
[ca41dd3ab346]:Previous write at 0x00c4202787d0 by goroutine 42:
[ca41dd3ab346]:  github.com/coreos/go-iptables/iptables.NewWithProtocol()
[ca41dd3ab346]:      /home/ubuntu/workspace/code/go/src/github.com/coreos/go-iptables/iptables/iptables.go:74 +0x1f1
[ca41dd3ab346]:  github.com/coreos/go-iptables/iptables.New()
[ca41dd3ab346]:      /home/ubuntu/workspace/code/go/src/github.com/coreos/go-iptables/iptables/iptables.go:60 +0x33
[ca41dd3ab346]:  github.com/aporeto-inc/trireme/supervisor/provider.NewGoIPTablesProvider()
[ca41dd3ab346]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/trireme/supervisor/provider/iptablesprovider.go:20 +0x2f
[ca41dd3ab346]:  github.com/aporeto-inc/trireme/supervisor/iptablesctrl.NewInstance()
[ca41dd3ab346]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/trireme/supervisor/iptablesctrl/iptables.go:40 +0x33
[ca41dd3ab346]:  github.com/aporeto-inc/trireme/supervisor.NewSupervisor()
[ca41dd3ab346]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/trireme/supervisor/supervisor.go:77 +0x7a6
[ca41dd3ab346]:  github.com/aporeto-inc/trireme/cmd/remoteenforcer.(*Server).InitSupervisor()
[ca41dd3ab346]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/trireme/cmd/remoteenforcer/remoteenforcer_linux.go:142 +0x15e
[ca41dd3ab346]:  runtime.call128()
[ca41dd3ab346]:      /usr/local/go/src/runtime/asm_amd64.s:516 +0x65
[ca41dd3ab346]:  reflect.Value.Call()
[ca41dd3ab346]:      /usr/local/go/src/reflect/value.go:302 +0xc0
[ca41dd3ab346]:  net/rpc.(*service).call()
[ca41dd3ab346]:      /usr/local/go/src/net/rpc/server.go:387 +0x257
[ca41dd3ab346]:
[ca41dd3ab346]:Goroutine 42 (finished) created at:
[ca41dd3ab346]:  net/rpc.(*Server).ServeCodec()
[ca41dd3ab346]:      /usr/local/go/src/net/rpc/server.go:481 +0x496
[ca41dd3ab346]:  net/rpc.(*Server).ServeConn()
[ca41dd3ab346]:      /usr/local/go/src/net/rpc/server.go:458 +0x62b
[ca41dd3ab346]:  net/rpc.(*Server).ServeHTTP()
[ca41dd3ab346]:      /usr/local/go/src/net/rpc/server.go:700 +0x506
[ca41dd3ab346]:  net/http.(*ServeMux).ServeHTTP()
[ca41dd3ab346]:      /usr/local/go/src/net/http/server.go:2238 +0xa2
[ca41dd3ab346]:  net/http.serverHandler.ServeHTTP()
[ca41dd3ab346]:      /usr/local/go/src/net/http/server.go:2568 +0xbc
[ca41dd3ab346]:  net/http.(*conn).serve()
[ca41dd3ab346]:      /usr/local/go/src/net/http/server.go:1825 +0x71a
[ca41dd3ab346]:==================
[a274b1e1e157]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/trireme/supervisor/iptablesctrl/iptables.go:40 +0x33
[46c8cb939ace]:  net/rpc.(*Server).ServeHTTP()
[46c8cb939ace]:      /usr/local/go/src/net/rpc/server.go:700 +0x506
[46c8cb939ace]:  net/http.(*ServeMux).ServeHTTP()
[46c8cb939ace]:      /usr/local/go/src/net/http/server.go:2238 +0xa2
[46c8cb939ace]:  net/http.serverHandler.ServeHTTP()
[46c8cb939ace]:      /usr/local/go/src/net/http/server.go:2568 +0xbc
[46c8cb939ace]:  net/http.(*conn).serve()
[46c8cb939ace]:      /usr/local/go/src/net/http/server.go:1825 +0x71a
[46c8cb939ace]:==================
[84cdf04e2c04]:==================
[84cdf04e2c04]:WARNING: DATA RACE
[84cdf04e2c04]:Read at 0x00c4203036f2 by main goroutine:
[a274b1e1e157]:      /home/ubuntu/workspace/code/go/src/github.com/coreos/go-iptables/iptables/iptables.go:193 +0x84
[84cdf04e2c04]:  github.com/coreos/go-iptables/iptables.(*IPTables).runWithOutput()
[a274b1e1e157]:  github.com/aporeto-inc/trireme/supervisor/iptablesctrl.(*Instance).cleanACLSection()
[84cdf04e2c04]:      /home/ubuntu/workspace/code/go/src/github.com/coreos/go-iptables/iptables/iptables.go:228 +0x16f
[a274b1e1e157]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/trireme/supervisor/iptablesctrl/acls.go:751 +0xb0
[84cdf04e2c04]:  github.com/coreos/go-iptables/iptables.(*IPTables).run()
[a274b1e1e157]:  github.com/aporeto-inc/trireme/supervisor/iptablesctrl.(*Instance).cleanACLs()
[84cdf04e2c04]:      /home/ubuntu/workspace/code/go/src/github.com/coreos/go-iptables/iptables/iptables.go:221 +0x6c
[a274b1e1e157]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/trireme/supervisor/iptablesctrl/acls.go:741 +0xf9
[84cdf04e2c04]:  github.com/coreos/go-iptables/iptables.(*IPTables).NewChain()
[a274b1e1e157]:  github.com/aporeto-inc/trireme/supervisor/iptablesctrl.(*Instance).Stop()
[84cdf04e2c04]:      /home/ubuntu/workspace/code/go/src/github.com/coreos/go-iptables/iptables/iptables.go:187 +0x129
[a274b1e1e157]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/trireme/supervisor/iptablesctrl/iptables.go:299 +0x201
[84cdf04e2c04]:  github.com/coreos/go-iptables/iptables.(*IPTables).ClearChain()
[84cdf04e2c04]:      /home/ubuntu/workspace/code/go/src/github.com/coreos/go-iptables/iptables/iptables.go:193 +0x84
[a274b1e1e157]:  github.com/aporeto-inc/trireme/supervisor.(*Config).Stop()
[a274b1e1e157]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/trireme/supervisor/supervisor.go:142 +0x53
[84cdf04e2c04]:  github.com/aporeto-inc/trireme/supervisor/iptablesctrl.(*Instance).cleanACLSection()
[a274b1e1e157]:  github.com/aporeto-inc/trireme/cmd/remoteenforcer.(*Server).EnforcerExit()
[84cdf04e2c04]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/trireme/supervisor/iptablesctrl/acls.go:751 +0xb0
[e3d08ff45647]:      /home/ubuntu/workspace/code/go/src/github.com/coreos/go-iptables/iptables/iptables.go:221 +0x6c
[a274b1e1e157]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/trireme/cmd/remoteenforcer/remoteenforcer_linux.go:280 +0x60
[84cdf04e2c04]:  github.com/aporeto-inc/trireme/supervisor/iptablesctrl.(*Instance).cleanACLs()
[84cdf04e2c04]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/trireme/supervisor/iptablesctrl/acls.go:741 +0xf9
[e3d08ff45647]:  github.com/coreos/go-iptables/iptables.(*IPTables).NewChain()
[84cdf04e2c04]:  github.com/aporeto-inc/trireme/supervisor/iptablesctrl.(*Instance).Stop()
[a274b1e1e157]:  main.EnforceOnly()
[e3d08ff45647]:      /home/ubuntu/workspace/code/go/src/github.com/coreos/go-iptables/iptables/iptables.go:187 +0x129
[a274b1e1e157]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/enforcerd/main.go:543 +0x7a4
[84cdf04e2c04]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/trireme/supervisor/iptablesctrl/iptables.go:299 +0x201
[e3d08ff45647]:  github.com/coreos/go-iptables/iptables.(*IPTables).ClearChain()
[a274b1e1e157]:  main.main()
[84cdf04e2c04]:  github.com/aporeto-inc/trireme/supervisor.(*Config).Stop()
[a274b1e1e157]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/enforcerd/main.go:637 +0xb7
[84cdf04e2c04]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/trireme/supervisor/supervisor.go:142 +0x53
[a274b1e1e157]:
[e3d08ff45647]:      /home/ubuntu/workspace/code/go/src/github.com/coreos/go-iptables/iptables/iptables.go:193 +0x84
[84cdf04e2c04]:  github.com/aporeto-inc/trireme/cmd/remoteenforcer.(*Server).EnforcerExit()
[84cdf04e2c04]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/trireme/cmd/remoteenforcer/remoteenforcer_linux.go:280 +0x60
[84cdf04e2c04]:  main.EnforceOnly()
[00d98ff6e87e]:Previous write at 0x00c42029bff0 by goroutine 42:
[e3d08ff45647]:  github.com/aporeto-inc/trireme/supervisor/iptablesctrl.(*Instance).cleanACLSection()
[84cdf04e2c04]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/enforcerd/main.go:543 +0x7a4
[84cdf04e2c04]:  main.main()
[84cdf04e2c04]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/enforcerd/main.go:637 +0xb7
[00d98ff6e87e]:  github.com/coreos/go-iptables/iptables.NewWithProtocol()
[84cdf04e2c04]:
[84cdf04e2c04]:Previous write at 0x00c4203036f0 by goroutine 42:
[00d98ff6e87e]:      /home/ubuntu/workspace/code/go/src/github.com/coreos/go-iptables/iptables/iptables.go:74 +0x1f1
[84cdf04e2c04]:  github.com/coreos/go-iptables/iptables.NewWithProtocol()
[84cdf04e2c04]:      /home/ubuntu/workspace/code/go/src/github.com/coreos/go-iptables/iptables/iptables.go:74 +0x1f1
[84cdf04e2c04]:  github.com/coreos/go-iptables/iptables.New()
[00d98ff6e87e]:  github.com/coreos/go-iptables/iptables.New()
[84cdf04e2c04]:      /home/ubuntu/workspace/code/go/src/github.com/coreos/go-iptables/iptables/iptables.go:60 +0x33
[84cdf04e2c04]:  github.com/aporeto-inc/trireme/supervisor/provider.NewGoIPTablesProvider()
[00d98ff6e87e]:      /home/ubuntu/workspace/code/go/src/github.com/coreos/go-iptables/iptables/iptables.go:60 +0x33
[84cdf04e2c04]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/trireme/supervisor/provider/iptablesprovider.go:20 +0x2f
[00d98ff6e87e]:  github.com/aporeto-inc/trireme/supervisor/provider.NewGoIPTablesProvider()
[84cdf04e2c04]:  github.com/aporeto-inc/trireme/supervisor/iptablesctrl.NewInstance()
[00d98ff6e87e]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/trireme/supervisor/provider/iptablesprovider.go:20 +0x2f
[00d98ff6e87e]:  github.com/aporeto-inc/trireme/supervisor/iptablesctrl.NewInstance()
[84cdf04e2c04]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/trireme/supervisor/iptablesctrl/iptables.go:40 +0x33
[e3d08ff45647]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/trireme/supervisor/iptablesctrl/acls.go:751 +0xb0
[00d98ff6e87e]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/trireme/supervisor/iptablesctrl/iptables.go:40 +0x33
[84cdf04e2c04]:  github.com/aporeto-inc/trireme/supervisor.NewSupervisor()
[00d98ff6e87e]:  github.com/aporeto-inc/trireme/supervisor.NewSupervisor()
[00d98ff6e87e]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/trireme/supervisor/supervisor.go:77 +0x7a6
[e3d08ff45647]:  github.com/aporeto-inc/trireme/supervisor/iptablesctrl.(*Instance).cleanACLs()
[00d98ff6e87e]:  github.com/aporeto-inc/trireme/cmd/remoteenforcer.(*Server).InitSupervisor()
[e3d08ff45647]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/trireme/supervisor/iptablesctrl/acls.go:741 +0xf9
[84cdf04e2c04]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/trireme/supervisor/supervisor.go:77 +0x7a6
[84cdf04e2c04]:  github.com/aporeto-inc/trireme/cmd/remoteenforcer.(*Server).InitSupervisor()
[84cdf04e2c04]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/trireme/cmd/remoteenforcer/remoteenforcer_linux.go:142 +0x15e
[00d98ff6e87e]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/trireme/cmd/remoteenforcer/remoteenforcer_linux.go:142 +0x15e
[84cdf04e2c04]:  runtime.call128()
[84cdf04e2c04]:      /usr/local/go/src/runtime/asm_amd64.s:516 +0x65
[00d98ff6e87e]:  runtime.call128()
[00d98ff6e87e]:      /usr/local/go/src/runtime/asm_amd64.s:516 +0x65
[84cdf04e2c04]:  reflect.Value.Call()
[00d98ff6e87e]:  reflect.Value.Call()
[84cdf04e2c04]:      /usr/local/go/src/reflect/value.go:302 +0xc0
[84cdf04e2c04]:  net/rpc.(*service).call()
[84cdf04e2c04]:      /usr/local/go/src/net/rpc/server.go:387 +0x257
[84cdf04e2c04]:
[84cdf04e2c04]:Goroutine 42 (finished) created at:
[00d98ff6e87e]:      /usr/local/go/src/reflect/value.go:302 +0xc0
[00d98ff6e87e]:  net/rpc.(*service).call()
[84cdf04e2c04]:  net/rpc.(*Server).ServeCodec()
[84cdf04e2c04]:      /usr/local/go/src/net/rpc/server.go:481 +0x496
[84cdf04e2c04]:  net/rpc.(*Server).ServeConn()
[84cdf04e2c04]:      /usr/local/go/src/net/rpc/server.go:458 +0x62b
[84cdf04e2c04]:  net/rpc.(*Server).ServeHTTP()
[84cdf04e2c04]:      /usr/local/go/src/net/rpc/server.go:700 +0x506
[84cdf04e2c04]:  net/http.(*ServeMux).ServeHTTP()
[84cdf04e2c04]:      /usr/local/go/src/net/http/server.go:2238 +0xa2
[84cdf04e2c04]:  net/http.serverHandler.ServeHTTP()
[84cdf04e2c04]:      /usr/local/go/src/net/http/server.go:2568 +0xbc
[84cdf04e2c04]:  net/http.(*conn).serve()
[e3d08ff45647]:  github.com/aporeto-inc/trireme/supervisor/iptablesctrl.(*Instance).Stop()
[e3d08ff45647]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/trireme/supervisor/iptablesctrl/iptables.go:299 +0x201
[e3d08ff45647]:  github.com/aporeto-inc/trireme/supervisor.(*Config).Stop()
[84cdf04e2c04]:      /usr/local/go/src/net/http/server.go:1825 +0x71a
[e3d08ff45647]:      /home/ubuntu/workspace/code/go/src/github.com/aporeto-inc/trireme/supervisor/supervisor.go:142 +0x53
[84cdf04e2c04]:==================

BUG: Need to retry RPC call when resource temporarily unavailable

During a soaking run, I ran into the following error:

ERROR	Failed to execute command run: Cannot connect to policy process dial unix /var/run/trireme.sock: connect: resource temporarily unavailable	{"error": "Cannot connect to policy process dial unix /var/run/trireme.sock: connect: resource temporarily unavailable"}

Talking with @amitlimaye, the root cause was narrowed down to the following line in cmd/systemdutil/systemdutil.go:

client, err := net.Dial("unix", rpcmonitor.DefaultRPCAddress)

We need to retry here if the error is due to resource temporarily unavailable, what ever error code that resolves to.

package import

go.aporeto.io/trireme-lib/controller/internal/

  • The following packages cannot be imported. Prompt not allowed

Metadata extractor for SELinux Labels

Requested behavior

The Trireme example can associate with the Trireme identity the SELInux labels of the
process that is being executed. This will help us to define SELinux based network
policies that span hosts, similar to what SELinux was trying to achieve with much
more complex implementations.

Expected behavior

When a process is instantiated through Trireme, the SELinux labels associated with
the process are captured and become part of the identity.

I can then use these SELinux labels to define a policy. For example if a file
has the labels: system_u:object_r:user_a these labels can become part of an authorization
policy.

Add a new monitor for Kubernetes.

Add a new monitor for Kubernetes, to be used directly by projects plugging into Kubernetes such as Trireme-Kubernetes and the Enforcer launched on Kubernetes

Kubernetes: unable to find pu id associated given context id

Actual behavior

Description of the actual behavior.
Deployed Trireme using the "Getting Started" guide in: https://github.com/aporeto-inc/trireme-kubernetes and seeing a ton of errors"

[4388f9e84a6a]:ERROR	destNFLogsHandler: create flow record	{"error": "nflog: unable to find pu id associated given context id: 4388f9e84a6a"}

Expected behavior

Description of the expected behavior.

Steps to reproduce

Description of the various steps required to reproduce the error.
Deploy using https://github.com/aporeto-inc/trireme-kubernetes

Solution proposal

Description of what you thingk would need to be done.

Installation type

  • console.aporeto.com
  • on-prem
  • dev

Version: ?
Customer: ?
ETA: ?

metadataExtractor for each Monitor type should be packaged with the monitor type

Having all the extractors on a completely separate package is weird to me and doesn't seen particulartly a good pattern as it distribute almost redundant code accross the monitor and extractor inplementation all over the place (such as helper functions).

I would suggest to move the extractors for each type in the same package as the monitor implementation itself

trirme-example not working as documented with container labels

Am running trireme-example as process on rhel7 box running docker 1.12-cs3 and the step i took are as below.

  1. start docker engine.

  2. run trireme-example binary as process(_not as container) on the host as below
    /local/trireme/trireme-example -log_dir /var/log/trireme -v=10

  3. run a python simhttpserver as a container with label=web
    docker run -l app=web -d -t qa.registry.docker.xxxx.xx.com/cpe/gs-linux6 python -m SimpleHTTPServer 80

  4. run another container with different label as documented
    docker run -i -l app=database -t qa.registry.docker.xxxx.xx.com/cpe/xx-linux6 /bin/bash

  5. curl is able to access it. log to follow

BUG: Enforcing an enforcer that has already exited

While testing trireme, the test was removing PUs when it encountered the following:

{"level":"info","ts":1501555386.8419542,"caller":"policy/engine.go:507","msg":"Processing unit removed","ID":"597fe629df7696000117f505","nativeID":"/32344","name":"python-server-0c12212a-ac9c-457c-ae09-a3b2e1c5e09c-0-2"}
{"level":"info","ts":1501555386.8789887,"caller":"policy/engine.go:471","msg":"Processing unit stopped","ID":"597fe608df7696000117f43d","nativeID":"/31074","name":"python-server-0c12212a-ac9c-457c-ae09-a3b2e1c5e09c-0-0"}
[1a2a4263fd4b]:{"level":"fatal","ts":1501555386.9435802,"caller":"remoteenforcer/remoteenforcer_linux.go:377","msg":"Enforcer not inited"}
{"level":"error","ts":1501555386.9456024,"caller":"proxy/enforcerproxy.go:136","msg":"Failed to Enforce remote enforcer","error":"unexpected EOF"}
{"level":"warn","ts":1501555386.945639,"caller":"trireme/trireme.go:303","msg":"Re-initializing enforcers - connection lost"}
{"level":"error","ts":1501555386.9457128,"caller":"processmon/processmon.go:202","msg":"Cannot determine namespace of new container","error":"stat /proc/31846/ns/net: no such file or directory"}
panic: interface conversion: interface {} is nil, not *rpcwrapper.RPCHdl

goroutine 135 [running]:
github.com/aporeto-inc/enforcerd/vendor/github.com/aporeto-inc/trireme/enforcer/utils/rpcwrapper.(*RPCWrapper).DestroyRPCClient(0xc4201567e0, 0xc4212b8cc0, 0xc)
	/tmp/build/80754af9/go/src/github.com/aporeto-inc/enforcerd/vendor/github.com/aporeto-inc/trireme/enforcer/utils/rpcwrapper/rpc_handle.go:188 +0xac3
github.com/aporeto-inc/enforcerd/vendor/github.com/aporeto-inc/trireme/processmon.(*ProcessMon).KillProcess(0xc42000e070, 0xc4212b8cc0, 0xc)
	/tmp/build/80754af9/go/src/github.com/aporeto-inc/enforcerd/vendor/github.com/aporeto-inc/trireme/processmon/processmon.go:171 +0x2fd
github.com/aporeto-inc/enforcerd/vendor/github.com/aporeto-inc/trireme/supervisor/proxy.(*ProxyInfo).Unsupervise(0xc42019cc40, 0xc4212b8cc0, 0xc, 0xc42040d228, 0xc420261fc0)
	/tmp/build/80754af9/go/src/github.com/aporeto-inc/enforcerd/vendor/github.com/aporeto-inc/trireme/supervisor/proxy/supervisorproxy.go:81 +0xb6
github.com/aporeto-inc/enforcerd/vendor/github.com/aporeto-inc/trireme.(*trireme).doHandleDelete(0xc4201a3680, 0xc4212b8cc0, 0xc, 0xb43711, 0x4)
	/tmp/build/80754af9/go/src/github.com/aporeto-inc/enforcerd/vendor/github.com/aporeto-inc/trireme/trireme.go:255 +0x3ce
github.com/aporeto-inc/enforcerd/vendor/github.com/aporeto-inc/trireme.(*trireme).HandlePUEvent(0xc4201a3680, 0xc4212b8cc0, 0xc, 0xb43711, 0x4, 0x0, 0xb55b8f)
	/tmp/build/80754af9/go/src/github.com/aporeto-inc/enforcerd/vendor/github.com/aporeto-inc/trireme/trireme.go:94 +0x146
github.com/aporeto-inc/enforcerd/vendor/github.com/aporeto-inc/trireme/monitor/dockermonitor.(*dockerMonitor).stopDockerContainer(0xc42028d720, 0xc4212b8cc0, 0x40, 0x3, 0x1)
	/tmp/build/80754af9/go/src/github.com/aporeto-inc/enforcerd/vendor/github.com/aporeto-inc/trireme/monitor/dockermonitor/docker.go:514 +0x105
github.com/aporeto-inc/enforcerd/vendor/github.com/aporeto-inc/trireme/monitor/dockermonitor.(*dockerMonitor).handleDieEvent(0xc42028d720, 0xc421b04380, 0x8eb8f4, 0xc420339450)
	/tmp/build/80754af9/go/src/github.com/aporeto-inc/enforcerd/vendor/github.com/aporeto-inc/trireme/monitor/dockermonitor/docker.go:582 +0x42
github.com/aporeto-inc/enforcerd/vendor/github.com/aporeto-inc/trireme/monitor/dockermonitor.(*dockerMonitor).(github.com/aporeto-inc/enforcerd/vendor/github.com/aporeto-inc/trireme/monitor/dockermonitor.handleDieEvent)-fm(0xc421b04380, 0xc42060e750, 0xc421a7cdbc)
	/tmp/build/80754af9/go/src/github.com/aporeto-inc/enforcerd/vendor/github.com/aporeto-inc/trireme/monitor/dockermonitor/docker.go:232 +0x34
github.com/aporeto-inc/enforcerd/vendor/github.com/aporeto-inc/trireme/monitor/dockermonitor.(*dockerMonitor).eventProcessors.func1(0xc42028d720, 0x4)
	/tmp/build/80754af9/go/src/github.com/aporeto-inc/enforcerd/vendor/github.com/aporeto-inc/trireme/monitor/dockermonitor/docker.go:319 +0x15b
created by github.com/aporeto-inc/enforcerd/vendor/github.com/aporeto-inc/trireme/monitor/dockermonitor.(*dockerMonitor).eventProcessors
	/tmp/build/80754af9/go/src/github.com/aporeto-inc/enforcerd/vendor/github.com/aporeto-inc/trireme/monitor/dockermonitor/docker.go:334 +0x54

I see two issues here:

  1. While removing PUs we hit a fatal due to s.Enforcer == nil in Server.Enforce, most probably due to us calling Server.EnforcerExit in Server.KillProcess. I am not entirely sure why these two ran into one another.
  2. The panic was due to rpcHdl being set to nil when trying to convert it at this line:
if err := rpcHdl.(*RPCHdl).Client.Close(); err != nil {

We do not check if any error comes back from rpcHdl, _ := r.rpcClientMap.Get(contextID). Will make a PR for this.

Is this project supported for use on Openshift Origin?

This project has show the most potential for network isolation for microservices we are deploying to Openshift. It was featured on the openshift blog, however it was demo'd under kubernetes. We are trying to deploy this to the latest release of Openshift Origin, currently v.1.5.0-alpha. We were able to create the PSK secret, and the DaemonSet deploys, but the pods never fully spin up.

oc v1.4.1+3f9807a
kubernetes v1.4.0+776c994
features: Basic-Auth

openshift v1.5.0-alpha.2+e4b43ee
kubernetes v1.5.2+43a9be4

--

Failed validation Error validating pod trireme-5vfgm.demo from api, ignoring: spec.containers[0].env[4].valueFrom.fieldRef.fieldPath: Unsupported value: "spec.host": supported values: metadata.name, metadata.namespace, status.podIP

--

It seems that the Openshift API does not accept the same values that k8s does. Is there any evidence that this project is supported to run on Openshift, or are there future plans to test / develop for use on the Openshift platform? We would absolutely use it.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.