Code Monkey home page Code Monkey logo

missinglink's Introduction

The Missing Link

Given two sets of entities - a sample group and a control group - identify relationships the sample entities have in common with each other but not with the control group.

For example, given ...

  • A sample set of IP addresses exhibiting malicious behavior and assumed to be infected with malware
  • A control group of IPs assumed (but not known) to be clean
  • A list of network connections made by both groups

... determine what network traffic the infected IPs have in common with each other but not in common with the control group. This raises the botnet control channel to the top of the list while minimizing traffic that all hosts have in common such as social media and content delivery networks.

Missing Link Diagram

For each relationship target, the algorithm outputs these fields:

  • target - the target of the relationship (e.g. destination IP)
  • ratio - the ratio of related sample entities to related control entities. A ratio > 1 means the target is over represented in the sample group. A ratio == 1 means the target is equally represented in both groups. A ratio < 1 means the target is underrepresented in the sample group.
  • deviations_from_mean - the number of standard deviations from the average ratio
  • sample_count - the number of entities in the sample group with this relationship
  • sample_percent - the percent of entities in the sample group with this relationship
  • control_count - the number of entities in the control group with this relationship
  • control_percent - the percent of entities in the control group with this relationship

Note: if labels are provided for the two sets when the object is instantiated, those labels will be used instead of "sample" and "control."

missinglink's People

Contributors

davedittrich avatar mattcarothers avatar

Watchers

 avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.