Code Monkey home page Code Monkey logo

ransomwatch's Introduction

ransomwatch engine ransomwatch dockerimage builder ransomwatch codeql analysis

ransomwatch trails the extortion sites used by ransomware groups and surfaces an aggregated feed of claims

please use the issue template when submitting new groups


⚠️

content within ransomwatch.telemetry.ltd, posts.json, groups.json alongside the docs/ & source/ directories is dynamically generated based on hosting choices of real-world threat actors in near-real-time.

whilst sanitisation efforts have been taken, by viewing or accessing ransomwatch you acknowledge you are doing so at your own risk

if you leverage ransomwatch in commercial platforms, please consider becoming a sponsor 💞

key outputs

web:// ransomwatch.telemetry.ltd

json:// ransomwhat.telemetry.ltd/posts

json:// ransomwhat.telemetry.ltd/groups

  • groups.json contains hosts, nodes, relays and mirrors for a tracked group or actor
  • posts.json contains extracted posts, noted by their discovery time and accountable group

technicals

this is a live repository that utilizes a combination of GitHub actions and a service container. it visits, parses, and reports on monitored hosts in near-real-time in a self-contained manner

content fetching is done with psf/requests - if rendering is required mozilla/geckodriver and seleniumhq/selenium are leveraged.

The frontend is ultimately generated with markdown, using markdown.py and served with docsifyjs/docsify thanks to pages.github.com

graphs or visualisations are generated with plotting.py with the help of matplotlib/matplotlib

post indexing is done with a mix of grep, awk and sed within parsers.py - it's brittle and like any ̴̭́H̶̤̓T̸̙̅M̶͇̾L̷͑ͅ ̴̙̏p̸̡͆a̷̛̦r̵̬̿s̴̙͛ĩ̴̺n̸̔͜g̸̘̈, has a limited lifetime.

tools

rendered HTML for each page is viewable within the source directory

  • screenshotter.py a playwright script to generate high-resolution screenshots of online hosts
  • srcanalyser.py a basic extractor for emails, internal and external links found within page source
  • browse-hosts.sh a simple cURL based iterator for sweeping URL checks
  • sources.zsh an aggregator of various locations that surface new groups for ransomwatch
  • uptimekuma-importer.py a script to convert the group data into a uptime-kuma configuration file
  • parsers.sh a health-check script that provides details on parsers that are returning no fields

a flattened version of groups.json with each host as its own object can be found at assets/groups-kv.json. the structure is an array of objects, each representing a distinct entity/group with each containing all properties (like name, captcha, parser, etc.) at the same level, including potential repetition on elements such as profile and meta. some data analysis tools work with this structure in an easier manner requiring less transposing.

cli operations

fetching hidden services requires a tor circuit! establish one with;

docker run -p9050:9050 ghcr.io/joshhighet/torsocc:latest
➜  ransomwatch git:(main) ✗ ./ransomwatch.py --help

       _______________                        |*\_/*|________
      |  ___________  |                      ||_/-\_|______  |
      | |           | |                      | |           | |
      | |   0   0   | |                      | |   0   0   | |
      | |     -     | |                      | |     -     | |
      | |   \___/   | |                      | |   \___/   | |
      | |___     ___| |                      | |___________| |
      |_____|\_/|_____|                      |_______________|
        _|__|/ \|_|_.............💔.............._|________|_
       / ********** \                          / ********** \
     /  ************  \     ransomwhat?      /  ************  \
    --------------------                    --------------------

usage: ransomwatch.py [-h] [--name NAME] [--location LOCATION]
                      [--append APPEND]
                      {add,append,scrape,parse,list,markdown,check}

👀 🦅 ransomwatch

positional arguments:
  {add,append,scrape,parse,list,markdown,check}
                        operation to execute

options:
  -h, --help            show this help message and exit
  --name NAME           provider name
  --location LOCATION   onionsite fqdn
  --append APPEND       add onionsite fqdn to existing record

newly indexed posts can be sent to discord by providing a DISCORD_WEBHOOK var when running parse.

DISCORD_WEBHOOK=https://discord.com/api/webhooks/xxxxx/xxx ./ransomwatch.py parse

datamap

erDiagram
    groups_json ||--|{ group : contains
    group {
        string name "group name"
        boolean captcha "captcha status"
        boolean parser "parser status"
        boolean javascript_render "javascript status"
        string meta "freeform text"
        string url "notable articles and references"
    }
    group ||--|{ locations : has
    locations {
        string fqdn "fully qualified domain name"
        string title "page title"
        int version "hidden service version"
        string slug "full URI"
        boolean available "availability status"
        datetime updated "timestamp of last update"
        datetime lastscrape "timestamp of last scrape"
        boolean enabled "status"
    }
    group ||--|{ post : references
    post {
        string post_title "post title"
        string group_name "associated group name"
        datetime discovered "timestamp of discovery"
    }
Loading

accessing data with cURL and JQ

print last 10 claims by group lockbit3
curl -sL ransomwhat.telemetry.ltd/posts \
| jq -r '.[] | select(.group_name == "lockbit3") | .post_title' \
| tail -n 10
print all online URL's
curl -sL ransomwhat.telemetry.ltd/groups \
| jq -r '.[] | .locations[] | select(.available == true) | .slug'
print group data for "lockbit3"
curl -sL ransomwhat.telemetry.ltd/groups \
| jq -r '.[] | select(.name == "lockbit3")'
print the last 20 claims
curl -sL ransomwhat.telemetry.ltd/posts \
| jq -r '.[] | [.group_name, .post_title] | @tsv' \
| sed 's/ /_/g' | column -t | tail -n 20

ransomwatch is licensed under unlicense.org

ransomwatch's People

Contributors

dependabot[bot] avatar github-actions[bot] avatar joshhighet avatar weddige avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.