Code Monkey home page Code Monkey logo

tlosint-live's Introduction

Trace Labs Kali Linux build configuration

Overview

The repository includes a recipe file to build a Linux OSINT Distribution for Trace Labs based on the Kali Linux live-build-config (https://gitlab.com/kalilinux/build-scripts/live-build-config/-/tree/master).

image

The following changes have been made to the default Kali git repo:

  • Creation of a folder for Tracelabs under the kali-config/variant-tracelabs/package-lists/kali-list.chroot path. The kali-list.chroot can be modified to add additional packages or remove pre-configured packages that are required as part of the build process.
  • Creation of the kali-config/common/hooks/normal/osint-packages.chroot file to include the installation steps for all the git repositories that have been included in the build, and do not already have a package. The osint-packages.chroot file can be modified to add additional git repositories or remove pre-configured git repositories that are required as part of the build process. Please add any pre-requisite packages to the kali-live/kali-config/variant-tracelabs/package-lists/kali-list.chroot file.
  • Creation of the following folders under the directory kali-config/common/includes.chroot/usr/share/
    • applications: linked to the menu for applications
    • backgrounds: default Tracelabs background
    • desktop-directories: desktop directories with tools
    • firefox-esr/distribution: default Firefox policy

Build Steps

Building the ISO file on Docker

You will need a host\vm with Docker-engine installed. Installation guide can be found here: https://docs.docker.com/engine/install/ Once you have docker install, you just need to run:

docker build -t tlvm .
docker run --privileged -v $(pwd)/data:/data tlvm

Windows Powershell:

docker build -t tlvm .
docker run --privileged -v ${PWD}\data:/data tlvm

Windows CMD:

docker build -t tlvm .
docker run --privileged -v %cd%\data:/data tlvm

Once the docker container will finish running, you will be able to locate the ISO file inside a folder named "data" in the location you ran the commands. Now you can use the ISO file to install the tlosint vm.

Building the ISO file on your Debian host or Debian VM

Setup

This build has only been tested on a pre-existing Kali environment, as recommended by Offensive Security.

git clone https://github.com/tracelabs/tlosint-live.git
cd tl-osint
sudo ./build_tracelabsiso_recipe.sh

If the build process is successful, a .iso file will be created in the /opt/live-build-config/images directory. The .iso file can be used for live boot or to install the Virtual Machine. The .iso file can also be converted to a .ova file using the ovftool as outlined in the "Converting to an OVA" page (https://www.kali.org/docs/virtualization/converting-to-ova/).

OVA Download

We have set up Version 1.0 of this build in an OVA for you to easily try out. To get started, download the OVA file via the link below and run it in your choice of VM software (ie. VMware Workstation, Virtualbox etc.). The default credentials to log in to the TL OSINT VM are osint:osint

https://www.tracelabs.org/initiatives/osint-vm

Applications included in the build

Browsers

  • Firefox ESR
  • Tor Browser

Data Analysis

  • DumpsterDiver
  • Exifprobe
  • Exifscan
  • Stegosuite

Domains

  • Domainfy (OSRFramework)
  • Sublist3r

Downloaders

  • Browse Mirrored Websites
  • Metagoofil
  • Spiderpig
  • WebHTTrack Website Copier
  • Youtube-DL

Email

  • Buster
  • Checkfy (OSRFramework)
  • Infoga
  • Mailfy (OSRFramework)
  • theHarvester
  • h8mail

Frameworks

  • Little Brother
  • OSRFramework
  • sn0int
  • Spiderfoot
  • Maltego
  • OnionSearch

Phone Numbers

  • Phonefy (OSRFramework)
  • PhoneInfoga

Social Media

  • Instaloader
  • Twint
  • Searchfy (OSRFramework)
  • Tiktok Scraper
  • Twayback

Usernames

  • Alias Generator (OSRFramework)
  • Sherlock
  • Usufy (OSRFramework)

**Other tools

  • Photon
  • Sherlock
  • Shodan

Configuration Settings

Firefox

  • Delete cookies/history on shutdown
  • Block geo tracking
  • Block mic/camera detection
  • Block Firefox tracking
  • Preload OSINT Bookmarks

References:

tlosint-live's People

Contributors

5nacks avatar ballaswetha avatar borewalus avatar duckst3r avatar hodgegoblin avatar humandecoded avatar jcliolios avatar jspinel avatar kbrandwijk avatar kpcyrd avatar lowprivshighhopes avatar mmuyskens avatar xfreed0m avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

tlosint-live's Issues

Bookmarks are overwritten by Kali package

Describe the bug
When a new VM is created from OVA a Kali package overwrites the bookmarks.

To Reproduce
Steps to reproduce the behavior:

  1. Download OVA from Tracelabs
  2. Update OVA
  3. Bookmarks are removed

Expected behavior
Bookmarks shouldn't be removed

[Tool request] exiftool

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the tool you'd like
A clear and concise description of the tool you want added.

A link to the tool
A link to the tool code\repository.

Additional context
Add any other context or screenshots about the feature request here.

Chromium missing from Kali

Describe the bug
Build script is failing due to Chromium removal

To Reproduce
Steps to reproduce the behavior:

  1. Run script

Expected behavior
A clear and concise description of what you expected to happen.

Iso should build

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

  • OS: [e.g. iOS]
  • Browser [e.g. chrome, safari]
  • Version [e.g. 22]

Smartphone (please complete the following information):

  • Device: [e.g. iPhone6]
  • OS: [e.g. iOS8.1]
  • Browser [e.g. stock browser, safari]
  • Version [e.g. 22]

Additional context
Add any other context about the problem here.

Sherlock helper file not found

Describe the bug
Running 2021.2 OVA when attempting to execute /usr/bin/sherlock helper script results in file not found error

┌──(osint㉿osint)-[~]
└─$ sherlock -h                                                          
python3: can't open file '/usr/share/sherlock/sherlock.py': [Errno 2] No such file or directory

Reviewing /usr/bin/sherlock

!/bin/sh

exec python3 /usr/share/sherlock/sherlock.py "$@"

find that the sherlock.py file is in another directory.

$ sudo find / -name sherlock.py 2>/dev/null
/usr/share/sherlock/sherlock/sherlock.py
/usr/lib/python3/dist-packages/sherlock/sherlock.py

To Reproduce
Steps to reproduce the behavior:

  1. open terminal and run sherlock -h
    -or-
  2. launch from menu

Unable to use shodan-cli

Whenever I try to run shodan on the terminal, I get this:

Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 584, in _build_master
ws.require(requires)
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 901, in require
needed = self.resolve(parse_requirements(requirements))
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 792, in resolve
raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.VersionConflict: (shodan 1.23.1 (/usr/lib/python3/dist-packages), Requirement.parse('shodan==1.23.0'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/bin/shodan", line 6, in
from pkg_resources import load_entry_point
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 3259, in
def _initialize_master_working_set():
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 3242, in _call_aside
f(*args, **kwargs)
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 3271, in _initialize_master_working_set
working_set = WorkingSet._build_master()
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 586, in _build_master
return cls._build_from_requirements(requires)
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 599, in _build_from_requirements
dists = ws.resolve(reqs, Environment())
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 787, in resolve
raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'shodan==1.23.0' distribution was not found and is required by the application

How do I fix this?

Bookmarks menu issue

Describe the bug
Bookmarks menu is not showing as expected

To Reproduce
Steps to reproduce the behavior:
Open Chromium, browser bookmarks show at top of screen incorrectly

Expected behavior
Bookmarks should be one menu rather than split up

Screenshots
If applicable, add screenshots to help explain your problem.

Github Actions is about to break: "The `set-env` command is deprecated and will be disabled soon."

Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2020-10-01-github-actions-deprecating-set-env-and-add-path-commands/
A moderate security vulnerability has been identified in the GitHub Actions runner that can allow environment variable and path injection in workflows that log untrusted data to stdout. This can result in environment variables being introduced or modified without the intention of the workflow author. To address this issue we have introduced a new set of files to manage environment and path updates in workflows.
Starting today runner version 2.273.5 will begin to warn you if you use the add-path or set-env commands. We are monitoring telemetry for the usage of these commands and plan to fully disable them in the future.

Trace-Labs-OSINT-Search-Party-CTF-Contestant-Guide_v1.pdf contains incorrect credentials.

Bug Description
Incorrect credentials contained in desktop PDF, needs updated to the current credentials used.

Trace-Labs-OSINT-Search-Party-CTF-Contestant-Guide_v1.pdf file on the desktop contains old credentials for the Trace Labs OSINT VM. It instructs users to downlad the VM and log in with the username of kali, password kali. This should be updated to reflect the credentials used in the current version of Trace Labs OSINT VM 2021.2, which are username osint, password osint.

To Reproduce
Steps to reproduce the behavior:

  1. Open 'Trace-Labs-OSINT-Search-Party-CTF-Contestant-Guide_v1.pdf' on the desktop
  2. Scroll down to 'Page 4'
  3. See error:
    To log in use these default credentials:
    Username: Kali
    Password: Kali

Add more tools

Make menu dynamically update

Our menu is hard coded currently and we should look at either using the inbuilt kali menu or packaging up the menu changes to prevent breakage.

[Tool request] Toutatis

Describe the tool you'd like
Toutatis is a tool that can retrieve information such as phone number and email address (both public and obfuscated) from an instagram account

A link to the tool
https://github.com/megadose/toutatis

Additional context
This may be personal preference as i have not used many instagram tools, but i like the ability to see obfuscated email addresses and phone numbers. It is useful for cross referencing and potentially validating if the account belongs to the correct person.

[Tool request] onionsearch

Is your feature request related to a problem? Please describe.
We don't have many darkweb tools

Describe the tool you'd like
automate searching the dark web

A link to the tool
https://github.com/megadose/OnionSearch

Additional context
Add any other context or screenshots about the feature request here.

Todo:

  • update osint-packages.chroot, add OnionSearch
  • add OnionSearch launch helper script
  • add OnionSearch desktop file
  • update update.sh script for OnionSearch
  • update README.md

2021.2 VM Update script errors - git pull reference broken

Describe the bug
When running update script on desktop a number of update steps are failing related to pulling git repos. Found this is because the ORIG_HEAD file in the repo is zero bytes.

sudo git pull https://github.com/kpcyrd/sn0int.git --rebase
From https://github.com/kpcyrd/sn0int
 * branch            HEAD       -> FETCH_HEAD
fatal: update_ref failed for ref 'ORIG_HEAD': cannot lock ref 'ORIG_HEAD': unable to resolve reference 'ORIG_HEAD': reference broken

Identified additional repos with the ORIG_HEAD reference errors.

$ sudo find / -name ORIG_HEAD -size -1b
find: ‘/run/user/1000/gvfs’: Permission denied
/usr/share/LittleBrother/.git/ORIG_HEAD
/usr/share/sn0int/.git/ORIG_HEAD
/usr/share/WhatsMyName/.git/ORIG_HEAD
/usr/share/DumpsterDiver/.git/ORIG_HEAD
/usr/share/WikiLeaker/.git/ORIG_HEAD
/usr/share/OnionSearch/.git/ORIG_HEAD
/usr/share/Infoga/.git/ORIG_HEAD
/usr/share/Spiderpig/.git/ORIG_HEAD

To Reproduce
Steps to reproduce the behavior:
Double-click Update icon on desktop

Expected behavior
Update script to perform git pull --rebase on repos

Request to have a Github/Slack bot

Is your feature request related to a problem? Please describe.
I'd like to request a Github/Slack bot.

Describe the solution you'd like
Maybe use the integration described here: https://github.com/integrations/slack

Additional context
xFreedom suggested the use of a Github/Slack bot and 5nacks responded that it will be awesome. 😄

[Tool request] tiktok-scraper

Hey, I found this great tool for scraping information from TikTok which is very popular these days, but it's very hard to extract data by normal means so I think it should be added to the VM

github.com/drawrowfly/tiktok-scraper

Todo:

  • update kali.list.chroot, add nodejs and npm packages in pre-reqs
  • update osint-packages.chroot add tiktok-scraper
  • add tiktok-scraper launcher helper script in `/usr/bin/ folder
  • add .desktop file for tiktop-scraper tool
  • update the update.sh script: update npm to latest version as well as all npm package globally
  • update README.md file

[Tool request]

Will there be such tools like DaProfiler GHunt iKy Osintgram Profil3r?

Firefox ESR Policies Improvements

Firefox ESR Policies

The current policies for Firefox ESR are good, but we could consider shielding them a little more.

Here are the suggestions:

Policy Name Description Value
AppAutoUpdate Enable or disable automatic application update. TRUE
BlockAboutAddons Block access to the Add-ons Manager (about:addons). TRUE
BlockAboutConfig Block access to about:config. TRUE
BlockAboutProfiles Block access to About Profiles (about:profiles). TRUE
BlockAboutSupport Block access to Troubleshooting Information (about:support). TRUE
DefaultDownloadDirectory Set the default download directory. ${home}/Downloads
DisableAppUpdate Turn off application updates. FALSE
DisableBuiltinPDFViewer Disable the built in PDF viewer. TRUE
DisabledCiphers Disable ciphers. TLS_RSA_WITH_3DES_EDE_CBC_SHA
DisableDefaultBrowserAgent Prevent the default browser agent from taking any actions (Windows only). TRUE
DisableFormHistory Turn off saving information on web forms and the search bar. TRUE
DisablePasswordReveal Do not allow passwords to be revealed in saved logins. TRUE
DisableProfileImport Disables the "Import data from another browser" option in the bookmarks window. TRUE
DisableProfileRefresh Disable the Refresh Firefox button on about:support and support.mozilla.org TRUE
DisableSafeMode Disable safe mode within the browser. TRUE
DisableSecurityBypass Prevent the user from bypassing security in certain cases. InvalidCertificate = TRUE SafeBrowsing = TRUE
DisableSystemAddonUpdate Prevent system add-ons from being installed or update. FALSE
DisplayMenuBar Set the initial state of the menubar. default-on
DontCheckDefaultBrowser Don't check if Firefox is the default browser at startup. TRUE
DownloadDirectory Set and lock the download directory. ${home}/Downloads
EnableTrackingProtection Configure tracking protection. Tracking Protection Enabled: Cryptomining = TRUE Fingerprinting = TRUE
EncryptedMediaExtensions Configure tracking protection. DISABLED
Extensions/ExtensionSettings Control the installation, uninstallation and locking of extensions. Disable All / Force install uBlock Origin, HTTPS Everywhere
FlashPlugin Configure the default Flash plugin policy as well as origins for which Flash is allowed. Block All
InstallAddonsPermission Configure the default extension install policy as well as origins for extension installs are allowed. Disallow All
OfferToSaveLogins Control whether or not Firefox offers to save passwords. FALSE
OfferToSaveLoginsDefault Set the default value for whether or not Firefox offers to save passwords. FALSE
OverrideFirstRunPage Override the first run page.
OverridePostUpdatePage Override the upgrade page.
PasswordManagerEnabled Remove (some) access to the password manager. FALSE
PDFjs Disable or configure PDF.js, the built-in PDF viewer. DISABLED
Permissions: Autoplay Set permissions associated with video autoplay. Block All / Block New Requests
Permissions: Camera Set permissions associated with camera. Block All / Block New Requests
Permissions: Location Set permissions associated with location. Block All / Block New Requests
Permissions: Microphone Set permissions associated with microphone. Block All / Block New Requests
Permissions: Notifications Set permissions associated with notifications. Block All / Block New Requests
Permissions: VirtualReality Set permissions associated with virtual reality. Block All / Block New Requests
PopupBlocking Configure the default pop-up window policy as well as origins for which pop-up windows are allowed. Block All / Block New Requests
PromptForDownloadLocation Ask where to save each file before downloading. FALSE
SSLVersionMax Set and lock the maximum version of TLS. tls1.3
SSLVersionMin Set and lock the minimum version of TLS. tls1.2

Add a VPN helper script and notification for VPN connect

Is your feature request related to a problem? Please describe.
Make VPN access easier for non Linux users and potentially notification when VPN is active.

Describe the solution you'd like
A method to add VPN access to distro

Describe alternatives you've considered
Use the default method which is network manager

Additional context
enhance VPN support

Include a method to get the Anbox Android.img file

The android image for anbox is currently being manually placed into the correct location for the build. the android.img file is too large to be hosted in this repository.

Android image will have GApps and ARM support preloaded.

Investigate whether the update script can also update the distro

Is your feature request related to a problem? Please describe.
We have no way to update the distribution after release except for Kali updates

Describe the solution you'd like
Have the updater script have this capability

Describe alternatives you've considered
We could also package up the updates in apt

Additional context
See whether the updates can come from github

Squid proxy setting persists after ISO creation

Hi!

This had me stumped for a while, as I built my own ISO using the shell script then realised my network connection was being oddly flaky. It turns out this line:

grep -qxF "http_proxy=http://localhost:3128/" /etc/environment || echo "http_proxy=http://localhost:3128/" >> /etc/environment

Added the proxy to my /etc/environment - but it wasn't removed once the ISO had been built (and squid was no longer running). Should the environment file be restored (or backed up)?

Cheers,
Mike

Update to Github Actions Infrastructure

Currently there is a space limit on the free tier 14GB /dev/sdb and 20GB /dev/sda ubuntu OS github runners. (This is causing builds to fail)
A few options and avenues to research to address the issue

  1. Make smaller builds.
  2. Use a self-hosted runner.
  3. Slimming out the current runner, but I don't know how much we'll get out of that.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.