yogeshojha / rengine Goto Github PK

While installing rengine container on windows, the CRLF is causing problem.

Look at this:

https://github.com/dhoer/docker-on-windows-gotchas

Successfully tagged rengine_web:latest
Starting rengine_db_1 ... done                                                                                                    Recreating rengine_web_1 ... done                                                                                                 Attaching to rengine_db_1, rengine_web_1
db_1   |
db_1   | PostgreSQL Database directory appears to contain a database; Skipping initialization
db_1   |
db_1   | 2020-07-11 17:00:03.343 UTC [1] LOG:  starting PostgreSQL 12.2 (Debian 12.2-2.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
db_1   | 2020-07-11 17:00:03.343 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
db_1   | 2020-07-11 17:00:03.343 UTC [1] LOG:  listening on IPv6 address "::", port 5432
db_1   | 2020-07-11 17:00:03.792 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
db_1   | 2020-07-11 17:00:04.274 UTC [25] LOG:  database system was shut down at 2020-07-11 16:59:19 UTC
web_1  | standard_init_linux.go:211: exec user process caused "no such file or directory"
db_1   | 2020-07-11 17:00:04.450 UTC [1] LOG:  database system is ready to accept connections
rengine_web_1 exited with code 1
Gracefully stopping... (press Ctrl+C again to force)

I remove the docker images and reinstall it ，the data is still here. How should i do?

Hi
just updated to V3 and the login page not redirecting to the dashboard even after navigating manually to /Dashboard its prompting for login again

Found a bug? Please fill out the sections below. 👍

HTTP Crawler not working

A summary of the issue.
I tried with 4-5 target but the HTTP Crawler result is zero always

Steps to Reproduce

1. For a quick verification i created scan engine with Fetch All URLs only
2. scanned 4-5 targets
3. scan completed but no result

Is your feature request related to a problem? Please describe.

Nmap takes a lot of time to scan the ports as well as with masscan etc.

Describe the solution you'd like

Add this tool

https://github.com/brandonskerritt/RustScan

Describe alternatives you've considered

Scans all 65k ports in 8 seconds (on 10k batch size).

Additional context

https://github.com/brandonskerritt/RustScan/blob/master/pictures/with_rustscan.gif

https://github.com/brandonskerritt/RustScan/blob/master/pictures/without_rustscan.gif

When scan runs, and activity is saved, the time is not updated on every activity change.

In urlscan.io Submit URLs to be scanned, search for historical results by IP, domain or URL, retrieve results and pivot to further leads. You can search all Public scans as well as your own Private scans on our platform. You can subscribe to our API plans as a Team and share your available quota amongst multiple users on the urlscan.io platform.

Use the search for historical results feature for targets all subdomains and sort out alive endpoints .

Example : https://urlscan.io/api/v1/search/?q=domain:bugcrowd.com

Take a look at tweet for more info https://twitter.com/prial261/status/1254503609217150976

Cheers

I follow the setup steps (docker and docker-compose are working well) but the installation crash on rengine_web_1 process and say that it cannot find "docker-entrypoint.sh" for container_linux.go. I delete the folder, clone it and restart installation but the same error is still display.

Technical details

My OS is windows 10.

while going to add a new engine cancel button is not available over there

Is your feature request related to a problem? Please describe.

Tool need to be powered with http request smuggling identification mechanism.

Describe the solution you'd like

Adding this tool will

https://github.com/defparam/smuggler

The application is using dirsearch tool to brute-force the directories but this will give 429 status code because most of the website implementing the rate-limiting. So instead of the dirsearch use meg tool on all subdomains with a custom wordlist.

Please have a look at this tool.

https://github.com/tomnomnom/meg

In import target option we can feed multiple domains in txt list . But if you can add a feature where we can skip the Subdomain scanning part and import a subdomains list from external scanned tools as there is many other way to gather Subdomains 😇 . Here we can’t skip the Subdomain discovery from scan engine . Hope you will take a look to skip the Subdomain discovery option , so the workflow will be

• Import Subdomains list in txt what discovered by other tools and recon methodology .
• Fetch live domains
• Do port scan
• Takeover scan
• Directory search
• Fetch all urls from wayback, OTX .
• Also if possible add urlscan.io for fetching target data ex: https://twitter.com/prial261/status/1254503609217150976

Cheers

Issue Summary

While scan was in progress, subdomains discovered was showing and after scan is completed, subdomains discovered are not visible anymore. They are available to export but aren't displayed from dashboard.

Steps to Reproduce

1. Started scanning "droom.in".
2. Used full scan configuration.
3. After scan is completed, subdomains discovered aren't visible.

• I have confirmed that this issue can be reproduced as described on a latest version/pull of reNgine: YES

Technical details

Please list out any technical details such as operating environment.
OS: Ubuntu 16.04.6 LTS (GNU/Linux 4.15.0-1080-gcp x86_64)

Docker:
Client: Docker Engine - Community
Version: 19.03.12
API version: 1.40
Go version: go1.13.10
Git commit: 48a66213fe
Built: Mon Jun 22 15:45:49 2020
OS/Arch: linux/amd64
Experimental: false

Server: Docker Engine - Community
Engine:
Version: 19.03.12
API version: 1.40 (minimum version 1.12)
Go version: go1.13.10
Git commit: 48a66213fe
Built: Mon Jun 22 15:44:20 2020
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.2.13
GitCommit: 7ad184331fa3e55e52b890ea95e65ba581ae3429
runc:
Version: 1.0.0-rc10
GitCommit: dc9208a3303feef5b3839f4323d9beb36df0a9dd
docker-init:
Version: 0.18.0
GitCommit: fec3683

Docker-Compose:
docker-compose version 1.26.2, build eefe0d31
docker-py version: 4.2.2
CPython version: 3.7.7
OpenSSL version: OpenSSL 1.1.0l 10 Sep 2019

cat SecLists/Discovery/DNS/dns-Jhaddix.txt | subgen -d DOMAIN.TLD | zdns A --name-servers 1.1.1.1 --threads 500 | jq -r "select(.data.answers[0].name) | .name"

#bugbountytips

python3 dirsearch.py -e conf,config,bak,backup,swp,old,db,sql,asp,aspx,aspx~,asp~,py,py~,rb,rb~,php,php~,bak,bkp,cache,cgi,conf,csv,html,inc,jar,js,json,jsp,jsp~,lock,log,rar,old,sql,sql.gz,http://sql.zip,sql.tar.gz,sql~,swp,swp~,tar,tar.bz2,tar.gz,txt,wadl,zip,.log,.xml,.js.,.json -u https://coderedmarketing.eccouncil.org/ -t 100 -w /root/tools/bruteforce/ffufplus/wordlist/dicc.txt -b

From a UX prespective, I think it is better to disable the export buttons than hiding it.

I want to add $HOME/.config/subfinder/config.yaml file , but i get error when execute the command

docker exec -it rengine_web_1 /bin/bash

Hey

If possible implement https://github.com/projectdiscovery/nuclei
and https://github.com/projectdiscovery/nuclei-templates

create option to upload nuclei templates or auto-update through a button in the UI

While someone will be going to click on start the scan button he/she might get confused as the start scan button is coming below all the engines.

Is your feature request related to a problem? Please describe.
Unable to detect changes in a target over a period of time and schedule reoccurring scans.

Describe the solution you'd like
Schedule new scans, weekly/monthly, and then detecting if there were any changes in the title, status code, new subdomains for a target, or a previous subdomain that wasn't active has became active.

The notification feature would come in handy along with this feature. Imagine getting notifications after a scan has detected changes in a target, allowing you to act quickly.

Add Gospider and Also Github Domian Discover also Dns Permutation And Virtual Host Discovery Use https://github.com/dark-warlord14/ffufplus

For That

also Add Nuclei
Amass for some domain brute force or add some other tools for subdomain Brute forceing

https://github.com/pikpikcu/XRCross/blob/master/XRCross

Check This Out U Will Get Some idea and Scripts

Hi, @yogeshojha I have recently cloned and followed up the instructions as mentioned in README.md. Here is the screenshot of the problem.

My machine Info:
Linux 5.4.0-40-generic x86_64 GNU/Linux

Steps to reproduce:
docker-compose up -- build

But I have run this web application using virtual environment which is easy to setup. I think we have to docs about setting up the environment using another method too .

Since most of the people run their recon tools on cloud servers, it would be great to have a web login.

Hi @yogeshojha I ran rengine in my local machine with burpsuite to find any vulnerability and with an automated extension like retire.js give some issue regarding jquery one. Have a look at it.

Issue detail

The library jquery version 3.1.1.min has known security issues. Supporting materials and references are listed below:

• https://snyk.io/vuln/SNYK-JS-JQUERY-174006
• https://blog.jquery.com/2019/04/10/jquery-3-4-0-released/
• https://nvd.nist.gov/vuln/detail/CVE-2019-11358

Impact

An attacker can able to inject properties on Object.prototype

Affected versions

The vulnerability is affecting all versions prior 3.4.0 (between * and 3.4.0)

Solutions

Updated the patched version of jquery

While installing facing this problems

/rengine# docker-compose up
Creating network "rengine_rengine_network" with the default driver
Creating rengine_db_1 ...
Creating rengine_db_1 ... error

ERROR: for rengine_db_1  Cannot start service db: driver failed programming external connectivity on endpoint rengine_db_1 (ecf18ca60cf994048dd5d06cdddf6e842ef11c97247afc04f6699447e6fd2d7e): Error starting userland proxy: listen tcp 0.0.0.0:5432: bind: address already in use

ERROR: for db  Cannot start service db: driver failed programming external connectivity on endpoint rengine_db_1 (ecf18ca60cf994048dd5d06cdddf6e842ef11c97247afc04f6699447e6fd2d7e): Error starting userland proxy: listen tcp 0.0.0.0:5432: bind: address already in use
ERROR: Encountered errors while bringing up the project.

docker-compose up -d this is not worked

Starting rengine-master_db_1 ...
Starting rengine-master_db_1 ... error

ERROR: for rengine-master_db_1 Cannot start service db: driver failed programming external connectivity on endpoint rengine-master_db_1 (2df8216f6d26811d173c9e0471534928e41f890996067b9ec2481727b64228ea): Error starting userland proxy: listen tcp 0.0.0.0:5432: bind: address already in use

ERROR: for db Cannot start service db: driver failed programming external connectivity on endpoint rengine-master_db_1 (2df8216f6d26811d173c9e0471534928e41f890996067b9ec2481727b64228ea): Error starting userland proxy: listen tcp 0.0.0.0:5432: bind: address already in use

Hello there Yogesh.
Firstly, thanks a ton for creating this awesome tool. loving this tool.
If it will be possible to accommodate a small request by adding a feature where the tool can include seclists. This will help in directly selecting the pre-defined lists for discovery.

Thanks in advance.

docker exec -it rengine_web_1 python manage.py createsuperuser
not work !!

''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
┌─[root@parrot]─[/home/bikram/Downloads/rengine-master]
└──╼ #sudo docker exec -it rengine_web_1 python3 manage.py createsuperuser
Error: No such container: rengine_web_1

Add a button to delete scan.

Hi,
Thanks for your tool, great job, I had started something similar but what you have done is really clean / functional.

I offer you some suggestions in addition to what others have already asked for (especially GF & Nuclei & github-subdomains.py)

• The ability to add an Amass config file, the detection power of the tool lies in the use of this file which allows to search for subdomains on services requiring API keys (like SecurityTrails which is a musthave).
  • The file in question : https://raw.githubusercontent.com/OWASP/Amass/master/examples/config.ini
  • Also used the possibility of bruteforce subdomains with Amass via -brute options
  • Possibly, via Amass (-dir options) or manually by coupling the output of the different tools, to have alerts on sub-domain differences between two scans

All these things can be done with the following command from Amass :
amass enum -active -o results.txt -d domain.tld -brute -w wordlist.txt -config config.ini -dir aDirectory

The difference between two scans via Amass can then be found with the following command:

amass track -d $DOMAIN -dir aDirectory | grep 'Found:')

In addition, I suggest you add Subdomainizer or something similar and ParamSpider Which is a good complement to hakrawler (I found it more convenient to use GF on ParamSpider instead of the hakrawler output.)

Regards,
Jomar

Hi,

Thank you so much for making this tool publicly available. I encountered some errors when uploading a wordlist, And the error was disclosing many sensitive information as we know. As I am also running this tool on my server (Public IP). So, leaving DEBUG as True might be a problem in the future. So, Please make this little change in this tool.

Best regards,
Mubassir

There is a known issue for installing reNgine on windows based docker.

standard_init_linux.go:211: exec user process caused "no such file or directory"

Looks like this is the problem with CLRF.

Will fix this soon

Find below list of features that will make the tool perfect in my opinion:

1. Javascript files enumeration
https://github.com/KathanP19/JSFScan.sh
A JSfiles enumeration module to extract hidden endpoints and secrets such as APIs keys will be an awesome module.
i'm using a tool called JSFScan which is a combine of multiple tools that take a list of subdomains.txt file and do the below:

• Gather Js Files Links
• Gather Endpoints For JSFiles
• Find Secrets For JSFiles
• Use Meg for fetching JsFiles for manual testing
• Make a wordlist using words from jsfiles

2. Parameter Discovery
https://github.com/s0md3v/Arjun
Parameter Discovery module will be a very nice addition to the framework personally i use Arjun.

3. SSRF - SQLi - IDORs - RCE patterns using GF-Patterns tool and gf from Tomnomnom
https://github.com/1ndianl33t/Gf-Patterns

I use Gf-pattern/gf to help me on grep all suspicious parameters from url that helps in (SSRF/SQLi/IDORs/RCE/Openredirect..etc), its combine of multiple tools such as gf and waybackurls and gives a results of txt file contains results based in what you chose, below example for SSRF, this module is very helpful.

▶ cat subdomains.txt | waybackurls | sort -u >> waybackdata | gf ssrf | tee -a ssfrparams.txt

▶ cat waybackdata | gf redirect | tee -a redirect.txt

Issue Summary

When running the project using docker-compose up, we get the following error message:

web_1  | /app/docker-entrypoint.sh: 7: /app/docker-entrypoint.sh: nc: not found

Steps to Reproduce

docker-compose pull
docker-compose up

Technical details

The installation of netcat binary is missing from the Dockerfile.

while getting more than 20k subdomains in a scan one can not look to each subdomain manually .
so if we could filter this part like if someone directly searches for takeover phrase and gets the subdomain that can be took over.

Issue Summary

The results of Subdomains Discovered have shown too many urls/directory. This caused the interface to be broken.
Below is a picture of my experience

If possible please add Pause option to Stop Scan for one domain and being active for others
.

Is your feature request related to a problem? Please describe.
Github can contain subdomains and endpoints that can't be found anywhere else.

Describe the solution you'd like
There are a number of tools that implement the ability to search Github for endpoints/subdomains such as https://github.com/gwen001/github-search

Also, you may need to provide the ability to add github tokens/API keys to other tools from the GUI.

It would be great if Screnshots pops up and then can easily go back to where I was. It gets loaded directly in the same page and when I go back via browser back button, it leaves where I was previously.

Issue: Right now, all targets are separately placed and as such it is difficult to figure out who owns what and which one does a particular domain belong to.

Fix: Creating a grouping function where a group of targets can be listed under a single group body wherein targets of an asset can be easily located and tasks can be applied simultaneously.

TLDR: Grouping domains of a particular asset would be great as it would help organize the recon workflow and cause less headaches!

When scan is started without selecting scan type, an exception is occurred

`

`

Add a search field for searching using different filters such as technology,port,status code and same in uri like path,parameters

when i try docker-compose up -d https://google.com
then it will show ERROR: No such service: https://google.com

While on the dashboard, either zoom or resize the browser window, the cards do not resize properly. This has something to do with the Bootstrap grid system.

Maybe somebody can work on it, Good first issue!

Code for this should be inside /reNgine/dashboard/templates/dashboard/index.html

Optional Grepping the output after fetching the endpoints from gau and hakwaler.
Showing possibility of open redirect or other vulns using gf and gf patterns.

It will be nice to see the subdomain link clickable on detailed scan result

The default scan engines are loaded via fixtures from the entry point. In docker, entry point runs every time a container starts. So, if the default scan engines are deleted, on the next container run, the deleted scan engines are added back. The idea is to run

python manage.py loaddata fixtures/default_scan_engines.json --app scanEngine.EngineType

from RUN rather than

ENTRYPOINT ["/app/docker-entrypoint.sh"]

Hi
kindly provide or update in the readme the default credential for 1st time login.

I would love to run this in my VPS but I'm not sure if i would be able to access it from anywhere I want.

What should i modify to do it correctly?