Code Monkey home page Code Monkey logo

getjs's Introduction

GetJS

License contributions welcome

getJS is a tool to extract all the javascript files from a set of given urls.

The urls can also be piped to getJS, or you can specify a singel url with the -url argument. getJS offers a range of options,

varying from completing the urls, to resolving the files.

Prerequisites

Make sure you have GO installed on your system.

Installing

getJS is written in GO. You can install it with go get:

go install github.com/003random/getJS@latest

Usage

Note: When you supply urls from different sources, e.g. with stdin and an input file, it will add all the urls together :)
Example: echo "https://github.com" | getJS --url https://example.com --input domains.txt

To get all options, do:

getJS -h
Flag Description Example
--url The url to get the javascript sources from getJS --url https://poc-server.com
--method The request method. e.g. POST or GET. Default: "GET" getJS --url https://poc-server.com --method POST
--timeout The request timeout. Default: 10 (secs) getJS --url https://poc-server.com --timeout 15
--insecure Skip SSL certificate verification. Use when the cert is expired or invalid getJS --url https://poc-server.com --insecure
--header Custom request header(s) getJS --url https://poc-server.com --header "Authorization: Bearer token"
--input Input file with urls getJS --input domains.txt
--output The file where to save the output to getJS --output output.txt
--verbose Display info of what is going on getJS --verbose
--complete Complete the urls. e.g. /js/index.js -> https://example.com/js/index.js getJS --complete
--resolve Resolve the output and filter out the non existing files (Can only be used in combination with --complete) getJS --complete --resolve
--nocolors Don't color the output getJS --nocolors

Examples

screenshot

getJS supports stdin data. To pipe urls to getJS, use the following:

$ cat domains.txt | getJS

To save the js files, you can use:

$ getJS --complete --url https://poc-server.com | xargs wget

If you would like the output to be in JSON format, you can combine it with @Tomnomnom's toJSON:

$ getJS --url https://poc-server.com | tojson

To feed urls from a file use:

$ getJS --input domains.txt

To save the results to a file, and don't display anything, use:

$ getJS --url https://poc-server.com --output results.txt

If you want to have a list of full urls as output use:

$ getJS --url domains.txt -complete

If you want to only show the existing js files, use:

$ getJS --url domains.txt --complete --resolve

Built With

  • GO - GOlanguage
  • Goquery - HTML parser with syntaxes like jquery, in GO

Contributing

You are free to submit any issues and/or pull requests :)

License

This project is licensed under the MIT License.

Acknowledgments

  • @jimen0 for helping getting me started with GO

This is my first tool written in GO. I created it to learn the language more. (useful feeback is always welcome!)

getjs's People

Contributors

003random avatar abss0x7tbh avatar fededamian avatar jimen0 avatar njmulsqb avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

getjs's Issues

Cannot Get Source

[!] Couldn't get sources from subfinderAcrons1.txt
[!] Error: Get "subfinderAcrons1.txt": unsupported protocol scheme ""

Hang

This tool needs to implement a timeout for the requests do you think you could add that or if I missed out how to do so and you already did let me know this gets hung on some urls and I am using this in my static analysis tool

https://github.com/devcoinfet/Static_Analysis.py

I don't know go and think this would really bolsters this tools usefulness I have this in use in 2 diff tools I would be very appreciative if you did hook it up with a default timeout of like 10 seconds

I have seen it hang for a very long time on 1 url

For more readablity ,prefix the domain name to the stdout

@003random

For more readability can you please prefix the domain name to the result found to be printed in stdout.

It would make readability improve particularly when multiple domains are placed as input to getJS.

For example
In the output of this command
getJS -input=domain.txt -output out

one may provide several domains, the output will be something like
aaa.aaa/aaa.js
bbb.bbb.vbb.js

In this case if the first output corresponds to say AAA.com and second to BBB.com, please prefix them in their respective outputs.

so the modified output would correspond to
AAA.com/aaa.aaa/aaa.js and BBB.com/bbb.bbb/vbb.js
respectively.

broken pipe

is this my issue or the tools issue?full error below

Traceback (most recent call last):
File "/root/Astra/Astra.py", line 172, in
asyncio.get_event_loop().run_until_complete(start(urls, threads))
File "/usr/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
return future.result()
File "/root/Astra/Astra.py", line 142, in start
await asyncio.gather(*tasks)
File "/root/Astra/Astra.py", line 123, in parser
html = await fetch(session=session, url=url)
File "/root/Astra/Astra.py", line 86, in fetch
html = await response.text()
File "/usr/lib/python3/dist-packages/aiohttp/client_reqrep.py", line 1081, in text
return self._body.decode(encoding, errors=errors) # type: ignore
File "/usr/lib/python3.9/encodings/cp1254.py", line 15, in decode
return codecs.charmap_decode(input,errors,decoding_table)
UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 183079: character maps to

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/root/Astra/Astra.py", line 188, in
print(f'\n{col.magenta}[-]{col.red} Error : {repr(err)}{col.reset}')
BrokenPipeError: [Errno 32] Broken pipe

issue installing

HI
Currently trying to install running

go install github.com/003random/getJS@latest

im getting this

go: finding module for package github.com/logrusorgru/aurora
go: finding module for package github.com/PuerkitoBio/goquery
go: finding module for package github.com/spf13/pflag
go: found github.com/PuerkitoBio/goquery in github.com/PuerkitoBio/goquery v1.8.0
go: found github.com/logrusorgru/aurora in github.com/logrusorgru/aurora v2.0.3+incompatible
go: found github.com/spf13/pflag in github.com/spf13/pflag v1.0.5

i also tried

go get -u github.com/003random/getJS@latest                                                                                                                                                                                         
go: downloading github.com/logrusorgru/aurora v0.0.0-20200102142835-e9ef32dff381
go: downloading github.com/spf13/pflag v0.0.0-20170417173400-9e4c21054fa1
go: downloading golang.org/x/net v0.0.0-20220412020605-290c469a71a5
go: downloading golang.org/x/sys v0.0.0-20211216021012-1d35b9e2eb4e
go get: installing executables with 'go get' in module mode is deprecated.
        To adjust and download dependencies of the current module, use 'go get -d'.
        To install using requirements of the current module, use 'go install'.
        To install ignoring the current module, use 'go install' with a version,
        like 'go install example.com/cmd@latest'.
        For more information, see https://golang.org/doc/go-get-install-deprecation
        or run 'go help get' or 'go help install'.
go get: added github.com/003random/getJS v0.0.0-20220318111727-2f687c7fce08
go get: added github.com/PuerkitoBio/goquery v1.8.0
go get: upgraded github.com/logrusorgru/aurora v0.0.0-20200102142835-e9ef32dff381 => v2.0.3+incompatible
go get: upgraded github.com/spf13/pflag v0.0.0-20170417173400-9e4c21054fa1 => v1.0.5
go get: upgraded golang.org/x/net v0.0.0-20190620200207-3b0461eec859 => v0.0.0-20220412020605-290c469a71a

Thank you

Timeout Issue

Hello @003random ,
getting i/o timeout issue if the first resource is not resolving from list of domains.

no required module provides package

root@cyberghazi-vmwarevirtualplatform getJS]# go build main.go
main.go:14:2: no required module provides package github.com/PuerkitoBio/goquery: go.mod file not found in current directory or any parent directory; see 'go help modules'
main.go:15:2: no required module provides package github.com/logrusorgru/aurora: go.mod file not found in current directory or any parent directory; see 'go help modules'
main.go:16:2: no required module provides package github.com/spf13/pflag: go.mod file not found in current directory or any parent directory; see 'go help modules'
[root@cyberghazi-vmwarevirtualplatform getJS]# ls
main.go README.md

issue installing

compile: version "go1.17.6" does not match go tool version "go1.18.2"

thx

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.