A website downloader with a built-in JavaScript engine!

Purpose
Compatibility
Set Up
Functionality
Use Cases
Future Goals

The purpose of Web Loader is to give a user the ability to download portions of or entire websites.

Web Loader is compatible with PhantomJS version 2.1.1 and above.

1. Install PhantomJS for your operating system: http://phantomjs.org/download.html.
2. Clone this repository.
3. You're done! You are now able to run loader.js from the command line.

There are two main ways to download a website with Web Loader:

A host website download involves downloading every page that is present on a given host.

To run a host website download, run a command of the following form:

phantomjs loader.js https://example.com

A path website download involves downloading pages that are in a specific path on the host.

To run a path website download, run a command of the following form:

phatomjs loader.js https://example.com /example/path

There are a few use cases for this software:

Running this program multiple times over a given time period could allow a user to detect changes that are made to a website. This might be useful in tracking when a business announces a new product or updates an application.

The resulting text files generated by this program can be rapidly searched, allowing a user to quickly check if website contains certain verbiage.

In general, this program may be useful in any sort of research that involves extracting verbiage from the internet.

See Projects.