Code Monkey home page Code Monkey logo

imagescraper's Introduction

ImageScraper ๐Ÿ“ƒ

A high performance, easy to use, multithreaded command line tool which downloads images from the given webpage.

Build Status Downloads Test Coverage
Build Status PyPi downloads Coverage Status

Demo

Click here to see it in action!

Download

tar file:

Grab the latest stable build from - Pip: https://pypi.python.org/pypi/ImageScraper

pip install (recommended):

You can also download using pip:

$ pip install ImageScraper

Dependencies

Note that ImageScraper depends on lxml, requests, setproctitle, and future. If you run into problems in the compilation of lxml through pip, install the libxml2-dev and libxslt-dev packages on your system.

Usage

$ image-scraper [OPTIONS] URL

You can also use it in your Python scripts. (Deprecated)

import image_scraper
image_scraper.scrape_images(URL)

Options

-h, --help            show this help message and exit
-m MAX_IMAGES, --max-images MAX_IMAGES
                    Limit on number of images
-s SAVE_DIR, --save-dir SAVE_DIR
                    Directory in which images should be saved
-g, --injected        Scrape injected images
--proxy-server PROXY_SERVER
                    Proxy server to use
--min-filesize MIN_FILESIZE
                    Limit on size of image in bytes
--max-filesize MAX_FILESIZE
                    Limit on size of image in bytes
--dump-urls           Print the URLs of the images
--formats [FORMATS [FORMATS ...]]
                    Specify formats in a list without any separator. This
                    argument must be after the URL.
--scrape-reverse      Scrape the images in reverse order
--filename-pattern FILENAME_PATTERN
                    Only scrape images with filenames that match the given
                    regex pattern
--nthreads NTHREADS   The number of threads to use when downloading images.

If you downloaded the tar:

Extract the contents of the tar file.

$ cd ImageScraper/
$ python setup.py install
$ image-scraper --max-images 10 [url to scrape]

Examples

Scrape all images

$ image-scraper  ananth.co.in/test.html

Scrape at max 2 images

$ image-scraper -m 2 ananth.co.in/test.html

Scrape only gifs and download to folder ./mygifs

$ image-scraper -s mygifs ananth.co.in/test.html --formats gif

NOTE:

By default, a new folder called "images_" will be created in the working directory, containing all the downloaded images.

Issues

Q.)All images were not downloaded?

It could be that the content was injected into the page via JavaScript; this scraper doesn't run JavaScript.

Contribute

If you want to add features, improve them, or report issues, feel free to send a pull request!!

Contributors

Disclaimer

ImageScraper is to be used education/research purposes only. The authors takes NO responsibility and/or liability for how you choose to use any of the tools/source code/any files provided. By using ImageScraper, you understand that you are AGREEING TO USE AT YOUR OWN RISK.

License

GPL V3

Analytics

imagescraper's People

Contributors

sananth12 avatar ssundarraj avatar vigneshmanix avatar osborne6 avatar ananthjan avatar tsleyson avatar eugriner avatar bryant1410 avatar solomon1732 avatar dannyflax avatar

Watchers

James Cloos avatar Nischitha K N avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.