Pinterest-infinite-crawler

An infinite pinterest crawler, crawl image by page.

Requirements

Python 3.7+
Selenium, requests, beautifulsoup4, pyyaml
Chrome + Chromedriver

Installation

Download requirements

git clone https://github.com/mirusu400/Pinterest-infinite-crawler.git
cd Pinterest-infinite-crawler
pip install -r requirements.txt

Download chromedriver

You MUST download ChromeDriver as the same version of Chrome.

And replace it the same directory with main.py.

(Optional) Set config.yaml

Copy .config.yaml to config.yaml and fill your Pinterest's email, password and directorys to save images

email: [your email here]
password: [your password here]
directory: ./download

Usage

python main.py

Using argument

You can also run crawler by passing argument, here are full document:

usage: main.py [-h] [-e EMAIL] [-p PASSWORD] [-d DIRECTORY] [-l LINK] [-g PAGE]

optional arguments:
  -h, --help                            show this help message and exit
  -e EMAIL, --email EMAIL               Your Pinterest account email
  -p PASSWORD, --password PASSWORD      Your Pinterest account password
  -d DIRECTORY, --directory DIRECTORY   Directory you want to download
  -l LINK, --link LINK                  Link of Pinterest which you want to scrape
  -g PAGE, --page PAGE                  Number of pages which you want to scrape
  -b BATCH, --batch BATCH               Enable batch mode (Please read README.md!!)

Example:

main.py -e [email protected] -p [your_password] -d download_image -l https://pinterest.com/ -g 10

Batch mode

You can download multiple Pinterest links in a one, using batch mode

Copy and paste .batch.json to batch.json and modify json array files.

[
    {
        "index": "1",
        "link": "https://www.pinterest.co.kr/pin/362750944993136496/",
        "dir": "./download1"
    },
    {
        "index": "2",
        "link": "https://www.pinterest.co.kr/",
        "dir": "./download2"
    },
    ...
]

Use Batch mode in command line

main.py -b

Q & A

What is `Link to scrape` mean?

You can select any pages what you want to scrape in Pinterest, not only main page. Such as:

Does it can download video?

No, you can only download jpg images from this tool. Video is not support for now.

Contribute

If you find an issue or wants to contribute, please issue or pull request.

xenlata / pinterest-infinite-crawler Goto Github PK