Code Monkey home page Code Monkey logo

botasaurus's Introduction

botasaurus

โœจ Botasaurus Framework ๐Ÿค–

๐Ÿš€ Swiss Army Knife for Bot Development ๐Ÿค–

(Programming Language - Python 3)

botasaurus forks Repo stars botasaurus License issues

View

Botasaurus - Your Ultimate Selenium-Based Bot Development Framework

๐Ÿค– Hi, I'm Botasaurus, a Selenium-based bot development framework designed to help you become a 10x Bot Developer!

๐Ÿ” What is Botasaurus and How Can it Help Me Become a 10x Bot Developer?

Think of Botasaurus as a supercharged version of Selenium. If you know Selenium, you're off to a great start because Botasaurus is a framework for Selenium with goodies like:

1. Account Generation

  • My Account Generation Module enables you to create humane user accounts effortlessly. This is particularly handy when you need multiple accounts while evading detection.

2. Temporary Email

  • Need to verify those accounts? I've got you covered with an inbuilt Temporary Email Module.

3. Simple Configuration

  • Configuring profiles, user agents, and proxies can be a headache in Selenium. But with Botasaurus, it's as easy as pie!

4. Parallel Bots

  • Run multiple bots in parallel and turn hours of automation into mere minutes with Botasaurus.

5. Easy Debugging

  • Debugging made painless! When a crash occurs due to an incorrect selector, etc., Botasaurus pauses the browser instead of closing it, facilitating on-the-spot debugging.

  • Capture important details like screenshots, page HTML, error logs, and execution time for easy debugging and generating time estimates of how long to keep the Bot Running.

6. Run Bots at Google's Scale with Kubernetes

  • Seamlessly integrate Botasaurus with Kubernetes to run your bots at Google's Scale. Easily control the number of bots and when to launch them via a simple UI interface.

7. Turbocharge Your Web Scraping

  • Say goodbye to redundant requests with our powerful caching feature. Botasaurus allows you to cache web scraping results, ensuring lightning-fast performance on subsequent scrapes.

8. Supercharged Selenium Driver

  • Elevate your Selenium game with Botasaurus. We've added Anti-Bot Detection features and added shortcut methods for common operations, all designed to save your valuable time.

9. Persist 1000 Profiles in 1 MB

  • Forget about heavy 100 MBs Chrome profiles. Use the Tiny Profile feature to save cookies in just a few KBs.

10. Evade IP Suspensions like Ninja Hatori

  • Using a single IP to create accounts and getting suspended?

No problem! Ditch expensive $15/GB Residential Proxies and use the FreeProxy Module to connect to the Tor Network and access a vast pool of unique Tor IPs all for free.

So, if you're comfortable with Selenium and want to level up your bot development game with easy debugging, simple configuration, parallel execution, and temporary emails, then you're in for a treat. Botasaurus is like Selenium on steroids, and you'll love building Bots with it!

So, if you're comfortable with Selenium and want to level up your bot development game with easy debugging, simple configuration, parallel execution, and temporary emails, then you're in for a treat. Botasaurus is like Selenium on steroids, and you'll love building Bots with it!

๐Ÿš€ Getting Started with Botasaurus

Let's quickly grasp how Botasaurus works with a simple example.

In this simple example, we'll walk through the process of scraping the heading text from https://www.omkar.cloud/.

Step 1: Install Botasaurus

Start by installing Botasaurus with the help of this command:

python -m pip install botasaurus

Step 2: Set Up Your Botasaurus Project

  1. Create a directory for our Botasaurus project and navigate to it:
mkdir my-botasaurus-project
cd my-botasaurus-project
code .  # Optionally, open the project in VSCode

Step 3: Write the Scraping Code

Within your project directory, create a Python script named main.py and paste the following code into main.py:

from botasaurus.launch_tasks import launch_tasks
from botasaurus import *

# Define a custom scraping Task
class ScrapeHeadingTask(BaseTask):

    def run(self, driver: BotasaurusDriver, data):
        # Visit the Omkar Cloud website
        driver.get("https://www.omkar.cloud/")

        # Get the heading element text
        heading = driver.text("h1")
    
        # Return the data to be saved as a JSON file in output/all.json
        return {
            "heading": heading
        }

if __name__ == "__main__":
    # Launch the web scraping task
    launch_tasks(ScrapeHeadingTask)

Let's break down this code:

  • We define a custom scraping task class named ScrapeHeadingTask.
class ScrapeHeadingTask(BaseTask):
  • Inside the run method, we are automatically passed a Selenium driver by Botasaurus.
    def run(self, driver: BotasaurusDriver, data):
  • In the run method:
    • We visit Omkar Cloud
    • Extract the heading text
    • Finally, return data to be saved as JSON and CSV files.
    driver.get("https://www.omkar.cloud/")

    # Get the heading element text
    heading = driver.text("h1")
  
    # Return the data to be saved as a JSON file in output/all.json
    return {
        "heading": heading
    }
  • Lastly, we launch the scraping task.
if __name__ == "__main__":
    # Launch the web scraping task
    launch_tasks(ScrapeHeadingTask)

Step 4: Run the Scraping Task

Now, let's run the bot:

python main.py

After running, the script will:

  • Launch Google Chrome
  • Visit omkar.cloud
  • Extract the heading text
  • Automatically save it as output/finished.json.

๐Ÿš€ Deep Dive into Botasaurus

Power of Bots is Immense, A Bot

  • Can apply on your behalf to Linkedin Jobs 24 Hours
  • Scrape Phone Number of Thousands of Buisnesses from Google Maps to sell your Products to.
  • Mass Message People on Twitter/LinkedIn/Reddit for selling your Product
  • Sign Up 100's of Accounts on MailChimp to send 50,000 (500 emails * 100) Emails all for free

So, Ready to unlock the power of Bots?

๐Ÿ“š Read the Tutorial here and embark on your journey of bot mastery! ๐Ÿš€

Sponsors

Special Sponsor

special sponsor Bright Data

Love It? Star It! โญ

botasaurus's People

Contributors

chetan11-dev avatar chetanjainsirsa avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.