Code Monkey home page Code Monkey logo

laurentbroering / wbmbot_v2 Goto Github PK

View Code? Open in Web Editor NEW

This project forked from vel-san/wbmbot_v2

0.0 0.0 0.0 939 KB

WBMBOT_v2 is a Selenium-based Python bot designed to automate the application process for new flats listed by WBM GmbH. It prioritizes speed and efficiency to ensure your application is among the first to be considered in the random selection process for apartment viewings.

Home Page: https://hub.docker.com/r/vel7an/wbmbot_v2

License: MIT License

Python 97.62% Dockerfile 2.38%

wbmbot_v2's Introduction

Docker Pulls Docker Image Size

GitHub commit activity GitHub repo size GitHub forks GitHub Repo stars GitHub commits since latest release

GitHub Actions Workflow Status GitHub Actions Workflow Status GitHub Actions Workflow Status

Code Format

GitHub Release

Alt

WBMBOT_v2

An improved work-in-progress (WIP) and fully refactored version of the original WBMBOT by David Fischer.

WBMBOT_v2 is a Selenium-based Python bot designed to automate the application process for new flats listed by WBM (Wohnungsbaugesellschaft Berlin-Mitte) GmbH. It prioritizes speed and efficiency to ensure your application is among the first to be considered in the random selection process for apartment viewings.

IMPORTANT DISCLAIMER

The more you share this bot, the "less" your chances of finding an apartment will be. You do the math.

IN CASE YOU DON'T GET IT

Please do not over-share this bot to maximize your chances

Prerequisites

  • Python 3.10 or higher
  • Ubuntu Linux or Windows Docker (The bot didn't work properly on MacOS/Amazon Hosted instances)

Installation

To set up your environment and install the required dependencies, run the following command:

pip install -r wbmbot_v2/requirements.txt

Quick Start

To launch the bot, navigate to the project directory and execute:

python3 wbmbot_v2/main.py

On the first run, the bot will guide you through a setup process to gather necessary information for applications on wbm.de. This data will be stored in a local configs/wbm_config.json file in a human-readable format.

Security Notice: The information is saved unencrypted. Ensure the security of your configs/wbm_config.json file.

Alternatively, you can manually create the configs/wbm_config.json file with the following structure:

Configuration File Example

{
    "first_name": "John",
    "last_name": "Doe",
    "sex": "m",
    "emails": [
        "[email protected]"
    ],
    "notifications_email": "[email protected]",
    "street": "No-Add Str. 123",
    "zip_code": "12345",
    "city": "Berlin",
    "phone": "11223344",
    "wbs": "yes",
    "wbs_date": "23/04/1972",
    "wbs_num": "WBS 160",
    "wbs_rooms": "2",
    "wbs_special_housing_needs": "no",
    "exclude": [
        "wbs",
        "2-zimmer",
        "1-zimmer"
    ],
    "flat_rent_below": "600",
    "flat_size_above": "55",
    "flat_rooms_above": "1"
}

Notifications (E-mails)

The bot will be able to send you e-mail notifications once it applies for a flat from the e-mail you input in notifications_email within the config.

To do so, you need to export EMAIL_PASSWORD to your environment variables. If this variable is not found, no e-mails will be sent.

NOTE: If your email has 2FA (MFA), you need to create an App Password from your Outlook online settings.

NOTE: Only @outlook.com emails are currently supported

export EMAIL_PASSWORD=<password>

Outputs

You will get outputs such as:

  • Angebote HTML page (Entire HTML page) saved under offline_viewings/angebote_pages if ANY flats are found
  • EXPOSE PDF saved under offline_viewings/apartments_expose_pdfs if the bot applies to a flat

All of these files are saved per bot page-check. e.g. if the angebote page has at least 1 flat, everytime the bot wants to check that page it will download under a nested folder with date and time as its name.

Remember to do a clean-up if you don't want to view them!

Command-Line Interface

usage: main.py [-i INTERVAL] [-H] [-t]

A Selenium-based bot that scrapes 'WBM Angebote' page and auto applies on appartments based on user exclusion filters

options:
  -h, --help            show this help message and exit
  -i INTERVAL, --interval INTERVAL
                        Set the time interval in 'minutes' to check for new flats (refresh) on wbm.de. [default: 3 minutes]
  -H, --headless        If set, use 'headless' run. The bot will run in the background, otherwise, a chrome tab will show.
  -t, --test            If set, run test-run on the test data. This does not actually connect to wbm.de.

Docker

Build

docker build -f ci/docker/Dockerfile -t wbmbot_v2 .

Pull

docker pull vel7an/wbmbot_v2:latest

Run

If running for the first time, use -it to setup your config, if you already have the config ready in the correct directory, use -d

If you want to send emails as well to yourself as notifications, please add -e "EMAIL_PASSWORD=<password>" to the command

Without e-mail notifications

docker run -it \
    -v /PATH_HERE/offline_viewings:/home/offline_viewings \
    -v /PATH_HERE/logging:/home/logging \
    -v /PATH_HERE/configs:/home/configs \
    vel7an/wbmbot_v2:latest

With e-mail notifications

docker run -it \
    -e "EMAIL_PASSWORD=<password>" \
    -v /PATH_HERE/offline_viewings:/home/offline_viewings \
    -v /PATH_HERE/logging:/home/logging \
    -v /PATH_HERE/configs:/home/configs \
    vel7an/wbmbot_v2:latest

Filtering Strategy

The exclude list is designed to exclude listings based on specified keywords. Simply add your exclusion keywords to the list.

Alternatively you can also use the 3 variables in the config:

"flat_rent_below": "600"

"flat_size_above": "55"

"flat_rooms_above": "1"

Where the bot will "include" your options only. If the rent/size/rooms is equal OR below/above then it will consider the flat to apply.

Logging

Successful applications are recorded in logging/successful_applications.json.

Important: This log prevents reapplication to the same flats. DO NOT DELETE it unless you intend to re-apply to all available flats.

Additional Information

During setup, you can provide multiple email addresses. The bot will apply to each flat once per email address. By default, the bot refreshes wbm.de every 3 minutes to check for new listings.

As of now, there are no timeouts, bot checks, or captchas on the website (which we hope remains the case). However, given the limited number of flats available, frequent checks are not deemed necessary compared to platforms like immoscout24.

Embark on your apartment hunt with WBMBOT_v2. Good luck!

TODO

  • HIGH PRIORITY Fix form filling
  • HIGH PRIORITY Fix next page logic
  • HIGH PRIORITY Add an include only filter for zimmer numbers, rent cost & size
  • Add the reason if ignoring a flat (To make sure your filters are working properly) in logging
  • Add color_printer class into the works with the logger
  • Notify via e-mail whenever the bot applies to an application
  • Download the 'Angebote' page as an HTML for records keeping
  • Download the viewing of an apartment as a PDF (Available on WBM) for records keeping
  • Make a docker container out of the bot
  • CI/CD for Github
  • Fix test-data
  • Change "successful_applications.txt" to JSON type
  • Automatically detect if internet network connection is down and pause/restart once back
  • Add support for multi user wbm_config files
  • Add "excluded_applications.json" that shows all applications that were excluded by the filter
  • Make a compiled exec of the bot using pyinstaller

wbmbot_v2's People

Contributors

vel-san avatar fischer-hub avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.