vel-san / wbmbot_v2 Goto Github PK

Describe the bug
If the bot finds several flats on the page that match your requirements, it will attempt to apply to them 1 by 1. The problem here is that once it is done applying to the first flat, it goes back to the main page "Angebote" page.

It fails to jump to the second flat button with "flat_elem" getting stale, because the page was refreshed! But the "flat_obj" is properly updated.

The bug is really simple to fix but I just cannot fix it properly if there are no flats available to test it out 😢

To Reproduce
Attempt to apply on 2 flats, 2nd flat will fail (to be precise, it will apply to the first flat twice instead of apply to both)

Expected behavior
Should apply to all flats 1 by 1

Screenshots
N/A

Environment

• OS: Linux + Docker
• Version: Doesn't matter;

Additional context
Related bug lines:

flat_elem (Fails to update as intended and is always taking the first element in the list):

• wbmbot_v2/wbmbot_v2/helpers/webDriverOperations.py

  Line 411 in 0b36ed3

  flat_elem = all_flats[i]

• wbmbot_v2/wbmbot_v2/helpers/webDriverOperations.py

  Line 404 in 0b36ed3

  flat_elem = all_flats[i]

flat_obj (Works properly, updates its values as intended):

• wbmbot_v2/wbmbot_v2/helpers/webDriverOperations.py

  Line 407 in 0b36ed3

  flat_obj = flat.Flat(flat_elem.text)

Describe the bug
Bot crashes when there is an apartment on WBM Site. It tries to restart itself then crash again

To Reproduce
Just run the bot, one customisation i did was to set interval to 1 mionute

Expected behavior
Bot should not crash and apply for the apartment

Screenshots

Environment

• MAC OS 14.4.1

Additional context
[29.04.2024 - 15:54] [main.py:71] [INFO] Initializing Script (v1.1.4) (Headless? False) 🚀
[29.04.2024 - 15:54] [main.py:76] [INFO] Checking for internet connection 🔎
[29.04.2024 - 15:54] [main.py:84] [SUCCESS] Online 🟢
[29.04.2024 - 15:54] [io_operations.py:24] [INFO] Loading WBM config ⚙️
[29.04.2024 - 15:54] [main.py:107] [INFO] Connecting to 'https://www.wbm.de/wohnungen-berlin/angebote/' 🔗
[29.04.2024 - 15:54] [webDriverOperations.py:252] [SUCCESS] Cookies have been accepted 🍪
[29.04.2024 - 15:54] [webDriverOperations.py:380] [INFO] Looking for flats 👀
[29.04.2024 - 15:54] [webDriverOperations.py:387] [SUCCESS] Found 2 flat(s) in total 💡
[29.04.2024 - 15:54] [main.py:122] [ERROR] Bot has crashed... Attempting to restart it now! ❤️‍🩹
[29.04.2024 - 15:54] [main.py:125] [ERROR] Crash reason: lxml.html.clean module is now a separate project lxml_html_clean.
Install lxml[html_clean] or lxml_html_clean directly.
^CTraceback (most recent call last):
File "/Users/UsmanTatla/Desktop/wbmbot_v2-main/wbmbot_v2/main.py", line 111, in main
webDriverOperations.process_flats(
File "/Users/UsmanTatla/Desktop/wbmbot_v2-main/wbmbot_v2/helpers/webDriverOperations.py", line 391, in process_flats
hpd.save_viewing_offline(
File "/Users/UsmanTatla/Desktop/wbmbot_v2-main/wbmbot_v2/httpsWrapper/httpPageDownloader.py", line 30, in save_viewing_offline
save_webpage(
File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pywebcopy/init.py", line 107, in save_page
page = config.create_page()
^^^^^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pywebcopy/configs.py", line 241, in create_page
from .core import WebPage
File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pywebcopy/core.py", line 12, in
from .elements import HTMLResource
File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pywebcopy/elements.py", line 24, in
from .parsers import iterparse
File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pywebcopy/parsers.py", line 14, in
from lxml.html.clean import Cleaner
File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/lxml/html/clean.py", line 18, in
raise ImportError(
ImportError: lxml.html.clean module is now a separate project lxml_html_clean.
Install lxml[html_clean] or lxml_html_clean directly.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/Users/UsmanTatla/Desktop/wbmbot_v2-main/wbmbot_v2/main.py", line 134, in
main()
File "/Users/UsmanTatla/Desktop/wbmbot_v2-main/wbmbot_v2/main.py", line 129, in main
main()
File "/Users/UsmanTatla/Desktop/wbmbot_v2-main/wbmbot_v2/main.py", line 129, in main
main()
File "/Users/UsmanTatla/Desktop/wbmbot_v2-main/wbmbot_v2/main.py", line 129, in main
main()
File "/Users/UsmanTatla/Desktop/wbmbot_v2-main/wbmbot_v2/main.py", line 127, in main
time.sleep(5)
KeyboardInterrupt

Describe the bug
Bot ignores the apartment as users set limit is something greater than 0. Bot detects the apartment size as 0 m2

To Reproduce
Run the bot when apartment is available

Screenshots

Environment
MAC OS - 14.4.1

Additional context
[29.04.2024 - 16:17] [main.py:76] [INFO] Checking for internet connection 🔎
[29.04.2024 - 16:17] [main.py:84] [SUCCESS] Online 🟢
[29.04.2024 - 16:17] [io_operations.py:24] [INFO] Loading WBM config ⚙️
[29.04.2024 - 16:17] [main.py:107] [INFO] Connecting to 'https://www.wbm.de/wohnungen-berlin/angebote/' 🔗
[29.04.2024 - 16:17] [webDriverOperations.py:252] [SUCCESS] Cookies have been accepted 🍪
[29.04.2024 - 16:17] [webDriverOperations.py:380] [INFO] Looking for flats 👀
[29.04.2024 - 16:17] [webDriverOperations.py:387] [SUCCESS] Found 2 flat(s) in total 💡
[29.04.2024 - 16:17] [webDriverOperations.py:438] [WARNING] Ignoring flat '3-ZIMMER WOHNUNG IN MITTE WBS 140' because the size doesn't match our criteria --> Flat Size: 0.0 m² | User wants it above: 40 m² 🙈
[29.04.2024 - 16:17] [webDriverOperations.py:438] [WARNING] Ignoring flat '2-ZIMMER-WOHNUNG IN MITTE WBS 160/180/220' because the size doesn't match our criteria --> Flat Size: 0.0 m² | User wants it above: 40 m² 🙈
[29.04.2024 - 16:17] [webDriverOperations.py:61] [ERROR] Failed to switch page, last page reached ❌
[29.04.2024 - 16:18] [webDriverOperations.py:504] [INFO] Reloading main page 🔄
[29.04.2024 - 16:18] [webDriverOperations.py:380] [INFO] Looking for flats 👀
[29.04.2024 - 16:18] [webDriverOperations.py:387] [SUCCESS] Found 2 flat(s) in total 💡
[29.04.2024 - 16:18] [webDriverOperations.py:438] [WARNING] Ignoring flat '3-ZIMMER WOHNUNG IN MITTE WBS 140' because the size doesn't match our criteria --> Flat Size: 0.0 m² | User wants it above: 40 m² 🙈
[29.04.2024 - 16:18] [webDriverOperations.py:438] [WARNING] Ignoring flat '2-ZIMMER-WOHNUNG IN MITTE WBS 160/180/220' because the size doesn't match our criteria --> Flat Size: 0.0 m² | User wants it above: 40 m² 🙈
[29.04.2024 - 16:18] [webDriverOperations.py:61] [ERROR] Failed to switch page, last page reached ❌
[29.04.2024 - 16:19] [webDriverOperations.py:504] [INFO] Reloading main page 🔄
[29.04.2024 - 16:19] [webDriverOperations.py:380] [INFO] Looking for flats 👀
[29.04.2024 - 16:19] [webDriverOperations.py:387] [SUCCESS] Found 2 flat(s) in total 💡
[29.04.2024 - 16:19] [webDriverOperations.py:438] [WARNING] Ignoring flat '3-ZIMMER WOHNUNG IN MITTE WBS 140' because the size doesn't match our criteria --> Flat Size: 0.0 m² | User wants it above: 40 m² 🙈
[29.04.2024 - 16:19] [webDriverOperations.py:438] [WARNING] Ignoring flat '2-ZIMMER-WOHNUNG IN MITTE WBS 160/180/220' because the size doesn't match our criteria --> Flat Size: 0.0 m² | User wants it above: 40 m² 🙈
[29.04.2024 - 16:20] [webDriverOperations.py:61] [ERROR] Failed to switch page, last page reached ❌
[29.04.2024 - 16:21] [webDriverOperations.py:504] [INFO] Reloading main page 🔄
[29.04.2024 - 16:21] [webDriverOperations.py:380] [INFO] Looking for flats 👀
[29.04.2024 - 16:21] [webDriverOperations.py:387] [SUCCESS] Found 2 flat(s) in total 💡
[29.04.2024 - 16:21] [webDriverOperations.py:438] [WARNING] Ignoring flat '3-ZIMMER WOHNUNG IN MITTE WBS 140' because the size doesn't match our criteria --> Flat Size: 0.0 m² | User wants it above: 40 m² 🙈
[29.04.2024 - 16:21] [webDriverOperations.py:438] [WARNING] Ignoring flat '2-ZIMMER-WOHNUNG IN MITTE WBS 160/180/220' because the size doesn't match our criteria --> Flat Size: 0.0 m² | User wants it above: 40 m² 🙈
[29.04.2024 - 16:21] [webDriverOperations.py:61] [ERROR] Failed to switch page, last page reached ❌
[29.04.2024 - 16:22] [webDriverOperations.py:504] [INFO] Reloading main page 🔄
[29.04.2024 - 16:22] [webDriverOperations.py:380] [INFO] Looking for flats 👀
[29.04.2024 - 16:22] [webDriverOperations.py:387] [SUCCESS] Found 2 flat(s) in total 💡
[29.04.2024 - 16:22] [webDriverOperations.py:438] [WARNING] Ignoring flat '3-ZIMMER WOHNUNG IN MITTE WBS 140' because the size doesn't match our criteria --> Flat Size: 0.0 m² | User wants it above: 40 m² 🙈
[29.04.2024 - 16:22] [webDriverOperations.py:438] [WARNING] Ignoring flat '2-ZIMMER-WOHNUNG IN MITTE WBS 160/180/220' because the size doesn't match our criteria --> Flat Size: 0.0 m² | User wants it above: 40 m² 🙈
[29.04.2024 - 16:22] [webDriverOperations.py:61] [ERROR] Failed to switch page, last page reached ❌
^CTraceback (most recent call last):
File "/Users/UsmanTatla/Desktop/wbmbot_v2-main/wbmbot_v2/main.py", line 134, in
main()
File "/Users/UsmanTatla/Desktop/wbmbot_v2-main/wbmbot_v2/main.py", line 111, in main
webDriverOperations.process_flats(
File "/Users/UsmanTatla/Desktop/wbmbot_v2-main/wbmbot_v2/helpers/webDriverOperations.py", line 500, in process_flats
time.sleep(int(refresh_internal) * 60)
KeyboardInterrupt

UsmanTatla@MacBookPro wbmbot_v2-main %

Describe the bug
I've encountered an issue today where the bot has stopped identifying available flats on wbm.de, despite there being 5 flats currently listed (as per the archived page). This morning, the bot was functioning correctly; it detected a WBS flat (which was excluded based on my configuration) and attempted to apply for a non-WBS flat but was unsuccessful due to an issue. Following a fix issued by @Vel-San, I updated the Docker image, and since then, the bot has not detected any available flats.

To Reproduce

Expected behavior
The bot should consistently detect available flats on wbm.de, including after updates and fixes are applied.

Screenshots

Environment
Digest: sha256:a989a4dfe0a7817060d906620329f294f641ecaa8006970f94b60edf7b84745a

Additional context
Not a coding expert, please let me know if there's any additional information I can share to help solving this.