Code Monkey home page Code Monkey logo

Comments (5)

wkeeling avatar wkeeling commented on May 20, 2024 1

Actually, driver.get() may hand control back before the URL has been captured, so that print statement may display an empty list.

I'll investigate what's going on with wait_for_request().

from selenium-wire.

wkeeling avatar wkeeling commented on May 20, 2024

@jedie - thanks for raising this.

From your example code above, it doesn't look as though you made a call to driver.get('https://www.google.com').

Have you tried:

...

driver = webdriver.Chrome(...)

driver.get('https://www.google.com')  # Trigger the request

print("Request google.com...", flush=True)
driver.wait_for_request("https://www.google.com", timeout=3)  # Wait for it
print("OK")

The wait_for_request() method will wait for a request made by get() but it won't make any request itself.

from selenium-wire.

aflansburg avatar aflansburg commented on May 20, 2024

I'm experiencing the same issue and have tried using driver.get() before driver.wait_for_request() - great concept though - definitely what I'm looking for:

import logging
import os

logging.basicConfig(format='%(message)s', level=logging.DEBUG)

chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--headless")


def get_page_response(url):

    # check for current os
    if os.name == 'posix':
        # osx
        driver_path = '/usr/local/bin/chromedriver'
    elif os.name == 'nt':
        # win32
        driver_path = 'C:\chromedriver\chromedriver'
    else:
        print('Unknown operating system!!!')
        exit()

    driver = webdriver.Chrome(
        chrome_options=chrome_options,
        executable_path=driver_path
    )
    driver.get(url)
    request = driver.wait_for_request(url, timeout=3)
    print(request)


get_page_response("https://www.google.com")

output:

http://seleniumwire/find?path=https%3A%2F%2Fwww.google.com 200
http://seleniumwire/find?path=https%3A%2F%2Fwww.google.com 200
http://seleniumwire/find?path=https%3A%2F%2Fwww.google.com 200
http://seleniumwire/find?path=https%3A%2F%2Fwww.google.com 200
http://seleniumwire/find?path=https%3A%2F%2Fwww.google.com 200
http://seleniumwire/find?path=https%3A%2F%2Fwww.google.com 200
http://seleniumwire/find?path=https%3A%2F%2Fwww.google.com 200
http://seleniumwire/find?path=https%3A%2F%2Fwww.google.com 200
http://seleniumwire/find?path=https%3A%2F%2Fwww.google.com 200
http://seleniumwire/find?path=https%3A%2F%2Fwww.google.com 200
http://seleniumwire/find?path=https%3A%2F%2Fwww.google.com 200
http://seleniumwire/find?path=https%3A%2F%2Fwww.google.com 200
http://seleniumwire/find?path=https%3A%2F%2Fwww.google.com 200
http://seleniumwire/find?path=https%3A%2F%2Fwww.google.com 200
http://seleniumwire/find?path=https%3A%2F%2Fwww.google.com 200

selenium.common.exceptions.TimeoutException: Message: Timed out after 3s waiting for request https://www.google.com```

from selenium-wire.

wkeeling avatar wkeeling commented on May 20, 2024

@aflansburg Thanks. Your example code ought to work - I'll see if I can reproduce.

Out of curiosity, are you able to add print(driver.requests) before the wait_for_request() - so:

driver.get(url)
print(driver.requests)  # Should contain https://www.google.com
request = driver.wait_for_request(url, timeout=3)
print(request)

The output of print(driver.requests) should contain https://www.google.com if things are working correctly.

from selenium-wire.

wkeeling avatar wkeeling commented on May 20, 2024

@aflansburg Running your example code seems to work as expected for me. I see:

Created proxy listening on 127.0.0.1:44549
POST http://127.0.0.1:59249/session {"capabilities": {"firstMatch": [{}], "alwaysMatch": {"browserName": "chrome", "platformName": "any", "proxy": {"proxyType": "manual", "httpProxy": "127.0.0.1:44549", "sslProxy": "127.0.0.1:44549", "noProxy": ""}, "acceptInsecureCerts": true, "goog:chromeOptions": {"extensions": [], "args": ["--headless"]}}}, "desiredCapabilities": {"browserName": "chrome", "version": "", "platform": "ANY", "proxy": {"proxyType": "manual", "httpProxy": "127.0.0.1:44549", "sslProxy": "127.0.0.1:44549", "noProxy": ""}, "acceptInsecureCerts": true, "goog:chromeOptions": {"extensions": [], "args": ["--headless"]}}}
Finished Request
POST http://127.0.0.1:59249/session/fc0fff6090494290619eb54382023180/url {"url": "https://www.google.com", "sessionId": "fc0fff6090494290619eb54382023180"}
www.google.com:443 200
www.google.com:443 200
Capturing request: https://www.google.com/
Capturing response: https://www.google.com/ 200 OK

... lots more "Capturing request"s ...

https://www.google.com/
http://seleniumwire/find?path=https%3A%2F%2Fwww.google.com 200

I'm using the latest version of Selenium Wire with Selenium 3.8.0, Chrome 67 on Linux. I'll check macOS later.

What version of Selenium are you using?

from selenium-wire.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.