Comments (5)
Actually, driver.get()
may hand control back before the URL has been captured, so that print statement may display an empty list.
I'll investigate what's going on with wait_for_request()
.
from selenium-wire.
@jedie - thanks for raising this.
From your example code above, it doesn't look as though you made a call to driver.get('https://www.google.com')
.
Have you tried:
...
driver = webdriver.Chrome(...)
driver.get('https://www.google.com') # Trigger the request
print("Request google.com...", flush=True)
driver.wait_for_request("https://www.google.com", timeout=3) # Wait for it
print("OK")
The wait_for_request()
method will wait for a request made by get()
but it won't make any request itself.
from selenium-wire.
I'm experiencing the same issue and have tried using driver.get() before driver.wait_for_request() - great concept though - definitely what I'm looking for:
import logging
import os
logging.basicConfig(format='%(message)s', level=logging.DEBUG)
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--headless")
def get_page_response(url):
# check for current os
if os.name == 'posix':
# osx
driver_path = '/usr/local/bin/chromedriver'
elif os.name == 'nt':
# win32
driver_path = 'C:\chromedriver\chromedriver'
else:
print('Unknown operating system!!!')
exit()
driver = webdriver.Chrome(
chrome_options=chrome_options,
executable_path=driver_path
)
driver.get(url)
request = driver.wait_for_request(url, timeout=3)
print(request)
get_page_response("https://www.google.com")
output:
http://seleniumwire/find?path=https%3A%2F%2Fwww.google.com 200
http://seleniumwire/find?path=https%3A%2F%2Fwww.google.com 200
http://seleniumwire/find?path=https%3A%2F%2Fwww.google.com 200
http://seleniumwire/find?path=https%3A%2F%2Fwww.google.com 200
http://seleniumwire/find?path=https%3A%2F%2Fwww.google.com 200
http://seleniumwire/find?path=https%3A%2F%2Fwww.google.com 200
http://seleniumwire/find?path=https%3A%2F%2Fwww.google.com 200
http://seleniumwire/find?path=https%3A%2F%2Fwww.google.com 200
http://seleniumwire/find?path=https%3A%2F%2Fwww.google.com 200
http://seleniumwire/find?path=https%3A%2F%2Fwww.google.com 200
http://seleniumwire/find?path=https%3A%2F%2Fwww.google.com 200
http://seleniumwire/find?path=https%3A%2F%2Fwww.google.com 200
http://seleniumwire/find?path=https%3A%2F%2Fwww.google.com 200
http://seleniumwire/find?path=https%3A%2F%2Fwww.google.com 200
http://seleniumwire/find?path=https%3A%2F%2Fwww.google.com 200
selenium.common.exceptions.TimeoutException: Message: Timed out after 3s waiting for request https://www.google.com```
from selenium-wire.
@aflansburg Thanks. Your example code ought to work - I'll see if I can reproduce.
Out of curiosity, are you able to add print(driver.requests)
before the wait_for_request()
- so:
driver.get(url)
print(driver.requests) # Should contain https://www.google.com
request = driver.wait_for_request(url, timeout=3)
print(request)
The output of print(driver.requests)
should contain https://www.google.com if things are working correctly.
from selenium-wire.
@aflansburg Running your example code seems to work as expected for me. I see:
Created proxy listening on 127.0.0.1:44549
POST http://127.0.0.1:59249/session {"capabilities": {"firstMatch": [{}], "alwaysMatch": {"browserName": "chrome", "platformName": "any", "proxy": {"proxyType": "manual", "httpProxy": "127.0.0.1:44549", "sslProxy": "127.0.0.1:44549", "noProxy": ""}, "acceptInsecureCerts": true, "goog:chromeOptions": {"extensions": [], "args": ["--headless"]}}}, "desiredCapabilities": {"browserName": "chrome", "version": "", "platform": "ANY", "proxy": {"proxyType": "manual", "httpProxy": "127.0.0.1:44549", "sslProxy": "127.0.0.1:44549", "noProxy": ""}, "acceptInsecureCerts": true, "goog:chromeOptions": {"extensions": [], "args": ["--headless"]}}}
Finished Request
POST http://127.0.0.1:59249/session/fc0fff6090494290619eb54382023180/url {"url": "https://www.google.com", "sessionId": "fc0fff6090494290619eb54382023180"}
www.google.com:443 200
www.google.com:443 200
Capturing request: https://www.google.com/
Capturing response: https://www.google.com/ 200 OK
... lots more "Capturing request"s ...
https://www.google.com/
http://seleniumwire/find?path=https%3A%2F%2Fwww.google.com 200
I'm using the latest version of Selenium Wire with Selenium 3.8.0, Chrome 67 on Linux. I'll check macOS later.
What version of Selenium are you using?
from selenium-wire.
Related Issues (20)
- Websites with a domain in Cyrillic do not open HOT 3
- Don't load JS content on page HOT 1
- Valid proxies with credentials not working
- selenium-wire with proxy timing-out inside AWS Lambda
- Selenium Wire Remote instantiates with incorrect arguments HOT 2
- Full Traceback Displayed Reporting [WinError 10053] An established connection was aborted by the software in your host machine HOT 1
- ERR_PROXY_CONNECTION_FAILED while running selenium wire on Github Actions HOT 1
- Reduce HAR file size
- PFX/PKCS certificate not working
- How to use selenium-wire with dolphin anty webbrowser? + how to use proxy with interceptor? HOT 2
- Submit POST/PATCH/PUT instead of GET request HOT 1
- driver.requests is empty list with MSEdgedriver
- WebDriverException: Message: unknown error: session deleted because of page crash HOT 1
- Some error when use debuggerAddress on selenium-wire and how to fix it
- Problem to run code from terminal
- SeleniumWire Unable to Intercept Network with Samsung S23 android user-agent
- Error handling request in HAR with large timestamp in cookie
- Unable to get content of page from special characters (utf-8) URL HOT 1
- Dynamic Proxy Changing is not working (?) with Flag "--ignore-certificate-errors"
- Add `py.typed` marker
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from selenium-wire.