waitForNavigation doesn't work after clicking a link about pyppeteer HOT 12 CLOSED

Your solution is correct.

The problem of your first code is:

await page.goto("https://github.com")
await page.click('.HeaderMenu [href="/features"]')  # start navigation to the feature page here
await page.waitForNavigation()  # waiting for the "next" navigation from the feature page

To avoid this situation, you need to start click and waitForNavigation (almost) simultaneously, as your solution.

I will update document about page.click() by the next release.

from pyppeteer.

You need to wait for page contents loaded.
Use waitForSelector method before click.

print(await result_page.title())
await result_page.waitForSelector('a.ng-binding')
await result_page.click('a.ng-binding')

from pyppeteer.

Seems to have found a way to achieve, but I do not understand, do not know whether the correct, I hope to get your guidance, thank you
solution:

await asyncio.wait([page.click('.HeaderMenu [href="/features"]'), page.waitForNavigation()])

from pyppeteer.

sorry guys,i found the solution still not work in my case. any idea for this?

async def main():
browser = await launch({"headless":False})
page = await browser.newPage()
await page.goto('http://wsjs.saic.gov.cn')
await page.screenshot({'path': 'example.png'})
await page.click('#txnS02',{"waitUntil":"networkidle0"})
title = await page.title()
if title is None or title =="":
await page.reload({"waitUntil":"networkidle0"})
print (await page.title() + str(len(await browser.pages())))
await page.screenshot({'path': 'example1.png'})
await page.focus("input[name='request:sn']")
await page.type("input[name='request:sn']","26716759",{"delay": 50})
await page.screenshot({'path': 'example2.png'})
await asyncio.wait([page.click('#_searchButton'), page.waitForNavigation()])
print (await page.title())
# await browser.close()
asyncio.get_event_loop().run_until_complete(main())

from pyppeteer.

Since page.click does not accept waitUntil option, this line is incorrect.

await page.click('#txnS02',{"waitUntil":"networkidle0"})

So you need to use await asyncio.wait([page.click(...), page.waitForNavigation()]) here, too.

from pyppeteer.

Updated document of Page.click()

from pyppeteer.

actually it's working for me at that line ,but still got the Navigation Timeout Exceeded: 30000 ms exceeded at line await asyncio.wait([page.click('#_searchButton'), page.waitForNavigation()])

from pyppeteer.

Thank you for details.
I reproduced the error.
I will try to fix it.

from pyppeteer.

I understand the problem.
Since page.click('#_searchButton') opens a new tab, navigation event is fired in the different page instance.
So you need to watch targetcreated event on the browser.

Try the below code:

import asyncio
from pyppeteer import launch

async def main() -> None:
    browser = await launch({"headless": False, "args": ['--no-sandbox']})
    page = await browser.newPage()
    await page.goto('http://wsjs.saic.gov.cn')
    await page.screenshot({'path': 'example.png'})
    await asyncio.wait([page.waitForNavigation(), page.click('#txnS02')])

    title = await page.title()
    if title is None or title == "":
        await page.reload({"waitUntil": "networkidle0"})

    print(await page.title() + str(len(await browser.pages())))
    await page.screenshot({'path': 'example1.png'})
    await page.focus("input[name='request:sn']")
    await page.type("input[name='request:sn']", "26716759", {"delay": 50})
    await page.screenshot({'path': 'example2.png'})

    result_page_future = asyncio.get_event_loop().create_future()
    browser.once('targetcreated',
                 lambda target: result_page_future.set_result(target))
    await page.click('#_searchButton'),
    result_page = await (await result_page_future).page()
    print(await result_page.title())
    # await browser.close()

asyncio.get_event_loop().run_until_complete(main())

output is:

商标综合检索2
商标检索结果

from pyppeteer.

hi bro, but seems i'm not able to click the link inside the table. Means if this page is dynamic loading how can i access the content?Thanks

import asyncio
from pyppeteer import launch

async def main() -> None:
browser = await launch({"headless": False, "args": ['--no-sandbox']})
page = await browser.newPage()
await page.goto('http://wsjs.saic.gov.cn')
await page.screenshot({'path': 'example.png'})
await asyncio.wait([page.waitForNavigation(), page.click('#txnS02')])

title = await page.title()
if title is None or title == "":
    await page.reload({"waitUntil": "networkidle0"})

print(await page.title() + str(len(await browser.pages())))
await page.screenshot({'path': 'example1.png'})
await page.focus("input[name='request:sn']")
await page.type("input[name='request:sn']", "26716759", {"delay": 50})


result_page_future = asyncio.get_event_loop().create_future()
browser.once('targetcreated',
             lambda target: result_page_future.set_result(target))
await page.click('#_searchButton')
result_page = await (await result_page_future).page()

print(await result_page.title())
await result_page.click("a.ng-binding")  #it does not work

await browser.close()

asyncio.get_event_loop().run_until_complete(main())

from pyppeteer.

Thank u bro!!

from pyppeteer.

Hi @miyakogi , could you give this post https://stackoverflow.com/questions/60763644/cant-go-on-clicking-on-the-next-page-button-while-scraping-certain-fields-from a shot in case there is any solution you would like to offer. The post that I created in there is about pyppeteer. Thanks.

from pyppeteer.