python selenium - click button and get page_source

Question

I currently click a button into a new window:

browser.execute_script("arguments[0].value = 'test';", browser.find_element_by_xpath('//*[@id="email-signup-email-address"]'))
input_button = browser.find_element_by_xpath('//*[@id="submittracking"]')
browser.execute_script('arguments[0].target="_blank";', input_button.find_element_by_xpath('./ancestor::form'))
browser.execute_script('arguments[0].click();', input_button)

but must time.sleep(10) before asking browser.page_source otherwise the page does not load fully

Other articles/posts suggest to use seleniums explicit wait but this asks to provide an element to wait for such as visibilityOfAllElements() or some specific element. I would like to get the ALL the page_source and am not looking to identify any elements.

How can I make browser.page_source work as it normally does (when not clicking into a new window)?

you can wait until state is completed, see this https://stackoverflow.com/questions/5868439/wait-for-page-load-in-selenium — murali selenium, Dec 18 '18 at 08:28
Do you mean, readyState? I may look into this further. Curious if it has a timeout built in. Also the link is mostly in Java — Rhys, Dec 18 '18 at 08:39

score 1 · Accepted Answer · answered Dec 18 '18 at 09:02

1

Try to wait for page loaded (document.readyState == 'complete') as below:

from selenium.webdriver.support.ui import WebDriverWait

WebDriverWait(browser, 20).until(lambda browser: browser.execute_script("return document.readyState;") == "complete")

answered Dec 18 '18 at 09:02

Andersson

51,635
17
77
129

score 0 · Answer 2 · edited Dec 18 '18 at 09:06

0

You can use some timeout for the page load like below:

driver.set_page_load_timeout(30)

edited Dec 18 '18 at 09:06

undetected Selenium

183,867
41
278
352

answered Dec 18 '18 at 08:33

Kaushik

140
11

python selenium - click button and get page_source

2 Answers2