Selecting button using Selenium in Python 2.7

Question

A site I'm scraping recently changed the button ID I was using. For some reason, I can't find the new element. I've read through multiple sites (including Stack Overflow) on selecting a button, nothing I try works. I'm pretty much a newbie at Selenium. Here's the HTML extract:

                <div class="info">
                <h4 class="store-number">
                    Store Number: {{=storeId}}
                </h4>
                {{ if (closeForEcommerce == 0 ) { }}
                <button id="store-search-modal-make-this-my-store-{{=storeId}}" class="btn btn-make-this-my-store btn-block btn-primary-2 {{ if (ResultDisplayHelper.isMyStore(storeId)) { print("hidden"); } }}"
                        onclick="ResultDisplayHelper.setMyStoreMarker({{=storeId}});ResultDisplayHelper.setMyStore('store-search-modal-abc-store-card-info-', 'store-search-modal-make-this-my-store-', 'store-search-modal-my-store-', {{=storeId}})">
                    Make This My Store
                </button>
                {{ } }}

                {{ if (closeForEcommerce != 0 ) { }}
                <button id="btnStoreCloseForEcommerce" class="btn btn-store-temporarily-closed btn-block btn-primary-2 {{ if (ResultDisplayHelper.isMyStore(storeId)) { print("hidden"); } }}"
                        onclick="">
                    Store Temporarily Closed
                </button>
                {{ } }}

                <a id="store-search-modal-my-store-{{=storeId}}" href="{{=clickUri}}" class="CoveoResultLink my-store btn btn-gray-300 btn-block {{ if (!ResultDisplayHelper.isMyStore(storeId)) { print("hidden"); } }}">
                    My Store
                </a>
                <a class="CoveoResultLink" href="{{=clickUri}}">Visit Store Page</a>
                <div class="location">
                    {{ if (dist != null) { }}
                    <div><strong>Miles</strong>: {{=ResultDisplayHelper.metersToMiles(dist)}}</div>
                    {{ } }}
                    <address>
                        {{ if (shoppingcenter) { }}
                        {{=shoppingcenter}}<br/>
                        {{ } }}
                        {{=address1}}
                        {{ if (address2) { }}<br />{{=address2}} {{ } }}
                        <br />
                        {{=city}}, {{=state}} {{=zip}}
                    </address>
                </div>

I've tried

button_id = 'store-search-modal-make-this-my-store-'+shop
make_my_store = driver.find_element_by_id(button_id)

and

make_my_store = driver.find_element_by_xpath("//button[contains(text(),'Make 
This My Store')]")

with the results:

NoSuchElementException: no such element: Unable to locate element: {"method":"id","selector":"store-search-modal-make-this-my-store-231"}
  (Session info: headless chrome=67.0.3396.99)
  (Driver info: chromedriver=2.33.506120 (e3e53437346286c0bc2d2dc9aa4915ba81d9023f),platform=Windows NT 10.0.17134 x86_64)

and

    NoSuchElementException: no such element: Unable to locate element: {"method":"css selector","selector":"button[onclick^=ResultDisplayHelper]"}
  (Session info: headless chrome=67.0.3396.99)
  (Driver info: chromedriver=2.33.506120 (e3e53437346286c0bc2d2dc9aa4915ba81d9023f),platform=Windows NT 10.0.17134 x86_64)

What am I missing?

UPDATE: Thanks for the suggestions so far. Unfortunately when I tried the multiple variations, I keep getting timeout errors as the object isn't found. I grabbed driver.page the source and see:

    <button id="make-this-my-store" class="btn btn-block btn-primary-2" 
ng-show="model.store.storeId !== model.abcCartService.cart.pickupStore.storeId &amp;&amp; 
model.store.closeForEcommerce !== 'True'" ng-click="model.makeMyStore();">
        Make This My Store
</button>

I've tried looking for "Make This My Store" using XPATH, using "make-this-my-store" as the button ID, and "btn btn-block btn-primary-2" as the CSS class. All give object not found errors.

are u sure that you are using the correct xpath and that the element is visible on the UI when you try to find it? have a look here https://stackoverflow.com/questions/21350605/python-selenium-click-on-button/37279279#37279279 — Carlo 1585, Jul 19 '18 at 15:55
I see two buttons `make this my store` and `store temporarily closed` , which one you want to select and click ? — cruisepandey, Jul 19 '18 at 15:58
Look at the actual HTML being rendered and presented to the browser, rather than the code used to create the HTML. — pbuck, Jul 19 '18 at 15:59
The element is visible and I can click on it when I do it manually. I'm looking for "Make This My Store." The HTML I presented is from "show source" when I manually go to the site in Chrome, should I be doing something else to see it? Should I try print(driver.page_source) as I see referenced on the link Carlo 1585 provided? — ViennaMike, Jul 19 '18 at 16:44
@ViennaMike : You can inspect the element by simply right click on a web page and then click on inspect, and then under element tab you will get it. I have provided the answer.Please have a look. — cruisepandey, Jul 20 '18 at 06:15

cruisepandey · Accepted Answer · 2018-07-19T16:21:10.990

You can try with this xpath and can use explicit wait:

For clicking on Make This My Store button :

wait = WebDriverWait(driver,20)  
make_this_my_store = wait.until(EC.element_to_be_clickable((By.XPATH, '//button[contains(text(),'Make This My Store')]')))
make_this_my_store.click()

For clicking on Store Temporarily Closed button :

store_temporarily_closed= wait.until(EC.element_to_be_clickable((By.XPATH, '//button[contains(text(),'Store Temporarily Closed')]')))  
store_temporarily_closed.click()

Make sure to import these :

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

Explanation :

An explicit wait is code you define to wait for a certain condition to occur before proceeding further in the code. The worst case of this is Thread.sleep(), which sets the condition to an exact time period to wait. There are some convenience methods provided that help you write code that will wait only as long as required. WebDriverWait in combination with ExpectedCondition is one way this can be accomplished.

More about explicit wait, can be found here

As you have mentioned that

//button[contains(text(),'Make This My Store')]

is not working.

In case, if you would like to use css selector :

That would be :

h4.store-number+button[class*='btn btn-make-this-my-store btn-block btn-primary-2'][id*='store-search-modal-make-this-my-store']

In code something like : (for clicking on Make this my store)

make_this_my_store = wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, 'h4.store-number+button[class*='btn btn-make-this-my-store btn-block btn-primary-2'][id*='store-search-modal-make-this-my-store']')))
make_this_my_store.click()

For clicking on Store Temporarily Closed button :

store_temporarily_closed = wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, 'button[class*='btn btn-store-temporarily-closed btn-block btn-primary-2'][id='btnStoreCloseForEcommerce']')))
store_temporarily_closed.click()

Note that it is always good to have css selector as compared to xpath.
For more about xpath vs css selector can be found here

Hope this information will be helpful.Thanks !

Thanks for the extensive answer. I can't say why, but these still kept getting "no such element" if done without the waits, and timeout errors with the waits (which makes sense if it never could find the elements). I'm still baffled by why none of these searches worked, but I finally got it to work after more examination of the html and using: items = driver.find_elements_by_id('make-this-my-store') make_my_store = items[0] make_my_store.click() — ViennaMike, Jul 20 '18 at 17:20
@ViennaMike : It is very simple if these line of code work for you : ` `driver.find_elements_by_id('make-this-my-store') make_my_store = items[0] make_my_store.click()` . Explanation : all the web elements with this id `make-this-my-store` will be stored in a list and by using item[0] , you are retrieving first element of list and then clicking on it. It's quite obvious now that there are multiple elements are present with that id. When you inspect the elements in dev tool, for this id `make-this-my-store` how many entries are present? — cruisepandey, Jul 21 '18 at 07:07
Moreover you can try something like this also : `driver.find_element_by_xpath("//*[@id='make-this-my-store'][1]").click()` — cruisepandey, Jul 21 '18 at 07:10
cruisepandy: Yes, I kno know that that code does. There are two elements in the list. That's the strange thing. That code seems to work, but just directly selecting it doesn't seem to. Since it's the FIRST element, I should be able to find it without the list. But it works, so... — ViennaMike, Jul 22 '18 at 21:45
you will have to change the locator. That the only thing we can do. — cruisepandey, Jul 23 '18 at 09:35

Oleksandr Makarenko · Answer 2 · 2018-07-19T16:06:08.960

Are you using waits in your tests? Your variant of xpath must work.

Try with explicit wait:

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

wait = WebDriverWait(driver, 5)
make_my_store = wait.until(EC.presence_of_element_located((By.XPATH, "//button[contains(text(),'Make This My Store')]")))
make_my_store.click()

This waits up to 5 seconds before throwing a TimeoutException unless it finds the element to return within 5 seconds. WebDriverWait by default calls the ExpectedCondition every 500 milliseconds until it returns successfully. A successful return is for ExpectedCondition type is Boolean return true or not null return value for all other ExpectedCondition types.

score 0 · Answer 3 · answered Jul 19 '18 at 16:08

Try using the following xpath:

button = browser.find_element_by_xpath("//button[@id='store-search-modal-make-this-my-store-{{=storeId}}']")
button.click()

Now, if that does not work, there is a possibility that the driver needs to wait for a few seconds before looking for the element. In this case, I recommend using explicit waits. More on explicit waits can be found here = http://selenium-python.readthedocs.io/waits.html.

You can include an explicit wait as follows:

    xpath = "//button[@id='store-search-modal-make-this-my-store-{{=storeId}}']"
   button = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, xpath))).click()

This means that the driver will wait upto 10 seconds for the element to appear and be clickable. If the driver cannot find the element, it will throw a TimeoutException. You can use try/except blocks to deal with the TimeOutException. For example:

try:
   xpath = "//button[@id='store-search-modal-make-this-my-store-{{=storeId}}']"
    button = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, xpath))).click() 
except TimeoutException:
   print("Button not found!")

You will need the following imports:

import selenium.webdriver as webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import TimeoutException

Hope this helps!

score 0 · Answer 4 · answered Jul 19 '18 at 16:15

As per the HTML you have shared the element with text as Make This My Store is an Angular element, so to invoke click() on the desired element you have to induce WebDriverWait for the element to be clickable and you can use the following solution:

WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button.btn.btn-make-this-my-store.btn-block.btn-primary-2[id^=store-search-modal-make-this-my-store-]"))).click()

Note : You have to add the following imports :

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

Selecting button using Selenium in Python 2.7

4 Answers4