How to find position of an element using pyautogui and selenium automatically

Question

I'm writing an script and I need to find cursor position automatically for an element. The script will run on different computers, so I want it be done automatically.

Suppose I want to click on search box in the head of stackoverflow.com to find its position using selenium and pyautogui.

How to do that so that mouse is in that position and script is exited?

Edit 1

Suppose I want it to click on this element:

input which name="q" and class="s-input s-input__search js-search-field "

Edit 2:

My current code with the help of Prophet and undetected Selenium, but the output I get is not correct:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.action_chains import ActionChains
import pyautogui
from time import sleep
from datetime import datetime
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as ec
from selenium.webdriver.support.wait import WebDriverWait

path = r'path'
user_data = r'user'
options = webdriver.ChromeOptions()
options.add_argument(f'--user-data-dir={user_data}')
options.add_argument('--profile-directory=Default')
driver = webdriver.Chrome(executable_path=path, options=options)
actions = ActionChains(driver)
xpath = '//*[@id="search"]/div/input'

url = 'https://stackoverflow.com/'
driver.get(url)
position = WebDriverWait(driver, 20).until(ec.visibility_of_element_located((By.XPATH, xpath))).location_once_scrolled_into_view
# both tried the above position var and the below
# position = WebDriverWait(driver, 20).until(ec.visibility_of_element_located((By.XPATH, xpath))).location
# element = driver.find_element(By.XPATH, xpath)
# actions.move_to_element(element).perform()
# position = pyautogui.position()
file = open('position.txt', 'a')
file.write(f'{position}, {datetime.now()}\n')
driver.close()

# to test the exact or relative position to compare with above code
# while True:
#     sleep(1)
#     print(pyautogui.position())

Edit 3:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.action_chains import ActionChains
import pyautogui
from time import sleep
from datetime import datetime
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.wait import WebDriverWait

path = r'D:\\Downloads\\chromedriver_win32\\chromedriver.exe'
user_data = r'C:\\Users\\Saeed\\AppData\\Local\\Google\\Chrome\\User Data\\Default'
options = webdriver.ChromeOptions()
options.add_argument(f'--user-data-dir={user_data}')
options.add_argument('--profile-directory=Default')
driver = webdriver.Chrome(executable_path=path, options=options)
actions = ActionChains(driver)
xpath = '//*[@id="search"]/div/input'  # `search` box at header
url = 'https://stackoverflow.com//'
driver.get(url)
driver.maximize_window()
sleep(10)
position = WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, xpath))).location
print(position)
file = open('position.txt', 'a')
file.write(f'{position}, {datetime.now()}\n')
# pyautogui.click(793, 9)
driver.close()

# while True:
#     sleep(1)
#     print(pyautogui.position())

The video how my code now works: https://up.mazandhost.com/uploads/1028407950.bandicam-2022-03-27-21-33-27-729.mp4

Solution in my case:

Answer https://stackoverflow.com/a/56693949/5790653 helps me.

Answer 1

With Selenium you can perform mouse actions with the use of action_chains library.
Once you locate the element with selenium you can move the mouse to that element as following:

from selenium.webdriver.common.action_chains import ActionChains

actions = ActionChains(driver)

element = driver.find_element(By.XPATH, 'the_xpath_locator')
actions.move_to_element(element).perform()

Answer 2

To extract the location and/or position an element you need to induce WebDriverWait for the visibility_of_element_located() and you can use either of the following locator strategies:

• Using location attribute:

  driver.get('https://www.google.com/')
  print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.NAME, "q"))).location)
  

• Console Output:

  {'x': 439, 'y': 184}
  

• Using location_once_scrolled_into_view attribute:

  driver.get('https://www.google.com/')
  print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.NAME, "q"))).location_once_scrolled_into_view)
  

• Console Output:

  {'x': 439, 'y': 184}
  

• Note : You have to add the following imports :

  from selenium.webdriver.support.ui import WebDriverWait
  from selenium.webdriver.common.by import By
  from selenium.webdriver.support import expected_conditions as EC
  

---

This usecase

As per the HTML details:

input which name="q" and class="s-input s-input__search js-search-field "

To click on the clickable element you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following locator strategies:

• Using CSS_SELECTOR:

  WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input.s-input.s-input__search.js-search-field[name='q']"))).click()
  

• Using XPATH:

  WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//input[@class='s-input s-input__search js-search-field ' and @name='q']"))).click()
  

---

Alternative

As an alternative you can also use the move_to_element() method from ActionChains API as follows:

ActionChains(driver).move_to_element(WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//input[@class='s-input s-input__search js-search-field ' and @name='q']")))).perform()