Selenium snippets
Useful pieces of code for using Selenium and dealing with common errors
Importing
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import Select
from selenium.webdriver.support.ui import WebDriverWait
Starting up a browser
# If you want to open Chrome
driver = webdriver.Chrome()
# If you want to open Firefox
driver = webdriver.Firefox()
But Python can’t find chromedriver
driver_path = '/Users/yourname/Desktop/foundations/chromedriver'
driver = webdriver.Chrome(executable_path=driver_path)
But Python can’t find Chrome/Firefox
options = webdriver.ChromeOptions()
options.binary_location = "C:/Program Files (x86)/Google/Chrome/Application/chrome.exe"
driver = webdriver.Chrome(chrome_options=options, executable_path="C:/Utility/BrowserDrivers/chromedriver.exe")
Visiting a page
driver.get('http://www.nytimes.com')
Typing in a form
text_input = driver.find_element_by_id('name_input')
text_input.send_keys('Katherine')
Fill out a dropdown
from selenium.webdriver.support.ui import Select
select = Select(driver.find_element_by_name('cityname'))
select.select_by_visible_text('Houston')
Clicking buttons or links
search_button = driver.find_element_by_id('sch_button')
search_button.click()
Scrolling (if you get an error something’s not in view, ElementNotVisibleException
)
button = driver.find_element_by_class_name('load-more-btn')
driver.execute_script("arguments[0].scrollIntoView(true)", button)
button.click()
Trying to get something that might not exist
try:
search_button = driver.find_element_by_id('sch_button')
search_button.click()
except:
print("It didn't work")
Getting text and attributes
# Get the text of an element
element.text
# Get the href of a link
element.get_attribute('href')
# Get the HTML inside
element.get_attribute('innerHTML')