I’ve used like this. You can also use if you like this procedure.
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions
from selenium.webdriver.common.by import By
driver = webdriver.Chrome(PATH)
css_selector = "div.abcd"
results = WebDriverWait(driver, 10).until((expected_conditions.presence_of_all_elements_located((By.CSS_SELECTOR, css_selector))))
for result in results:
print(result.text)
I want to retrieve from a website every div class=’abcd’ element using Selenium together with ‘waiter’ and ‘XPATH’ classes from Explicit.
The source code is something like this:
<div class='abcd'>
<a> Something </a>
</div>
<div class='abcd'>
<a> Something else </a>
...
When I run the following code (Python) I get only ‘Something’ as a result. I’d like to iterate over every instance of the div class=’abcd’ appearing in the source code of the website.
from explicit import waiter, XPATH
from selenium import webdriver
driver = webdriver.Chrome(PATH)
result = waiter.find_element(driver, "//div[@class='abcd']/a", by=XPATH).text
Sorry if the explanation isn’t too technical, I’m only starting with webscraping. Thanks
Thanks! It worked. So, in the “div.something” nomenclature, “something” is always the class? Is there any way to target another attribute of the HTML block? (like name, type, etc.)
Sure!. You can also use (id, name, xpath, link_text, partial_link_text, tag_name, class_name and css_selector) according to their official documentation. You can check here.