Solution 1 :

I’ve used like this. You can also use if you like this procedure.

from selenium import webdriver
from import WebDriverWait
from import expected_conditions
from import By

driver = webdriver.Chrome(PATH)

css_selector = "div.abcd"
results = WebDriverWait(driver, 10).until((expected_conditions.presence_of_all_elements_located((By.CSS_SELECTOR, css_selector))))

for result in results:

Problem :

I want to retrieve from a website every div class=’abcd’ element using Selenium together with ‘waiter’ and ‘XPATH’ classes from Explicit.
The source code is something like this:

<div class='abcd'>
   <a> Something </a>
<div class='abcd'>
   <a> Something else </a>

When I run the following code (Python) I get only ‘Something’ as a result. I’d like to iterate over every instance of the div class=’abcd’ appearing in the source code of the website.

from explicit import waiter, XPATH
from selenium import webdriver
driver = webdriver.Chrome(PATH)
result = waiter.find_element(driver, "//div[@class='abcd']/a", by=XPATH).text

Sorry if the explanation isn’t too technical, I’m only starting with webscraping. Thanks


Comment posted by Juan Topo

Thanks! It worked. So, in the “div.something” nomenclature, “something” is always the class? Is there any way to target another attribute of the HTML block? (like name, type, etc.)

Comment posted by

Sure!. You can also use (id, name, xpath, link_text, partial_link_text, tag_name, class_name and css_selector) according to their official documentation. You can check here.