I am trying to scrape the name, image link and the price of some products in amazon.
This is the code so far
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import NoSuchElementException
options = webdriver.ChromeOptions()
options.add_argument('--headless')
options.add_argument('--no-sandbox')
options.add_argument('--disable-dev-shm-usage')
wd = webdriver.Chrome('chromedriver',options=options)
wd.get('https://www.amazon.fr/dp/000101742X')
product_title = wd.find_element(By.CLASS_NAME, 'a-size-extra-large')
print(product_title.text)
product_image_url = wd.find_element(By.ID, 'imgBlkFront')
print(product_image_url.get_attribute('src'))
WebDriverWait(wd, 20).until(EC.element_to_be_clickable((By.XPATH, "//input[@name='accept']"))).click()
product_price = WebDriverWait(wd, 20).until(EC.visibility_of_element_located((By.XPATH, "//span[text()='Partition']//following::span[1]"))).text.split(" ")[3].replace(',', '.')
print(product_price)
wd.quit()
This is working for the link in the code, but when I try to scrape another link, https://www.amazon.fr/dp/000101742X, It is giving me a TimeoutException.
These two websites have the same elements.
Is there any ways to get around this? Thanks.
