The scraping website got stuck in the pulling elements part. I was coding in VSC and this is working just fine in there. Now I'm trying to convert into Colab, but somehow it returns "TimeoutException: Message: "
The only different part between VSC and Colab is the first part. Which I copied from this thread.
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--headless')
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--disable-dev-shm-usage')
driver = webdriver.Chrome('chromedriver',chrome_options=chrome_options)
Code
website = 'https://www.bitkub.com/fee/cryptocurrency'
driver.get(website)
#coin_name
coin_name = [my_elem.text for my_elem in WebDriverWait(driver, 10).until(EC.visibility_of_all_elements_located((By.XPATH, "//tbody//tr//td[2]//span")))]
coin_name_res = [coin_name[i] for i in range(len(coin_name)) if i % 2 == 0]
#chain name
chain_name = [my_elem.text for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//tbody//tr//td[3]//div")))]
#fee
coin_icon_list = [my_elem.text for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//tbody//tr//td[2]//span//span")))]
coin_icon = ''.join(coin_icon_list)
withdrawal_fees = [my_elem.text for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//tbody//tr//td[4]//div")))]
fee_res = ([s.strip(coin_icon) for s in withdrawal_fees])
Error message
---------------------------------------------------------------------------
TimeoutException Traceback (most recent call last)
<ipython-input-32-b7dc6f81c817> in <module>()
1
2 #coin name
----> 3 coin_name = [my_elem.text for my_elem in WebDriverWait(driver, 10).until(EC.visibility_of_all_elements_located((By.XPATH, "//tbody//tr//td[2]//span")))]
4 coin_name_res = [coin_name[i] for i in range(len(coin_name)) if i % 2 == 0]
5 print (coin_name_res)
/usr/local/lib/python3.7/dist-packages/selenium/webdriver/support/wait.py in until(self, method, message)
87 if time.time() > end_time:
88 break
---> 89 raise TimeoutException(message, screen, stacktrace)
90
91 def until_not(self, method, message=''):
TimeoutException: Message:
I found out some other questions that got an error message like this too. but doesn't seem to be in case like me. So the solution won't fit.