0

I want to iteratively search for 30+ items through a search button in webpage and scrape the related data.

My search items are stored in a list: vol_list

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
driver = webdriver.Chrome("driver path")
driver.get("web url")

for item in vol_list :
mc_search_box = driver.find_element_by_name("search_str")
mc_search_box.clear()
search_box.send_keys(item)
search_box.send_keys(Keys.RETURN)

After search is complete I will proceed to scrape the data for each item and store in array/list. Is it possible to repeat this process without opening browser for every item in the loop?

DisappointedByUnaccountableMod
  • 6,407
  • 4
  • 17
  • 22
Stelios
  • 119
  • 1
  • 11
  • 1
    See [this SO answer](http://stackoverflow.com/questions/7593611/selenium-testing-without-browser)-- especially the one by Stéphane Bruckert – erapert Feb 24 '17 at 18:59

1 Answers1

1

You can't use chrome and other browsers without opening it.

In your case, headless browsers should do the job. Headless browsers simulates browser, but doesn't have GUI.

Try ghost driver/ html unit driver/ NodeJS. Then you will have to modify at least this line with the driver you want to use: 

driver = webdriver.Chrome("driver path")

Good luck!

vilkg
  • 565
  • 4
  • 7