0

I have a page I am trying to load periodically with Selenium and ChromeDriver. I use the driver.get() function and the website serves up an error page. Clicking the refresh button causes the browser to receive the same error page. Eventually I let the script controlling Chrome crashes from a timeout error. At this point the instance of chrome is still open and using the refresh button loads the expected page with no error.

Does anyone have any idea what could be causing this? It's definitely not a timing issue. No matter how many waits or how much I make the script sleep, the issue is the same. The page can only be loaded once webdriver is no longer controlling the browser.

Any help would be greatly appreciated!

ua = UserAgent()
global userAgent
userAgent = ua.random
print("[" + time.strftime('%a %H:%M:%S') + "] " + "User Agent: " + userAgent)
sys.stdout.flush()

#Start a new browser
options = webdriver.ChromeOptions()
options.add_argument("--headless")
options.add_argument("--no-sandbox")
options.add_argument(f'user-agent={userAgent}')
options.add_argument("--disable-popup-blocking")
options.add_argument("--disable-setuid-sandbox")
options.add_argument('ignore-certificate-errors')
options.add_argument('allow-cross-origin-auth-prompt')
options.add_argument('allow-profiles-outside-user-dir')
options.add_argument('allow-running-insecure-content')
options.add_argument('allow-silent-push')
options.add_argument('--blink-settings=imagesEnabled=false') #Disable images
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('useAutomationExtension', False)
service = Service("/usr/bin/chromedriver")
driver = webdriver.Chrome(service=service, options=options)
driver.set_page_load_timeout(15)
driver.execute_script("Object.defineProperty(navigator, 'webdriver', {get: () => undefined})")
# Selenium Stealth settings
stealth(driver,
    languages=["en-US", "en"],
    vendor="Google Inc.",
    platform="Win32",
    webgl_vendor="Intel Inc.",
    renderer="Intel Iris OpenGL Engine",
    fix_hairline=True,
)

driver.get("https://example.com")
Dan S
  • 1
  • 1
  • Your website might be detecting bot. Using the User Agent may solve the issue. See this [Thread](https://stackoverflow.com/questions/29916054/change-user-agent-for-selenium-web-driver) – Swaroop Humane Mar 01 '22 at 17:30
  • What url is it? – Arundeep Chohan Mar 01 '22 at 17:30
  • Include some code in your post... especially driver initialization. – pcalkins Mar 01 '22 at 17:56
  • Thank you all for you responses. I have added some code. Hopefully that will help. I would prefer not to give out the exact URL. I should also mention a random user agent is used every time. – Dan S Mar 01 '22 at 19:25

0 Answers0