0

I am new to RSelenium, and I am trying to iterate web search where I filter through some boxscores by date on ESPN's website. The code technically works correctly, but I have an issue where occasionally the loading of the page takes forever, meaning that the rest of the code cannot finish after running "navigate" (see code below for an example).

Manually, if I simply refresh the page in the browser when this page load error occurs, the code continues to run without any problems (until it hits a new link that takes too long to load). Obviously, I would prefer to automate this, as the slow page load occurs about once every ten page loads. So in the code below, I tried to use withTimeout so that the code will continue running and try the page load again automatically after 10 seconds (so that the rest of the code can run without interruption).

Whenever the withTimeout runs above the time limit, I get the following errors:

 "Error in .Call(R_curl_fetch_memory, enc2utf8(url), handle, nonblocking) : 
      reached elapsed time limit " and "Error in checkError(res) : 
      Undefined error in httr call. httr output: Operation was aborted by an application callback". 

I'm not sure what is going on, or how I can work the code so that it simply loads the page, and refreshes if the page load takes longer than 10s to load (I am iterating through thousands of days so just want to leave it running). The while loop is simply meant to refresh if there is a "page not found" error. This part works as intended.

Thanks!

library(xml2)
    library(rvest)
    library(jsonlite)
    library(RSelenium)
    library(R.utils)
    rD <- rsDriver(browser="firefox", port=4551L, verbose=F)
    remDr <- rD[["client"]]
    url_date <- "https://www.espn.com/nba/scoreboard/_/date/20031019"
    withTimeout({remDr$navigate(url_date); cat(month, day_str, "try: 0", "\n")},
                            timeout = 10, onTimeout="silent")
    while (remDr$getTitle()[[1]][1]!="NBA Basketball Scores - NBA Scoreboard - ESPN")
          {
         
            withTimeout({remDr$refresh(); cat(month, day_str, "try: ", counter, "\n")},
                             timeout = 10, onTimeout = "silent")
            
          }
Nad Pat
  • 2,995
  • 3
  • 9
  • 19
  • see if this helps https://stackoverflow.com/questions/43402237/r-waiting-for-page-to-load-in-rselenium-with-phantomjs – Nad Pat Oct 25 '21 at 04:43

0 Answers0