Goal Of the task
The task requires me to scrape the data from this site. If you will scroll down you will see a graph that displays data only when the mouse hovers over it. Here is a screenshot. I want to get the data displayed using selenium. One obvious choice is using ActionChains, but I can't use ActionChains in this case, cause It doesn't work when the browser is minimized. I want a solution which uses Javascript or any other means to somehow fetch the data and not use ActionChains.
Some Insights and Progress made by me
The site is sending a POST request to its GraphQl server and getting a response. I am not much familiar with JS or with React. But doing some research I found out that the site uses Recharts library to display the data. I also found out this react script named PriceHistoryGraph.react.tsx in the source. I can see that It detects the mouse hover using OnMouseMove and OnMouseLeave events. I have also looked at a lot of StackOverflow posts but none seems to help, for example this. I then tried to look for some way to get the data where the chrome has stored it as the script uses these imports, but I can't find any way or see if it is possible. If I were able to simulate the mouse movements as required by recharts so that the bubble appears then I can use XPath to extract the data. Here are the imports the script is using to work on the data.
import {
PriceHistoryGraph_data,
PriceHistoryGraph_data$key,
} from "lib/graphql/__generated__/PriceHistoryGraph_data.graphql"
Final Questions
- Is there a way to simulate the "mouse move" behavior using Javascript to trigger the bubble that appears on the graph, then I can just use XPath and get the data?
- Is there a workaround way to extract the data easily? (Can't use selenium wire to intercept the GraphQL requests as I am using a modified Selenium web driver, Github here)
