If you are looking to scrape or take information from a website look into puppeteer. It is a library designed for exactly that
Solution 1 :
Problem :
On yahoo finance there is an option to download the historical data for any given Ticker symbol. However I am most interested in Yahoo Finance Premium’s ‘Visitor Trends’ data. This ‘Visitor Trends’ data is shown with an interactive chart which overlays the visitor trends over the stock’s rising and falling performance. The interactive chart reacts to the users cursor and makes it easy to see where spikes in a given stock’s page visits correlates to a change in its price.
I want to download the visitor trends data to build a dataset, but there is no way to direct download through yahoo finance.
I have been looking for ways to scrape the data I need by intercepting it before it is rendered by the chart. Has anyone dealt with a similar puzzle/hurdle? If so, what should my approach be?
Sorry if am not specific enough. I can get more in depth if necessary.