-2

The title says it all. Please give me some way for how to scrape AJAX loaded Division.

There is one website which has Product Grid Division. I want this division for scrap the data from it but the problem is when i get the page content data in another html page there is no product grid because it's loaded via AJAX after some time. For Scrapping i tried both Perl and CURL with PHP.

Thanks in advance.

simbabque
  • 52,309
  • 8
  • 74
  • 127
Sahil Patel
  • 1,491
  • 14
  • 31
  • What page (url) would you like to scrape? – PHPhil Jul 31 '15 at 10:44
  • PHPhil sorry i can't declare url – Sahil Patel Jul 31 '15 at 11:27
  • @SinanÜnür Thanks for advice but you don't know about me and you have to respect each and every programmer even if he/she fresher. I just want to know how to scrape the Division which is load with AJAX. I can't get that division id when i print that page content. I am scrapping data from last 4 months so i have enough skill to do this. If you don't know the answer then please don't comment. – Sahil Patel Jul 31 '15 at 12:03

2 Answers2

1

From WWW::Mechanize::FAQ

Which modules work like Mechanize and have JavaScript support?

In no particular order: Gtk2::WebKit::Mechanize, Win32::IE::Mechanize, WWW::Mechanize::Firefox, WWW::Scripter, WWW::Selenium

Also see: How do you scrape AJAX pages?

Community
  • 1
  • 1
Chankey Pathak
  • 20,204
  • 11
  • 77
  • 126
1

Using Selenium, e.g. through Selenium::Remote::Driver, you will be operating real browsers to access the site. Sites can be quite sensitive to subtle differences in browser behavior.

reinierpost
  • 7,950
  • 1
  • 36
  • 70