I am trying to extract information such as prices and vendors from Amazon.
The way I do this now is to find the key words such as price and then to find the information I want.
The problem is if the websites such as Amazon changes their frame a little bit, the code might not work anymore.
I am wondering if there is some better way / algorithm doing similar things.
Thanks!
You want to access data from a website. What you suggested is a handcrafted API, or Application Programming Interface.
One of the main flaws of a handcrafted API is exactly what you mentioned, or that the supplier of the webpage could make a small change that would render your API unusable.
Generally, it is a better idea to use an API that has direct access to the data that belongs to the owners of the website. These APIs are created by the website owners themselves, so they have straight access to the data, and they get around all of the messy formatting that comes in between you and the data that you want when you use HTML scraping.
Specifically, Amazon's price API is located here.
IMPORTANT:
As mentioned here, please read Section 4b of the Licensing Agreement:
(b) You will use Product Advertising Content only (i) in a lawful manner; (ii) in accordance with the terms of this License Agreement and within the express scope of the license granted in Section 6; and (iii) to send end users to and drive sales on the Amazon Site. You will not use the Product Advertising API, Data Feed, or Product Advertising Content with any site or application, or in any other manner, that does not have the principal purpose of advertising and marketing the Amazon Site and driving sales of products and services on the Amazon Site.
