What is the easiest way to compare two web pages using python?

Question

Hello I want to Compare two webpages using python script. how can i achieve it? thanks in advance!

what do you want to compare? Do you just want to know whether they are exactly the same? Or whether they look the same? — Sören, Mar 08 '11 at 16:50

score 4 · Accepted Answer · edited May 23 '17 at 11:55

First, you want to retrieve both webpages. You can use wget, urlretrieve, etc.:
wget Vs urlretrieve of python

Second, you want to "compare" the pages. You can use a "diff" tool as Chinmay noted. You can also do a keyword analysis of the two pages:

Parse all keywords from page. e.g. How do I extract keywords used in text?
Optionally take the "stem" of the words with something like:
http://pypi.python.org/pypi/stemming/1.0
Use some math to compare the two pages' keywords, e.g. term frequency–inverse document frequency: http://en.wikipedia.org/wiki/Tf%E2%80%93idf with some of the python tools out there like these: http://wiki.python.org/moin/InformationRetrieval

Thanks for the detailed comment !! this helped a lot !! – Arackna Mar 08 '11 at 17:14 — Arackna, Mar 08 '11 at 17:14

score 2 · Answer 2 · answered Mar 08 '11 at 16:50

2

What do you mean by compare? If you just want to find the differences between two files, try difflib, which is part of the standard Python library.

answered Mar 08 '11 at 16:50

Chinmay Kanchi

2 Answers2