Use difflib instead of length to detect changes about website-watcher HOT 6 CLOSED

Great idea! I wasn't aware of difflib and that it's even part of the standard library. I'll adopt it as soon as possible.

This project started out as a very baisc, hacky script and the "change detection" logic wasn't changed since then. Now it's time to become a little more mature.

from website-watcher.

I'll look into it again these days!

from website-watcher.

Kudos for the quick implementation, there is a bug though. doc1 and doc2 are different, although the HTML source code didn't change. For example umlauts are treated differently, so are CRLF: here's the debug view. Don't know why atm, would have to look deeper but I would say f.read().encode("utf-8") and r.text.encode("utf-8") return different strings.

Okay found it, you will get the correct encoding when you open the file with utf-8 encoding like this, indead of reading with utf-8:

with open(tmp_location, 'r', encoding='utf8') as f:
doc1 = filter_document(get_nodes(args.xpath, f.read()))

Now there is still the problem with the CRLF. Okay solved the problem with the CRLF, you need to open it with newline='':

with open(tmp_location, 'r', encoding='utf8', newline='') as f:

Will send a push request soon.

from website-watcher.

Unfortunately, there's still some issue with the encoding. The content read from the file differs from the one it gets via requests. Not sure where and why, though.

from website-watcher.

Please try again.

from website-watcher.

Perfect, that was the problem. Thanks!

from website-watcher.