Comments (6)
Great idea! I wasn't aware of difflib and that it's even part of the standard library. I'll adopt it as soon as possible.
This project started out as a very baisc, hacky script and the "change detection" logic wasn't changed since then. Now it's time to become a little more mature.
from website-watcher.
I'll look into it again these days!
from website-watcher.
Kudos for the quick implementation, there is a bug though. doc1 and doc2 are different, although the HTML source code didn't change. For example umlauts are treated differently, so are CRLF: here's the debug view. Don't know why atm, would have to look deeper but I would say f.read().encode("utf-8")
and r.text.encode("utf-8")
return different strings.
Okay found it, you will get the correct encoding when you open the file with utf-8 encoding like this, indead of reading with utf-8:
with open(tmp_location, 'r', encoding='utf8') as f:
doc1 = filter_document(get_nodes(args.xpath, f.read()))
Now there is still the problem with the CRLF. Okay solved the problem with the CRLF, you need to open it with newline='':
with open(tmp_location, 'r', encoding='utf8', newline='') as f:
Will send a push request soon.
from website-watcher.
Unfortunately, there's still some issue with the encoding. The content read from the file differs from the one it gets via requests. Not sure where and why, though.
from website-watcher.
Please try again.
from website-watcher.
Perfect, that was the problem. Thanks!
from website-watcher.
Related Issues (17)
- SSL Doesn't support HOT 1
- Stdout adapter
- Subprocess adapter
- Add typings
- Include cookies when following redirects HOT 1
- Add example how to watch ebay-kleinanzeigen.de HOT 1
- Add possibility to monitor a list of URLs instead of single URL HOT 4
- Allow to set custom user agent
- Add possibility to exclude HTML tags from beeing monitored HOT 4
- Predicting Tolerance HOT 1
- Bug in batch.sh HOT 7
- `ntfy.sh` adapter
- XPath queries to only include certain parts of a document HOT 9
- Use argparse HOT 3
- WebSub adapter HOT 1
- Python3.8 breaking HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from website-watcher.