A distributed web crawler, using Redis for job and url store, Hbase for page store.
so-far-so-good / collie Goto Github PK
View Code? Open in Web Editor NEWThis project forked from mfan/collie
A distributed on demand web crawler, using Redis for job and url store, Hbase for page store.