loic-binet / crawlerx Goto Github PK
View Code? Open in Web Editor NEWThis project forked from leopardslab/crawlerx
CrawlerX - Develop Extensible, Distributed, Scalable Crawler System which is a web platform that can be used to crawl URLs in different kind of protocols in a distributed way.
License: Apache License 2.0