victorperez / spidr Goto Github PK
View Code? Open in Web Editor NEWThis project forked from postmodern/spidr
A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.
Home Page: http://spidr.rubyforge.org/
License: MIT License