Document how -u can accept multiple urls about spidy HOT 3 OPEN

Hey, I believe it should be -u url1.com -u url2.com.
Thank you for offering help with documentation that is really appreciated.
I'm working to release a new update, (branch: csv_url_reader) this will be update it.

from spidy.

Alright, when I have a little time i'll check that -u url1.com -u url2.com works. & try to make the PR if it does.

I am also in the process of learning go (previous js only andy hahaha), so might be able to contribute further.

Some of the things I would be interested in adding are:

• I would love to split the writer/verifier & crawler in such a way that you can spawn as multiple of both separately.
• I am not sure why but after running the same crawl for around 2-3 days it fills 1GB of ram on my pc. I imagine you are storing either the domains or the urls crawled in memory. Not sure the best solution for this, I had thought of moving those things to a SQLite or something similar

But first I got to learn go & i'm very busy with school so i'll do what I can.

from spidy.

Finally had time to test this

The code you suggest throws an error -> -u url1.com -u url2.com

This code -u url1.com url2.com does not throw an error but does not seem to work, it only picks up the first url

This code works -> -u url1.com,url2.com

from spidy.