Possibly collector adds artefactso to the end of URI about crawlers HOT 5 CLOSED

The error suggests that a URL existed with the double quote in it in the page where it was extracted. To find out whether it is the case or a bug with the URL Extractor logic, it would help if you could give the site URL, or maybe attach the page that contains the URL "http://www.somehost.com/hook/". If you can.

For the (optional) variables, they follow the Apache Velocity syntax so the use of ${braces} is recommended, but not always necessary. It is required when the variable names would blend with other characters if you do not have the ${variable} approach. ${like}this. To play it safe, you can always use the curly braces. No that the dollar sign goes outside the curly braces, not inside like your example.

from crawlers.

You may use for example http://www.site.com for the test crawl, getting such error messages:

INFO  [DefaultSitemapResolver] Resolving sitemap: http://www.site.com/sitemap.xml
INFO  [DefaultSitemapResolver]          Resolved: http://www.site.com/sitemap.xml
ERROR [URLProcessor] Could not process URL: http://www.site.com/"
com.norconex.commons.lang.url.URLException: Invalid URL syntax: http://www.site.com/"

The similar error I get in first minutes running the crawl for http://www.drupal.org

Search depth were set to 5.

from crawlers.

Hello Antonio,
It turns out as being a defect with URL Extractor. It was not handling empty href properly. Like from this code snippet found in your page:

<li><a href=""><a href="/imprint">Impress</a></a></li>

In addition, I noticed you have several URLs with spaces in them. Those were rejected too.

The good news is both these issues have been addressed in this new 1.3.2 snapshot release.

Please give it a try and let me know if it addresses your issues.

from crawlers.

The snapshot 1.3.2 works perfectly for me. It handles all cases disturbed me before.

Thanks a lot!

from crawlers.

Great! I'll release it soon.

from crawlers.