No Way to Specify Custom 404 about zaproxy HOT 17 CLOSED

The user guide will have to be updated to document the new functionality, added a task.

from zaproxy.

Completely agree with this - its been on my todo for a while:)
In the next release the brute forcing of files and directories will be handled by components
from DirBuster, which handles custom 404s much better.
However I think a more generic solution is also required so that custom error pages
are clearly flagged in the History tab.

Original issue reported on code.google.com by psiinon on 2010-10-06 17:45:02

from zaproxy.

If we added handling for definition of custom errors wouldn't every rule have to be
modified to account for such handling?

Original issue reported on code.google.com by kingthorin on 2014-07-06 01:22:10

from zaproxy.

Yeah, it would be very useful though.
Rules would be able to use SomeClass.is404(url) or similar.
They wouldnt _have_ to, but they would work better if they did...
Note that the custom 404s should be associated with contexts ;)

Original issue reported on code.google.com by psiinon on 2014-07-07 08:52:46

from zaproxy.

there is ... ask me ! 

Original issue reported on code.google.com by anandkhare78 on 2015-02-01 08:23:43

from zaproxy.

News to me.
Care to elaborate?

Original issue reported on code.google.com by psiinon on 2015-02-02 08:30:20

from zaproxy.

https://groups.google.com/d/msg/zaproxy-develop/41FgrOoggAw/HgrnP67rYgEJ

"It's something that would have big impact (from my perspective anyway) and could be
implemented in phases/waves/

P1
Implement options/configuration pane or dialog.
Implement a isError(response) that checks if one (or more) of the configured items
flag the response as an error. (This needs to be useable by both active and passive
scanners.)
Based on simple string matching within responses.
Implement isError(response) checks in 1 or 2 active and passive rules that other devs
can follow as a template/guide. (Perhaps the Example rules too.)
P2
Extend to handle matching URL patterns (absolute or relative)
P3
Extend to handle regex matching
P4
Implement functionality to update (remove) existing alerts that are affected by "new"
isError(response) definitions (so that if a user defines a new error pattern after
having scanned the alert tree is update, removing false positives)
Pn
Implement isError(response) in all relevant active and passive rules.

Why I think this is a great idea: I recently had a ton of false positives related to
Backup File Disclosure because the server I was testing actually replied to requests
that were not found with 302 to /FileNotFound.aspx?aspxerrorpath=/page_copy.aspx if
I could have defined this URL or some of the page content as an error page then these
false positives probably wouldn't have been identified."

Original issue reported on code.google.com by kingthorin on 2015-02-12 13:16:11

from zaproxy.

(No text was entered with this change)

Original issue reported on code.google.com by kingthorin on 2015-02-12 16:50:20

• Labels added: GSoC-Candidate

from zaproxy.

Dear all, I suffered the same trouble yesterday in a pentest when a nice programmer decided to use 200 and a page sying "this not exists" instead of 404 error... all the forced, svn, git, backup file plugins went crazy and the db became so big that everything has crashed!!!!
I don't know if someone is working on this, and I want to give my 2 cents.
Probably a good approach is using context, people could define custom 404 and 500 or other error messages using http response codes and regexes as you said before, then crawlers, active and passive plugins should use specific methods to check for a specific Response code (I don't agree for the isError, I think we can be more generic with isResponseCode(int Response code) then the implementation should apply the specific custom control or apply it to the Response Code directly).
But I think that a big improvement could be also to alert the user that a custom 404 exists. I saw that lot of scanners perform a check for an unexisting file and then, if the answer is 200, they alert the user that a custom error exists and the scanner hasn't been configured for this.
Sincerely I don't know how musch complex could be this implementation because I have any experience on ZAP contexts and extensions that can deal with them, but if this is simple to address I can give an hand...

from zaproxy.

I'd suggested isError because the solution shouldn't be limited to response codes. Since the issue is often that response codes are't always representative of content. Perhaps isCustomError or isCustomErrorResponse would be more indicative of the actual goal. I've been thinking about this lots lately and can see good arguments for both context and global implementation.

Trying to recognize and suggest setting custom error definitions might be harder. But definitely adds a user friendliness factor.

from zaproxy.

Well, check it could be simple. We can send some probes for unexisting pages and if the response is 200 probably a custom 404 is in place. We can arise a dialog for the user to alert about the situation and he can abort and configure custom pages.
For the others I agree that it's impossible, but this is also less important. The real trouble are custom 404

from zaproxy.

Good point, trying to detect 404s would probably be a 80% solution, perhaps even more.

from zaproxy.

Active scanner is already doing that (or trying to), with Analyser class. [1]
It's also exposed to scanners through AbstractPlugin.isFileExist(HttpMessage) (which seems to be the opposite of isError). [2]

[1] https://github.com/zaproxy/zaproxy/blob/55202a1d5becf6260ecb25cd3ebc49e271d477e8/src/org/parosproxy/paros/core/scanner/Analyser.java
[2]

from zaproxy.

We should document this much better - maybe an 'advanced a/pscan rules' guide?

from zaproxy.

Well I did a look to the Analyzer, but it seems to check if a page exists or not checking also for a "not found" pattern inside the content... so it seems more suitable to active plugins that currently work on the HTTP Status code... but I don't think that this soultion is good enought because it's not localized and don't take in care all the possibilities... I think that give to the user a custom pattern is the best idea.
Then we can force 404 status for pages that give 200 but are custom pages, or use the same method you described before and change all the active plugins and the crawlers to do this.
Because the real trouble is related also to crawlers that maybe are following not existing links...

My idea is instead to perform an initial probe before crawling to check for unexisting page (I se that Acunetix works in this way), then according to the result (HTTP OK for an unexisting page and maybe a pattern check in the body) alert the user that a custom 404 is probably in place (with evidence) and give to him the possibility to choice what to do next.
Then give to the user the possibility to go directly to the context panel, define a regex for the custom 404 (the default one could be the Analyzer existing pattern) and then we can approach control in one of the two models I described before (force status code or change all the plugins)....

from zaproxy.

I think changing the status code is a neat idea but I could see it shooting us in the foot later. We should minimize our impact on the traffic where/whenever possible. While it will be more work I think adapting all the scanners (as applicable) to leverage a check for custom errors is the superior solution.

As one example (and I'm sure there are others), some of the passive scanners check for client error (400 thru 499) and behave differently at certain attack thresholds. So if the app actually sends a 200 and then we alter it to 404, then analyze that response we might end up in a false negative or false positive condition.

from zaproxy.

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

from zaproxy.