Boar should detect md5 hash collisions about boar HOT 8 OPEN

One possible security impact scenario is the following:

Alice and Bob shares a repository. Alice uploads evil.exe, a malicious file. 
Alice has used a vulnerability in the md5 checksum and designed the file so 
that its checksum is identical to the checksum of dostuff.exe, a well-known 
useful program.

Bob uploads his dostuff.exe. However, since it has the same checksum as the 
existing evil.exe, it is not actually uploaded. Boar notices (wrongly) that it 
already has this file in the repo and uses that copy instead. 

Some time later, Bob downloads his dostuff.exe file again. However, instead he 
receives the evil.exe that Alice uploaded earlier. When he executes the 
program, bad things will happen.

Fixed in changeset 5518090482c9. All files now have a corresponding sha256 
checksum that ensures that no collisions can go undetected. 

Reopening the issue. As it turns out, the implemented solution is too slow. A 
verification on a repository will take about twice as long with md5 collision 
detection enabled (due to the verification of the sha256 database). I had hoped 
to mitigate this slowdown by using python multiprocessing features, but while 
that works well on Linux, I have not succeeded in making it work on windows. 

Due to md5 collision detection being a somewhat niche feature, I'm going to 
disable that feature for the next release as to not make boar slower for the 
current boar user base. 

Issue 80 has been merged into this issue.

One way to handle this is to store the first 8 bytes of the file as well and 
check against that as well as the md5, this makes it nearly impossible to have 
a collision even on purpose.

In response to comment 5: Do you have a reference for your statement? I've 
always assumed that md5 simply is inherently unsafe. If that can be mitigated 
with a simple check of the first part of the contents, that would certainly 
make things easier. 

Astronomical or not -- I'm slightly paranoid about it. Does it have to be 
SHA256 to detect md5 collisions or might something fast like the SpookyHash 
SnapRaid uses be an option as well?

Or maybe this:
https://github.com/SaberParker/xxHash-Python
https://code.google.com/p/xxhash/