lgautier / mashing-pumpkins Goto Github PK

View Code? Open in Web Editor NEW

19.0 19.0 3.0 1.41 MB

Minhash and maxhash library in Python, combining flexibility, expressivity, and performance.

License: MIT License

Python 35.74% C++ 8.57% C 55.69%

dice-similarity jaccard-similarity locality-sensitive-hashing minhash minhash-sketches murmurhash3 python xxhash

mashing-pumpkins's People

Contributors

Stargazers

Watchers

Forkers

pombredanne dfischer ersaree

mashing-pumpkins's Issues

Make default seed value 42 ?

Since this is what it in sourmash and MASH.

Fix ngrams at junction of batches between `hashfun` calls.

As the title says.

Current implementation is computing MaxHash sets

Since the general idea appears to be working and have a competitive runtime, implementing efficient MinHash sets would be needed.

Add Dockerfile w/ command line

Assess whether the container for dnasnout-client can be a good base
Implement the derived container (if the above is working)
Writing instructions to use as an in-container command line
Put benchmark code in a gist

Ensure it builds on OS X

I currently do not have dev environment on OS X for mashing-pumpkins.

Reports on whether it is building, or not building, would be greatly appreciated.

Codecov stopped reporting coverage

The coverage is visible in the Travis output, and this used to work but suddenly stopped...

Error pickling XXHash hashing function

>>> import mashingpumpkins._xxhash
>>> x = pickle.dumps(mashingpumpkins._xxhash.hasharray)
PicklingError: Can't pickle <built-in function hasharray>: import of module '_xxhash3' failed

Move tests to directory / submodule `tests/`

Default seed not programmatically accessible

Expose the default seed in the C extensions (murmurhash3 and xxhash)

Implement JSON format discussed in MASH

While the format is still discussed (see marbl/Mash#44), the design should already make the implementation of JSON signature both easy and sufficiently performant.

Implement in-place add operator `iadd` as an alias to existing method `update`.

Additional example / notebook showing how easily alternative minhash approaches can be compared

Showing how to write the code leading to the figure in comment for marbl/Mash#45 would make a cute notebook to showcase what the library's design can do.