spaolacci / murmur3 Goto Github PK

View Code? Open in Web Editor NEW

934.0 934.0 126.0 31 KB

Native MurmurHash3 Go implementation

License: BSD 3-Clause "New" or "Revised" License

Go 100.00%

murmur3's People

Contributors

Stargazers

Watchers

Forkers

venliong pedronis bububa alex-golang wheelcomplex pombredanne catinred2 kalafut navneetk mengzhuo runeaune seago jonnyhopper huydx guojun1992 jeddenlea dynomitedb pfransenadb samir80 chennqqi busystone yannstad sabey insionng mbyczkowski zjtheone bigbaser ravenhurst jzhongming splitio surechembl m3db mw-jn kangkona salixleaf aweffr dono cespare luciham guchangyuan 2892931976 zhanglei etsangsplk iwoylxb mahall tyktechnologies maciej akmsg mistshi smira wojnosystems kuangxc isgasho qinzhao168 wlwgcdxc ghp268 hellastorm zxfonline 2haodb pubgo dramirez-qb arvancs bdfnl pdalinis funny-falcon wangjunemse abeusher buger lobicode ismiyati yl365 banana513a zhuxinquan andypeng2015 wuhua988 rlds calmh geoah segmentio imgoogege iyefeng jksroot kelvinwang000 lonisy junxie6 sreekanth370 sfstoneforest nicksherron kiyonlin hsclangzi luckymeef xiaobinqt 0xcarlrock qhchen96 daniel-007 sefxl why2go zsyaoo francoisjacques kratos-coder-hash

murmur3's Issues

Output not compatible with other implementations

Because the seed is reset to 1 instead of 0, the output doesn't match other implementations of murmur3 hash :(

Hash implementations don't implement hash.Hash interface

The way that the hash functions are implemented - containing the hash.Hash instance, and the constructor, doesn't allow for them to be used in place of standard hash.Hash implementations.

I.e. if a program is designed to have swappable hash implementations, they will commonly take as a parameter of type (hashFunc func() hash.Hash)

However, since the objects return their own types - Hash32, Hash64 or Hash128, the New functions do not have a matching signature, and cannot be easily used in place of the other hash functions.

Tests fail on big-endian arches

When building on s390x, tests fail:

--- FAIL: TestRefStrings (0.00s)
    murmur_test.go:41: [Hash32] key: 'hello', seed: '0': 0x84c85fd5 (want 0x248bfa47)
    murmur_test.go:48: [Hash32] key: 'hello', seed: '0': 84c85fd5 (want 248bfa47)
    murmur_test.go:52: [Hash32] key 'hello', seed: '0': 0x84c85fd5 (want 0x248bfa47)
    murmur_test.go:41: [Hash32] key: 'hello, world', seed: '0': 0xa98a45f8 (want 0x149bbb7f)
    murmur_test.go:48: [Hash32] key: 'hello, world', seed: '0': a98a45f8 (want 149bbb7f)
    murmur_test.go:52: [Hash32] key 'hello, world', seed: '0': 0xa98a45f8 (want 0x149bbb7f)
    murmur_test.go:41: [Hash32] key: '19 Jan 2038 at 3:14:07 AM', seed: '0': 0xa74dd6ac (want 0xe31e8a70)
    murmur_test.go:48: [Hash32] key: '19 Jan 2038 at 3:14:07 AM', seed: '0': a74dd6ac (want e31e8a70)
    murmur_test.go:52: [Hash32] key '19 Jan 2038 at 3:14:07 AM', seed: '0': 0xa74dd6ac (want 0xe31e8a70)
    murmur_test.go:58: '[Hash64] key: '19 Jan 2038 at 3:14:07 AM', seed: '0': 0xcef175caccc2d982 (want 0xb89e5988b737affc)
    murmur_test.go:65: [Hash64] key: '19 Jan 2038 at 3:14:07 AM', seed: '0': cef175caccc2d982 (want b89e5988b737affc)
    murmur_test.go:69: [Hash64] key: '19 Jan 2038 at 3:14:07 AM', seed: '0': 0xcef175caccc2d982 (want 0xb89e5988b737affc)
    murmur_test.go:76: [Hash128] key: '19 Jan 2038 at 3:14:07 AM', seed: '0': 0xcef175caccc2d982-0x43fd5acb8ce000a8 (want 0xb89e5988b737affc-0x664fc2950231b2cb)
    murmur_test.go:83: [Hash128] key: '19 Jan 2038 at 3:14:07 AM', seed: '0': cef175caccc2d98243fd5acb8ce000a8 (want b89e5988b737affc664fc2950231b2cb)
    murmur_test.go:87: [Hash128] key: '19 Jan 2038 at 3:14:07 AM', seed: '0': 0xcef175caccc2d982-0x43fd5acb8ce000a8 (want 0xb89e5988b737affc-0x664fc2950231b2cb)
    murmur_test.go:41: [Hash32] key: 'The quick brown fox jumps over the lazy dog.', seed: '0': 0x4f6a2b71 (want 0xd5c48bfc)
    murmur_test.go:48: [Hash32] key: 'The quick brown fox jumps over the lazy dog.', seed: '0': 4f6a2b71 (want d5c48bfc)
    murmur_test.go:52: [Hash32] key 'The quick brown fox jumps over the lazy dog.', seed: '0': 0x4f6a2b71 (want 0xd5c48bfc)
    murmur_test.go:58: '[Hash64] key: 'The quick brown fox jumps over the lazy dog.', seed: '0': 0xa9dcb521e865357b (want 0xcd99481f9ee902c9)
    murmur_test.go:65: [Hash64] key: 'The quick brown fox jumps over the lazy dog.', seed: '0': a9dcb521e865357b (want cd99481f9ee902c9)
    murmur_test.go:69: [Hash64] key: 'The quick brown fox jumps over the lazy dog.', seed: '0': 0xa9dcb521e865357b (want 0xcd99481f9ee902c9)
    murmur_test.go:76: [Hash128] key: 'The quick brown fox jumps over the lazy dog.', seed: '0': 0xa9dcb521e865357b-0x31929daa7f2d439e (want 0xcd99481f9ee902c9-0x695da1a38987b6e7)
    murmur_test.go:83: [Hash128] key: 'The quick brown fox jumps over the lazy dog.', seed: '0': a9dcb521e865357b31929daa7f2d439e (want cd99481f9ee902c9695da1a38987b6e7)
    murmur_test.go:87: [Hash128] key: 'The quick brown fox jumps over the lazy dog.', seed: '0': 0xa9dcb521e865357b-0x31929daa7f2d439e (want 0xcd99481f9ee902c9-0x695da1a38987b6e7)
    murmur_test.go:41: [Hash32] key: 'hello', seed: '1': 0x3abb1755 (want 0xbb4abcad)
    murmur_test.go:48: [Hash32] key: 'hello', seed: '1': 3abb1755 (want bb4abcad)
    murmur_test.go:52: [Hash32] key 'hello', seed: '1': 0x3abb1755 (want 0xbb4abcad)
    murmur_test.go:41: [Hash32] key: 'hello, world', seed: '1': 0x603764bb (want 0x6f5cb2e9)
    murmur_test.go:48: [Hash32] key: 'hello, world', seed: '1': 603764bb (want 6f5cb2e9)
    murmur_test.go:52: [Hash32] key 'hello, world', seed: '1': 0x603764bb (want 0x6f5cb2e9)
    murmur_test.go:41: [Hash32] key: '19 Jan 2038 at 3:14:07 AM', seed: '1': 0x4a7fcb64 (want 0xf50e1f30)
    murmur_test.go:48: [Hash32] key: '19 Jan 2038 at 3:14:07 AM', seed: '1': 4a7fcb64 (want f50e1f30)
    murmur_test.go:52: [Hash32] key '19 Jan 2038 at 3:14:07 AM', seed: '1': 0x4a7fcb64 (want 0xf50e1f30)
    murmur_test.go:58: '[Hash64] key: '19 Jan 2038 at 3:14:07 AM', seed: '1': 0x9c2486a8436dc6d4 (want 0x2a929de9c8f97b2f)
    murmur_test.go:65: [Hash64] key: '19 Jan 2038 at 3:14:07 AM', seed: '1': 9c2486a8436dc6d4 (want 2a929de9c8f97b2f)
    murmur_test.go:69: [Hash64] key: '19 Jan 2038 at 3:14:07 AM', seed: '1': 0x9c2486a8436dc6d4 (want 0x2a929de9c8f97b2f)
    murmur_test.go:76: [Hash128] key: '19 Jan 2038 at 3:14:07 AM', seed: '1': 0x9c2486a8436dc6d4-0x8b246db6d41ca4cd (want 0x2a929de9c8f97b2f-0x56a41d99af43a2db)
    murmur_test.go:83: [Hash128] key: '19 Jan 2038 at 3:14:07 AM', seed: '1': 9c2486a8436dc6d48b246db6d41ca4cd (want 2a929de9c8f97b2f56a41d99af43a2db)
    murmur_test.go:87: [Hash128] key: '19 Jan 2038 at 3:14:07 AM', seed: '1': 0x9c2486a8436dc6d4-0x8b246db6d41ca4cd (want 0x2a929de9c8f97b2f-0x56a41d99af43a2db)
    murmur_test.go:41: [Hash32] key: 'The quick brown fox jumps over the lazy dog.', seed: '1': 0x30ce84fd (want 0x846f6a36)
    murmur_test.go:48: [Hash32] key: 'The quick brown fox jumps over the lazy dog.', seed: '1': 30ce84fd (want 846f6a36)
    murmur_test.go:52: [Hash32] key 'The quick brown fox jumps over the lazy dog.', seed: '1': 0x30ce84fd (want 0x846f6a36)
    murmur_test.go:58: '[Hash64] key: 'The quick brown fox jumps over the lazy dog.', seed: '1': 0x628443d6482f2887 (want 0xfb3325171f9744da)
    murmur_test.go:65: [Hash64] key: 'The quick brown fox jumps over the lazy dog.', seed: '1': 628443d6482f2887 (want fb3325171f9744da)
    murmur_test.go:69: [Hash64] key: 'The quick brown fox jumps over the lazy dog.', seed: '1': 0x628443d6482f2887 (want 0xfb3325171f9744da)
    murmur_test.go:76: [Hash128] key: 'The quick brown fox jumps over the lazy dog.', seed: '1': 0x628443d6482f2887-0x3dde79f9da7a3d93 (want 0xfb3325171f9744da-0xaaf8b92a5f722952)
    murmur_test.go:83: [Hash128] key: 'The quick brown fox jumps over the lazy dog.', seed: '1': 628443d6482f28873dde79f9da7a3d93 (want fb3325171f9744daaaf8b92a5f722952)
    murmur_test.go:87: [Hash128] key: 'The quick brown fox jumps over the lazy dog.', seed: '1': 0x628443d6482f2887-0x3dde79f9da7a3d93 (want 0xfb3325171f9744da-0xaaf8b92a5f722952)
    murmur_test.go:41: [Hash32] key: 'hello', seed: '42': 0xb4fc956f (want 0xe2dbd2e1)
    murmur_test.go:48: [Hash32] key: 'hello', seed: '42': b4fc956f (want e2dbd2e1)
    murmur_test.go:52: [Hash32] key 'hello', seed: '42': 0xb4fc956f (want 0xe2dbd2e1)
    murmur_test.go:41: [Hash32] key: 'hello, world', seed: '42': 0x8cdde10 (want 0x7ec7c6c2)
    murmur_test.go:48: [Hash32] key: 'hello, world', seed: '42': 08cdde10 (want 7ec7c6c2)
    murmur_test.go:52: [Hash32] key 'hello, world', seed: '42': 0x8cdde10 (want 0x7ec7c6c2)
    murmur_test.go:41: [Hash32] key: '19 Jan 2038 at 3:14:07 AM', seed: '42': 0x4e1b5ffd (want 0x58f745f6)
    murmur_test.go:48: [Hash32] key: '19 Jan 2038 at 3:14:07 AM', seed: '42': 4e1b5ffd (want 58f745f6)
    murmur_test.go:52: [Hash32] key '19 Jan 2038 at 3:14:07 AM', seed: '42': 0x4e1b5ffd (want 0x58f745f6)
    murmur_test.go:58: '[Hash64] key: '19 Jan 2038 at 3:14:07 AM', seed: '42': 0x74da3b584f708419 (want 0xfd8f19ebdc8c6b6a)
    murmur_test.go:65: [Hash64] key: '19 Jan 2038 at 3:14:07 AM', seed: '42': 74da3b584f708419 (want fd8f19ebdc8c6b6a)
    murmur_test.go:69: [Hash64] key: '19 Jan 2038 at 3:14:07 AM', seed: '42': 0x74da3b584f708419 (want 0xfd8f19ebdc8c6b6a)
    murmur_test.go:76: [Hash128] key: '19 Jan 2038 at 3:14:07 AM', seed: '42': 0x74da3b584f708419-0x70bc8f344c720268 (want 0xfd8f19ebdc8c6b6a-0xd30fdc310fa08ff9)
    murmur_test.go:83: [Hash128] key: '19 Jan 2038 at 3:14:07 AM', seed: '42': 74da3b584f70841970bc8f344c720268 (want fd8f19ebdc8c6b6ad30fdc310fa08ff9)
BUILDSTDERR:     murmur_test.go:87: [Hash128] key: '19 Jan 2038 at 3:14:07 AM', seed: '42': 0x74da3b584f7error: Bad exit status from /var/tmp/rpm-tmp.nLSukM (%check)
BUILDSTDERR:     Bad exit status from /var/tmp/rpm-tmp.nLSukM (%check)
08419-0x70bc8f344c720268 (want 0xfd8f19ebdc8c6b6a-0xd30fdc310fa08ff9)
    murmur_test.go:41: [Hash32] key: 'The quick brown fox jumps over the lazy dog.', seed: '42': 0xb30cdd6a (want 0xc02d1434)
    murmur_test.go:48: [Hash32] key: 'The quick brown fox jumps over the lazy dog.', seed: '42': b30cdd6a (want c02d1434)
    murmur_test.go:52: [Hash32] key 'The quick brown fox jumps over the lazy dog.', seed: '42': 0xb30cdd6a (want 0xc02d1434)
    murmur_test.go:58: '[Hash64] key: 'The quick brown fox jumps over the lazy dog.', seed: '42': 0x273479948464724d (want 0x74f33c659cda5af7)
    murmur_test.go:65: [Hash64] key: 'The quick brown fox jumps over the lazy dog.', seed: '42': 273479948464724d (want 74f33c659cda5af7)
    murmur_test.go:69: [Hash64] key: 'The quick brown fox jumps over the lazy dog.', seed: '42': 0x273479948464724d (want 0x74f33c659cda5af7)
    murmur_test.go:76: [Hash128] key: 'The quick brown fox jumps over the lazy dog.', seed: '42': 0x273479948464724d-0x41792c09e77ee500 (want 0x74f33c659cda5af7-0x4ec7a891caf316f0)
    murmur_test.go:83: [Hash128] key: 'The quick brown fox jumps over the lazy dog.', seed: '42': 273479948464724d41792c09e77ee500 (want 74f33c659cda5af74ec7a891caf316f0)
    murmur_test.go:87: [Hash128] key: 'The quick brown fox jumps over the lazy dog.', seed: '42': 0x273479948464724d-0x41792c09e77ee500 (want 0x74f33c659cda5af7-0x4ec7a891caf316f0)
hel|lo|
hel|lo, wo|rld|
19 |Jan 20|38 at 3:14:0|7 AM|
The| quick| brown fox j|umps over the lazy dog.|
hel|lo|
hel|lo, wo|rld|
19 |Jan 20|38 at 3:14:0|7 AM|
The| quick| brown fox j|umps over the lazy dog.|
hel|lo|
hel|lo, wo|rld|
19 |Jan 20|38 at 3:14:0|7 AM|
The| quick| brown fox j|umps over the lazy dog.|
--- FAIL: TestIncremental (0.00s)
    murmur_test.go:109: [Hash32] key: 'hello', seed: '0': 0x84c85fd5 (want 0x248bfa47)
    murmur_test.go:109: [Hash32] key: 'hello, world', seed: '0': 0xa98a45f8 (want 0x149bbb7f)
    murmur_test.go:109: [Hash32] key: '19 Jan 2038 at 3:14:07 AM', seed: '0': 0xa74dd6ac (want 0xe31e8a70)
    murmur_test.go:112: [Hash128] key: '19 Jan 2038 at 3:14:07 AM', seed: '0': 0xcef175caccc2d982-0x43fd5acb8ce000a8 (want 0xb89e5988b737affc-0x664fc2950231b2cb)
    murmur_test.go:109: [Hash32] key: 'The quick brown fox jumps over the lazy dog.', seed: '0': 0x4f6a2b71 (want 0xd5c48bfc)
    murmur_test.go:112: [Hash128] key: 'The quick brown fox jumps over the lazy dog.', seed: '0': 0xa9dcb521e865357b-0x31929daa7f2d439e (want 0xcd99481f9ee902c9-0x695da1a38987b6e7)
    murmur_test.go:109: [Hash32] key: 'hello', seed: '1': 0x3abb1755 (want 0xbb4abcad)
    murmur_test.go:109: [Hash32] key: 'hello, world', seed: '1': 0x603764bb (want 0x6f5cb2e9)
    murmur_test.go:109: [Hash32] key: '19 Jan 2038 at 3:14:07 AM', seed: '1': 0x4a7fcb64 (want 0xf50e1f30)
    murmur_test.go:112: [Hash128] key: '19 Jan 2038 at 3:14:07 AM', seed: '1': 0x9c2486a8436dc6d4-0x8b246db6d41ca4cd (want 0x2a929de9c8f97b2f-0x56a41d99af43a2db)
    murmur_test.go:109: [Hash32] key: 'The quick brown fox jumps over the lazy dog.', seed: '1': 0x30ce84fd (want 0x846f6a36)
    murmur_test.go:112: [Hash128] key: 'The quick brown fox jumps over the lazy dog.', seed: '1': 0x628443d6482f2887-0x3dde79f9da7a3d93 (want 0xfb3325171f9744da-0xaaf8b92a5f722952)
    murmur_test.go:109: [Hash32] key: 'hello', seed: '42': 0xb4fc956f (want 0xe2dbd2e1)
    murmur_test.go:109: [Hash32] key: 'hello, world', seed: '42': 0x8cdde10 (want 0x7ec7c6c2)
    murmur_test.go:109: [Hash32] key: '19 Jan 2038 at 3:14:07 AM', seed: '42': 0x4e1b5ffd (want 0x58f745f6)
    murmur_test.go:112: [Hash128] key: '19 Jan 2038 at 3:14:07 AM', seed: '42': 0x74da3b584f708419-0x70bc8f344c720268 (want 0xfd8f19ebdc8c6b6a-0xd30fdc310fa08ff9)
    murmur_test.go:109: [Hash32] key: 'The quick brown fox jumps over the lazy dog.', seed: '42': 0xb30cdd6a (want 0xc02d1434)
    murmur_test.go:112: [Hash128] key: 'The quick brown fox jumps over the lazy dog.', seed: '42': 0x273479948464724d-0x41792c09e77ee500 (want 0x74f33c659cda5af7-0x4ec7a891caf316f0)
FAIL

I'm fairly certain that murmur2 is not endian-neutral, so using the same hash result as little-endian won't work. MurmerHash3 should be endian-neutral, so maybe this has to do with #11.

Sum() method doesn't work after small Write()s

I expect this code to print out the hash value in byte slice form, but it produces a slice with all zeros:

h := murmur3.New64()
h.Write([]byte{1, 2, 3, 4})

fmt.Println(h.Sum(nil))
// Output: [0 0 0 0 0 0 0 0]

fmt.Println(h.Sum64())
// Output: 720734999560851427

GoModules incompatible release versions

On release page versions have form vX.Y, by GoModules cannot work with that form..

It will be nice if you add two additional tags on same revisions in the form of vX.Y.Z

v1.1.0 -> v1.1 (9f5d223)
v1.0.0 -> v1.0 (0d12bf8)

Thanks!

[Question] generate positive only numbers

Hi,
Thanks a lot for this library.
Is there is a way to generate positive number only?

Many thanks,
Marc

fatal error: checkptr: pointer arithmetic result points to invalid allocation

related issue: pingcap/tidb#29086

We got the following error when running go test -race on Go1.15 and go test -race -d=checkptr on Go1.13. Does anyone know why this happens?

fatal error: checkptr: pointer arithmetic result points to invalid allocation

goroutine 1052 [running]:
runtime.throw(0x44c01fb, 0x40)
	/root/go/src/runtime/panic.go:1116 +0x72 fp=0xc000c9fc70 sp=0xc000c9fc40 pc=0x10e8892
runtime.checkptrArithmetic(0xc000c9fd78, 0x0, 0x0, 0x0)
	/root/go/src/runtime/checkptr.go:43 +0xbe fp=0xc000c9fca0 sp=0xc000c9fc70 pc=0x10b755e
github.com/spaolacci/murmur3.Sum32WithSeed(0xc000c9fd78, 0x14, 0x20, 0xc000000000, 0x14)
	/root/go/pkg/mod/github.com/spaolacci/[email protected]/murmur32.go:129 +0x8a fp=0xc000c9fd08 sp=0xc000c9fca0 pc=0x29cbe0a
github.com/spaolacci/murmur3.Sum32(...)

please version murmur3

Hello,

Could you please tag this repository?

I am the Debian Maintainer for this project and tags would help Debian keep up with new releases/bugfixes.

See:

Flaky "unsafe pointer conversion"-panics with -race and go1.14rc1

I cannot reproduce the panic below with a minimal example and it occurs very often, but neither all the time nor in the same test - so this report might be entirely unhelpful, but I am reporting it anyway in case it makes any sense to you:

When running the Syncthing database testsuite on go1.14rc1 with race detection enabled on linux/amd64 I get the following panic:

$ go test -v -race -short -count 1 github.com/syncthing/syncthing/lib/db/ 2>&1 | tee test.out
fatal error: checkptr: unsafe pointer conversion

goroutine 148 [running]:
runtime.throw(0xd106c0, 0x23)
	/media/ext4_data/Linux/source/go/src/runtime/panic.go:1112 +0x72 fp=0xc00022ba50 sp=0xc00022ba20 pc=0x4630d2
runtime.checkptrAlignment(0xc00011e3f1, 0xc40880, 0x1)
	/media/ext4_data/Linux/source/go/src/runtime/checkptr.go:13 +0xd0 fp=0xc00022ba80 sp=0xc00022ba50 pc=0x4348a0
github.com/spaolacci/murmur3.(*digest128).bmix(0xc000123c80, 0xc00011e3f1, 0x20, 0x2f, 0x0, 0x48, 0x58)
	/media/ext4_data/Coding/go/pkg/mod/github.com/spaolacci/[email protected]/murmur128.go:67 +0xc0 fp=0xc00022bae8 sp=0xc00022ba80 pc=0xb649f0
github.com/spaolacci/murmur3.(*digest).Write(0xc000123c80, 0xc00011e3f1, 0x20, 0x2f, 0xb64474, 0xc000123c80, 0xc000123c80)
	/media/ext4_data/Coding/go/pkg/mod/github.com/spaolacci/[email protected]/murmur.go:51 +0x127 fp=0xc00022bb98 sp=0xc00022bae8 pc=0xb63dd7
github.com/spaolacci/murmur3.(*digest128).Write(0xc000123c80, 0xc00011e3f1, 0x20, 0x2f, 0x10, 0x10, 0x21)
	<autogenerated>:1 +0x6a fp=0xc00022bbf8 sp=0xc00022bb98 pc=0xb6558a
github.com/willf/bloom.baseHashes(0xc00011e3f1, 0x20, 0x2f, 0x0, 0x0, 0x0, 0x0)
	/media/ext4_data/Coding/go/pkg/mod/github.com/willf/[email protected]+incompatible/bloom.go:97 +0xb9 fp=0xc00022bc68 sp=0xc00022bbf8 pc=0xb6d179
github.com/willf/bloom.(*BloomFilter).Test(0xc000134700, 0xc00011e3f1, 0x20, 0x2f, 0xdf6fe0)
	/media/ext4_data/Coding/go/pkg/mod/github.com/willf/[email protected]+incompatible/bloom.go:183 +0x6b fp=0xc00022bd20 sp=0xc00022bc68 pc=0xb6deab
github.com/syncthing/syncthing/lib/db.(*Lowlevel).gcBlocks(0xc000098870, 0x0, 0x0)
	/media/ext4_data/Coding/go/src/github.com/syncthing/syncthing/lib/db/lowlevel.go:570 +0x4d4 fp=0xc00022beb8 sp=0xc00022bd20 pc=0xb798b4
github.com/syncthing/syncthing/lib/db.(*Lowlevel).gcRunner(0xc000098870)
	/media/ext4_data/Coding/go/src/github.com/syncthing/syncthing/lib/db/lowlevel.go:483 +0x279 fp=0xc00022bfd8 sp=0xc00022beb8 pc=0xb78fb9
runtime.goexit()
	/media/ext4_data/Linux/source/go/src/runtime/asm_amd64.s:1375 +0x1 fp=0xc00022bfe0 sp=0xc00022bfd8 pc=0x4969c1
created by github.com/syncthing/syncthing/lib/db.NewLowlevel
	/media/ext4_data/Coding/go/src/github.com/syncthing/syncthing/lib/db/lowlevel.go:65 +0x36a

(rest of the backtrace does not include murmur3 nor bloom).

Do you have any ideas what this might be about? Thanks in advance for any help.

Python mmh3 compatibility

Hello @spaolacci thank you so much for providing this murmur3 implementation and for all the work you do in open source; we've been very happy to use this library in our code bases.

I was recently investigating some compatibility issues with the output produced by this package and mmh3, a Python library we are using internally. I created an issue in the mmh3 repository: hajimes/mmh3#46.

The author of that package responded that there is a conflict in the 128-bit byte sequence on little-endian machines that is creating the incompatibility and they provided a Go snippet for the fix:

https://gist.github.com/hajimes/b174e3cd7b0d0c14be97aa39010f2932

They reference the issue in the following snippet: https://github.com/spaolacci/murmur3/blob/master/murmur128.go#L52-L61

There is also a related Stack Overflow question: https://stackoverflow.com/questions/75921577/murmur3-hash-compatibility-between-go-and-python

The author of mmh3 has issued a compatibility notice on their README and package documentation; but I was wondering if you would be interested in my opening a PR into this repository to help resolve the compatibility issue? Let me know the best way I can assist looking into this; I'm sure you're very busy.

How about the performance compare to c/c++?

Consistent result with Guava's implementation

I created a function to create a hash string that is consistent with Guava's implementation. Happy to submit a PR for the README since this might be a fairly common use case.

var hexDigits = []byte("0123456789abcdef")
func guavaHash(h1, h2 uint64) string {
	buf := new(bytes.Buffer)
	binary.Write(buf, binary.LittleEndian, h1)
	binary.Write(buf, binary.LittleEndian, h2)
	var result []byte
	for _, curr := range buf.Bytes() {
		result = append(result, hexDigits[(curr>>4)&0xf])
		result = append(result, hexDigits[curr&0xf])
	}
	return string(result)
}

Originally wanted to do a BigInt version as well but since bytes are signed Java and unsigned in Go I passed on that exercise ;)

[Documentation] Generating hash strings

I think most bystanders would come here and desire to get a hash string like 6c1b07bc7bbc4be347939ac4a93c437a using Sum128WithSeed()

However, two things still need to be done to achieve that:

Use strconv.FormatUint to convert each number output (h1 and h2)
Concatenate the output

This should probably be in the README somewhere, to help newbies along :)

Difference between C++ and Go output

I've been running both the C++ and this version of Murmur3 side by side and i've noticed that in cases where a null byte occurs in the middle of the input data, the C++ version will output a different hash.

Now I guess this is a bug in the C++ implementation, as Go will iterate over the correct array length, and not just stop at a null byte. But, given that the C++ version is the "official" one, should this version implement the bug as a feature?

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.