Code Monkey home page Code Monkey logo

go-bloomfilter's Introduction

go-bloomfilter

build workflow Go Reference

go-bloomfilter is implemented by Golang which supports in-memory and Redis. Moreover, it’s available for a duration-based rotation.

Resources

Features

Installation

go get github.com/x0rworld/go-bloomfilter

Quickstart

package main

import (
	"context"
	"github.com/bits-and-blooms/bloom/v3"
	"github.com/x0rworld/go-bloomfilter/config"
	"github.com/x0rworld/go-bloomfilter/factory"
	"log"
)

func main() {
	m, k := bloom.EstimateParameters(100, 0.01)
	// configure factory config
	cfg := config.NewDefaultFactoryConfig()
	// modify config
	cfg.FilterConfig.M = uint64(m)
	cfg.FilterConfig.K = uint64(k)
	// create factory by config
	ff, err := factory.NewFilterFactory(cfg)
	if err != nil {
		log.Println(err)
		return
	}
	// create filter by factory
	f, err := ff.NewFilter(context.Background())
	if err != nil {
		log.Println(err)
		return
	}
	// manipulate filter: Exist & Add
	data := "hello world"
	exist, err := f.Exist(data)
	if err != nil {
		log.Println(err)
		return
	}
	// data: hello world, exist: false
	log.Printf("data: %v, exist: %v\n", data, exist)
	err = f.Add(data)
	if err != nil {
		log.Println(err)
		return
	}
	// add data: hello world
	log.Printf("add data: %s\n", data)
	exist, err = f.Exist(data)
	if err != nil {
		log.Println(err)
		return
	}
	// data: hello world, exist: true
	log.Printf("data: %v, exist: %v\n", data, exist)
}

More examples such as rotation could be found in Examples.

go-bloomfilter's People

Contributors

x0rror avatar deepsourcebot avatar

Stargazers

 avatar astrolemonade avatar Elvis_LEE avatar

Watchers

 avatar

go-bloomfilter's Issues

[bitmap] Max size of bitmap on Redis

Description

According to Redis official mentioned about SETBIT:

The offset argument is required to be greater than or equal to 0, and smaller than 2^32 (this limits bitmaps to 512MB).

However, go-bloomfilter doesn't handle the limitation in bitmap/redis.go.

Solution

Having bucket number which is divided m by 512MB and adding bucket number to the key of bitmap, then calculating where the location of buckets should the item to go when calls CheckBits and SetBits.

Reference

https://redis.io/commands/setbit/

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.