Code Monkey home page Code Monkey logo

gosnowflake's Introduction

Terry-Mao/gosnowflake

Terry-Mao/gosnowflake is a network service for generating unique ID numbers at high scale with some simple guarantees (golang).

Requeriments

golang 1.2 is required.

zookeeper is required.

Installation

Just pull Terry-Mao/gosnowflake from github using go get:

# install golang from https://code.google.com/p/go/downloads/list
# here golang 1.2(linux-amd64)
$ wget https://go.googlecode.com/files/go1.2.linux-amd64.tar.gz
$ tar -xvf go1.2.linux-amd64.tar.gz
$ cp -R go /usr/local/
$ vim /etc/profile.d/golang.sh
# add below to golang.sh
# export GOROOT=/usr/local/go
# export PATH=$PATH:$GOROOT/bin
# export GOPATH=/data/apps/go
$ source /etc/profile.d/golang.sh
# download the code
$ go get -u github.com/Terry-Mao/gosnowflake
# find the dir
$ cd $GOPATH/src/github.com/Terry-Mao/gosnowflake
# compile
$ go install
$ cp ./gosnowflake-example.conf $GOPATH/bin/gosnowflake.conf
$ cd $GOPATH/bin
# run
$ ./gosnowflake -conf=./gosnowflake.conf
# for help
$ ./gosnowflake -h
# test
$ cd $GOPATH/src/github.com/Terry-Mao/gosnowflake/client
$ go test -conf=./test-examples.conf

Document

# gosnowflake configuration file example

# Note on units: when memory size is needed, it is possible to specify
# it in the usual form of 1k 5GB 4M and so forth:
#
# 1kb => 1024 bytes
# 1mb => 1024*1024 bytes
# 1gb => 1024*1024*1024 bytes
#
# units are case insensitive so 1GB 1Gb 1gB are all the same.

# Note on units: when time duration is needed, it is possible to specify
# it in the usual form of 1s 5M 4h and so forth:
#
# 1s => 1000 * 1000 * 1000 nanoseconds
# 1m => 60 seconds
# 1h => 60 minutes
#
# units are case insensitive so 1h 1H are all the same.

[base]
# When running daemonized, gosnowflake writes a pid file in 
# /tmp/gosnowflake.pid by default. You can specify a custom pid file 
# location here.
pid /tmp/gosnowflake.pid

# Sets the maximum number of CPUs that can be executing simultaneously.
# This call will go away when the scheduler improves. By default the number of 
# logical CPUs is set.
# 
# maxproc 4

# By default gosnowflake listens for connections from all the network interfaces
# available on the server on 8080 port. It is possible to listen to just one or 
# multiple interfaces using the "rpc.bind" configuration directive, 
# followed by one or more IP addresses and port.
#
# Examples:
#
# Note this directive is only support "golang rpc" protocol
# rpc.bind 192.168.1.100:8080,10.0.0.1:8080
# rpc.bind 127.0.0.1:8080
# rpc.bind :8080

# By default gosnowflake thrift listens for connections from all the network interfaces
# available on the server on 8080 port. It is possible to listen to just one or 
# multiple interfaces using the "rpc.bind" configuration directive, 
# followed by one or more IP addresses and port.
#
# Examples:
#
# Note this directive is only support "thrift" protocol
# thrift.bind 192.168.1.100:8080,10.0.0.1:8080
# thrift.bind 127.0.0.1:8080
# thrift.bind :8080

# This is used by gosnowflake service profiling (pprof).
# By default gosnowflake pprof listens for connections from local interfaces on 6971
# port. It's not safty for listening internet IP addresses.
#
# Examples:
#
# pprof.bind 192.168.1.100:6971,10.0.0.1:6971
# pprof.bind 127.0.0.1:6971
# pprof.bind :6971

# This is used by gosnowflake service get stat info by http.
# By default gosnowflake pprof listens for connections from local interfaces on 6972
# port. It's not safty for listening internet IP addresses.
#
# Examples:
#
# stat.bind 192.168.1.100:6971,10.0.0.1:6971
# stat.bind 127.0.0.1:6971
# stat.bind :6971

# The working directory.
#
# The log will be written inside this directory, with the filename specified
# above using the 'logfile' configuration directive.
#  
# Note that you must specify a directory here, not a file name.
dir ./

# Log4go configuration path
log ./log.xml

################################## ZOOKEEPER ##################################
[zookeeper]
# The zookeeper cluster section. When gosnowflake start, it will register data 
# in the zookeeper cluster and create a ephemeral node. When gosnowflake died, 
# the node will drop by zookeeper cluster. 

# Zookeeper cluster addresses. Mutiple address split by a ",".
# Examples:
#
# addr 192.168.1.100:2181,10.0.0.1:2181
# addr 127.0.0.1:2181
# addr 10.20.216.122:2181

# Zookeeper cluster session idle timeout seconds. Zookeeper will close the 
# connection after a client is idle for N seconds.
# Examples:
#
# timeout 30s
# timeout 15s
timeout 30s

# gosnowflake zookeeper root path.
path /gosnowflake-servers

################################## GOSNOWFLAKE ################################
[snowflake]
# snowflake must set a datacenter [0, 31], must be unique in all datacenter.
# Examples:
#
# datacenter 0
# datacenter 1
datacenter 0

# register which worker, must be unique in one datacenter.
# multiple worker id register can split by a ",".
# Examples:
#
# worker 0
# worker 0,1,2
worker 0,1,2

RPC API

SnowflakeRPC.NextId: generate a snowflake id.

SnowflakeRPC.DatacenterId: get gosnowflake service's datacenterId.

SnowflakeRPC.Timestamp: get gosnowflake service's current timestamp.

SnowflakeRPC.Ping: get gosnowflake service's status.

Usage

if err := Init(MyConf.ZKServers, MyConf.ZKPath, MyConf.ZKTimeout); err != nil {
    panic(err)
}
c := NewClient(MyConf.WorkerId)                                             
defer c.Close()
defer c.Destroy()
time.Sleep(1 * time.Second)                                             
id, err := c.Id()                                                       
if err != nil {                                                         
    panic(err)
}                                                                       
fmt.Printf("gosnwoflake id: %d\n", id)                                  

Highly Available

use heartbeat or keepalived apply a VIP for the client.

a easy keepalived configration: here

(1.0.1)

use zookeeper for client failover

golang client sdk: here

(1.1)

LICENSE

gosnowflake is is distributed under the terms of the GNU General Public License, version 3.0 GPLv3

gosnowflake's People

Contributors

terry-mao avatar felixhao avatar weilii avatar

Watchers

Marcelle von Wendland avatar James Cloos avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.