Code Monkey home page Code Monkey logo

go-tarantool's Introduction

go-tarantool GoDoc Build Status

The go-tarantool package has everything necessary for interfacing with Tarantool 1.6+.

The advantage of integrating Go with Tarantool, which is an application server plus a DBMS, is that Go programmers can handle databases with responses that are faster than other packages according to public benchmarks.

Table of contents

Key features

  • Support for both encoding and decoding of Tarantool queries/commands, which leads us to the following advantages:
    • implementing services that mimic a real Tarantool DBMS is relatively easy; for example, you can code a service which would relay queries and commands to a real Tarantool instance; the server interface is documented here;
    • replication support: you can implement a service which would mimic a Tarantool replication slave and get on-the-fly data updates from the Tarantool master, an example is provided here.
  • The interface for sending and packing queries is different from other go-tarantool implementations, which you may find more aesthetically pleasant to work with: all queries are represented with different types that follow the same interface rather than with individual methods in the connector, e.g. conn.Exec(&Update{...}) vs conn.Update({}).

Installation

Pre-requisites:

  • Tarantool version 1.6 or 1.7,
  • a modern Linux, BSD or Mac OS operating system,
  • a current version of go, version 1.8 or later (use go version to check the version number).

If your go version is older than 1.8, or if go is not installed, download the latest tarball from golang.org and say:

sudo tar -C /usr/local -xzf go1.8.3.linux-amd64.tar.gz
sudo chmod -R a+rwx /usr/local/go

Make sure go and go-tarantool are on your path. For example:

export PATH=$PATH:/usr/local/go/bin
export GOPATH="/usr/local/go/go-tarantool"

The go-tarantool package is in the viciious/go-tarantool repository. To download and install, say:

go get github.com/viciious/go-tarantool

This should bring source and binary files into subdirectories of /usr/local/go, making it possible to access by adding github.com/viciious/go-tarantool in the import {...} section at the start of any Go program.

Hello World

Here is a very short example Go program which tries to connect to a Tarantool server.

package main

import (
    "context"
    "fmt"
    "github.com/viciious/go-tarantool"
)

func main() {
    opts := tarantool.Options{User: "guest"}
    conn, err := tarantool.Connect("127.0.0.1:3301", &opts)
    if err != nil {
        fmt.Printf("Connection refused: %s\n", err.Error())
	return
    }

    query := &tarantool.Insert{Space: "examples", Tuple: []interface{}{uint64(99999), "BB"}}
    resp := conn.Exec(context.Background(), query)

    if resp.Error != nil {
        fmt.Println("Insert failed", resp.Error)
    } else {
        fmt.Println(fmt.Sprintf("Insert succeeded: %#v", resp.Data))
    }

    conn.Close()
}

Cut and paste this example into a file named example.go.

Start a Tarantool server on localhost, and make sure it is listening on port 3301. Set up a space named examples exactly as described in the Tarantool manual's Connectors section.

Again, make sure PATH and GOPATH point to the right places. Then build and run example.go:

go build example.go
./example

You should see: messages saying "Insert failed" or "Insert succeeded".

If that is what you see, then you have successfully installed go-tarantool and successfully executed a program that connected to a Tarantool server and manipulated the contents of a Tarantool database.

Walking through the example

We can now have a closer look at the example.go program and make some observations about what it does.

Observation 1: the line "github.com/viciious/go-tarantool" in the import(...) section brings in all Tarantool-related functions and structures. It is common to bring in context and fmt as well.

Observation 2: the line beginning with "Opts :=" sets up the options for Connect(). In this example, there is only one thing in the structure, a user name. The structure can also contain:

  • ConnectTimeout (the number of milliseconds the connector will wait a new connection to be established before giving up),
  • QueryTimeout (the default maximum number of milliseconds to wait before giving up - can be overriden on per-query basis),
  • DefaultSpace (the name of default Tarantool space)
  • Password (user's password)
  • UUID (used for replication)
  • ReplicaSetUUID (used for replication)

Observation 3: the line containing "tarantool.Connect" is one way to begin a session. There are two parameters:

  • a string with host:port format (or "/path/to/tarantool.socket"), and
  • the option structure that was set up earlier.

There is an alternative way to connect, we will describe it later.

Observation 4: the err structure will be nil if there is no error, otherwise it will have a description which can be retrieved with err.Error().

Observation 5: the conn.exec request, like many requests, is preceded by "conn." which is the name of the object that was returned by Connect(). In this case, for Insert, there are two parameters:

  • a space name (it could just as easily have been a space number), and
  • a tuple.

All the requests described in the Tarantool manual can be expressed in a similar way within connect.Exec(), with the format "&name-of-request{arguments}". For example: &ping{}. For a long example:

    data, err := conn.Exec(context.Background(), &Update{
        Space: "tester",
        Index: "primary",
        Key:   1,
        Set: []Operator{
            &OpAdd{
                Field:    2,
                Argument: 17,
            },
            &OpAssign{
                Field:    1,
                Argument: "Hello World",
            },
        },
    })

API reference

Read the Tarantool manual to find descriptions of terms like "connect", "space", "index", and the requests for creating and manipulating database objects or Lua functions.

The source files for the requests library are:

  • connection.go for the Connect() function plus functions related to connecting, and
  • insert_test.go for an example of a data-manipulation function used in tests.

See comments in these files for syntax details:

The supported requests have parameters and results equivalent to requests in the Tarantool manual. Browsing through the other *.go programs in the package will show how the packagers have paid attention to some of the more advanced features of Tarantool, such as vclock and replication.

Alternative way to connect

Here we show a variation of example.go, where the connect is done a different way.

package main

import (
    "context"
    "fmt"
    "github.com/viciious/go-tarantool"
)

func main() {
    opts := tarantool.Options{User: "guest"}
    tnt := tarantool.New("127.0.0.1:3301", &opts)
    conn, err := tnt.Connect()
    if err != nil {
        fmt.Printf("Connection refused: %s\n", err.Error())
	return
    }

    query := &tarantool.Insert{Space: "examples", Tuple: []interface{}{uint64(99999), "BB"}}
    resp := conn.Exec(context.Background(), query)

    if resp.Error != nil {
        fmt.Println("Insert failed", resp.Error)
    } else {
        fmt.Println(fmt.Sprintf("Insert succeeded: %#v", resp.Data))
    }

    conn.Close()
}

In this variation, tarantool.New returns a Connector instance, which is a goroutine-safe singleton object that can transparently handle reconnects.

Help

To contact go-tarantool developers on any problems, create an issue at viciious/go-tarantool.

The developers of the Tarantool server will also be happy to provide advice or receive feedback.

go-tarantool's People

Contributors

alexx8487 avatar darthunix avatar ehrmanngit avatar ifn avatar lenkis avatar lomik avatar oleggator avatar rkravchik avatar shelomentsevd avatar shmel1k avatar skythet avatar tsukanov-as avatar viciious avatar zimnukhov avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

go-tarantool's Issues

Error on calling box.info function

Hi,

I'm trying to get the information about an instance calling a function box.info or returning 'box.info.replication'. As a result I got the error like that: "Error decoding packet type 0: msgp: attempted to decode type "int" with method for "str""

I suppose that library unable to parse output like vclock: {1: 1203983550}, e.g.:

  replication:
    1:
      id: 1
      uuid: 294e7310-13f0-4690-b136-169599e87ba0
      lsn: 1203755966
    2:
      id: 2
      uuid: 62f23532-0291-42bf-b133-2c9db163aef6
      lsn: 0
      upstream:
        peer: user@replication:14301
        lag: 0.00088262557983398
        status: follow
        idle: 0.3704827
    downstream:
       status: follow
       idle: 0.077566780149937
       vclock: {1: 1203983550}

For anyone who are looking for a workaround to fetch the box info use eval:

local repl = {}
for i, r in pairs(box.info.replication) do
  if r.downstream ~= nil then
    r.downstream.vclock = string.format("%s", r.downstream.vclock)
  end
  repl[r.id] = r
end
return repl

int64 or uint64

When returning a numeric value (number) from the tarantool, sometimes it returns int64, sometimes uint64. It depends on the value of the number. 0 is recognized as int64, 126 as uint64.
Why can not you always use the same type?
For example:

  1. the return of the number of records
  2. tuple with a number value

Subscribe with full vector clock

If there are many replicas in replica set and two or greater "masters" it is needed to set more than one clock in SUBSCRIBE request.

Support UUID and Decimal types

Hello!
Now decimal and UUID values aren't decoded in packets.
For example, if I have space with one field of type 'uuid', packets which I get from joining with snap like here: https://pkg.go.dev/github.com/viciious/go-tarantool#example-Slave.JoinWithSnap-Sync, look something like this - Request:&tarantool.Insert{Space:0x201, Tuple:[]interface {}{(*msgp.RawExtension)(0xc00037a260)}}
If I get tuple as packet.Request.(*tarantool.Insert).Tuple, it looks like this - [{"Data":"YUKSdM85T7e5ndpKCK3GeQ==","Type":2}].
I want to insert this tuple in tarantool space as UUID, is there any way it's possible?
I saw similar issues for other connector: tarantool/go-tarantool#90, tarantool/go-tarantool#96

How to handle situation when master has changed?

Hi! I love this feature of "mimicing a Tarantool replication slave and get on-the-fly data updates from the Tarantool master". The only thing I don't understand is what should I do if the master goes down and a new master is chosen after election between real replicas? Do you have any advice how to handle this?

Support IPROTO_PUSH

box.session.push() in 1.10 impements a new iproto command, IPROTO_PUSH. Please add support for it.

Add Unwrap interface implementation for Errors types

Current error handling scheme is inconvenient and requires additional wrapping on go-tarantool package's client

For example - it is not possible to call errors.Is(err, context.Canceled) because of the current implementation.

My suggestion is to add Unwrap() error method for each error struct in https://github.com/viciious/go-tarantool/blob/master/error.go#L29 so it would be possible to compare go-tarantool errors with golang built-in errors by using errors.As or errors.Is functions

More information about Unwrap interface could be found here:
https://github.com/golang/go/blob/master/src/errors/errors.go#L12

Reconnects

What is the best way to handle reconnects? Docs say that Connector instance "transparently handle reconnects", but I do not see any reconnect logic in sources.

Possible issue with terminology

There is a global trend in programming culture to migrate to a different terminology than master-slave, specifically one that has no reference to slavery.
Recently that was made by such projects like Python, Redis, Django, CouchDB and many others.

Should we change "Slave" struct name to something more polite?
I have no ideas what could it be. Maybe "Shadow"? :)

Access violation in connection writer

Reported with revision 9bf2a85

Oct 02 17:13:05 srv.i daemon[5584]: [signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x88058f]
Oct 02 17:13:05 srv.i daemon[5584]: goroutine 601 [running]:
Oct 02 17:13:05 srv.i daemon[5584]: daemon/vendor/github.com/viciious/go-tarantool.(*BinaryPacket).WriteTo(0x0, 0xd044c0, 0xc421985400, 0xe0, 0x0, 0x0)
Oct 02 17:13:05 srv.i daemon[5584]: /root/rpm/BUILD/daemon-f4a656d/_vendor/src/daemon/vendor/github.com/viciious/go-tarantool/binpacket.go:23 +0x2f
Oct 02 17:13:05 srv.i daemon[5584]: daemon/vendor/github.com/viciious/go-tarantool.(*Connection).writer.func1(0xd044c0, 0xc421985400, 0xc4277f3a00, 0x0, 0x0)
Oct 02 17:13:05 srv.i daemon[5584]: /root/rpm/BUILD/daemon-f4a656d/_vendor/src/daemon/vendor/github.com/viciious/go-tarantool/connection.go:478 +0x87
Oct 02 17:13:05 srv.i daemon[5584]: daemon/vendor/github.com/viciious/go-tarantool.(*Connection).writer(0xc42192a1c0, 0x45c2e6, 0xd0b1c0)
Oct 02 17:13:05 srv.i daemon[5584]: /root/rpm/BUILD/daemon-f4a656d/_vendor/src/daemon/vendor/github.com/viciious/go-tarantool/connection.go:507 +0x371
Oct 02 17:13:05 srv.i daemon[5584]: daemon/vendor/github.com/viciious/go-tarantool.(*Connection).worker.func1(0xc42192a1c0, 0xc4201b9ff0)
Oct 02 17:13:05 srv.i daemon[5584]: /root/rpm/BUILD/daemon-f4a656d/_vendor/src/daemon/vendor/github.com/viciious/go-tarantool/connection.go:415 +0x2b
Oct 02 17:13:05 srv.i daemon[5584]: created by daemon/vendor/github.com/viciious/go-tarantool.(*Connection).worker
Oct 02 17:13:05 srv.i daemon[5584]: /root/rpm/BUILD/daemon-f4a656d/_vendor/src/daemon/vendor/github.com/viciious/go-tarantool/connection.go:414 +0x79

Support 1.7+ binary protocol for procedure calls

[since 1.7.2] by default, net.box connections in Lua comply with a new binary protocol command for CALL, which is not backward compatible with previous versions. The new CALL no longer restricts a function to returning an array of tuples and allows returning an arbitrary MsgPack/JSON result, including scalars, nil and void (nothing). The old CALL is left intact for backward compatibility. It will be removed in the next major release. All programming language drivers will be gradually changed to use the new CALL.

We need to add support for the new call command in the protocol while keeping it easy to switch between the old and the new versions.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.