Code Monkey home page Code Monkey logo

bucket's Introduction

DEPRECATED, WARNING: The Couchbase Group introduced collections. The main feature of this repository was to focus on one-bucket collection-like usage. Since the collections as feature already implemented in Couchbase v2, it's no longer necessary, so this repository no longer maintained.

Bucket

Build Status Go Report Card Documentation

Simple Couchbase framework.

Project specifically focuses on the one bucket as database approach, and makes it easier to manage complex data sets. It tries to get rid of the embedded jsons per document and separates them into different documents behind the scene.

Disclaimer:

This is still a work in progress. We will not take responsibility for any breaks in the code that happen after a new version comes out.

Features:

  • Automatic index generator with indexable tags.
  • Simple usage through the handler.
  • Following best practices of Couchbase usage behind the scene, which doesn't affect the user of the library.

Rules:

  1. Only struct can be referenced

How to use:

Create a new handler with the New function, that needs a configuration.

bucket.New( {bucket.Configuration} )

type Configuration struct {
    // The address of the couchbase server
	ConnectionString string 

    // Username and password to access couchbase
	Username         string 
	Password         string 
	
    // The name and password of the bucket you want to use
	BucketName       string 
	BucketPassword   string 

    // The separator of your choice, this will separate the prefix from the field name
	Separator        string 
}

After that you can use the Insert, Get, Remove, Upsert, Touch, GetAndTouch and Ping methods of the handler.

package main

import (
    "context"
    "fmt"

    "github.com/PumpkinSeed/bucket"
    "github.com/couchbase/gocb"
)

var conf = &bucket.Configuration{
    ConnectionString: "myServer.org:1234",
    Username: "cbUsername",
    Password: "cbPassword",
    BucketName: "testBucket",
    BucketPassword: "testBucketPassword",
    Separator: "::",
}

type myStruct struct {
    justAField string `json:"just_a_field"`
}

func main() {
    var in = &myStruct{
        justAField: "basic",
    }
    var cas bucket.Cas
    var typ = "prefix"
    ctx := context.Background()

    h, err := bucket.New(conf)
    if err != nil {
        // handle error
    }
    
    // insert
    cas, id, err := h.Insert(ctx, typ, "myID", in, 0)
    if err != nil {
        // handle error
    }

    // get
    var out = &myStruct{}
    err = h.Get(ctx, typ, id, out)
    if err != nil {
        // handle error
    }

    // touch
    err = h.Touch(ctx, typ, id, in, 0)
    if err != nil {
        // handle error
    }

    // get and touch
    var secondOut = &myStruct{}
    err = h.GetAndTouch(ctx, typ, id, secondOut, 0)
    if err != nil {
        // handle error
    }

    // ping
    var services []gocb.ServiceType
    res, err := h.Ping(ctx, services)
    if err != nil {
        // handle error
    }

    fmt.Println(res)

    // upsert
    in.justAField = "updated"
    cas, newID, err := h.Upsert(ctx, typ, id, in, 0)

    // remove
    err = h.Remove(ctx, typ, newID, in)
    if err != nil {
        // handle error
    }
}

Important:

  • The typ parameter will be the prefix of the initial struct, so you should use the same value for the same types!
  • IDs should be unique, if the parameter is an empty string ("") a globally unique ID will be automatically generated!

Additional:

Embedded structs can be separated into a a new entry with the cb_referenced tag. The value will decide the typ of the struct.

type example struct {
    refMe       *refMe      `json:"ref_me" cb_referenced:"separate_entry"`
    ignoreMe    *ignoreMe   `json:"ignore_me"`
}

type refMe struct {
    referencedField int `json:"referenced_field"`
}

type ignoreMe struct {
    notReferencedField int `json:"not_referenced_field"`
}

You can also index structs adding the cb_indexable:"true" tag to the field, and then calling bucket.Index({context.Context}, yourStruct).

type example struct {
    indexMe         string `json:"index_me" cb_indexable:"true"`
    butNotThisOne   string `json:"but_not_this_one"`
}

bucket's People

Contributors

borosr avatar hajnalandor avatar pumpkinseed avatar u5surf avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

Forkers

u5surf kbranurman

bucket's Issues

Operations tests

  • Test nearly all cases for Write and Read.
  • Rename Read to Get
  • Rename Write to Insert

TTL to params

Let the user set the ttl, this is something what we should allow for all of the write related methods.

Type field

Is that make sense to add type field because of secondary indexes or can we use the document type from the document key?

State validate

Validate the state

  • document_types and iterate over existing keys

Implement N1QL features

Create a struct Filter with fieldName, value, operator fields, which represents the where condition.
This is would be an alternative method for N1QL's select usage.

TestRangeSearch fails occasionally

--- FAIL: TestRangeSearch (3.04s)
    fts_test.go:446: 
        	Error Trace:	fts_test.go:446
        	Error:      	Expected nil, but got: &errors.errorString{s:"[{\"error\":\"rest_auth: preparePerms, err: index not found\",\"request\":{\"ctl\":{\"timeout\":75000},\"query\":{\"end\":\"2019-09-16T22:13:03+02:00\",\"field\":\"something\",\"start\":\"2019-07-16T10:13:03+02:00\"}},\"status\":\"fail\"}\n]"}
        	Test:       	TestRangeSearch
210.815371ms
--- FAIL: TestRangeSearchWithFacet (3.23s)
    fts_test.go:513: 
        	Error Trace:	fts_test.go:513
        	Error:      	Expected nil, but got: &errors.errorString{s:"[{\"error\":\"rest_auth: preparePerms, err: index not found\",\"request\":{\"ctl\":{\"timeout\":75000},\"facets\":{\"BillingAddressAddress1\":{\"field\":\"BillingAddressAddress1\",\"size\":10}},\"query\":{\"end\":\"2019-09-16T22:13:06+02:00\",\"field\":\"something\",\"start\":\"2019-07-16T10:13:06+02:00\"}},\"status\":\"fail\"}\n]"}
        	Test:       	TestRangeSearchWithFacet
2019/10/07 18:13:37 Error when create secondary index An invalid index name was specified.
2019/10/07 18:13:37 Error when dropping index[] An invalid index name was specified.
{Service:1 Endpoint:127.0.0.1:11210 Success:true Latency:560.066µs}
{Service:4 Endpoint:http://127.0.0.1:8093 Success:true Latency:2.069186ms}
{Service:5 Endpoint:http://127.0.0.1:8094 Success:true Latency:25.111139ms}
{Service:6 Endpoint: Success:false Latency:0s}
{Service:1 Endpoint:127.0.0.1:11210 Success:true Latency:416.271µs}
{Service:4 Endpoint:http://127.0.0.1:8093 Success:true Latency:726.847µs}
{Service:5 Endpoint:http://127.0.0.1:8094 Success:true Latency:1.209528ms}

Usage of Cas

Until we return the Cas we should figure out how can we use it in our framework.

Search only checks the base struct nor the embedded ones

For example we have the following json.

{ 
   "token":"bmau53eg8ubfah4bg7eg",
   "creation_date":"2019-10-03 11:59:09.271784777 +0000 UTC",
   "modification_date":"",
   "status":"processed",
   "payment_method":"card",
   ...
   "shipping_method":"Free shipping",
   "will_be_paid_later":false,
   "payment_transaction_id":"bmau53eg8ubfah4bg7f0",
   "product":{ 
      "id":"bmau53eg8ubfah4bg7fg",
      ...
      "status":"active"
   },
   "store":{ 
      "id":"bmau53eg8ubfah4bg7h0",
      ...
      "description":"Product shop"
   }
}

Since the main principles of bucket that it shouldn't store embedded fields, so because of the referenced tags it will stored as:

Key: webshop::1234
Value: {"token":"bmau53eg8ubfah4bg7eg","creation_date":"2019-10-03 11:59:09.271784777 +0000 UTC","modification_date":"","status":"processed","payment_method":"card","invoice_number":"inv-0069947","email":"[email protected]","card_holder_name":"Sheldon Kassulke","credit_card_last_4_digits":"4541" .... "shipping_address_postal_code":"84157","payment_transaction_id":"bmau53eg8ubfah4bg7f0"}
Key: product::1234
Value: {"id":"bmau53eg8ubfah4bg7fg", ... "status":"active"}
Key: store::1234
Value: {"id":"bmau53eg8ubfah4bg7h0", ... "description":"Product shop"}

Issue

Inside the fts.go we have many Search methods. We should make sure if we perform a search on webshop it will do the search in product and store as well and in case of result it will return the webshop's document key in all cases.

Remove pre-seed

Remove pre-seeding from the tests. Seed data only necessary for certain tests.

Read not find document id

What happens if we have an embedded struct but one of the embedded field is empty, do we save it anyway, or leave it? If we leave it, what if the read can't find it?

Remove old Get method

There is the new Get method introduced in #80 so we should remove the old one and make sure tests are working with it.

Write individual benchmarks without inner fmt

There are benchmark test in the operations_test.go where there is inner measurements inside the for loop of the benchmark. Separate them or pull them out from the for loop, but the main issue is the fmt's functions inside the for loop.

Benchmarks:

  • BenchmarkGetSingle
  • BenchmarkGetEmbedded
  • BenchmarkGetPtr
  • BenchmarkRemoveEmbedded

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.