Code Monkey home page Code Monkey logo

gqlite's Introduction

GQLite

Join the chat at https://gitter.im/webbery/gqlite Github

GQLite is a graph database for testing abilities in ending device. The target is to make a small, fast, light-weight graph database.

1. Designed

Here is GQlite's framework:
framework

2. Build

Use command with git clone --recursive https://github.com/webbery/gqlite.git to clone this repository.

2.1. ubuntu

Install latest version of bison.

# Build type can be Release or Debug. If Build type is `Debug`, 'gcorv' should be install before build.
cmake -DCMAKE_BUILD_TYPE=Release gqlite_root_dir

2.2. macos

Install latest version of bison.

2.3. windows

An version of flex&bison is placed in dir tool. So it's not need to install dependency.

2.4. android

C++: 17
Please use cross-compile tools on Ubuntu/MacOS. Some mistakes of libzstd occur on Windows.

3. How to use in C/C++

An simple example shows how to use in your program is here:

#include "gqlite.h"

int gqlite_exec_callback(gqlite_result* params)
{
  if (params) {
    switch (params->type)
    {
    case gqlite_result_type_node:
    {
      gqlite_node* node = params->nodes;
      while (node) {
        switch (node->_type)
        {
        case gqlite_node_type_vertex:
        {
          gqlite_vertex* v = node->_vertex;
          if (v->type == gqlite_id_type::integer) {
            printf("[%d, %s]\n", v->uid, v->properties);
          }
          else {
            printf("[%s, %s]\n", v->cid, v->properties);
          }
        }
          break;
        case gqlite_node_type_edge:
          break;
        default:
          break;
        }
        node = node->_next;
      }
    }
      break;
    case gqlite_result_type_cmd:
      for (size_t idx = 0; idx < params->count; ++idx) {
        printf("%s\n", params->infos[idx]);
      }
      break;
    default:
      break;
    }
  }
  return 0;
}

int main() {
  gqlite* pHandle = 0;
  gqlite_open(&pHandle);
  char* ptr = nullptr;
  gqlite_exec(pHandle,
    "{create: 'example_db'};",
    gqlite_exec_callback, nullptr, &ptr);
  gqlite_free(ptr);
  gqlite_close(pHandle);
}

4. Graph Query Language

4.1. Create Graph

Create a graph is simply use create keyword. The keyword of group, means that all entity node which group belongs to. If we want to search vertex by some property, index keyword will regist it.

{
    create: 'movielens',
    group: [
        {movie: ['title', 'genres']},
        {tag: ['user_id', 'tag', 'movie_id'], index: ['tag']},  // <-- relationship's property must write center if it is a edge
        {rate: ['user_id', 'rate', 'movie_id']}
    ]
};

Here we create an index called tag. The tag will create revert index from tag to group tag's id.
So after upset a new tag, the revert index will be added.

4.2. Data Types

Normaly, basic data type as follows:
string: 'string'
number: 10 means integer, 10.0 means real number.
array: start as [ and end with ]
binary: start with 0b, then follow as base64 string, it will save as binary data. Such as 0b'df32099'
datetime: start with 0d, then will try to convert following string to datetime, such as 0d1642262159
vector: a special type of array, which items are same type.
hash: a special type of string, start with 0h like 0h'hash'

4.3. Add Vertex & Edge

add or update vertex:

{
    upset: 'movie',
    vertex:[
        [21, {'title': 'Get Shorty', genres: ['Comedy', 'Crime', 'Thriller']}],
        [53, {title: 'Lamerica (1994)', genres: ['Adventure','Drama']}],
        [88, {title: 'Black Sheep (1996)'}]
    ]
};

Note that current graph is created graph before called movielens. The 3 of vertexes is added to group movie.

add or update edge:

{
    upset: 'tag',
    edge: [
        [{user_id: 2}, {'--': 'Martin Scorsese'}, {movie_id: [106782, 89774]}],
        [{user_id: 21}, {'--': ['romantic comedy', 'wedding']}, {movie_id: 1569}],
    ]
};

For simply use, it can be write as follows, but id is automatic generated by database:

{
    upset: 'edge',
    edge: [
        ['Tom', ->, 'Lamerica'],
        ['Kitty', <-, 'Black Sheep'],
    ]
};

or simply use bidirection:

{
    upset: 'tag',
    edge: [
        ['Tom', --, 'Lamerica'],
        ['Kitty', --, 'Black Sheep'],
    ]
};

4.4. Remove Vertex & Edge

{remove: 'graph', vertex: [21, 88]};

4.5. Query

4.5.1. intrinct function

count()
{// this is used to count the number of vertex
    query: count(vertex),
    group: 'movie'
};

4.5.2. condition

query all movie that has tag:

{
    query: [movie.title, movie.genres],
    where: [
        [user_id, {--: *}, movie_id]        // here is an edge condition, user_id and movie_id are in group `tag`, * represent all relationship here.
    ],
    in: 'movielens' // the graph instance can be written here or not.
};

Or:

{
    query: movie,
    where: {tag: ['black comedy']}
};

query points from graph by relationship:

{
    query: user,
    where: {
        ->: 'son'
    }
};
{
    query: user,
    where: [
        {
            user: function(user) { return user.age > 10}
        }
    ],
};

query a list of neighbors, where 1 mean 1'st neighbors:

{query: user, from: 'v1', where: {--: 1}};

In order to get a search way

4.6. Inference

Here we define a kind of inference operator, and apply it to a graph.
HMM:

{
    query: hidden_variant,
    event: [{e1: 'sun'}, {e2: 'rain'}, {e3: 'wind'}],
    where: [
        [hidden_variant.v1, {->: 0.2}, e1],
        [hidden_variant.v2, {->: gassian(0.2, 0.1)}, e2],
        [hidden_variant.v3, {->: gassian(0.2, 0.1)}, e3],
        [hidden_variant.v1, {->: 0.2}, hidden_variant.v2],
        [hidden_variant.v2, {->: 0.2}, hidden_variant.v3],
        [hidden_variant.v1, {->: 0.2}, hidden_variant.v4],
    ]
};

5. Utility

5.1. Show Graphs

show graph
show graph 'xxx'

Use Graph

use graph 'xxx'

6. Reference Papers

  1. Yihan Sun, Daniel Ferizovic, Guy E. Belloch. PAM: Parallel Augmented Maps.
  2. Laxman Dhulipala, Guy Blelloch, Yan Gu, Yihan Sun. PaC-trees: Supporting Parallel and Compressed Purely-Functional Collections.
  3. Amine Ghrab, Oscar Romero, Sabri Skhiri etc. GRAD: On Graph Database Modeling.
  4. 向量索引算法HNSW和NSG的比较
  5. Daniel Lemirea, Leonid Boytsov. Decoding billions of integers per second through vectorization.
  6. Thorup Mikkel. Undirected single-source shortest paths with positive integer weights in linear time.
  7. Xuanhua Shi, Xuan Luo, Junling Liang etc. Frog: Asynchronous Graph Processing on GPU with Hybrid Coloring Model.
  8. Xuanhua Shi, Zhigao Zheng, Yongluan Zhou etc. Graph Processing on GPUs: A Survey.
  9. Merrill Duane, Garland Michael, Grimshaw Andrew. High-Performance and Scalable GPU Graph Traversal.
  10. Sakr Sherif, Pardede Eric. Graph Data Management: Techniques and Applications.

gqlite's People

Contributors

webbery avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

gqlite's Issues

syntax error

Error: syntax error, unexpected ',', expecting '}':
{ create: 'movielens', group : [ { movie: ['title', 'genres'] }, { tag: ['user_id', 'tag', 'movie_id'] }, { rate: ['user_id', 'rate', 'movie_id'] } ], index:'tag'};

core dump on Ubuntu 22.04

master branch code, built on Ubuntu 22.04, both customized example application and the tool will core dump, Coudl you help to check?

cannot increment end list iterator

hello, i test you HungorianAlgorithm , errors occurred!!

code

#include <iostream>
#include <Eigen/Dense>
#include <vector>
#include <algorithm>
#include <set>
#include <numeric>
#include <list>
using namespace std;
using Eigen::MatrixXd;

template< typename T >
std::vector<size_t> sort_indexes(const std::vector<T>& v) {
    std::vector<size_t> indx(v.size());
    iota(indx.begin(), indx.end(), 0);
    stable_sort(indx.begin(), indx.end(), [&v](size_t ia, size_t ib) {
        return v[ia] > v[ib];
        });
    return indx;
}

bool is_exist(size_t col, const std::set<size_t>& cols, size_t N) {
    if (cols.find(col) != cols.end()) return true;
    return false;
}

bool find_new(
    const Eigen::MatrixXd& input,
    size_t row,
    std::set<size_t> usedCols,
    std::list<std::pair<size_t, size_t>>& result)
{
    // fmt::print("row: {}, new: {}\n", row, usedCols);
    if (row == input.rows() && usedCols.size() == input.cols()) return true;
    for (size_t col = 0; col < input.cols(); ++col) {
        if (usedCols.count(col)) continue;
        if (input(row, col) == 0) {
            auto status = usedCols.insert(col);
            if (find_new(input, row + 1, usedCols, result)) {
                // fmt::print("get it: {}, {}; {}\n", row, col, usedCols);
                result.push_front({ row, col });
                return true;
            }
            else {
                usedCols.erase(status.first);
                // fmt::print("try: {}, {}\n", row, col);
            }
        }
    }
    return false;
}

std::list<std::pair<size_t, size_t>> get_valid_indexes(const Eigen::MatrixXd& input) {
    std::set<size_t> usedCols;
    std::list<std::pair<size_t, size_t>> result;
    find_new(input, 0, usedCols, result);
    return result;
}

int solve(const Eigen::MatrixXd& input, std::list<std::pair<size_t, size_t>>& matched) {
    // step 0: weight must positive
    Eigen::MatrixXd m(input);


    const auto minRow = m.rowwise().minCoeff();
    Eigen::MatrixXd m1 = m.colwise() - minRow;

    const auto minCol = m1.colwise().minCoeff();
    Eigen::MatrixXd result = m1.rowwise() - minCol;

    do {
        // get zero count of each row and col, then sort it by count
        size_t total = std::count(result.data(), result.data() + result.size(), 0);
        // fmt::print("step 2:\ntotal {}\n", total);
        size_t times = 0;
        Eigen::MatrixXd mTempInf = result;
        Eigen::MatrixXd mTempLine = Eigen::MatrixXd::Zero(result.rows(), result.cols());
        Eigen::MatrixXd mTempLineCross = Eigen::MatrixXd::Zero(result.rows(), result.cols());
        do {
            size_t ridx = 0, cidx = 0;
            std::vector<size_t> vRowCounts, vColCounts;
            for (size_t row = 0; row < mTempInf.rows(); ++row) {
                const auto& r = mTempInf.row(row);
                size_t c = r.size() - r.count();
                vRowCounts.push_back(c);
            }
            for (size_t col = 0; col < mTempInf.cols(); ++col) {
                const auto& c = mTempInf.col(col);
                size_t value = c.size() - c.count();
                vColCounts.push_back(value);
            }
            // sort
            std::vector<size_t> vRows = sort_indexes(vRowCounts);
            std::vector<size_t> vCols = sort_indexes(vColCounts);

            size_t rValue = vRowCounts[vRows[ridx]];
            size_t cValue = vColCounts[vCols[cidx]];
            if (rValue > cValue) {
                auto rows = mTempInf.row(vRows[ridx]);
                rows = (rows.array() == 0).select(INFINITY, rows);
                mTempLine.row(vRows[ridx]) = result.row(vRows[ridx]);
                mTempLineCross.row(vRows[ridx]) += Eigen::VectorXd::Ones(result.cols());
                ridx += 1;
            }
            else {
                auto cols = mTempInf.col(vCols[cidx]);
                cols = (cols.array() == 0).select(INFINITY, cols);
                mTempLine.col(vCols[cidx]) = result.col(vCols[cidx]);
                mTempLineCross.col(vCols[cidx]) += Eigen::VectorXd::Ones(result.cols());
                cidx += 1;
            }
            times += 1;

        } while (mTempInf.count() < mTempInf.size());
        if (times >= input.rows()) {

            break;
        }
        double minimal = mTempInf.minCoeff();
        double maximal = mTempLineCross.maxCoeff();


        mTempInf -= Eigen::MatrixXd::Ones(mTempInf.rows(), mTempInf.cols()) * minimal;
        mTempInf = (mTempInf.array() == INFINITY).select(0, mTempInf);


        auto pluss = (mTempLineCross.array() == maximal).select(minimal, Eigen::MatrixXd::Zero(result.rows(), result.cols()));


        result = mTempInf + mTempLine + pluss;


    } while (true);
    // get position of matched
    auto indexes = get_valid_indexes(result);
    auto itr = indexes.begin();
    for (size_t offset = 0; offset < input.rows(); ++itr, ++offset);
    matched.assign(indexes.begin(), itr);


    return 1;
}

int main()
{
	MatrixXd m(6, 6);
    m << 62, 75, 80, 93, 95, 97,
        75, 80, 82, 85, 71, 97,
        80, 75, 81, 98, 90, 97,
        78, 82, 84, 80, 50, 98,
        90, 85, 85, 80, 85, 99,
        65, 75, 80, 75, 68, 96;

	cout << m << endl;

    std::list<std::pair<size_t, size_t>> indexes;
    int ret = solve(m, indexes);
    int weight;
    if (ret == 1) {
        weight = 0;
        for (auto indx = indexes.begin(); indx != indexes.end(); ++indx) {
            weight += m(indx->first, indx->second);
            cout << indx->first << "-->" << indx->second << endl;
        }
    }
    cout << weight << endl;
    return ret;
}

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.