Code Monkey home page Code Monkey logo

opensrv's Introduction

OpenSrv

Build Status

Async bindings for emulating database servers. Currently, support for ClickHouse and MySQL/MariaDB is provided.

Status

OpenSrv is in alpha stage, the goal is to provide high performance and highly reliable service compatibility for Databend and others. Welcome any feedback at Discussions!

Examples

You may be looking for:

Projects

  • Databend: A powerful cloud data warehouse. Built for elasticity and efficiency. Free and open. Also available in the cloud: https://app.databend.com
  • GreptimeDB: An open-source, cloud-native, distributed time-series database.
  • CeresDB: A high-performance, distributed, cloud native time-series database that can handle both time-series and analytics workloads.

Contributing

Check out the CONTRIBUTING.md guide for more details on getting started with contributing to this project.

Getting help

Submit issues for bug report or asking questions in discussion.

License

Licensed under Apache License, Version 2.0.

opensrv's People

Contributors

arthur-zhang avatar b41sh avatar bohutang avatar dantengsky avatar junnplus avatar kezhenxu94 avatar michaelscofield avatar psiace avatar rinchannowww avatar ssebo avatar sundy-li avatar sunng87 avatar tceason avatar tisonkun avatar xuanwo avatar youngsofun avatar zhang2014 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

opensrv's Issues

saving authentication related

Currently the shim.authenticate does not pass a &mut self and so it is not possible to save the username for later use.
Also the on_init does not pass username or the entire handshake but rather just the db.
Can we add the username to the on_init ?

Feature Request: 32 bit (ARM) Support

I am trying to get greptimedb running on a Raspberry PI Zero 2W (512MB RAM, 64 bit processor but 32 bit PI OS seems to be recommended for stability).

Greptimedb depends on this library, which currently does not build on the PI, I think because it is a 32 bit system. The compilation error is:

  Compiling opensrv-mysql v0.7.0 (/home/gareth/opensrv/mysql)
error[E0277]: the trait bound `u64: ToUsize` is not satisfied
   --> mysql/src/commands.rs:76:44
    |
76  |                 nom::bytes::complete::take(size)(i)?
    |                 -------------------------- ^^^^ the trait `ToUsize` is not implemented for `u64`
    |                 |
    |                 required by a bound introduced by this call
    |
    = help: the following other types implement trait `ToUsize`:
              usize
              u8
              u16
              u32
note: required by a bound in `nom::bytes::complete::take`
   --> /home/gareth/.cargo/registry/src/index.crates.io-1cd66030c949c28d/nom-7.1.3/src/bytes/complete.rs:408:6
    |
403 | pub fn take<C, Input, Error: ParseError<Input>>(
    |        ---- required by a bound in this function
...
408 |   C: ToUsize,
    |      ^^^^^^^ required by this bound in `take`

Is there any chance of 32 bit support for this project?

Add a basic workflow for this repo

We need to add the following information to ease collaboration

Basic documentation

  • README
  • LICENSE
  • history & credits

Collaboration

  • toolchain / other useful settings
  • CONTRIBUTING
  • PULL_REQUEST_TEMPLATE
  • Workflows

proposal: Simplify ClickHouseSession

ClickHouseSession is too complex and contains many small functions:

fn timezone(&self) -> &str {
    "UTC"
}

fn server_display_name(&self) -> &str {
    "clickhouse-server"
}

fn dbms_version_patch(&self) -> u64 {
    1
}

As a result, it's hard to use and implement one: our simple clickhouse contains nearly 200 lines.

How about returning a ClickHouseMetadata instead?

Make opensrv-mysql support async only

msql-srv is the source of opensrv-mysql, which is a synchronous implementation.

I think we don't need to keep maintaining the sync methods in opensrv-mysql. Let's focus on advancing the asynchronous implementation.

truncate already parsed bytes instead of unparsed bytes.

I found in the code the change of the truncate method in order to improve performance.
but the change cause an issue in case the rest field is greater than 0.

if rest greater than 0 the truncate method will truncate from self.bytes the first remaining bytes number which included some of the parsed bytes.

that scenario cause the next iteration to parse the same packet instead of the remaining packet it should be read from the beginning.

match packet(bytes) {
Ok((rest, p)) => {
self.remaining = rest.len();
self.bytes.truncate(self.remaining);
return Ok(Some(p));
}
Err(nom::Err::Incomplete(_)) | Err(nom::Err::Error(_)) => {}
Err(nom::Err::Failure(ctx)) => {
self.bytes.truncate(self.remaining);
return Err(io::Error::new(
io::ErrorKind::InvalidData,
format!("{:?}", ctx),
));

one optional fix - maybe instead of truncate the bytes vector you can set the start value to be the correct one to start from for next iteration.

but I want to consult with you guys before I change some low level code like that.

thanks.

[opensrv-mysql] Memory safety problem in `PacketReader`

In mysql/src/packet_reader.rs, the following code has a memory safety problem:

let bytes = {
    // NOTE: this is all sorts of unfortunate. what we really want to do is to give
    // &self.bytes[self.start..] to `packet()`, and the lifetimes should all work
    // out. however, without NLL, borrowck doesn't realize that self.bytes is no
    // longer borrowed after the match, and so can be mutated.
    let bytes = &self.bytes[self.start..];
    unsafe { ::std::slice::from_raw_parts(bytes.as_ptr(), self.remaining) }
};
match packet(bytes) {
    Ok((rest, p)) => {
        self.remaining = rest.len();
        if self.remaining > 0 {
            self.bytes = rest.to_vec();
            self.start = 0;
        }
        return Ok(Some(p));
    }
    Err(nom::Err::Incomplete(_)) | Err(nom::Err::Error(_)) => {}
    Err(nom::Err::Failure(ctx)) => {
        self.bytes.truncate(self.remaining);
        return Err(io::Error::new(
            io::ErrorKind::InvalidData,
            format!("{:?}", ctx),
        ));
    }
}

We can see that bytes borrows self.bytes, then p in the Ok match arm also borrows self.bytes (the signature of packet function is pub(crate) fn packet(i: &[u8]) -> nom::IResult<&[u8], (u8, Packet<'_>)>. If self.remaining > 0 holds, then the old self.bytes object will be dropped by self.bytes = rest.to_vec();, then p borrowing that object will be invalidated, but p will still be returned.

Another problem relating to pointer aliasing rule violation may exist: the statement self.bytes.truncate(self.remaining); creates a mutable borrow of self.bytes, while another immutable borrow of self.bytes is live (consider the variable ctx). By language reference, breaking the pointer aliasing rules will cause undefined behavior. This can be solved by rewriting the corresponding match arm to:

Err(nom::Err::Failure(ctx)) => {
    let err = Err(io::Error::new(
        io::ErrorKind::InvalidData,
        format!("{:?}", ctx),
    ));
    self.bytes.truncate(self.remaining);
    return err;
}

valid data length for mysql timestamp binary encoding

In decode.rs:

impl<'a> From<Value<'a>> for NaiveDateTime {
fn from(val: Value<'a>) -> Self {
if let ValueInner::Datetime(mut v) = val.0 {
assert!(v.len() == 7 || v.len() == 11);

The code asserts that v must have length of either 7 or 11.

However, it seems that 0 and 4 are also valid:

image

from https://mariadb.com/kb/en/resultset-row/#timestamp-binary-encoding

Any specific considerations why this assertion only makes 7 and 11 valid here?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.