Code Monkey home page Code Monkey logo

open-coroutine's Introduction

open-coroutine

The open-coroutine is a simple, efficient and generic stackful-coroutine library.

我有故事,你有酒吗?

Status

Still under development, please do not use this library in the production environment !

How to use this library ?

step1: add dependency to your Cargo.toml

[dependencies]
# check https://crates.io/crates/open-coroutine
open-coroutine = "x.y.z"

step2: add macro

#[open_coroutine::main]
fn main() {
    //......
}

step3: enjoy the performance improvement brought by open-coroutine !

Examples

Amazing preemptive schedule

Note: not supported for windows

#[open_coroutine::main]
fn main() -> std::io::Result<()> {
    cfg_if::cfg_if! {
        if #[cfg(all(unix, feature = "preemptive-schedule"))] {
            use open_coroutine_core::scheduler::Scheduler;
            use std::sync::{Arc, Condvar, Mutex};
            use std::time::Duration;

            static mut TEST_FLAG1: bool = true;
            static mut TEST_FLAG2: bool = true;
            let pair = Arc::new((Mutex::new(true), Condvar::new()));
            let pair2 = Arc::clone(&pair);
            let handler = std::thread::Builder::new()
                .name("preemptive".to_string())
                .spawn(move || {
                    let scheduler = Scheduler::new();
                    _ = scheduler.submit(
                        |_, _| {
                            println!("coroutine1 launched");
                            while unsafe { TEST_FLAG1 } {
                                println!("loop1");
                                _ = unsafe { libc::usleep(10_000) };
                            }
                            println!("loop1 end");
                            1
                        },
                        None,
                    );
                    _ = scheduler.submit(
                        |_, _| {
                            println!("coroutine2 launched");
                            while unsafe { TEST_FLAG2 } {
                                println!("loop2");
                                _ = unsafe { libc::usleep(10_000) };
                            }
                            println!("loop2 end");
                            unsafe { TEST_FLAG1 = false };
                            2
                        },
                        None,
                    );
                    _ = scheduler.submit(
                        |_, _| {
                            println!("coroutine3 launched");
                            unsafe { TEST_FLAG2 = false };
                            3
                        },
                        None,
                    );
                    scheduler.try_schedule();

                    let (lock, cvar) = &*pair2;
                    let mut pending = lock.lock().unwrap();
                    *pending = false;
                    // notify the condvar that the value has changed.
                    cvar.notify_one();
                })
                .expect("failed to spawn thread");

            // wait for the thread to start up
            let (lock, cvar) = &*pair;
            let result = cvar
                .wait_timeout_while(
                    lock.lock().unwrap(),
                    Duration::from_millis(3000),
                    |&mut pending| pending,
                )
                .unwrap();
            if result.1.timed_out() {
                Err(std::io::Error::new(
                    std::io::ErrorKind::Other,
                    "preemptive schedule failed",
                ))
            } else {
                unsafe {
                    handler.join().unwrap();
                    assert!(!TEST_FLAG1);
                }
                Ok(())
            }
        } else {
            println!("please enable preemptive-schedule feature");
            Ok(())
        }
    }
}

outputs

coroutine1 launched
loop1
coroutine2 launched
loop2
coroutine3 launched
loop1
loop2 end
loop1 end

Arbitrary use of blocking syscalls

#[open_coroutine::main]
fn main() {
    std::thread::sleep(std::time::Duration::from_secs(1));
}

outputs

nanosleep hooked

Features

todo

  • support scalable stack

  • support and compatibility for AF_XDP socket

  • hook other syscall maybe interrupt by signal

    syscalls
    • open
    • chdir
    • chroot
    • mkdir
    • rmdir
    • link
    • unlink
    • readlink
    • stat
    • dup
    • dup2
    • umask
    • mount
    • umount
    • mknod
    • fcntl
    • truncate
    • ftruncate
    • setjmp
    • longjmp
    • chown
    • lchown
    • fchown
    • chmod
    • fchmod
    • fchmodat
    • semop
    • ppoll
    • pselect
    • io_getevents
    • semop
    • semtimedop
    • msgrcv
    • msgsnd
  • support #[open_coroutine::join] macro to wait coroutines

0.5.x

  • refactor syscall state, distinguish between state and innerState

0.4.x

  • Supports and is compatible with io_uring in terms of local file IO
  • elegant shutdown
  • use log instead of println
  • enhance #[open_coroutine::main] macro
  • refactor hook impl, no need to publish dylibs now
  • Monitor follow the thread-per-core guideline
  • EventLoop follow the thread-per-core guideline

0.3.x

  • support genawaiter as low_level stackless coroutine (can't support it due to hook)
  • use corosensei as low_level coroutine
  • support backtrace
  • support #[open_coroutine::co] macro
  • refactor WorkStealQueue

0.2.x

  • use correct epoll_event struct

  • use rayon for parallel computing

  • support #[open_coroutine::main] macro

  • hook almost all read syscall

    read syscalls
    • recv
    • readv
    • pread
    • preadv
    • recvfrom
    • recvmsg
  • hook almost all write syscall

    write syscalls
    • send
    • writev
    • sendto
    • sendmsg
    • pwrite
    • pwritev
  • hook other syscall

    other syscalls
    • sleep
    • usleep
    • nanosleep
    • connect
    • listen
    • accept
    • shutdown
    • poll
    • select

0.1.x

  • basic suspend/resume supported
  • use jemalloc as memory pool
  • higher level coroutine abstraction supported
  • preemptive scheduling supported
  • work stealing supported
  • sleep system call hooks supported

open-coroutine's People

Contributors

dependabot[bot] avatar loongs-zhang avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

open-coroutine's Issues

[Feature] support event and condition

Feature Request

No response

Is your feature request related to a problem? Please describe

No response

Describe the solution you'd like

No response

Describe alternatives you've considered

No response

Additional context

No response

[Feature] add stack overflow Err

Feature Request

No response

Is your feature request related to a problem? Please describe

No response

Describe the solution you'd like

No response

Describe alternatives you've considered

No response

Additional context

No response

[BUG] compile fail in linux environment

Is there an existing issue for this?

  • I have searched the existing issues

Current Behavior

image

Expected Behavior

No response

Steps To Reproduce

No response

Environment

libfiber-rs version(s):

Debug logs

No response

Anything else?

No response

[Feature] support work-steal

Feature Request

No response

Is your feature request related to a problem? Please describe

No response

Describe the solution you'd like

No response

Describe alternatives you've considered

No response

Additional context

No response

[BUG] 02hooked_echo_server run failed

Is there an existing issue for this?

  • I have searched the existing issues

Current Behavior

run failed.

Expected Behavior

run successfully.

Steps To Reproduce

cargo clean
cargo run --example 02hooked_echo_server

Environment

libfiber-rs version(s): current

Debug logs

server started !
Exception: EXC_BAD_ACCESS (code=1, address=0x30)

Anything else?

nop

[BUG] concurrency problem in 02hooked_echo_server

Is there an existing issue for this?

  • I have searched the existing issues

Current Behavior

No response

Expected Behavior

No response

Steps To Reproduce

  1. run 02hooked_echo_server.rs;
  2. run 02base_echo_client.rs;

Environment

libfiber-rs version(s): master

Debug logs

No response

Anything else?

echo_client 110
echo_client 110
echo_client 110

[BUG] The dynamic link library generated by `libhook` cannot be loaded correctly

Is there an existing issue for this?

  • I have searched the existing issues

Current Behavior

run failed

Expected Behavior

run successfully

Steps To Reproduce

[dependencies]
libc = "0.2.119"
base-coroutine = "0.0.1"
open-coroutine = "0.0.1"
use base_coroutine::OpenYielder;
use open_coroutine::co;
use std::os::raw::c_void;

fn main() {
    extern "C" fn f1(
        _yielder: &OpenYielder<Option<&'static mut c_void>>,
        _input: Option<&'static mut c_void>,
    ) -> Option<&'static mut c_void> {
        println!("hello1 from coroutine");
        None
    }
    co(f1, None, 2048);
    extern "C" fn f2(
        _yielder: &OpenYielder<Option<&'static mut c_void>>,
        _input: Option<&'static mut c_void>,
    ) -> Option<&'static mut c_void> {
        println!("hello2 from coroutine");
        None
    }
    co(f2, None, 2048);
    unsafe {
        libc::sleep(1);
    }
}

Environment

open-coroutine version(s):

Debug logs

error: linking with cc failed: exit status: 1
|
= note: "cc" "-arch" "arm64" "/var/folders/yv/k8yn4gt95zbgqvmv4pkx0zfw0000gn/T/rustcxSewLr/symbols.o" "/Users/admin/Downloads/study/rust-study/target/debug/deps/43test_hook-1094d3b3ce98550d.2m6f053bv5v2znn9.rcgu.o" "/Users/admin/Downloads/study/rust-study/target/debug/deps/43test_hook-1094d3b3ce98550d.3pk1jj45xptudnpg.rcgu.o" "/Users/admin/Downloads/study/rust-study/target/debug/deps/43test_hook-1094d3b3ce98550d.3swcxoaurcdh28pl.rcgu.o" "/Users/admin/Downloads/study/rust-study/target/debug/deps/43test_hook-1094d3b3ce98550d.4gwpj4tr63m7ppnt.rcgu.o" "/Users/admin/Downloads/study/rust-study/target/debug/deps/43test_hook-1094d3b3ce98550d.4z1itdfg8o4xapbj.rcgu.o" "/Users/admin/Downloads/study/rust-study/target/debug/deps/43test_hook-1094d3b3ce98550d.5ef6y44j888sl2j3.rcgu.o" "/Users/admin/Downloads/study/rust-study/target/debug/deps/43test_hook-1094d3b3ce98550d.wnngfpdgapu04xv.rcgu.o" "/Users/admin/Downloads/study/rust-study/target/debug/deps/43test_hook-1094d3b3ce98550d.2iiyqsgczzjoa0dq.rcgu.o" "-L" "/Users/admin/Downloads/study/rust-study/target/debug/deps" "-L" "/Users/admin/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib" "/Users/admin/Downloads/study/rust-study/target/debug/deps/libopen_coroutine-a8f3e2d950a92730.rlib" "/Users/admin/Downloads/study/rust-study/target/debug/deps/libbase_coroutine-12832087162b6da7.rlib" "/Users/admin/Downloads/study/rust-study/target/debug/deps/libonce_cell-0f1302befc59200f.rlib" "/Users/admin/Downloads/study/rust-study/target/debug/deps/libtimer_utils-8138146dbd419fc4.rlib" "/Users/admin/Downloads/study/rust-study/target/debug/deps/libobject_collection-9509479e2f448b54.rlib" "/Users/admin/Downloads/study/rust-study/target/debug/deps/libcorosensei-95c540b47f02f612.rlib" "/Users/admin/Downloads/study/rust-study/target/debug/deps/liblibc-3f493df9e27c93d1.rlib" "/Users/admin/Downloads/study/rust-study/target/debug/deps/libscopeguard-d7dce28efe117e2a.rlib" "/Users/admin/Downloads/study/rust-study/target/debug/deps/libcfg_if-a08cfa42db5d7bec.rlib" "/Users/admin/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libstd-f6811f31090e0239.rlib" "/Users/admin/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libpanic_unwind-585aa2ea4819ce90.rlib" "/Users/admin/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libobject-1b4ea1dc4a3823f8.rlib" "/Users/admin/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libmemchr-f653b27121920f5c.rlib" "/Users/admin/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libaddr2line-ef1f6d7964edf865.rlib" "/Users/admin/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libgimli-8d0a95997d48da47.rlib" "/Users/admin/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/librustc_demangle-68d9470ea5ba2fa3.rlib" "/Users/admin/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libstd_detect-d4ed7f63db7e9953.rlib" "/Users/admin/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libhashbrown-d2ad324cf0492211.rlib" "/Users/admin/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libminiz_oxide-a00c8eba7ff52d83.rlib" "/Users/admin/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libadler-92e25d254b309fb5.rlib" "/Users/admin/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/librustc_std_workspace_alloc-9cf27ed549592a86.rlib" "/Users/admin/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libunwind-ebee0ebd9a510319.rlib" "/Users/admin/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libcfg_if-718033b8fe200c22.rlib" "/Users/admin/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/liblibc-5ec3532fb120beec.rlib" "/Users/admin/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/liballoc-80ef5ce8d4cd7e19.rlib" "/Users/admin/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/librustc_std_workspace_core-b5600cb14c277fae.rlib" "/Users/admin/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libcore-760870d297bc6b81.rlib" "/Users/admin/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libcompiler_builtins-931e7dc6a4f959e6.rlib" "-lhook" "-liconv" "-lSystem" "-lresolv" "-lc" "-lm" "-liconv" "-L" "/Users/admin/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib" "-o" "/Users/admin/Downloads/study/rust-study/target/debug/deps/43test_hook-1094d3b3ce98550d" "-Wl,-dead_strip" "-nodefaultlibs"
= note: ld: library not found for -lhook
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Anything else?

No response

[BUG] windows hook not works

Is there an existing issue for this?

  • I have searched the existing issues

Current Behavior

No response

Expected Behavior

No response

Steps To Reproduce

No response

Environment

open-coroutine version(s):

Debug logs

No response

Anything else?

No response

[Feature] add CI/CD workflow

Feature Request

No response

Is your feature request related to a problem? Please describe

No response

Describe the solution you'd like

No response

Describe alternatives you've considered

No response

Additional context

No response

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.