async-task allows arbitrary metadata to be attached to tasks. It would be nice if this were available in async-executor as well.

Task::local no longer exists, and LocalExecutor requires some setup in order to use. Please consider providing an example (or a pointer to a crate implementing an appropriate pattern) for how to keep a set of local executors around for running non-Send futures, concurrently with the set of normal executors.

In the async-std test suite, at least android and mips target are failing because of

error[E0432]: unresolved import `std::sync::atomic::AtomicU64`
  --> /cargo/registry/src/github.com-1ecc6299db9ec823/async-executor-1.1.0/src/lib.rs:29:47
   |
29 | use std::sync::atomic::{AtomicBool, Ordering, AtomicU64, AtomicUsize};
   |                                               ^^^^^^^^^
   |                                               |
   |                                               no `AtomicU64` in `sync::atomic`
   |                                               help: a similar name exists in the module: `AtomicU8`

bevyengine/bevy#12044 reports that the update to 1.9.0 is SemVer compatible with the prior 1.7.x and 1.8.x releases, and is currently causing all new fresh fetches and builds of Bevy to lock up on start. Likely due to a bug with #93. Still trying to root cause the issue.

related to: PolyMeilex/rfd#136

After update from 1.5.1 to 1.5.2 futures that use async-executor internally suddenly became non send, braking the build. Which is kinda surprising for a patch releases.

error: future cannot be sent between threads safely
   --> src/backend/xdg_desktop_portal.rs:43:9
    |
43  | /         Box::pin(async {
44  | |             OpenFileRequest::default()
45  | |                 .accept_label("Pick file")
46  | |                 .multiple(false)
...   |
59  | |                 .map(FileHandle::from)
60  | |         })
    | |__________^ future created by async block is not `Send`
    |
    = help: within `[async block@src/backend/xdg_desktop_portal.rs:43:18: 60:10]`, the trait `std::marker::Send` is not implemented for `*const async_executor::State`
note: future is not `Send` as this value is used across an await
   --> /home/poly/.cargo/registry/src/index.crates.io-6f17d22bba15001f/async-executor-1.5.2/src/lib.rs:936:18
    |
889 |                 let mut old = with_waker(|waker| {
    |                     ------- has type `std::option::Option<async_executor::LocalQueue>` which is not `Send`
...
936 |                 .await
    |                  ^^^^^ await occurs here, with `mut old` maybe used later
937 |             }
    |             - `mut old` is later dropped here
    = note: required for the cast from `Pin<Box<[async block@src/backend/xdg_desktop_portal.rs:43:18: 60:10]>>` to `Pin<Box<(dyn futures_util::Future<Output = std::option::Option<FileHandle>> + std::marker::Send + 'static)>>`

error: future cannot be sent between threads safely
   --> src/backend/xdg_desktop_portal.rs:64:9
    |
64  | /         Box::pin(async {
65  | |             OpenFileRequest::default()
66  | |                 .accept_label("Pick file(s)")
67  | |                 .multiple(true)
...   |
85  | |                 })
86  | |         })
    | |__________^ future created by async block is not `Send`
    |
    = help: within `[async block@src/backend/xdg_desktop_portal.rs:64:18: 86:10]`, the trait `std::marker::Send` is not implemented for `*const async_executor::State`
note: future is not `Send` as this value is used across an await
   --> /home/poly/.cargo/registry/src/index.crates.io-6f17d22bba15001f/async-executor-1.5.2/src/lib.rs:936:18
    |
889 |                 let mut old = with_waker(|waker| {
    |                     ------- has type `std::option::Option<async_executor::LocalQueue>` which is not `Send`
...
936 |                 .await
    |                  ^^^^^ await occurs here, with `mut old` maybe used later
937 |             }
    |             - `mut old` is later dropped here
    = note: required for the cast from `Pin<Box<[async block@src/backend/xdg_desktop_portal.rs:64:18: 86:10]>>` to `Pin<Box<(dyn futures_util::Future<Output = std::option::Option<Vec<FileHandle>>> + std::marker::Send + 'static)>>`

error: future cannot be sent between threads safely
   --> src/backend/xdg_desktop_portal.rs:109:9
    |
109 | /         Box::pin(async {
110 | |             OpenFileRequest::default()
111 | |                 .accept_label("Pick folder")
112 | |                 .multiple(false)
...   |
126 | |                 .map(FileHandle::from)
127 | |         })
    | |__________^ future created by async block is not `Send`
    |
    = help: within `[async block@src/backend/xdg_desktop_portal.rs:109:18: 127:10]`, the trait `std::marker::Send` is not implemented for `*const async_executor::State`
note: future is not `Send` as this value is used across an await
   --> /home/poly/.cargo/registry/src/index.crates.io-6f17d22bba15001f/async-executor-1.5.2/src/lib.rs:936:18
    |
889 |                 let mut old = with_waker(|waker| {
    |                     ------- has type `std::option::Option<async_executor::LocalQueue>` which is not `Send`
...
936 |                 .await
    |                  ^^^^^ await occurs here, with `mut old` maybe used later
937 |             }
    |             - `mut old` is later dropped here
    = note: required for the cast from `Pin<Box<[async block@src/backend/xdg_desktop_portal.rs:109:18: 127:10]>>` to `Pin<Box<(dyn futures_util::Future<Output = std::option::Option<FileHandle>> + std::marker::Send + 'static)>>`

error: future cannot be sent between threads safely
   --> src/backend/xdg_desktop_portal.rs:131:9
    |
131 | /         Box::pin(async {
132 | |             OpenFileRequest::default()
133 | |                 .accept_label("Pick folders")
134 | |                 .multiple(true)
...   |
153 | |                 })
154 | |         })
    | |__________^ future created by async block is not `Send`
    |
    = help: within `[async block@src/backend/xdg_desktop_portal.rs:131:18: 154:10]`, the trait `std::marker::Send` is not implemented for `*const async_executor::State`
note: future is not `Send` as this value is used across an await
   --> /home/poly/.cargo/registry/src/index.crates.io-6f17d22bba15001f/async-executor-1.5.2/src/lib.rs:936:18
    |
889 |                 let mut old = with_waker(|waker| {
    |                     ------- has type `std::option::Option<async_executor::LocalQueue>` which is not `Send`
...
936 |                 .await
    |                  ^^^^^ await occurs here, with `mut old` maybe used later
937 |             }
    |             - `mut old` is later dropped here
    = note: required for the cast from `Pin<Box<[async block@src/backend/xdg_desktop_portal.rs:131:18: 154:10]>>` to `Pin<Box<(dyn futures_util::Future<Output = std::option::Option<Vec<FileHandle>>> + std::marker::Send + 'static)>>`

error: future cannot be sent between threads safely
   --> src/backend/xdg_desktop_portal.rs:172:9
    |
172 | /         Box::pin(async move {
173 | |             SaveFileRequest::default()
174 | |                 .accept_label("Save")
175 | |                 .title(&*self.title.unwrap_or_else(|| "Save file".to_string()))
...   |
190 | |                 .map(FileHandle::from)
191 | |         })
    | |__________^ future created by async block is not `Send`
    |
    = help: within `[async block@src/backend/xdg_desktop_portal.rs:172:18: 191:10]`, the trait `std::marker::Send` is not implemented for `*const async_executor::State`
note: future is not `Send` as this value is used across an await
   --> /home/poly/.cargo/registry/src/index.crates.io-6f17d22bba15001f/async-executor-1.5.2/src/lib.rs:936:18
    |
889 |                 let mut old = with_waker(|waker| {
    |                     ------- has type `std::option::Option<async_executor::LocalQueue>` which is not `Send`
...
936 |                 .await
    |                  ^^^^^ await occurs here, with `mut old` maybe used later
937 |             }
    |             - `mut old` is later dropped here
    = note: required for the cast from `Pin<Box<[async block@src/backend/xdg_desktop_portal.rs:172:18: 191:10]>>` to `Pin<Box<(dyn futures_util::Future<Output = std::option::Option<FileHandle>> + std::marker::Send + 'static)>>`

error: could not compile `rfd` (lib) due to 5 previous errors

Upon investigating bevyengine/bevy#6941, we've found that thread wakeup times have been a significant bottleneck when spinning up large batches of tasks, like those mentioned in #91.

Would it be possible to more aggressively wake up threads (i.e. 2+ at a time), if and only if we know there is sufficient work for all of them? Whenever a thread is stealing work from another, it has an approximation of how much remaining work is in both threads, so it may be reasonable to weigh the potential contention against the cost of sequentially context switches as threads wake up.

This is a question, but I hope asking it here would be useful to other people to find whether it fits the use case.

I'm building a library which has to do some async work (like background processing, async cleanup in Drop::drop() that doesn't necessarily have to block its call), but I don't want to pull heavier dependencies like async-std, tokio or even smol.

I ended up with API that can either create async_executor::Executor and run on background thread or expects one to be provided explicitly:
https://github.com/nazar-pc/mediasoup/blob/ee38b3ad9bbad8638d57350902992ee2e3760073/rust/src/worker_manager.rs#L53-L89

Is this a good idea in general to expose async_executor::Executor as a public API of the library in this way?

I see it is used completely internally in nats crate, but it felt like a good idea to have an option to provide it in case there is already an executor available, potentially running on multiple threads so there is no need to create yet another thread.

Would it make sense for this crate to support no_std + alloc? I thought of smol when searching for a async executor in embedded rust.

Bevy's use of async_executor tends to be very bursty, with a single thread that adds tens or hundreds of tasks in short bursts. This might be running up against the executor's need to acquire a lock on the internal state for every task that is spawned or terminated. Is it possile to expose an function on Executor that would allow the caller to grab the lock once and the spawn an arbitrary number of tasks before releasing it?

This was originally implemented in #37, but the implementation turned out to be very buggy so I removed it in #61. Still, I think it's an important optimization even for a reference executor, as it nearly doubles our performance in micro-benchmarks.

What I would want to do is something like block_on, but which would wait on the passwed futures and all the ones that have been spawned on this executor.
While it doesnt' really make sense for Executor (as long as you keep one running), as it's work stealing across threads, dropping a LocalExecutor ofter the future passed to block_on has finished would end up losing all the spawned ones on the LocalExecutor, unless I'm missing something obvious

What is the point of it? It is not reasonable to statically track the lifetime of Executor which only contain a Arc.

A 'static Executor that has been leaked never has its Drop impl called. This may not be uncommon for use cases where an executor is initialized at startup and never dropped until program termination. In such a use case, the active field in the executor's state isn't really all that useful, as it's only used in the Drop impl and Executor::is_empty.

I understand the intent is to keep the API footprint of the crate fairly low, but would it be feasible to add something like this to the public API of the crate?

#[repr(transparent)]
pub struct LeakedExecutor(State);

impl Executor<'static> {
  fn leak(self) -> &'static LeakedExecutor;
}

impl LeakedExecutor {
  fn spawn<'a: 'static, T: Send + 'a, F: Future<Output = T> +  'a>(&self, fut: F) -> Task<T>;
  async fn tick(&self);
  async fn run(&self);
}

LeakedExecutor should be able to omit all changes to active, and shouldn't require a lock when spawning or finishing tasks. Leaking is also not const, so the atomic check for the whether the state's initialized is also omitted from all calls to the type.

Can you give me an example on how to spawn tasks in a loop but limit total number of tasks?

I want to spawn async tasks from a loop iterating over a huge amount of data, in order to utilize all my CPU cores and execute the tasks in parallell.
How can i make sure i don't spawn more than x amount of tasks?
If to many tasks are spawned they are rejected, i assume there is a max amount of async tasks that can exist at once?

Is there a way to limit this by making the iteration wait for more "spots" to be available in the async tasks list?

An cargo update updated async-executor to version 1.5.3 from 1.5.1. Since this update, I am getting the attached panic when running my application. Note that when pinning back to 1.5.1 and keeping all other crates the same, this panic does not happen.

I don't have a minimal reproducible exampe as I don't know how to use this crate (I just use it as a transitive dependency in ashpd > async-std > async-global-executor > async-executor and ashpd > zbus > async-executor). The full code I use it in is here.

thread 'tokio-runtime-worker' panicked at 'missing local queue', /path/to/flare/build/cargo-home/registry/src/index.crates.io-6f17d22bba15001f/async-executor-1.5.3/src/lib.rs:918:57
stack backtrace:
   0: rust_begin_unwind
   1: core::panicking::panic_fmt
   2: core::option::expect_failed
   3: core::option::Option<T>::expect
             at /build/rustc-1.71.1-src/library/core/src/option.rs:898:21
   4: async_executor::LocalQueue::with::<impl async_executor::LocalQueue>::set::{{closure}}::{{closure}}::{{closure}}
             at ./build/cargo-home/registry/src/index.crates.io-6f17d22bba15001f/async-executor-1.5.3/src/lib.rs:918:43
   5: std::thread::local::LocalKey<T>::try_with
             at /build/rustc-1.71.1-src/library/std/src/thread/local.rs:270:16
   6: async_executor::LocalQueue::with::<impl async_executor::LocalQueue>::set::{{closure}}::{{closure}}
             at ./build/cargo-home/registry/src/index.crates.io-6f17d22bba15001f/async-executor-1.5.3/src/lib.rs:913:21
   7: <futures_lite::future::PollFn<F> as core::future::future::Future>::poll
             at ./build/cargo-home/registry/src/index.crates.io-6f17d22bba15001f/futures-lite-1.13.0/src/future.rs:246:9
   8: async_executor::LocalQueue::with::<impl async_executor::LocalQueue>::set::{{closure}}
             at ./build/cargo-home/registry/src/index.crates.io-6f17d22bba15001f/async-executor-1.5.3/src/lib.rs:936:18
   9: async_executor::Executor::run::{{closure}}
             at ./build/cargo-home/registry/src/index.crates.io-6f17d22bba15001f/async-executor-1.5.3/src/lib.rs:258:10
  10: zbus::abstractions::executor::Executor::run::{{closure}}
             at ./build/cargo-home/registry/src/index.crates.io-6f17d22bba15001f/zbus-3.14.1/src/abstractions/executor.rs:118:39
  11: <core::pin::Pin<P> as core::future::future::Future>::poll
             at /build/rustc-1.71.1-src/library/core/src/future/future.rs:125:9
  12: zbus::connection_builder::ConnectionBuilder::build::{{closure}}
             at ./build/cargo-home/registry/src/index.crates.io-6f17d22bba15001f/zbus-3.14.1/src/connection_builder.rs:329:74
  13: zbus::connection::Connection::session::{{closure}}
             at ./build/cargo-home/registry/src/index.crates.io-6f17d22bba15001f/zbus-3.14.1/src/connection.rs:1232:47
  14: flare::dbus::feedbackd::Feedbackd::new::{{closure}}
             at ./src/dbus/feedbackd.rs:31:48
  15: flare::backend::manager::Manager::init::{{closure}}::{{closure}}
             at ./src/backend/manager.rs:341:65
  16: tokio::runtime::task::core::Core<T,S>::poll::{{closure}}
             at ./build/cargo-home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.32.0/src/runtime/task/core.rs:334:17
  17: tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut
             at ./build/cargo-home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.32.0/src/loom/std/unsafe_cell.rs:16:9
  18: tokio::runtime::task::core::Core<T,S>::poll
             at ./build/cargo-home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.32.0/src/runtime/task/core.rs:323:13
  19: tokio::runtime::task::harness::poll_future::{{closure}}
             at ./build/cargo-home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.32.0/src/runtime/task/harness.rs:485:19
  20: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /build/rustc-1.71.1-src/library/core/src/panic/unwind_safe.rs:271:9
  21: std::panicking::try::do_call
             at /build/rustc-1.71.1-src/library/std/src/panicking.rs:500:40
  22: __rust_try
  23: std::panicking::try
             at /build/rustc-1.71.1-src/library/std/src/panicking.rs:464:19
  24: std::panic::catch_unwind
             at /build/rustc-1.71.1-src/library/std/src/panic.rs:142:14
  25: tokio::runtime::task::harness::poll_future
             at ./build/cargo-home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.32.0/src/runtime/task/harness.rs:473:18
  26: tokio::runtime::task::harness::Harness<T,S>::poll_inner
             at ./build/cargo-home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.32.0/src/runtime/task/harness.rs:208:27
  27: tokio::runtime::task::harness::Harness<T,S>::poll
             at ./build/cargo-home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.32.0/src/runtime/task/harness.rs:153:15
  28: tokio::runtime::task::raw::poll
             at ./build/cargo-home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.32.0/src/runtime/task/raw.rs:276:5
  29: tokio::runtime::task::raw::RawTask::poll
             at ./build/cargo-home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.32.0/src/runtime/task/raw.rs:200:18
  30: tokio::runtime::task::LocalNotified<S>::run
             at ./build/cargo-home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.32.0/src/runtime/task/mod.rs:400:9
  31: tokio::runtime::scheduler::multi_thread::worker::Context::run_task::{{closure}}
             at ./build/cargo-home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.32.0/src/runtime/scheduler/multi_thread/worker.rs:576:13
  32: tokio::runtime::coop::with_budget
             at ./build/cargo-home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.32.0/src/runtime/coop.rs:107:5
  33: tokio::runtime::coop::budget
             at ./build/cargo-home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.32.0/src/runtime/coop.rs:73:5
  34: tokio::runtime::scheduler::multi_thread::worker::Context::run_task
             at ./build/cargo-home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.32.0/src/runtime/scheduler/multi_thread/worker.rs:575:9
  35: tokio::runtime::scheduler::multi_thread::worker::Context::run
             at ./build/cargo-home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.32.0/src/runtime/scheduler/multi_thread/worker.rs:526:24
  36: tokio::runtime::scheduler::multi_thread::worker::run::{{closure}}::{{closure}}
             at ./build/cargo-home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.32.0/src/runtime/scheduler/multi_thread/worker.rs:491:21
  37: tokio::runtime::context::scoped::Scoped<T>::set
             at ./build/cargo-home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.32.0/src/runtime/context/scoped.rs:40:9
  38: tokio::runtime::context::set_scheduler::{{closure}}
             at ./build/cargo-home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.32.0/src/runtime/context.rs:176:26
  39: std::thread::local::LocalKey<T>::try_with
             at /build/rustc-1.71.1-src/library/std/src/thread/local.rs:270:16
  40: std::thread::local::LocalKey<T>::with
             at /build/rustc-1.71.1-src/library/std/src/thread/local.rs:246:9
  41: tokio::runtime::context::set_scheduler
             at ./build/cargo-home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.32.0/src/runtime/context.rs:176:9
  42: tokio::runtime::scheduler::multi_thread::worker::run::{{closure}}
             at ./build/cargo-home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.32.0/src/runtime/scheduler/multi_thread/worker.rs:486:9
  43: tokio::runtime::context::runtime::enter_runtime
             at ./build/cargo-home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.32.0/src/runtime/context/runtime.rs:65:16
  44: tokio::runtime::scheduler::multi_thread::worker::run
             at ./build/cargo-home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.32.0/src/runtime/scheduler/multi_thread/worker.rs:478:5
  45: tokio::runtime::scheduler::multi_thread::worker::Launch::launch::{{closure}}
             at ./build/cargo-home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.32.0/src/runtime/scheduler/multi_thread/worker.rs:447:45
  46: <tokio::runtime::blocking::task::BlockingTask<T> as core::future::future::Future>::poll
             at ./build/cargo-home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.32.0/src/runtime/blocking/task.rs:42:21
  47: tokio::runtime::task::core::Core<T,S>::poll::{{closure}}
             at ./build/cargo-home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.32.0/src/runtime/task/core.rs:334:17
  48: tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut
             at ./build/cargo-home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.32.0/src/loom/std/unsafe_cell.rs:16:9
  49: tokio::runtime::task::core::Core<T,S>::poll
             at ./build/cargo-home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.32.0/src/runtime/task/core.rs:323:13
  50: tokio::runtime::task::harness::poll_future::{{closure}}
             at ./build/cargo-home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.32.0/src/runtime/task/harness.rs:485:19
  51: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /build/rustc-1.71.1-src/library/core/src/panic/unwind_safe.rs:271:9
  52: std::panicking::try::do_call
             at /build/rustc-1.71.1-src/library/std/src/panicking.rs:500:40
  53: __rust_try
  54: std::panicking::try
             at /build/rustc-1.71.1-src/library/std/src/panicking.rs:464:19
  55: std::panic::catch_unwind
             at /build/rustc-1.71.1-src/library/std/src/panic.rs:142:14
  56: tokio::runtime::task::harness::poll_future
             at ./build/cargo-home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.32.0/src/runtime/task/harness.rs:473:18
  57: tokio::runtime::task::harness::Harness<T,S>::poll_inner
             at ./build/cargo-home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.32.0/src/runtime/task/harness.rs:208:27
  58: tokio::runtime::task::harness::Harness<T,S>::poll
             at ./build/cargo-home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.32.0/src/runtime/task/harness.rs:153:15
  59: tokio::runtime::task::raw::poll
             at ./build/cargo-home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.32.0/src/runtime/task/raw.rs:276:5
  60: tokio::runtime::task::raw::RawTask::poll
             at ./build/cargo-home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.32.0/src/runtime/task/raw.rs:200:18
  61: tokio::runtime::task::UnownedTask<S>::run
             at ./build/cargo-home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.32.0/src/runtime/task/mod.rs:437:9
  62: tokio::runtime::blocking::pool::Task::run
             at ./build/cargo-home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.32.0/src/runtime/blocking/pool.rs:159:9
  63: tokio::runtime::blocking::pool::Inner::run
             at ./build/cargo-home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.32.0/src/runtime/blocking/pool.rs:513:17
  64: tokio::runtime::blocking::pool::Spawner::spawn_thread::{{closure}}
             at ./build/cargo-home/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.32.0/src/runtime/blocking/pool.rs:471:13

I noticed that although this crate has been updated to use fastrand 2.0.0, that update hasn't yet been published. Is there anything blocking the publication of a new version? Any way I can help?

It is my current opinion that try_tick is almost never useful for the purposes that I've seen people use it.

The intended purpose is for FFI interop with other runtimes. Even then it's use case is niche. You almost never want to pop a task off of and not be notified of the next task later. In the worst cases it lends itself to creating a spin loop, where someone polls try_tick in a loop.

Even in the best cases the patterns that it creates are buggy. Here are issues caused by the misuse of the ticking APIs.

• https://github.com/orgs/smol-rs/discussions/298
• #44
• No doubt constant misuses of this function in production code that is never reported.

For its intended use case of FFI interop, using tick() with a waker that wakes up the event loop that it's in is the preferred option. Even then run() would be the preferred option, as it runs forever.

My intention is as follows:

• Mark try_tick as deprecated.
• In the next breaking release of async-executor remove try_tick.
• Specify in documentation that run is the preferred way of driving the executor.
• Specify in the documentation that tick and try_tick are exclusively used in executor interop, and that pure smol use cases should use run().

I am seeing my server crash very occasionally like this:

Oct 15 17:21:49 sd-177498 geph4-exit-start[26711]: thread 'sscale-wkr-c' panicked at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/async-executor-1.5.4/src/lib.rs:950:49:
Oct 15 17:21:49 sd-177498 geph4-exit-start[26711]: already mutably borrowed: BorrowError
Oct 15 17:21:49 sd-177498 geph4-exit-start[26711]: note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

How to reproduce:

• Clone async-executor
• cargo build --target=wasm32-unknown-unknown

This leads to:

error[E0599]: no method named `get_or_init_blocking` found for struct `async_lock::OnceCell` in the current scope
   --> src/lib.rs:266:20
    |
266 |         self.state.get_or_init_blocking(|| Arc::new(State::new()))
    |                    ^^^^^^^^^^^^^^^^^^^^ help: there is a method with a similar name: `get_or_init`

For more information about this error, try `rustc --explain E0599`.
error: could not compile `async-executor` (lib) due to previous error

Version 1.6 was compiling for wasm32-unknown-unknown just fine.

Potential unaligned read

On windows, atty dereferences a potentially unaligned pointer.

In practice however, the pointer won't be unaligned unless a custom global allocator is used.

In particular, the System allocator on windows uses HeapAlloc, which guarantees a large enough alignment.

atty is Unmaintained

A Pull Request with a fix has been provided over a year ago but the maintainer seems to be unreachable.

Last release of atty was almost 3 years ago.

Possible Alternative(s)

The below list has not been vetted in any way and may or may not contain alternatives;

• is-terminal
• std::io::IsTerminal nightly-only experimental

See advisory page for additional details.

st3 looks to be a fairly promising public crate for implementing tokio's !Send local worker queues which eliminate most of the atomic operations when popping from the local queue.

Right now async_executor uses concurrent_queue for these purposes, but realistically only the workstealing operations need to be threadsafe, the local worker popping from the queue does not actually need to be Send.

Some additional scrutiny on it's thread safety model and some API compatibility additions are definitely needed (see asynchronics/st3#2).

I'm integrating LocalExecutor into OCaml single-threaded event loop, so far it works great, but what I'm missing is the ability for me to register some non-async thread-safe function that will notify OCaml single-threaded event loop (via a pipe) that it needs to call LocalExecutor::try_tick, I have to resort to calling LocalExecutor::try_tick "often enough", but that's obviously a suboptimal solution...

Maybe there is something in the API that already allows me to do this without busy-polling?

I'd like to see Spawn and LocalSpawn trait implementations from futures (now futures-task actually) to be implemented for executors. That would make it possible to use async-executor with generic code that doesn't necessarily care about specific executor being used (if I understand it correctly).

Running clippy on nightly triggers the incompatible_msrv, which highlights that async/await only became available in Rust 1.64:

warning: current MSRV (Minimum Supported Rust Version) is `1.60.0` but this item is stable since `1.64.0`
   --> src\lib.rs:160:20
    |
160 |             future.await
    |                    ^^^^^
    |
    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#incompatible_msrv
    = note: `#[warn(clippy::incompatible_msrv)]` on by default

This crate's MSRV might need to be bumped.

Hello,

I was trying to understand how the Executor and LocalExecutor are working by running some easy examples, but I think there is an issue.

The following code doesn't loop forever, it prints a bunch of times and stops running.

use async_executor::Executor;
use easy_parallel::Parallel;
use smol::future::yield_now;
use std::thread::sleep;
use std::time::Duration;

fn main() {
    let ex = Executor::new();
    Parallel::new()
        .each(0..4, |i| {
            ex.spawn(async move {
                let mut j = 0;
                loop {
                    j += 1;
                    println!("{}: Hello, world! {}", i, j);
                    yield_now().await;
                }
            })
            .detach();
            ex.run(yield_now());
        })
        .run();
    sleep(Duration::from_secs(10));
}

Moreover, I tried to add a Timer instead of yield_now and it only prints 4 times and stops running.

use async_executor::Executor;
use easy_parallel::Parallel;
use smol::future::yield_now;
use smol::Timer;
use std::thread::sleep;
use std::time::Duration;

fn main() {
    let ex = Executor::new();
    Parallel::new()
        .each(0..4, |i| {
            ex.spawn(async move {
                let mut j = 0;
                loop {
                    j += 1;
                    println!("{}: Hello, world! {}", i, j);
                    Timer::new(Duration::from_millis(200)).await;
                }
            })
            .detach();
            ex.run(yield_now());
        })
        .run();
    sleep(Duration::from_secs(10));
}

Do you know what is happening?

With smol 0.2 and below, we could spawn tasks from anywhere, starting with smol 0.3 which uses async-executor, it's no longer possible. It makes it impossible to spawn tasks from e.g. a custom reactor thread which doesn't run on the Executor.
This seem to be a pretty sad regression.

It would be great to at least have some king of handle to spawn tasks on the executor from the outside?

Currently, LocalExecutor wraps Executor and uses Executor's State, which is designed to support multithreading and has the overhead of Mutex, Arc, ConcurrentQueue, and so on. It would be nice if there was a version of LocalExecutor that did not make this assumption and didn't have to pay for these structures in a strictly single-threaded context. Would this be possible to do now that rust-lang/rust#95985 is live in 1.68?

A specific test has started failing with a stack overflow in the Rust mongodb driver when using async-std without any code change on our part; I've narrowed down that this happens when the dependency tree includes async-executor 1.5.3, but not when it's pinned to 1.5.1. This can be reproduced by running this test against a replica set topology:

$ cargo test --no-default-features --features tracing-unstable,async-std-runtime test::spec::retryable_writes::run_legacy
    Finished test [unoptimized + debuginfo] target(s) in 0.16s
     Running unittests src/lib.rs (target/debug/deps/mongodb-a10b17d95b5decb5)

running 1 test

thread 'test::spec::retryable_writes::run_legacy' has overflowed its stack
fatal runtime error: stack overflow
error: test failed, to rerun pass `--lib`

Caused by:
  process didn't exit successfully: `/Users/abraham.egnor/src/mongo-rust-driver/debug_stack-overflow-async-executor/target/debug/deps/mongodb-a10b17d95b5decb5 'test::spec::retryable_writes::run_legacy'` (signal: 6, SIGABRT: process abort signal)

Unfortunately, I haven't had much success constructing a minimal reproduction - I strongly suspect some badly-behaved unsafe code is at the root of this, since changes that affect memory layout (unrelated struct fields) have caused this behavior to unpredictably fail to occur. Let me know if there's more information I can provide that would be useful!

In many actor systems (most notable Erlang), executors are optimized by having a "next task" slot in each executor queue. Whenever a new Runnable is scheduled, the future is first pushed to a slot separate from the normal queue. If a Runnable is already in this slot, it is pushed to the back of the queue. When it comes time to read from this queue, the slot is checked and popped from before the normal queue is.

This is optimal because if a task wakes another task to be immediately executed, the second task will be queued up immediately, which can emulate sequential computations very well.

Hello! I'm not sure whether it's a stack overflow but indeed our app was also broken when updating from 1.5.1 to 1.5.3. It's a complex app based on the Bevy game engine, where running it normally freezes, running it after locking the async-executor through cargo update -p async-executor --precise 1.5.1 works well.
It's a bit hard to understand what is happening and I don't want to bring in any red herring but bevy uses this executor to handle its tasks and it seems to get stuck in a sort of endless busy loop, without any significant memory leak but without any progress on the tasks it has assigned.

The app is here but I wouldn't really ask to run it and troubleshoot. Just thought I'd report the issue since we encountered it in a quite different package from the OP, so maybe it can shed some light.

Originally posted by @luca-della-vedova in #53 (comment)

This segment of code has assumptions that are incorrect for projects making use of multithreaded Wasm:

Multithreaded Wasm is possible with Rust, but the chief limitation is that the main thread panics if it tries to block/wait. However it is allowed to busy-loop as a form of waiting. Crates like wasm_sync reimplement synchronization primitives and use busy-looping instead of blocking on the main thread.

So here's the puzzle: projects like Bevy depend on async-executor, and right now I'm trying to make Bevy multithreaded. There needs to be a workaround to prevent outright crashing if contention is encountered on the main thread due to the above snippet of code.

It could be modified to work like this on Wasm (preferably only on the main thread):

loop {
    if let Ok(state) = self
        .state
        .get_or_try_init_blocking::<()>(|| Ok(Arc::new(State::new())))
    {
        return state;
    }
}

Alternatively the fix could go into the async-lock project.

Would this project be willing to take on a fix like that to unblock upstream users of multithreaded Wasm?

Perhaps eventually this workaround could land in the standard library itself, but there are challenges with that as well. In an even better world browsers would allow waiting on the main thread, but that will not be considered any time soon.

Example:

use futures::future;
use smol::{Task, Timer};
use std::task::Poll;
use std::time::Duration;

async fn sleep_loop() {
    for counter in 1i32.. {
        Timer::after(Duration::from_millis(100)).await;
        eprintln!(
            "beep beep, beep beep. *snooze* (alarm snoozed {} times)",
            counter
        )
    }
}

pub fn main() {
    let mut poll_count: i32 = 0;
    Task::spawn(sleep_loop()).detach();
    smol::run(future::poll_fn(|_cx| {
        poll_count += 1;
        if poll_count == 20 {
            return Poll::Ready(());
        }
        eprintln!("polled {} times", poll_count);
        Poll::Pending
    }));
}

Gives me:

polled 1 times
polled 2 times
polled 3 times
polled 4 times
beep beep, beep beep. *snooze* (alarm snoozed 1 times)
polled 5 times
polled 6 times
polled 7 times
beep beep, beep beep. *snooze* (alarm snoozed 2 times)
polled 8 times
polled 9 times
polled 10 times
beep beep, beep beep. *snooze* (alarm snoozed 3 times)
polled 11 times
polled 12 times
polled 13 times
beep beep, beep beep. *snooze* (alarm snoozed 4 times)
polled 14 times
polled 15 times
polled 16 times
beep beep, beep beep. *snooze* (alarm snoozed 5 times)
polled 17 times
polled 18 times
polled 19 times
beep beep, beep beep. *snooze* (alarm snoozed 6 times)

This came up in smol-rs/smol#113, but I this is a bit more general than that, so I figured it'd give it its own issue.

While it's true that spurious polls should be handled gracefully by futures it seems wasteful to be polling the main future all the time if it hasn't been woken.

https://github.com/smol-rs/async-executor/blob/master/src/lib.rs#L825