Preemption is broken about hermit-rs HOT 7 CLOSED

Note that setting HERMIT_CPUS=2 fixes the symptoms, even when having three or more threads.

from hermit-rs.

I found that the behavior is identical when running on QEMU. One CPU does not preempt, more than one do.

from hermit-rs.

I don't really have any knowledge about the APIC or the scheduler implementation, but here my two cents on the symptoms:

AFAIK the scheduler in rusty-hermit has some concept of priorities. It seems possible to me that the priority of threads generated via thread::spawn is lower than that of the main thread. In that case, the main thread would always be scheduled (if only a single core is available). Linux uses CFS (Completly Fair Scheduler) by default, so the main thread can't starve the spawned thread on Linux.
You could set a breakpoint / debug print in the scheduler to see if it really is never called.

from hermit-rs.

Thanks for the fast reply, @jschwe!

Good idea with the priorities! With two cores, the above code also fires the timer handler often, which calls the scheduler, but with one core, it does not. Not firing the interrupt handler is suspicious, but the starving could still be caused by priorities. The timer interrupt seems to operate in one-shot mode and not fire periodically, so if the scheduler would not want to preempt the current thread again in the future because of priorities, this might fit.

Another setup stills seems weird though:

Spawn a second thread, which does not print to Stdout:

use std::thread;

fn main() {
	thread::spawn(|| loop {
		println!(" 2")
	});

	thread::spawn(|| loop {});

	loop {
		println!("1 ")
	}
}

Running this code on uhyve with two cores only prints “ 2”, which I interpret as the first thread running on one core and the second thread and the main thread on the other core with the second thread starving the main thread. When printing in the second thread as well, I can read all three outputs again. So to me, it seems like something is going wrong with the one-shot timer setup when not yielding to the kernel by waiting on Stdout's mutex hidden in println! in this case.

I hope this makes some sense.

from hermit-rs.

I don't have any further ideas or time to investigate this currently. Pinging @stlankes since it looks like he might have overlooked this.

from hermit-rs.

RustyHermit uses currently a cooperative scheduler. If you not block a task, the task will never give up the CPU. Your main loop should call std::thread::yield_now.

from hermit-rs.

All right, that makes sense. Thanks for the clarification!

from hermit-rs.