[Bug] Boot timer seems off about firecracker HOT 4 OPEN

What CPU type are you using? The [Firmware Bug]: TSC doesn't count with P0 frequency! seems similar to #4099 Is it also an AMD CPU? Can you reproduce these differences in an Intel or an ARM CPU?

Also, could you try to run your test with --level Debug ? That outputs some information with event_start/event_end that could be useful for timing.

from firecracker.

I'm on an AMD Ryzen 5 7640U, I can try to repro on ARM

from firecracker.

Hi, can you try the same workaround as in #4099, i.e. use lapic=notscdeadline as a Linux boot option? Does that workaround the issue?

from firecracker.

i've been looking at firecracker source in recent days and i saw this issue so thought i would investigate. i see same behaviour on ubuntu kernel 6 host/guest.

if we look at the source, we can see the initial timestamp for boot_timer is recorded here:

https://github.com/firecracker-microvm/firecracker/blob/v1.7.0/src/vmm/src/builder.rs#L233

this is before all the work is done to load kernel, initrd (optionally) from disk, configure any boot disk, setup the cpus and guest memory, start the guest kernel and attach devices etc.

you can see the "kick devices" message is happening after all that work has been done when resume_vm is called here

00.036482 2024-02-07T14:14:16.726643784 [anonymous-instance:main] Artificially kick devices.

https://github.com/firecracker-microvm/firecracker/blob/v1.7.0/src/vmm/src/builder.rs#L385

so the 8ms you are talking about is just the time to give the guest thread state machines a kick and start running i think? 🤔

imo, 78ms and 43 ms are the correct timings if what we want to measure includes all the setup and configuration of the vm to the time the port gets kicked in the guest.

from firecracker.