Code Monkey home page Code Monkey logo

Comments (50)

aristocratos avatar aristocratos commented on June 16, 2024 1

@jmakov
If this happens consistently for you would you mind trying the following:

  1. Start btop with gdb -ex run btop you will se some gdb output next to the clock, just ignore that.
  2. When btop hangs, press CTRL+C , if gdb prompts about more information just press enter.
  3. Press CTRL+L to clear the screen
  4. Type command bt to get the backtrace
  5. Post the backtrace here

That way we could maybe figure out were it gets stuck.

from btop.

aristocratos avatar aristocratos commented on June 16, 2024 1

@jmakov
Fixed in release v1.0.11

from btop.

dfuehrer avatar dfuehrer commented on June 16, 2024 1

It's been stable for the last 2 days so that was the issue, thanks

from btop.

aristocratos avatar aristocratos commented on June 16, 2024 1

@jmakov
For best result make distclean and make OPTFLAGS="-O0 -g" to get as much debugging info as possible with gdb.
And then when stalled, try to get as much of the current stack as possible to find out exactly what's going wrong.

I'm in no way an expert in the workings of gdb so can't say for sure, but logging "live" gdb debugging output to a file would probably be a bit tricky (and quickly get large) with the amount of data it likely would generate.

from btop.

jmakov avatar jmakov commented on June 16, 2024 1

Just a dumb suggestion: spdlog + rotating file logger e.g. max 10MB. Will try out installing from the repo as you suggest.

from btop.

aristocratos avatar aristocratos commented on June 16, 2024 1

@mh166 @jmakov @vaygr
Last one didn't work I just noticed. But I've replaced the clock_nanosleep() trigger with cpu pause instructions in the loop causing the problem in the latest commits.

If you need static binaries you can get them from the continuous build workflow

from btop.

aristocratos avatar aristocratos commented on June 16, 2024 1

@vaygr
I believe I might have fixed it in latest commit. Still stable in my test setup so far.

Static binaries in latest build

from btop.

aristocratos avatar aristocratos commented on June 16, 2024 1

@dfuehrer @vaygr @tomterl
I've removed the use of async threads in the secondary thread which I believe might have been the cause of the issue, and they weren't really serving any use. Probably reduced cpu usage a bit by removing them.

So please make clean and build with latest commit or run the latest static build to see if it's still stalling for you.

from btop.

aristocratos avatar aristocratos commented on June 16, 2024 1

@vaygr
It's possible there was two seperate issues causing stalls and the removal of the async threads fixed on of them.
But there might still be the possibility of a stall from the ifstream issue in the build you are running.
Both are fixed in latest commit that have been running stable in my test setup for over 7 hours now.

Will wait for at least 24 hours stable before pushing a new release with the current fixes.
After that we can hopefully close this issue for good :)

from btop.

aristocratos avatar aristocratos commented on June 16, 2024 1

@vaygr @tomterl @dfuehrer @mh166
It has now been stable for 24 hours, I believe this to be fixed finally in v1.0.16

from btop.

vaygr avatar vaygr commented on June 16, 2024 1

Thanks @aristocratos. After 2 weeks of uptime of 1.2.0 I think it's safe to say it's fully resolved.

from btop.

aristocratos avatar aristocratos commented on June 16, 2024

@jmakov
I managed to replicate it and found it had stalled on a atomic wait operation, not sure why. But I've changed some things in latest commit.
If you compiled yourself you can try pulling and recompiling.

Otherwise you're gonna have to wait a while until I've confirmed my fix worked. Want to have it running for at least 24 hours to make sure.

from btop.

jmakov avatar jmakov commented on June 16, 2024

Thanks! I'm still running with gdb in hope I can replicate but it's working as expected. I think I'll just run with gdb until a new release :).

from btop.

dfuehrer avatar dfuehrer commented on June 16, 2024

I still see this bug when I leave btop running for a long time. The backtrace shows it failing at the atomic_wait at the top of the run function.

from btop.

aristocratos avatar aristocratos commented on June 16, 2024

@dfuehrer
Which version are you running and are you using the static binaries or compiling yourself?

from btop.

dfuehrer avatar dfuehrer commented on June 16, 2024

I build it myself from the latest commit.

from btop.

aristocratos avatar aristocratos commented on June 16, 2024

@dfuehrer
And you ran make clean before compiling the one that stalled?

from btop.

dfuehrer avatar dfuehrer commented on June 16, 2024

I haven't done a make clean recently so I'll do that and then comment back if it crashes again

from btop.

aristocratos avatar aristocratos commented on June 16, 2024

@dfuehrer
That's probably it, should be impossible for atomic_wait() to stall after the changes in v1.0.11.

from btop.

jmakov avatar jmakov commented on June 16, 2024

Hangs again in Byobu shell, btop version: 1.0.14:

(gdb) bt
#0  0x000000000067155f in clock_nanosleep ()
#1  0x000000000063f927 in nanosleep ()
#2  0x000000000048551b in Runner::run(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool, bool) ()
#3  0x0000000000419947 in main ()

from btop.

vaygr avatar vaygr commented on June 16, 2024

Same issue for me. Using pre-built 1.0.14 on Manjaro.

After running for 2-3 hours went into an infinite loop with this:

…
clock_nanosleep(CLOCK_REALTIME, 0, {tv_sec=0, tv_nsec=1000000}, 0x7fff90d16880) = 0
clock_nanosleep(CLOCK_REALTIME, 0, {tv_sec=0, tv_nsec=1000000}, 0x7fff90d16880) = 0
clock_nanosleep(CLOCK_REALTIME, 0, {tv_sec=0, tv_nsec=1000000}, 0x7fff90d16880) = 0
clock_nanosleep(CLOCK_REALTIME, 0, {tv_sec=0, tv_nsec=1000000}, 0x7fff90d16880) = 0
clock_nanosleep(CLOCK_REALTIME, 0, {tv_sec=0, tv_nsec=1000000}, 0x7fff90d16880) = 0
clock_nanosleep(CLOCK_REALTIME, 0, {tv_sec=0, tv_nsec=1000000}, 0x7fff90d16880) = 0
…

from btop.

aristocratos avatar aristocratos commented on June 16, 2024

@jmakov @vaygr
Need some more information:

  1. Does it stall with 100% cpu usage on one core like before?
  2. Does it happen without any user interaction, no resizing, wake from sleep etc.?
  3. What time do you have set on the update timer update_ms?
  4. Does CTRL+c fail?

from btop.

jmakov avatar jmakov commented on June 16, 2024
  1. yes
  2. yes, I just notice nothing is happening and open htop in another window where I see btop using 100% of 1 core
  3. default settings, I just run btop
  4. didn't try that out yet, will update if happens again

from btop.

vaygr avatar vaygr commented on June 16, 2024
  1. yes
  2. yes
  3. default 2000ms
  4. yes

from btop.

aristocratos avatar aristocratos commented on June 16, 2024

@jmakov @vaygr
Any more debugging info you can give me would be very helpful. I'm having a hard time reproducing the issue, so really have no idea of the chain of events leading up to the stall right now.

from btop.

jmakov avatar jmakov commented on June 16, 2024

Is there a way to run with debug/instrumentation info enabled and we can send logs when this happens?

from btop.

aristocratos avatar aristocratos commented on June 16, 2024

@jmakov
By all means, try whatever you like, more information can only be good :)

from btop.

mh166 avatar mh166 commented on June 16, 2024

I experience the same issues:

  • btop++ version: 1.0.14
  • Architecture: x86_64
  • Platform: Debian GNU/Linux 10 (buster)
  • Linux Kernel: 4.19.0-17-amd64
  • Terminal used: byobu (byobu version 5.112; tmux 2.8) via PuTTY (p0.67-t029)

In the options menu of btop++, I set the log level to DEBUG. Yet, the only output I received was the following:

2021/10/11 (19:26:30) | ===> btop++ v.1.0.14
2021/10/11 (19:26:30) | INFO: Logger set to DEBUG
2021/10/11 (19:26:30) | DEBUG: Using locale de_DE.UTF-8
2021/10/11 (19:26:30) | INFO: Running on /dev/pts/17
2021/10/11 (19:26:30) | WARNING: No good candidate for cpu sensor found, using random from all found sensors.

About an hour later it hang. At 20:23:36.

  1. Does it stall with 100% cpu usage on one core like before?
    • Unfortunately I pressed CTRL+C too early, so I can't tell wether it got stuck with 100% cpu load. But at least Zabbix monitoring doesn't seem to suggest that there was an extra load put on the CPU. Regarding the other three questions:
  2. Does it happen without any user interaction, no resizing, wake from sleep etc.?
    • Yes, I was not doing anything on the server at that time.
  3. What time do you have set on the update timer update_ms?
    • 1500ms
  4. Does CTRL+c fail?
    • Well, it used to fail and I had to kill it. This time I ran gdb -ex run btop and so gdb took action when I pressed CTRL+C.

The gdb stack trace is identical to the one posted by @jmakov earlier:

(gdb) bt
#0  0x0000000000668a5f in clock_nanosleep ()
#1  0x0000000000639577 in nanosleep ()
#2  0x0000000000458b2b in Runner::run(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool, bool) ()
#3  0x0000000000419db7 in main ()

from btop.

mh166 avatar mh166 commented on June 16, 2024

I would love to support you with further details on the error. Unfortunately, when running make OPTFLAGS="-O0 -g", I get the following error messages:

[...]
Building btop++ (v1.0.14) Linux x86_64
Compiling src/btop.cpp
Compiling src/btop_draw.cpp
Compiling src/btop_input.cpp
Compiling src/btop_config.cpp
g++: error: unrecognized command line option '-std=c++20'; did you mean '-std=c++2a'?
make: *** [Makefile:187: obj/btop.o] Error 1
make: *** Waiting for unfinished jobs....
g++: error: unrecognized command line option '-std=c++20'; did you mean '-std=c++2a'?
make: *** [Makefile:187: obj/btop_input.o] Error 1
g++: error: unrecognized command line option '-std=c++20'; did you mean '-std=c++2a'?
make: *** [Makefile:187: obj/btop_draw.o] Error 1
g++: error: unrecognized command line option '-std=c++20'; did you mean '-std=c++2a'?
make: *** [Makefile:187: obj/btop_config.o] Error 1

So I assume, that Debian Buster's g++ compiler doesn't support this standard. 😕 Could you maybe provide a stically linked debug build? Maybe with even more debug routines all around that clock_nanosleep (), so I can provide you with more information?

from btop.

aristocratos avatar aristocratos commented on June 16, 2024

@mh166 @jmakov @vaygr
There are some fixes in v1.0.15 that if it doesn't fix the issue, should at least not stall (but maybe crash instead).
However if it crashes instead, we should at least gain some useful information about what's going wrong :)

Statically compiled debug binary (v1.0.15) for more debug symbols: btop-debug.zip

from btop.

vaygr avatar vaygr commented on June 16, 2024

Tried that last one. After a few hours it froze again. strace -fp output was a bit different this time though:

[pid 983489] futex(0x7ffa7402c1c0, FUTEX_WAIT_BITSET, 2147483648, {tv_sec=289229, tv_nsec=39438448}, FUTEX_BITSET_MATCH_ANY) = -1 ETIMEDOUT (Connection timed out)
[pid 983489] futex(0x7ffa7402c1c0, FUTEX_WAIT_BITSET, 2147483648, {tv_sec=289229, tv_nsec=39515309}, FUTEX_BITSET_MATCH_ANY) = -1 ETIMEDOUT (Connection timed out)
[pid 983489] futex(0x7ffa7402c1c0, FUTEX_WAIT_BITSET, 2147483648, {tv_sec=289229, tv_nsec=39611302}, FUTEX_BITSET_MATCH_ANY) = -1 ETIMEDOUT (Connection timed out)

from btop.

aristocratos avatar aristocratos commented on June 16, 2024

@vaygr
Are you running through byobu also?

That's so far the only way I've been able to reproduce it.

from btop.

vaygr avatar vaygr commented on June 16, 2024

No, plain xfce4-terminal 0.8.10.

from btop.

dfuehrer avatar dfuehrer commented on June 16, 2024
  1. Yes, it does stall at 100% CPU on 1 core
  2. Yes, I just leave it running in the background
  3. 1000ms
  4. CTRL+c doesn't fail when run in gdb, it may when run standalone
(gdb) bt
#0  0x00007ffff7bff600 in __lll_lock_wait () from /usr/lib/libpthread.so.0
#1  0x00007ffff7bf8503 in pthread_mutex_lock () from /usr/lib/libpthread.so.0
#2  0x00005555555ee04f in Runner::thread_lock::thread_lock (mtx=..., this=<synthetic pointer>) at src/btop.cpp:296
#3  Runner::run (box="clock", no_update=<optimized out>, force_redraw=<optimized out>) at src/btop.cpp:597
#4  0x0000555555571195 in main (argc=<optimized out>, argv=<optimized out>) at src/btop.cpp:880

Like it says in the backtrace, it hangs at the construction of a thread_lock at the start of run.
This should have been on the latest commit

from btop.

aristocratos avatar aristocratos commented on June 16, 2024

@dfuehrer
Try with latest commit, reverted the thread locks to atomic bool but added a timeout to the waiting.
Static binaries

from btop.

dfuehrer avatar dfuehrer commented on June 16, 2024

This is the backtrace I got from gdb. It failed in alacritty but not in st (at least so far). Ctrl+c does fail when not run in gdb.

(gdb) bt
#0  0x00007ffff7c028ca in __futex_abstimed_wait_common64 () from /usr/lib/libpthread.so.0
#1  0x00007ffff7bf7743 in __pthread_clockjoin_ex () from /usr/lib/libpthread.so.0
#2  0x00005555555de0f9 in clean_quit (sig=<optimized out>) at src/btop.cpp:203
#3  0x00007ffff7a604a7 in __run_exit_handlers () from /usr/lib/libc.so.6
#4  0x00007ffff7a6064e in exit () from /usr/lib/libc.so.6
#5  0x00005555555ee627 in Runner::run (box="clock", no_update=<optimized out>, force_redraw=<optimized out>) at src/btop.cpp:606
#6  0x00005555555711a5 in main (argc=<optimized out>, argv=<optimized out>) at src/btop.cpp:886

from btop.

aristocratos avatar aristocratos commented on June 16, 2024

Still stalls, but at least exits out with a error message now.
Noticed I missed some stuff from @tomterl debug log in #83
It looks like it might possibly be stalling on ifstreams ignore in Proc::collect()
Added some more ifstream error checks and added an attempt at restarting the secondary thread in case of a stall.

Please try with latest commit or new static binaries

Edit: Note that if the thread restart is successful in case of a stall it will only print out a message to the logfile. So please check your btop.log file after running for a while too see if the underlying issue is still present. Or set a break point in gdb at line 544 if running a debug build.

from btop.

aristocratos avatar aristocratos commented on June 16, 2024

Also if anyone still get stalls and are runnig in gdb.
Please use thread apply all bt instead of just bt to get stack trace for all threads.

from btop.

tomterl avatar tomterl commented on June 16, 2024

Looks good - 4 Hours so far and still running fine (self-compiled) - I'm off soon and won't be able to test much till next wednesday

from btop.

vaygr avatar vaygr commented on June 16, 2024

It's been 14 hours and still no issues.

from btop.

mh166 avatar mh166 commented on June 16, 2024

@aristocratos Thank you very much for solving this issue. Can confirm it's been running way beyond 24 hours without any issues. Amazing work! 🎉

from btop.

vaygr avatar vaygr commented on June 16, 2024

Seems like the issue is back after I upgraded to 1.1.0. However, this time 100% CPU usage is intermittent.

The first time btop froze around 10:30am, and 100% CPU usage spiked only at about 3pm same day.

Right now it froze at 6:30am with the following strace context:

ioctl(0, FIONREAD, [0])                 = 0
clock_nanosleep(CLOCK_REALTIME, 0, {tv_sec=0, tv_nsec=10000000}, 0x7ffe210d4d30) = 0
ioctl(0, FIONREAD, [0])                 = 0
clock_nanosleep(CLOCK_REALTIME, 0, {tv_sec=0, tv_nsec=10000000}, 0x7ffe210d4d30) = 0
ioctl(0, FIONREAD, [0])                 = 0
clock_nanosleep(CLOCK_REALTIME, 0, {tv_sec=0, tv_nsec=10000000}, 0x7ffe210d4d30) = 0
ioctl(0, FIONREAD, [0])                 = 0
clock_nanosleep(CLOCK_REALTIME, 0, {tv_sec=0, tv_nsec=10000000}, 0x7ffe210d4d30) = 0
ioctl(0, FIONREAD, [0])                 = 0
clock_nanosleep(CLOCK_REALTIME, 0, {tv_sec=0, tv_nsec=10000000}, 0x7ffe210d4d30) = 0
ioctl(0, FIONREAD, [0]) 

from btop.

tomterl avatar tomterl commented on June 16, 2024

I have 1.1.1 running without hitch for nine days.

from btop.

vaygr avatar vaygr commented on June 16, 2024

Yeah, this one is pretty rare. It happened again today 13 days later.

from btop.

aristocratos avatar aristocratos commented on June 16, 2024

@vaygr
I've had a build running 30+ days without encountering any issues.
The ioctl(0, FIONREAD, [0]) could indicate that this is happening when trying to check the state of standard input and for some reason getting stuck there. The only possibility for stalling there (to my knowledge) would be if the number of characters in standard input for some reason was below zero, which shouldn't be possible.
But could maybe indicate a issue on your system (hardware or software).

Added a fix in latest commit so the suspected loop will break if the value is 0 or below, and we can see if that's the cause of the stall. But guess it will take some time before we know if it worked considering how rare it seems to be.

But I don't have any way to for sure figure out what's happening (or how to fix it) without a proper backtrace leading up to the stall.

from btop.

ezzieyguywuf avatar ezzieyguywuf commented on June 16, 2024

I seem to be running into a similar issue:

(gdb) bt
#0  0x00007ffff7b6ee7a in __GI___clock_nanosleep (clock_id=clock_id@entry=0, flags=flags@entry=0, req=0x7fffffffd2f0, rem=0x7fffffffd2f0) at ../sysdeps/unix/sysv/linux/clock_nanosleep.c:48
#1  0x00007ffff7b73713 in __GI___nanosleep (req=<optimized out>, rem=<optimized out>) at ../sysdeps/unix/sysv/linux/nanosleep.c:25
#2  0x00005555555ede85 in ?? ()
#3  0x000055555556f56b in ?? ()
#4  0x00007ffff7ac48ca in __libc_start_call_main (main=main@entry=0x55555556df30, argc=argc@entry=1, argv=argv@entry=0x7fffffffd618) at ../sysdeps/nptl/libc_start_call_main.h:58
#5  0x00007ffff7ac4985 in __libc_start_main_impl (main=0x55555556df30, argc=1, argv=0x7fffffffd618, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffd608) at ../csu/libc-start.c:360
#6  0x0000555555585351 in ?? ()

from btop.

correabuscar avatar correabuscar commented on June 16, 2024

This is the backtrace I got from gdb. It failed in alacritty but not in st (at least so far). Ctrl+c does fail when not run in gdb.

(gdb) bt
#0  0x00007ffff7c028ca in __futex_abstimed_wait_common64 () from /usr/lib/libpthread.so.0
#1  0x00007ffff7bf7743 in __pthread_clockjoin_ex () from /usr/lib/libpthread.so.0
#2  0x00005555555de0f9 in clean_quit (sig=<optimized out>) at src/btop.cpp:203
#3  0x00007ffff7a604a7 in __run_exit_handlers () from /usr/lib/libc.so.6
#4  0x00007ffff7a6064e in exit () from /usr/lib/libc.so.6
#5  0x00005555555ee627 in Runner::run (box="clock", no_update=<optimized out>, force_redraw=<optimized out>) at src/btop.cpp:606
#6  0x00005555555711a5 in main (argc=<optimized out>, argv=<optimized out>) at src/btop.cpp:886

I got this (after a system upgrade on Gentoo), it's 100% cpu usage, and happened after i pressed 'q' to quit.

(gdb) thread apply all bt4

Thread 2 (Thread 0x76bd72a006c0 (LWP 1058579) "btop"):
executing: 'bt -full -frame-info source-and-location -past-entry on -entry-values both -frame-arguments all -past-main on'
#0  syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
38		cmpq $-4095, %rax	/* Check %rax for error.  */
No locals.
#1  0x00005d9700243e2b in std::__detail::__platform_wait<int> (__addr=0x5d9700368788 <Runner::do_work>, __addr@entry=<optimized out>, __val=0, __val@entry=<optimized out>) at /usr/lib/gcc/x86_64-pc-linux-gnu/13/include/g++-v13/bits/atomic_wait.h:111
111	      __platform_wait(const _Tp* __addr, __platform_wait_t __val) noexcept
        __e = <optimized out>
        __e = <optimized out>
#2  std::__atomic_wait_address_bare<std::__atomic_semaphore::_M_acquire()::{lambda()#1}>(int const*, std::__atomic_semaphore::_M_acquire()::{lambda()#1}) (__addr=<optimized out>, __addr@entry=<optimized out>, __pred=<optimized out>, __pred@entry=<optimized out>) at /usr/lib/gcc/x86_64-pc-linux-gnu/13/include/g++-v13/bits/atomic_wait.h:449
449		  __detail::__platform_wait(__addr, __val);
        __val = 0
#3  std::__atomic_semaphore::_M_acquire (this=<optimized out>, this@entry=<optimized out>) at /usr/lib/gcc/x86_64-pc-linux-gnu/13/include/g++-v13/bits/semaphore_base.h:215
215	      std::__atomic_wait_address_bare(&_M_counter, __pred);
        __pred = <optimized out>
        __pred = <optimized out>
#4  std::counting_semaphore<1l>::acquire (this=<optimized out>, this@entry=<optimized out>) at /usr/lib/gcc/x86_64-pc-linux-gnu/13/include/g++-v13/semaphore:74
74	      { _M_sem._M_acquire(); }
No locals.
#5  Runner::thread_wait () at /usr/src/debug/sys-process/btop-1.3.2/btop-1.3.2/src/btop.cpp:418
418		inline void thread_wait() { do_work.acquire(); }
No locals.
#6  Runner::_runner () at /usr/src/debug/sys-process/btop-1.3.2/btop-1.3.2/src/btop.cpp:538
538				thread_wait();
        lck = {atom = std::atomic<bool> = { false }, not_true = false}
        powers = <optimized out>
        conf = <optimized out>
        pt_lck = {pt_mutex = @0x5d97003687a0, status = <optimized out>}
        loc = <error reading variable: Cannot access memory at address 0x20>
#7  0x000076bd79aaaff1 in start_thread (arg=<optimized out>, arg@entry=<optimized out>) at pthread_create.c:447
447		ret = pd->start_routine (pd->arg);
        ret = <optimized out>
        pd = <optimized out>
        out = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {130556043986624, 2075342563026190060, 130556043986624, -152, 22, 140729689797520, 2075342563072327404, 2075332021641723628}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
#8  0x000076bd79b2aadc in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78
78		call	*%rdx		/* Call function.  */
No locals.

Thread 1 (Thread 0x76bd79f09740 (LWP 16517) "btop"):
executing: 'bt -full -frame-info source-and-location -past-entry on -entry-values both -frame-arguments all -past-main on'
#0  0x000076bd79aa768c in __futex_abstimed_wait_common64 (private=128, private@entry=<optimized out>, cancel=true, cancel@entry=<optimized out>, abstime=0x7ffe2f2b7050, abstime@entry=<optimized out>, op=265, op@entry=<optimized out>, expected=1058579, expected@entry=<optimized out>, futex_word=0x76bd72a00990, futex_word@entry=<optimized out>) at futex-internal.c:57
57	    return INTERNAL_SYSCALL_CANCEL (futex_time64, futex_word, op, expected,
        sc_cancel_oldtype = 0
        __arg6 = <optimized out>
        __arg3 = <optimized out>
        _a5 = <optimized out>
        _a2 = <optimized out>
        sc_ret = <optimized out>
        __arg4 = <optimized out>
        __arg1 = <optimized out>
        _a6 = <optimized out>
        _a3 = <optimized out>
        resultvar = <optimized out>
        __arg5 = <optimized out>
        __arg2 = <optimized out>
        _a4 = <optimized out>
        _a1 = <optimized out>
#1  __futex_abstimed_wait_common (futex_word=0x76bd72a00990, futex_word@entry=0x76bd72a00990, expected=1058579, expected@entry=<optimized out>, clockid=0, clockid@entry=0, abstime=0x7ffe2f2b7050, abstime@entry=0x7ffe2f2b7050, private=128, private@entry=128, cancel=true, cancel@entry=true) at futex-internal.c:87
87	  err = __futex_abstimed_wait_common64 (futex_word, expected, op, abstime,
        err = <optimized out>
        clockbit = 256
        op = 265
#2  0x000076bd79aa770f in __GI___futex_abstimed_wait_cancelable64 (futex_word=0x76bd72a00990, futex_word@entry=0x76bd72a00990, expected=<optimized out>, expected@entry=<optimized out>, clockid=0, clockid@entry=0, abstime=0x7ffe2f2b7050, abstime@entry=0x7ffe2f2b7050, private=128, private@entry=128) at futex-internal.c:139
139	  return __futex_abstimed_wait_common (futex_word, expected, clockid,
No locals.
#3  0x000076bd79aacca3 in __pthread_clockjoin_ex (threadid=130556043986624, threadid@entry=<optimized out>, thread_return=0x0, thread_return@entry=<optimized out>, clockid=0, clockid@entry=<optimized out>, abstime=0x7ffe2f2b7050, abstime@entry=<optimized out>, block=<optimized out>, block@entry=<optimized out>) at pthread_join_common.c:102
102		  int ret = __futex_abstimed_wait_cancelable64 (
        ret = <optimized out>
        _buffer = {__routine = 0x76bd79aacb80 <cleanup>, __arg = 0x76bd72a00ce0, __canceltype = 2045359872, __prev = 0x0}
        tid = <optimized out>
        pd = 0x76bd72a006c0
        self = <optimized out>
        result = 0
        pd_result = <optimized out>
#4  0x00005d97002429b6 in clean_quit (sig=0, sig@entry=<optimized out>) at /usr/src/debug/sys-process/btop-1.3.2/btop-1.3.2/src/btop.cpp:286
286			if (pthread_timedjoin_np(Runner::runner_id, nullptr, &ts) != 0) {
        ts = {tv_sec = 5, tv_nsec = -4822960917442847232}
        excode = <optimized out>
#5  0x00005d97002a5077 in Input::process (key="q", key@entry=<optimized out>) at /usr/src/debug/sys-process/btop-1.3.2/btop-1.3.2/src/btop_input.cpp:217
217						clean_quit(0);
        keep_going = false
        filtering = <optimized out>
        vim_keys = false
        help_key = <optimized out>
        kill_key = 0x5d970033707e "k"
        boxes = {_M_elems = {"gpu5", "cpu", "mem", "net", "proc", "gpu0", "gpu1", "gpu2", "gpu3", "gpu4"}}
        last_press = 0
#6  0x00005d9700226659 in main (argc=1, argc@entry=1, argv=0x7ffe2f2b79a8, argv@entry=0x7ffe2f2b79a8) at /usr/src/debug/sys-process/btop-1.3.2/btop-1.3.2/src/btop.cpp:1169
1169						else Input::process(Input::get());
        current_time = <optimized out>
        mask = {__val = {512, 102903124808160, 30, 111567764994150, 102903147693248, 0, 16098195381762339, 7, 102903147794432, 125, 125, 140729689798712, 140729689798720, 7, 14690799022258211, 120}}
        update_ms = 2000
        future_time = 1713167011872
#7  0x000076bd79a4444a in __libc_start_call_main (main=0x5d97002259f0 <main(int, char**)>, main@entry=0x5d97002259f0 <main(int, char**)>, argc=1, argc@entry=1, argv=0x7ffe2f2b79a8, argv@entry=0x7ffe2f2b79a8) at ../sysdeps/nptl/libc_start_call_main.h:58
58	      result = main (argc, argv, __environ MAIN_AUXVEC_PARAM);
        self = <optimized out>
        result = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140729689799080, 1030095583795003116, 1, 0, 130556168114176, 102903124990840, 1030095583843237612, 2075331941406523116}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x76bd7a061000 <_rtld_local>}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
#8  0x000076bd79a4450b in __libc_start_main_impl (main=0x5d97002259f0 <main(int, char**)>, main@entry=<optimized out>, argc=1, argc@entry=<optimized out>, argv=0x7ffe2f2b79a8, argv@entry=<optimized out>, init=<optimized out>, init@entry=<optimized out>, fini=<optimized out>, fini@entry=<optimized out>, rtld_fini=<optimized out>, rtld_fini@entry=<optimized out>, stack_end=0x7ffe2f2b7998, stack_end@entry=<optimized out>) at ../csu/libc-start.c:360
360	  __libc_start_call_main (main, argc, argv MAIN_AUXVEC_PARAM);
No locals.
#9  0x00005d970023cef5 in _start ()
No symbol table info available.
(gdb)

This is what got updated:

1713121395:  === (1 of 110) Merging (dev-python/ensurepip-setuptools-69.5.1::/var/db/repos/gentoo/dev-python/ensurepip-setuptools/ensurepip-setuptools-69.5.1.ebuild)
1713121396:  === Unmerging... (dev-python/ensurepip-setuptools-69.2.0)
1713121401:  === (5 of 110) Merging (dev-libs/tree-sitter-0.22.4::/var/db/repos/gentoo/dev-libs/tree-sitter/tree-sitter-0.22.4.ebuild)
1713121402:  === Unmerging... (dev-libs/tree-sitter-0.22.2)
1713121407:  === (4 of 110) Merging (dev-util/xdelta-3.0.11-r1::/var/db/repos/gentoo/dev-util/xdelta/xdelta-3.0.11-r1.ebuild)
1713121408:  === Unmerging... (dev-util/xdelta-3.0.11-r1)
1713121412:  === (3 of 110) Merging (sys-kernel/linux-firmware-20240410::/var/db/repos/gentoo/sys-kernel/linux-firmware/linux-firmware-20240410.ebuild)
1713121423:  === Unmerging... (sys-kernel/linux-firmware-20240312)
1713121428:  === (2 of 110) Merging (dev-lang/go-1.22.2::/var/db/repos/gentoo/dev-lang/go/go-1.22.2.ebuild)
1713121432:  === Unmerging... (dev-lang/go-1.22.1)
1713121728:  === (9 of 110) Merging (sys-fs/squashfs-tools-4.6.1::/var/db/repos/gentoo/sys-fs/squashfs-tools/squashfs-tools-4.6.1.ebuild)
1713121730:  === Unmerging... (sys-fs/squashfs-tools-4.6.1)
1713121733:  === (6 of 110) Merging (sys-libs/libunwind-1.8.1::/var/db/repos/gentoo/sys-libs/libunwind/libunwind-1.8.1.ebuild)
1713121735:  === Unmerging... (sys-libs/libunwind-1.8.1)
1713121739:  === (8 of 110) Merging (app-shells/fzf-0.49.0::/var/db/repos/gentoo/app-shells/fzf/fzf-0.49.0.ebuild)
1713121741:  === Unmerging... (app-shells/fzf-0.46.1)
1713121745:  === (10 of 110) Merging (x11-misc/compose-tables-1.8.9::/var/db/repos/gentoo/x11-misc/compose-tables/compose-tables-1.8.9.ebuild)
1713121747:  === Unmerging... (x11-misc/compose-tables-1.8.8)
1713121751:  === (7 of 110) Merging (dev-libs/boost-1.84.0-r3::/var/db/repos/gentoo/dev-libs/boost/boost-1.84.0-r3.ebuild)
1713121758:  === Unmerging... (dev-libs/boost-1.84.0-r3)
1713121793:  === (11 of 110) Merging (x11-libs/libX11-1.8.9::/var/db/repos/gentoo/x11-libs/libX11/libX11-1.8.9.ebuild)
1713121794:  === Unmerging... (x11-libs/libX11-1.8.8)
1713121816:  === (12 of 110) Merging (app-arch/xz-utils-5.4.2::/var/db/repos/gentoo/app-arch/xz-utils/xz-utils-5.4.2.ebuild)
1713121817:  === Unmerging... (app-arch/xz-utils-5.4.2)
1713122013:  === (13 of 110) Merging (sys-libs/ncurses-6.4_p20240330::/var/db/repos/gentoo/sys-libs/ncurses/ncurses-6.4_p20240330.ebuild)
1713122015:  === Unmerging... (sys-libs/ncurses-6.4_p20230401)
1713122060:  === (14 of 110) Merging (sys-apps/util-linux-2.39.3-r7::/var/db/repos/gentoo/sys-apps/util-linux/util-linux-2.39.3-r7.ebuild)
1713122063:  === Unmerging... (sys-apps/util-linux-2.39.3-r6)
1713122143:  === (15 of 110) Merging (dev-lang/python-3.11.9::/var/db/repos/gentoo/dev-lang/python/python-3.11.9.ebuild)
1713122150:  === Unmerging... (dev-lang/python-3.11.8_p1)
1713122181:  === (16 of 110) Merging (dev-libs/libxml2-2.12.6::/var/db/repos/gentoo/dev-libs/libxml2/libxml2-2.12.6.ebuild)
1713122183:  === Unmerging... (dev-libs/libxml2-2.12.6)
1713122200:  === (17 of 110) Merging (sys-kernel/linux-headers-6.8-r1::/var/db/repos/gentoo/sys-kernel/linux-headers/linux-headers-6.8-r1.ebuild)
1713122202:  === Unmerging... (sys-kernel/linux-headers-6.7-r1)
1713122214:  === (18 of 110) Merging (sys-libs/libseccomp-2.5.5-r1::/var/db/repos/gentoo/sys-libs/libseccomp/libseccomp-2.5.5-r1.ebuild)
1713122216:  === Unmerging... (sys-libs/libseccomp-2.5.5-r1)
1713122238:  === (19 of 110) Merging (sys-apps/file-5.45-r4::/var/db/repos/gentoo/sys-apps/file/file-5.45-r4.ebuild)
1713122240:  === Unmerging... (sys-apps/file-5.45-r4)
1713122274:  === (20 of 110) Merging (dev-libs/elfutils-0.191::/var/db/repos/gentoo/dev-libs/elfutils/elfutils-0.191.ebuild)
1713122276:  === Unmerging... (dev-libs/elfutils-0.191)
1713122311:  === (21 of 110) Merging (app-arch/libarchive-3.7.3::/var/db/repos/gentoo/app-arch/libarchive/libarchive-3.7.3.ebuild)
1713122313:  === Unmerging... (app-arch/libarchive-3.7.2-r3)
1713122374:  === (22 of 110) Merging (net-libs/gnutls-3.8.3::/var/db/repos/gentoo/net-libs/gnutls/gnutls-3.8.3.ebuild)
1713122376:  === Unmerging... (net-libs/gnutls-3.8.3)
1713122385:  === (23 of 110) Merging (dev-python/trove-classifiers-2024.4.10::/var/db/repos/gentoo/dev-python/trove-classifiers/trove-classifiers-2024.4.10.ebuild)
1713122386:  === Unmerging... (dev-python/trove-classifiers-2024.3.25)
1713122395:  === (24 of 110) Merging (dev-python/hatchling-1.22.5-r1::/var/db/repos/gentoo/dev-python/hatchling/hatchling-1.22.5-r1.ebuild)
1713122397:  === Unmerging... (dev-python/hatchling-1.22.4)
1713122406:  === (25 of 110) Merging (dev-python/idna-3.7::/var/db/repos/gentoo/dev-python/idna/idna-3.7.ebuild)
1713122408:  === Unmerging... (dev-python/idna-3.6)
1713122417:  === (26 of 110) Merging (dev-python/urllib3-2.2.1::/var/db/repos/gentoo/dev-python/urllib3/urllib3-2.2.1.ebuild)
1713122418:  === Unmerging... (dev-python/urllib3-2.2.1)
1713122631:  === (27 of 110) Merging (dev-build/cmake-3.29.2::/var/db/repos/gentoo/dev-build/cmake/cmake-3.29.2.ebuild)
1713122637:  === Unmerging... (dev-build/cmake-3.29.0)
1713122653:  === (28 of 110) Merging (net-libs/nghttp2-1.61.0::/var/db/repos/gentoo/net-libs/nghttp2/nghttp2-1.61.0.ebuild)
1713122655:  === Unmerging... (net-libs/nghttp2-1.60.0)
1713122680:  === (29 of 110) Merging (net-misc/rsync-3.3.0::/var/db/repos/gentoo/net-misc/rsync/rsync-3.3.0.ebuild)
1713122681:  === Unmerging... (net-misc/rsync-3.2.7-r4)
1713122763:  === (30 of 110) Merging (dev-lang/python-3.12.3::/var/db/repos/gentoo/dev-lang/python/python-3.12.3.ebuild)
1713122770:  === Unmerging... (dev-lang/python-3.12.2_p1)
1713122794:  === (31 of 110) Merging (dev-lang/nasm-2.16.02::/var/db/repos/gentoo/dev-lang/nasm/nasm-2.16.02.ebuild)
1713122796:  === Unmerging... (dev-lang/nasm-2.16.01-r1)
1713122809:  === (32 of 110) Merging (dev-util/vulkan-headers-1.3.280.0::/var/db/repos/gentoo/dev-util/vulkan-headers/vulkan-headers-1.3.280.0.ebuild)
1713122810:  === Unmerging... (dev-util/vulkan-headers-1.3.275)
1713122814:  === (33 of 110) Merging (sys-libs/mtdev-1.1.7::/var/db/repos/gentoo/sys-libs/mtdev/mtdev-1.1.7.ebuild)
1713122816:  === Unmerging... (sys-libs/mtdev-1.1.6)
1713122831:  === (34 of 110) Merging (sys-devel/llvm-common-18.1.3::/var/db/repos/gentoo/sys-devel/llvm-common/llvm-common-18.1.3.ebuild)
1713122833:  === Unmerging... (sys-devel/llvm-common-18.1.2)
1713122865:  === (37 of 110) Merging (dev-util/spirv-headers-1.3.280.0::/var/db/repos/gentoo/dev-util/spirv-headers/spirv-headers-1.3.280.0.ebuild)
1713122866:  === Unmerging... (dev-util/spirv-headers-1.3.275)
1713122870:  === (38 of 110) Merging (dev-python/typing-extensions-4.11.0::/var/db/repos/gentoo/dev-python/typing-extensions/typing-extensions-4.11.0.ebuild)
1713122872:  === Unmerging... (dev-python/typing-extensions-4.10.0)
1713122876:  === (36 of 110) Merging (media-libs/vulkan-loader-1.3.280.0::/var/db/repos/gentoo/media-libs/vulkan-loader/vulkan-loader-1.3.280.0.ebuild)
1713122877:  === Unmerging... (media-libs/vulkan-loader-1.3.275)
1713122882:  === (35 of 110) Merging (sys-libs/libomp-18.1.3::/var/db/repos/gentoo/sys-libs/libomp/libomp-18.1.3.ebuild)
1713122883:  === Unmerging... (sys-libs/libomp-18.1.2)
1713123046:  === (44 of 110) Merging (dev-python/backports-tarfile-1.0.0::/var/db/repos/gentoo/dev-python/backports-tarfile/backports-tarfile-1.0.0.ebuild)
1713123051:  === (43 of 110) Merging (dev-util/volk-1.3.280.0::/var/db/repos/gentoo/dev-util/volk/volk-1.3.280.0.ebuild)
1713123052:  === Unmerging... (dev-util/volk-1.3.275)
1713123056:  === (42 of 110) Merging (sys-apps/gptfdisk-1.0.10-r1::/var/db/repos/gentoo/sys-apps/gptfdisk/gptfdisk-1.0.10-r1.ebuild)
1713123057:  === Unmerging... (sys-apps/gptfdisk-1.0.10)
1713123062:  === (40 of 110) Merging (media-libs/libbluray-1.3.4-r1::/var/db/repos/gentoo/media-libs/libbluray/libbluray-1.3.4-r1.ebuild)
1713123063:  === Unmerging... (media-libs/libbluray-1.3.4)
1713123067:  === (41 of 110) Merging (media-libs/tiff-4.6.0::/var/db/repos/gentoo/media-libs/tiff/tiff-4.6.0.ebuild)
1713123069:  === Unmerging... (media-libs/tiff-4.6.0)
1713123073:  === (39 of 110) Merging (dev-util/spirv-tools-1.3.280.0::/var/db/repos/gentoo/dev-util/spirv-tools/spirv-tools-1.3.280.0.ebuild)
1713123076:  === Unmerging... (dev-util/spirv-tools-1.3.275)
1713123123:  === (45 of 110) Merging (dev-util/glslang-1.3.280.0::/var/db/repos/gentoo/dev-util/glslang/glslang-1.3.280.0.ebuild)
1713123125:  === Unmerging... (dev-util/glslang-1.3.275)
1713123134:  === (46 of 110) Merging (dev-python/jaraco-context-5.3.0::/var/db/repos/gentoo/dev-python/jaraco-context/jaraco-context-5.3.0.ebuild)
1713123136:  === Unmerging... (dev-python/jaraco-context-4.3.0)
1713123147:  === (47 of 110) Merging (app-misc/pax-utils-1.3.7::/var/db/repos/gentoo/app-misc/pax-utils/pax-utils-1.3.7.ebuild)
1713123148:  === Unmerging... (app-misc/pax-utils-1.3.7)
1713123224:  === (50 of 110) Merging (dev-ruby/rake-13.2.1::/var/db/repos/gentoo/dev-ruby/rake/rake-13.2.1.ebuild)
1713123226:  === Unmerging... (dev-ruby/rake-13.1.0)
1713123230:  === (51 of 110) Merging (dev-python/setuptools-69.5.1::/var/db/repos/gentoo/dev-python/setuptools/setuptools-69.5.1.ebuild)
1713123232:  === Unmerging... (dev-python/setuptools-69.2.0-r1)
1713123236:  === (48 of 110) Merging (sys-libs/libblockdev-3.1.1::/var/db/repos/gentoo/sys-libs/libblockdev/libblockdev-3.1.1.ebuild)
1713123238:  === Unmerging... (sys-libs/libblockdev-3.0.4)
1713123242:  === (49 of 110) Merging (app-text/poppler-24.04.0::/var/db/repos/gentoo/app-text/poppler/poppler-24.04.0.ebuild)
1713123244:  === Unmerging... (app-text/poppler-24.03.0)
1713125641:  === (52 of 110) Merging (dev-ruby/reline-0.5.1::/var/db/repos/gentoo/dev-ruby/reline/reline-0.5.1.ebuild)
1713125644:  === Unmerging... (dev-ruby/reline-0.5.0)
1713125660:  === (53 of 110) Merging (dev-python/mako-1.3.3::/var/db/repos/gentoo/dev-python/mako/mako-1.3.3.ebuild)
1713125661:  === Unmerging... (dev-python/mako-1.3.2)
1713125665:  === (54 of 110) Merging (dev-lang/vala-0.56.16::/var/db/repos/gentoo/dev-lang/vala/vala-0.56.16.ebuild)
1713125667:  === Unmerging... (dev-lang/vala-0.56.16)
1713125672:  === (55 of 110) Merging (sys-devel/llvm-18.1.3::/var/db/repos/gentoo/sys-devel/llvm/llvm-18.1.3.ebuild)
1713125698:  === Unmerging... (sys-devel/llvm-18.1.2)
1713126377:  === (57 of 110) Merging (media-libs/mesa-24.0.5::/var/db/repos/gentoo/media-libs/mesa/mesa-24.0.5.ebuild)
1713126380:  === Unmerging... (media-libs/mesa-24.0.4)
1713126386:  === (56 of 110) Merging (media-video/ffmpeg-6.1.1-r5::/var/db/repos/gentoo/media-video/ffmpeg/ffmpeg-6.1.1-r5.ebuild)
1713126389:  === Unmerging... (media-video/ffmpeg-6.1.1-r5)
1713126393:  === (58 of 110) Merging (dev-qt/qtbase-6.7.0-r1::/var/db/repos/gentoo/dev-qt/qtbase/qtbase-6.7.0-r1.ebuild)
1713126398:  === Unmerging... (dev-qt/qtbase-6.6.3)
1713126507:  === (61 of 110) Merging (x11-base/xwayland-23.2.6::/var/db/repos/gentoo/x11-base/xwayland/xwayland-23.2.6.ebuild)
1713126509:  === Unmerging... (x11-base/xwayland-23.2.4-r1)
1713126514:  === (59 of 110) Merging (x11-base/xorg-server-21.1.13::/var/db/repos/gentoo/x11-base/xorg-server/xorg-server-21.1.13.ebuild)
1713126515:  === Unmerging... (x11-base/xorg-server-21.1.11)
1713126520:  === (60 of 110) Merging (media-libs/libsdl2-2.30.2::/var/db/repos/gentoo/media-libs/libsdl2/libsdl2-2.30.2.ebuild)
1713126522:  === Unmerging... (media-libs/libsdl2-2.30.1)
1713126525:  === (62 of 110) Merging (dev-qt/qtwayland-6.7.0::/var/db/repos/gentoo/dev-qt/qtwayland/qtwayland-6.7.0.ebuild)
1713126527:  === Unmerging... (dev-qt/qtwayland-6.6.3)
1713126543:  === (63 of 110) Merging (sys-devel/clang-common-18.1.3-r3::/var/db/repos/gentoo/sys-devel/clang-common/clang-common-18.1.3-r3.ebuild)
1713126545:  === Unmerging... (sys-devel/clang-common-18.1.2-r3)
1713129021:  === (66 of 110) Merging (sys-apps/less-643-r2::/var/db/repos/gentoo/sys-apps/less/less-643-r2.ebuild)
1713129025:  === Unmerging... (sys-apps/less-643-r1)
1713129042:  === (65 of 110) Merging (sys-apps/iproute2-6.8.0-r2::/var/db/repos/gentoo/sys-apps/iproute2/iproute2-6.8.0-r2.ebuild)
1713129044:  === Unmerging... (sys-apps/iproute2-6.6.0-r3)
1713129048:  === (67 of 110) Merging (dev-python/lxml-5.2.1::/var/db/repos/gentoo/dev-python/lxml/lxml-5.2.1.ebuild)
1713129050:  === Unmerging... (dev-python/lxml-5.2.0)
1713129054:  === (64 of 110) Merging (sys-devel/clang-18.1.3::/var/db/repos/gentoo/sys-devel/clang/clang-18.1.3.ebuild)
1713129096:  === Unmerging... (sys-devel/clang-18.1.2)
1713129377:  === (70 of 110) Merging (x11-drivers/xf86-input-evdev-2.10.6::/var/db/repos/gentoo/x11-drivers/xf86-input-evdev/xf86-input-evdev-2.10.6.ebuild)
1713129378:  === Unmerging... (x11-drivers/xf86-input-evdev-2.10.6)
1713129382:  === (69 of 110) Merging (dev-ruby/json-2.7.2::/var/db/repos/gentoo/dev-ruby/json/json-2.7.2.ebuild)
1713129384:  === Unmerging... (dev-ruby/json-2.7.1-r1)
1713129388:  === (68 of 110) Merging (sys-libs/compiler-rt-sanitizers-18.1.3::/var/db/repos/gentoo/sys-libs/compiler-rt-sanitizers/compiler-rt-sanitizers-18.1.3.ebuild)
1713129390:  === Unmerging... (sys-libs/compiler-rt-sanitizers-18.1.2)
1713129394:  === (73 of 110) Merging (cross-aarch64-unknown-linux-gnu/binutils-2.42-r1::/var/db/repos/crossdev/cross-aarch64-unknown-linux-gnu/binutils/binutils-2.42-r1.ebuild)
1713129396:  === Unmerging... (cross-aarch64-unknown-linux-gnu/binutils-2.42-r1)
1713129401:  === (71 of 110) Merging (app-emulation/qemu-8.2.2::/var/db/repos/localrepo/app-emulation/qemu/qemu-8.2.2.ebuild)
1713129406:  === Unmerging... (app-emulation/qemu-8.2.2)
1713133758:  === (2 of 37) Merging (dev-ruby/debug-1.9.2::/var/db/repos/gentoo/dev-ruby/debug/debug-1.9.2.ebuild)
1713133761:  === Unmerging... (dev-ruby/debug-1.9.1)
1713133769:  === (3 of 37) Merging (x11-drivers/xf86-input-libinput-1.4.0::/var/db/repos/gentoo/x11-drivers/xf86-input-libinput/xf86-input-libinput-1.4.0.ebuild)
1713133773:  === Unmerging... (x11-drivers/xf86-input-libinput-1.4.0)
1713133781:  === (13 of 37) Merging (dev-python/snakeoil-0.10.8::/var/db/repos/gentoo/dev-python/snakeoil/snakeoil-0.10.8.ebuild)
1713133784:  === Unmerging... (dev-python/snakeoil-0.10.7)
1713133792:  === (5 of 37) Merging (cross-aarch64-unknown-linux-gnu/linux-headers-6.8-r1::/var/db/repos/crossdev/cross-aarch64-unknown-linux-gnu/linux-headers/linux-headers-6.8-r1.ebuild)
1713133795:  === Unmerging... (cross-aarch64-unknown-linux-gnu/linux-headers-6.7-r1)
1713133805:  === (8 of 37) Merging (cross-aarch64-unknown-linux-musl/linux-headers-6.8-r1::/var/db/repos/crossdev/cross-aarch64-unknown-linux-musl/linux-headers/linux-headers-6.8-r1.ebuild)
1713133808:  === Unmerging... (cross-aarch64-unknown-linux-musl/linux-headers-6.7-r1)
1713133818:  === (1 of 37) Merging (sys-libs/compiler-rt-18.1.3::/var/db/repos/gentoo/sys-libs/compiler-rt/compiler-rt-18.1.3.ebuild)
1713133821:  === Unmerging... (sys-libs/compiler-rt-18.1.2)
1713133830:  === (14 of 37) Merging (dev-qt/qtsensors-6.7.0::/var/db/repos/gentoo/dev-qt/qtsensors/qtsensors-6.7.0.ebuild)
1713133834:  === Unmerging... (dev-qt/qtsensors-6.6.3)
1713133842:  === (9 of 37) Merging (cross-x86_64-w64-mingw32/binutils-2.42-r1::/var/db/repos/crossdev/cross-x86_64-w64-mingw32/binutils/binutils-2.42-r1.ebuild)
1713133845:  === Unmerging... (cross-x86_64-w64-mingw32/binutils-2.42-r1)
1713133854:  === (6 of 37) Merging (cross-aarch64-unknown-linux-musl/binutils-2.42-r1::/var/db/repos/crossdev/cross-aarch64-unknown-linux-musl/binutils/binutils-2.42-r1.ebuild)
1713133858:  === Unmerging... (cross-aarch64-unknown-linux-musl/binutils-2.42-r1)
1713133866:  === (12 of 37) Merging (dev-lang/php-8.3.6::/var/db/repos/gentoo/dev-lang/php/php-8.3.6.ebuild)
1713133871:  === Unmerging... (dev-lang/php-8.3.4)
1713133880:  === (11 of 37) Merging (dev-debug/gdb-14.2::/var/db/repos/gentoo/dev-debug/gdb/gdb-14.2.ebuild)
1713133884:  === Unmerging... (dev-debug/gdb-14.2)
1713133892:  === (10 of 37) Merging (cross-x86_64-w64-mingw32/gcc-13.2.1_p20240210::/var/db/repos/crossdev/cross-x86_64-w64-mingw32/gcc/gcc-13.2.1_p20240210.ebuild)
1713133898:  === Unmerging... (cross-x86_64-w64-mingw32/gcc-13.2.1_p20240210)
1713133911:  === (4 of 37) Merging (cross-aarch64-unknown-linux-gnu/gcc-13.2.1_p20240210::/var/db/repos/crossdev/cross-aarch64-unknown-linux-gnu/gcc/gcc-13.2.1_p20240210.ebuild)
1713133916:  === Unmerging... (cross-aarch64-unknown-linux-gnu/gcc-13.2.1_p20240210)
1713133928:  === (7 of 37) Merging (cross-aarch64-unknown-linux-musl/gcc-13.2.1_p20240210::/var/db/repos/crossdev/cross-aarch64-unknown-linux-musl/gcc/gcc-13.2.1_p20240210.ebuild)
1713133933:  === Unmerging... (cross-aarch64-unknown-linux-musl/gcc-13.2.1_p20240210)
1713133949:  === (15 of 37) Merging (sys-devel/clang-runtime-18.1.3::/var/db/repos/gentoo/sys-devel/clang-runtime/clang-runtime-18.1.3.ebuild)
1713133952:  === Unmerging... (sys-devel/clang-runtime-18.1.2)
1713134669:  === (16 of 37) Merging (dev-debug/lldb-18.1.3::/var/db/repos/gentoo/dev-debug/lldb/lldb-18.1.3.ebuild)
1713134677:  === Unmerging... (dev-debug/lldb-18.1.2)
1713134713:  === (17 of 37) Merging (dev-util/vulkan-tools-1.3.280.0::/var/db/repos/gentoo/dev-util/vulkan-tools/vulkan-tools-1.3.280.0.ebuild)
1713134715:  === Unmerging... (dev-util/vulkan-tools-1.3.275)
1713134838:  === (18 of 37) Merging (media-fonts/noto-20240325::/var/db/repos/gentoo/media-fonts/noto/noto-20240325.ebuild)
1713134841:  === Unmerging... (media-fonts/noto-20240229)
1713134851:  === (21 of 37) Merging (net-firewall/nftables-1.0.9::/var/db/repos/gentoo/net-firewall/nftables/nftables-1.0.9.ebuild)
1713134852:  === Unmerging... (net-firewall/nftables-1.0.9)
1713134856:  === (22 of 37) Merging (net-misc/yt-dlp-2024.04.09::/var/db/repos/gentoo/net-misc/yt-dlp/yt-dlp-2024.04.09.ebuild)
1713134860:  === Unmerging... (net-misc/yt-dlp-2024.03.10)
1713134864:  === (20 of 37) Merging (media-libs/gst-plugins-bad-1.22.11-r1::/var/db/repos/gentoo/media-libs/gst-plugins-bad/gst-plugins-bad-1.22.11-r1.ebuild)
1713134866:  === Unmerging... (media-libs/gst-plugins-bad-1.22.11)
1713134870:  === (19 of 37) Merging (media-gfx/imagemagick-7.1.1.25::/var/db/repos/gentoo/media-gfx/imagemagick/imagemagick-7.1.1.25.ebuild)
1713134873:  === Unmerging... (media-gfx/imagemagick-7.1.1.25)
1713135285:  === (27 of 37) Merging (sys-process/iotop-0.6_p20230124::/var/db/repos/gentoo/sys-process/iotop/iotop-0.6_p20230124.ebuild)
1713135287:  === Unmerging... (sys-process/iotop-0.6_p20230124)
1713135291:  === (26 of 37) Merging (sys-process/cronie-1.7.2::/var/db/repos/gentoo/sys-process/cronie/cronie-1.7.2.ebuild)
1713135294:  === Unmerging... (sys-process/cronie-1.7.1-r1)
1713135299:  === (30 of 37) Merging (sys-apps/pciutils-3.12.0::/var/db/repos/gentoo/sys-apps/pciutils/pciutils-3.12.0.ebuild)
1713135300:  === Unmerging... (sys-apps/pciutils-3.11.1)
1713135304:  === (31 of 37) Merging (media-libs/shaderc-2024.0::/var/db/repos/gentoo/media-libs/shaderc/shaderc-2024.0.ebuild)
1713135306:  === Unmerging... (media-libs/shaderc-2023.8)
1713135310:  === (33 of 37) Merging (net-libs/libproxy-0.5.5::/var/db/repos/gentoo/net-libs/libproxy/libproxy-0.5.5.ebuild)
1713135312:  === Unmerging... (net-libs/libproxy-0.5.4-r1)
1713135317:  === (23 of 37) Merging (sys-apps/man-db-2.12.1::/var/db/repos/gentoo/sys-apps/man-db/man-db-2.12.1.ebuild)
1713135318:  === Unmerging... (sys-apps/man-db-2.12.0)
1713135323:  === (25 of 37) Merging (sys-kernel/gentoo-sources-6.8.6::/var/db/repos/gentoo/sys-kernel/gentoo-sources/gentoo-sources-6.8.6.ebuild)
1713135348:  === (28 of 37) Merging (x11-drivers/xf86-video-intel-2.99.917_p20230201::/var/db/repos/gentoo/x11-drivers/xf86-video-intel/xf86-video-intel-2.99.917_p20230201.ebuild)
1713135351:  === Unmerging... (x11-drivers/xf86-video-intel-2.99.917_p20230201)
1713135355:  === (34 of 37) Merging (sys-apps/xdg-desktop-portal-1.18.3::/var/db/repos/gentoo/sys-apps/xdg-desktop-portal/xdg-desktop-portal-1.18.3.ebuild)
1713135356:  === Unmerging... (sys-apps/xdg-desktop-portal-1.18.2)
1713135360:  === (24 of 37) Merging (sys-boot/grub-2.12-r4::/var/db/repos/gentoo/sys-boot/grub/grub-2.12-r4.ebuild)
1713135364:  === Unmerging... (sys-boot/grub-2.12-r2)
1713135368:  === (36 of 37) Merging (dev-libs/botan-2.19.3-r2::/var/db/repos/gentoo/dev-libs/botan/botan-2.19.3-r2.ebuild)
1713135370:  === Unmerging... (dev-libs/botan-2.19.3-r2)
1713135375:  === (35 of 37) Merging (sci-libs/libqalculate-5.0.0::/var/db/repos/gentoo/sci-libs/libqalculate/libqalculate-5.0.0.ebuild)
1713135376:  === Unmerging... (sci-libs/libqalculate-4.8.1)
1713135380:  === (32 of 37) Merging (media-libs/gegl-0.4.48::/var/db/repos/gentoo/media-libs/gegl/gegl-0.4.48.ebuild)
1713135382:  === Unmerging... (media-libs/gegl-0.4.48)
1713135386:  === (29 of 37) Merging (dev-libs/efl-1.27.0::/var/db/repos/gentoo/dev-libs/efl/efl-1.27.0.ebuild)
1713135391:  === Unmerging... (dev-libs/efl-1.27.0)
1713135408:  === (37 of 37) Merging (dev-libs/libgamin-0.1.10-r7::/var/db/repos/gentoo/dev-libs/libgamin/libgamin-0.1.10-r7.ebuild)
1713135410:  === Unmerging... (dev-libs/libgamin-0.1.10-r6)
1713141890: === Unmerging... (dev-python/zstandard-0.22.0)
1713142154:  === (1 of 3) Merging (dev-lang/go-1.22.2::/var/db/repos/gentoo/dev-lang/go/go-1.22.2.ebuild)
1713142164:  === Unmerging... (dev-lang/go-1.22.2)
1713142200:  === (2 of 3) Merging (app-shells/fzf-0.49.0::/var/db/repos/gentoo/app-shells/fzf/fzf-0.49.0.ebuild)
1713142203:  === Unmerging... (app-shells/fzf-0.49.0)
1713142354:  === (3 of 3) Merging (dev-util/android-tools-34.0.5::/var/db/repos/gentoo/dev-util/android-tools/android-tools-34.0.5.ebuild)
1713142359:  === Unmerging... (dev-util/android-tools-34.0.5)
1713142589:  === (1 of 1) Merging (app-misc/mc-4.8.31::/var/db/repos/gentoo/app-misc/mc/mc-4.8.31.ebuild)
1713142593:  === Unmerging... (app-misc/mc-4.8.30-r2)
1713166234:  === (1 of 1) Merging (sys-kernel/genkernel-4.3.10::/var/db/repos/gentoo/sys-kernel/genkernel/genkernel-4.3.10.ebuild)
1713166236:  === Unmerging... (sys-kernel/genkernel-4.3.10)
1713166466:  === (1 of 1) Merging (app-emulation/virtualbox-modules-7.0.14-r1::/var/db/repos/gentoo/app-emulation/virtualbox-modules/virtualbox-modules-7.0.14-r1.ebuild)
1713166467:  === Unmerging... (app-emulation/virtualbox-modules-7.0.14-r1)

I believe I've found the log entry for this one:

2024/04/14 (20:11:47) | ===> btop++ v.1.3.2
2024/04/14 (20:11:47) | INFO: Logger set to DEBUG
2024/04/14 (20:11:47) | DEBUG: Using locale en_US.utf8
2024/04/14 (20:11:47) | INFO: Running on /dev/pts/0
2024/04/14 (20:11:47) | INFO: Failed to load libnvidia-ml.so, NVIDIA GPUs will not be detected: libnvidia-ml.so.1: cannot open shared object file: No such file or directory
2024/04/14 (20:11:47) | INFO: Failed to load librocm_smi64.so, AMD GPUs will not be detected: librocm_smi64.so.6: cannot open shared object file: No such file or directory
2024/04/14 (20:11:47) | DEBUG: Shared::init() : Initialized.
2024/04/15 (00:14:53) | ERROR: Stall in Runner thread, restarting!

from btop.

correabuscar avatar correabuscar commented on June 16, 2024

I got another hang but without any cpu usage (but the uptime was 6+ days, same system), it is hung for like 8 hours and i just noticed:

(gdb) thread apply all bt

Thread 2 (Thread 0x7ce1172006c0 (LWP 380176) "btop"):
#0  syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
#1  0x00005a51fc3f7e2b in std::__detail::__platform_wait<int> (__addr=0x5a51fc51c788 <Runner::do_work>, __val=0) at /usr/lib/gcc/x86_64-pc-linux-gnu/13/include/g++-v13/bits/atomic_wait.h:111
#2  std::__atomic_wait_address_bare<std::__atomic_semaphore::_M_acquire()::{lambda()#1}>(int const*, std::__atomic_semaphore::_M_acquire()::{lambda()#1}) (__addr=<optimized out>, __pred=...) at /usr/lib/gcc/x86_64-pc-linux-gnu/13/include/g++-v13/bits/atomic_wait.h:449
#3  std::__atomic_semaphore::_M_acquire (this=<optimized out>) at /usr/lib/gcc/x86_64-pc-linux-gnu/13/include/g++-v13/bits/semaphore_base.h:215
#4  std::counting_semaphore<1l>::acquire (this=<optimized out>) at /usr/lib/gcc/x86_64-pc-linux-gnu/13/include/g++-v13/semaphore:74
#5  Runner::thread_wait () at /usr/src/debug/sys-process/btop-1.3.2/btop-1.3.2/src/btop.cpp:418
#6  Runner::_runner () at /usr/src/debug/sys-process/btop-1.3.2/btop-1.3.2/src/btop.cpp:538
#7  0x00007ce11a1aaff1 in start_thread (arg=<optimized out>) at pthread_create.c:447
#8  0x00007ce11a22aadc in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78

Thread 1 (Thread 0x7ce11a6a6740 (LWP 16565) "btop"):
#0  0x00007ce11a2275aa in pselect64_syscall (sigmask=<optimized out>, timeout=<optimized out>, exceptfds=0x0, writefds=0x0, readfds=0x7ffe8696ee60, nfds=1) at ../sysdeps/unix/sysv/linux/pselect.c:34
#1  __pselect (nfds=1, readfds=0x7ffe8696ee60, writefds=0x0, exceptfds=0x0, timeout=<optimized out>, sigmask=<optimized out>) at ../sysdeps/unix/sysv/linux/pselect.c:56
#2  0x00005a51fc44caf8 in Input::poll (timeout=<optimized out>) at /usr/src/debug/sys-process/btop-1.3.2/btop-1.3.2/src/btop_input.cpp:106
#3  0x00005a51fc3da5d0 in main (argc=<optimized out>, argv=<optimized out>) at /usr/src/debug/sys-process/btop-1.3.2/btop-1.3.2/src/btop.cpp:1165
(gdb)
(gdb) thread apply all bt2

Thread 2 (Thread 0x7ce1172006c0 (LWP 380176) "btop"):
executing: 'frame apply all -q frame'
#0  syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
38		cmpq $-4095, %rax	/* Check %rax for error.  */
#1  0x00005a51fc3f7e2b in std::__detail::__platform_wait<int> (__addr=0x5a51fc51c788 <Runner::do_work>, __val=0) at /usr/lib/gcc/x86_64-pc-linux-gnu/13/include/g++-v13/bits/atomic_wait.h:111
111	      __platform_wait(const _Tp* __addr, __platform_wait_t __val) noexcept
#2  std::__atomic_wait_address_bare<std::__atomic_semaphore::_M_acquire()::{lambda()#1}>(int const*, std::__atomic_semaphore::_M_acquire()::{lambda()#1}) (__addr=<optimized out>, __pred=...) at /usr/lib/gcc/x86_64-pc-linux-gnu/13/include/g++-v13/bits/atomic_wait.h:449
449		  __detail::__platform_wait(__addr, __val);
#3  std::__atomic_semaphore::_M_acquire (this=<optimized out>) at /usr/lib/gcc/x86_64-pc-linux-gnu/13/include/g++-v13/bits/semaphore_base.h:215
215	      std::__atomic_wait_address_bare(&_M_counter, __pred);
#4  std::counting_semaphore<1l>::acquire (this=<optimized out>) at /usr/lib/gcc/x86_64-pc-linux-gnu/13/include/g++-v13/semaphore:74
74	      { _M_sem._M_acquire(); }
#5  Runner::thread_wait () at /usr/src/debug/sys-process/btop-1.3.2/btop-1.3.2/src/btop.cpp:418
418		inline void thread_wait() { do_work.acquire(); }
#6  Runner::_runner () at /usr/src/debug/sys-process/btop-1.3.2/btop-1.3.2/src/btop.cpp:538
538				thread_wait();
#7  0x00007ce11a1aaff1 in start_thread (arg=<optimized out>) at pthread_create.c:447
447		ret = pd->start_routine (pd->arg);
#8  0x00007ce11a22aadc in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78
78		call	*%rdx		/* Call function.  */

Thread 1 (Thread 0x7ce11a6a6740 (LWP 16565) "btop"):
executing: 'frame apply all -q frame'
#0  0x00007ce11a2275aa in pselect64_syscall (sigmask=<optimized out>, timeout=<optimized out>, exceptfds=0x0, writefds=0x0, readfds=0x7ffe8696ee60, nfds=1) at ../sysdeps/unix/sysv/linux/pselect.c:34
34	  return SYSCALL_CANCEL (pselect6_time64, nfds, readfds, writefds, exceptfds,
#1  __pselect (nfds=1, readfds=0x7ffe8696ee60, writefds=0x0, exceptfds=0x0, timeout=<optimized out>, sigmask=<optimized out>) at ../sysdeps/unix/sysv/linux/pselect.c:56
56	  return pselect64_syscall (nfds, readfds, writefds, exceptfds, timeout,
#2  0x00005a51fc44caf8 in Input::poll (timeout=<optimized out>) at /usr/src/debug/sys-process/btop-1.3.2/btop-1.3.2/src/btop_input.cpp:106
106			if(pselect(STDIN_FILENO + 1, &fds, nullptr, nullptr, waitptr, &signal_mask) > 0) {
#3  0x00005a51fc3da5d0 in main (argc=<optimized out>, argv=<optimized out>) at /usr/src/debug/sys-process/btop-1.3.2/btop-1.3.2/src/btop.cpp:1165
1165					else if (Input::poll(min((uint64_t)1000, future_time - current_time))) {
(gdb)
(gdb) thread apply all bt3

Thread 2 (Thread 0x7ce1172006c0 (LWP 380176) "btop"):
executing: 'frame apply all list *$pc' (but line numbers are a bit innacurate!)
#0  syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
0x7ce11a22882d is in syscall (../sysdeps/unix/sysv/linux/x86_64/syscall.S:38).
33		movq %rcx, %rdx
34		movq %r8, %r10
35		movq %r9, %r8
36		movq 8(%rsp),%r9	/* arg6 is on the stack.  */
37		syscall			/* Do the system call.  */
38		cmpq $-4095, %rax	/* Check %rax for error.  */
39		jae SYSCALL_ERROR_LABEL	/* Jump to error handler if error.  */
40		ret			/* Return to caller.  */
41	
42	PSEUDO_END (syscall)
#1  0x00005a51fc3f7e2b in std::__detail::__platform_wait<int> (__addr=0x5a51fc51c788 <Runner::do_work>, __val=0) at /usr/lib/gcc/x86_64-pc-linux-gnu/13/include/g++-v13/bits/atomic_wait.h:111
0x5a51fc3f7e2b is in Runner::_runner(void*) (/usr/lib/gcc/x86_64-pc-linux-gnu/13/include/g++-v13/bits/atomic_wait.h:116).
111	      __platform_wait(const _Tp* __addr, __platform_wait_t __val) noexcept
112	      {
113		auto __e = syscall (SYS_futex, static_cast<const void*>(__addr),
114				    static_cast<int>(__futex_wait_flags::__wait_private),
115				    __val, nullptr);
116		if (!__e || errno == EAGAIN)
117		  return;
118		if (errno != EINTR)
119		  __throw_system_error(errno);
120	      }
#2  std::__atomic_wait_address_bare<std::__atomic_semaphore::_M_acquire()::{lambda()#1}>(int const*, std::__atomic_semaphore::_M_acquire()::{lambda()#1}) (__addr=<optimized out>, __pred=...) at /usr/lib/gcc/x86_64-pc-linux-gnu/13/include/g++-v13/bits/atomic_wait.h:449
0x5a51fc3f7e2b is in Runner::_runner(void*) (/usr/lib/gcc/x86_64-pc-linux-gnu/13/include/g++-v13/bits/atomic_wait.h:116).
111	      __platform_wait(const _Tp* __addr, __platform_wait_t __val) noexcept
112	      {
113		auto __e = syscall (SYS_futex, static_cast<const void*>(__addr),
114				    static_cast<int>(__futex_wait_flags::__wait_private),
115				    __val, nullptr);
116		if (!__e || errno == EAGAIN)
117		  return;
118		if (errno != EINTR)
119		  __throw_system_error(errno);
120	      }
#3  std::__atomic_semaphore::_M_acquire (this=<optimized out>) at /usr/lib/gcc/x86_64-pc-linux-gnu/13/include/g++-v13/bits/semaphore_base.h:215
0x5a51fc3f7e2b is in Runner::_runner(void*) (/usr/lib/gcc/x86_64-pc-linux-gnu/13/include/g++-v13/bits/atomic_wait.h:116).
111	      __platform_wait(const _Tp* __addr, __platform_wait_t __val) noexcept
112	      {
113		auto __e = syscall (SYS_futex, static_cast<const void*>(__addr),
114				    static_cast<int>(__futex_wait_flags::__wait_private),
115				    __val, nullptr);
116		if (!__e || errno == EAGAIN)
117		  return;
118		if (errno != EINTR)
119		  __throw_system_error(errno);
120	      }
#4  std::counting_semaphore<1l>::acquire (this=<optimized out>) at /usr/lib/gcc/x86_64-pc-linux-gnu/13/include/g++-v13/semaphore:74
0x5a51fc3f7e2b is in Runner::_runner(void*) (/usr/lib/gcc/x86_64-pc-linux-gnu/13/include/g++-v13/bits/atomic_wait.h:116).
111	      __platform_wait(const _Tp* __addr, __platform_wait_t __val) noexcept
112	      {
113		auto __e = syscall (SYS_futex, static_cast<const void*>(__addr),
114				    static_cast<int>(__futex_wait_flags::__wait_private),
115				    __val, nullptr);
116		if (!__e || errno == EAGAIN)
117		  return;
118		if (errno != EINTR)
119		  __throw_system_error(errno);
120	      }
#5  Runner::thread_wait () at /usr/src/debug/sys-process/btop-1.3.2/btop-1.3.2/src/btop.cpp:418
0x5a51fc3f7e2b is in Runner::_runner(void*) (/usr/lib/gcc/x86_64-pc-linux-gnu/13/include/g++-v13/bits/atomic_wait.h:116).
111	      __platform_wait(const _Tp* __addr, __platform_wait_t __val) noexcept
112	      {
113		auto __e = syscall (SYS_futex, static_cast<const void*>(__addr),
114				    static_cast<int>(__futex_wait_flags::__wait_private),
115				    __val, nullptr);
116		if (!__e || errno == EAGAIN)
117		  return;
118		if (errno != EINTR)
119		  __throw_system_error(errno);
120	      }
#6  Runner::_runner () at /usr/src/debug/sys-process/btop-1.3.2/btop-1.3.2/src/btop.cpp:538
0x5a51fc3f7e2b is in Runner::_runner(void*) (/usr/lib/gcc/x86_64-pc-linux-gnu/13/include/g++-v13/bits/atomic_wait.h:116).
111	      __platform_wait(const _Tp* __addr, __platform_wait_t __val) noexcept
112	      {
113		auto __e = syscall (SYS_futex, static_cast<const void*>(__addr),
114				    static_cast<int>(__futex_wait_flags::__wait_private),
115				    __val, nullptr);
116		if (!__e || errno == EAGAIN)
117		  return;
118		if (errno != EINTR)
119		  __throw_system_error(errno);
120	      }
#7  0x00007ce11a1aaff1 in start_thread (arg=<optimized out>) at pthread_create.c:447
0x7ce11a1aaff1 is in start_thread (pthread_create.c:447).
442		     to void are able to represent all values of int.  */
443		  int (*start)(void*) = (int (*) (void*)) pd->start_routine;
444		  ret = (void*) (uintptr_t) start (pd->arg);
445		}
446	      else
447		ret = pd->start_routine (pd->arg);
448	      THREAD_SETMEM (pd, result, ret);
449	    }
450	
451	  /* Call destructors for the thread_local TLS variables.  */
#8  0x00007ce11a22aadc in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78
0x7ce11a22aadc is in clone3 (../sysdeps/unix/sysv/linux/x86_64/clone3.S:80).
75	
76		/* Set up arguments for the function call.  */
77		mov	%R8_LP, %RDI_LP	/* Argument.  */
78		call	*%rdx		/* Call function.  */
79		/* Call exit with return value from function call. */
80		movq	%rax, %rdi
81		movl	$SYS_ify(exit), %eax
82		syscall
83		cfi_endproc
84	

Thread 1 (Thread 0x7ce11a6a6740 (LWP 16565) "btop"):
executing: 'frame apply all list *$pc' (but line numbers are a bit innacurate!)
#0  0x00007ce11a2275aa in pselect64_syscall (sigmask=<optimized out>, timeout=<optimized out>, exceptfds=0x0, writefds=0x0, readfds=0x7ffe8696ee60, nfds=1) at ../sysdeps/unix/sysv/linux/pselect.c:34
0x7ce11a2275aa is in __pselect (../sysdeps/unix/sysv/linux/pselect.c:34).
29	  /* NB: This is required by ARGIFY used in x32 internal_syscallN.  */
30	  __syscall_ulong_t data[2] =
31	    {
32	      (uintptr_t) sigmask, __NSIG_BYTES
33	    };
34	  return SYSCALL_CANCEL (pselect6_time64, nfds, readfds, writefds, exceptfds,
35				 timeout, data);
36	}
37	
38	int
#1  __pselect (nfds=1, readfds=0x7ffe8696ee60, writefds=0x0, exceptfds=0x0, timeout=<optimized out>, sigmask=<optimized out>) at ../sysdeps/unix/sysv/linux/pselect.c:56
0x7ce11a2275aa is in __pselect (../sysdeps/unix/sysv/linux/pselect.c:34).
29	  /* NB: This is required by ARGIFY used in x32 internal_syscallN.  */
30	  __syscall_ulong_t data[2] =
31	    {
32	      (uintptr_t) sigmask, __NSIG_BYTES
33	    };
34	  return SYSCALL_CANCEL (pselect6_time64, nfds, readfds, writefds, exceptfds,
35				 timeout, data);
36	}
37	
38	int
#2  0x00005a51fc44caf8 in Input::poll (timeout=<optimized out>) at /usr/src/debug/sys-process/btop-1.3.2/btop-1.3.2/src/btop_input.cpp:106
0x5a51fc44caf8 is in Input::poll(unsigned long) (/usr/src/debug/sys-process/btop-1.3.2/btop-1.3.2/src/btop_input.cpp:106).
101				wait.tv_sec = timeout / 1000;
102				wait.tv_nsec = (timeout % 1000) * 1000000;
103				waitptr = &wait;
104			}
105	
106			if(pselect(STDIN_FILENO + 1, &fds, nullptr, nullptr, waitptr, &signal_mask) > 0) {
107				input.clear();
108				char buf[1024];
109				ssize_t count = 0;
110				while((count = read(STDIN_FILENO, buf, sizeof(buf))) > 0) {
#3  0x00005a51fc3da5d0 in main (argc=<optimized out>, argv=<optimized out>) at /usr/src/debug/sys-process/btop-1.3.2/btop-1.3.2/src/btop.cpp:1165
0x5a51fc3da5d0 is in main(int, char**) (/usr/src/debug/sys-process/btop-1.3.2/btop-1.3.2/src/btop.cpp:1165).
1160					}
1161					else if (future_time - current_time > update_ms) {
1162						future_time = current_time;
1163					}
1164					//? Poll for input and process any input detected
1165					else if (Input::poll(min((uint64_t)1000, future_time - current_time))) {
1166						if (not Runner::active) Config::unlock();
1167	
1168						if (Menu::active) Menu::process(Input::get());
1169						else Input::process(Input::get());
(gdb)
(gdb) thread apply all bt4

Thread 2 (Thread 0x7ce1172006c0 (LWP 380176) "btop"):
executing: 'bt -full -frame-info source-and-location -past-entry on -entry-values both -frame-arguments all -past-main on'
#0  syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
38		cmpq $-4095, %rax	/* Check %rax for error.  */
No locals.
#1  0x00005a51fc3f7e2b in std::__detail::__platform_wait<int> (__addr=0x5a51fc51c788 <Runner::do_work>, __addr@entry=<optimized out>, __val=0, __val@entry=<optimized out>) at /usr/lib/gcc/x86_64-pc-linux-gnu/13/include/g++-v13/bits/atomic_wait.h:111
111	      __platform_wait(const _Tp* __addr, __platform_wait_t __val) noexcept
        __e = <optimized out>
        __e = <optimized out>
#2  std::__atomic_wait_address_bare<std::__atomic_semaphore::_M_acquire()::{lambda()#1}>(int const*, std::__atomic_semaphore::_M_acquire()::{lambda()#1}) (__addr=<optimized out>, __addr@entry=<optimized out>, __pred=<optimized out>, __pred@entry=<optimized out>) at /usr/lib/gcc/x86_64-pc-linux-gnu/13/include/g++-v13/bits/atomic_wait.h:449
449		  __detail::__platform_wait(__addr, __val);
        __val = 0
#3  std::__atomic_semaphore::_M_acquire (this=<optimized out>, this@entry=<optimized out>) at /usr/lib/gcc/x86_64-pc-linux-gnu/13/include/g++-v13/bits/semaphore_base.h:215
215	      std::__atomic_wait_address_bare(&_M_counter, __pred);
        __pred = <optimized out>
        __pred = <optimized out>
#4  std::counting_semaphore<1l>::acquire (this=<optimized out>, this@entry=<optimized out>) at /usr/lib/gcc/x86_64-pc-linux-gnu/13/include/g++-v13/semaphore:74
74	      { _M_sem._M_acquire(); }
No locals.
#5  Runner::thread_wait () at /usr/src/debug/sys-process/btop-1.3.2/btop-1.3.2/src/btop.cpp:418
418		inline void thread_wait() { do_work.acquire(); }
No locals.
#6  Runner::_runner () at /usr/src/debug/sys-process/btop-1.3.2/btop-1.3.2/src/btop.cpp:538
538				thread_wait();
        lck = {atom = std::atomic<bool> = { false }, not_true = false}
        powers = <optimized out>
        conf = <optimized out>
        pt_lck = {pt_mutex = @0x5a51fc51c7a0, status = <optimized out>}
        loc = <error reading variable: Cannot access memory at address 0x20>
#7  0x00007ce11a1aaff1 in start_thread (arg=<optimized out>, arg@entry=<optimized out>) at pthread_create.c:447
447		ret = pd->start_routine (pd->arg);
        ret = <optimized out>
        pd = <optimized out>
        out = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {137306197460672, -7015127914872298842, 137306197460672, -152, 22, 140731156459552, -7015127914817772890, -7015103680042884442}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
#8  0x00007ce11a22aadc in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78
78		call	*%rdx		/* Call function.  */
No locals.

Thread 1 (Thread 0x7ce11a6a6740 (LWP 16565) "btop"):
executing: 'bt -full -frame-info source-and-location -past-entry on -entry-values both -frame-arguments all -past-main on'
#0  0x00007ce11a2275aa in pselect64_syscall (sigmask=<optimized out>, sigmask@entry=<optimized out>, timeout=<optimized out>, timeout@entry=<optimized out>, exceptfds=0x0, exceptfds@entry=<optimized out>, writefds=0x0, writefds@entry=<optimized out>, readfds=0x7ffe8696ee60, readfds@entry=<optimized out>, nfds=1, nfds@entry=<optimized out>) at ../sysdeps/unix/sysv/linux/pselect.c:34
34	  return SYSCALL_CANCEL (pselect6_time64, nfds, readfds, writefds, exceptfds,
        sc_ret = -514
        sc_cancel_oldtype = 0
        sc_ret = <optimized out>
        data = {99308172067968, 8}
        data = <optimized out>
        sc_ret = <optimized out>
        sc_ret = <optimized out>
        resultvar = <optimized out>
        __arg6 = <optimized out>
        __arg5 = <optimized out>
        __arg4 = <optimized out>
        __arg3 = <optimized out>
        __arg2 = <optimized out>
        __arg1 = <optimized out>
        _a6 = <optimized out>
        _a5 = <optimized out>
        _a4 = <optimized out>
        _a3 = <optimized out>
        _a2 = <optimized out>
        _a1 = <optimized out>
        sc_cancel_oldtype = <optimized out>
        sc_ret = <optimized out>
        resultvar = <optimized out>
        __arg6 = <optimized out>
        __arg5 = <optimized out>
        __arg4 = <optimized out>
        __arg3 = <optimized out>
        __arg2 = <optimized out>
        __arg1 = <optimized out>
        _a6 = <optimized out>
        _a5 = <optimized out>
        _a4 = <optimized out>
        _a3 = <optimized out>
        _a2 = <optimized out>
        _a1 = <optimized out>
#1  __pselect (nfds=1, nfds@entry=<optimized out>, readfds=0x7ffe8696ee60, readfds@entry=<optimized out>, writefds=0x0, writefds@entry=<optimized out>, exceptfds=0x0, exceptfds@entry=<optimized out>, timeout=<optimized out>, timeout@entry=<optimized out>, sigmask=<optimized out>, sigmask@entry=<optimized out>) at ../sysdeps/unix/sysv/linux/pselect.c:56
56	  return pselect64_syscall (nfds, readfds, writefds, exceptfds, timeout,
        tval = {tv_sec = 0, tv_nsec = 328879256}
#2  0x00005a51fc44caf8 in Input::poll (timeout=<optimized out>, timeout@entry=<optimized out>) at /usr/src/debug/sys-process/btop-1.3.2/btop-1.3.2/src/btop_input.cpp:106
106			if(pselect(STDIN_FILENO + 1, &fds, nullptr, nullptr, waitptr, &signal_mask) > 0) {
        lck = {atom = std::atomic<bool> = { true }, not_true = false}
        fds = {fds_bits = {1, 0 <repeats 15 times>}}
        wait = {tv_sec = 1, tv_nsec = 0}
        waitptr = <optimized out>
        __d = <optimized out>
#3  0x00005a51fc3da5d0 in main (argc=1, argc@entry=1, argv=0x7ffe8696f638, argv@entry=0x7ffe8696f638) at /usr/src/debug/sys-process/btop-1.3.2/btop-1.3.2/src/btop.cpp:1165
1165					else if (Input::poll(min((uint64_t)1000, future_time - current_time))) {
        current_time = <optimized out>
        mask = {__val = {512, 99308171858400, 30, 111567764994150, 99308185212096, 0, 16098195381762339, 7, 99308185313280, 125, 125, 140731156460744, 140731156460752, 7, 14690799022258211, 120}}
        update_ms = 1000
        future_time = 1715380758515
#4  0x00007ce11a14444a in __libc_start_call_main (main=0x5a51fc3d99f0 <main(int, char**)>, main@entry=0x5a51fc3d99f0 <main(int, char**)>, argc=1, argc@entry=1, argv=0x7ffe8696f638, argv@entry=0x7ffe8696f638) at ../sysdeps/nptl/libc_start_call_main.h:58
58	      result = main (argc, argv, __environ MAIN_AUXVEC_PARAM);
        self = <optimized out>
        result = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140731156461112, -7450535678886272346, 1, 0, 137306253025280, 99308172041080, -7450535678900952410, -7015103798918110554}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x7ce11a6fe000 <_rtld_local>}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
#5  0x00007ce11a14450b in __libc_start_main_impl (main=0x5a51fc3d99f0 <main(int, char**)>, main@entry=<optimized out>, argc=1, argc@entry=<optimized out>, argv=0x7ffe8696f638, argv@entry=<optimized out>, init=<optimized out>, init@entry=<optimized out>, fini=<optimized out>, fini@entry=<optimized out>, rtld_fini=<optimized out>, rtld_fini@entry=<optimized out>, stack_end=0x7ffe8696f628, stack_end@entry=<optimized out>) at ../csu/libc-start.c:360
360	  __libc_start_call_main (main, argc, argv MAIN_AUXVEC_PARAM);
No locals.
#6  0x00005a51fc3f0ef5 in _start ()
No symbol table info available.
(gdb)
$ btop --version
btop version: 1.3.2

sys-process/btop-1.3.2::gentoo

This time for sure I didn't update anything of the system. But I did install one new package: app-misc/jq-1.7.1 (Description: A lightweight and flexible command-line JSON processor)

Hope this helps? :)

I haven't killed it yet, let me know if you need me to try something else on it?

Ah looks like i forgot this even existed, anyway here's what I've found:

2024/05/04 (09:01:00) | ===> btop++ v.1.3.2
2024/05/04 (09:01:00) | INFO: Logger set to DEBUG
2024/05/04 (09:01:00) | DEBUG: Using locale en_US.utf8
2024/05/04 (09:01:00) | INFO: Running on /dev/pts/2
2024/05/04 (09:01:00) | INFO: Failed to load libnvidia-ml.so, NVIDIA GPUs will not be detected: libnvidia-ml.so.1: cannot open shared object file: No such file or directory
2024/05/04 (09:01:00) | INFO: Failed to load librocm_smi64.so, AMD GPUs will not be detected: librocm_smi64.so.6: cannot open shared object file: No such file or directory
2024/05/04 (09:01:00) | DEBUG: Shared::init() : Initialized.
2024/05/10 (16:51:39) | ERROR: Stall in Runner thread, restarting!
/home/user/.config/btop/btop.log lines 4306-4345/4345 (END)

Anyway here's the screen, frozen:
screen-2024-05-11-09-36-21

so finally pressing q after all this time, did insta quit it, with no high cpu usage or anything.

from btop.

correabuscar avatar correabuscar commented on June 16, 2024

I haven't killed it yet, let me know if you need me to try something else on it?

from btop.

correabuscar avatar correabuscar commented on June 16, 2024

I'm by far no expert (I mean look at this to see how noobish I r), but i took a look at the code and the handling of atomics seems wrong in some places, like you have:

-       if (Global::quitting) return;
-       Global::quitting = true;

instead of something like compare_exchange:

+	bool expected = false;
+	if (!Global::quitting.compare_exchange_strong(expected, true)) {
+		//return of false ^ means the exchange wasn't made
+		// because expected value wasn't false
+		assert_eq(true,expected);//aka the prev. value was 'true'
+			return; // quitting was already true
+	} else {
+		assert_eq(false,expected);//prev. value
+	}

But anyway, the problem in my ante-prev comment with 100% cpu usage on quit, happens if the runner thread got restarted at least once(I emulate this in the patch below by using activated_once atomic), and thus to make it quit instead of use 100% CPU forever, a quick hack is adding thread_trigger(); right before the last statement stopping = false; in the Runner::stop(), but I'm not sure if fixing the other atomics' usage is also needed as well, too ( :D ) and whether a sleep_ms() is needed or not after the pthread_cancel since it's doing things async... so I've left more things for you to consider.

Anyway this is what I did as mentioned:

diff --git a/src/btop.cpp b/src/btop.cpp
index 8eae107..355e22d 100644
--- a/src/btop.cpp
+++ b/src/btop.cpp
@@ -56,6 +56,10 @@ tab-size = 4
 #include "fmt/core.h"
 #include "fmt/ostream.h"
 
+#include <cassert>
+// Define custom assert_eq macro
+#define assert_eq(expected, actual) assert((expected) == (actual))
+
 using std::atomic;
 using std::cout;
 using std::flush;
@@ -289,8 +293,14 @@ void term_resize(bool force) {
 
 //* Exit handler; stops threads, restores terminal and saves config changes
 void clean_quit(int sig) {
-	if (Global::quitting) return;
-	Global::quitting = true;
+	bool expected = false;
+	if (!Global::quitting.compare_exchange_strong(expected, true)) {
+		//return of false ^ means the exchange wasn't made because expected value wasn't false
+		assert_eq(true,expected);//aka the prev. value was 'true'
+			return; // quitting was already true
+	} else {
+		assert_eq(false,expected);//prev. value
+	}
 	Runner::stop();
 	if (Global::_runner_started) {
 	#if defined __APPLE__ || defined __OpenBSD__
@@ -419,6 +429,7 @@ void init_config(){
 //* Manages secondary thread for collection and drawing of boxes
 namespace Runner {
 	atomic<bool> active (false);
+	atomic<bool> activated_once (false);
 	atomic<bool> stopping (false);
 	atomic<bool> waiting (false);
 	atomic<bool> redraw (false);
@@ -808,16 +819,33 @@ namespace Runner {
 	//* Runs collect and draw in a secondary thread, unlocks and locks config to update cached values
 	void run(const string& box, bool no_update, bool force_redraw) {
 		atomic_wait_for(active, true, 5000);
-		if (active) {
+		//^ so that waits for 5 sec or until active becomes 'true', whichever's first.
+		if (!activated_once) {//FIXME: remove this, was emulating this: https://github.com/aristocratos/btop/issues/47#issuecomment-2056046776  which requires pressing 'q' to quit, once this happens.
+			active=true;
+			activated_once=true;
+		}
+		bool expected_active=true;
+		if (active.compare_exchange_strong(expected_active, false)) {
+			//^ true return means was changed from 'true' to 'false'
+			assert_eq(true,expected_active);//prev value was.
+		//if (active) {
 			Logger::error("Stall in Runner thread, restarting!");
-			active = false;
 			// exit(1);
 			pthread_cancel(Runner::runner_id);
+			sleep_ms(1000);//TODO: ^ that's async, do we need to wait at all before creating new?
 			if (pthread_create(&Runner::runner_id, nullptr, &Runner::_runner, nullptr) != 0) {
+				Logger::error("Failed to re-create _runner thread!1!");
 				Global::exit_error_msg = "Failed to re-create _runner thread!";
 				clean_quit(1);
+				Logger::error("Failed to re-create _runner thread!2!");
 			}
+			//active = false;
+			Logger::error("Succeeded in restarting runner thread!");
+		} else {
+			//^ false means was NOT changed from 'true' to 'false', because prev. value was false
+			assert_eq(false,expected_active);//prev. value was.
 		}
+
 		if (stopping or Global::resized) return;
 
 		if (box == "overlay") {
@@ -849,29 +877,55 @@ namespace Runner {
 
 	//* Stops any work being done in runner thread and checks for thread errors
 	void stop() {
+			Logger::error("stop() starting");
 		stopping = true;
 		int ret = pthread_mutex_trylock(&mtx);
+		bool expected_active=true;
 		if (ret != EBUSY and not Global::quitting) {
-			if (active) active = false;
+			if (active.compare_exchange_strong(expected_active,false)) {
+				//was changed, so:
+				assert_eq(true, expected_active);//prev value was.
+			} else {
+				//wasn't changed because prev. value was not the expected 'true' one
+				assert_eq(false, expected_active);
+			}
 			Global::exit_error_msg = "Runner thread died unexpectedly!";
+			Logger::error("stop() Runner thread died unexpectedly! quitting clean");
 			clean_quit(1);
 		}
 		else if (ret == EBUSY) {
+			Logger::error("stop() EBUSY");
 			atomic_wait_for(active, true, 5000);
-			if (active) {
-				active = false;
+			if (active.compare_exchange_strong(expected_active, false)) {
+				//was changed
+				assert_eq(true, expected_active);//prev. value was.
+					Logger::error("stop() was active");
+			//if (active) {
+				//active = false;
 				if (Global::quitting) {
+					Logger::error("stop() quitting");
 					return;
 				}
 				else {
 					Global::exit_error_msg = "No response from Runner thread, quitting!";
+					Logger::error("stop() No response from Runner thread, quitting!");
 					clean_quit(1);
 				}
+			} else {
+				//wasn't changed, so:
+				assert_eq(false, expected_active);//this was the prev. value
 			}
+
+			Logger::error("stop() about to thread_trigger");
 			thread_trigger();
+			Logger::error("stop() waiting for active=false");
 			atomic_wait_for(active, false, 100);
+			Logger::error("stop() waited for active=false");
 			atomic_wait_for(active, true, 100);
+			Logger::error("stop() waited for active=true");
 		}
+			thread_trigger();
+			Logger::error("stop() stopping false");
 		stopping = false;
 	}
 

EDIT: any updates to the above patch would be here or the one at the time of this edit permalinked (somewhat already different from above)

from btop.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.