redhat-plumbers / systemd-rhel8 Goto Github PK

a systemd-udevd crash in udev_monitor_receive_device()

version:systemd-239-41.el8_3.1.x86_64

#gdb /usr/lib/systemd/systemd-udevd /var/lib/systemd/coredump/core.systemd-udevd.0.0d9072fbe44241cb98b5c3bf8185ce26.1144.1615470074000000
(gdb) where
#0  0x00007fa943b79785 in recvmsg () from /lib64/libpthread.so.0
#1  0x00007fa94466f6b9 in udev_monitor_receive_device () from /usr/lib/systemd/libsystemd-shared-239.so
#2  0x00005595b3e17c91 in on_uevent ()
#3  0x00005595b3e18afc in on_inotify ()
#4  0x00007fa944773b40 in source_dispatch () from /usr/lib/systemd/libsystemd-shared-239.so
#5  0x00007fa9447758a5 in sd_event_dispatch () from /usr/lib/systemd/libsystemd-shared-239.so
#6  0x00007fa944775a30 in sd_event_run () from /usr/lib/systemd/libsystemd-shared-239.so
#7  0x00007fa944775c53 in sd_event_loop () from /usr/lib/systemd/libsystemd-shared-239.so
#8  0x00005595b3e14f0c in main ()
(gdb)

PR #136 and #170 were manually reverted. Please resubmit these PRs again against master branch and rhel-8.5.0 if needed.

systemd version the issue has been seen with

239

Used distribution

CentOS 8

Expected behaviour you didn't see

dig hrz.uni-bonn.de @127.0.0.53 A should always respond with an A record of hrz.uni-bonn.de

Unexpected behaviour you saw

systemd-resolved may respond with an A record of rhrz.uni-bonn.de and a DNAME entry instead.
Also, systemd-resolved reply depends on previous queries.

Steps to reproduce the problem
Case A

# resolvectl flush-caches
# dig rhrz.uni-bonn.de @127.0.0.53 A
;; ANSWER SECTION:
rhrz.uni-bonn.de.       86400   IN      A       131.220.14.100
# dig hrz.uni-bonn.de @127.0.0.53 DNAME
;; ANSWER SECTION:
hrz.uni-bonn.de.        86400   IN      DNAME   rhrz.uni-bonn.de.
# dig hrz.uni-bonn.de @127.0.0.53 A
;; ANSWER SECTION:
hrz.uni-bonn.de.        7167    IN      DNAME   rhrz.uni-bonn.de.
rhrz.uni-bonn.de.       7163    IN      A       131.220.14.100

Note that the final reply here does not answer the query for an A record of hrz.uni-bonn.de directly, but requires further evaluation on the client end.
This reply is completely ignored by some clients, notably, mDNSresponder on macOS, since it does not answer the actual question (only indirectly).

Case B

# dig hrz.uni-bonn.de @127.0.0.53 A
;; ANSWER SECTION:
hrz.uni-bonn.de.        86400   IN      A       131.220.14.100

Note that the result is now an actual reply to the query. So the reply by systemd-resolved depends on earlier queries filling the cache accordingly.

Other DNS cachers (such as dnsmasq) always reply as shown in B when asked for an A record, and that's also the way "real" DNS servers usually respond.

This can likely be reproduced with any domain with a "DNAME" entry.

Depending on what the specs say, this might be ok (and the client has to evaluate, i.e. mDNSresponder is broken). I'm not sure who is at fault here, but the fact that the answer of systemd-resolved depends on previous queries seems fishy.

There's a memory leak in the path cache rebuilding... path, which is unrelated to this PR, but should be fixed nonetheless in the future (https://github.com/systemd/systemd/pull/15954/commits)

Originally posted by @mrc0mmand in #256 (comment)

systemd version the issue has been seen with

...systemd-239-58.el8.x86_64

Used distribution

…Rocky Linux release 8.6 (Green Obsidian)

Expected behaviour you didn't see

…(systemd-logind) Sessions to close when user disconnects

Unexpected behaviour you saw

…Sessions leaking until D-bus was saturated and slow, the load spiked 100X what it should be, hitting the configured sessionmax

Steps to reproduce the problem run this version of systemd with lots of uses running short-lived ssh sessions, many that perform a bunch of automount activity

Before the switch to SuccessAction=exit-force in /usr/lib/systemd/user/systemd-exit.service, user sessions were ended with systemctl --force exit as an ExecStart command. Now it uses SuccessAction, which waits for either failed or not active state to occur, but my sessions that are leaked appear to be in the opening or closing state (for days).

Is there no timer that can be used here to prevent the session leak?

systemd version

239

Used distribution

CentOS 8.1

Expected behaviour you didn't see

Respect /sys/device/system/memory/auto_online_blocks settings

Unexpected behaviour you saw

Despite /sys/device/system/memory/auto_online_blocks = offline, /lib/udev/rules.d/40-redhat.rules always unconditionally online hotplugged memory blocks.

Steps to reproduce the problem

1. Create VM
2. In qemu monitor, hotplug memory
3. You'll see though by default /sys/device/system/memory/auto_online_blocks = offline, those new memory blocks are onlined by udev.

(I'm opening this mainly as a reminder to myself, so I don't debug the same stuff twice.)

In #433 CI started failing with:

 120/298 test-fs-util                                       FAIL              0.05s   killed by signal 6 SIGABRT
>>> SYSTEMD_KBD_MODEL_MAP=/build/src/locale/kbd-model-map SYSTEMD_LANGUAGE_FALLBACK_MAP=/build/src/locale/language-fallback-map MALLOC_PERTURB_=236 PATH=/build/build:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin /build/build/test-fs-util
 ✀  
stderr:
Assertion 'touch_file(a, false, test_mtime, test_uid, test_gid, 0640) >= 0' failed at ../src/test/test-fs-util.c:549, function test_touch_file(). Aborting.

The fail itself is caused by touch_file() calling chmod() on a symlink, which then returns EOPNOTSUPP. I suspect this is something that has changed in later kernels (and in this case it affects CI, because it runs in a docker container on Ubuntu Jammy). On C8S kernel this seems to work, but only on tmpfs:

# cat main.c 
#include <assert.h>
#include <errno.h>
#include <fcntl.h>
#include <stdio.h>
#include <sys/stat.h>
#include <sys/types.h>

int main(int argc, char *argv[]) {
        int fd, r;
        char fdpath[64];

        if (argc < 2)
                return 1;

        fd = open(argv[1], O_PATH|O_CLOEXEC|O_NOFOLLOW);
        if (fd < 0) {
                fprintf(stderr, "open() failed: %m\n");
                return 1;
        }

        sprintf(fdpath, "/proc/self/fd/%i", fd);
        r = chmod(fdpath, 0640);
        printf("chmod(%s)=%d (%d)\n", fdpath, r, errno);

        return r;
}

# mount -t tmpfs tmpfs /tmp
# (cd /tmp; ln -s symlink dangling; ln -s /etc/os-release symlink)
# (cd /var/tmp; ln -s symlink dangling; ln -s /etc/os-release symlink)
# gcc -o main main.c -D_GNU_SOURCE
# ./main /tmp/dangling
chmod(/proc/self/fd/3)=0 (0)
# ./main /tmp/symlink
chmod(/proc/self/fd/3)=0 (0)
# ./main /var/tmp/dangling 
chmod(/proc/self/fd/3)=-1 (95)
# ./main /var/tmp/symlink 
chmod(/proc/self/fd/3)=-1 (95)

And since the test uses /dev/shm/ for the test files (which is either tmpfs or some other form of tmpfs), it works.

On newer kernels all attempts to call chmod() on a symlink fail:

# uname -r
6.7.4-200.fc39.x86_64
# ./main /tmp/dangling 
chmod(/proc/self/fd/3)=-1 (95)
# ./main /tmp/symlink 
chmod(/proc/self/fd/3)=-1 (95)
# ./main /var/tmp/dangling 
chmod(/proc/self/fd/3)=-1 (95)
# ./main /var/tmp/symlink 
chmod(/proc/self/fd/3)=-1 (95)

The fix on the touch_file() side would be to call fchmod_and_chown() instead. Unfortunately, our fchmod_and_chown() in RHEL 8 is not clever enough and still does chmod() on a symlink, suffering from the same issue. For that we'd need 2dbb7e9 (+ commits before and after that change) to make it work properly, and that seems way too risky given how far in we're in RHEL 8 cycle.

So, when the need arises, I think it would be safer to just skip the offending test case, i.e.:

when running in GH Actions (or docker, whatever).

systemd version the issue has been seen with
systemd-239-41.el8_3.2.x86_64

Used distribution
CentOS 8.3 with lastest udpate

Expected behaviour you didn't see
'find /sys -follow -printf '' ' reports nothing.

Unexpected behaviour you saw
'find /sys -follow -printf '' ' reported a lot of 'File system loop detected'

Steps to reproduce the problem
just run 'find /sys -follow -printf '' '

systemd version the issue has been seen with
systemd-239-41.el8_3.2.x86_64

Used distribution
CentOS 8.3 with lastest udpate

Expected behaviour you didn't see
'find /sys -follow -printf '' ' reports nothing.

Unexpected behaviour you saw
'find /sys -follow -printf '' ' reported a lot of 'File system loop detected'

Steps to reproduce the problem
just run 'find /sys -follow -printf '' '

systemd version the issue has been seen with
systemd-239-41.el8_3.2.x86_64

Used distribution
CentOS 8.3 with lastest udpate

Expected behaviour you didn't see
'find /sys -follow -printf '' ' reports nothing.

Unexpected behaviour you saw
'find /sys -follow -printf '' ' reported a lot of 'File system loop detected'

Steps to reproduce the problem
just run 'find /sys -follow -printf '' '

systemd version the issue has been seen with
v239

Used distribution
Photon OS

Expected behaviour you didn't see
No fix for CVE-2022-4415 is available yet in systemd v239

Unexpected behaviour you saw

…

Steps to reproduce the problem
Please refer:
https://www.openwall.com/lists/oss-security/2022/12/21/3