smuellerdd / esdm Goto Github PK

View Code? Open in Web Editor NEW

10.0 10.0 3.0 6.92 MB

Entropy Source and DRNG Manager

Home Page: https://www.chronox.de/esdm

License: Other

C 93.12% Makefile 0.10% Shell 0.92% Meson 4.68% C++ 1.19%

entropy linux rng

esdm's People

Contributors

Stargazers

Watchers

Forkers

sirlucjan dj-on-github thillux

esdm's Issues

esdm-server is using up CPU checking file /proc/sys/crypto/fips_enabled and seeding

esdm version 0.5.0 has been compiled with the following meson flags:

meson --prefix=/usr --buildtype=debug -D strip=false -D es_jent=disabled -D selinux=disabled -D l\
inux-devfiles=enabled -D linux-getrandom=disabled -D esdm-server=enabled -D libdir=lib

From the resulting build the esdm-server and the esdm-cuse-random binary has been installed (with the needed libraries) and has been successfully started with the accordings systemd units. The test is run on a linux kernel v6.0.

The access to /dev/random (provided via fuse) seems to work fine.

However, the esdm-server process is utilizing a CPU for itself basically repeating the following sequence of syscalls:

FD 3 is /dev/hwrng

12:06:47.611925 getrandom("\x9b\xda\x43\x06\x68\xf1\xc1\x55\x8e\x88\xfc\xc2\x95\xb8\x51\xc1\xfd\x8a\x24\xaf\x34\xda\xa6\x03\xec\x16\x97\x94\x91\x65\xe4\x8c", 32, GRND_NONBLOCK) = 32
12:06:47.611934 read(3, 0x7ffce93663b0, 32) = -1 ENODEV (No such device)
12:06:47.611941 openat(AT_FDCWD, "/proc/sys/crypto/fips_enabled", O_RDONLY) = -1 ENOENT (No such file or directory)
12:06:47.611951 openat(AT_FDCWD, "/proc/sys/crypto/fips_enabled", O_RDONLY) = -1 ENOENT (No such file or directory)
12:06:47.611961 openat(AT_FDCWD, "/proc/sys/crypto/fips_enabled", O_RDONLY) = -1 ENOENT (No such file or directory)
12:06:47.611970 openat(AT_FDCWD, "/proc/sys/crypto/fips_enabled", O_RDONLY) = -1 ENOENT (No such file or directory)
12:06:47.611984 openat(AT_FDCWD, "/proc/sys/crypto/fips_enabled", O_RDONLY) = -1 ENOENT (No such file or directory)
12:06:47.611993 openat(AT_FDCWD, "/proc/sys/crypto/fips_enabled", O_RDONLY) = -1 ENOENT (No such file or directory)
12:06:47.612048 openat(AT_FDCWD, "/proc/sys/crypto/fips_enabled", O_RDONLY) = -1 ENOENT (No such file or directory)
12:06:47.612062 openat(AT_FDCWD, "/proc/sys/crypto/fips_enabled", O_RDONLY) = -1 ENOENT (No such file or directory)
12:06:47.612073 openat(AT_FDCWD, "/proc/sys/crypto/fips_enabled", O_RDONLY) = -1 ENOENT (No such file or directory)
12:06:47.612084 openat(AT_FDCWD, "/proc/sys/crypto/fips_enabled", O_RDONLY) = -1 ENOENT (No such file or directory)
12:06:47.612095 openat(AT_FDCWD, "/proc/sys/crypto/fips_enabled", O_RDONLY) = -1 ENOENT (No such file or directory)
12:06:47.612110 openat(AT_FDCWD, "/proc/sys/crypto/fips_enabled", O_RDONLY) = -1 ENOENT (No such file or directory)
12:06:47.612121 getrandom("\x92\xe3\xe4\x42\xde\x7c\xcc\x0e\xa0\x8d\x43\x34\xc0\xfd\x38\xee\x4c\x7e\x7c\x68\x30\xa1\x67\xfc\x11\x7e\xd1\xcb\x35\xa7\xc4\x14", 32, GRND_NONBLOCK) = 32
12:06:47.612142 read(3, 0x7ffce93663b0, 32) = -1 ENODEV (No such device)
12:06:47.612151 openat(AT_FDCWD, "/proc/sys/crypto/fips_enabled", O_RDONLY) = -1 ENOENT (No such file or directory)
12:06:47.612161 openat(AT_FDCWD, "/proc/sys/crypto/fips_enabled", O_RDONLY) = -1 ENOENT (No such file or directory)
12:06:47.612171 openat(AT_FDCWD, "/proc/sys/crypto/fips_enabled", O_RDONLY) = -1 ENOENT (No such file or directory)
12:06:47.612181 openat(AT_FDCWD, "/proc/sys/crypto/fips_enabled", O_RDONLY) = -1 ENOENT (No such file or directory)
12:06:47.612194 openat(AT_FDCWD, "/proc/sys/crypto/fips_enabled", O_RDONLY) = -1 ENOENT (No such file or directory)
12:06:47.612205 openat(AT_FDCWD, "/proc/sys/crypto/fips_enabled", O_RDONLY) = -1 ENOENT (No such file or directory)
12:06:47.612215 openat(AT_FDCWD, "/proc/sys/crypto/fips_enabled", O_RDONLY) = -1 ENOENT (No such file or directory)
12:06:47.612224 openat(AT_FDCWD, "/proc/sys/crypto/fips_enabled", O_RDONLY) = -1 ENOENT (No such file or directory)
12:06:47.612234 openat(AT_FDCWD, "/proc/sys/crypto/fips_enabled", O_RDONLY) = -1 ENOENT (No such file or directory)
12:06:47.612243 openat(AT_FDCWD, "/proc/sys/crypto/fips_enabled", O_RDONLY) = -1 ENOENT (No such file or directory)
12:06:47.612253 openat(AT_FDCWD, "/proc/sys/crypto/fips_enabled", O_RDONLY) = -1 ENOENT (No such file or directory)
12:06:47.612266 openat(AT_FDCWD, "/proc/sys/crypto/fips_enabled", O_RDONLY) = -1 ENOENT (No such file or directory)

When attaching with a debugger the stracktrace usually looks like the following:

Program received signal SIGINT, Interrupt.
0x00007f937e1edc07 in open64 () from target:/lib/libc.so.6
#0  0x00007f937e1edc07 in open64 () from target:/lib/libc.so.6
#1  0x00007f937e1800c2 in _IO_file_open () from target:/lib/libc.so.6
#2  0x00007f937e18029a in __GI__IO_file_fopen () from target:/lib/libc.so.6
#3  0x00007f937e173bf9 in __fopen_internal () from target:/lib/libc.so.6
#4  0x00007f937e2fb3f4 in fips_enabled () at ../esdm/fips.c:103
#5  esdm_config_fips_enabled () at ../esdm/esdm_config.c:245
#6  0x00007f937e2fbde5 in esdm_sp80090c_compliant () at ../esdm/esdm_drng_mgr.c:278
#7  0x00007f937e2fcbfb in esdm_get_seed_entropy_osr (fully_seeded=false) at ../esdm/esdm_es_aux.h:51
#8  esdm_fully_seeded (collected_entropy=<optimized out>, fully_seeded=false) at ../esdm/esdm_es_mgr.c:259
#9  esdm_drng_seed_es (drng=0x7f937e307500 <esdm_drng_init>) at ../esdm/esdm_drng_mgr.c:383
#10 0x00007f937e2fd03f in esdm_drng_seed_work_one (node=0, drng=0x7f937e307500 <esdm_drng_init>) at ../esdm/esdm_drng_mgr.c:421
#11 __esdm_drng_seed_work () at ../esdm/esdm_drng_mgr.c:453
#12 esdm_drng_seed_work () at ../esdm/esdm_drng_mgr.c:475
#13 esdm_es_add_entropy () at ../esdm/esdm_es_mgr.c:555
#14 esdm_es_add_entropy () at ../esdm/esdm_es_mgr.c:545
#15 0x00007f937e2fed87 in esdm_es_mgr_initialize () at ../esdm/esdm_es_mgr.c:504
#16 esdm_init () at ../esdm/esdm_lib.c:45
#17 0x000055fd3f4e38af in daemon_init () at ../frontends/server/server_main.c:143
#18 main (argc=2, argv=<optimized out>) at ../frontends/server/server_main.c:292

No output from esd-server is found in the journal logs, only the startup message from systemd.

After checking the source code I tried adding the meson option -D oversample_es=false. This has no influence on the problem.

I also added esdm-proc and started in with the systemd unit, this did provide the /proc/sys/kernel/random directory with the esdm_type file but did not solve the problem.

systemd service for esdm-server does not allow debugfs reads

If esdm-server is started from the systemd service in the frontend path, privileges are dropped to nobody, which does not allow reading irq/sched entropy from debugfs on most systems.

Race condition on /dev/esdm_es

If esdm_es is loaded as a kernel module at some point in time during boot and esdm-server gets started independently, a race condition exists. Sometimes esdm-server finds kernel entropy sources, sometimes not.

Possible fixes/workarounds:

Add condition to systemd-service to check if /dev/esdm_es exists before starting esdm-server, but only if kernel support is compiled in
Periodically retry opening/finding /dev/esdm_es in esdm-server

SEGV when using SHA3-512 in esdm_get_cpu_data_compress

I use an old Haswell system which only supports RDRAND, if this should be of any influence here.

ESDM (22:39:23) (esdm-server) Debug - ESDM server [../frontends/server/server_main.c:install_term:189]: Install termination signal handler
ESDM (22:39:23) (esdm-server) Verbose - ESDM server [../frontends/server/server_main.c:daemon_init:142]: Starting ESDM server
ESDM (22:39:23) (esdm-server) Verbose [../esdm/esdm_builtin_hash_drbg.c:esdm_hash_drbg_alloc:55]: Hash DRBG core allocated
ESDM (22:39:23) (esdm-server) Debug - DRNG [../esdm/esdm_drng_mgr.c:esdm_drng_reset:136]: reset DRNG
ESDM (22:39:23) (esdm-server) Verbose - DRNG [../esdm/esdm_drng_mgr.c:esdm_drng_mgr_initialize:243]: DRNG with prediction resistance allocated
ESDM (22:39:23) (esdm-server) Verbose [../esdm/esdm_builtin_hash_drbg.c:esdm_hash_drbg_alloc:55]: Hash DRBG core allocated
ESDM (22:39:23) (esdm-server) Debug - DRNG [../esdm/esdm_drng_mgr.c:esdm_drng_reset:136]: reset DRNG
ESDM (22:39:23) (esdm-server) Verbose - DRNG [../esdm/esdm_drng_mgr.c:esdm_drng_mgr_initialize:251]: DRNG without prediction resistance allocated
ESDM (22:39:23) (esdm-server) Debug - DRNG [../esdm/esdm_drng_mgr.c:esdm_drng_mgr_initialize:258]: ESDM for general use is available
ESDM (22:39:23) (esdm-server) Debug - DRNG [../esdm/esdm_drng_mgr.c:esdm_drng_mgr_selftest:190]: Hash self test passed successfully
ESDM (22:39:23) (esdm-server) Debug - DRNG [../esdm/esdm_drng_mgr.c:esdm_drng_mgr_selftest:201]: DRNG self test passed successfully
ESDM (22:39:23) (esdm-server) Verbose - Entropy Source [../esdm/esdm_es_mgr.c:esdm_es_mgr_initialize:618]: Initialize ES manager
ESDM (22:39:23) (esdm-server) Debug - Entropy Source [../esdm/esdm_es_mgr.c:esdm_es_mgr_init_es:594]: Initialize ES Auxiliary
ESDM (22:39:23) (esdm-server) Verbose [../esdm/esdm_es_aux.c:esdm_aux_init:126]: Aux ES hash allocated
ESDM (22:39:23) (esdm-server) Debug - Entropy Source [../esdm/esdm_es_mgr.c:esdm_es_mgr_init_es:594]: Initialize ES Interrupt
ESDM (22:39:23) (esdm-server) Warning - Entropy Source [../esdm/esdm_es_irq.c:esdm_irq_initialize:150]: Disabling interrupt-based entropy source which is not present in kernel
ESDM (22:39:23) (esdm-server) Debug - Entropy Source [../esdm/esdm_es_mgr.c:esdm_es_mgr_init_es:594]: Initialize ES Scheduler
ESDM (22:39:23) (esdm-server) Warning - Entropy Source [../esdm/esdm_es_sched.c:esdm_sched_initialize:149]: Disabling scheduler-based entropy source which is not present in kernel
ESDM (22:39:23) (esdm-server) Debug - Entropy Source [../esdm/esdm_es_mgr.c:esdm_es_mgr_init_es:594]: Initialize ES JitterRNG
ESDM (22:39:23) (esdm-server) Debug - Entropy Source [../esdm/esdm_es_jent.c:esdm_jent_initialize:70]: Jitter RNG working on current system
ESDM (22:39:23) (esdm-server) Debug - Entropy Source [../esdm/esdm_es_mgr.c:esdm_es_mgr_init_es:594]: Initialize ES CPU
ESDM (22:39:23) (esdm-server) Debug - Entropy Source [../esdm/esdm_es_mgr.c:esdm_es_mgr_init_es:594]: Initialize ES KernelRNG
ESDM (22:39:23) (esdm-server) Debug - Entropy Source [../esdm/esdm_es_mgr.c:esdm_es_mgr_init_es:594]: Initialize ES LinuxHWRand
ESDM (22:39:23) (esdm-server) Debug [../esdm/es_cpu/cpu_random_x86.h:cpu_es_x86_rdseed:96]: RDSEED support not detected
ESDM (22:39:23) (esdm-server) Debug [../esdm/es_cpu/cpu_random_x86.h:cpu_es_x86_rdrand:119]: RDRAND support detected
ESDM (22:39:23) (esdm-server) Debug - DRNG [../esdm/esdm_drng_mgr.c:esdm_drng_seed_work_one:486]: reseed triggered by system events for DRNG on node 0
ESDM (22:39:23) (esdm-server) Debug - Entropy Source [../esdm/esdm_es_jent.c:esdm_jent_get:122]: obtained 256 bits of entropy from Jitter RNG noise source
ESDM (22:39:23) (esdm-server) Debug [../esdm/es_cpu/cpu_random_x86.h:cpu_es_x86_rdseed:96]: RDSEED support not detected
ESDM (22:39:23) (esdm-server) Debug - Entropy Source [../esdm/esdm_es_cpu.c:esdm_cpu_multiplier:181]: Setting CPU ES multiplier to 512
ESDM (22:39:23) (esdm-server) Debug [../esdm/es_cpu/cpu_random_x86.h:cpu_es_x86_rdrand:119]: RDRAND support detected

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff7fb01b2 in esdm_get_cpu_data_compress (multiplier=<optimized out>, requested_bits=256, 
    outbuf=0x7fffffffc88c "\207h\376\b\356Ľ\030Ґiz\351\345\002\3775Q\202\347-(\241cA/\023l\020|\301", <incomplete sequence \355>) at ../esdm/esdm_es_cpu.c:122
122			if (hash_cb->hash_update(shash, outbuf, ent_bits >> 3))
(gdb) bt
#0  0x00007ffff7fb01b2 in esdm_get_cpu_data_compress (multiplier=<optimized out>, requested_bits=256, 
    outbuf=0x7fffffffc88c "\207h\376\b\356Ľ\030Ґiz\351\345\002\3775Q\202\347-(\241cA/\023l\020|\301", <incomplete sequence \355>) at ../esdm/esdm_es_cpu.c:122
#1  esdm_cpu_get (eb_es=0x7fffffffc88c, requested_bits=<optimized out>, unsused=<optimized out>) at ../esdm/esdm_es_cpu.c:225
#2  0x00007ffff7faecfb in esdm_fill_seed_buffer (eb=<optimized out>, requested_bits=64, force=<optimized out>) at ../esdm/esdm_es_mgr.c:728
#3  0x00007ffff7fac89c in esdm_drng_seed_es_nolock (drng=drng@entry=0x7ffff7fba140 <esdm_drng_init>, drng_type=0x7ffff7fb3a31 "regular", init_ops=true)
    at ../esdm/esdm_drng_mgr.c:416
#4  0x00007ffff7face5e in esdm_drng_seed_es (drng=0x7ffff7fba140 <esdm_drng_init>) at ../esdm/esdm_drng_mgr.c:469
#5  esdm_drng_seed (drng=0x7ffff7fba140 <esdm_drng_init>) at ../esdm/esdm_drng_mgr.c:479
#6  esdm_drng_seed_work_one (drng=drng@entry=0x7ffff7fba140 <esdm_drng_init>, node=node@entry=0) at ../esdm/esdm_drng_mgr.c:489
#7  0x00007ffff7facf76 in __esdm_drng_seed_work (force=true) at ../esdm/esdm_drng_mgr.c:543
#8  __esdm_drng_seed_work (force=<optimized out>) at ../esdm/esdm_drng_mgr.c:496
#9  0x00007ffff7fad0c7 in esdm_force_fully_seeded () at ../esdm/esdm_drng_mgr.c:791
#10 esdm_force_fully_seeded () at ../esdm/esdm_drng_mgr.c:785
#11 0x00007ffff7faea9b in esdm_es_mgr_initialize () at ../esdm/esdm_es_mgr.c:645
#12 0x00007ffff7faf0db in esdm_init () at ../esdm/esdm_lib.c:45
#13 0x00005555555598ca in daemon_init () at ../frontends/server/server_main.c:143
#14 main (argc=<optimized out>, argv=<optimized out>) at ../frontends/server/server_main.c:294

Dangerous usage of the min_t macro

Definition of the min_t macro:

#define min_t(type, a, b) ((type)a < (type)b) ? (type)a : (type)b

This is no problem when a and b are variables or constants, but if they are functions, the functions are potentially executed twice. If those functions will yield different results at first and second execution, the macro wil not work as intended.

An example can be observed in this code snippet (from esdm_auc_get_pool)
collected_ent_bits = min_t(uint32_t, digestsize_bits, atomic_xchg(&pool->aux_entropy_bits, 0));
First call to atomic_xchg(&pool->aux_entropy_bits, 0) will return the correct entropy (for the comparision), but the second call will return 0 and the subsequent code will work as if no entropy were available and fail.

Including esdm_rpc_client.h from C++ c

There are two issues with including esdm_rpc_client.h from C++ code:

new is used as a variable name, which is not allowed in C++ since it is a keyword
bool.h does not compile at all, but is not needed anyway.

At the moment I am using this work-around:
#define new new_
#define BOOL_H
#include <esdm/esdm_rpc_client.h>
#undef new

write_wakeup_threshold does not work

I tested a smartcard-like setup (watch auxiliary pool and feed entropy in it, while additional entropy sources where active). The output of /proc/sys/kernel/random/entropy_avail dropped to 0 when I started an endless loop drawing random numbers from ESDM with prediction resistance. My small helper which selects on /dev/random for writeability only gets notified once and not permanently, while my test tool is running. The tests seem to not cover this case of multiple write readiness notifications.

esdm-cuse-random continues to deliver "random" numbers, when esdm-server is stopped

Test:

cat /dev/random
systemctl stop esdm-server

esdm_rpc_client.h includes local header files.

I am trying to build an extrenal rpc client. My program includes esdm_rpc_client.h. This header includes atomic.h and other headers that are not exported, thus in the current configuration I cannot use the library from out side.

esdm_es kernel module: scheduled while atomic

While testing esdm on a recent kernel 6.3.5, I noticed the following kernel warning:

[    2.765344] BUG: scheduling while atomic: kworker/6:1/138/0x00000002
[    2.765505] Modules linked in: esdm_es(O+) v4l2loopback(O) videodev mc led_class kvm_amd ccp kvm drm irqbypass fuse backlight i2c_core deflate efi_pstore configfs efivarfs tpm rng_core dmi_sysfs ip_tables x_tables autofs4 sd_mod xhci_pci xhci_pci_renesas xhci_hcd ahci libahci libata nvme usbcore scsi_mod nvme_core t10_pi usb_common scsi_common crc64_rocksoft crc64 crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common rtc_cmos dm_mod dax btrfs blake2b_generic xor libcrc32c crc32c_generic crc32c_intel raid6_pq
[    2.765520] CPU: 6 PID: 138 Comm: kworker/6:1 Tainted: G           O       6.3.5 #1-NixOS
[    2.765521] Hardware name: Micro-Star International Co., Ltd. MS-7C37/X570-A PRO (MS-7C37), BIOS H.J0 02/28/2023
[    2.765522] Workqueue: events esdm_es_irq_set_callbackfn [esdm_es]
[    2.765526] Call Trace:
[    2.765528]  <TASK>
[    2.765529]  dump_stack_lvl+0x47/0x60
[    2.765533]  __schedule_bug+0x56/0x70
[    2.765536]  __schedule+0xfc1/0x1380
[    2.765539]  ? find_busiest_group+0x41/0x330
[    2.765541]  ? select_task_rq_fair+0x135/0x11f0
[    2.765542]  schedule+0x5e/0xd0
[    2.765543]  schedule_timeout+0x151/0x160
[    2.765546]  wait_for_completion_state+0x14f/0x220
[    2.765548]  call_usermodehelper_exec+0x14e/0x1a0
[    2.765550]  __request_module+0x1a9/0x420
[    2.765553]  crypto_alg_mod_lookup+0x165/0x230
[    2.765556]  crypto_alloc_tfm_node+0x42/0xd0
[    2.765558]  esdm_kcapi_hash_name_alloc+0x25/0xc0 [esdm_es]
[    2.765561]  esdm_es_irq_set_callbackfn+0x57/0xc0 [esdm_es]
[    2.765564]  process_one_work+0x1c8/0x3c0
[    2.765566]  worker_thread+0x51/0x390
[    2.765568]  ? __pfx_worker_thread+0x10/0x10
[    2.765570]  kthread+0xde/0x110
[    2.765571]  ? __pfx_kthread+0x10/0x10
[    2.765572]  ret_from_fork+0x2c/0x50
[    2.765575]  </TASK>

relying on a library without a release in a security product?

while packaging esdm for opensuse (hi from lars too) i noticed that protobuf-c-rpc library is not released yet. is there no released library that we can use?

Running into fuse limitation

When running ESDM and some /dev/random poller on my notebook, suspend no longer works. ESDM uses all fuse daemons. In the example here, I used jitterentropy-rngd as poller (just in order to have an easy reproducer).

Steps to reproduce:

Start esdm with all fuse/cuse daemons
Start jitterentropy-rngd
Enter suspend (I do this with "systemctl suspend")

Observed behavior:
Suspend is entered after a long waiting time and sometimes never.

[ 2754.945068] PM: suspend entry (s2idle)
[ 2754.951728] Filesystems sync: 0.006 seconds
[ 2754.955664] Freezing user space processes
[ 2774.963892] Freezing user space processes failed after 20.008 seconds (1 tasks refusing to freeze, wq_busy=0):
[ 2774.964260] task:jitterentropy-r state:D stack:0     pid:527252 ppid:1      flags:0x00004006
[ 2774.964355] Call Trace:
[ 2774.964362]  <TASK>
[ 2774.964377]  __schedule+0x388/0x1390
[ 2774.964418]  schedule+0x5e/0xd0
[ 2774.964439]  request_wait_answer+0xd1/0x2a0 [fuse]
[ 2774.964501]  ? __pfx_autoremove_wake_function+0x10/0x10
[ 2774.964526]  fuse_simple_request+0x18f/0x2b0 [fuse]
[ 2774.964583]  fuse_file_poll+0x1a5/0x210 [fuse]
[ 2774.964657]  do_select+0x352/0x820
[ 2774.964697]  ? __pfx_pollwake+0x10/0x10
[ 2774.964758]  core_sys_select+0x236/0x450
[ 2774.964806]  do_pselect.constprop.0+0xe9/0x180
[ 2774.964832]  __x64_sys_pselect6+0x4c/0x70
[ 2774.964849]  do_syscall_64+0x3b/0x90
[ 2774.964868]  entry_SYSCALL_64_after_hwframe+0x6e/0xd8
[ 2774.964880] RIP: 0033:0x7f4981a5ce94
[ 2774.964966] RSP: 002b:00007ffc5724e1f0 EFLAGS: 00000246 ORIG_RAX: 000000000000010e
[ 2774.964980] RAX: ffffffffffffffda RBX: 0000000000000004 RCX: 00007f4981a5ce94
[ 2774.964990] RDX: 00007ffc5724e280 RSI: 0000000000000000 RDI: 0000000000000004
[ 2774.964998] RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000
[ 2774.965005] R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
[ 2774.965012] R13: 0000000000000000 R14: 0000563709d4ac30 R15: 00007ffc5724e280
[ 2774.965037]  </TASK>

long blocking in esdm-proc files

I'm currently investigating why cat /proc/sys/kernel/random/* sometimes blocks on my system for some seconds per file. Details will follow if I understand more of this issue.

systemd service esdm-server gets deactivated

When running as a systemd service the call to unshare(CLONE_NEWNS | CLONE_NEWCGROUP | CLONE_NEWNET) in linux_isolate_namespace fails with EPERM and systemd deactivates the service.
This can be fixed by removing this line from frontends/server/esdm-server.service.in
RestrictNamespaces=yes

First request takes long time with AIS2031 mode enabled

Test setup:

configure esdm with -Dais2031=true
start esdm-server and wait some time (e.g. 20s), all DRGs should be fully seeded now
connect client and request random numbers -> esdm needs around 1-2s on my system until it returns random numbers on the first request
connect client again -> random numbers are returned instantly

Without ais2031 I obtain random numbers instantly in step 3.

Linking fails due to missing thread dependency

I tried to build version 0.5.0 in a glibc 2.28 environment, based on Debian bullseye.

fuse3.pc and protobuf-c.pc are available with their usual content, only prefix is modified to match our build system:

fuse3.pc

prefix=/build/client/random/fuse/_/
libdir=${prefix}/lib/x86_64-linux-gnu
includedir=${prefix}/include

Name: fuse3
Description: Filesystem in Userspace
Version: 3.12.0
Libs: -L${libdir} -lfuse3 -lpthread
Libs.private: -ldl -pthread -lrt
Cflags: -I${includedir}/fuse3

protobuf-c.pc

prefix=/build/client/random/protobuf-c/_/usr
libnames=protobuf-c
bindirs=
aclocaldirs=
adaprjdirs=${prefix}/lib/gnat

Name: protobuf-c
Description: C implementation of the Google Protocol Buffers
Version: 1.4.1
Requires.private: protobuf
Libs: -L${prefix}/lib -lprotobuf-c 
Cflags: -I${prefix}/include

The meson call:

meson --prefix=/usr -D es_jent=disabled -D selinux=disabled -D linux-devfiles=enabled -D linux-getrandom=disabled -D esdm-server=enabled
The Meson build system
Version: 0.56.2
Source dir: /build/client/random/esdm/work/esdm-0.5.0
Build dir: /build/client/random/esdm/work/esdm-0.5.0/build
Build type: native build
Project name: esdm
Project version: 0.5.0
C compiler for the host machine: ccache cc (gcc 10.2.1 "cc (Debian 10.2.1-6) 10.2.1 20210110")
C linker for the host machine: cc ld.bfd 2.35.2
Host machine cpu family: x86_64
Host machine cpu: x86_64
Program addon/sanity_checks.sh found: YES (/build/client/random/esdm/work/esdm-0.5.0/addon/sanity_checks.sh)
Found pkg-config: /build/client/random/bscripts/tools/pkg-config (0.29.2)
Run-time dependency libprotobuf-c found: YES 1.4.1
Configuring config.h using configuration
Run-time dependency threads found: YES
Configuring esdm-server.service using configuration
Run-time dependency fuse3 found: YES 3.12.0
Configuring esdm-cuse-random.service using configuration
Configuring esdm-cuse-urandom.service using configuration
Configuring esdm-proc.service using configuration
Build targets in project: 41

Found ninja-1.10.1 at /usr/bin/ninja

The linking then fails with:

[193/209] Linking target esdm/libesdm.so.0.5.0
FAILED: esdm/libesdm.so.0.5.0 
cc  -o esdm/libesdm.so.0.5.0 esdm/libesdm.so.0.5.0.p/.._common_binhexbin.c.o esdm/libesdm.so.0.5.0.p/.._common_buffer.c.o esdm/libesdm.so.0.5.0.p/.._common_helper.c.o esdm/libesdm.so.0.5.0.p/.._common_logger.c.o esdm/libesdm.so.0.5.0.p/.._common_threading_support.c.o esdm/libesdm.so.0.5.0.p/.._crypto_hash.c.o esdm/libesdm.so.0.5.0.p/.._crypto_hash_drbg.c.o esdm/libesdm.so.0.5.0.p/.._crypto_hmac.c.o esdm/libesdm.so.0.5.0.p/.._crypto_sha256.c.o esdm/libesdm.so.0.5.0.p/.._crypto_sha512.c.o esdm/libesdm.so.0.5.0.p/.._crypto_drbg.c.o esdm/libesdm.so.0.5.0.p/esdm_config.c.o esdm/libesdm.so.0.5.0.p/esdm_drng_mgr.c.o esdm/libesdm.so.0.5.0.p/esdm_es_aux.c.o esdm/libesdm.so.0.5.0.p/esdm_es_mgr.c.o esdm/libesdm.so.0.5.0.p/esdm_info.c.o esdm/libesdm.so.0.5.0.p/esdm_interface_dev_common.c.o esdm/libesdm.so.0.5.0.p/esdm_lib.c.o esdm/libesdm.so.0.5.0.p/esdm_shm_status.c.o esdm/libesdm.so.0.5.0.p/esdm_es_cpu.c.o esdm/libesdm.so.0.5.0.p/esdm_es_irq.c.o esdm/libesdm.so.0.5.0.p/esdm_es_krng.c.o esdm/libesdm.so.0.5.0.p/esdm_es_sched.c.o esdm/libesdm.so.0.5.0.p/esdm_es_hwrand.c.o esdm/libesdm.so.0.5.0.p/esdm_node.c.o esdm/libesdm.so.0.5.0.p/fips.c.o esdm/libesdm.so.0.5.0.p/fips_integrity.c.o esdm/libesdm.so.0.5.0.p/esdm_builtin_hash_drbg.c.o esdm/libesdm.so.0.5.0.p/esdm_builtin_sha512.c.o -flto -Wl,--as-needed -Wl,--no-undefined -shared -fPIC -Wl,--start-group -Wl,-soname,libesdm.so.0 -Wl,--gc-sections -Wl,-z,relro -Wl,-z,now -Wl,--as-needed /build/client/random/protobuf-c/_/usr/lib/libprotobuf-c.so -Wl,--end-group
/usr/bin/ld: /tmp/libesdm.so.0.5.0.TzKbUI.ltrans0.ltrans.o: in function `thread_set_name':
/build/client/random/esdm/work/esdm-0.5.0/build/../common/threading_support.c:475: undefined reference to `pthread_setname_np'
/usr/bin/ld: /tmp/libesdm.so.0.5.0.TzKbUI.ltrans0.ltrans.o: in function `thread_get_name':
/build/client/random/esdm/work/esdm-0.5.0/build/../common/threading_support.c:482: undefined reference to `pthread_getname_np'
/usr/bin/ld: /tmp/libesdm.so.0.5.0.TzKbUI.ltrans0.ltrans.o: in function `thread_worker':
/build/client/random/esdm/work/esdm-0.5.0/build/../common/threading_support.c:232: undefined reference to `pthread_sigmask'
/usr/bin/ld: /tmp/libesdm.so.0.5.0.TzKbUI.ltrans0.ltrans.o: in function `thread_worker':
/build/client/random/esdm/work/esdm-0.5.0/build/../common/mutex_w.h:91: undefined reference to `pthread_mutexattr_destroy'
/usr/bin/ld: /tmp/libesdm.so.0.5.0.TzKbUI.ltrans0.ltrans.o: in function `esdm_shm_status_up':
/build/client/random/esdm/work/esdm-0.5.0/build/../esdm/esdm_shm_status.c:58: undefined reference to `sem_getvalue'
/usr/bin/ld: /build/client/random/esdm/work/esdm-0.5.0/build/../esdm/esdm_shm_status.c:62: undefined reference to `sem_post'
/usr/bin/ld: /tmp/libesdm.so.0.5.0.TzKbUI.ltrans0.ltrans.o: in function `thread_start':
/build/client/random/esdm/work/esdm-0.5.0/build/../common/mutex_w.h:103: undefined reference to `pthread_mutex_trylock'
/usr/bin/ld: /tmp/libesdm.so.0.5.0.TzKbUI.ltrans0.ltrans.o: in function `thread_start':
/build/client/random/esdm/work/esdm-0.5.0/build/../common/threading_support.c:281: undefined reference to `pthread_create'
/usr/bin/ld: /tmp/libesdm.so.0.5.0.TzKbUI.ltrans0.ltrans.o: in function `thread_start':
/build/client/random/esdm/work/esdm-0.5.0/build/../common/mutex_w.h:91: undefined reference to `pthread_mutexattr_destroy'
/usr/bin/ld: /tmp/libesdm.so.0.5.0.TzKbUI.ltrans0.ltrans.o: in function `thread_release':
/build/client/random/esdm/work/esdm-0.5.0/build/../common/threading_support.c:542: undefined reference to `pthread_cancel'
/usr/bin/ld: /tmp/libesdm.so.0.5.0.TzKbUI.ltrans0.ltrans.o:/build/client/random/esdm/work/esdm-0.5.0/build/../common/threading_support.c:543: undefined reference to `pthread_join'
/usr/bin/ld: /tmp/libesdm.so.0.5.0.TzKbUI.ltrans0.ltrans.o: in function `thread_release':
/build/client/random/esdm/work/esdm-0.5.0/build/../common/mutex_w.h:91: undefined reference to `pthread_mutexattr_destroy'
/usr/bin/ld: /tmp/libesdm.so.0.5.0.TzKbUI.ltrans0.ltrans.o: in function `thread_release':
/build/client/random/esdm/work/esdm-0.5.0/build/../common/threading_support.c:506: undefined reference to `pthread_join'
/usr/bin/ld: /tmp/libesdm.so.0.5.0.TzKbUI.ltrans0.ltrans.o: in function `thread_release':
/build/client/random/esdm/work/esdm-0.5.0/build/../common/mutex_w.h:91: undefined reference to `pthread_mutexattr_destroy'
/usr/bin/ld: /tmp/libesdm.so.0.5.0.TzKbUI.ltrans0.ltrans.o: in function `esdm_drng_mgr_selftest':
/build/client/random/esdm/work/esdm-0.5.0/build/../common/mutex.h:87: undefined reference to `pthread_rwlock_rdlock'
/usr/bin/ld: /build/client/random/esdm/work/esdm-0.5.0/build/../common/mutex.h:96: undefined reference to `pthread_rwlock_unlock'
/usr/bin/ld: /tmp/libesdm.so.0.5.0.TzKbUI.ltrans0.ltrans.o:/build/client/random/esdm/work/esdm-0.5.0/build/../common/mutex.h:96: undefined reference to `pthread_rwlock_unlock'
/usr/bin/ld: /tmp/libesdm.so.0.5.0.TzKbUI.ltrans0.ltrans.o: in function `esdm_drng_mgr_initialize.part.0':
/build/client/random/esdm/work/esdm-0.5.0/build/../common/mutex_w.h:71: undefined reference to `pthread_mutexattr_init'
/usr/bin/ld: /tmp/libesdm.so.0.5.0.TzKbUI.ltrans0.ltrans.o:/build/client/random/esdm/work/esdm-0.5.0/build/../common/mutex_w.h:75: undefined reference to `pthread_mutexattr_setrobust'
/usr/bin/ld: /tmp/libesdm.so.0.5.0.TzKbUI.ltrans0.ltrans.o: in function `esdm_drng_mgr_initialize.part.0':
/build/client/random/esdm/work/esdm-0.5.0/build/../common/mutex_w.h:71: undefined reference to `pthread_mutexattr_init'
/usr/bin/ld: /tmp/libesdm.so.0.5.0.TzKbUI.ltrans0.ltrans.o:/build/client/random/esdm/work/esdm-0.5.0/build/../common/mutex_w.h:75: undefined reference to `pthread_mutexattr_setrobust'
/usr/bin/ld: /tmp/libesdm.so.0.5.0.TzKbUI.ltrans0.ltrans.o: in function `esdm_aux_pool_insert_locked':
/build/client/random/esdm/work/esdm-0.5.0/build/../common/mutex.h:87: undefined reference to `pthread_rwlock_rdlock'
/usr/bin/ld: /build/client/random/esdm/work/esdm-0.5.0/build/../common/mutex.h:96: undefined reference to `pthread_rwlock_unlock'
/usr/bin/ld: /tmp/libesdm.so.0.5.0.TzKbUI.ltrans0.ltrans.o: in function `esdm_aux_get_backtrack':
/build/client/random/esdm/work/esdm-0.5.0/build/../common/mutex.h:87: undefined reference to `pthread_rwlock_rdlock'
/usr/bin/ld: /build/client/random/esdm/work/esdm-0.5.0/build/../common/mutex.h:96: undefined reference to `pthread_rwlock_unlock'
/usr/bin/ld: /tmp/libesdm.so.0.5.0.TzKbUI.ltrans0.ltrans.o: in function `esdm_fill_seed_buffer':
/build/client/random/esdm/work/esdm-0.5.0/build/../common/queue.h:59: undefined reference to `pthread_mutex_trylock'
/usr/bin/ld: /tmp/libesdm.so.0.5.0.TzKbUI.ltrans0.ltrans.o: in function `esdm_drng_seed':
/build/client/random/esdm/work/esdm-0.5.0/build/../common/queue.h:59: undefined reference to `pthread_mutex_trylock'
/usr/bin/ld: /tmp/libesdm.so.0.5.0.TzKbUI.ltrans0.ltrans.o: in function `esdm_es_add_entropy':
/build/client/random/esdm/work/esdm-0.5.0/build/../common/mutex_w.h:103: undefined reference to `pthread_mutex_trylock'
/usr/bin/ld: /tmp/libesdm.so.0.5.0.TzKbUI.ltrans0.ltrans.o: in function `esdm_cpu_get':
/build/client/random/esdm/work/esdm-0.5.0/build/../common/mutex.h:87: undefined reference to `pthread_rwlock_rdlock'
/usr/bin/ld: /build/client/random/esdm/work/esdm-0.5.0/build/../common/mutex.h:96: undefined reference to `pthread_rwlock_unlock'
/usr/bin/ld: /tmp/libesdm.so.0.5.0.TzKbUI.ltrans0.ltrans.o: in function `esdm_drng_get_sleep':
/build/client/random/esdm/work/esdm-0.5.0/build/../common/mutex_w.h:103: undefined reference to `pthread_mutex_trylock'
/usr/bin/ld: /build/client/random/esdm/work/esdm-0.5.0/build/../common/mutex_w.h:103: undefined reference to `pthread_mutex_trylock'
/usr/bin/ld: /tmp/libesdm.so.0.5.0.TzKbUI.ltrans0.ltrans.o: in function `esdm_shm_status_init':
/build/client/random/esdm/work/esdm-0.5.0/build/../esdm/esdm_shm_status.c:124: undefined reference to `sem_open'
/usr/bin/ld: /tmp/libesdm.so.0.5.0.TzKbUI.ltrans0.ltrans.o:/build/client/random/esdm/work/esdm-0.5.0/build/../esdm/esdm_shm_status.c:127: undefined reference to `sem_open'
/usr/bin/ld: /tmp/libesdm.so.0.5.0.TzKbUI.ltrans0.ltrans.o: in function `esdm_init':
/build/client/random/esdm/work/esdm-0.5.0/build/../common/mutex_w.h:71: undefined reference to `pthread_mutexattr_init'
/usr/bin/ld: /tmp/libesdm.so.0.5.0.TzKbUI.ltrans0.ltrans.o:/build/client/random/esdm/work/esdm-0.5.0/build/../common/mutex_w.h:75: undefined reference to `pthread_mutexattr_setrobust'
/usr/bin/ld: /tmp/libesdm.so.0.5.0.TzKbUI.ltrans0.ltrans.o: in function `esdm_init':
/build/client/random/esdm/work/esdm-0.5.0/build/../common/mutex.h:65: undefined reference to `pthread_rwlock_init'
/usr/bin/ld: /tmp/libesdm.so.0.5.0.TzKbUI.ltrans0.ltrans.o: in function `esdm_reinit':
/build/client/random/esdm/work/esdm-0.5.0/build/../esdm/esdm_shm_status.c:100: undefined reference to `sem_close'
/usr/bin/ld: /tmp/libesdm.so.0.5.0.TzKbUI.ltrans0.ltrans.o: in function `esdm_fini':
/build/client/random/esdm/work/esdm-0.5.0/build/../esdm/esdm_shm_status.c:100: undefined reference to `sem_close'
/usr/bin/ld: /tmp/libesdm.so.0.5.0.TzKbUI.ltrans0.ltrans.o: in function `esdm_fini':
/build/client/random/esdm/work/esdm-0.5.0/build/../common/mutex.h:43: undefined reference to `pthread_rwlock_wrlock'
/usr/bin/ld: /build/client/random/esdm/work/esdm-0.5.0/build/../common/mutex.h:52: undefined reference to `pthread_rwlock_unlock'
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.

Applying a simple patch solved this problem:

diff -Narup esdm-0.5.0.orig/meson.build esdm-0.5.0/meson.build
--- esdm-0.5.0.orig/meson.build	2022-08-14 17:33:20.000000000 +0200
+++ esdm-0.5.0/meson.build	2022-10-07 11:34:25.609583975 +0200
@@ -71,7 +71,7 @@ meson.add_dist_script(dist_script, meson
 ################################################################################
 
 cc = meson.get_compiler('c')
-dependencies = [ dependency('libprotobuf-c') ]
+dependencies = [ dependency('threads'), dependency('libprotobuf-c') ]
 
 include_user_files = [ ]

adapt ESDM_SEED_BUFFER_INIT_ADD_BITS in kernel module for ais2031 and oversample_es flag

Otherwise buffer sizes mismatch on communication between esdm and kernel modules.

aux_src should be initialized first

irq_es otherwise tries to update aux src's hash on its init, while this hash is a nullptr.

diff --git a/esdm/esdm_es_mgr.c b/esdm/esdm_es_mgr.c
index 908c72c..a07cf49 100644
--- a/esdm/esdm_es_mgr.c
+++ b/esdm/esdm_es_mgr.c
@@ -87,6 +87,7 @@ static DECLARE_WAIT_QUEUE(esdm_monitor_wait);
  * enum esdm_external_es
  */
 struct esdm_es_cb *esdm_es[] = {
+       &esdm_es_aux,
 #ifdef ESDM_ES_IRQ
        &esdm_es_irq,
 #endif
@@ -105,7 +106,6 @@ struct esdm_es_cb *esdm_es[] = {
 #ifdef ESDM_ES_HWRAND
        &esdm_es_hwrand,
 #endif
-       &esdm_es_aux
 };
 
 /******************************** ES monitor **********************************/