Code Monkey home page Code Monkey logo

rustix's Introduction

rustix

Safe Rust bindings to POSIX/Unix/Linux/Winsock syscalls

A Bytecode Alliance project

Github Actions CI Status zulip chat crates.io page docs.rs docs

rustix provides efficient memory-safe and I/O-safe wrappers to POSIX-like, Unix-like, Linux, and Winsock syscall-like APIs, with configurable backends. It uses Rust references, slices, and return values instead of raw pointers, and I/O safety types instead of raw file descriptors, providing memory safety, I/O safety, and provenance. It uses Results for reporting errors, bitflags instead of bare integer flags, an Arg trait with optimizations to efficiently accept any Rust string type, and several other efficient conveniences.

rustix is low-level and, and while the net API supports Windows Sockets 2 (Winsock), the rest of the APIs do not support Windows; for higher-level and more portable APIs built on this functionality, see the cap-std, memfd, timerfd, and io-streams crates, for example.

rustix currently has two backends available:

  • linux_raw, which uses raw Linux system calls and vDSO calls, and is supported on Linux on x86-64, x86, aarch64, riscv64gc, powerpc64le, arm (v5 onwards), mipsel, and mips64el, with stable, nightly, and 1.63 Rust.

    • By being implemented entirely in Rust, avoiding libc, errno, and pthread cancellation, and employing some specialized optimizations, most functions compile down to very efficient code, which can often be fully inlined into user code.
    • Most functions in linux_raw preserve memory, I/O safety, and pointer provenance all the way down to the syscalls.
  • libc, which uses the libc crate which provides bindings to native libc libraries on Unix-family platforms, and windows-sys for Winsock on Windows, and is portable to many OS's.

The linux_raw backend is enabled by default on platforms which support it. To enable the libc backend instead, either enable the "use-libc" cargo feature, or set the RUSTFLAGS environment variable to --cfg=rustix_use_libc when building.

Cargo features

The modules rustix::io, rustix::fd, and rustix::ffi are enabled by default. The rest of the API is conditional with cargo feature flags:

Name Description
event rustix::event—Polling and event operations.
fs rustix::fs—Filesystem operations.
io_uring rustix::io_uring—Linux io_uring.
mm rustix::mm—Memory map operations.
mount rustix::mount—Linux mount API.
net rustix::net—Network-related operations.
param rustix::param—Process parameters.
pipe rustix::pipe—Pipe operations.
process rustix::process—Process-associated operations.
procfs rustix::procfs—Utilities for reading /proc on Linux.
pty rustix::pty—Pseudoterminal operations.
rand rustix::rand—Random-related operations.
shm rustix::shm—POSIX shared memory.
stdio rustix::stdio—Stdio-related operations.
system rustix::system—System-related operations.
termios rustix::termios—Terminal I/O stream operations.
thread rustix::thread—Thread-associated operations.
time rustix::time—Time-related operations.
use-libc Enable the libc backend.

64-bit Large File Support (LFS) and Year 2038 (y2038) support

rustix automatically uses 64-bit APIs when available, and avoids exposing 32-bit APIs that would have the year-2038 problem or fail to support large files. For instance, rustix::fstatvfs calls fstatvfs64, and returns a struct that's 64-bit even on 32-bit platforms.

Similar crates

rustix is similar to nix, simple_libc, unix, nc, uapi, and rusl. rustix is architected for I/O safety with most APIs using OwnedFd and AsFd to manipulate file descriptors rather than File or even c_int, and supporting multiple backends so that it can use direct syscalls while still being usable on all platforms libc supports. Like nix, rustix has an optimized and flexible filename argument mechanism that allows users to use a variety of string types, including non-UTF-8 string types.

relibc is a similar project which aims to be a full "libc", including C-compatible interfaces and higher-level C/POSIX standard-library functionality; rustix just aims to provide safe and idiomatic Rust interfaces to low-level syscalls. relibc also doesn't tend to support features not supported on Redox, such as *at functions like openat, which are important features for rustix.

rustix has its own code for making direct syscalls, similar to the syscall, sc, and scall crates, using the Rust asm! macro. rustix can also use Linux's vDSO mechanism to optimize Linux clock_gettime on all architectures, and all Linux system calls on x86. And rustix's syscalls report errors using an optimized Errno type.

rustix's *at functions are similar to the openat crate, but rustix provides them as free functions rather than associated functions of a Dir type. rustix's CWD constant exposes the special AT_FDCWD value in a safe way, so users don't need to open . to get a current-directory handle.

rustix's openat2 function is similar to the openat2 crate, but uses I/O safety types rather than RawFd. rustix does not provide dynamic feature detection, so users must handle the NOSYS error themselves.

rustix's termios module is similar to the termios crate, but uses I/O safety types rather than RawFd, and the flags parameters to functions such as tcsetattr are enums rather than bare integers. And, rustix calls its tcgetattr function tcgetattr, rather than Termios::from_fd.

Minimum Supported Rust Version (MSRV)

This crate currently works on the version of Rust on Debian stable, which is currently Rust 1.63. This policy may change in the future, in minor version releases, so users using a fixed version of Rust should pin to a specific version of this crate.

Minimum Linux Version

On Linux platforms, rustix requires at least Linux 3.2. This is at most the oldest Linux version supported by:

  • any current Rust target, or
  • kernel.org at the time of rustix's MSRV release. The specifics of this policy may change in the future, but we intend it to always reflect “very old” Linux versions.

rustix's People

Contributors

arctic-alpaca avatar asomers avatar bjorn3 avatar carbotaniuman avatar cgwalters avatar chenx97 avatar ecnelises avatar etkneil avatar ids1024 avatar imbearchild avatar inteon avatar jordanisaacs avatar joshtriplett avatar koutheir avatar mek101 avatar morr0ne avatar newpavlov avatar niluxv avatar nivkner avatar notgull avatar oxalica avatar pchickey avatar psychon avatar ratmice avatar sunfishcode avatar supercilex avatar taiki-e avatar urgau avatar valpackett avatar yujincheng08 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

rustix's Issues

Avoid `MaybeUninit::uninit().assume_init()`

MaybeUninit::uninit().assume_init() technically has undefined behavior, even though our specific usage of it in the riscv64 and arm outline-assembly backends is very unlikely to be problematic in practice. We should make the outline assembly move the nr argument into the desired register, so that we don't need to pass uninit arguments.

I'll update the riscv64 backend first; @Urgau if you want to update the arm backend, feel free, otherwise I'll update it after I finish the riscv64 parts.

Rename "imp" to "backend"

Most documentation and comments refer to the contents of the imp directory as "backends", so it might make sense to rename the imp directory to backend.

Build failures on OpenBSD

Unfortunately it looks like there have been some regressions since #11 (completely understandable as OpenBSD is not tested on CI and is quite an odd duck wrt platform APIs).

The build failures are mostly undefined symbols in libc; here is the full build log: link

(This was using Rust 1.54.0 on OpenBSD 7.0-current; the current stable, 6.9, has an old Rust 1.51.0 that fails to build a bunch of stuff, including unsafe-io.)

I'm not sure how much work it would take to resolve these and I can't put much time into it at the moment but I figured I should record these in an issue.

`INIT_ARRAY` is forced into the binary + Strong coupling between `syscalls` and `impl::process`

Some features of imp::process such as auxv.rs's INIT_ARRAY are forced into the binary. I think their inclusion should be a feature.
It will further separate the dependencies between raw syscalls and usermode tricks.
Such a feature would be called startfile or disable-initfini-array right?

I did not dive too deep into this. Let me know what you think...

P.S. At first I tried to just comment out ::process but turns out syscalls.rs uses it.

&& crate::process::getuid() == crate::process::geteuid()

Rename rsix to rustix?

I've heard some confusion about how to pronounce rsix, and it's a little awkward that the name comes from "rs" + "ix" but is pronounced as "r" + "six". "rustix" seems like it only has one plausible pronunciation, everyone in the target audience knows what "rust" is and will surely parse it as "rust" + "ix", and "ix" is a somewhat established name for Unix-like things.

So maybe we should rename to "rustix".

Fail to build rustix with nightly

I was trying to install wizer and failed to install it.

I used the command :

cargo install wizer --all-features

with cargo 1.62.0-nightly (1ef1e0a 2022-03-31)

and got the error

error: cannot find macro `asm` in this scope
   --> /home/lola/.cargo/registry/src/github.com-1ecc6299db9ec823/rustix-0.26.2/src/imp/linux_raw/arch/inline/x86_64.rs:295:5
    |
295 |     asm!(
    |     ^^^
    |
    = note: consider importing one of these items:
            std::arch::asm
            core::arch::asm

error: could not compile `rustix` due to 14 previous errors
warning: build failed, waiting for other jobs to finish...
error: failed to compile `wizer v1.3.5`, intermediate artifacts can be found at `/tmp/cargo-installedMD7S`

Using rust 1.61 fixed my problem.

Support for BSD platforms?

Greetings! One of my occasional hobby activities is to try to compile wasmtime on unsupported platforms. I managed to get it to work last October on OpenBSD (branch) with a number of hacks in the platform code. I thought I would get back to this and try with the latest main this morning, and saw a number of build errors in posish, which seems to be the new platform glue we use -- a representative example:

error[E0432]: unresolved import `libc::posix_fadvise`                                                                                                
 --> /home/cfallin/.cargo/registry/src/github.com-1ecc6299db9ec823/posish-0.6.1/src/fs/fadvise.rs:8:5                                                
  |                                                                                                                                                  
8 | use libc::posix_fadvise as libc_posix_fadvise;                                                                                                   
  |     ^^^^^^-------------^^^^^^^^^^^^^^^^^^^^^^                                                                                                    
  |     |     |                                                                                                                                      
  |     |     help: a similar name exists in the module: `posix_madvise`                                                                             
  |     no `posix_fadvise` in the root          

and a bunch more related to fadvise, posix_fallocate, makedev, CLOCK_PROCESS_CPUTIME_ID, etc. (OpenBSD seems not to have the first two at all; and last time I tried this I had to rip out some of the process cpu-time stuff too.)

This is somewhat unsurprising, given that low-level details like this will undoubtedly (i) be missing on some platforms, or (ii) at least not be implemented in the right places in lower-level crates like libc, and BSD platforms are much more niche.

That said, I am curious: are there plans at any point to try to build/run on BSD systems, even at a best-effort ("fallocate is stubbed out with unimplemented!("system does not support this syscall")") level?

Take this as a feature request at the lowest-possible priority; it would be neat to be able to run Wasm on my little OpenBSD router, and various other folks might be using one of the BSDs in their infrastructure, but it's far from critical :-)

net/sockopt::test_sockopts test failure on s390x

I'm trying to update the rustix crate we have packaged for Fedora Linux in preparation for updating wasmtime to the latest version. However, I've encountered a failure when running the rustix test suite on s390x that has me slightly concerned.

Here's the relevant log output from cargo test from a build of rustix 0.33.5 (I need ^0.33 for wasmtime, and I can't easily test arbitrary versions on s390x due to limited hardware access; but it looks like the relevant code has not changed between v0.33.5...main anyway):

     Running `/builddir/build/BUILD/rustix-0.33.5/target/release/deps/net-c22b1583489e3ba8`
running 18 tests
test addr::encode_decode ... ok
test addr::test_unix_addr ... ok
test connect_bind_send::net_v4_acceptfrom ... ok
test connect_bind_send::net_v4_bind_any ... ok
test connect_bind_send::net_v4_connect_any ... ok
test connect_bind_send::net_v4_connect ... ok
test connect_bind_send::net_v4_sendto ... ok
test connect_bind_send::net_v4_sendto_any ... ok
test connect_bind_send::net_v6_acceptfrom ... ok
test connect_bind_send::net_v6_bind_any ... ok
test connect_bind_send::net_v6_connect ... ok
test connect_bind_send::net_v6_connect_any ... ok
test connect_bind_send::net_v6_sendto ... ok
test connect_bind_send::net_v6_sendto_any ... ok
test poll::test_poll ... ok
test sockopt::test_sockopts ... FAILED
test v4::test_v4 ... ok
test v6::test_v6 ... ok
failures:
---- sockopt::test_sockopts stdout ----
thread 'sockopt::test_sockopts' panicked at 'assertion failed: `(left == right)`
  left: `false`,
 right: `true`', tests/net/sockopt.rs:101:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
failures:
    sockopt::test_sockopts
test result: FAILED. 17 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

It looks like setting the broadcast flag does not work as expected, according to the line number in the failed assert (tests/net/sockopt.rs:101:9 from rustix 0.33.5).

I see that the commit that added those tests references some bugs in QEMU, is this one of the problems that's expected? If so, can I safely ignore this failure?

Split `futex` into separate functions

Rustix's futex function can perform several different operations, as determined by a FutexOperation argument. However, as documented in the Arguments section of Linux's man page, in some operations, some of the arguments are unused. We should consider splitting futex into multiple functions, one per signature at least, so that we can provide customized signatures for each operation.

A related crate in this space is the linux-futex crate. Another option here would be to simply port linux-futex to use rustix's existing low-level futex API internally.

Rename `Error` to `Errno`?

Renaming rustix::io::Error to rustix::io::Errno would:

  • Make it easier for users to do use rustix::io::Errno; and have it not conflict with any Error types which might be in scope.
  • More closely reflect the Unix entity this represents: an errno code.

no_std possible?

I am new to Rust and just took a peek at the source.

Would it be possible to make the linux_raw backend no_std? Or have a no_std backend?

I am looking for something just like posish: higher level than syscalls-rs, but I don't want to manually human-ize all the syscalls (and half of them are not even needed).

support: I misunderstand something about a borrowed `OwnedFd` and drop.

I'm working on examples a little more advanced than basic. I'd like to see other examples of this. I'm wanting to socat a command with a socket.

https://gitlab.com/cheako/rust-general/-/blob/0a352884935086da7e82e1866be56efd8d855a0a/agis_phiwraek/src/lib.rs#L59-100

The unix socket gets closed, causing the write to fail, and I don't understand why. Could be something to-do with try_clone()? Also, the program gets into a state where the listening socket is closed but two places it should panic never fire.

Figure out an 390x build error

In coreos/rpm-ostree#3839, they're seeing this error:

--> /builddir/build/BUILD/rpm-ostree-2022.10.110.gad3fecdc/vendor/rustix/src/imp/libc/fs/dir.rs:47:18
   |
47 | use libc_errno::{errno, set_errno, Errno};
   |                  ^^^^^  ^^^^^^^^^  ^^^^^ no `Errno` in the root
   |                  |      |
   |                  |      no `set_errno` in the root
   |                  no `errno` in the root

This is a surprising error message; it's not the "unresolved import libc_errno" error one might expect if we got the complex cfg wrong in Cargo.toml. Here, the build system seems to have correctly decided that we're using the libc dependencies, and the compiler appears to have found libc_errno. It appears it isn't finding the things inside of it 🤔 .

The errno crate doesn't have any way to disable errno, set_errno, and Errno; those are always unconditionally defined.

So I don't currently have any theories for what might be causing this.

[Q&A] Are smaller program startup times possible if `libc` isn't used?

Having recently read this article that describes how the launch time of a C program could be reduced to 1ms from 10ms I thought that maybe with rustix + origin, it would be possible to emit more optimal program startup sequences based on the actual usage of features in the program.

Is this achievable or is there a good reason for libc based programs not to typically do that?

Thanks

What might porting `std` to use `rustix` look like?

Porting Rust's std to use rustix instead of using libc directly for I/O on Unix-family platforms would factor out unsafe code, both for memory safety and I/O safety, and raw error handling, from std's main implementation code, making it easier to maintain. Secondarily, it'd also bring a number of optimizations, such as avoiding the allocation that std does on every system call that takes a string, and making system calls directly and avoiding errno.

cap-std and mustang demonstrate that rustix can do most of the I/O that std does, and the pieces that are missing should be straightforward to add. rustix and libc can coexist, so the work could be done incrementally.

rustix does depend on some things from std, though there's a working no_std branch, so we know the dependencies. The most interesting here are:

  • std::os::raw::*, CStr, Ip*Addr, SocketAddr*, IoSlice* - Rust has avoided moving these into core because they invove OS-specific types, so perhaps we could introduce a new core_ffi library and move these into it? Or, perhaps rustix should define its own versions of these which convert to and from the std versions. rustix doesn't need them to be full-featured, so it should be doable.

  • BorrowedFd, OwnedFd, etc. - Could these also make sense in a core_ffi library? Alternatively, rustix could similarly define its own.

  • CString - Similarly, this could go into a new alloc_ffi library, or rustix could similarly define its own.

Once we have the dependencies sorted out, I imagine the next steps would be to make a temporary fork of std, and port some calls to use rustix, and then make a proposal to the std maintainers and use the fork to show how it simplifies the code and factors out unsafe, as the primary motivators.

For any of this to happen, I'll need help! Both because it's a lot of work, and because if there isn't community interest in helping with this, it's a sign that this doesn't actually make sense to do.

  • Does the above sound reasonable? Is it missing anything major? Is there anything about rustix's design that should be different? What questions should I be asking here that I'm not?

  • Are there people interested and available to implement pieces of this? Many of the pieces should be straightforward, and I'd be happy to mentor people and explain what needs to be done.

Add tests for `wait`

As discussed in #165, wait is a tricky function to test from within the test harness because it depends on knowing that nothing else in the process is creating child processes which the wait might wait for. To test it, we should ideally create a new process, and have the new process create a new process and wait for it, so we can be sure of what it's waiting for.

Find a better way to handle `dup2` vs `dup3` differences

dup2 silently does nothing if the file descriptors passed to it are equal. dup3 fails. Rustix calls these functions dup2 and dup2_with, which suggests that they behave the same except for the presence of a flag word, which due to this differing failure mode, is misleading.

Options include:

  • Rename dup2_with to dup3, to match the Linux name
  • Change the behavior of rustix's dup2 to match Linux's dup3, on the theory that Linux made this change because it makes the API better.

Compile Android arm64 libwasmtime.a happens link error?

/Users/lukiyang/Work/sourcecode/DynamicFramework/core/third_party/wasmtime/lib/android/arm64-v8a/libwasmtime.a(rustix-7ff7823763dd1c95.rustix.57691c6b-cgu.7.rcgu.o): In function rustix::imp::io::syscalls::preadv::h9ff81e210c739d18': /Users/lukiyang/.cargo/registry/src/github.com-1ecc6299db9ec823/rustix-0.33.0/src/imp/libc/io/syscalls.rs:122: undefined reference to preadv64'
/Users/lukiyang/Work/sourcecode/DynamicFramework/core/third_party/wasmtime/lib/android/arm64-v8a/libwasmtime.a(rustix-7ff7823763dd1c95.rustix.57691c6b-cgu.7.rcgu.o): In function rustix::imp::io::syscalls::pwritev::hb6ef9b4b3c378517': /Users/lukiyang/.cargo/registry/src/github.com-1ecc6299db9ec823/rustix-0.33.0/src/imp/libc/io/syscalls.rs:137: undefined reference to pwritev64'

Share a common richly-typed description of Linux syscalls with other projects

Hi, I've followed Rustix (and other downstream projects) with great interest - very exciting! I'm excited for the safer future it can bring!

A really detailed, richly-typed, machine-readable description of the types/behavior of Linux system calls would, I think, be really useful for Rustix and other projects like strace, gvisor, WSL1, etc. This is a big missing part of the Linux ecosystem, I think. (And I've seen other projects, e.g. strace, bemoan the lack of it before too.)

In theory, you could use it to generate Rustix instead of implementing it by hand - or at least to sanity-check that Rustix is compatible with ground truth. I've been wanting this kind of shared description from my work on https://github.com/catern/rsyscall which I think has a similar philosophy to Rustix. (specifically the "type-safe" and "low-level" bullet points in the summary)

Have you thought about such a description? Do you think it would be useful to you?

powerpc64le linux_raw support

Unless someone else is already working on it, I'm thinking about adding powerpc64le support in the linux_raw backend soon.

Use `dlsym` to resolve `__vdso_clock_gettime` etc.?

In dynamically linked executables, we should be able to dlopen the "linux-vdso.so" library and use dlsym to obtain symbols like "__vdso_clock_gettime" instead of parsing the vDSO ELF image manually.

An alternative would be to say that since a dynamically-linked executable has libc anyway, the time routines should just use the libc clock_gettime APIs.

std::process::Command and I/O safety

Following up on a discussion in #97: std::process::Command has a safe API for spawning child processes. Its documentation doesn't mention non-O_CLOEXEC file descriptors, but from some experimenting, child processes do inherit non-O_CLOEXEC file descriptors.

Does this violate I/O safety?

When a program calls Command::spawn, all existing memory in the process is replaced with a new program image, so Rust's safety guarantees have typically considered that to be the end of one program and the beginning of a new one. However, this is an area where file descriptors are different from pointers. FIle descriptors not marked O_CLOEXEC persist into the next program. In effect, spawn is implicitly obtaining the values of all such file descriptors in the process, even those meant to be encapsulated, and "passing" them to another program where they could be accessed. That has the shape of an I/O safety violation. And in practice, leaking file descriptors to child processes is a real security problem.

One possible way out of this is to say that safe code can't create non-O_CLOEXEC file descriptors. Rust's std already does open all file descriptors as O_CLOEXEC whenever it can, and sets them to O_CLOEXEC even when it can't create them that way. Perhaps then, rustix's openat function should always set O_CLOEXEC too, and perhaps there should be a special function named something like unsafe fn dup2_no_cloexec for creating non-O_CLOEXEC file descriptors for when those are really needed.

One tricky issue is that not all OS's can set O_CLOEXEC atomically in all cases. Rust uses ioctl with FIOCLEX to set the CLOEXEC flag as soon as it can, but there is a window where another thread could call exec and capture the file descriptor before it has O_CLOEXEC set. A related issue is that users of std::process::Command may already be creating non-O_CLOEXEC file descriptors within safe functions. A possible long-term path out of this is to say that post-exec programs accessing file descriptors passed through exec is in the same category as using a debugger on the pre-exec program, so it's out of scope for I/O safety, and then to provide a new safe way to pass file descriptors to child processes and gradually encourage the Rust ecosystem to adopt it.

Support opting-in to libc even on x86_64 etc.

As you know I like a lot of the rustix design, the I/O safety, etc. But the "no-libc" path from my use cases I feel brings more risk than reward. One blunt way to look at this is my employer also employs people to work on glibc. We need to support 4 different architectures. The current status quo of using direct syscalls on e.g. x86_64 but libc on ppc64le means that suddenly my software could have ppc64le specific bugs. (And we already have problems with CI capacity on that arch)

Basically, WDYT about exposing use-libc which would turn on rustix_use_libc? (Or is this possible today?)

0.35 Release Plan

The main branch has several semver-incompatible changes, and a fair amount of overall code churn. Also, rustix has a lot of users that need stability. So here's the plan:

At some point, the I/O safety feature in Rust nightly will hopefully be stabilized. Once that happens, I plan to update io-lifetimes' API to match it, and also to switch to using the types and traits in std instead of its own copies, on Rust versions that support it. That will require a semver bump in io-lifetimes, which will also require a semver bump in rustix, so my plan is to do this in the rustix 0.35 release along with the other semver-incompatible changes, to minimize the number of semver bumps for users.

Before releasing 0.35, I'll also go through the test-users.yml script, figuring out what changes downstream users will need, and adding some of rustix's new users. I'll also do some testing on platforms not covered in CI, including old Linux versions and FreeBSD. This should help ensure stability before the release, and prepare patches to send to downstream users to help them with the API changes once the release is up.

Switch from OsString to CString

For functions like readlinkat, we should return a CString instead of an OsString. OsString abstracts over Unix and Windows, which rsix doesn't need to do. And keeping things as CStrings means we can pass them back into rsix APIs without requiring it to re-add NUL terminators.

Whats the deal with the `use std::` in the middle of `linux_raw::io::syscalls.rs` when `target_arch = "aarch64"`

I am compiling a no_std code using this library, it works fine with x86_64 but when I set target_arch = "aarch64" it fails with:

error[E0433]: failed to resolve: use of undeclared crate or module `std`
   --> /home/a/.cargo/registry/src/github.com-1ecc6299db9ec823/rustix-0.34.6/src/imp/linux_raw/io/syscalls.rs:560:13
    |
560 |         use std::os::unix::io::AsRawFd;
    |             ^^^ use of undeclared crate or module `std`

#[cfg(any(target_arch = "aarch64", target_arch = "riscv64"))]

Provide low-level access to poll, epoll, dir APIs?

Most of rsix is simple wrappers around syscalls, however rsix currently has a few higher-level APIs: poll, epoll, where it's difficult to make the low-level APIs safe. And most of rsix doesn't hide significant differences between platforms, but it has Dir which hides differences between libc readdir and Linux getdents. These high-level APIs also include dynamic allocations, where most of the rest of rsix doesn't.

Is this the right approach? Or should we expose just the low-level APIs, using unsafe and being non-portable when it's unavoidable? Or, should we provide both?

RawPid should be i32

pid_t is universally signed, however rustix's linux_raw backend makes RawPid be u32. This complicates porting code from libc to rustix.

Replace `ZStr` with `CStr`

Rust has moved CStr into libcore and CString into liballoc: rust-lang/rust#94079

As an aside, I'm still fond of the idea of using u8 instead of c_char for this, because there isn't any reason to care about the signedness of C's char in nul-terminated string contexts—if you want to interpret the contents, you need to know the encoding, and the signedness of char doesn't help with that. Exposing c_char at this level just means that more places are exposed to gratuitous platform differences. But it's also true that in practice this issue is just a minor nuisance. Rust already had CStr with c_char, and replacing it with something that uses u8 would create a lot of churn, so keeping it makes sense.

So now that CStr has moved to core in nightly, that eliminates the need for ZStr in rustix, so rustix should switch to using CStr now.

Split `proc_self_fd` and `ttyname` into a separate crate

Now that rsix no longer uses /proc/self/auxv to locate the auxv vector, and rsix has matured to the point where it seems pretty clear it won't need /proc/self/maps or other things in /proc, the only thing in rsix using /proc now is ttyname, so I'm considering splitting the /proc code, and ttyname, into a separate crate.

This would mean that rsix itself would no longer need to depend on once_cell, and would no longer have this implicit static file descriptor code in it, which be nice for users that don't need that functionality.

@Urgau Would you be ok be ok if ttyname were split out, along with the proc_self_fd code, to a separate crate?

Reading into uninitialized buffers

Currently, read takes a &mut [u8]. That's fine for many use cases, but it does require the buffer to be initialized before calling read, so it's currently not possible to read directly into uninitialized memory.

For Mustang, the C read API takes a raw pointer which may be pointing to an uninitialized buffer. Currently mustang does slice::from_raw_parts_mut, however this has undefined behavior if the slice is uninitialized.

Check in precompiled .o files for the outline assembly?

In some places, rsix's dependency on cc has been one barrier to adoption.

The cc dependency exists just to assemble Rsix's .S files containing assembly code for making syscalls on stable Rust:

src/imp/linux_raw/arch/outline/riscv64.S
src/imp/linux_raw/arch/outline/x86_64.S
src/imp/linux_raw/arch/outline/arm.S
src/imp/linux_raw/arch/outline/aarch64.S
src/imp/linux_raw/arch/outline/x86.S

The Rustonomicon discusses shipping pre-assembled .o files. We should consider that in rsix: these .S files are pretty small and don't change very often, and they're already specific to one OS. Shipping pre-assembled .o files would also speed up build times for downstream users.

Fork and safety

This is motivated by #76, a port of std to rustix.

std calls fork and execve (via execvp) to launch child processes in some situations. Rustix's execve can allocate, to convert args to CString, and to nul-terminate the args array. But POSIX says that child processes of fork can't call anything async-signal-unsafe, including allocation. So how can we port std's fork and execve calls to Rustix?

This is std, so, unlike mustang, we can't assume anything special about the global allocator.

Rust doesn't require users to pick a global allocator which registers its locks with pthread_atfork. It isn't documented, and there are notable allocators today that don't always do it. Getting them to do it is tricky, as registering with pthread_atfork at the right time is tricky. And even when allocators do it, they're often on thin ice because malloc, pthread_mutex_lock, pthread_mutex_unlock, and other relevant functions aren't guaranteed to be async-signal-safe in libc.

It's looking like what we'd really need to do to make fork truly usable would be to define a new set of rules defining what Rust code and Rust allocator implementations need to do to play well with fork, and then work with independent projects to get them to follow the rules.

But even if we do that, and get all the independent projects to follow all the rules, the environment in the child between fork and execve will always be really awkward. No matter how safe we make things, it'll still have things like file locks inherited from the parent, MAP_SHARED regions shared with the parent, file descriptors dup'd from the parent and sometimes renumbered, and a whole copy of all the parent's memory, which can lead to many surprising situations. Related considerations are that fork also isn't a great fit for wasm programs, and therefore also WASI. And it's not portable to Windows and other non-Unix platforms. And in C, POSIX says that the child can only call async-signal-safe functions, but the Rust ecosystem in general doesn't document what's async-signal-safe and what's not. And, in general there are good reasons for wanting to move away from fork anyway.

So far, the overall direction in the rustix ecosystem has been to push for making fork safer and more usable, because that's rustix's overall API approach. However, the above issues motivate looking for other directions.

Another possible direction is to treat fork as an inherently limited tool. It's a tool to use with execve to launch child processes in specialized ways that posix_spawn isn't quite flexible enough to support, and nothing more. As such, it isn't as important to have a safe interface to fork and execve themselves, and to support general-purpose things like allocating or computing random numbers. We just need to support the things that Rust's std needs in its fork and execve code.

We could support that by adding a non-allocating version of execve and execvp to rustix::runtime, which are unsafe and take char** arguments and NUL-terminated arrays just like libc's execve does, and just like the Unix syscalls do. And similar for any other function Rust needs to be non-allocating between fork and exec (though at a first pass through the current code, I don't see any—Rust is already careful to avoid anything that allocates in this path).

That way, std can call rustix's fork and execve directly, rather than calling them through origin. This will mean not running any pthread_atfork handlers, but that seems to be ok, because posix_spawn says it's unspecified whether the fork handlers run, and we're thinking of fork as just a generalized posix_spawn use case.

Mustang, for its part, can then move its fork and at_fork out of origin and into c-scape. Perhaps initially it won't support pthread_atfork at all, but that could be added if needed for C compatibility.

time-related problems with powerpc64le linux_raw support

It's me again :) After expanding the architectures I ran the rustix test suite for, I found another architecture-specific failure in the test suite. This time, it looks like the - relatively new - linux_raw support for powerpc64le still has some problems.

In particular, the time tests seem to be problematic, in not obviously deterministic ways, but always with an assertion failure. Looks like the powerpc64le machine I have access to doesn't necessarily behave as the code expects (line number is from rustix 0.33.6, but according to the commit history, this part of the code has not been changed since, as far as I can tell).

thread '<foo>' panicked at 'assertion failed: !ptr.is_null()', src/imp/linux_raw/vdso_wrappers.rs:375:13

What's weird is that the test that failed changed, depending on which tests I skipped:

  • first, all tests from tests/time passed, except dynamic_clocks::test_conditional_clocks
  • second, I added --skip test_conditional_clocks to cargo test --, and then dynamic_clocks::test_dynamic_clocks failed instead
  • third, I did --skip dynamic_clocks, but then monotonic::test_monotonic_clock failed instead
  • last, I did both --skip dynamic_clocks and --skip test_monotonic_clock, at which point no new test failures occurred

Since actual failing test depended on which other tests were running (or not), this looks like some "hidden shared global state" to me ... or it's just a Heisenbug :)

Possibly relevant environment info:

  • Fedora Rawhide, 36, 35, 34 on powerpc64le
  • running on Linux 5.16.18 host
  • Rust / Cargo 1.60.0 stable

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.