Not sure when Nightly introduced a change that broke the compilation of cap-primtives here, I'm on latest from today cargo 1.64.0-nightly (85b500cca 2022-07-24).

latest stable (1.62) and beta (1.63.0-beta.7) works fine.

this is the error one runs into when building with Nightly:

error[E0277]: the trait bound `file_type::FileType: std::sealed::Sealed` is not satisfied
   --> cap-primitives\src\fs\file_type.rs:134:6
    |
134 | impl std::os::windows::fs::FileTypeExt for FileType {
    |      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `std::sealed::Sealed` is not implemented for `file_type::FileType`
    |
    = help: the following other types implement trait `std::sealed::Sealed`:
              Child
              Command
              ExitCode
              ExitStatus
              ExitStatusError
              OsStr
              OsString
              Simd<f32, N>
            and 2 others
note: required by a bound in `std::os::windows::fs::FileTypeExt`

Repro:

• cargo +nightly build --workspace

Several functions in src/fs/dir.rs and cap-async-std/src/fs/dir.rs have an AsRef<Path> parameter, meaning they are monomorphized for each kind of type passed as a path. We should use the pattern in rust-lang/rust@564c569 to monorphize functions containing non-trivial code.

Is it worth defining Path, PathBuf and associated types in cap-std? They could mostly be thin wrappers around their std::path counterparts, but they'd omit several ambient-authority methods: metadata, read_link, read_dir, symlink_metadata, and canonicalize.

Hi, I'm working on porting away from openat to cap-std. I maintain the openat-ext crate that has a lot of "high level" helpers:
https://docs.rs/openat-ext/latest/openat_ext/trait.OpenatDirExt.html

(We can ignore the FileExt trait for now)

Of these, I find myself using e.g. open_file_optional() quite often.

But the other bigger and more interesting hunk of functionality is https://docs.rs/openat-ext/latest/openat_ext/trait.OpenatDirExt.html#tymethod.new_file_writer

Basically we have a nice wrapper around O_TMPFILE + linkat.

Would it make sense to try to add these to cap-fs-ext or should I start with a new, separate crate?

On Linux, /proc/self/fd/* are symlink-like objects on which one can do readlink to obtain canonical paths to open file descriptors. Musl for example does this to implement realpath, which is what Rust's libstd uses to implement fs::canonicalize on Unix-family platforms. cap-std's canonicalize doesn't provide an absolute path, so it would need to strip leading components up to the point where the Dir's directory starts.

Caution is indicated; Linux's /proc has a history of being tricky to work with. These symlink-like objects are "magic links" and have special behavior. And /proc may be mounted in a context where users may have the ability to manipulate it.

Yay! I'm really glad to see this work. It's been on my wish-list since 2012 dckc/rust#2 . And again in https://www.madmode.com/2017/ocap-wish-list.html

I just added this to https://github.com/dckc/awesome-ocap#lib

see also hickory-dns/hickory-dns#71

This isn't so much an issue as just getting in touch. Feel free to close this issue. (I'll close it myself after a bit, otherwise.)

test code

let path = std::path::Path::new("C:\\Users\\path\\to\\folder"); //works fine
let path = std::path::Path::new("\\\\?\\UNC\\Mac\\path\\to\\folder"); //throw an error
let dir = std::fs::OpenOptions::new()
  .read(true)
  .custom_flags(33554432u32)
  .open(path).unwrap();
let entries = cap_std::fs::Dir::from_std_file(dir).entries();
dbg!(entries);

reason

After debugged, I found cap_primitives/src/windows/fs get_path cause this problem.

    let wide_final = if wide.starts_with(&['\\' as u16, '\\' as _, '?' as _, '\\' as _]) {
        &wide[4..] //throw error
        //&wide //change to this and it works
    } else {
        &wide
    };

when path start with \\\\?\\, \\\\?\\ will be remove, and \\\\?\\UNC\\Mac\\path\\to\\folder changed to UNC\\Mac\\path\\to\\folder, so a path not found error be throwed.

dependency chain

the dependency chain is:

• cap_std::fs::dir::Dir::entries
• cap_primitives::fs::read_dir::read_base_dir
• cap_primitives::windows::fs::read_dir_inner::ReadDirInner::read_base_dir
• cap_primitives::windows::fs::read_dir_inner::ReadDirInner::new_unchecked
• cap_primitives::windows::fs::get_path::concatenate
• cap_primitives::windows::fs::get_path::get_path

cargo.toml

cap-std = "1.0.14"

cap-std's fs_utf8 module was originally built as part of an experiment with UTF-8 paths in WASI, and one of the questions was whether we could make it possible to access files with non-Unicode encodings while still giving applications valid UTF-8 strings. This led to an experiment known as arf-strings, which is a scheme for losslessly encoding non-UTF-8 bytes within a UTF-8 filename. arf-strings are currently built into fs_utf8, so specially crafted UTF-8 strings can be used to create and manipulate non-UTF-8 filenames.

However, at least some users of fs_utf8 will not likely want this behavior, so cap-std should have a way to disable it, and possibly should disable it by default to be less surprising.

@repi

In #321 the cap-primitives crate updated its dependency on fs-set-times from 0.19.x to 0.20.x, but the fs-set-times types are part of the public API of cap-primitives via the from_std and into_std methods, meaning that upgrading a major version of a dependency is a breaking change on behalf of the cap-primitives crate.

This is causing downstream issues where Wasmtime 10.0.1 no longer compiles with crates.io, for example.

FreeBSD 13.1 introduced an F_KINFO fcntl command, which looks up a file descriptor's path. It's exactly what cap-primitives/rustix/fs/file_path needs. It's much more efficient than file_path_by_searching, and isn't vulnerable to the #330 bug. However, it can fail in certain circumstances, so file_path_by_searching should still be used as a fallback.

https://man.freebsd.org/cgi/man.cgi?query=fcntl&sektion=2

file_path_by_searching tries to identify a directory's pathname by iterating through its parent's children and matching the dev and ino fields. The problem is that if the directory in question is a file system root, then its ino won't match. Readdir will show the ino of the underlying mountpoint, whereas the directory's own ino method will show the ino of its root directory.

Steps to Reproduce

On a non-Linux, non-Darwin system (tested on FreeBSD 15.0-CURRENT), set TMPDIR to a file system's mountpoint. This is the default on a ZFS-root system, where /tmp is a separate ZFS file system.

cd cap-primitives
cargo test --lib fs::dir_paths

We currently use a simple target-independent implementation of Dir::copy. libstd has several platform-specific optimizations that would be good to port to cap-primitives and then expose via Dir::copy:

• https://github.com/rust-lang/rust/blob/master/src/libstd/sys/unix/fs.rs#L1171
• https://github.com/rust-lang/rust/blob/master/src/libstd/sys/windows/fs.rs#L852

Note this repo's convention for citing code copied from Rust's libstd: https://github.com/sunfishcode/cap-std/blob/a0b507c4790ccf743cd594b26d491d628d02d75e/src/fs/dir.rs#L207

I'm trying to run some code that uses cap-std on Android 10 (on a kernel that definitely does not support the syscall), and I hit the following crash in openat2, called by open_beneath from open_impl:

 signal 31 (SIGSYS), code 1 (SYS_SECCOMP), fault addr --------
 Cause: seccomp prevented call to disallowed arm64 system call 437

 backtrace:
       #00 pc 00000000000703d0  /apex/com.android.runtime/lib64/bionic/libc.so (syscall+32) 
       #01 pc 0000000002e8e404  ourapp.so (rsix::imp::libc::syscalls::openat2::h04809da1ec63d9ef+44) 
       #02 pc 0000000002e8aa9c  ourapp.so (cap_primitives::rsix::linux::fs::open_impl::open_beneath::h3811b95d26505c84+312) 
       #03 pc 0000000002e8a800  ourapp.so (cap_primitives::rsix::linux::fs::open_impl::open_impl::h725b24260261d337+40)
       ...

Looking back up into cap-std, it seems that the detection method for presence of openat2 is inadequate, because instead of simply returning ENOSYS as the linked code assumes, Android forcibly kills the process with a SIGSYS signal.

So there needs to be some sort of kernel version check instead I guess?

walkdir is a popular cross platform Rust library for efficiently walking a directory recursively. It'd be interesting to either wrap or port this crate to cap-std to allow recursively walking the subtree of a Dir.

Or would trying to Dir::open(allowed_dir, "COM1") lead to opening thing not intended by the giver of allowed_dir?

Feel free to close this as out of scope, but the idea is:

1. tag functions that are designed to follow ocap discipline:

#[tagged_safe(ocap="tagsafe_std_ocap.txt")]
extern crate std as _std;

#[tag_safe(ocap)]
fn cap_main<W>(out: &mut W) -> io::Result<()>

2. tag functions that use ambient authority as unsafe w.r.t. ocap discipline. The tagsafe_std_ocap.txt file db would express the equivalent of:

#[tag_unsafe(ocap)]
use std::fs::File::open;

3. Use the tag_safe linter to go BZZZT if an ocap safe function calls, directly or indirectly, an ocap unsafe function.

refs from around Nov 2016:
thepowersgang/tag_safe#1
https://github.com/dckc/larust-tame
dckc/rust#2 (comment)

See coreos/rpm-ostree#3771

Basically we have a use case where we want a read_link() that does escape checking for the link name, but not the link target. (The current process is not going to follow the link target, so this is safe)

If you agree with this need I'm happy to do a PR to add those APIs here.

Currently Dir::symlink, Dir::symlink_file, Dir::symlink_dir, and the Unix-domain socket functions are conditionally declared, depending on platform. libstd tends to not do things like this; instead, it has traits like MetadataExt, OpenOptionsExt, and so on, to allow platforms to add additional functionality to otherwise portable types. Should cap_std do something similar, and have platform-specific DirExt traits, with the platform-specific symlink functions in them?

create_dir_all uses the algorithm from libstd for recursively creating all components. However, cap-primitives' mkdir function performs a system call per path component per call, so calling it once per path component leads to a quadratic number of system calls.

We should add a create_dir_all function in cap-primitives which, for each path component, creates a directory for that component (ignoring a io::ErrorKind::AlreadyExists error), opens the newly created directory, and then uses the resulting handle as the base for handling the next component.

Similar to open_manually, it should handle .. components by keeping a stack of handles as it goes, so that it can simply pop an entry from that stack to ascend to the parent.

And then, the cap-std and cap-async-std create_dir_all routines can use it.

I have just discovered the cap-std crates and I'm really excited about them!

My understanding of rand is that it's full of code that doesn't rely on ambient authority, then also includes some ambient authorities such as rand::thread_rng.

Would it work better for everyone if we could separate out the ambient authorities of rand in order to reuse everything else?

How much of cap-rand is duplicate code?

FreeBSD 13.0 added a O_RESOLVE_BENEATH flag to open which appears to implement similar behavior to Linux's openat2's RESOLVE_BENEATH flag, which should allow us to implement sandboxed open with a single system call.

Inside the Dir::entries iterator, calling dir_entry.metadata()? succeeds, but the .ino() accessor panicks, only on windows, with a message that says:

`ino` depends on a Metadata constructed from an open `File`

When running a simple Rust directory listing module on my C:\ drive as a test, cap-std will throw a panic :
'ino' depends on a Metadata constructed from an open 'File'

This can be reproduced through wasmtime with :
wasmtime run --dir C:\ list_dir.wasm -- C:\

This older issue #142 seems to be the same problem although it was closed with claims that it had been fixed ?
Did the bug get re-introduced somehow ?

I unfortunately don't have access to my list_dir project but its essentially only std::fs::read_dir(argv[1])

The important parts in my panic backtrace are :

wasmtime 0.26.0
    wasi-common 0.26.0    (wasi_common::snapshots::preview_1::wasi_snapshot_preview1::fd_readdir)
        wasi-cap-std-sync 0.26.0    (wasi_cap_std_sync::dir::{{impl}}::readdir)
            cap-fs-ext 0.13.9   (cap_fs_ext::metadata_ext::{{impl}}::ino)

The following line assumes that a file's timestamp is nonnegative. But that's not always a valid assumption. Files can date from before 1970. In particular, fuse-ext2 sets all files' st_birthtime fields to -1, leading to crashes like this:

called `Result::unwrap()` on an `Err` value: TryFromIntError(())
stack backtrace:
...
  19:          0x160fe5d - cap_primitives::rustix::fs::metadata_ext::system_time_from_rustix::h952f446b7ef9aa09
                               at /usr/home/asomers/.cargo/registry/src/index.crates.io-6f17d22bba15001f/cap-primitives-2.0.0/src/rustix/fs/metadata_ext.rs:274:55
  20:          0x160fb4c - cap_primitives::rustix::fs::metadata_ext::MetadataExt::from_rustix::h18ac86e5705d6acb
                               at /usr/home/asomers/.cargo/registry/src/index.crates.io-6f17d22bba15001f/cap-primitives-2.0.0/src/rustix/fs/metadata_ext.rs:145:22
  21:          0x16094ff - core::ops::function::FnOnce::call_once::he60a80a9d504abb8
                               at /rustc/2f5df8a94bb3c5fae4e3fcbfc8ef20f1f976cb19/library/core/src/ops/function.rs:250:5
  22:          0x161cc81 - core::result::Result<T,E>::map::hafcd0d645ffa7c5b
                               at /rustc/2f5df8a94bb3c5fae4e3fcbfc8ef20f1f976cb19/library/core/src/result.rs:746:25
  23:          0x1615139 - cap_primitives::rustix::fs::stat_unchecked::stat_unchecked::h51e92270a2f5fa53
                               at /usr/home/asomers/.cargo/registry/src/index.crates.io-6f17d22bba15001f/cap-primitives-2.0.0/src/rustix/fs/stat_unchecked.rs:78:8
  24:          0x16176b9 - cap_primitives::fs::manually::open::stat::h9a9a53c599c6d4ae
                               at /usr/home/asomers/.cargo/registry/src/index.crates.io-6f17d22bba15001f/cap-primitives-2.0.0/src/fs/manually/open.rs:477:32
  25:          0x14385c0 - cap_primitives::fs::stat::stat::h93661c54d61d894b
                               at /usr/home/asomers/.cargo/registry/src/index.crates.io-6f17d22bba15001f/cap-primitives-2.0.0/src/fs/stat.rs:15:18
  26:          0x155ac2a - cap_std::fs::dir::Dir::symlink_metadata::he991c86cd1a40ff7
                               at /usr/home/asomers/.cargo/registry/src/index.crates.io-6f17d22bba15001f/cap-std-2.0.0/src/fs/dir.rs:386:9
...

Here is a minimal test case:

fn main() {
    let f = tempfile::NamedTempFile::new().unwrap();
    let negative_one = nix::sys::time::TimeVal::new(-2, 0);
    nix::sys::stat::utimes(f.path(), &negative_one, &negative_one).unwrap();
    let aa = cap_std::ambient_authority();
    let d = cap_std::fs::Dir::open_ambient_dir(f.path().parent().unwrap(), aa).unwrap();
    dbg!(d.symlink_metadata(f.path().file_name().unwrap()).unwrap());
}

Rust 1.56 stablizes std::os::unix::fs::chroot. Should cap-std add a chroot function that can be called on a Dir?

It'd be implemented in terms of fchroot on platforms that have that, or chroot on /proc/self/fd/{} on Linux.

I have a slight question about attenuation, let me take cap_std::fs::Dir::remove_file as an example,
It seems to rely upon (from the rust perspective) interior mutability, as in it takes an &self, rather than an &mut self.

I assume to attenuate a Dir into say a ReadOnlyDir, i'd basically need to build a wrapper around Dir which only exposes those functions which do not exercise write permission.

If for instance it did take an &mut, Dir itself could be used to express read-only and writable directories... I'm not sure actually taking an &mut would be the right way to expose this, it entails uniqueness as well as mutation, I'd need to think about it, But looking through the API it the first question that came to mind.

cap-std crates currently use unsafe to indicate function which utilize ambient authorities, for example Dir::open_ambient_dir. As discussed here, this follows the precedent of Rust's standard library in making File::from_raw_fd be unsafe.

However, when porting code to cap-std, such as this Web server example and looking at it from the perspective of a user of the API incrementally adopting it, the use of unsafe has felt somewhat uncomfortable. So, what's the best approach here?

Some possible options include:

• Write a linting system. See also this issue. This is a fair amount of work, and would be great for users that want to scan large codebases for ambient authorities, however it'd be less convenient for incremental users, such as the Web server example mentioned above.
• Put unsafe functions in a separate module, or even a separate crate.
• Continue to use unsafe.

How would one create a new root directory with cap-std::fs (that doesn't exist on disk already) with ambient authority and get a Dir for it?

Dir::open_ambient_dir fails on opening directories that do not exist, and there is no Dir::create_ambient_dir, and you need to have a Dir in the first place to use Dir::create_dir or Dir::create_dir_all.

For now working around this by using std::fs::create_dir_all before using Dir::open_ambient_dir but would like to be able to get rid of that.

Are we missing some functionality here in in cap-std or am I missing something obvious?

Syscalls open2 and statx are not available on some Android devices and thus calling them causes seccomp to kill the process directly.

Here are the crash logs from two devices:

04-09 16:31:57.811 19810 19810 F DEBUG   : *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
04-09 16:31:57.811 19810 19810 F DEBUG   : Build fingerprint: 'HUAWEI/HMA-AL00/HWHMA:10/HUAWEIHMA-AL00/10.1.0.163C00:user/release-keys'
04-09 16:31:57.812 19810 19810 F DEBUG   : Revision: '0'
04-09 16:31:57.812 19810 19810 F DEBUG   : ABI: 'arm64'
04-09 16:31:57.814 19810 19810 F DEBUG   : SYSVMTYPE: Maple
04-09 16:31:57.814 19810 19810 F DEBUG   : APPVMTYPE: Art
04-09 16:31:57.815 19810 19810 F DEBUG   : Timestamp: 2023-04-09 16:31:57+0800
04-09 16:31:57.815 19810 19810 F DEBUG   : pid: 19607, tid: 19649, name: Thread-3  >>> org.flos.phira <<<
04-09 16:31:57.815 19810 19810 F DEBUG   : uid: 10397
04-09 16:31:57.815 19810 19810 F DEBUG   : signal 31 (SIGSYS), code 1 (SYS_SECCOMP), fault addr --------
04-09 16:31:57.815 19810 19810 F DEBUG   : Cause: seccomp prevented call to disallowed arm64 system call 291
04-09 16:31:57.815 19810 19810 F DEBUG   :     x0  0000000000000066  x1  0000007487e2af40  x2  0000000000000100  x3  0000000000000fff
04-09 16:31:57.815 19810 19810 F DEBUG   :     x4  0000007487e2ac60  x5  0000000074617473  x6  0000000074617473  x7  0000000078746174
04-09 16:31:57.815 19810 19810 F DEBUG   :     x8  0000000000000123  x9  0000000000000001  x10 0000000000004001  x11 0000000000000000
04-09 16:31:57.815 19810 19810 F DEBUG   :     x12 0000007580e21223  x13 0000007580e23a00  x14 00000074881b45cc  x15 0000000000000003
04-09 16:31:57.815 19810 19810 F DEBUG   :     x16 0000007488d592e8  x17 000000757e42cba0  x18 0000007486fd2000  x19 0000007487e2ae38
04-09 16:31:57.815 19810 19810 F DEBUG   :     x20 0000007487e2af40  x21 0000000000000fff  x22 0000000000000100  x23 0000000000000066
04-09 16:31:57.815 19810 19810 F DEBUG   :     x24 00000074e8fed840  x25 0000007487e2af40  x26 0000000000000025  x27 0000007488d66000
04-09 16:31:57.815 19810 19810 F DEBUG   :     x28 00000074880ce58e  x29 0000000000000000
04-09 16:31:57.815 19810 19810 F DEBUG   :     sp  0000007487e2ab60  lr  00000074889908e8  pc  000000757e42cbc0

04-09 00:34:33.772  5968  5968 F DEBUG   : *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
04-09 00:34:33.772  5968  5968 F DEBUG   : Build fingerprint: 'Redmi/rubens/rubens:13/TP1A.220624.014/V14.0.23.4.2.DEV:user/release-keys'
04-09 00:34:33.772  5968  5968 F DEBUG   : Revision: '0'
04-09 00:34:33.772  5968  5968 F DEBUG   : ABI: 'arm64'
04-09 00:34:33.772  5968  5968 F DEBUG   : Timestamp: 2023-04-09 00:34:32.513516823+0800
04-09 00:34:33.772  5968  5968 F DEBUG   : Process uptime: 6s
04-09 00:34:33.772  5968  5968 F DEBUG   : Cmdline: org.phira
04-09 00:34:33.772  5968  5968 F DEBUG   : pid: 5723, tid: 5755, name: Thread-4  >>> org.phira <<<
04-09 00:34:33.772  5968  5968 F DEBUG   : uid: 10195
04-09 00:34:33.772  5968  5968 F DEBUG   : tagged_addr_ctrl: 0000000000000001 (PR_TAGGED_ADDR_ENABLE)
04-09 00:34:33.772  5968  5968 F DEBUG   : signal 31 (SIGSYS), code 1 (SYS_SECCOMP), fault addr --------
04-09 00:34:33.772  5968  5968 F DEBUG   : Cause: seccomp prevented call to disallowed arm64 system call 437
04-09 00:34:33.772  5968  5968 F DEBUG   :     x0  0000000000000094  x1  00000078e7d29418  x2  00000078e7d293a0  x3  0000000000000018
04-09 00:34:33.772  5968  5968 F DEBUG   :     x4  000000000000000a  x5  00000000666e693a  x6  00000000666e693a  x7  000000006f666e69
04-09 00:34:33.772  5968  5968 F DEBUG   :     x8  00000000000001b5  x9  0000000000000000  x10 0000000a00000000  x11 000000000000000a
04-09 00:34:33.772  5968  5968 F DEBUG   :     x12 0000000000000000  x13 00000000ffffffbf  x14 00000078ea360b99  x15 0000000000000003
04-09 00:34:33.772  5968  5968 F DEBUG   :     x16 00000078eae163b8  x17 0000007a046a40c0  x18 00000078e78e8000  x19 00000078e7d29630
04-09 00:34:33.772  5968  5968 F DEBUG   :     x20 b4000078db6322d0  x21 0000000000080000  x22 0000000000000000  x23 0000000000000005
04-09 00:34:33.772  5968  5968 F DEBUG   :     x24 00000078e7d29418  x25 0000000000000006  x26 0000000000000004  x27 0000000500000000
04-09 00:34:33.772  5968  5968 F DEBUG   :     x28 00000078eae23000  x29 0000000000000000
04-09 00:34:33.772  5968  5968 F DEBUG   :     lr  00000078eaa50610  sp  00000078e7d293a0  pc  0000007a046a40e0  pst 0000000000001000

cap_std version: 1.0.12

The 1.0 release is blocked, but users are encountering #270 now. We have a 0.26-patch2 prerelease with the fix, but I think it now makes sense to do a regular 0.26 release, to get the fix out, even though it will be another semver bump before 1.0.

The 0.26 branch has the needed patches already, so I'm expecting to just do the 0.26 release from there.

Can I start a process that is constrained to only having access to within a Dir?

(I understand that not all OSes may enforce this restriction).

I can use system_interface::GetSetFdFlags::set_fd_flags on a file to set FdFlags::{DSYNC, SYNC, NONBLOCK, RSYNC} after opening it. Can OpenOptions support these as well?

The present crate could be quite convenient a way to get some std-ish functionality into embedded systems that otherwise don't implement all of std (and may not even have a generic allocator, let alone a generic one).

Usages

I see two cases where this would be useful:

• Embedded operating systems often provide some functionalities of std, but only conditionally (eg. RIOT's file system is fully optional), and with reduced functionality.

  If a generic library (say, SHA-1'ing a file) were implemented using trait-based cap-std, it might have a fn checksum(impl cap_std::fs::File) -> Hash;, and could be used with a File passed in from the embedded OS's file system.

• Using non-OS file systems.

  Some desktop environments that are not OSes in the Rust platform sense provide their own virtual file system; KDE's kio and Gnome's gvfs are prime examples. They offer transparent integration of network file systems as well as things like treating zip files as folders.

  These could provide their own implementations of the cap_std traits, and again allow the direct use of generic libraries with them.

(I initially thought that given this is written in a WASM ecosystem this'd also be useful to give, say, an in-browser WASM program access to a file system through WebDAV, but that can more easily be done by just configuring cap-std to do that internally, given it already has platform specific code for different OSes anyway).

Trouble

Apart from being quite some work to do, two aspects are particularly tricky:

• Versioning. Unlike structs, traits can only compatibly grow provided methods.

  Possible solutions are "know the API well in advance" (ie. develop this on structs to he point where it's unlikely that much needs to be added), "just bump versions" (possibly with compatibility mechanisms that implement the old trait on the new implementations) and "seal the trait, provide a generic impl based on a for-implementers crate, bump for-implementers crate version more frequently".

• GATs. Each impl Dir for X needs its own DirEntry type, own File type et cetera. That may need some GATs if any of the types is not owned (but I didn't check all the API yet to see whether that applies here).

Prior Art

The embedded-hal ecosystem does pretty much that to great success.

Question

Would it be in scope of cap-std to explore traits covering the capabilites?

The kf_path field of libc::kinfo_file holds the path.

briefly: the way I recommend applying ocap discipline in rust is:

• only the main bin crate gets access to ambient authority
• lib crates do not

so in crates such as the rand crate, separate the interfaces for accessing the thread RNG and such to their own crate, for use from the main bin crate.

see also dckc/rust#2
https://github.com/dckc/rust-sqlite3/blob/master/src/access/mod.rs

As @kubkon observed, NtOpenFile provides a more efficient and likely more reliable way to implement openat-like behavior on Windows. As an example, Rust is now using it this way in its remove_dir_all implementation.

Microsoft officially documents NtOpenFile as an Internal API which may change between releases or even service packs of Windows, however this particular function, NtOpenFile, has apparently been stable for a long time, and since Rust itself is now depending on it, it would seem to be sufficiently stable for cap-std to use as well.

This isn't a sandbox escape, because .. still fails when it goes below the starting directory, and the missing check only checks for access to things that the host process would need to have access to anyway in order to trigger them. It's just a case where POSIX says we should issue an error and we don't, and it triggers asserts in the fuzzer (typically with a "we already canonicalized this" expect failure).

To fix this, we should check faccessat when resolving . or .. paths. This depends on yanix supporting faccessat, which is added here: bytecodealliance/wasmtime#2046.

This looks like the perfect library for the Capsicum sandbox, I'm happy to see more capability stuff!
I've tried to run an example:

--- i/examples/std_fs_misc.rs
+++ w/examples/std_fs_misc.rs
@@ -33,6 +33,8 @@ fn touch(dir: &mut Dir, path: &Path) -> io::Result<()> {
 
 fn main() {
     let mut cwd = unsafe { Dir::open_ambient_dir(".") }.expect("!");
+    extern "C" { fn cap_enter(); }
+    unsafe { cap_enter(); }
 
     println!("`mkdir a`");

And it's mostly great, but accessing the symlink fails:

`cat a/c/b.txt`
write(1,"`cat a/c/b.txt`\n",16)                  = 16 (0x10)
openat(3,"a",O_RDONLY|O_NOFOLLOW|O_DIRECTORY|O_CLOEXEC,00) = 4 (0x4)
openat(4,"c",O_RDONLY|O_NOFOLLOW|O_DIRECTORY|O_CLOEXEC,00) = 5 (0x5)
openat(5,"b.txt",O_RDONLY|O_NOFOLLOW|O_CLOEXEC,00) ERR#31 'Too many links'
readlinkat(5,"b.txt","../b.txt",256)             = 8 (0x8)
faccessat(5,"..",X_OK,AT_EACCESS)                ERR#93 'Capabilities insufficient'
close(4)                                         = 0 (0x0)
close(5)                                         = 0 (0x0)
! Os { code: 93, kind: Other, message: "Capabilities insufficient" }

It would be really nice if the regular posix implementation could resolve the symlink between the opened components by itself, i.e. instead of trying to access .. at the a/c directory's descriptor, it would walk back its own "chain" of descriptors to a. Then it would actually be perfect :)

According to this blog post, macos 11 has a new O_NOFOLLOW_ANY flag that causes open to fail if any path component is a symlink. Assuming symlinks are relatively rare, cap-std could use this as a fast path -- check that the path contains no explicit .. and then open with O_NOFOLLOW_ANY; if that succeeds, then we're done. If either of those fails, open with the slow path.

(Hi, I've come here from @sunfishcode's cap-std announcement blog post. Please redirect me to a better place if this is not the best forum for a discussion around this).

In my experience when auditing source code or trying to maintain certain security guarantees, it has always been useful to be able to distinguish between an arbitrary string and a hardcoded string ('static str). The hardcoded string can be scrutinized more easily and ideally just once during a code-review, whereas a dynamic string is potentially dangerous.

I understand that cap-std is mostly about runtime guarantees for sandboxing and this notion might be orthogonal to the guarantees you'd like to uphold with AmbientAuthority, (e.g., denote all interactions with the outside world).

But would it be relevant to this project to introduce the notion of dynamic versus hardcoded strings?

I guess one could build some sort of static analysis (a clippy check?) that can restrict functions to only accept a static lifetime or declare additional shim functions that require their parameters to be 'static str, which internally just call into the existing functions.

Maybe I'm missing something, but I can't find a way to convert a sync Dir obtained with cap_directories to an async Dir compatible with cap_async_std.
The only solution at the moment seems to use the conversion Dir -> std::fs::File + ambient_authority-> async Dir.
It would be nicer to convert a Dir directly, without using ambient_authority.

In latest released version of cap-async-std (0.24.4 at the time of writing)

canonicalize method is defined on path::Path: https://docs.rs/cap-async-std/latest/cap_async_std/path/struct.Path.html#method.canonicalize

The documentation states that it is an alias for fs::canonicalize, however the link in the documentation is broken, since fs::canonicalize does not exist and (righfully so) instead is defined on fs::Dir https://docs.rs/cap-async-std/latest/cap_async_std/fs/struct.Dir.html#method.canonicalize

On further investigation, I realized that path module is re-exported from async-std

I understand that the suggested approach is to configure a custom linter with a list of disallowed methods to prevent usage of these. However, the documentation is confusing, references are broken and it's very easy to make a mistake for new users.

If it's not desired to maintain a custom Path type implementation, is there a reason cap-async-std crate has to re-export the path module at all?

Rust removed its workarounds for missing CLOEXEC support in old Linux versions in rust-lang/rust#74606, so we can remove the corresponding workarounds in cap-std as well. See ensure_cloexec and related functions.

The ambient_authority function is in scope for the whole file:

Is there some way to leave it out of scope of everything but the main function? I should re-read how rust modules work.

Maybe main() would only call open_ambient_dir and then pass cwd to a function that does the rest?

cap-std and cap-async-std will need a semver bump in order to pick up the fix for #270, which is needed in order to compile with changes to nightly Rust.

I'm also considering making the same change as #271 for FileExt, MetadataExt, DirBuilderExt, PermissionExt, OpenOptionsExt, to future-proof against std sealing those traits in the future. [edit: see comments below]

To minimize the number of semver bumps, I'm also planning to coordinate these changes with another change, when I/O lifetimes makes it to stable Rust (expected Aug. 11), and I update io-lifetimes to use it when available.

That's the main change contemplated in #192, so I'm tentatively planning to have this next release be version 1.0.

I was trying to make a crate for getting cap_std::fs::Dirs from wasi's preopened dirs, but hit a bit of a roadblock
It seems even when compiling to wasi, cap-primitives still tries to access std::os::unix

error[E0433]: failed to resolve: could not find `unix` in `os`
  --> cap-primitives\src\rsix\fs\copy_impl.rs:34:18
   |
34 |     use std::os::unix::fs::PermissionsExt;
   |                  ^^^^ could not find `unix` in `os`

+16 other errors

(on nightly)

I'm planning to publish a 0.25 release soon, after

• #256
• #252
• #258
• #259
• #260
• #261

Hello 👋,

I'm trying to implement some logic that extracts a zip file using cap-primitives and ran into a snag with how fs::set_permissions is currently implemented. In the general UNIX implementation it says even AT_NOFOLLOW_SYMLINK with fchmodat is not enough because it would modify the symlink itself, and that it is undesirable behavior. So, because of that, its implemented as a regular fchmod. Its not clear to me why this is undesirable at a glance though.

In my case however, I am actually trying to change the symlink itself based on permission bits that come from the zip file and the current behavior makes that impossible as it always dereferences the symlink and changes the permissions of the linked item instead. This is an odd use case, but I have the constraint of the process umask set at startup being more restrictive then what the zipped file permissions are, so I need to change everything written out to disk after writing to get the correct resulting permissions.

Is this a feature that you would consider adding to cap-primitives, or is "weird" symlink handling something that's considered out-of-scope?

cap-std's coverage of std's fs, net, and time APIs is roughly as complete as it can be, and has been fairly stable for some time now. The one upcoming breaking change I'm aware of is that once the I/O safety features in std are stabilized, cap-std should use those and remove its OwnsRaw trait impls. That may be a good occasion to make a 1.0 semver release.

I could imagine potential future changes, such as changing the way the Pool type in cap_std::net works, but there's always 2.0 and beyond.

If there's anything that should go into a 1.0 release, please post about it!

As reported here, cap-std's PermissionExt::mode returns a mode value with the format bits masked out, while std's PermissionsExt::mode returns the raw mode value. It's cap-std'd intention to match std, so it should remove the masking.