External disk support about kerla HOT 16 OPEN

The Block I/O subsystem is a bit off-topic so the first goal would be writing ext2 reader/writer library.

from kerla.

Great! If I work on this topic I will:

1. Survey how existing OS kernels implement file systems, block I/O subsystem, and how they cache file/disk contents.
2. Write virtio-blk device driver.
3. Write block IO and caching subsystem.
4. Look for file system which is easy to implement: ext2 is a promising candidate. Newer feature-rich filesystems like btrfs and zfs would be too complicated for the first step.
5. Add ext2 (or something) output support in docker2initramfs.py. You might want to use virt-make-fs to do this task.
6. Implement filesystem reader/writer no_std library: it will depends on a simple Disk trait like:

trait Disk {
    fn read_block(&mut self, lba: usize, buf: &mut [u8]) -> Result<usize /* # of bytes actually read */, DiskError>;
    fn write_block(&mut self, lba: usize, buf: &[u8]) -> Result<usize /* # of bytes actually written */, DiskError>;
}

8. Integrate the library into Kerla.
9. Attach the disk to QEMU (-device virtio-blk-pci,...).

First, I recommend you to learn how file I/O works in major operating systems like Linux and FreeBSD. I think this book is a good start point to learn the internals.

from kerla.

You don't need to say that! Please feel free to ask me anything even if the question is trivial. It also helps me to write docs 👍

from kerla.

I see https://github.com/pi-pi3/ext2-rs

There is an updated fork https://github.com/rcore-os/ext2-rs which is in better shape than original

from kerla.

@pi-pi3 the original author of that crate is my countryman, but it seems he abandoned GitHub after Microsoft bought it. His profile on GitLab is private, so we cannot count on his contribution.

I wonder if @jiegec would like to contribute.

from kerla.

I am more animalistic about implementing a file system.
But I can only start with some simple tasks, Can we first determine the specific plan and then list the task list?

from kerla.

A great plan, thanks, I will first understand the implementation of the linux file system by reading this book.
Are we sure to implement the ex2 file system?

from kerla.

Are we sure to implement the ex2 file system?

Yes. I agree ext2 is the best for the first file system implementation because ext2 and its descendants ext3/ext4 are popular and well-known: there're good articles, sample implementations, and tools.

from kerla.

Can I suggest starting from simpler filesystem, potentially some FAT derivative might be easier for start.

from kerla.

Of course! Other file system implementations are also welcome. That said, I think FAT is a good for us because:

• It's not designed for UNIX-like operating systems: it lacks attributes such as permissions.
• Part of the spec is protected by Microsoft's patent.

Perhaps minixfs could be interesting.

from kerla.

That might be interesting, maybe we should have a VFS?

from kerla.

Kerla already has a sort of file system abstraction 😃 You can mount an arbitrary file system here:

from kerla.

cool,It seems that I did not understand kerla carefully,That's not great

from kerla.

That said, I think FAT is a good for us because:

• It's not designed for UNIX-like operating systems: it lacks attributes such as permissions.
• Part of the spec is protected by Microsoft's patent.

While it's not designed for UNIX, it's very simple to implement - and I suggest that as starting point and test thing. Keep in mind to use SD Cards you pretty often need FAT and/or exFAT support as this filesystem in part of the SD Card specification. Moreover, (FAT is already implemented in Rust](https://github.com/rafalh/rust-fatfs). Metadata can be mapped via hidden files.

Most FAT patents expired already, potentially there are some for exFAT.

from kerla.

Makes sense. We need to support FAT anyway in the future to support some boards like Raspberry Pi.

from kerla.

I see https://github.com/pi-pi3/ext2-rs which could be a starting point but not sure how usable it is for this purpose

from kerla.