psankar / simplefs Goto Github PK
View Code? Open in Web Editor NEWA simple, kernel-space, on-disk filesystem from the scratch
License: Other
A simple, kernel-space, on-disk filesystem from the scratch
License: Other
A simple filesystem to understand things. This is a Work In Progress. Do not use this yet. If you are planning to learn filesystems, start from the scratch. You can look from the first commit in this repository and move the way up. The source files are licensed under Creative Commons Zero License. More information at: http://creativecommons.org/publicdomain/zero/1.0/ Full license text at: http://creativecommons.org/publicdomain/zero/1.0/legalcode simplefs 1.0 Architecture + Notes --------------------------------- Block Zero = Super block Block One = Inode store Block Two = Occupied by the initial file that is created as part of the mkfs. Only a limited number of filesystem objects are supported. Files and Directories can be created. Support for .create and .mkdir is implemented. Nested directories can be created. A file cannot grow beyond one block. ENOSPC will be returned as an error on attempting to do. Directories store the children inode number and name in their data blocks. Read support is implemented. Basic write support is implemented. Writes may not succeed if done in an offset. Works when you overwrite the entire block. Locks are not well thought-out. The current locking scheme works but needs more analysis + code reviews. Memory leaks may (will ?) exist. Credits -------- The initial source code is written by psankar and patches are contributed by other github members. azat being the first contributor. An O_LARGETHANKS to the guidance of VijaiBabu M and Santosh Venugopal. Their excellent talks on filesystems motivated me to implement a file system from the scratch. Without their inspirational speeches, I would not have focussed on filesystems. A big thanks should go to the kernelnewbies mailing list for helping me with clarifying the nitty-gritties of the linux kernel, especially people like Mulyadi Santosa, Valdis Kletnieks, Manish Katiyar, Rajat Sharma etc. Special thanks to Ankit Jain who provides interesting conversations to help my intellectual curiosities. Patch Submission ----------------- Please send a merge request only if you are ready to publish your changes in the same creative commons zero license. We would like to keep the filesystem simple and minimal so that it can be used as a good teaching material. We are not planning to use this filesystem in a production machine. So some design choices are driven by learning/teaching simplicity over performance/scalability. This is intentional. So do not try to fix those things :) TODO ----- - After the 1.0 release, start with support for extents, which on completion will be 2.0 - After the 2.0 release, start with journalling support, which on completion will be 3.0 To compile: ------------ install linux kernel sources and run make from the checkedout directory. To test: --------- And here is the build environment for simplefs: https://github.com/azat/simplefs-bld psankar@linux-9dni:~/src/simplefs> make psankar@linux-9dni:~/src/simplefs> dd bs=4096 count=100 if=/dev/zero of=image 100+0 records in 100+0 records out 409600 bytes (410 kB) copied, 0.00106827 s, 383 MB/s psankar@linux-9dni:~/src/simplefs> ./mkfs-simplefs image Super block written succesfully root directory inode written succesfully welcomefile inode written succesfully inode store padding bytes (after the two inodes) written sucessfully root directory datablocks (name+inode_no pair for welcomefile) written succesfully padding after the rootdirectory children written succesfully welcomefilebody has been written succesfully psankar@linux-9dni:~/src/simplefs> Now as root: linux-9dni:/home/psankar/src/simplefs # insmod simplefs.ko ; mount -o loop -t simplefs image /home/psankar/src/simplefs/mount/ ; dmesg [41932.281196] Sucessfully registered simplefs [41932.284811] The magic number obtained in disk is: [268640275] [41932.284813] simplefs filesystem of version [1] formatted with a block size of [4096] detected in the device. [41932.284823] simplefs is succesfully mounted on [/dev/loop16] linux-9dni:/home/psankar/src/simplefs # cd mount/ linux-9dni:/home/psankar/src/simplefs/mount # ls vanakkam linux-9dni:/home/psankar/src/simplefs/mount # cat vanakkam Love is God. God is Love. Anbe Murugan. linux-9dni:/home/psankar/src/simplefs/mount # cp vanakkam hello linux-9dni:/home/psankar/src/simplefs/mount # cat hello Love is God. God is Love. Anbe Murugan. linux-9dni:/home/psankar/src/simplefs/mount # echo "Hello World" > hello linux-9dni:/home/psankar/src/simplefs/mount # cat hello Hello World linux-9dni:/home/psankar/src/simplefs/mount # mkdir dir1 linux-9dni:/home/psankar/src/simplefs/mount # cd dir1 linux-9dni:/home/psankar/src/simplefs/mount/dir1 # cp ../hello . linux-9dni:/home/psankar/src/simplefs/mount/dir1 # cat hello Hello World linux-9dni:/home/psankar/src/simplefs/mount/dir1 # echo "First level directory" > hello linux-9dni:/home/psankar/src/simplefs/mount/dir1 # cat hello First level directory linux-9dni:/home/psankar/src/simplefs/mount/dir1 # mkdir dir2 linux-9dni:/home/psankar/src/simplefs/mount/dir1 # cd dir2 linux-9dni:/home/psankar/src/simplefs/mount/dir1/dir2 # touch hello linux-9dni:/home/psankar/src/simplefs/mount/dir1/dir2 # cat hello linux-9dni:/home/psankar/src/simplefs/mount/dir1/dir2 # echo "Second level directory" > hello linux-9dni:/home/psankar/src/simplefs/mount/dir1/dir2 # cat hello Second level directory linux-9dni:/home/psankar/src/simplefs/mount/dir1/dir2 # ... linux-9dni:/home/psankar/src/simplefs/mount # .. linux-9dni:/home/psankar/src/simplefs # umount mount ; rmmod simplefs.ko ; linux-9dni:/home/psankar/src/simplefs # insmod simplefs.ko ; mount -o loop -t simplefs image /home/psankar/src/simplefs/mount/ linux-9dni:/home/psankar/src/simplefs # cd mount linux-9dni:/home/psankar/src/simplefs/mount # ls -lR .: total 0 drwxr-xr-x 1 root root 0 Aug 5 22:43 dir1 ---------- 1 root root 0 Aug 5 22:43 hello ---------- 1 root root 0 Aug 5 22:43 vanakkam ./dir1: total 0 drwxr-xr-x 1 root root 0 Aug 5 22:43 dir2 ---------- 1 root root 0 Aug 5 22:43 hello ./dir1/dir2: total 0 -rw-r--r-- 1 root root 0 Aug 5 22:43 hello linux-9dni:/home/psankar/src/simplefs/mount # cat hello Hello World linux-9dni:/home/psankar/src/simplefs/mount # cd dir1 linux-9dni:/home/psankar/src/simplefs/mount/dir1 # cat hello First level directory linux-9dni:/home/psankar/src/simplefs/mount/dir1 # cd dir2/ linux-9dni:/home/psankar/src/simplefs/mount/dir1/dir2 # cat hello Second level directory linux-9dni:/home/psankar/src/simplefs/mount/dir1/dir2 # ... linux-9dni:/home/psankar/src/simplefs/mount # .. linux-9dni:/home/psankar/src/simplefs # umount mount ; rmmod simplefs.ko ; linux-9dni:/home/psankar/src/simplefs # dmesg [41932.281196] Sucessfully registered simplefs [41932.284811] The magic number obtained in disk is: [268640275] [41932.284813] simplefs filesystem of version [1] formatted with a block size of [4096] detected in the device. [41932.284823] simplefs is succesfully mounted on [/dev/loop16] [41951.102689] No inode found for the filename [hello] [41951.102706] No inode found for the filename [hello] [41951.102724] No inode found for the filename [hello] [41951.102731] New file creation request [41951.102986] The new filesize that is written is: [41] and len was: [41] [41961.195423] The new filesize that is written is: [12] and len was: [12] [41968.055946] No inode found for the filename [dir1] [41968.055959] New directory creation request [41980.265026] No inode found for the filename [hello] [41980.265073] No inode found for the filename [hello] [41980.265082] New file creation request [41980.265168] The new filesize that is written is: [12] and len was: [12] [41991.857978] The new filesize that is written is: [22] and len was: [22] [41999.993579] No inode found for the filename [dir2] [41999.993592] New directory creation request [42005.461745] No inode found for the filename [hello] [42005.461756] New file creation request [42016.694882] The new filesize that is written is: [23] and len was: [23] [42024.480843] simplefs superblock is destroyed. Unmount succesful. [42024.485339] Sucessfully unregistered simplefs [42031.157983] Sucessfully registered simplefs [42031.162331] The magic number obtained in disk is: [268640275] [42031.162334] simplefs filesystem of version [1] formatted with a block size of [4096] detected in the device. [42031.162360] simplefs is succesfully mounted on [/dev/loop17] [42104.389807] simplefs superblock is destroyed. Unmount succesful. [42104.395575] Sucessfully unregistered simplefs linux-9dni:/home/psankar/src/simplefs #
The filesystem driver fails to build on 2.6.32 kernel. Kindly specify for which kernel you have written your driver
I am getting tainted kernel message on ubuntu 16. Does it mean that without signing kernel module I cannot load kernel module ?
I checked signing process docs but those are not easy to follow. Is there any workarounds to load kernel module without signing in ubuntu16?
mount command says "can't find image2 in /etc/fstab" , any idea what could have gone wrong. This is simplefs built on ubuntu 16
ubuntu:/code/sankar/simplefs-master$ dd bs=4096 count=100 if=/dev/zero of=image2/code/sankar/simplefs-master$ ./mkfs-simplefs image
100+0 records in
100+0 records out
409600 bytes (410 kB, 400 KiB) copied, 0.0290435 s, 14.1 MB/s
ubuntu:
Super block written succesfully
root directory inode written succesfully
journal inode written succesfully
welcomefile inode written succesfully
inode store padding bytes (after the three inodes) written sucessfully
Journal written successfully
root directory datablocks (name+inode_no pair for welcomefile) written succesfully
padding after the rootdirectory children written succesfully
block has been written succesfully
ubuntu:/code/sankar/simplefs-master$ sudo mount -o loop -t simplefs image2/code/sankar/simplefs-master$
mount: can't find image2 in /etc/fstab
ubuntu:
I tried to mount the file system after inserting the simplefs kernel module. But it is giving error while initializing journal block. The dmesg report is given below.
kernel version: 3.13.0-32-generic
[24820.367435] The magic number obtained in disk is: [268640275]
[24820.367440] simplefs filesystem of version [1] formatted with a block size of [4096] detected in the device.
[24820.367465] jbd2_journal_bmap: journal block not found at offset 0 on loop0-2
[24820.367466] jbd2_journal_init_inode: Cannot locate journal superblock
[24820.367469] Can't load journal
[24820.367471] simplefs superblock is destroyed. Unmount succesful.
[24820.367475] Freeing private data of inode ffff8801e940d060 (1)
[24820.367506] VFS: Busy inodes after unmount of loop0. Self-destruct in 5 seconds. Have a nice day...
[24820.367512] simplefs is succesfully mounted on [/dev/loop0]
[24820.367531] BUG: unable to handle kernel NULL pointer dereference at 0000000000000069
[24820.367577] IP: [] mount_fs+0x48/0x1b0
[24820.369308] PGD 161fda067 PUD 1e7c98067 PMD 0
[24820.370693] Oops: 0000 [#2] SMP
[24820.372138] Modules linked in: usb_storage simplefs(POF) bbswitch(OF) vmnet(OF) vmw_vsock_vmci_transport vsock vmw_vmci vmmon(OF) rfcomm bnep bluetooth snd_hda_codec_hdmi binfmt_misc nls_iso8859_1 hp_wmi sparse_keymap intel_rapl x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm crct10dif_pclmul crc32_pclmul ghash_clmulni_intel uvcvideo videobuf2_vmalloc videobuf2_memops videobuf2_core videodev aesni_intel arc4 aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd rt2800pci rt2800mmio rt2800lib rt2x00pci rt2x00mmio rt2x00lib mac80211 cfg80211 rtsx_pci_ms snd_hda_codec_realtek snd_seq_midi snd_seq_midi_event eeprom_93cx6 memstick nvidia(POF) crc_ccitt snd_rawmidi snd_seq i915 joydev snd_hda_intel serio_raw snd_hda_codec lpc_ich snd_hwdep snd_pcm shpchp snd_page_alloc snd_seq_device drm_kms_helper mei_me snd_timer mei drm snd i2c_algo_bit soundcore wmi video mac_hid parport_pc ppdev lp parport hid_generic usbhid hid rtsx_pci_sdmmc psmouse rtsx_pci ahci r8169 libahci mii
[24820.378482] CPU: 3 PID: 17469 Comm: mount Tainted: PF D O 3.13.0-32-generic #57-Ubuntu
[24820.379566] Hardware name: Hewlett-Packard HP 15 Notebook PC/2212, BIOS F.16 04/24/2014
[24820.380630] task: ffff8800472017f0 ti: ffff88021b89e000 task.ti: ffff88021b89e000
[24820.381669] RIP: 0010:[] [] mount_fs+0x48/0x1b0
[24820.382724] RSP: 0018:ffff88021b89fe40 EFLAGS: 00010203
[24820.383890] RAX: 0000000000000001 RBX: ffff88024f42b180 RCX: 0000000000000006
[24820.385158] RDX: 0000000000000007 RSI: 0000000000000046 RDI: 0000000000000246
[24820.386208] RBP: ffff88021b89fe78 R08: 0000000000000086 R09: 000000000000060c
[24820.387225] R10: 0000000000000000 R11: ffff88021b89fb4e R12: ffffffffa0304000
[24820.388227] R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000000
[24820.389233] FS: 00007ffad29d1880(0000) GS:ffff88025f2c0000(0000) knlGS:0000000000000000
[24820.390257] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[24820.391262] CR2: 0000000000000069 CR3: 00000002512af000 CR4: 00000000001407e0
[24820.392299] Stack:
[24820.393743] ffffffff8116f260 ffff88021b89fe78 ffff88024f42b180 ffffffffa0304000
[24820.394641] ffff880060e0f590 0000000000000000 0000000000000000 ffff88021b89feb0
[24820.395556] ffffffff811dbcd7 ffffffff81c3f880 0000000000000020 0000000000000000
[24820.396505] Call Trace:
[24820.397448] [] ? __alloc_percpu+0x10/0x20
[24820.398343] [] vfs_kern_mount+0x67/0x110
[24820.399228] [] do_mount+0x23e/0xad0
[24820.400103] [] ? strndup_user+0x4b/0xf0
[24820.401023] [] SyS_mount+0x83/0xc0
[24820.401885] [] tracesys+0xe1/0xe6
[24820.402739] Code: ec 10 48 85 c9 74 0a f6 47 08 02 0f 84 d2 00 00 00 44 89 fe 4c 89 e7 41 ff 54 24 10 48 3d 00 f0 ff ff 49 89 c6 0f 87 8e 00 00 00 <48> 8b 58 68 48 85 db 0f 84 f7 00 00 00 48 8b 83 d0 00 00 00 48
[24820.404661] RIP [] mount_fs+0x48/0x1b0
[24820.405564] RSP
[24820.406464] CR2: 0000000000000069
[24820.411360] ---[ end trace 468305eeb07c0f4d ]---
Here is list of build errors
jsridhara@ubuntu:~/code/sankar/simplefs-master$ make
make -C /lib/modules/4.4.0-65-generic/build M=/home/jsridhara/code/sankar/simplefs-master modules
make[1]: Entering directory '/usr/src/linux-headers-4.4.0-65-generic'
CC [M] /home/jsridhara/code/sankar/simplefs-master/simple.o
/home/jsridhara/code/sankar/simplefs-master/simple.c: In function ‘simplefs_iterate’:
/home/jsridhara/code/sankar/simplefs-master/simple.c:187:14: error: ‘struct file’ has no member named ‘f_dentry’
inode = filp->f_dentry->d_inode;
^
/home/jsridhara/code/sankar/simplefs-master/simple.c:203:14: error: ‘struct file’ has no member named ‘f_dentry’
filp->f_dentry->d_name.name);
^
/home/jsridhara/code/sankar/simplefs-master/simple.c: In function ‘simplefs_write’:
/home/jsridhara/code/sankar/simplefs-master/simple.c:378:32: warning: passing argument 1 of ‘generic_write_checks’ from incompatible pointer type [-Wincompatible-pointer-types]
retval = generic_write_checks(filp, ppos, &len, 0);
^
In file included from /home/jsridhara/code/sankar/simplefs-master/simple.c:12:0:
include/linux/fs.h:2688:16: note: expected ‘struct kiocb *’ but argument is of type ‘struct file *’
extern ssize_t generic_write_checks(struct kiocb *, struct iov_iter *);
^
/home/jsridhara/code/sankar/simplefs-master/simple.c:378:38: warning: passing argument 2 of ‘generic_write_checks’ from incompatible pointer type [-Wincompatible-pointer-types]
retval = generic_write_checks(filp, ppos, &len, 0);
^
In file included from /home/jsridhara/code/sankar/simplefs-master/simple.c:12:0:
include/linux/fs.h:2688:16: note: expected ‘struct iov_iter *’ but argument is of type ‘loff_t * {aka long long int *}’
extern ssize_t generic_write_checks(struct kiocb *, struct iov_iter *);
^
/home/jsridhara/code/sankar/simplefs-master/simple.c:378:11: error: too many arguments to function ‘generic_write_checks’
retval = generic_write_checks(filp, ppos, &len, 0);
^
In file included from /home/jsridhara/code/sankar/simplefs-master/simple.c:12:0:
include/linux/fs.h:2688:16: note: declared here
extern ssize_t generic_write_checks(struct kiocb *, struct iov_iter *);
^
scripts/Makefile.build:258: recipe for target '/home/jsridhara/code/sankar/simplefs-master/simple.o' failed
make[2]: *** [/home/jsridhara/code/sankar/simplefs-master/simple.o] Error 1
Makefile:1420: recipe for target 'module/home/jsridhara/code/sankar/simplefs-master' failed
make[1]: *** [module/home/jsridhara/code/sankar/simplefs-master] Error 2
make[1]: Leaving directory '/usr/src/linux-headers-4.4.0-65-generic'
Makefile:8: recipe for target 'ko' failed
make: *** [ko] Error 2
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.