jaegeuk / f2fs-tools Goto Github PK

I created the filesystem with mkfs.f2fs -w 4096 DEV, but it always prints Info: sector size = 512, not Info: sector size = 4096.

I checked the latest source code on https://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs-tools.git/ and found 2 problems.

1. The value passed to -w is assigned to struct f2fs_configuration's member c.wanted_sector_size, but it is never used.

   It seems that c.sector_size is used to create a fs of certain sector size. So, if c.wanted_sector_size is set, c.sector_size must be also set. However, c.wanted_sector_size is not used, so c.sector_size is always DEFAULT_SECTOR_SIZE which is 512.

2. The position of add_default_options() and the parsing while loop in the function f2fs_parse_options() should be switched.

   The parsing while loop parse -w, but after that add_default_options() is called and it changes a few options including c.wanted_sector_size to default values. This renders -w option useless.

When cross-compiling with the --build flag, the test for .git fails. AC_CHECK_FILE can only be used to check for files on the native system, not the build system.

A few years ago, some email in the f2fs development mailing list mentioned a filesystem corruption, which would only be recoverable by rescanning all the inodes and reconstructing the checkpoint. I have had a similar corruption issue. My fsck output follows:

Info: Debug level = 3
Info: Segments per section = 1
Info: Sections per zone = 1
Info: sector size = 512
Info: total sectors = 536870912 (262144 MB)
Info: MKFS version
  "Linux version 5.4.0-72-generic (buildd@lcy01-amd64-019) (gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04)) #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021"
Info: FSCK version
  from "Linux version 5.4.0-74-generic (buildd@lgw01-amd64-038) (gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04)) #83-Ubuntu SMP Sat May 8 02:35:39 UTC 2021"
    to "Linux version 6.2.0-35-generic (buildd@bos03-amd64-016) (x86_64-linux-gnu-gcc-11 (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #35~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Fri Oct  6 10:23:26 UTC 2"
Info: superblock features = 0 : 
Info: superblock encrypt level = 0, salt = 00000000000000000000000000000000

+--------------------------------------------------------+
| Super block                                            |
+--------------------------------------------------------+
magic                         		[0xf2f52010 : 4076150800]
major_ver                     		[0x       1 : 1]
volum_name                    		[]
minor_ver                     		[0x       b : 11]
log_sectorsize                		[0x       9 : 9]
log_sectors_per_block         		[0x       3 : 3]
log_blocksize                 		[0x       c : 12]
log_blocks_per_seg            		[0x       9 : 9]
segs_per_sec                  		[0x       1 : 1]
secs_per_zone                 		[0x       1 : 1]
checksum_offset               		[0x       0 : 0]
block_count                   		[0x 4000000 : 67108864]
section_count                 		[0x   1fe85 : 130693]
segment_count                 		[0x   1ffff : 131071]
segment_count_ckpt            		[0x       2 : 2]
segment_count_sit             		[0x       a : 10]
segment_count_nat             		[0x      6e : 110]
segment_count_ssa             		[0x     100 : 256]
segment_count_main            		[0x   1fe85 : 130693]
segment0_blkaddr              		[0x     200 : 512]
cp_blkaddr                    		[0x     200 : 512]
sit_blkaddr                   		[0x     600 : 1536]
nat_blkaddr                   		[0x    1a00 : 6656]
ssa_blkaddr                   		[0x    f600 : 62976]
main_blkaddr                  		[0x   2f600 : 194048]
root_ino                      		[0x       3 : 3]
node_ino                      		[0x       1 : 1]
meta_ino                      		[0x       2 : 2]
cp_payload                    		[0x       0 : 0]
crc                           		[0x       0 : 0]
version                       Linux version 5.4.0-74-generic (buildd@lgw01-amd64-038) (gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04)) #83-Ubuntu SMP Sat May 8 02:35:39 UTC 2021
Info: total FS sectors = 536870912 (262144 MB)
	Invalid CP CRC offset: 0
	Invalid CP CRC offset: 0
[f2fs_do_mount:3512] Can't find valid checkpoint

The rescue tool should satisfy the following requirements:

1. In case the tool finds multiple inodes with overlapping blocks, the one updated the latest should take precedence.
2. Where possible, the tool should be written in a manner that prefers bogus files to unlinked files.
3. If checksums fail during the rescue, they should just be ignored, in accordance with requirement 2.
4. When there is enough RAM, the tool should construct the tables (superblock & checkpoints) at RAM, then write the result to the disk all at once. This is necessary to allow multiple FS rescue attempts going in parallel.

I have discovered (the hard way of course) that mkfs.f2fs -m does not reset zone pointers on a SMR disk. This leads to unaligned write errors once the file system hits one of those zones. I believe mkfs should reset the write pointers of any sequential zones when the -m option is used.

If you have a drive that has been use before (some zone WP's not zero), in this example /dev/sdb1 is a 16TiB partition of a large host-managed SMR drive:

1. blkzone report /dev/sdb1 | grep "(fu)" | wc -l # note number of full zones
2. mkfs.f2fs -m /dev/sdb1
3. blkzone report /dev/sdb1 | grep "(fu)" | wc -l # same number of full zones!

The worst part is that, depending on where the full zones are, the file system may work fine for some amount of time before you hit a previously-used zone. I have also noticed that fsck.f2fs doesn't fix this problem, despite patches that indicate it should. I've built and installed the latest f2fs-tools from source. I am running on Ubuntu 20.04 LTS, kernel 5.4.0-73-generic.

The work-around is to make sure you run blkzone reset /dev/sdb1 before creating the file system.

HI,

I got the build failed - log below, please help. autoconf and configure is fine all dependencies are met.

make[2]: Entering directory '/home/stevek/src/f2fs-tools/mkfs'
gcc -DHAVE_CONFIG_H -I. -I..  -I/usr/include/uuid -I/usr/include/blkid -I../include  -Wall -DWITH_BLKDISCARD -g -O2 -MT f2fs_format_main.o -MD -MP -MF .deps/f2fs_format_main.Tpo -c -o f2fs_format_main.o f2fs_format_main.c
mv -f .deps/f2fs_format_main.Tpo .deps/f2fs_format_main.Po
/bin/bash ../libtool  --tag=CC   --mode=link gcc -Wall -DWITH_BLKDISCARD -g -O2   -o mkfs.f2fs f2fs_format_main.o f2fs_format.o f2fs_format_utils.o -luuid -lblkid ../lib/libf2fs.la 
libtool: link: gcc -Wall -DWITH_BLKDISCARD -g -O2 -o .libs/mkfs.f2fs f2fs_format_main.o f2fs_format.o f2fs_format_utils.o  -luuid -lblkid ../lib/.libs/libf2fs.so
/usr/bin/ld: ../lib/.libs/libf2fs.so: undefined reference to `minor'
/usr/bin/ld: ../lib/.libs/libf2fs.so: undefined reference to `major'
collect2: error: ld returned 1 exit status
make[2]: *** [Makefile:487: mkfs.f2fs] Error 1
make[2]: Leaving directory '/home/stevek/src/f2fs-tools/mkfs'
make[1]: *** [Makefile:415: all-recursive] Error 1
make[1]: Leaving directory '/home/stevek/src/f2fs-tools'
make: *** [Makefile:347: all] Error 2

Many suggestion is to add #include <sys/sysmacros.h> into a .c file which I think does not make sense as it is error in the linking phase (maybe we forgot to add some lib?) - however I did try to add to ./mkfs/f2fs_format_main.c and rebuild without success as well.

Thanks

Introductory remarks:

• I'm a bit wondering why a f2fs partition that's shifted some GiB to the right is no longer mountable ("mount(2) system call failed: Structure needs cleaning"), despite fsck.f2fs giving no errors but the following remark: checkpoint state = 181 : trimmed nat_bits unmount
• My experience is not unique. Here is a link to someone's reported comparable problem: https://beroal.livejournal.com/80945.html

I post this issue because it seems that the behaviour is related to f2fs-tools.

After having tried testdisk tools (specifically: photorec - which recovered only a part of the files) I gave ddrescue a try.
No changes/error corrections to the partition are executed.
Then I found this sourceforge entry, where a patch to f2fs-tools 1.14 is successfully tested and which fixed "wrong sit/nat bitmap during rebuild_checkpoint()". Therefore I cloned this repository and built v1.15, where this patch is included. No file system changes result in applying this version or even g-dev-test branch.
The attached transcript contains a final execution of fsck.f2fs, which is based on f2fs-tools 1.14, but using 1.15 the result is the same.

ddrescue-transcript.txt

Using f2fs tools on Ubuntu:
I wanted to format half-a-dozen USB flash drives to be identical and have the F2FS filesystem. I'm 'lazy' and wanted to write a simple shell script to do the job. But first, sensibly, I needed to see what options mkfs.f2fs had and then use them in the script.

After a series of attempts I couldn't get the mkfs.f2fs to work. It simply said it couldn't find the device. So I tried mkfs.ext4 and it worked just fine.

Frustrated I gave up on the idea of writing the script and used the disk drive GUI instead. However, even then it seems that mkfs.f2fs would only work if the drive was already ext4 formatted -- not the plain FAT format that the Samsung USB flash drives normally come with.

I also note that when a drive is F2FS formatted the disk drive GUI says that the contents are: "unknown (f2fs 1.11)" -- quite ugly.

My sense is that the F2FS tools need a serious clean up.

I tried to figure out how to change the f2fs label and I could not come up with a way.

Hello, I tried to use a F2FS SD card on android 7 kernel 3.18 but it is not recognized

Why?
Do I need to install this? like I installed it on my computer to format the SD card

thanks

Hello,

I have the problem that a f2fs filesystem together with systemd (and an installed fsck.f2fs) will endlessly reboot. (systemd/systemd#15106)

The fsck.f2fs tool behaves different to the ext variants, and does not seem to support the -a switch,
and fails on a mounted device (by default). ie. it would not help calling fsck from the initramfs either.
fsck tools seem wildy different, but systemd assumes some basic uniformity

systemd version the issue has been seen with

systemd 245 (245)
-PAM -AUDIT -SELINUX -IMA -APPARMOR -SMACK -SYSVINIT -UTMP -LIBCRYPTSETUP -GCRYPT -GNUTLS -ACL -XZ +LZ4 -SECCOMP +BLKID -ELFUTILS +KMOD -IDN2 -IDN -PCRE2 default-hierarchy=unified

fsck.f2fs 1.13.0 (2019-09-24)

the root filesystem is f2fs, mounted read-only from the initrd.
Systemd will endlessly reboot the system as the fsck attempt fails.

The systemd-fsck tool will end up calling fsck.f2fs with the parameter below,
and will fail as this device is mounted (tries to open it in exclusive mode).

$ fsck.f2fs -a /dev/mmcblk0p5; echo $?  
Info: Fix the reported corruption.
Info: Mounted device!
Info: Check FS only on RO mounted device
	Error: Failed to open the device!
255

A workaround would be to force or skip the check

fsck.f2fs -a -f /dev/mmcblk0p5; echo $?   
Info: Fix the reported corruption.
.....
Done: 0.232165 secs
0

The repository seems to have silently moved to https://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs-tools.git/

It is therefore useless to report issues here.

Test condition:
Allwinner A133 Android 10 tablet.

• Data partition is formated to f2fs.
• fsck enabled during android boot
• 32GB emmc flash, and data partition size is 28GB+

Steps to reproduce:

1. create 20GB file on data partition:
   belsito:/data # dd if=/dev/urandom of=/data/test.bin bs=1k count=20m 20971520+0 records in 20971520+0 records out 21474836480 bytes (20 G) copied, 679.607325 s, 30 M/s
2. create 500MB file on data partition:
   dd if=/dev/urandom of=/data/test2.bin bs=1k count=500k
   don't wait this command finish, open another adb shell, type command:
   reboot # this makes device reboot
3. after reboot fsck.f2fs found data corruption, and took very long time >90s to finish check:

Hello!

Getting a SIGSEGV on attempt to format f2fs filesystem on a sparc64 machine:

f2fs-tools$ git describe
v1.14.0-45-g820b5e3
f2fs-tools$ git branch
* dev
  master

f2fs-tools# export LD_LIBRARY_PATH=./lib/.libs
f2fs-tools# gdb -q ./mkfs/.libs/mkfs.f2fs
Reading symbols from ./mkfs/.libs/mkfs.f2fs...
(gdb) set args -O extra_attr,inode_checksum,quota,verity,compression /dev/loop0
(gdb) run
Starting program: /1/mator/f2fs-tools/mkfs/.libs/mkfs.f2fs -O extra_attr,inode_checksum,quota,verity,compression /dev/loop0

        F2FS-tools: mkfs.f2fs Ver: 1.14.0 (2021-04-22)

Info: Disable heap-based policy
Info: Debug level = 0
Info: Trim is enabled
Info: Enable Compression
Info: Segments per section = 1
Info: Sections per zone = 1
Info: sector size = 512
Info: total sectors = 16777216 (8192 MB)
Info: zone aligned segment0 blkaddr: 512
Info: add quota type = 0 => 4
Info: add quota type = 1 => 5
Info: format version with
  "Linux version 5.13.0-rc3-00043-gad9f25d33860 (mator@ttip) (gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2) #220 SMP Wed May 26 16:38:54 MSK 2021"
Info: [/dev/loop0] Discarding device
Info: This device doesn't support BLKSECDISCARD
Info: Discarded 8192 MB

Program received signal SIGSEGV, Segmentation fault.
0x000001000001485c in f2fs_update_nat_root () at f2fs_format.c:1462
1462                    nat_blk->entries[sb->qf_ino[qtype]].block_addr =
(gdb) bt
#0  0x000001000001485c in f2fs_update_nat_root () at f2fs_format.c:1462
#1  0x0000010000017e24 in f2fs_create_root_dir () at f2fs_format.c:1732
#2  0x00000100000181ec in f2fs_format_device () at f2fs_format.c:1780
#3  0x00000100000049fc in main (argc=4, argv=0x7fefffff598) at f2fs_format_main.c:422

Can you please have a look?! Thanks.

There is not enough explanation for what each option does in mkfs.f2fs.

Here are the supported features for one's current f2fs-tools version, in my case g-dev-test branch. This is not in the manpages for mkfs.f2fs.
ls /sys/fs/f2fs/features:

atomic_write block_zoned encryption extra_attr flexible_inline_xattr inode_checksum inode_crtime lost_found project_quota quota_ino sb_checksum
So...
cat /sys/fs/f2fs/features/*:

supported
supported
supported
supported
supported
supported
supported
supported
supported
supported
supported

Great!

But now I have to read f2fs-tools/mkfs/f2fs_format_main.c to know proper syntax.

The full list with correct wording: encrypt extra_attr flexible_inline_xattr inode_checksum inode_crtime project_quota compression

Notes: atomic_write and quota_ino are supported but not available as an option; compression is only available in this most recent commit. block_zoned is handled by the -m flag.

Proposed resolution: please help users understand the filesystem parameters better.

Using the latest package in Archlinux it doesnt recognize compression or compress option with -O

Observed behavior

Running fsck.f2fs -p* [device], where * is any sequence of non-spaces with at least a non-digit character, freezes fsck.f2fs. It enters an infinite loop, printing Info: Use default preen mode continuously. CPU usage is increased, but there's no disk activity and used up RAM is constant.

This could be a problem if the user wants to pass multiple flags (e.g. -pf, which should be valid) or they misinterpret the syntax. Other combined options like -ap, -fp or -fS work perfectly. Contrast this with -pf.

Note: this only happens with -p, not with -d.

Tests performed

1.13.0

On this version I've verified that -pf creates an infinite loop. I haven't tested other combinations.

1.11.0

On this version I've tested all of -pf, -pa, -py, -pp, -p3a, p*, -p-, -p$, -p%, -p., -p123 and -pbhydfs. All of them raise an infinite loop, printing the same message over and over.

Intended behavior

-p<option> should work like -<option>p, and -p<garbage> like -<option><garbage>.

Summary

the ext4 partition, i google search found this https://www.reddit.com/r/linux_gaming/comments/cbrc7s/52_kernel_supports_caseinsensitive_directories/
mkfs -t ext4 -E encoding=utf8

but, how to format f2fs partition to enable casefold ?

For example, from mkfs.f2fs

.BI \-O " feature-list"
Specify a feature list in order f2fs filesystem will supports.
e.g "encrypt" and so on.

The source code for the mkfs tool shows that there are several supported options, some of which can be added after FS creation by the fsck tool.

It would be great if the man pages for mkfs showed:

• all the optional features available
• explanations for F2FS only features

and the man pages for fsck showed:

• all the features that can be enabled after the FS has been created (such as encryption)

I don't know how the development works (if it's here or via mailing list), but if you can give me some tips I can try to update the man pages.

Is there a way to enable casefolding option on an existing partition, without having to reformat?