Code Monkey home page Code Monkey logo

go-ext4's Introduction

Build Status Coverage Status GoDoc

Overview

This package allows you to browse an ext4 filesystem directly. It does not use FUSE or touch the kernel, so no privileges are required.

This package also exposes the data in the journal (if one is available).

Example

Recursively walk all of the files in the filesystem:

inodeNumber := InodeRootDirectory

filepath := path.Join(assetsPath, "hierarchy_32.ext4")

f, err := os.Open(filepath)
log.PanicIf(err)

defer f.Close()

_, err = f.Seek(Superblock0Offset, io.SeekStart)
log.PanicIf(err)

sb, err := NewSuperblockWithReader(f)
log.PanicIf(err)

bgdl, err := NewBlockGroupDescriptorListWithReadSeeker(f, sb)
log.PanicIf(err)

bgd, err := bgdl.GetWithAbsoluteInode(inodeNumber)
log.PanicIf(err)

dw, err := NewDirectoryWalk(f, bgd, inodeNumber)
log.PanicIf(err)

allEntries := make([]string, 0)

for {
	fullPath, de, err := dw.Next()
	if err == io.EOF {
		break
	} else if err != nil {
		log.Panic(err)
	}

	description := fmt.Sprintf("%s: %s", fullPath, de.String())
	allEntries = append(allEntries, description)
}

sort.Strings(allEntries)

for _, entryDescription := range allEntries {
	fmt.Println(entryDescription)
}

// Output:
//
// directory1/fortune1: DirectoryEntry<NAME=[fortune1] INODE=(15) TYPE=[regular]-(1)>
// directory1/fortune2: DirectoryEntry<NAME=[fortune2] INODE=(14) TYPE=[regular]-(1)>
// directory1/fortune5: DirectoryEntry<NAME=[fortune5] INODE=(20) TYPE=[regular]-(1)>
// directory1/fortune6: DirectoryEntry<NAME=[fortune6] INODE=(21) TYPE=[regular]-(1)>
// directory1/subdirectory1/fortune3: DirectoryEntry<NAME=[fortune3] INODE=(17) TYPE=[regular]-(1)>
// directory1/subdirectory1/fortune4: DirectoryEntry<NAME=[fortune4] INODE=(18) TYPE=[regular]-(1)>
// directory1/subdirectory1: DirectoryEntry<NAME=[subdirectory1] INODE=(16) TYPE=[directory]-(2)>
// directory1/subdirectory2/fortune7: DirectoryEntry<NAME=[fortune7] INODE=(22) TYPE=[regular]-(1)>
// directory1/subdirectory2/fortune8: DirectoryEntry<NAME=[fortune8] INODE=(23) TYPE=[regular]-(1)>
// directory1/subdirectory2: DirectoryEntry<NAME=[subdirectory2] INODE=(19) TYPE=[directory]-(2)>
// directory1: DirectoryEntry<NAME=[directory1] INODE=(13) TYPE=[directory]-(2)>
// directory2/fortune10: DirectoryEntry<NAME=[fortune10] INODE=(26) TYPE=[regular]-(1)>
// directory2/fortune9: DirectoryEntry<NAME=[fortune9] INODE=(25) TYPE=[regular]-(1)>
// directory2: DirectoryEntry<NAME=[directory2] INODE=(24) TYPE=[directory]-(2)>
// lost+found: DirectoryEntry<NAME=[lost+found] INODE=(11) TYPE=[directory]-(2)>
// thejungle.txt: DirectoryEntry<NAME=[thejungle.txt] INODE=(12) TYPE=[regular]-(1)>

This example and others are documented here.

Notes

  • Modern filesystems are supported, including both 32-bit and 64-bit addressing. Obscure filesystem options may not be compatible. See the compatibility assertions in NewSuperblockWithReader.
    • 64-bit addressing should be fine, as the high addressing should likely be zero when 64-bit addressing is turned-off (which is primarily what our unit-tests test with). However, the available documentation is limited on the subject. It's specifically not clear which of the various high/low addresses are affected by the 64-bit mode.

To Do

  • Finish implementing checksum calculation and validation. Currently all checksums are readable but with no additional functionality.

go-ext4's People

Contributors

ayushr2 avatar dsoprea avatar ml-bnr avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

go-ext4's Issues

Error instead of Panic

Currently go-ext4 panics on most of the errors paths. Think it would be worth returning an error (which the current code supports with return statements) so that client code can deal with the error as they like. Currently all errors returned from most functions are always nil.

This will allow flexibility in use. Client code can panic if they like or continue gracefully.

Add support for Symlinks

Currently symlinks are not supported because only inodes having extent trees are supported. According to Ext4 Disk Layout for symlinks, if the symlink contains a path shorter than 60 bytes then inode.i_block (Inode.IBlock in this lib) is used to store the path (which is usually the case). When that happens, the inode is not using extents and hence InodeFlagExtents is not set. Hence, this library refuses to create the symlink's inode. (Reference)

BGD Size Incorrect

The function NewBlockGroupDescriptorListWithReadSeeker seems to be assuming that the block group descriptor is 64 bytes in length. However, that is not always true according to docs.

You need to read only half of that struct if the 64-bit feature is not set and not use the rest of the fields. For the next entry you only increment offset by 32, not 64.
If the 64-bit feature is set, the BGD size could be larger than 64 bytes also sometimes. You need to increment the offset by that amount (>= 64).

Impact:
32-bit ext4 filesystems would not work with this library since block group descriptors are one of the most important metadata besides the superblock.

Binary Search Extent Indices

From performance gains for large files, I think it would be better to do a binary search on the array of ExtentIndexNode since it is sorted. This library currently does a linear search. Linux kernel does a binary search too (see ext4_ext_binsearch_idx in fs/ext4/extents.c).

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.