go-git / go-billy Goto Github PK

I'm unable to compile my code with go-billy dependencies.

> env GOOS=js GOARCH=wasm go build -v -o main.wasm cmd/addbranch/main/main.go 
/home/capsci/go/pkg/mod/github.com/go-git/go-billy/[email protected]/osfs/os_posix.go:8:2: build constraints exclude all Go files in /home/capsci/go/pkg/mod/golang.org/x/[email protected]/unix

Chroot does not process path target ​​from symlink? It is normal behavior?

Follow concept code

baseDir := "/foo/bar"
fs := chroot.New(&billyFilesystemSomeImpl{}, baseDir)
// lrwxr-xr-x /foo/bar/lint-to-tmp -> /tmp
fs.Open("/foo/bar/link-to-tmp")
// opened without errors

github.com/u-root/cpu depends on go-billy, and it would be nice to have a new release with the BoundsOS fix in.

This works:

package main

import (
	"fmt"

	"github.com/go-git/go-billy/v5/memfs"
)

func main() {
	mfs := memfs.New()
	if err := mfs.MkdirAll("test", 0755); err != nil {
		panic(err)
	}

	st, err := mfs.Stat(".")
	if err != nil {
		panic(err)
	}
	fmt.Printf("%#v\n", st)
}

If you comment out the MkdirAll line, it returns error not found.

This means that the root directory is created in the MkdirAll command.

The fresh memfs.New() should have a root / directory just after new() is called, right?

BoundOS has two function to check whether or not a file is inside the base directory. Their implementation rely on appending filepath.Separator to fs.baseDir and using strings.HasPrefix with filename. This causes a problem when fs.baseDir is the root directory. That's because strings.HasPrefix("/" + filepath.Separator, filename) will never return true with a clean filename.

Just wondering if there are any thoughts about adding an abstraction for os.Chown (which for windows does work pretty badly as you need string SSIDs and the semantics differ from Linux in the regard that you don't change user (uid) and group (gid) at the same time like os.Chown does.

Seemingly ACLs are common on those platforms or changing file owner and file owner group independently of each other might be an approach.

Also getting the current owner and owner group or a list of all ACLs of a file might be another interesting feature which would be necessary for like copying files and applying the same ownership permissions as the source file had.

How does this library relate to the io/fs draft which has been accepted recently and will be a part of go 1.16, which is scheduled to be released in February? How will this affect go-git and will go-git migrate to io/fs?

I haven't dug too deep/tested this yet, but it seems to me that in a lot of instances methods are referencing the stdlib os rather than fs for operations, i.e. https://github.com/go-git/go-billy/blob/master/osfs/os.go#L45 there is no reference to the chroot

Please feel free to ignore this if I'm being a wally.

We're using memfs alongside go-git to perform a shallow clone of Git repositories. We'd like to be able to limit the data written to the memfs (other than heap memory / crashing).

Would additions to the memfs interface to support limiting the filesystem contents be of interest / accepted?

I'm trying to use go-git for a tool but I'm getting stuck with the lack of interoperability of go-billy. Particularly, I need to cast billy.File to another custom File interface we have that implements Readdir but billy.File does not. How reasonable is to get this added to go-billy?

Thanks!

As a bit of a test as one of my projects uses spf13/afero for all other disk operations I implemented the go-billy interface as a wrapper for afero for use in go-git.

https://github.com/Maldris/go-billy-afero

This was done as a quick and dirty implementation to test the concept, but as part of it, I have gotten optional support for Symlink and ReadLink merged into afero.

To revisit the idea and look at potentially cleaning it up for actual use, I would appreciate some feedback to see what needs to be done, and if this could be imported as a new filesystem option.

My main concern for support is the file Lock and Unlock methods, which do not have equivalents in afero. To control the behaviour within memory I've provided a mutex, however if multiple instances are created in different processes, or multiple applications try to access the same file, this is an inadequate solution, requiring additional application level logic.

Hello folks,

I know it sounds like a rookie question and that surely is the case ...

not being such a golang guru, and trying to use https://github.com/go-git, I happened to come along the billy filesystem abstraction.

Well, I like abstraction and billy looks very fine so far, so I tried to get some instances but not achieving somehow. Then tried to get some example code and the only thing I find is the projects' homepage; but to no avail, because there is no source code where I could find out how to get an instance of type billy.Filesystem, only what to do with it if one has gotten one from somewhere. What I tried to code was something like:

myFileSystem := Filesystem (pathToFileSystem) and
myFileSystem := billy.Filesystem (pathToFileSystem)

and following the projects' statement on the homepage, stating that

each Filesystem implementation gives you a New method

i tried

myFileSystem := Filesystem.New (pathToFileSystem)
and myFileSystem := Basic.New (pathToFileSystem)
and myFileSystem := Dir.New (pathToFileSystem)

and even, prefixing the above with billy, like so:

myFileSystem := billy.Filesystem.New (pathToFileSystem)

I also tried a little bit around with ioutil ... but whatever I do, I dont find a constructor that takes a string path and gives back an instance of billy.Filesystem.

As mentioned, yes it might be a golang rookies' question and I might be as dumb as hell, but... please relieve me,...

Best Regards,

Michael

Edit: wayyy too much typos

Version

v5.4.1

Description

memfs.New().ReadDir("asdf") returns nil, nil instead of nil, <directory-not-found-err>.

Minimal example

func TestMemoryReadDir(t *testing.T) {
	if _, err := osfs.New("/tmp").ReadDir("asdf"); err == nil {
		t.Fatal("osfs: expected error; found <nil>")
	} else {
		t.Logf("osfs: error was correctly returned!")
	}

	if _, err := memfs.New().ReadDir("asdf"); err == nil {
		t.Fatal("memfs: expected error; found <nil>")
	} else {
		t.Logf("memfs: error was correctly returned!")
	}
}

// Output
//   osfs: error was correctly returned!
//   memfs: expected error; found <nil>

Proposed changes

func (fs *Memory) ReadDir(path string) ([]os.FileInfo, error) {
	if f, has := fs.s.Get(path); has {
		if target, isLink := fs.resolveLink(path, f); isLink {
			return fs.ReadDir(target)
		}
+	} else {
+		return nil, &fs.PathError{Op: "open", Path: path, Err: unix.ENOENT}
}

	var entries []os.FileInfo
	for _, f := range fs.s.Children(path) {
		fi, _ := f.Stat()
		entries = append(entries, fi)
	}

	sort.Sort(ByName(entries))

	return entries, nil
}

See #23

That PR also updates to Go 1.18.

If you call New() on the storage objects from multiple goroutines at the same time, go runtime may error with fatal error: concurrent map writes. This happens because the files map inside the storage struct has no concurrency protection around multiple access.

Other concurrency issues probably exist for accessing the children map inside the storage object.

Would it be appropriate to add support a billy.Filesystem method equivalent to os.Link? I gave it a try and it doesn't seem too hard to implement. The os.Link method is useful for implementing certain types of safe file moves, for example.

Hi,

Could you issue a new release to include the latest commits (walk code) that didn't make it in the last one? Thanks.

Eric

Would it be possible to have a minor version released which contains util.Walk? We are currently pinning to v5.0.0-20210804024030-7ab80d7c013d but it would be great if we could just have v5.3.2 instead.

If there is any outstanding work left for an official version to be cut, please let us know and we can try to contribute towards it.

import "github.com/go-git/go-billy/v5" // with go modules enabled (GO111MODULE=on or outside GOPATH)
import "github.com/go-git/go-billy" // with go modules disabled

The second line does not work: the code expects import "github.com/go-git/go-billy/v5".

This project includes in source an Apache-2.0 license text, with appendix changed to list the actual copyright holders of this project.

Please don't do that.

Instead, please do as instructed right above that copyright line:

To apply the Apache License to your work, attach the following boilerplate notice, with the fields enclosed by brackets "{}" replaced with your own identifying information. (Don't include the brackets!) The text should be enclosed in the appropriate comment syntax for the file format. We also recommend that a file or class name and description of purpose be included on the same "printed page" as the copyright notice for easier identification within third-party archives.

I.e. copy the boilerplate text and paste it at the top of each code source file, commented as appropriate for the coding language.

There is no legal need to state copyright since 1989.
The reason we do it anyway is as a curtesy for our users: We want to promote the code as Free Software, which requires it to be licensed as such, which can be done only by copyright holders - we therefore want our users to most easily locate not only licensing statement but also who made those statements for which code pieces.

When you list copyright notices in a project-wide file instead of inside each code file, you make it more difficult for your users to reuse code and still keep track of owers and licensing of the mixed parts.

When you edit general public licenses (concretely, the Apache-2.0 license text) you make it more difficult for users to rely on such files being general.

By the interface definition of ReadDir

// ReadDir reads the directory named by dirname and returns a list of
// directory entries sorted by filename.
ReadDir(path string) ([]os.FileInfo, error)

The current implementation returns the files in random order because it is retrieved from a map in a for loop. This can be fixed by sorting the result. I will submit a PR.

Thank you for this awesome project!

See #23

That PR also updates to Go 1.18.