bmatcuk / doublestar Goto Github PK

From @cequintana:

I keep getting this error when I attempt to build fossa-cli. Thoughts?

Solving failure: (1) failed to clean up git repository at C:\Users\cquintana\go\pkg\dep\s ources\https---github.com-bmatcuk-doublestar - dirty? corrupted? status output: M test/broken-symlink (2) failed to list versions for ssh://[email protected]/bmatcuk/doublestar: [email protected]: Permission denied (publickey). fatal: Could not read from remote repository. Please make sure you have the correct access rights and the repository exists. : exit status 128 (3) failed to clean up git repository at C:\Users\cquintana\go\pkg\dep\s ources\git---github.com-bmatcuk-doublestar - dirty? corrupted? status output: M test/broken-symlink (4) failed to clean up git repository at C:\Users\cquintana\go\pkg\dep\s ources\http---github.com-bmatcuk-doublestar - dirty? corrupted? status output: M test/broken-symlink

Even if not released in this versions it is a good practice to test in multiple go versions.

Hey folks,

Is there any way to ignore subdirectories based on the directory name? We currently have a lerna monorepo and I find it difficult to exclude multiple node_modules folders, while using addlicense, which mentions that is using doublestar for the file matching.

The optimal would be to use it like this

addlicense -c "Neo4j Inc." -f license.txt -check ./**/*.{js,ts,css} -ignore **/node_modules/*

but it doesn't seem to work.

The directories also are in the following tree structure

.
├── CONTRIBUTING.md
├── README.md
├── lerna.json
├── license.js
├── license.txt
├── node_modules
├── package.json
├── packages
│   ├── base
│   │   ├── node_modules
│   │   ├── package.json
│   │   ├── src
│   ├── html-storybook
│   │   ├── node_modules
│   │   ├── package.json
│   ├── react
│   │   ├── node_modules
│   │   ├── package.json
│   │   ├── src
│   │   └── tsconfig.json
│   └── react-storybook
│       ├── node_modules
│       ├── package.json
│       └── src
├── tsconfig.json
└── yarn.lock

Any clue if I am missing something?

I would have expected

abc**

to match

abc

but it does not. If ** matches one or more occurrences of any character then this is incompatible with the fact that **abc matches abcc just fine.

When attempting to build v1 in Fedora Rawhide, which uses go 1.20:

+ GO_TEST_EXT_LD_FLAGS='-Wl,-z,relro -Wl,--as-needed  -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -Wl,--build-id=sha1 -specs=/usr/lib/rpm/redhat/redhat-package-notes  '
+ go-rpm-integration check -i github.com/bmatcuk/doublestar -b /builddir/build/BUILD/doublestar-1.3.4/_build/bin -s /builddir/build/BUILD/doublestar-1.3.4/_build -V 1.3.4-1.fc39 -p /builddir/build/BUILDROOT/golang-github-bmatcuk-doublestar-1.3.4-1.fc39.x86_64 -g /usr/share/gocode -r '.*example.*'
Testing    in: /builddir/build/BUILD/doublestar-1.3.4/_build/src
         PATH: /builddir/build/BUILD/doublestar-1.3.4/_build/bin:/usr/lib64/ccache:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/sbin
       GOPATH: /builddir/build/BUILD/doublestar-1.3.4/_build:/usr/share/gocode
  GO111MODULE: off
      command: go test -buildmode pie -compiler gc -ldflags " -X github.com/bmatcuk/doublestar/version=1.3.4 -extldflags '-Wl,-z,relro -Wl,--as-needed  -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -Wl,--build-id=sha1 -specs=/usr/lib/rpm/redhat/redhat-package-notes  '"
      testing: github.com/bmatcuk/doublestar
github.com/bmatcuk/doublestar
--- FAIL: TestMatch (0.00s)
    doublestar_test.go:155: #57. Match(`a[`, `a`) != path.Match(...). Got false, <nil> want false, syntax error in pattern
FAIL
exit status 1
FAIL	github.com/bmatcuk/doublestar	0.015s

Hi,

Thanks for a great library. However, it seems that I've found a bug.

match, _ := doublestar.Match("/@tests**", "/@tests/AppTest/TestGetCategories")
revel.ERROR.Print("Match: " + strconv.FormatBool(match))

This Match function returns false. I believe it matches incorrectly when first "*" in the pattern has same position as "/" in the matched string.

Best,

Imagine I might be missing something obvious. If I have a file like /a/b/c/d.go and I want to let a process within /a/b/c/ match on *.go but also on ../c/d.go how do I do that?

It seems if I create an fs with os.DirFS("/a/b/c/") I can get the first example to work, but then I can't get the second example to work since passing ../c/d.go to the fs.FS created by the os.DirFS will error. fs.Sub also doesn't seem to solve this problem.

It seems like I could get around this by providing my own fs.FS with the properties needed to move up directories, but I'm wondering if there is a suggested solution here outside of moving back to v3?

Edit: oh, do I just pre-process the input and compute the absolute path?

The globbing expression dir1/**/dir2/* is descending into child directories within dir2. So it is matching dir1/dir1_1/dir2/dir2_1/dir3, where I'd only expect it to match dir1/dir1_1/dir2/dir2_1.

From doublestar v3 to doublestar v4, it appears that the matching in PathMatch() has changed.

The same pattern/path pair returns true in v3 and returns false in v4.

A test that demonstrates the behavior:

package doublestar_windows_bug

import (
	"testing"

	doublestarv3 "github.com/bmatcuk/doublestar/v3"
	doublestarv4 "github.com/bmatcuk/doublestar/v4"
	"github.com/stretchr/testify/assert"
)

func TestDoublestarWindows(t *testing.T) {
	pattern := "foo/**/*.md"
	path := "foo\\test\\bar\\baz\\1.md"

	match, _ := doublestarv3.PathMatch(pattern, path)
	assert.True(t, match)

	match, _ = doublestarv4.PathMatch(pattern, path)
	assert.False(t, match)
}

I was able to replicate the doublestar v4 PathMatch() behavior in Windows versions 21H2, 22H2 and 1809, but I'm unsure of how many versions are affected in total.

The above test was confirmed on a machine with the following:
Edition: Windows 10 Home
Version: 22H2
OS build: 19045.2251
Experience: Windows Feature Experience Pack 120.2212.4180.0

Note: due to the \\ path separators, the above test will fail in MacOS / Linux.

Would the maintainers of this package be comfortable with supporting the exclamation mark (!) as an alternative to the caret (^) in non-matching character classes? e.g.: [!class] instead of [^class]
Relevant issue: golang/go#41394

I'd be happy to make a pull request, looking at the code it looks like a straightforward change. Thank you!

Using a globstar in glob patterns seems to return more than it should.

Adding a MatchTest of:

{"**/c", "a/b", false, nil, true},

and running the tests gives:

--- FAIL: TestGlob (0.01s)
    doublestar_test.go:125: #60. Glob(`**/c`) = []string{"test/a", "test/a/b", "test/a/b/c", "test/a/b/c", "test/a/c", "test/a/c", "test/abc", "test/axbxcxdxe", "test/axbxcxdxe/xxx", "test/axbxcxdxexxx", "test/b", "test/b/c", "test/c"} - contains a/b, but shouldn't
FAIL
exit status 1

or have I misunderstand how globstar is supposed to work?

Best regards, Poffe.

hello, thanks for a great project.. found a typo in the project description (can't make a PR for this!)

Implements support for double star (**) matches in golang's path.Match and filepatch.Glob.

should be filepath.Glob not filepatch.Glob

regards,

n

First thanks for this wonderful doublestar glob, it makes file matching very easy.

A possible bug we encountered in our usage of doublestar.Glob:
When a file has its name starting with "【" ( Lenticular bracket wikipedia , ulookup ) it doesnt seem to match. Having the "【" in the path or in the filename but not as first character the file matches.

An example to illustrate the issue.

touch "/home/username/test/file.mp4"
touch "/home/username/test/【file.mp4"
touch "/home/username/test/ 【file.mp4"

note: the second file has the filename starting with "【" while the third has an extra space added " 【" ( can be any normal char instead of space )

Running the below code ( close to our use case)

package main

import (
	"fmt"
	"github.com/bmatcuk/doublestar"
	"path/filepath"
)

func main() {
	path := "/home/username/test"
	pattern := "**/*.{mp4}"
	re, err := doublestar.Glob(filepath.Join(path, pattern))

	if err != nil {
		fmt.Printf("error %v", err)
	}

	for _, file := range re {
		fmt.Println(file)
	}
}

The files printed ( matched ) are the first and the last not the second.

/home/username/test/ 【file.mp4
/home/username/test/file.mp4

Apart from "【" i would suspect more special unicode characters won't match.

doublestar.FilepathGlob("..") returns nil. Since it's meant to be a drop-in replacement for filepath.Glob(), it should return []string{".."}

Hi there. Thanks for the a neat and well-put-together module. I have two projects that use doublestar devd and modd. Both of these are cross-platform, with releases on Windows. Unfortunately the doublestar repo includes files that can't exist on Windows, which means that a "go get" of my projects won't work. Here's an excerpt:

I am happy to see that the doublestar has been updated, but after test, the match between /* and // is true, which is not compatible with the statement you made in the documentation:

 *	matches any sequence of non-path-separators

The way to reproduce this mistake:

Match(`/*`, `//`)

Hey there! I tried to do a glob searching using /Users/keithpitt/Desktop/*.doc but it didn't pick up any files. I checked out the source and thought it would be because the empty string here: https://github.com/bmatcuk/doublestar/blob/master/doublestar.go#L161 should be "/". I hacked it in to test, and it still didn't work. Perhaps that type of search isn't supported? Thanks for you help!

Hi,
a change was introduced in 1.30 to return the error from readdir, but this change is causing the glob results to get corrupted due to nil getting returned and overwriting all previous glob results.

https://github.com/bmatcuk/doublestar/blob/master/doublestar.go#L452

From what I can tell the line should just return rather than return nil, err.

It seems that \{ and \} are not handled properly in both pattern patching and path evaluation:

• The pattern **/log file \{????-??-??\}.txt will fail with a syntax error
• The input path src/logs/log file {2021-05-11}.txt will fail with any pattern (this is a valid path)

If I remove both from my test the other patterns and path combinations work fine. This does not seem to be the case for brackets, e.g. src/logs/log file [2021-05-11].txt
and **/log file [????-??-??].txt. I am using doublestar.Match.

Is it possible to configure Glob in such a way that it will return matching files but not matching directories?

For example, supposing that I have no knowledge of naming or structure, I just want to find all files immediately in the /var/log directory.

/var/log/some_file
/var/log/another_file
/var/log/some_dir/foo

If I specify the pattern /var/log/*, the result is [ "some_file", "another_file", "some_dir" ].

Is there a way to match only [ "some_file", "another_file" ]? If I'm understanding correctly, this is not currently possible. If that is the case, would you be open to adding a new GlobOption that specifies whether the result should include only files?

For context, this library is being used in the OpenTelemetry Collector, primarily for finding log files.

if **/ is followed by {...}, Match() does not work correctly in v4.
it works fine until v3.

https://go.dev/play/p/chnmf65L8EK

From Glob:

"By default, file and directory names starting with . are only matched with literal .. The patterns *, **, ?, and [] will not match a leading .. To alter this behavior, you can use the DOTGLOB flag."

Hi Bob,

thanks for this awesome package!

I created a version for the upcoming io/fs: https://github.com/forensicanalysis/fsdoublestar

I just wanted to let you know and say thanks.

Would it be possible to add support for fs.SkipDir on GlobWalk ?

doublestar.FilepathGlob should return filepath.ErrBadPattern, not path.ErrBadPattern. That's because doublestar.FilepathGlob should be consistent with the filepath.Glob behavior.

doublestar.Glob correctly returns path.ErrBadPattern for consistency with io/fs.Glob.

I might be missing something here, but I want to be able to use the glob syntax to match all files except for those with certain file extensions. In glob I can use the following:

./{app,src,resources,public}/**/*.!(css|scss)

As best as I can tell there is no combination of patterns accepted by this project that would allow for a series of strings in a negative match. If I have missed something I appreciate the feedback, but otherwise consider this a feature request! 😊

Sadly, I'm writing some code that need to work on windows (and the dependency I'm using is actually fileb0x which uses doublestar) and I couldn't figure out why no paths worked. I then noticed your latest commit (d8311f3) causes all of your tests to fail on windows. For now, I've reverted to v1.0.10 but I see what you're trying to do and still think it's a great improvement (so paths don't have to be OS specific inside the patterns). If I get a chance later I'll have a peek myself, but upon quick inspection, there's going to be a little more work with the way that paths will have to be translated into/out of \ and /

For reference:

$ go test
--- FAIL: TestGlob (0.02s)
        doublestar_test.go:178: #0. Glob([`abc`]) = []string(nil) - doesn't contain [abc], but should
        doublestar_test.go:178: #0. Glob([`abc`]) = []string(nil) - doesn't contain [abc], but should
        doublestar_test.go:178: #1. Glob([`*`]) = []string(nil) - doesn't contain [abc], but should
        doublestar_test.go:178: #1. Glob([`*`]) = []string(nil) - doesn't contain [abc], but should
        doublestar_test.go:178: #2. Glob([`*c`]) = []string(nil) - doesn't contain [abc], but should
        doublestar_test.go:178: #2. Glob([`*c`]) = []string(nil) - doesn't contain [abc], but should
        doublestar_test.go:178: #3. Glob([`a*`]) = []string(nil) - doesn't contain [a], but should
        doublestar_test.go:178: #3. Glob([`a*`]) = []string(nil) - doesn't contain [a], but should
        doublestar_test.go:178: #4. Glob([`a*`]) = []string(nil) - doesn't contain [abc], but should
        doublestar_test.go:178: #4. Glob([`a*`]) = []string(nil) - doesn't contain [abc], but should
        doublestar_test.go:178: #6. Glob([`a*` `b`]) = []string(nil) - doesn't contain [abc b], but should
        doublestar_test.go:178: #6. Glob([`a*` `b`]) = []string(nil) - doesn't contain [abc b], but should
        doublestar_test.go:178: #8. Glob([`a*b*c*d*e*` `f`]) = []string(nil) - doesn't contain [axbxcxdxe f], but should
        doublestar_test.go:178: #8. Glob([`a*b*c*d*e*` `f`]) = []string(nil) - doesn't contain [axbxcxdxe f], but should
        doublestar_test.go:178: #9. Glob([`a*b*c*d*e*` `f`]) = []string(nil) - doesn't contain [axbxcxdxexxx f], but should
        doublestar_test.go:178: #9. Glob([`a*b*c*d*e*` `f`]) = []string(nil) - doesn't contain [axbxcxdxexxx f], but should
        doublestar_test.go:178: #12. Glob([`a*b?c*x`]) = []string(nil) - doesn't contain [abxbbxdbxebxczzx], but should
        doublestar_test.go:178: #12. Glob([`a*b?c*x`]) = []string(nil) - doesn't contain [abxbbxdbxebxczzx], but should
        doublestar_test.go:178: #14. Glob([`ab[c]`]) = []string(nil) - doesn't contain [abc], but should
        doublestar_test.go:178: #14. Glob([`ab[c]`]) = []string(nil) - doesn't contain [abc], but should
        doublestar_test.go:178: #15. Glob([`ab[b-d]`]) = []string(nil) - doesn't contain [abc], but should
        doublestar_test.go:178: #15. Glob([`ab[b-d]`]) = []string(nil) - doesn't contain [abc], but should
        doublestar_test.go:178: #19. Glob([`ab[^e-g]`]) = []string(nil) - doesn't contain [abc], but should
        doublestar_test.go:178: #19. Glob([`ab[^e-g]`]) = []string(nil) - doesn't contain [abc], but should
        doublestar_test.go:178: #21. Glob([`a?b`]) = []string(nil) - doesn't contain [a☺b], but should
        doublestar_test.go:178: #21. Glob([`a?b`]) = []string(nil) - doesn't contain [a☺b], but should
        doublestar_test.go:178: #22. Glob([`a[^a]b`]) = []string(nil) - doesn't contain [a☺b], but should
        doublestar_test.go:178: #22. Glob([`a[^a]b`]) = []string(nil) - doesn't contain [a☺b], but should
        doublestar_test.go:178: #25. Glob([`[a-ζ]*`]) = []string(nil) - doesn't contain [α], but should
        doublestar_test.go:178: #25. Glob([`[a-ζ]*`]) = []string(nil) - doesn't contain [α], but should
        doublestar_test.go:184: #37. Glob([`[]a]`]) has error <nil>, but should be syntax error in pattern
        doublestar_test.go:184: #37. Glob([`[]a]`]) has error <nil>, but should be syntax error in pattern
        doublestar_test.go:184: #38. Glob([`[-]`]) has error <nil>, but should be syntax error in pattern
        doublestar_test.go:184: #38. Glob([`[-]`]) has error <nil>, but should be syntax error in pattern
        doublestar_test.go:184: #39. Glob([`[x-]`]) has error <nil>, but should be syntax error in pattern
        doublestar_test.go:184: #39. Glob([`[x-]`]) has error <nil>, but should be syntax error in pattern
        doublestar_test.go:184: #40. Glob([`[x-]`]) has error <nil>, but should be syntax error in pattern
        doublestar_test.go:184: #40. Glob([`[x-]`]) has error <nil>, but should be syntax error in pattern
        doublestar_test.go:184: #41. Glob([`[x-]`]) has error <nil>, but should be syntax error in pattern
        doublestar_test.go:184: #41. Glob([`[x-]`]) has error <nil>, but should be syntax error in pattern
        doublestar_test.go:184: #42. Glob([`[-x]`]) has error <nil>, but should be syntax error in pattern
        doublestar_test.go:184: #42. Glob([`[-x]`]) has error <nil>, but should be syntax error in pattern
        doublestar_test.go:184: #43. Glob([`[-x]`]) has error <nil>, but should be syntax error in pattern
        doublestar_test.go:184: #43. Glob([`[-x]`]) has error <nil>, but should be syntax error in pattern
        doublestar_test.go:184: #44. Glob([`[-x]`]) has error <nil>, but should be syntax error in pattern
        doublestar_test.go:184: #44. Glob([`[-x]`]) has error <nil>, but should be syntax error in pattern
        doublestar_test.go:184: #46. Glob([`[a-b-c]`]) has error <nil>, but should be syntax error in pattern
        doublestar_test.go:184: #46. Glob([`[a-b-c]`]) has error <nil>, but should be syntax error in pattern
        doublestar_test.go:184: #47. Glob([`[`]) has error <nil>, but should be syntax error in pattern
        doublestar_test.go:184: #47. Glob([`[`]) has error <nil>, but should be syntax error in pattern
        doublestar_test.go:184: #48. Glob([`[^`]) has error <nil>, but should be syntax error in pattern
        doublestar_test.go:184: #48. Glob([`[^`]) has error <nil>, but should be syntax error in pattern
        doublestar_test.go:184: #49. Glob([`[^bc`]) has error <nil>, but should be syntax error in pattern
        doublestar_test.go:184: #49. Glob([`[^bc`]) has error <nil>, but should be syntax error in pattern
        doublestar_test.go:184: #51. Glob([`a[`]) has error <nil>, but should be syntax error in pattern
        doublestar_test.go:184: #51. Glob([`a[`]) has error <nil>, but should be syntax error in pattern
        doublestar_test.go:178: #52. Glob([`*x`]) = []string(nil) - doesn't contain [xxx], but should
        doublestar_test.go:178: #52. Glob([`*x`]) = []string(nil) - doesn't contain [xxx], but should
        doublestar_test.go:178: #53. Glob([`[abc]`]) = []string(nil) - doesn't contain [b], but should
        doublestar_test.go:178: #53. Glob([`[abc]`]) = []string(nil) - doesn't contain [b], but should
        doublestar_test.go:178: #55. Glob([`a` `**`]) = []string(nil) - doesn't contain [a b], but should
        doublestar_test.go:178: #55. Glob([`a` `**`]) = []string(nil) - doesn't contain [a b], but should
        doublestar_test.go:178: #56. Glob([`a` `**`]) = []string(nil) - doesn't contain [a b c], but should
        doublestar_test.go:178: #56. Glob([`a` `**`]) = []string(nil) - doesn't contain [a b c], but should
        doublestar_test.go:178: #57. Glob([`**` `c`]) = []string(nil) - doesn't contain [c], but should
        doublestar_test.go:178: #57. Glob([`**` `c`]) = []string(nil) - doesn't contain [c], but should
        doublestar_test.go:178: #58. Glob([`**` `c`]) = []string(nil) - doesn't contain [b c], but should
        doublestar_test.go:178: #58. Glob([`**` `c`]) = []string(nil) - doesn't contain [b c], but should
        doublestar_test.go:178: #59. Glob([`**` `c`]) = []string(nil) - doesn't contain [a b c], but should
        doublestar_test.go:178: #59. Glob([`**` `c`]) = []string(nil) - doesn't contain [a b c], but should
        doublestar_test.go:178: #63. Glob([`a` `**` `b`]) = []string(nil) - doesn't contain [a b], but should
        doublestar_test.go:178: #63. Glob([`a` `**` `b`]) = []string(nil) - doesn't contain [a b], but should
        doublestar_test.go:178: #64. Glob([`a` `**` `c`]) = []string(nil) - doesn't contain [a b c], but should
        doublestar_test.go:178: #64. Glob([`a` `**` `c`]) = []string(nil) - doesn't contain [a b c], but should
        doublestar_test.go:178: #65. Glob([`a` `**` `d`]) = []string(nil) - doesn't contain [a b c d], but should
        doublestar_test.go:178: #65. Glob([`a` `**` `d`]) = []string(nil) - doesn't contain [a b c d], but should
        doublestar_test.go:178: #67. Glob([`ab{c,d}`]) = []string(nil) - doesn't contain [abc], but should
        doublestar_test.go:178: #67. Glob([`ab{c,d}`]) = []string(nil) - doesn't contain [abc], but should
        doublestar_test.go:178: #68. Glob([`ab{c,d,*}`]) = []string(nil) - doesn't contain [abcde], but should
        doublestar_test.go:178: #68. Glob([`ab{c,d,*}`]) = []string(nil) - doesn't contain [abcde], but should
        doublestar_test.go:184: #69. Glob([`ab{c,d}[`]) has error <nil>, but should be syntax error in pattern
        doublestar_test.go:184: #69. Glob([`ab{c,d}[`]) has error <nil>, but should be syntax error in pattern
        doublestar_test.go:178: #70. Glob([`abc` `**`]) = []string(nil) - doesn't contain [abc b], but should
        doublestar_test.go:178: #70. Glob([`abc` `**`]) = []string(nil) - doesn't contain [abc b], but should
        doublestar_test.go:178: #71. Glob([`**` `abc`]) = []string(nil) - doesn't contain [abc], but should
        doublestar_test.go:178: #71. Glob([`**` `abc`]) = []string(nil) - doesn't contain [abc], but should
FAIL
exit status 1
FAIL    github.com/bmatcuk/doublestar   0.099s

I don't know how much you're trying to mimic path/filepath.Glob with this but on Windows, path/filepath.Glob converts forward slash to the appropriate path separator.

fmt.Println(filepath.Glob("*/*/*.txt"))                      // rel path to my txt files
fmt.Println(doublestar.Glob("**\\*.txt"))                    // works on windows only
fmt.Println(doublestar.Glob(filepath.FromSlash("**/*.txt"))) // FromSlash conversion , should work cross platform
fmt.Println(doublestar.Glob("**/*.txt"))                     // Doesn't work on windows

Using filepath.FromSlash works for me in my current problem but could be useful to use this in doublestar.Glob and Match anyway?

Neat package :)

The pattern "*/" does not match to the name "a/".
This is different from standard library path and path/filepath.

https://play.golang.org/p/KNjSNRHQ7HH

func main() {
	name := "a/"
	pattern := "*/"
	pResult, _ := path.Match(pattern, name)
	fpResult, _ := filepath.Match(pattern, name)
	dsResult, _ := doublestar.Match(pattern, name)
	fmt.Println(pResult, fpResult, dsResult)
}

Result: true true false

Firstly, thank you for this great library!

I'd like to be able to use the functions in my unit tests that run in a "virtual" root filesystem that is really a temporary directory in a real filesystem. This temporary directory is created just for the test and is cleaned up after the tests are run. This means that my tests are hermetic and bugs in my code do not damage the real filesystem.

This means that the paths passed to os.Open, os.Stat, and os.Lstat need to be modified, for example os.Open("/home/user/file") needs to be translated to os.Open("/tmp/dir/somewhere/with-random-characters/home/user/file").

Would you accept the PR that does the following:

• Does not touch the existing doublestar API.

• Adds a new function, say GlobOS(vos OS, pattern string) ([]string, error) that is just like the existing Glob(pattern string) ([]string, error) function except it takes an interface as the first parameter:

type OS interface {
	Open(name string) (*os.File, error)
    Stat(name string) (os.FileInfo, error)
    Lstat(name string) (os.FileInfo, error)
    PathSeparator() rune
}

• Provides an StandardOS value that implements the above OS interface and calls the underlying standard library functions.

• Refactors Glob to be a call GlobOS(StandardOS, pattern).

This would have the following benefits:

• No changes to the API for existing users.

• You run tests for Windows filesystems that run on Linux.

• Users can use doublestar with virtual filesystem libraries like go-vfs, afero, and billy.

Would you accept such a PR?

Hello,
Thank you for your work on this library.

I have a use case where I'd exclude files and include others. I skimmed through the test-cases as well but I couldn't find, hence the query.

Reproducing steps:

$ mkdir -p level1/level2/

$ touch level1/level2/{a.stderr.0,a.stdout.0,ignore.stderr.0,ignore.stdout.0,b.stderr.0,b.stdout.0}

$ ls level1/level2/!(ignore*).std* 
level1/level2/a.stderr.0  level1/level2/a.stdout.0  level1/level2/b.stderr.0  level1/level2/b.stdout.0

Corresponding go-snippet:

$ cat main.go
package main

import (
	"fmt"

	"github.com/bmatcuk/doublestar/v2"
)

func main() {
	matched, err := doublestar.Glob("level1/level2/!(ignore*).std* ")
	if err != nil {
		panic(err)
	}

        // So here we find no matched files
	if len(matched) == 0 {
		panic(fmt.Errorf("No matched files found"))
	}
	for _, match := range matched {
		fmt.Println(match)
	}
}

Is this by design or it could be a feature worth supporting in the library?

Just wanna say thank you for this great package! I use this package for ls-lint.

Keep up the great work!

Hello,

Thanks for this useful library!
If you try something like this

match, err := doublestar.Match("/mydir/**", "/mydir")
fmt.Printf("match %v, err %v\n", match, err)

you will see that there is a match. I looked at the test cases and I know this is by purpose.

I would have expected something like this

https://www.digitalocean.com/community/tools/glob?comments=true&glob=%2Fmydir%2F%2A%2A&matches=false&tests=%2Fmydir&tests=%2Fmydir1&tests=%2Fmydir%2Fa&tests=%2Fmydir%2Fa%2Fb&tests=%2Fmydir%2Fa%2Fb%2Ffile.txt

But I could also be misunderstanding something here. Any clarification would be appreciated, thanks!

Documentation for ** in doublestar.Match() says:

  '**'        matches any sequence of characters, including
              path separators.

The comment in the matching code also seems to imply similar properties:

Despite this, doublestar.Match("**.go", "dir/file.go") returns false: https://play.golang.org/p/BUl4Dsrlxlp.

I'm trying to install v2 of doublestar according to the instructions in the v2 readme, and I get:

$ go get github.com/bmatcuk/doublestar/v2
cannot find package "github.com/bmatcuk/doublestar/v2" in any of:
/snap/go/6745/src/github.com/bmatcuk/doublestar/v2 (from $GOROOT)
/home/me/gocode/src/github.com/bmatcuk/doublestar/v2 (from $GOPATH)

I'm running ubuntu 20.04 if that makes any difference.....

Hi 👋 ,

Thanks so much for the work on this project! At @reviewpad, we are trying to support the OSS projects that we use as dependencies to our main repository reviewpad/reviewpad.

Do you plan to join any sponsoring program? We are also happy to help the community to maintain the project.

Thanks!

Unfortunately, I found that your document does not match the behavior of the doublestar.Match()

1. As mentioned in the document , * will not match any path separators, but doublestar.Match("*", "/") will return true, and golang's native path.Match() will return false.
2. doublestar.Match("*", "") will return true, which looks like "*" can match 0 or more characters, but doublestar.Match("/*", "/") returns false
3. As mentioned in the documentation, * will not match any path separators, but doublestar.Match("/*", "/debug/") will return true.
4. doublestar.Match("/**/profile", "/debug/pprof/profile/") will return true. Although in most cases "profile" has the same meaning as "profile/", but the result is not rigorous and not intuitive.

The document of doublestar.Match() is so similar with the golang's native path.Match(), which leads me to think that you just added support for "**" and "{}", but now it seems that they return the opposite result in many cases.

I appreciate your work, maybe there is some misunderstanding in the above content, I look forward to your reply, thank you

Hello,

Alternatives in glob search patterns seem to not be working properly, when they contain slashes. According to documentation, it should be possible to escape special characters by using \, but it doesn't seem to work:

Steps to reproduce:

$ mkdir /tmp/test && touch /tmp/test/example{1,2}
$ ls /tmp/test
example1 example2

$ cat example.go
package main

import (
    "fmt"
    "github.com/bmatcuk/doublestar"
)

func main() {
    patterns := [6]string{
        "/tmp/test/*",
        "/tmp/test/{example1}",
        "/tmp/test/{example*}",
        "/tmp/test/{example1,example2}",
        "{/tmp/test/example*}",       // not working
        "{\\/tmp\\/test\\/example1}"} // not working
    for _, pattern := range patterns {
        fmt.Println("pattern:", pattern)
        matches, err := doublestar.Glob(pattern)
        if err != nil {
            fmt.Println(err)
        } else {
            fmt.Println(" ", matches, "\n")
        }
    }
}

$ go run example.go
pattern: /tmp/test/*
  [/tmp/test/example1 /tmp/test/example2] 

pattern: /tmp/test/{example1}
  [/tmp/test/example1] 

pattern: /tmp/test/{example*}
  [/tmp/test/example1 /tmp/test/example2] 

pattern: /tmp/test/{example1,example2}
  [/tmp/test/example1 /tmp/test/example2] 

pattern: {/tmp/test/example*}
syntax error in pattern
pattern: {\/tmp\/test\/example1}
  [] 

These patterns are working perfectly fine in both ksh and bash:

$ echo $0
ksh

$ ls /tmp/test/{example1,example2}
/tmp/test/example1  /tmp/test/example2

$ ls {/tmp/test/example1,/tmp/test/example2}
/tmp/test/example1  /tmp/test/example2

$ bash

$ echo $0
bash

$ ls /tmp/test/{example1,example2}
/tmp/test/example1  /tmp/test/example2

$ ls {/tmp/test/example1,/tmp/test/example2}
/tmp/test/example1  /tmp/test/example2

Being able to have full path (and slashes) in pattern is required to be able to find log files in different directories on the system (like /var/log & /opt/application/logs), more details about existing issue in Loki, which relies on doublestar, are available here.

Thanks!

It would be great to have functionality to compile globs similar to https://github.com/gobwas/glob (which is seemingly no longer actively maintained and has bugs causing issues)

This would have performance benefits as one would not be parsing the pattern each time

Hello,

It seems doublestar.Glob does not sort the results, like filepath.Glob does.

Steps to reproduce:

$ mkdir /tmp/test && cd /tmp/test && touch {1,2,a,b}

$ cat main.go
package main

import (
        "fmt"
        "path/filepath"

        "github.com/bmatcuk/doublestar"
)

func main() {
        pattern := "*"

        fpath, _ := filepath.Glob(pattern)
        dstar, _ := doublestar.Glob(pattern)

        fmt.Println("filepath:", fpath)
        fmt.Println("doublestar:", dstar)
}

$ go run main.go
filepath:   [1 2 a b main.go]
doublestar: [2 1 a main.go b]

Hi,

there is a bug when pattern starts with alternatives. Glob is called recursively instead of GlobOS.

Best regards

Pavel

Hi, the link https://github.com/bmatcuk/doublestar/blob/master/test/abc/working-symlink references itself. I'm seeing this cause issues when copying the doublestar project into a go vendor directory and then attempting to add the results to git. There's some strange corner cases in that the project we're seeing this issue in is vendoring a different project which then vendors doublestar.

Here are the errors we see:

$ git add vendor
error: readlink("vendor/github.com/cloudfoundry/bosh-utils/vendor/github.com/bmatcuk/doublestar/test/abc/working-symlink"): Permission denied
error: unable to index file vendor/github.com/cloudfoundry/bosh-utils/vendor/github.com/bmatcuk/doublestar/test/abc/working-symlink
fatal: adding files failed

$ ll vendor/github.com/cloudfoundry/bosh-utils/vendor/github.com/bmatcuk/doublestar/test/abc/working-symlink

ls: vendor/github.com/cloudfoundry/bosh-utils/vendor/github.com/bmatcuk/doublestar/test/abc/working-symlink: Permission denied
l---------  1 pivotal  staff  15 Sep  6 15:52 vendor/github.com/cloudfoundry/bosh-utils/vendor/github.com/bmatcuk/doublestar/test/abc/working-symlink

You can see that it is the file which is in the vendor director of the "bosh-utils" project that is causing issues.

Please let me know if there's any more information I can provide.
Best,
Zachary Auerbach and the CF BOSH team

Hello.
My use of this library is to extract files in a particular programming language from a git repository.

I have found that the program hangs in some repositories.

The following is sample code.

package main

import (
	"fmt"
	"os"

	"github.com/bmatcuk/doublestar/v4"
)

func main() {
	pattern := "**/*.go"
	path := "../k3os"
	fsys := os.DirFS(path)
	files, err := doublestar.Glob(fsys, pattern)
	if err != nil {
		panic(err)
	}
	fmt.Println(files)
}

The two I have found are rancher/k3os and google/google-ctf.

doublestar version: 4.2.0

This code:

matches, err := doublestar.Glob(fsys, "non-existent-path", doublestar.WithFailOnIOErrors())

incorrectly returns an error if non-existent-path does not exist. It should return an empty list of matches and no error.

The exists function is certainly buggy, as it returns an error when a file does not exist, whereas it should return false and no error. It should be something like:

// Returns true if the path exists
func (g *glob) exists(fsys fs.FS, name string) (bool, error) {
	_, err := fs.Stat(fsys, name)
	if errors.Is(err, fs.ErrNotExist) { // name not existing is not an error
		return false, nil
	}
	return err == nil, g.forwardErrIfFailOnIOErrors(err)
}

I am trying to figure out a way to expand files recursively, but can not seem to get it working:

 % find stack/provisioning/tenant 
stack/provisioning/tenant
stack/provisioning/tenant/ui
stack/provisioning/tenant/ui/main.libsonnet
stack/provisioning/tenant/oathkeeper
stack/provisioning/tenant/oathkeeper/main.libsonnet
stack/provisioning/tenant/oathkeeper/access-rules.libsonnet
stack/provisioning/tenant/utils.libsonnet
stack/provisioning/tenant/kratos
stack/provisioning/tenant/kratos/main.libsonnet
stack/provisioning/tenant/kratos/schemas
stack/provisioning/tenant/kratos/schemas/simple.json
stack/provisioning/tenant/kratos/schemas/ory.json
stack/provisioning/tenant/kratos/config.libsonnet
stack/provisioning/tenant/main.jsonnet

%  go install github.com/bmatcuk/doublestar/v2/examples

% examples "stack/provisioning/tenant/{,**/}*"
Searching on disk for pattern: stack/provisioning/tenant/{,**/}*

stack/provisioning/tenant/kratos/config.libsonnet
stack/provisioning/tenant/kratos/main.libsonnet
stack/provisioning/tenant/kratos/schemas
stack/provisioning/tenant/oathkeeper/access-rules.libsonnet
stack/provisioning/tenant/oathkeeper/main.libsonnet
stack/provisioning/tenant/ui/main.libsonnet

# Missing: stack/provisioning/tenant/main.jsonnet

Found 6 items.

I also tried several other approaches including stack/provisioning/tenant/{*,**/*} and others but it looks like none of them really work.

I am trying to match /var/log/plex/Plex Media Scanner Analysis.log but not /var/log/plex/Plex Media Scanner Analysis.2.log which as a regexp would work as [a-zA-Z ]*.log for the file name and I thought /var/log/plex/[a-zA-Z ]*.log in doublestar. But this is interpreted as (one letter, upper case or lower case or one space) followed by whatever followed by .log and not as (one letter, upper case or lower case or one space) zero to many times exactly followed by .log

Am I missing a way to do so?

Both on Windows and CircleCI (via Docker RUN) it seems that packages that depend on this are getting empty repos rather than code. See UnnoTed/fileb0x#50

Ubuntu and Mac OS seem unaffected.

I forked your repo and used a replace directive in the go.mod to my fork, no code changes whatsoever, and go get now works on Windows.

This leads me to believe that there's some sort of weird issue with the repo and Github's CDNs or some such.

I'm wondering if you could try force pushing an old commit and then updating, or deleting the v1.1.1 tag and then force updating it (to the same commit of course) or something like that that might trigger Github's caching mechanisms to bust and deliver code (and maybe just pushing a new tag to the same commit).

Dependabot is a free service bought by Github that analise all dependencies listed in your project and send pull requests whenever a new version of any of these dependencies is released.

If the project has a CI the tests are automaticly trigged and the new dependency is tested wihout affect the master branch.

When I thought about developing a wheel, I found it, which would save me a lot of time, thanks!
So will there be any official benchmark?

This makes easier to use doublestar when the calling code is already using a filesystem interface like go-billy or afero. I have a fork with the changes here: c4milo@af79f84

If there is interest, I can send a PR.

Thanks 🙇