Code Monkey home page Code Monkey logo

Comments (5)

haraldrudell avatar haraldrudell commented on July 24, 2024

Appears to be caused by, in this case, a Go project having a fakeGoPATH directory structure eventually symlinked back to its own go.mod in a relative parent directory
ls -l …
xyz -> ../../../..

I think this was something before Go modules became widely used

What one could do here is to model that scandir code on Go’s filepath.EvalSymlinks https://pkg.go.dev/path/filepath#EvalSymlinks
@google wrote that, it works

this code here walks anything avoiding recursion from cyclic paths. It doesn’t include the latest learnings but won’t crash
I don’t know for anybody else doing this. Like @facebook doesn’t
https://pkg.go.dev/github.com/haraldrudell/[email protected]/pfs#Traverser

walking symlinks is a huge problem space

This code could be easily transpiled to ECMAScript and it will work every time

from storybook.

shilman avatar shilman commented on July 24, 2024

@valentinpalkovic I am able to reproduce this in a sandbox:

  1. Remove /sandbox from .gitignore
  2. Run the sandbox and search for button => it will return a bunch of results
  3. Link a parent directory ln -s ../../.. boom
  4. Run the sandbox and search for button => it will hang -- I think it is traversing the symlink infinitely

I tried adding the following options to globby but it did not fix the problem:

    followSymbolicLinks: false,
    onlyFiles: true,

When I remove gitignore: true from the options:

  1. It does fix the problem (followSymLinks is respected)
  2. The glob runs twice as fast (500ms vs 1000+ms with gitignore)

Therefore it seems globby gitignore is buggy and also introduces a performance regression. I wonder if we can get a fix into globby?

from storybook.

valentinpalkovic avatar valentinpalkovic commented on July 24, 2024

In my opinion, setting followSymbolicLinks: false wouldn't be an option. Symlinks are common, and when properly set up, they usually don't cause loops.

Since globby respects the user's gitignore, wouldn't a further entry in the gitignore file solve the issue by excluding the symlinked folder?

@shilman Also, could you experiment with the deep option? By default, it is set to Infinity. Maybe we should limit the folder depth to, for example, 100.

from storybook.

valentinpalkovic avatar valentinpalkovic commented on July 24, 2024

@haraldrudell Just for context: We haven't implemented the glob search by ourselves, but we rely on the globby library. If you think cyclical symlinks should be treated in a different way, then I think it makes sense to open a bug report there, unless it is not configurable by globby itself.

from storybook.

shilman avatar shilman commented on July 24, 2024

@valentinpalkovic I don't think deep is a good solution. deep: 50 still blows up, and deep: 20 works but takes 10 seconds to complete a single search.

This bug is important to me because I want to use fileSearch as part of some telemetry and I need it to be performant. It would run on startup once per day (I'd cache the results). I suppose I could pass followSymbolicLinks: false JUST for the telemetry case and we could leave it to true for the search UI. But it's a bummer that gitignore doesn't play nice with followSymLinks.

from storybook.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.