| Library | Speed |
|---|---|
| os.walk (std lib) | 225.98 |
| faster_than_walk | 70.0 |
- Benchmarks run on Docker from Dockerfile on this repo.
- Speed is IRL time to complete 10000 full recursive directory walks to
/usr/share/. - This depends a lot on I/O but overall most of times is faster.
- Stats as of year 2019.
- x86_64 64Bit AMD, SSD, Arch Linux.
import faster_than_walk as ftw
print(ftw.walk("/path/to/some/folder")) # From path string
print(ftw.walk("/path/to/some/folder", [".py", ".txt"])) # Filter by Extension
print(ftw.walk("/path/to/some/folder", [".py"], followlinks=True)) # Follow SymLinks
print(ftw.walk("/path/to/some/folder", [".py"], followlinks=True, yieldfiles=False)) # Folders only
print(ftw.walk_glob("*.txt")) # Standard POSIX Globs
print(ftw.walk_glob("**/*.txt")) # Recursive standard POSIX GlobsDescription: Takes a path string, makes a full recursive walk of folders and returns a list with the result. Recursive directory walk from path to list of strings.
Arguments:
folderpathA local path,strtype, required, must not be empty string.extensionslist of file Extensions,listtype, optional, list must containstr, must not be empty list.followlinksSet toTrueto follow SymLinks,booltype, optional.yieldfilesSet toFalseto return folders only,booltype, optional.debugsSet toTruefor Debug mode,booltype, optional, not needed for normal usage.
Returns:
Result, list type, values of the list are str type.
Description: Takes a path string, makes a full recursive walk of folders and returns a list with the result. Recursive directory walk from standard POSIX like Glob Pattern string to list of strings, allows Recursive and Non-Recursive Globs.
Arguments:
globpatternA standard POSIX like Glob Pattern string,strtype, required, must not be empty string.
Returns:
Result, list type, values of the list are str type.
Description: Takes a path string, makes a full recursive walk of folders and returns a list with the result. Recursive directory walk from path to list of strings.
Arguments:
folderpathA local path,strtype, required, must not be empty string.recursiveSet toTrueto be Recursive,booltype, optional.relativeSet toTrueto Relative paths,booltype, optional.
Returns:
Result, list type, values of the list are str type.
Description: Takes a path string, makes a full recursive walk of folders and returns a list with the result. Recursive directory walk from path to list of strings.
Arguments:
globpatternA standard POSIX like Glob Pattern string,strtype, required, must not be empty string.folders_onlySet toTrueto be only return Folders,booltype, optional.
Returns:
Result, list type, values of the list are str type.
Description: Debug the internal Configuration of the library, takes no arguments, returns nothing, prints the pretty-printed human-friendly multi-line JSON Configuration to standard output terminal. Use it for copy&paste into new Bug Reports!!!.
Arguments: None.
Returns: None.
For more Examples check the Examples.
Instead of having a pair of functions with a lot of arguments that you should provide to make it work, we have tiny functions with very few arguments that do one thing and do it as fast as possible.
pip install faster_than_walk
- None
- Make a quick test drive on Docker!.
$ ./build-docker.sh
$ ./run-docker.sh
$ ./run-benchmark.sh # Inside Docker.- ✅ Linux
- ✅ Windows
- ✅ Mac
- ✅ Android
- ✅ Raspberry Pi
- ✅ BSD
More Faster Libraries...
- https://github.com/juancarlospaco/faster-than-csv#faster-than-csv
- https://github.com/juancarlospaco/faster-than-requests#faster-than-requests
- We want to make Open Source faster, better, stronger.
- Python 3.
- GCC.
- 64 Bit.
- Whats the idea, inspiration, reason, etc ?.
- This requires Cython ?.
No.
- This runs on PyPy ?.
No.
- This runs on Python2 ?.
I dunno. (Not supported)
- This runs on 32Bit ?.
No.
- This runs with Clang ?.
No.
- How can I Install it ?.
pip install faster_than_walk
- How can be faster than
os.walk?.
I dunno.
- How to Filter by File Extensions ?.
extensions = [".py", ".txt", ".tar.gz"]
(Glob Pattern may be faster for several file extensions than the extensions argument)
- How to force Follow SymLinks ?.
followlinks = True
- How to force return Folders only and Not Files ?.
yieldfiles = False
- How to enable Debug ?.
debugs = True
- How to use a Glob Pattern ?.
globpattern = "*.txt"
- How to use a Recursive Glob Pattern ?.
globpattern = "**/*.txt"
- I need to
import globon my code to use a Glob Pattern ?.
No.
- Whats a Glob Pattern ?.
https://en.wikipedia.org/wiki/Glob_%28programming%29
- Why needs 64Bit ?.
Maybe it works on 32Bit, but is not supported, integer sizes are too small, and performance can be worse.
- Why needs Python 3 ?.
Maybe it works on Python 2, but is not supported, and performance can be worse, we suggest to migrate to Python3.
- Can I wrap the functions on a
try: except:block ?.
Functions do not have internal try: except: blocks,
so you can wrap them inside try: except: blocks if you need very resilient code.
- PIP fails to install or fails build the wheel ?.
Add at the end of the PIP install command:
--isolated --disable-pip-version-check --no-cache-dir --no-binary :all:
Not my Bug.
- How to Build the project ?.
build.sh
- How to Package the project ?.
package.sh
- This requires Nimble ?.
No.
- Whats the unit of measurement for speed ?.
Unmmodified raw output of Python timeit module.
Please send Pull Request to Python to improve the output of timeit.
- How does this compare to
scandir?.
Scandir:
- Not Recursive.
- Returns iterator.
- Items yielded in random order.
- Lenght of return unknown, you can not assert it return something or nothing.
ftw:
- Recursive.
- Returns
list. - Items preserve order.
- Lenght of return known, you can assert it returns something.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent