bootandy / dust Goto Github PK

View Code? Open in Web Editor NEW

7.9K 7.9K 165.0 657 KB

A more intuitive version of du in rust

License: Apache License 2.0

Rust 77.92% PowerShell 7.99% Shell 6.91% Elvish 4.12% Roff 3.06%

dust's People

Contributors

Stargazers

Watchers

Forkers

nebkor abhi-jha drupalio happy-ferret amilajack chrisjohnson00 m4d3bug csu-anzai denisgolius csu-xiao-an mvertescher zerothabhishek adminxvii znielsen lespea rivy celti rasmushalland picassio mybigman mbrukman rustyforks j2ghz waltarix chaosstudygroup page2me aarzhanov anangnovriadi huzairuje sidvud98 skibaa phdenzel rylwin rdrcode olivierh59500 tgotwig ototot sylvestre treble37 isgasho sherifabdlnaby alexmaco jcaesar rustbunker ksxgithub xomyakus thomastay ibotty karmaz95 combinatorist mu-l wa7sa34cx alerque jaroslavpachola xzlinux ethanhs bozyao zva90 ishaanahuja7 davidoster rtpg coderboy24x7 gustavobb tychomagneticanomaly-1 sthagen zhaopufeng psy-repos-rust wizard-28 studiovc voidexpr u240 horki ajunlonglive ibuk01 standardgalactic skt041959 jacke x0f5c3 naterichman phial3 playfloor nico-abram haywire2099 dev-msp ibrother08 mt0803 noscripter liqizha qaison durant icodein cssolomon michuan lucretiel cuishuang hexiyou carlosjordandev xxoolm jedsek kianmeng

dust's Issues

build fails on Ubuntu 16.04

cargo bails out whilst compiling:

error[E0432]: unresolved import self::std::hint--> .cargo/registry/src/github.com-1ecc6299db9ec823/lazy_static-1.4.0/src/inline_lazy.rs:13:16 | 13 | use self::std::hint::unreachable_unchecked; | ^^^^ Could not findhintinstd`

error: aborting due to previous error

error: Could not compile lazy_static.
warning: build failed, waiting for other jobs to finish...
error: failed to compile du-dust v0.4.0, intermediate artifacts can be found at /tmp/cargo-install.9fTx6Q9xf9f3

Caused by:
build failed
`
any help appreciated,

Phil.

Add --reverse flag

Problem

The results of dust -s -d=1 in a folder containing many files will be displayed and having to scroll back to the top is, meh.

Solution

Add a --reverse flag so the important information is displayed right at the end of the command execution.

Great work on this friend! 👍

Any room for performance improvement?

Benchmark

hyperfine 'dutree app/'
Benchmark #1: dutree app/
  Time (mean ± σ):      57.4 ms ±   1.0 ms    [User: 19.2 ms, System: 37.8 ms]
  Range (min … max):    55.9 ms …  61.9 ms    51 runs

hyperfine 'dust app/'
Benchmark #1: dust app/
  Time (mean ± σ):      20.6 ms ±   2.8 ms    [User: 29.0 ms, System: 15.6 ms]
  Range (min … max):    17.2 ms …  27.0 ms    108 runs
 
hyperfine 'du app/'
Benchmark #1: du app/
  Time (mean ± σ):       6.3 ms ±   0.2 ms    [User: 2.6 ms, System: 3.6 ms]
  Range (min … max):     5.9 ms …   7.7 ms    401 runs

dust is ~3x slower than du but ~2.5x faster than duetree

Improvement suggestions from nachoparker/dutree#18 may apply

Using https://github.com/BurntSushi/walkdir
Minimize # sys calls
Using pointer for the tree structure

Performance

Dust is currently about 4 times slower than du.

It would be nice to know why and how it's slower. Did you add benchmarks of some sorts?

In a second phase, it's probably easy to optimize it?

When we getting v2?

When can we expect du_dust 2.0.0?

add man page

There doesn't seem to be a man page.

I've installed on macos via Hombrew:

$ brew info dust
dust: stable 0.5.2 (bottled), HEAD

Clone should probably be ignore / size computed differently

Hello,

First, thx for this amazing tools.
Thanks to it I was able to clean my computer from a lot of bull***

I open this issue on a subject i don't rely understand by I hope it can help.

Thx to bsd/apple clone things, i am able to clone dir/ file with cp -c .

Clone are, as understand it, a kind of weird hard link but when you write over it, it save the diff.

In my example, you can see that I clone the dankest movie of my library few time and df -h doesn't report a disk usage difference.

dust repeat the clones as there take more space on disk.

[Movies] df -h
Filesystem      Size   Used  Avail Capacity     iused      ifree %iused  Mounted on
/dev/disk1s1   466Gi   10Gi   90Gi    11%      484283 4881968597    0%   /
devfs          403Ki  403Ki    0Bi   100%        1404          0  100%   /dev
/dev/disk1s2   466Gi  352Gi   90Gi    80%     3172969 4879279911    0%   /System/Volumes/Data
/dev/disk1s5   466Gi   12Gi   90Gi    12%          12 4882452868    0%   /private/var/vm
map auto_home    0Bi    0Bi    0Bi   100%           0          0  100%   /System/Volumes/Data/home
/dev/disk2s2   105Mi  105Mi    0Bi   100%           3 4294967276    0%   /Volumes/Install Google Drive File Stream
drivefs         30Gi  7.0Gi   23Gi    24% 18446744069414596880 4294967295 146880675765702656%   /Volumes/GoogleDrive
drivefs         30Gi  7.0Gi   23Gi    24% 18446744069414740697 4294967295 11796403584574832%   /Volumes/GoogleDrive
[Movies] cp -cR Sonic.the.Hedgehog.2020.720p.HDRip.XviD.MP3-STUTTERSHIT clone1
[Movies] cp -cR Sonic.the.Hedgehog.2020.720p.HDRip.XviD.MP3-STUTTERSHIT clone2
[Movies] cp -cR Sonic.the.Hedgehog.2020.720p.HDRip.XviD.MP3-STUTTERSHIT clone3
[Movies] cp -cR Sonic.the.Hedgehog.2020.720p.HDRip.XviD.MP3-STUTTERSHIT clone4
[Movies] cp -cR Sonic.the.Hedgehog.2020.720p.HDRip.XviD.MP3-STUTTERSHIT clone5
[Movies] cp -cR Sonic.the.Hedgehog.2020.720p.HDRip.XviD.MP3-STUTTERSHIT clone6
[Movies] df -h
Filesystem      Size   Used  Avail Capacity     iused      ifree %iused  Mounted on
/dev/disk1s1   466Gi   10Gi   90Gi    11%      484283 4881968597    0%   /
devfs          403Ki  403Ki    0Bi   100%        1404          0  100%   /dev
/dev/disk1s2   466Gi  352Gi   90Gi    80%     3172987 4879279893    0%   /System/Volumes/Data
/dev/disk1s5   466Gi   12Gi   90Gi    12%          12 4882452868    0%   /private/var/vm
map auto_home    0Bi    0Bi    0Bi   100%           0          0  100%   /System/Volumes/Data/home
/dev/disk2s2   105Mi  105Mi    0Bi   100%           3 4294967276    0%   /Volumes/Install Google Drive File Stream
drivefs         30Gi  7.0Gi   23Gi    24% 18446744069414596880 4294967295 146880675765702656%   /Volumes/GoogleDrive
drivefs         30Gi  7.0Gi   23Gi    24% 18446744069414740697 4294967295 11796403584574832%   /Volumes/GoogleDrive
[Movies] dust
  46G ─┬ .
  24G  ├─┬ Star.Wars.The.Clone.Wars.S01.1080p.BluRay.x264-FLHD[rartv]
 1.1G  │ ├── Star.Wars.The.Clone.Wars.S01E22.1080p.BluRay.x264-FLHD.mkv
 1.1G  │ ├── Star.Wars.The.Clone.Wars.S01E20.1080p.BluRay.x264-FLHD.mkv
 1.1G  │ └── Star.Wars.The.Clone.Wars.S01E16.1080p.BluRay.x264-FLHD.mkv
 2.9G  ├─┬ Sonic.the.Hedgehog.2020.720p.HDRip.XviD.MP3-STUTTERSHIT
 2.9G  │ └── Sonic.the.Hedgehog.2020.720p.HDRip.XviD.MP3-STUTTERSHIT.avi
 2.9G  ├─┬ clone1
 2.9G  │ └── Sonic.the.Hedgehog.2020.720p.HDRip.XviD.MP3-STUTTERSHIT.avi
 2.9G  ├─┬ clone2
 2.9G  │ └── Sonic.the.Hedgehog.2020.720p.HDRip.XviD.MP3-STUTTERSHIT.avi
 2.9G  ├─┬ clone3
 2.9G  │ └── Sonic.the.Hedgehog.2020.720p.HDRip.XviD.MP3-STUTTERSHIT.avi
 2.9G  ├─┬ clone4
 2.9G  │ └── Sonic.the.Hedgehog.2020.720p.HDRip.XviD.MP3-STUTTERSHIT.avi
 2.9G  ├─┬ clone5
 2.9G  │ └── Sonic.the.Hedgehog.2020.720p.HDRip.XviD.MP3-STUTTERSHIT.avi
 2.9G  ├─┬ clone6
 2.9G  │ └── Sonic.the.Hedgehog.2020.720p.HDRip.XviD.MP3-STUTTERSHIT.avi
 1.4G  └─┬ Zero.Dark.Thirty.2012.1080p.BluRay.x265.10bit-z97
 1.4G    └── Zero.Dark.Thirty.2012.1080p.BluRay.x265.10bit-z97.mkv

I never ever developed in rust except 10 minute ago while try to see if metadata and filetype structure could help us here. seems not.

Actually, I have no clue to know if a file is a clone or not : (https://stackoverflow.com/questions/46417747/apple-file-system-apfs-check-if-file-is-a-clone-on-terminal-shell)

Would be glad to help you if you don't have any OSx to try thing out, but I don't think I would be able to PR anything.

Best Regards,

Low memory

I had dust fail when it ran out of memory:

memory allocation of 26214400 bytes failedAbandon

GNU sort uses temporary files so that it offloads big allocations to disk. Could this be considered ?

Installation problem on CentOS7

Hi,

This looks great. I wanted to install this on our CentOS7 cluster, but it runs into an error, for which I believe the key part is:

 = note: /usr/bin/ld: cannot find Scrt1.o: No such file or directory
          collect2: error: ld returned 1 exit status

Full error below:

$ cargo install du-dust
    Updating crates.io index
  Installing du-dust v0.4.4
   Compiling autocfg v0.1.7
   Compiling semver-parser v0.7.0
   Compiling cfg-if v0.1.10
   Compiling lazy_static v1.4.0
   Compiling libc v0.2.66
   Compiling scopeguard v1.0.0
   Compiling rayon-core v1.7.0
   Compiling bitflags v1.2.1
   Compiling either v1.5.3
   Compiling unicode-width v0.1.7
   Compiling ansi_term v0.11.0
   Compiling strsim v0.8.0
   Compiling vec_map v0.8.1
   Compiling ansi_term v0.12.1
   Compiling textwrap v0.11.0
error: linking with `cc` failed: exit code: 1
  |
  = note: "cc" "-Wl,--as-needed" "-Wl,-z,noexecstack" "-m64" "-L" "/home/wdecoster/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "/tmp/cargo-install7sSh8x/release/build/rayon-core-f7f2f1f46b6e30ee/build_script_build-f7f2f1f46b6e30ee.build_script_build.211rs0xc-cgu.0.rcgu.o" "/tmp/cargo-install7sSh8x/release/build/rayon-core-f7f2f1f46b6e30ee/build_script_build-f7f2f1f46b6e30ee.build_script_build.211rs0xc-cgu.1.rcgu.o" "-o" "/tmp/cargo-install7sSh8x/release/build/rayon-core-f7f2f1f46b6e30ee/build_script_build-f7f2f1f46b6e30ee" "/tmp/cargo-install7sSh8x/release/build/rayon-core-f7f2f1f46b6e30ee/build_script_build-f7f2f1f46b6e30ee.3g6uguohz55x28e8.rcgu.o" "-Wl,--gc-sections" "-pie" "-Wl,-zrelro" "-Wl,-znow" "-Wl,-O1" "-nodefaultlibs" "-L" "/tmp/cargo-install7sSh8x/release/deps" "-L" "/home/wdecoster/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,--start-group" "-Wl,-Bstatic" "/home/wdecoster/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-74488c47a41eb313.rlib" "/home/wdecoster/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libpanic_unwind-efbc2c947951b8cc.rlib" "/home/wdecoster/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libhashbrown-b177382d9a04ffbe.rlib" "/home/wdecoster/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_alloc-ba7d2244d33447d5.rlib" "/home/wdecoster/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libbacktrace-39569dc87e4ea301.rlib" "/home/wdecoster/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libbacktrace_sys-cd9e255bd82ddb5c.rlib" "/home/wdecoster/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_demangle-f83165e777dccb8d.rlib" "/home/wdecoster/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libunwind-10b085de263b1750.rlib" "/home/wdecoster/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcfg_if-9a4a22edf44da957.rlib" "/home/wdecoster/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-68fabc677efa98de.rlib" "/home/wdecoster/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc-026dc0061b48e8b6.rlib" "/home/wdecoster/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_core-7bf75bb619341145.rlib" "/home/wdecoster/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-ef54709e300503ed.rlib" "-Wl,--end-group" "/home/wdecoster/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-4ed27d8420cb4abc.rlib" "-Wl,-Bdynamic" "-ldl" "-lrt" "-lpthread" "-lgcc_s" "-lc" "-lm" "-lrt" "-lpthread" "-lutil" "-lutil"
  = note: /usr/bin/ld: cannot find Scrt1.o: No such file or directory
          collect2: error: ld returned 1 exit status
          

error: aborting due to previous error

error: could not compile `rayon-core`.
warning: build failed, waiting for other jobs to finish...
error: linking with `cc` failed: exit code: 1
  |
  = note: "cc" "-Wl,--as-needed" "-Wl,-z,noexecstack" "-m64" "-L" "/home/wdecoster/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "/tmp/cargo-install7sSh8x/release/build/bitflags-3969f63208505ddf/build_script_build-3969f63208505ddf.build_script_build.681e3too-cgu.0.rcgu.o" "/tmp/cargo-install7sSh8x/release/build/bitflags-3969f63208505ddf/build_script_build-3969f63208505ddf.build_script_build.681e3too-cgu.1.rcgu.o" "/tmp/cargo-install7sSh8x/release/build/bitflags-3969f63208505ddf/build_script_build-3969f63208505ddf.build_script_build.681e3too-cgu.10.rcgu.o" "/tmp/cargo-install7sSh8x/release/build/bitflags-3969f63208505ddf/build_script_build-3969f63208505ddf.build_script_build.681e3too-cgu.11.rcgu.o" "/tmp/cargo-install7sSh8x/release/build/bitflags-3969f63208505ddf/build_script_build-3969f63208505ddf.build_script_build.681e3too-cgu.12.rcgu.o" "/tmp/cargo-install7sSh8x/release/build/bitflags-3969f63208505ddf/build_script_build-3969f63208505ddf.build_script_build.681e3too-cgu.2.rcgu.o" "/tmp/cargo-install7sSh8x/release/build/bitflags-3969f63208505ddf/build_script_build-3969f63208505ddf.build_script_build.681e3too-cgu.3.rcgu.o" "/tmp/cargo-install7sSh8x/release/build/bitflags-3969f63208505ddf/build_script_build-3969f63208505ddf.build_script_build.681e3too-cgu.4.rcgu.o" "/tmp/cargo-install7sSh8x/release/build/bitflags-3969f63208505ddf/build_script_build-3969f63208505ddf.build_script_build.681e3too-cgu.5.rcgu.o" "/tmp/cargo-install7sSh8x/release/build/bitflags-3969f63208505ddf/build_script_build-3969f63208505ddf.build_script_build.681e3too-cgu.6.rcgu.o" "/tmp/cargo-install7sSh8x/release/build/bitflags-3969f63208505ddf/build_script_build-3969f63208505ddf.build_script_build.681e3too-cgu.7.rcgu.o" "/tmp/cargo-install7sSh8x/release/build/bitflags-3969f63208505ddf/build_script_build-3969f63208505ddf.build_script_build.681e3too-cgu.8.rcgu.o" "/tmp/cargo-install7sSh8x/release/build/bitflags-3969f63208505ddf/build_script_build-3969f63208505ddf.build_script_build.681e3too-cgu.9.rcgu.o" "-o" "/tmp/cargo-install7sSh8x/release/build/bitflags-3969f63208505ddf/build_script_build-3969f63208505ddf" "/tmp/cargo-install7sSh8x/release/build/bitflags-3969f63208505ddf/build_script_build-3969f63208505ddf.2tulew8g0xjtlar5.rcgu.o" "-Wl,--gc-sections" "-pie" "-Wl,-zrelro" "-Wl,-znow" "-Wl,-O1" "-nodefaultlibs" "-L" "/tmp/cargo-install7sSh8x/release/deps" "-L" "/home/wdecoster/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,--start-group" "-Wl,-Bstatic" "/home/wdecoster/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-74488c47a41eb313.rlib" "/home/wdecoster/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libpanic_unwind-efbc2c947951b8cc.rlib" "/home/wdecoster/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libhashbrown-b177382d9a04ffbe.rlib" "/home/wdecoster/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_alloc-ba7d2244d33447d5.rlib" "/home/wdecoster/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libbacktrace-39569dc87e4ea301.rlib" "/home/wdecoster/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libbacktrace_sys-cd9e255bd82ddb5c.rlib" "/home/wdecoster/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_demangle-f83165e777dccb8d.rlib" "/home/wdecoster/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libunwind-10b085de263b1750.rlib" "/home/wdecoster/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcfg_if-9a4a22edf44da957.rlib" "/home/wdecoster/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-68fabc677efa98de.rlib" "/home/wdecoster/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc-026dc0061b48e8b6.rlib" "/home/wdecoster/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_core-7bf75bb619341145.rlib" "/home/wdecoster/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-ef54709e300503ed.rlib" "-Wl,--end-group" "/home/wdecoster/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-4ed27d8420cb4abc.rlib" "-Wl,-Bdynamic" "-ldl" "-lrt" "-lpthread" "-lgcc_s" "-lc" "-lm" "-lrt" "-lpthread" "-lutil" "-lutil"
  = note: /usr/bin/ld: cannot find Scrt1.o: No such file or directory
          collect2: error: ld returned 1 exit status
          

error: aborting due to previous error

error: could not compile `bitflags`.
warning: build failed, waiting for other jobs to finish...
error: linking with `cc` failed: exit code: 1
  |
  = note: "cc" "-Wl,--as-needed" "-Wl,-z,noexecstack" "-m64" "-L" "/home/wdecoster/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "/tmp/cargo-install7sSh8x/release/build/libc-024ef14981899728/build_script_build-024ef14981899728.build_script_build.8innslw3-cgu.0.rcgu.o" "/tmp/cargo-install7sSh8x/release/build/libc-024ef14981899728/build_script_build-024ef14981899728.build_script_build.8innslw3-cgu.1.rcgu.o" "/tmp/cargo-install7sSh8x/release/build/libc-024ef14981899728/build_script_build-024ef14981899728.build_script_build.8innslw3-cgu.10.rcgu.o" "/tmp/cargo-install7sSh8x/release/build/libc-024ef14981899728/build_script_build-024ef14981899728.build_script_build.8innslw3-cgu.11.rcgu.o" "/tmp/cargo-install7sSh8x/release/build/libc-024ef14981899728/build_script_build-024ef14981899728.build_script_build.8innslw3-cgu.12.rcgu.o" "/tmp/cargo-install7sSh8x/release/build/libc-024ef14981899728/build_script_build-024ef14981899728.build_script_build.8innslw3-cgu.13.rcgu.o" "/tmp/cargo-install7sSh8x/release/build/libc-024ef14981899728/build_script_build-024ef14981899728.build_script_build.8innslw3-cgu.14.rcgu.o" "/tmp/cargo-install7sSh8x/release/build/libc-024ef14981899728/build_script_build-024ef14981899728.build_script_build.8innslw3-cgu.15.rcgu.o" "/tmp/cargo-install7sSh8x/release/build/libc-024ef14981899728/build_script_build-024ef14981899728.build_script_build.8innslw3-cgu.2.rcgu.o" "/tmp/cargo-install7sSh8x/release/build/libc-024ef14981899728/build_script_build-024ef14981899728.build_script_build.8innslw3-cgu.3.rcgu.o" "/tmp/cargo-install7sSh8x/release/build/libc-024ef14981899728/build_script_build-024ef14981899728.build_script_build.8innslw3-cgu.4.rcgu.o" "/tmp/cargo-install7sSh8x/release/build/libc-024ef14981899728/build_script_build-024ef14981899728.build_script_build.8innslw3-cgu.5.rcgu.o" "/tmp/cargo-install7sSh8x/release/build/libc-024ef14981899728/build_script_build-024ef14981899728.build_script_build.8innslw3-cgu.6.rcgu.o" "/tmp/cargo-install7sSh8x/release/build/libc-024ef14981899728/build_script_build-024ef14981899728.build_script_build.8innslw3-cgu.7.rcgu.o" "/tmp/cargo-install7sSh8x/release/build/libc-024ef14981899728/build_script_build-024ef14981899728.build_script_build.8innslw3-cgu.8.rcgu.o" "/tmp/cargo-install7sSh8x/release/build/libc-024ef14981899728/build_script_build-024ef14981899728.build_script_build.8innslw3-cgu.9.rcgu.o" "-o" "/tmp/cargo-install7sSh8x/release/build/libc-024ef14981899728/build_script_build-024ef14981899728" "/tmp/cargo-install7sSh8x/release/build/libc-024ef14981899728/build_script_build-024ef14981899728.3q2jtolj7xqfp7y.rcgu.o" "-Wl,--gc-sections" "-pie" "-Wl,-zrelro" "-Wl,-znow" "-Wl,-O1" "-nodefaultlibs" "-L" "/tmp/cargo-install7sSh8x/release/deps" "-L" "/home/wdecoster/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,--start-group" "-Wl,-Bstatic" "/home/wdecoster/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-74488c47a41eb313.rlib" "/home/wdecoster/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libpanic_unwind-efbc2c947951b8cc.rlib" "/home/wdecoster/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libhashbrown-b177382d9a04ffbe.rlib" "/home/wdecoster/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_alloc-ba7d2244d33447d5.rlib" "/home/wdecoster/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libbacktrace-39569dc87e4ea301.rlib" "/home/wdecoster/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libbacktrace_sys-cd9e255bd82ddb5c.rlib" "/home/wdecoster/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_demangle-f83165e777dccb8d.rlib" "/home/wdecoster/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libunwind-10b085de263b1750.rlib" "/home/wdecoster/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcfg_if-9a4a22edf44da957.rlib" "/home/wdecoster/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-68fabc677efa98de.rlib" "/home/wdecoster/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc-026dc0061b48e8b6.rlib" "/home/wdecoster/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_core-7bf75bb619341145.rlib" "/home/wdecoster/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-ef54709e300503ed.rlib" "-Wl,--end-group" "/home/wdecoster/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-4ed27d8420cb4abc.rlib" "-Wl,-Bdynamic" "-ldl" "-lrt" "-lpthread" "-lgcc_s" "-lc" "-lm" "-lrt" "-lpthread" "-lutil" "-lutil"
  = note: /usr/bin/ld: cannot find Scrt1.o: No such file or directory
          collect2: error: ld returned 1 exit status
          

error: aborting due to previous error

error: could not compile `libc`.
warning: build failed, waiting for other jobs to finish...
error: failed to compile `du-dust v0.4.4`, intermediate artifacts can be found at `/tmp/cargo-install7sSh8x`

Caused by:
  build failed

Dust 0.5.3 reports wrong version number

Download dust 0.5.3 from https://github.com/bootandy/dust/releases/download/v0.5.3/dust-v0.5.3-x86_64-unknown-linux-musl.tar.gz
Run dust -V from the downloaded package
Observe it reports version 0.5.2

Fix extra line that can occur on small outputs.

Sometimes the output can look like this:

4.0K ─┬ 
 8.0K ─┬ .
 8.0K  └─┬ many
 4.0K    ├── hello_file
   0B    └── a_file

Instead of:

 8.0K ─┬ .
 8.0K  └─┬ many
 4.0K    ├── hello_file
   0B    └── a_file

This seems to be down to a call to: e.path().ancestors() in utils/mod.rs which can return '/' as an ancestor.

Investigating.

Tests on Linux are filesystem-dependent

When running cargo test while putting together #59, it came to my attention that tests on Linux are filesystem-dependent. test_dir2 is 12K on my XFS home filesystem or my tmpfs /tmp, whereas the tests expect 24K (perhaps from ext4?).

This should at minimum be documented, but I'd recommend copying the test files to /tmp and running them there, as the vast majority of Linux systems default to /tmp being a tmpfs.

Add an option to exclude a directory

Hey there! Love dust and find it very useful. I would like to have an option to exclude directories from the search/listing though. Is there a way to do this (I don't see it in the help menu)? If not, I think adding a -X / --exclude option would be useful. I can do the work if you think it is technically feasible. Any thoughts?

Display filename with newline or other special character

When displaying files with special character like '\n', '\t', '\r', and so on, it might cause the bar to be placed wrong.
I think it is great to encode filenames like what ls do before printing it out.

Packaging

I'm mostly writing this so I get the close notification when this happens, but it would be neat if there were rpm/deb packages for dust.

What do 4 kinds of percentage-bar-characters mean?

Hello, I am new to dust.
I am curious about the percentage bar.

There are 4 kinds of characters which displays the percentage bar: ░, ▒, ▓, █
Which means what?

Thanks,

Option to not display total.

I have a directory which in total consumes over 1TB, but the subdirectories don't really go above 60GB a piece. The total bar consumes and crushes the rest of the values and makes it not very useful. The option to not show the total row would be great.

(Also thank you for the neat tool! I've been using it nearly daily since I've found it.)

Support --width flag for non-terminal output

I want to show dust output inside some environments where it is not possible to read the terminal width. The simplest example is if piping the output into a pager like less:

These environments normally have variables available that can be passed into the command, so being able to specify something like --width=$COLUMNS would be useful.

Scoop package available in unofficial scoop bucket

Hi. I managed to put together scoop manifest for dust. It is available here in my personal scoop. I would like to promote it to extras bucket after some evaluation period depending on the feedback. Let me know what you think.

Optiong to not color output

It'd be great if there was an option to not color the output, or if automatically detected that the output was not a TTY and skipped coloring automatically. This would help when using dust | less or with things like watch.

Option -V doenst print version

When i execute dust -V only the program name is printed, nor the version.

ignore .git by default

Thinking that this could be configurable but IMO makes sense to ignore .git by default.

Display bug in the way names work

Create two files where one is a substring of the other. The ASCII tree drawing is then broken as you get this ┬ instead of ─:

 1.2G  ├─┬ hanson-1
 1.2G  ├── hanson-13

Suggestion: stick to semantic versioning

Looking through your release history, your versioning scheme is all over the place.

I'm packaging your software on my Centos and Debian repositories and it's easier for me if the version is of a consistent standard, v0.0.0. In these package managers, v4 is newer than your latest release of v0.4.5.0. I would argue that you don't need 4 levels of version numbers either.

Just a suggestion to make things easier for the consumers of your software :)

Output is padded even when bars are turned off

Output is always padded with spaces to the longest filename even when bars are turned off - makes it a bit harder to use dust's output in other programs.

Incorrect calculation in apparent size

dust -s seems to have contradicting output compared to du --apparent-size. Apparent size should usually be less than block size.

I ran dust -sn0 /gnu on the guix package store and have 21G, while du --apparent-size -sh /gnu returns 8.4G. I think dust counts hardlinks twice (since guix uses hardlinks to save space) although I'm not sure.

Panics on windows in cmd.exe: 'Couldn't enable color support: 87'

> dust
thread 'main' panicked at 'Couldn't enable color support: 87', C:\Users\me\.cargo\registry\src\github.com-1ecc6299db9ec823\du-dust-.4.5\src/main.rs:16:5

That's on Win 8.1. On another machine with Win 10 it works.

Make nested bar charts better

So this isn't technically wrong but it looks weird

What is happening here is we have 2 large files and 1 small file deep in a nested tree.
Because we only have a few shads of greyscale in the terminal we 'use' the lighter shades up in the first subdirectory so they aren't available deeper in the tree.

If we start to cd into the directories it looks 'correct'

Cleaner Tree View

What about only show the last leaf of the path per line?

Instead of this:

djin:git/dust> dust
  65M  .
  65M └─┬ ./target
  49M   ├─┬ ./target/debug
  26M   │ ├─┬ ./target/debug/deps
  21M   │ │ └── ./target/debug/deps/libclap-9e6625ac8ff074ad.rlib
  13M   │ ├── ./target/debug/dust
 8.9M   │ └─┬ ./target/debug/incremental
 6.7M   │   ├─┬ ./target/debug/incremental/dust-2748eiei2tcnp
 6.7M   │   │ └─┬ ./target/debug/incremental/dust-2748eiei2tcnp/s-ezd6jnik5u-163pyem-1aab9ncf5glum
 3.0M   │   │   └── ./target/debug/incremental/dust-2748eiei2tcnp/s-ezd6jnik5u-163pyem-1aab9ncf5glum/dep-graph.bin
 2.2M   │   └─┬ ./target/debug/incremental/dust-1dlon65p8m3vl
 2.2M   │     └── ./target/debug/incremental/dust-1dlon65p8m3vl/s-ezd6jncecv-1xsnfd0-4dw9l1r2th2t
  15M   └─┬ ./target/release
 9.2M     ├─┬ ./target/release/deps
 6.7M     │ └── ./target/release/deps/libclap-87bc2534ea57f044.rlib
 5.9M     └── ./target/release/dust

...show this:

djin:git/dust> dust
  65M  .
  65M └─┬ target
  49M   ├─┬ debug
  26M   │ ├─┬ deps
  21M   │ │ └── libclap-9e6625ac8ff074ad.rlib
  13M   │ ├── dust
 8.9M   │ └─┬ incremental
 6.7M   │   ├─┬ dust-2748eiei2tcnp
 6.7M   │   │ └─┬ s-ezd6jnik5u-163pyem-1aab9ncf5glum
 3.0M   │   │   └── dep-graph.bin
 2.2M   │   └─┬ dust-1dlon65p8m3vl
 2.2M   │     └── s-ezd6jncecv-1xsnfd0-4dw9l1r2th2t
  15M   └─┬ release
 9.2M     ├─┬ deps
 6.7M     │ └── libclap-87bc2534ea57f044.rlib
 5.9M     └── dust

Or maybe, show the short version as default and the complete path via an argument.

File walking crates

I'm not sure if these crates will satisfy all the use cases in this application but they may be worth checking out.

Badly wrapped under terminal with short width

It does not take $COLUMNS into account and stuff gets wrapped being hardly readable under terminal less than 100 width. I have no issue with dust shortening the file name if the terminal is short.

1005M   │ ┌─┴ iams-dev-environment

 3.1G   ├─┴ hlai

 616M   │ ┌── student

 994M   │ │ ┌── node_modules

 1.4G   │ ├─┴ apspace

 608M   │ │   ┌── objects

 691M   │ │ ┌─┴ .git

 2.1G   │ │ │   ┌── Components

 2.3G   │ │ │ ┌─┴ stage

 2.3G   │ │ ├─┴ database

 3.6G   │ ├─┴ attendix

 6.7G   ├─┴ apu

I think at the very least, it should support terminal with 80 widths with a fallback even if stuff does not display well, without the terminal wrapped stuff up, it will still look pretty neat.

Use of -d flag should not change calculated sizes

I was giving dust a try and noticed that the use of the -d flag changes the displayed sizes of the directories, whereas I would have expect each directory to be the same size, but with fewer sub-levels displayed. Is this the expected behavior, a bug, or user error?

Here's the output of the dust repo after cloning it, comparing du, dust -bd1, and dust -b (using -b for more readable output below):

$ du -d 1 -h | sort -h
24K	./ci
24K	./.github
52K	./src
68K	./media
84K	./tests
552K	./.git
153M	./target
154M	.

$ cargo run -- -bd1
    Finished dev [unoptimized + debuginfo] target(s) in 0.02s
     Running `target/debug/dust -bd1`
 4.0K   ┌── README.md
 4.0K   ├── Cargo.toml
 4.0K   ├── .gitignore
 8.0K   ├── .github
  12K   ├── LICENSE
  16K   ├── target
  24K   ├── ci
  24K   ├── Cargo.lock
  32K   ├── src
  40K   ├── .git
  52K   ├── tests
  68K   ├── media
 292K ┌─┴ .

$ cargo run -- -b  
    Finished dev [unoptimized + debuginfo] target(s) in 0.03s
     Running `target/debug/dust -b`
 948K             ┌── query-cache.bin
 3.9M             ├── dep-graph.bin
  10M           ┌─┴ s-fqutna58o2-1mf6920-1dfcidj0qrb4
  10M         ┌─┴ dust-1u7onte9p4g5s
  10M       ┌─┴ incremental
 2.8M       │   ┌── build_script_build-95c8841d5eeef07a
 2.8M       │ ┌─┴ log-95c8841d5eeef07a
 2.8M       │ │ ┌── build_script_build-98b6810859c9cb39
 2.8M       │ ├─┴ memchr-98b6810859c9cb39
 3.1M       │ │ ┌── build-script-build
 3.1M       │ ├─┴ bitflags-5ac1d02a416a61b2
 3.2M       │ │ ┌── build_script_build-9091c386f088b04c
 3.2M       │ ├─┴ libc-9091c386f088b04c
 3.5M       │ │ ┌── build-script-build
 3.5M       │ ├─┴ crossbeam-utils-408f2589721c053b
  15M       ├─┴ build
 912K       │ ┌── libtextwrap-1a0030a681f6ac5d.rlib
 1.0M       │ ├── libregex-818c9001a7df9eb5.rmeta
 1.0M       │ ├── libaho_corasick-39d24709df92682f.rmeta
 1.1M       │ ├── libstrsim-c37e1feb7d72bc60.rlib
 1.3M       │ ├── libnum_cpus-7c7ca82d2b6385a6.rlib
 1.4M       │ ├── liblscolors-d48fed5629317cfd.rlib
 1.5M       │ ├── libbstr-8fff9e7d83cfd68c.rlib
 1.6M       │ ├── libwalkdir-d08a403e2772aeb4.rlib
 1.6M       │ ├── libcrossbeam_utils-4e0e4e297bcafc8b.rlib
 1.6M       │ ├── libautocfg-b11cbca6ecbb3954.rlib
 1.7M       │ ├── liblibc-c64d0ff9390f98b5.rmeta
 1.7M       │ ├── libclap-aeb92d621b503b05.rmeta
 1.8M       │ ├── libcrossbeam_channel-7116bcca9ad76a0b.rlib
 2.0M       │ ├── liblibc-c64d0ff9390f98b5.rlib
 4.9M       │ ├── libaho_corasick-39d24709df92682f.rlib
 5.2M       │ ├── libregex_syntax-d37ef5e024ea9658.rmeta
 6.0M       │ ├── libglobset-3998d750ef9972f8.rlib
 8.8M       │ ├── libignore-92c56c21ac8db7b8.rlib
  10M       │ ├── libregex-818c9001a7df9eb5.rlib
  13M       │ ├── libregex_syntax-d37ef5e024ea9658.rlib
  16M       │ ├── libclap-aeb92d621b503b05.rlib
  31M       │ ├── dust-321965f5bb0f1679
 125M       ├─┴ deps
 152M     ┌─┴ debug
 152M   ┌─┴ target
 153M ┌─┴ .

Option to emulate tree's behavior

It would be nice to have an options shortcut (--tree?) that makes dust behave more like tree, by that I mean:

Unsorted (alphabetically sorted?) output
No line-limit
No bars
Optionally: --no-filesizes

bug: behaviour difference on root directory.

on root directory
dust /
behaves differently to
dust

dust / - misses out the top level directories.

This is probably a bug counting the '/'s

Give a sensible error on nonexistent directories

$ ls foo
ls: foo: No such file or directory
$ dust foo
Did not have permissions for all directories
$

This error message is wrong. I'd expect "no such file or directory" instead like ls.

Consider using getattrlistbulk on macOS

DaisyDisk uses this and it is very fast at gather diskspace information.

Creating a dust Chocolatey Packages

Is there any interest in creating a dust chocolatey package (https://chocolatey.org/docs/create-packages)? I think it could be beneficial.

We could base the package on the Windows release artifacts:
https://github.com/bootandy/dust/releases

Incorrect position of folder where prefix is identical to filename

Thanks for the nice work on dust!

I ran into an issue on Ubuntu 18.04 where I have a folder called folder.bag.map and a file called file.bag. The resulting dust output when run in the parent folder is:

user:~/developer/$ dust
  62G ─┬ .
  19G  ├─┬ file.bag
 179M  │ └─┬ folder.bag.map
 119M  │   ├─┬ subfolder1
 119M  │   | └── file1.txt
  59M  │   └─┬ subfolder2
  38M  │     └── file2.txt

i.e. it seems like the file file.bag is treated like a folder and folder.bag.map is treated as a subfolder of the file. Maybe there's a parsing issue here somewhere?

Add an option to skip directories on other filesystems

Consider adding an -x option, which is available in both du and ncdu. The option would exclude any directories that reside on other filesystems. This is useful when you want to find what directories are taking all the space on a particular filesystem, but have network or other disks mounted in other directories.

For example, ncdu -x from the root directory will show this on my system:

ncdu 1.13 ~ Use the arrow keys to navigate, press ? for help
--- / ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
   38.2 GiB [##########] /var
   35.4 GiB [######### ] /home
    4.4 GiB [#         ] /nix
    4.4 GiB [#         ] /usr
...
    4.0 KiB [          ] /mnt
...

With dust it shows:

$ dust -d1
 277G ─┬ .
 192G  ├── mnt
  40G  ├── var
  35G  ├── home
 4.4G  ├── nix
 4.4G  ├── usr
...

/mnt has a CIFS shared mounted from a NAS, so it throws off the totals.

#46 is related, but would be less convenient to use if there were other filesystems mounted in various locations.

Fail to install on Windows

Versions:

rustc 1.43.0-nightly (c9290dcee 2020-02-04)

Error message:

 $ cargo install du-dust
    Updating crates.io index
  Downloaded du-dust v0.4.4
  Downloaded 1 crate (21.7 KB) in 3.21s
  Installing du-dust v0.4.4
   Compiling autocfg v0.1.7
   Compiling semver-parser v0.7.0
   Compiling lazy_static v1.4.0
   Compiling cfg-if v0.1.10
   Compiling libc v0.2.66
   Compiling scopeguard v1.0.0
   Compiling winapi v0.3.8
   Compiling rayon-core v1.7.0
   Compiling bitflags v1.2.1
   Compiling either v1.5.3
   Compiling unicode-width v0.1.7
   Compiling vec_map v0.8.1
   Compiling strsim v0.8.0
   Compiling semver v0.9.0
   Compiling textwrap v0.11.0
   Compiling rustc_version v0.2.3
   Compiling num_cpus v1.12.0
   Compiling crossbeam-utils v0.7.0
   Compiling crossbeam-epoch v0.8.0
   Compiling memoffset v0.5.3
   Compiling crossbeam-queue v0.2.1
   Compiling crossbeam-channel v0.4.0
   Compiling crossbeam-deque v0.7.2
   Compiling crossbeam v0.7.3
   Compiling rayon v1.3.0
   Compiling atty v0.2.14
   Compiling ansi_term v0.12.1
   Compiling clap v2.33.0
   Compiling jwalk v0.4.0
   Compiling du-dust v0.4.4
error[E0425]: cannot find function `get_filesystem` in this scope
   --> C:\Users\Winterreise\.cargo\registry\src\github.com-1ecc6299db9ec823\du-dust-0.4.4\src\utils\mod.rs:103:24
    |
103 |         if let Ok(a) = get_filesystem(file_name) {
    |                        ^^^^^^^^^^^^^^ not found in this scope

error[E0658]: use of unstable library feature 'windows_by_handle'
  --> C:\Users\Winterreise\.cargo\registry\src\github.com-1ecc6299db9ec823\du-dust-0.4.4\src\utils\platform.rs:29:52
   |
29 |         let windows_equivalent_of_inode = Some((md.file_index(), md.volume_serial_number()));
   |                                                    ^^^^^^^^^^
   |
   = note: for more information, see https://github.com/rust-lang/rust/issues/63010
   = help: add `#![feature(windows_by_handle)]` to the crate attributes to enable

error[E0658]: use of unstable library feature 'windows_by_handle'
  --> C:\Users\Winterreise\.cargo\registry\src\github.com-1ecc6299db9ec823\du-dust-0.4.4\src\utils\platform.rs:29:69
   |
29 |         let windows_equivalent_of_inode = Some((md.file_index(), md.volume_serial_number()));
   |                                                                     ^^^^^^^^^^^^^^^^^^^^
   |
   = note: for more information, see https://github.com/rust-lang/rust/issues/63010
   = help: add `#![feature(windows_by_handle)]` to the crate attributes to enable

error[E0308]: mismatched types
  --> C:\Users\Winterreise\.cargo\registry\src\github.com-1ecc6299db9ec823\du-dust-0.4.4\src\utils\platform.rs:28:5
   |
26 |   pub fn get_metadata(d: &DirEntry, use_apparent_size: bool) -> Option<(u64, Option<(u64, u64)>)> {        |                                                                 --------------------------------- expected `std::option::Option<(u64, std::option::Option<(u64, u64)>)>` because of return type
27 |       use std::os::windows::fs::MetadataExt;
28 | /     d.metadata.as_ref().unwrap().as_ref().ok().map(|md| {
29 | |         let windows_equivalent_of_inode = Some((md.file_index(), md.volume_serial_number()));
30 | |         (md.file_size(), windows_equivalent_of_inode)
31 | |     })
   | |______^ expected `u64`, found enum `std::option::Option`
   |
   = note: expected enum `std::option::Option<(_, std::option::Option<(u64, u64)>)>`
              found enum `std::option::Option<(_, std::option::Option<(std::option::Option<u64>, std::option::Option<u32>)>)>`

error[E0658]: use of unstable library feature 'windows_by_handle'
  --> C:\Users\Winterreise\.cargo\registry\src\github.com-1ecc6299db9ec823\du-dust-0.4.4\src\utils\platform.rs:55:17
   |
55 |     Ok(metadata.volume_serial_number())
   |                 ^^^^^^^^^^^^^^^^^^^^
   |
   = note: for more information, see https://github.com/rust-lang/rust/issues/63010
   = help: add `#![feature(windows_by_handle)]` to the crate attributes to enable

error[E0308]: mismatched types
  --> C:\Users\Winterreise\.cargo\registry\src\github.com-1ecc6299db9ec823\du-dust-0.4.4\src\utils\platform.rs:55:8
   |
55 |     Ok(metadata.volume_serial_number())
   |        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected `u64`, found enum `std::option::Option`
   |
   = note: expected type `u64`
              found enum `std::option::Option<u32>`

error: aborting due to 6 previous errors

Some errors have detailed explanations: E0308, E0425, E0658.
For more information about an error, try `rustc --explain E0308`.
error: failed to compile `du-dust v0.4.4`, intermediate artifacts can be found at `C:\Users\WINTER~1\AppData\Local\Temp\cargo-installTgeiQE`

Caused by:
  could not compile `du-dust`.

To learn more, run the command again with --verbose.

Thanks!

man page

Awesome work!

Any chance of adding a man page?
Maybe something like ripgrep has using asciidoc?
https://github.com/BurntSushi/ripgrep/blob/7cbc535d70a53c81dfa3e58552c01f21c2e38d28/build.rs#L66

❯ man dust
No manual entry for dust

I see dust --help works, but it'd be handy to have a man page too, since it's many peoples first instinct when trying a new cli tool.

no rlwrap or pixie-vm error after installing with nix

This is using a nixos-unstable pin from the past few days.

Here's a nix-shell command with that exact pin for you to reproduce:

$ nix-shell -I https://github.com/NixOS/nixpkgs-channels/archive/61525137fd1002f6f2a5eb0ea27d480713362cd5.tar.gz -p dust --pure --run dust
these paths will be fetched (11.28 MiB download, 143.87 MiB unpacked):
  /nix/store/a9p65g3q268clb8cbk1hhkm0yjki60dn-dust-0-91
  /nix/store/c30p181sp4jy5dr8lswgl431lyvfjmxz-pixie-0-r1364-jit
  /nix/store/k8bpj097am7lyasaamassdx6rjpxl98j-boost-1.69.0
  /nix/store/w3gv4wmbp0nwb69zgxairdw7rq9060nr-boost-1.69.0-dev
copying path '/nix/store/k8bpj097am7lyasaamassdx6rjpxl98j-boost-1.69.0' from 'https://cache.nixos.org'...
copying path '/nix/store/w3gv4wmbp0nwb69zgxairdw7rq9060nr-boost-1.69.0-dev' from 'https://cache.nixos.org'...
copying path '/nix/store/c30p181sp4jy5dr8lswgl431lyvfjmxz-pixie-0-r1364-jit' from 'https://cache.nixos.org'...
copying path '/nix/store/a9p65g3q268clb8cbk1hhkm0yjki60dn-dust-0-91' from 'https://cache.nixos.org'...
/nix/store/a9p65g3q268clb8cbk1hhkm0yjki60dn-dust-0-91/bin/dust: line 26: which: command not found
/nix/store/a9p65g3q268clb8cbk1hhkm0yjki60dn-dust-0-91/bin/dust: line 29: /nix/store/c30p181sp4jy5dr8lswgl431lyvfjmxz-pixie-0-r1364-jit/bin/pixie-vm: No such file or directory

Validate input parameters

dust -n STR
Crashes.

dust -d ASDF
Should warn instead of silence

Readme: sort -h can sort human-readable numbers

The following part in the README.md isn't quite correct

[du -b] for when you have a bunch and need to sort the output numerically, and you’re obligated to either further pass it into something like awk to turn bytes into the appropriate human-friendly unit like mega or gigabytes, or you just do some rough math in your head and use the ordering to sanity check.

Given that sort has a -h flag for sorting human-readable numbers correctly, that part is a bit misleading.

Edit: I just noticed that sort -h is already given as an alternative at the bottom. Changed comment text.

dust hangs on a single-core computer

Even on an empty directory, dust hangs:

$ mkdir empty
$ dust empty
^C

It works fine if I specify the number of threads:

$ dust -t1 empty
 4.0K ─┬ empty

It works fine on all my other boxes, so I'm guessing it's the fact that this one has only 1 core that is to blame. Here's a stack trace:

Thread 1 "dust" received signal SIGINT, Interrupt.
0x00007ffff7e9562b in sched_yield () from /usr/lib/libc.so.6
(gdb) bt
#0  0x00007ffff7e9562b in sched_yield () from /usr/lib/libc.so.6
#1  0x00005555555ff045 in jwalk::core::ordered_queue::OrderedQueueIter<T>::next_strict (self=0x7fffffff6d38) at /home/tavianator/.cargo/registry/src/github.com-1ecc6299db9ec823/jwalk-0.4.0/src/core/ordered_queue.rs:150
#2  0x00005555555ff817 in <jwalk::core::ordered_queue::OrderedQueueIter<T> as core::iter::traits::iterator::Iterator>::next (self=0x7fffffff6d38) at /home/tavianator/.cargo/registry/src/github.com-1ecc6299db9ec823/jwalk-0.4.0/src/core/ordered_queue.rs:170
#3  0x0000555555618ec0 in <jwalk::core::iterators::ReadDirIter as core::iter::traits::iterator::Iterator>::next (self=0x7fffffff6d30) at /home/tavianator/.cargo/registry/src/github.com-1ecc6299db9ec823/jwalk-0.4.0/src/core/iterators.rs:40
#4  0x00005555555cd3b5 in <core::iter::adapters::Peekable<I> as core::iter::traits::iterator::Iterator>::next (self=0x7fffffff6d30) at /build/rust/src/rustc-1.40.0-src/src/libcore/iter/adapters/mod.rs:1260
#5  0x00005555556194a6 in jwalk::core::iterators::DirEntryIter::push_next_read_dir_iter (self=0x7fffffff6d18) at /home/tavianator/.cargo/registry/src/github.com-1ecc6299db9ec823/jwalk-0.4.0/src/core/iterators.rs:65
#6  0x0000555555619984 in <jwalk::core::iterators::DirEntryIter as core::iter::traits::iterator::Iterator>::next (self=0x7fffffff6d18) at /home/tavianator/.cargo/registry/src/github.com-1ecc6299db9ec823/jwalk-0.4.0/src/core/iterators.rs:96
#7  0x000055555559cf38 in dust::utils::examine_dir (top_dir=..., apparent_size=false, inodes=0x7fffffff7638, data=0x7fffffff7670, file_count_no_permission=0x7fffffff7630, threads=...) at src/utils/mod.rs:112
#8  0x000055555559cbc4 in dust::utils::get_dir_tree (top_level_names=0x7fffffffdd58, apparent_size=false, threads=...) at src/utils/mod.rs:79
#9  0x0000555555590fdb in dust::main () at src/main.rs:115
(gdb) thread 2
[Switching to thread 2 (Thread 0x7ffff7dac700 (LWP 4624))]
#0  0x00007ffff7e9562b in sched_yield () from /usr/lib/libc.so.6
(gdb) bt
#0  0x00007ffff7e9562b in sched_yield () from /usr/lib/libc.so.6
#1  0x00005555555fe9ad in jwalk::core::ordered_queue::OrderedQueueIter<T>::next_relaxed (self=0x7ffff7da9960) at /home/tavianator/.cargo/registry/src/github.com-1ecc6299db9ec823/jwalk-0.4.0/src/core/ordered_queue.rs:122
#2  0x00005555555ff7cb in <jwalk::core::ordered_queue::OrderedQueueIter<T> as core::iter::traits::iterator::Iterator>::next (self=0x7ffff7da9960) at /home/tavianator/.cargo/registry/src/github.com-1ecc6299db9ec823/jwalk-0.4.0/src/core/ordered_queue.rs:169
#3  0x000055555560ab64 in <rayon::iter::par_bridge::IterParallelProducer<Iter> as rayon::iter::plumbing::UnindexedProducer>::fold_with (self=..., folder=...) at /home/tavianator/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-1.3.0/src/iter/par_bridge.rs:177
#4  0x00005555555e4184 in rayon::iter::plumbing::bridge_unindexed_producer_consumer (migrated=false, splitter=..., producer=..., consumer=...) at /home/tavianator/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-1.3.0/src/iter/plumbing/mod.rs:482
#5  0x00005555555e468a in rayon::iter::plumbing::bridge_unindexed_producer_consumer::{{closure}} (context=...) at /home/tavianator/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-1.3.0/src/iter/plumbing/mod.rs:474
#6  0x00005555555f9abe in rayon_core::join::join_context::call_a::{{closure}} () at /home/tavianator/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.7.0/src/join/mod.rs:125
#7  0x00005555555d8a36 in <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once (self=..., _args=()) at /build/rust/src/rustc-1.40.0-src/src/libstd/panic.rs:317
#8  0x00005555555e30d1 in std::panicking::try::do_call (data=0x7ffff7da7e20 "\250\223\332\367\377\177\000") at /build/rust/src/rustc-1.40.0-src/src/libstd/panicking.rs:287
#9  0x000055555581463a in __rust_maybe_catch_panic ()
#10 0x00005555555e2778 in std::panicking::try (f=...) at /build/rust/src/rustc-1.40.0-src/src/libstd/panicking.rs:265
#11 0x00005555555d9bc6 in std::panic::catch_unwind (f=...) at /build/rust/src/rustc-1.40.0-src/src/libstd/panic.rs:396
#12 0x00005555555ffe23 in rayon_core::unwind::halt_unwinding (func=...) at /home/tavianator/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.7.0/src/unwind.rs:17
#13 0x00005555555f91df in rayon_core::join::join_context::{{closure}} (worker_thread=0x7ffff7dab100, injected=false) at /home/tavianator/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.7.0/src/join/mod.rs:146
#14 0x000055555560a2ff in rayon_core::registry::in_worker (op=...) at /home/tavianator/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.7.0/src/registry.rs:799
#15 0x00005555555f8ea7 in rayon_core::join::join_context (oper_a=..., oper_b=...) at /home/tavianator/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.7.0/src/join/mod.rs:133
#16 0x00005555555e3f49 in rayon::iter::plumbing::bridge_unindexed_producer_consumer (migrated=false, splitter=..., producer=..., consumer=...) at /home/tavianator/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-1.3.0/src/iter/plumbing/mod.rs:473
#17 0x00005555555e3804 in rayon::iter::plumbing::bridge_unindexed (producer=..., consumer=...) at /home/tavianator/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-1.3.0/src/iter/plumbing/mod.rs:452
#18 0x000055555560f44d in <rayon::iter::par_bridge::IterBridge<Iter> as rayon::iter::ParallelIterator>::drive_unindexed (self=..., consumer=...) at /home/tavianator/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-1.3.0/src/iter/par_bridge.rs:87
#19 0x00005555555f6d20 in <rayon::iter::map_with::MapWith<I,T,F> as rayon::iter::ParallelIterator>::drive_unindexed (self=..., consumer=...) at /home/tavianator/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-1.3.0/src/iter/map_with.rs:53
#20 0x00005555555f63c7 in rayon::iter::from_par_iter::<impl rayon::iter::FromParallelIterator<()> for ()>::from_par_iter (par_iter=...) at /home/tavianator/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-1.3.0/src/iter/from_par_iter.rs:226
#21 0x00005555555f6404 in rayon::iter::ParallelIterator::collect (self=...) at /home/tavianator/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-1.3.0/src/iter/mod.rs:1886
#22 0x000055555560ee79 in rayon::iter::ParallelIterator::for_each_with (self=..., init=..., op=...) at /home/tavianator/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-1.3.0/src/iter/mod.rs:393
#23 0x000055555561b334 in jwalk::core::multi_threaded_walk::{{closure}} () at /home/tavianator/.cargo/registry/src/github.com-1ecc6299db9ec823/jwalk-0.4.0/src/core/mod.rs:142
#24 0x00005555555d8ab5 in <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once (self=..., _args=()) at /build/rust/src/rustc-1.40.0-src/src/libstd/panic.rs:317
#25 0x00005555555e2ff1 in std::panicking::try::do_call (data=0x7ffff7daa800 "Њ\217UUU\000") at /build/rust/src/rustc-1.40.0-src/src/libstd/panicking.rs:287
#26 0x000055555581463a in __rust_maybe_catch_panic ()
#27 0x00005555555e2dd8 in std::panicking::try (f=...) at /build/rust/src/rustc-1.40.0-src/src/libstd/panicking.rs:265
#28 0x00005555555d9c26 in std::panic::catch_unwind (f=...) at /build/rust/src/rustc-1.40.0-src/src/libstd/panic.rs:396
#29 0x00005555555ffea3 in rayon_core::unwind::halt_unwinding (func=...) at /home/tavianator/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.7.0/src/unwind.rs:17
#30 0x00005555555fa018 in rayon_core::spawn::spawn_job::{{closure}} () at /home/tavianator/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.7.0/src/spawn/mod.rs:98
#31 0x00005555555fcb98 in <rayon_core::job::HeapJob<BODY> as rayon_core::job::Job>::execute (this=0x5555558f80b0) at /home/tavianator/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.7.0/src/job.rs:167
#32 0x0000555555631d06 in rayon_core::job::JobRef::execute (self=0x7ffff7daadc0) at /home/tavianator/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.7.0/src/job.rs:59
#33 0x00005555556233fd in rayon_core::registry::WorkerThread::execute (self=0x7ffff7dab100, job=...) at /home/tavianator/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.7.0/src/registry.rs:681
#34 0x0000555555622d1c in rayon_core::registry::WorkerThread::wait_until_cold (self=0x7ffff7dab100, latch=0x5555558f7cf0) at /home/tavianator/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.7.0/src/registry.rs:665
#35 0x0000555555622b16 in rayon_core::registry::WorkerThread::wait_until (self=0x7ffff7dab100, latch=0x5555558f7cf0) at /home/tavianator/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.7.0/src/registry.rs:639
#36 0x0000555555623c78 in rayon_core::registry::main_loop (worker=..., registry=..., index=0) at /home/tavianator/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.7.0/src/registry.rs:759
#37 0x00005555556206c0 in rayon_core::registry::ThreadBuilder::run (self=...) at /home/tavianator/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.7.0/src/registry.rs:56
#38 0x0000555555620d01 in <rayon_core::registry::DefaultSpawn as rayon_core::registry::ThreadSpawn>::spawn::{{closure}} () at /home/tavianator/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.7.0/src/registry.rs:101
#39 0x000055555562c652 in std::sys_common::backtrace::__rust_begin_short_backtrace (f=...) at /build/rust/src/rustc-1.40.0-src/src/libstd/sys_common/backtrace.rs:129
#40 0x00005555556478c1 in std::thread::Builder::spawn_unchecked::{{closure}}::{{closure}} () at /build/rust/src/rustc-1.40.0-src/src/libstd/thread/mod.rs:469
#41 0x00005555556246c1 in <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once (self=..., _args=()) at /build/rust/src/rustc-1.40.0-src/src/libstd/panic.rs:317
#42 0x000055555564122e in std::panicking::try::do_call (data=0x7ffff7dab900 "\000") at /build/rust/src/rustc-1.40.0-src/src/libstd/panicking.rs:287
#43 0x000055555581463a in __rust_maybe_catch_panic ()
#44 0x0000555555640ea8 in std::panicking::try (f=...) at /build/rust/src/rustc-1.40.0-src/src/libstd/panicking.rs:265
#45 0x0000555555625683 in std::panic::catch_unwind (f=...) at /build/rust/src/rustc-1.40.0-src/src/libstd/panic.rs:396
#46 0x00005555556476a6 in std::thread::Builder::spawn_unchecked::{{closure}} () at /build/rust/src/rustc-1.40.0-src/src/libstd/thread/mod.rs:468
#47 0x000055555562cdf4 in core::ops::function::FnOnce::call_once{{vtable-shim}} () at /build/rust/src/rustc-1.40.0-src/src/libcore/ops/function.rs:227
#48 0x000055555580699f in <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once ()
#49 0x0000555555813b10 in std::sys::unix::thread::Thread::new::thread_start ()
#50 0x00007ffff7f994cf in start_thread () from /usr/lib/libpthread.so.0
#51 0x00007ffff7eae2d3 in clone () from /usr/lib/libc.so.6

Add flag to limit depth

From Reddit:

Once thing I notice however is that you elected for limiting the number of lines rather then the depth of the tree. It would be a lot more useful to specify the depth of the tree like the way tree -L 3 works.

Not yet sure how this would work. Would "dust -d 2" show:

All folders of depth 2 and below
Show the biggest 15 but not show any subdirectories that were deeper than 2.

I imagine the (1) is correct

Doesn't seem to work on Windows 8.1

It doesn't seem to work on Windows 8.1:

> dust
←[38;5;196m 597M←[0m ─┬ .

I get the same output, no matter which flags I pass. Any idea why?

Add dutree to list of alternatives

https://github.com/nachoparker/dutree

Commit bdc3d40 breaks windows builds

v0.4.3 builds and installs, but the changes in bdc3d40 break windows compilation (even when using 'nightly').

I just noticed when updating to the most recent version (v0.4.4).

I've looked through the code and it looks like using the unstable metadata.file_index() and metadata.volume_serial_number() as well as missing get_filesystem() are the culprits.

Here's the build output:

C:>cargo +nightly build
   Compiling du-dust v0.4.3 (C:\Users\Roy\OneDrive\Projects\rust\rs.dust)
error[E0425]: cannot find function `get_filesystem` in this scope
   --> src\utils\mod.rs:102:26
    |
102 |         if let Some(a) = get_filesystem(file_name) {
    |                          ^^^^^^^^^^^^^^ not found in this scope

error[E0658]: use of unstable library feature 'windows_by_handle'
  --> src\utils\platform.rs:28:52
   |
28 |         let windows_equivalent_of_inode = Some((md.file_index(), md.volume_serial_number()));
   |                                                    ^^^^^^^^^^
   |
   = note: for more information, see https://github.com/rust-lang/rust/issues/63010
   = help: add `#![feature(windows_by_handle)]` to the crate attributes to enable

error[E0658]: use of unstable library feature 'windows_by_handle'
  --> src\utils\platform.rs:28:69
   |
28 |         let windows_equivalent_of_inode = Some((md.file_index(), md.volume_serial_number()));
   |                                                                     ^^^^^^^^^^^^^^^^^^^^
   |
   = note: for more information, see https://github.com/rust-lang/rust/issues/63010
   = help: add `#![feature(windows_by_handle)]` to the crate attributes to enable

error[E0308]: mismatched types
  --> src\utils\platform.rs:27:5
   |
25 |   pub fn get_metadata(d: &DirEntry, use_apparent_size: bool) -> Option<(u64, Option<(u64, u64)>)> {
   |                                                                 --------------------------------- expected `std::option::Option<(u64, std::option::Option<(u64, u64)>)>` because of return type
26 |       use std::os::windows::fs::MetadataExt;
27 | /     d.metadata.as_ref().unwrap().as_ref().ok().map(|md| {
28 | |         let windows_equivalent_of_inode = Some((md.file_index(), md.volume_serial_number()));
29 | |         (md.file_size(), windows_equivalent_of_inode)
30 | |     })
   | |______^ expected `u64`, found enum `std::option::Option`
   |
   = note: expected enum `std::option::Option<(_, std::option::Option<(u64, u64)>)>`
              found enum `std::option::Option<(_, std::option::Option<(std::option::Option<u64>, std::option::Option<u32>)>)>`

error[E0658]: use of unstable library feature 'windows_by_handle'
  --> src\utils\platform.rs:54:19
   |
54 |     Some(metadata.volume_serial_number())
   |                   ^^^^^^^^^^^^^^^^^^^^
   |
   = note: for more information, see https://github.com/rust-lang/rust/issues/63010
   = help: add `#![feature(windows_by_handle)]` to the crate attributes to enable

error[E0308]: mismatched types
  --> src\utils\platform.rs:54:10
   |
54 |     Some(metadata.volume_serial_number())
   |          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected `u64`, found enum `std::option::Option`
   |
   = note: expected type `u64`
              found enum `std::option::Option<u32>`

error: aborting due to 6 previous errors

Some errors have detailed explanations: E0308, E0425, E0658.
For more information about an error, try `rustc --explain E0308`.
error: could not compile `du-dust`.

To learn more, run the command again with --verbose.

performance notes:

try using: https://github.com/servo/rust-fnv

try minimizing string creation and pathbuf creation