ossobv / dutree Goto Github PK

View Code? Open in Web Editor NEW

10.0 6.0 1.0 77 KB

dutree (python lib/cli) helps quickly find which files/dirs are taking up space. Compare it with a summary of ncdu.

License: GNU General Public License v3.0

Python 100.00%

cli library disk-usage

dutree's People

Contributors

Stargazers

Watchers

Forkers

leonzhao2020

dutree's Issues

dutree dies if EPERM is encountered

PermissionError: [Errno 13] Permission denied: '/./sys/kernel/debug'

dutree attempts to read special files like /proc/kcore

$ sudo python dutree/dutree.py / 
dutree/dutree.py:262: UserWarning: [Errno 2] No such file or directory: '/proc/14532/task/14532/fd/3'
  warnings.warn(str(e))
dutree/dutree.py:262: UserWarning: [Errno 2] No such file or directory: '/proc/14532/task/14532/fdinfo/3'
  warnings.warn(str(e))
dutree/dutree.py:262: UserWarning: [Errno 2] No such file or directory: '/proc/14532/fd/3'
  warnings.warn(str(e))
dutree/dutree.py:262: UserWarning: [Errno 2] No such file or directory: '/proc/14532/fdinfo/3'
  warnings.warn(str(e))
dutree/dutree.py:262: UserWarning: [Errno 13] Permission denied: '/run/user/1000/gvfs'
  warnings.warn(str(e))
 128.0 T  /proc/kcore
  17.0 G  /*
   -----
 128.0 T  TOTAL (140755721649795)

Should we blacklist /proc? Should we stay on the same filesystem? We might be able to use st_dev or st_rdev for this.

RFE: count block sizes as well and make the grouping optional (per apparent vs. per real size)

In the PlanB backup solution we bill the real (LZ4 compressed) size, but we list the apparent size. This can be confusing:

backing up 120GB of sql shows 30GB of usage
the listing shows 120GB: the user may think that lz4-ing the file will win him, but won't gain him anything

We should sum both and list both. Make the grouping optionally by apparent vs. by real size.

dutree scan of root dir shows "/./"

We should undo that ugliness.

UserWarning default output is 2 (long) lines

/usr/local/lib/python3.5/dist-packages/dutree/dutree.py:291: UserWarning: [Errno 13] Permission denied: '/dev/.lxd-mounts'
  warnings.warn(str(e))

Change to:

dutree.py:291: UserWarning: [Errno 13] Permission denied: '/dev/.lxd-mounts'

listdir can also give EPERM, also warn and allow..

    def _scan(self, path, parent_node):
        fraction = self._subtotal // 20  # initialize fraction
        children = []               # large separate child nodes
        mixed_total = 0             # "rest of the dir", add to this node

        try:
            files = listdir(path or '/')
        except OSError:   # <--- here
            # warn, and /sys/kernel/debug
            files = []

        for file_ in files:

Sometimes *-dirs are leftover without any any other dir

 755.0 M  invoice/renderedinvoices/2012/
   1.1 G  invoice/renderedinvoices/2013/
   1.5 G  invoice/renderedinvoices/2014/
   1.9 G  invoice/renderedinvoices/2015/*
   3.2 G  invoice/renderedinvoices/2016/
   4.2 G  invoice/renderedinvoices/2017/*
   1.1 G  invoice/renderedinvoices/*
 801.9 M  invoice/*
   -----
  14.5 G  TOTAL (15551665153)

--help does not work (and --count-blocks is awkward)

See title.

--xdev option needed to skip going to other filesystems

And maybe also something that skips certain special filesystems: proc and sys?

(This will require adding a bit of argparse.)

Fetching first/last mtime over a range is a good idea

If you want to find old/new versions of files, it is nice if we can show the min/max mtime for resultsets.

dutree dies if a file is removed during scan race

# dutree /somewhere
Traceback (most recent call last):
  File "/usr/local/bin/dutree", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python2.7/dist-packages/dutree/dutree.py", line 331, in main
    tree = scanner.scan()
  File "/usr/local/lib/python2.7/dist-packages/dutree/dutree.py", line 230, in scan
    self._tree, self._path, 0)
  File "/usr/local/lib/python2.7/dist-packages/dutree/dutree.py", line 248, in _scan
    st = lstat(fn)
OSError: [Errno 2] No such file or directory: '/somewhere/.pytree.py.4.swp'

Does not work on Windows

Throws

AttributeError: 'os.stat_result' object has no attribute 'st_blocks'

instead