dankamongmen / growlight Goto Github PK

View Code? Open in Web Editor NEW

85.0 8.0 12.0 5.08 MB

notcurses block device manager / system installation tool

Home Page: https://nick-black.com/dankwiki/index.php/Growlight

License: GNU General Public License v3.0

C 97.61% CMake 1.16% Shell 0.32% C++ 0.91%

notcurses blockdevice filesystem readline

growlight's Introduction

growlight by nick black ([email protected])

Block device manager and system installation tool.

https://nick-black.com/dankwiki/index.php/Growlight

Dependencies:

libatasmart 0.19+
libblkid 2.20.1
libcap 2.24+
libcryptsetup 2.1.5+
libdevmapper 1.02.74+
libnettle 3.5.1+
libnotcurses 3.0.5+
libpci 3.1.9+
libpciaccess 0.13.1+
libudev 175+
libz 1.2.11+
mkswap(8) from util-linux
badblocks(8), mkfs.ext4(8), mkfs.ext3(8), mkfs.ext2(8) from e2fsprogs

Kernel options:

CONFIG_DM_CRYPT (for device mapper encrypt aka LUKS)
CONFIG_MD_RAID* (for MDRAID)
CONFIG_MSDOS_PARTITION (for msdos partition tables)
CONFIG_EFI_PARTITION (for GPT partition tables) ... almost certainly more

Build-only dependencies:

pkg-config 0.29+
cmake 3.14+
pandoc 2.9.2.1+ (if building man pages)
doctest 2.3.5+ (if building unit tests)

Building:

mkdir build && cd build
cmake ..
make
(optionally) make check

User's guide

In almost all cases, growlight needs to be run as root. It will attempt to start otherwise, but will generally be unable to discover or manipulate disks. You'll definitely need at least CAP_SYS_RAWIO and CAP_SYS_ADMIN.

Help can be found by pressing 'H' or 'F1' in growlight, or running help in growlight-readline.

growlight's first action is to install inotify watches in several directories, and then enumerate the current devices by walking same (/sys/class/block, etc.). This way, it immediately learns of devices added or removed after startup. growlight discovers block devices via these directories, and through those block devices finds controllers. Controllers which do not have block devices attaches will thus not generally be found (growlight will remain aware of an adapter from which all devices are removed while it's running).

The highest level of structure in growlight is the controller ("controller" and "adapter" are used interchangeably in growlight). A virtual controller is also defined, to collect various virtual devices (especially aggregates). In the fullscreen view, controllers are boxes labeled by their type, bus path, and bandwidth. Below, we see a machine with one SATA SSD, a dmcrypt device mapper block built atop that, and an unloaded SD card reader hanging off USB 3.0:

Navigate among the adapters using PgUp and PgDn. Bring up the details subscreen with v to see full details about the adapter (along with other information). Within an adapter, up and down moves between block devices, and left and right move between partitions. Vi keys are also supported.

In the readline mode, adapters are listed via the adapter command (-v can be provided to adapter for full details of attached devices and filesystems):

[growlight](0)> adapter
[ahci-0] Southbridge device 0000:00.17.0
 Intel Corporation Sunrise Point-LP SATA Controller [AHCI mode]
Virtual devices
[xhci_pci-0] Southbridge device 0000:00.14.0
 Intel Corporation Sunrise Point-LP USB 3.0 xHCI Controller
[growlight](0)>

growlight's People

Contributors

Stargazers

Watchers

Forkers

dorlaor amlouden acidburn0zzz dustins conrad784 cutff jawar1mx isrsanti dawnadvent mreff555 cschech developers81828182

growlight's Issues

Add functionality to engage sg identification (LEDs)

u/tracernz of reddit's r/datahoarders requests cage LED illumination functionality. This is a particular implementation of SCSI Generic Identification, I believe. Figure out a U/I for it, and expose the functionality, along with other relevant parts of the SG ioctls.

format-truncation warnings with gcc8+

While we're doing work here, let's finally handle these:

src/growlight.h:457:28: warning: '%02ju' directive output may be truncated writing between 2 and 17 bytes into a region of size between 0 and 8 [-Wformat-truncation=]
    snprintf(buf,bsize,"%ju.%02ju%c%c",val / dv,(val % dv) / ((dv + 99) / 100),
                            ^~~~~
src/growlight.h:457:23: note: directive argument in the range [0, 18014398509481982]
    snprintf(buf,bsize,"%ju.%02ju%c%c",val / dv,(val % dv) / ((dv + 99) / 100),
                       ^~~~~~~~~~~~~~~
src/growlight.h:457:4: note: 'snprintf' output between 7 and 41 bytes into a destination of size 10
    snprintf(buf,bsize,"%ju.%02ju%c%c",val / dv,(val % dv) / ((dv + 99) / 100),
    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      prefixes[consumed - 1],uprefix);
      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/growlight.h:457:28: warning: '%02ju' directive output may be truncated writing between 2 and 17 bytes into a region of size between 0 and 8 [-Wformat-truncation=]
    snprintf(buf,bsize,"%ju.%02ju%c%c",val / dv,(val % dv) / ((dv + 99) / 100),
                            ^~~~~
src/growlight.h:457:23: note: directive argument in the range [0, 18014398509481982]
    snprintf(buf,bsize,"%ju.%02ju%c%c",val / dv,(val % dv) / ((dv + 99) / 100),
                       ^~~~~~~~~~~~~~~
src/growlight.h:457:4: note: 'snprintf' output between 7 and 41 bytes into a destination of size 10
    snprintf(buf,bsize,"%ju.%02ju%c%c",val / dv,(val % dv) / ((dv + 99) / 100),
    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      prefixes[consumed - 1],uprefix);

I think explicit length controls would suffice, and also help to document the code.

starting woeusb while growlight is up leads to segfault

on grimes, i started up woeusb to put windows10 on my samsung flash usb. boom! down comes growlight. starting it during this operation also frequently crashed.

Thread 41 (Thread 0x7fffedffb700 (LWP 94034)):
#0  0x0000555555576251 in print_dev (bo=0x7fffd0000e60, line=10, rows=<optimized out>, cols=80, topp=0, endp=0, rb=<optimized out>, rb=<optimized out>) at src/ncurses.c:1445
#1  0x000055555557860b in print_adapter_devs (endp=0, topp=0, cols=80, rows=15, as=0x7ffff0000e60) at src/ncurses.c:1487
#2  redraw_adapter (rb=<optimized out>) at src/ncurses.c:1620
#3  0x000055555557df4f in block_callback (d=<optimized out>, v=<optimized out>) at src/ncurses.c:6876
#4  0x00005555555602fd in rescan (d=0x7fffd0000b60, name=<optimized out>) at src/growlight.c:1133
#5  rescan (name=<optimized out>, d=<optimized out>) at src/growlight.c:915
#6  0x0000555555563a29 in rescan_device (name=0x7fffc0005320 "loop0") at src/growlight.c:2246

#7  0x000055555556ba90 in udev_event (gui=0x7fffffffe490) at src/udev.c:29
#8  0x00005555555634bc in event_posix_thread (unsafe=0x555555653850) at src/growlight.c:1658
#9  0x00007ffff780757f in start_thread () from /usr/lib/libpthread.so.0
#10 0x00007ffff77350e3 in clone () from /usr/lib/libc.so.6

Thread 1 (Thread 0x7ffff749e780 (LWP 93992)):
#0  0x00007ffff7810a6c in read () from /usr/lib/libpthread.so.0
#1  0x00007ffff7f134f0 in _nc_wgetch () from /usr/lib/libncursesw.so.6
#2  0x00007ffff7f13f29 in wgetch () from /usr/lib/libncursesw.so.6
#3  0x000055555555c1c4 in handle_ncurses_input (w=0x5555555e73e0) at src/ncurses.c:6280
#4  main (argc=<optimized out>, argv=<optimized out>) at src/ncurses.c:7118
(gdb)

growlight tries to read a bunch of nonsense /sys/class/block entries

At startup, I see the following:

Couldn't read link at /sys/class/block/securityfs (No such file or directory)
Couldn't read link at /sys/class/block/cgroup2 (No such file or directory)
Couldn't read link at /sys/class/block/cgroup (No such file or directory)
Couldn't read link at /sys/class/block/pstore (No such file or directory)
Couldn't read link at /sys/class/block/efivarfs (No such file or directory)
Couldn't read link at /sys/class/block/bpf (No such file or directory)
Couldn't read link at /sys/class/block/cgroup (No such file or directory)
Couldn't read link at /sys/class/block/cgroup (No such file or directory)
Couldn't read link at /sys/class/block/cgroup (No such file or directory)
Couldn't read link at /sys/class/block/cgroup (No such file or directory)
Couldn't read link at /sys/class/block/cgroup (No such file or directory)
Couldn't read link at /sys/class/block/cgroup (No such file or directory)
Couldn't read link at /sys/class/block/cgroup (No such file or directory)
Couldn't read link at /sys/class/block/cgroup (No such file or directory)
Couldn't read link at /sys/class/block/cgroup (No such file or directory)
Couldn't read link at /sys/class/block/mqueue (No such file or directory)
Couldn't read link at /sys/class/block/hugetlbfs (No such file or directory)
Couldn't read link at /sys/class/block/zhomez (No such file or directory)
Couldn't read link at /sys/class/block/chungus (No such file or directory)
Couldn't read link at /sys/class/block/tracefs (No such file or directory)

I wouldn't really care, except that one day someone will have an actual device called "cgroup2" or whatnot, and the world will end.

Test with iSCSI block devices

I have no idea how we react to iSCSI block devices. Set some up and find out.

Retrieve disk activity levels at 1Hz

To implement #5 , we'll need get kernel disk statistics. I say we use /proc/diskstats rather than the /sysfs entries, both to keep the number of filesystem operations to a minimum O(1) vs O(N) and to be able to break them down by partition, which I don't think sysfs provides. We have to lex some crap either way. sysfs does provide more detailed stats, IIRC.

https://www.kernel.org/doc/Documentation/block/stat.txt
https://www.kernel.org/doc/Documentation/iostats.txt

Offer to set up filesystem after creating partition

Imported from https://www.sprezzatech.com/bugs/show_bug.cgi?id=269

Assertion in close_safe() down in libatasmart -> libudev

If I sit there and rapidly start and kill sudo ./growlight-curses, about one out of ten times I'll get this:

╭──────[nvme-0 [0] (32Gbps to Southbridge)]────────────────────────────────[-]─╮
│    nvme0n1┌⇗⇨⇨⇨empty space──────────────────────────────────────────────────┐│
│ solidstate│eeeeeeeeeeeeeeeeeeee 18446.7 empty space eeeeeeeeeeeeeeeeeeee123m││
│38╭─────────────────╮X0C-00SJ  n/a   1.00T  512B gpt   1908E1805012     NVMe├┘│
╰──│                 │ice 0000:04.00.0 (x4, gen 3.0)]──────────────────────────╯
   │ Initializing... │
   │Assertion 'close_noAborted) != -EBADF' failed at ../src/basic/fd-util.c:67, [schwarzgerat](134) $  Aborting.

that src/basic/fd-util.c business is from system-fucking-d, which how the hell is it getting involved?

AUR PKGBUILD optdepends on nonexistent package "hfsprofs"

As noted in #2, we've got a typo in our PKGBUILD file, optdepping on hfsprofs instead of hfsprogs.

Extract panelreel core into its own library

The ncurses code is horrible intertwined with all kinds of core crap. Get it extracted, and move it into its own library. Then, unify it with the omphalos implementation, and get that refactored as well.

ZFS partitions show up as code "6"

In the details view of a device with ZFS partitions, we see something like:

╭─press 'v' to dismiss details───────────────────────────────────────────────╮─╯
│Sandisk Corp WD Black 2018/PC SN720 NVMe SSD                                │
│Firmware: 2101 BIOS: American Megatrends Inc. Load: 0bps                    │─╮
│nvme1n1: WDS100T3X0C-00SJG0 (931.51GiB) S/N: 1908E1801188 WC- WRVx RO-      │┐│
│Sectors: 1953525168 (512B logical / 512B physical) NVMe connect             │││
│Partitioning: gpt I/O scheduler: [none] mq-deadline kyber                   │┘│
│ 831.51GiB P₀₃ 209717248→1953525134 nvme1n1p3 “Solaris /usr & Mac ZFS” 0x6 1M─╯
i           zfs_member “zhomez”                                              │
╰────────────────────────────────────────────────────────────────────────────╯

note the '0x6'. here's another example:

╭─press 'v' to dismiss details───────────────────────────────────────────────╮─╯
│Broadcom / LSI SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon]                │
│Firmware: 2101 BIOS: American Megatrends Inc. Load: 36.13Gbps               │─╮
│sdj: ST12000NM0007-2ASN02 (10.91TiB) S/N: ZJV2VZG8 WC+ WRV- RO-             │┐│
│Sectors: 23437770752 (512B logical / 4096B physical) SAT3 connect (6Gbps)   │││
│Partitioning: gpt I/O scheduler: [mq-deadline] kyber none                   │┘│
│  10.91TiB P₀₁ 2048→23437752319 sdj1 “zfs-7f5ed1aa1ce6dbc4” 0x6 1MiB align  M─╯
i           zfs_member “chungus”                                             │
╰────────────────────────────────────────────────────────────────────────────╯

The actual codes are BF01 and BF07.

In growlight-readline, we just see "Oth", sigh. But what the hell is 0x6?

readline blockdev detail hits hdparm error for nvme

When using the readline mode, attempting to get block details on an NVMe drive using hdparm hits a predictable error:

[growlight](-1)> blockdev detail nvme1n1
nvme1n1    WDS100T3X0C-00SJ  n/a   1.00T  512B ✓.... gpt   1908E1801188     NVMe
Unused sectors 0:2047 (1023.46Ki)
nvme1n1p1  08cbe0c1-6771-4574-8ddb-913b24278f92   1.07G Lnx  Linux filesystem
nvme1n1p2  dbe112e1-aa09-4cb1-bf18-e3f01531b942 106.30G Oth  Linux RAID
nvme1n1p3  d1cc37e8-0879-4b51-92b5-845dd89b930f 892.82G Oth  Solaris /usr & Mac ZFS
Unused sectors 1953525135:1953525168 (16.46Ki)

Logical sector size: 512 Physical: 512
I/O scheduler: [none] mq-deadline kyber

BIOS boot SHA-1: 63:9a:c5:cd:f8:a5:cf:32:45:97:59:32:c6:a4:21:54:50:a7:b9:8f
Serial number: 1908E1801188        
Transport: NVMe
Multibyte ended unexpectedly: 
Running "hdparm -I /dev/nvme1n1 2>&1 < /dev/null"...
 HDIO_DRIVE_CMD(identify) failed: Inappropriate ioctl for device

/dev/nvme1n1:
[growlight](0)>

We probably want nvme id-ctrl device output here, though maybe only a subset of it.

Add support for Intel Optane via libipmctl

https://github.com/intel/ipmctl just hit Debian Unstable. It provides extended functionality for Intel Optane DCPMMs. Add support for it. This of course means we'll need buy ourselves some Optane =].

growlight 1.1.1.1-1 immediately exits with "buffer overflow"

I just uploaded 1.1.1.1-1 growlight to the AUR. The binaries created by this package (growlight-ncurses only) immediately exit with the message "buffer overflow", rather worrying...

If i build 1.1.1.1 from source on the same Arch machine, and run growlight-ncurses, I do not get this failure mode. Perhaps makepkg is adding some compiler flags?

Add disk stats to readline UI

#5 addresses how to integrate disk activity statistics with the curses UI, but we also ought expose them in our readline UI. Since there's no regular display update here, they'd need be explicit. I'd say....

include blockdev stats in the blockdev detail command
include partition stats in the partition detail command, if it's ever written
add a stats command to the toplevel

Partition confusion when block device is identified as filesystem by libblkid

I added two 1TB Western Digital Black NVMe M.2 drives to my machine, one requiring an Asus Hyper PCIe card. I carved out a part3 on each:

nvme1n1:

Command (? for help): p
Disk /dev/nvme1n1: 1953525168 sectors, 931.5 GiB
Model: WDS100T3X0C-00SJG0                      
Sector size (logical/physical): 512/512 bytes
Disk identifier (GUID): BF1C425C-02B7-4C9D-9E37-402BE6F84798
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 1953525134
Partitions will be aligned on 2048-sector boundaries
Total free space is 209717214 sectors (100.0 GiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   3       209717248      1953525134   831.5 GiB   BF01  Solaris /usr & Mac ZFS

Command (? for help):

nvme2n1:

Command (? for help): p
Disk /dev/nvme2n1: 1953525168 sectors, 931.5 GiB
Model: WDS100T3X0C-00SJG0                      
Sector size (logical/physical): 512/512 bytes
Disk identifier (GUID): E5B3CDE0-CB86-4347-B302-4A229C8928CB
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 1953525134
Partitions will be aligned on 2048-sector boundaries
Total free space is 209717214 sectors (100.0 GiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   3       209717248      1953525134   831.5 GiB   BF01  Solaris /usr & Mac ZFS

Command (? for help):

and enrolled them into a mirrored zfs vdev, zhomez.

Here's zpool status zhomez:

  pool: zhomez
 state: ONLINE
  scan: resilvered 120G in 0 days 00:02:02 with 0 errors on Thu Jul 18 16:51:30 2019
config:

	NAME                                            STATE     READ WRITE CKSUM
	zhomez                                          ONLINE       0     0     0
	  mirror-0                                      ONLINE       0     0     0
	    nvme-WDS100T3X0C-00SJG0_1908E1801188-part3  ONLINE       0     0     0
	    nvme-WDS100T3X0C-00SJG0_1908E1805012-part3  ONLINE       0     0     0

errors: No known data errors

Upon loading growlight, both the inline display of both devices seems screwed up (it shows almost entirely empty space; the '3' only gets one character), and the details box also has some problems.

Not sure whether the former is due to discontiguous partitions or what, but it's gotta get fixed.

See what can be done about the detail view, as well.

Add cryptfs setup functionality

See https://www.sprezzatech.com/bugs/show_bug.cgi?id=369. We can format cryptfs, but we can't yet set up slots etc.

Color scheme weirdly changes between NVMe SSDs

See the attached screenshot from schwarzgerat. Why does the color scheme change between nvme1n1 and nvme0n1? It doesn't change for the various spinning disks...

External programs in growlight-readline come back with "Multibyte ended unexpectedly"

Pretty much any time we shell out in at least growlight-readline, we see something like e.g.:

Multibyte ended unexpectedly: 
Running "mkswap --version 2>&1 < /dev/null"...
mkswap from util-linux 2.34

Multibyte ended unexpectedly: 
Running "grub-mkdevicemap --version 2>&1 < /dev/null"...
grub-mkdevicemap (GRUB) 2.04-2
No ZFS support in this build.

growlight 1.0.6.1

Also happens with e.g. hdparm and nvme. We get the output, and it looks fine, but this line is worrying and confusing for the user.

Show block device activity level

It would be cool to show the amount of disk usage, both in the past second (tenth of a second?) and perhaps over some history. Doing the latter would require space that I'm not sure is available assuming an 80-character display.

Even the former will be a squeeze. The only unallocated space is on the far left of the device. There are three lines:

device name
rotation speed or "solidstate"
temp/SMART status

the third line is completely full. the second line offers a single character ("solidstate" otherwise occupies all space). the first line can presumably offer a character, though exceptions could conceivably arise(?). we could use these two characters for a vertical indicator made up of Unicode Block Elements. This would give us 16 nominal levels of indicator, and we could combine them with color to indicate 3*16 == 48 levels. Assuming we wanted to support up through a petabit per second, a simple linear relation would afford us 3 levels per order of magnitude. We could do an exponential, say:

3 for sub-K (one per OOM)
9 for K (three per OOM)
24 for M (8 per OOM)
9 for G (three per OOM)
3 for supra-T (one per OOM)

and that's probably more useful. assuming linear distribution within a range, that would give us:

blue less than 1.25Mbit/s
orange up through 500Mbit/s
red greater than 500Mbit/s

If we want a historical view, the only real choice I see would be to indicate it via intensity of the border of the device, beginning on the lower right, moving counterclockwise. This will be complicated by the case of a selected device. Here's a device:

╭──────[nvme-1 [0] (32Gbps to Southbridge)]────────────────────────────────[-]─╮
│    nvme1n1┌⇗⇨⇨⇨empty space──────────────────────────────────────────────────┐│
│ solidstate│eeeeeeeeeeeeeeeeeeee 18446.7 empty space eeeeeeeeeeeeeeeeeeee123m││
│51°C smart+└┤WDS100T3X0C-00SJ  n/a   1.00T  512B gpt   1908E1801188     NVMe├┘│
╰─────[PCI Express device 0000:02.00.0 (x4, gen 3.0)]──────────────────────────╯

This of course offers only a very few levels (three or four at the max), and doesn't mix well with the partition browser on the top border of the device. There would also be no place to put a legend. It sounds cool, but it just doesn't seem practical.

Another option would be some kind of indicator outside the device, i.e. a panel somewhere which could presumably be enabled and disabled at runtime. Where to put this widget would be a question, but this might be the best thing?

Normalize utilization stats against time period

We retain the actual time difference between statistics samples in each device. We ought be normalizing the delta against this number when printing B/s in the summary (see #5).

readline stats are since last second, not since last request

Since reimplementing #20 to use the device-backed stats, running stats shows deltas for the last second, not since the last time we ran stats. It should probably go back to the latter behavior, but this isn't a huge deal.

Coredump in console mode scrolling through full screen

I was in a text console (25 rows or so) on schwarzgerat, and started growlight 1.0.6.2. Moving down through the adapters, I was able to generate a sigfault (NOT an abort()). Seems pretty reproducible, so get at it.

Track maximum bandwidth to each device for better demand estimates

Right now, we assume a SATAIII spinning disk can demand 6Gbps. This will never be the case, and results in SATA complexes looking horribly oversubscribed. With the new diskstats, we can track an actual maximum, and make a better estimate. Think about how to do so.

Test with Fiber Channel hardware

I have no idea if we work with Fiber Channel crap. Get access to some, and find out.

Navigating to metadata zone in detail block gets moved back to non-metadata

If I have a blcok device with metadata followed by partitioned space (like any GPT device), and I navigate to the metadata area, within a second I'm moved back to the next non-metadata zone. Not sure when this started.

Devices are showing up as "removable" when they most certainly are not

Seeing this on killermike, using growlight 1.1.1-pre.

╭──────[nvme-0 [0] (32Gbps to Southbridge, 32Gbps (100%) demanded)]────────[-]─╮
│    nvme0n1┌─⇗⇨⇨⇨nvme0n1p1───────────────────────────────────────────────────┐│
│✔solidstate│m1113333 linux_raid_member “debian:intel 750 nvme” (378.61G) 333m││
│30°  no i/o└┤INTEL SSDPE2MW40  n/a 400.08G  512B gpt   CVCQ5135007B400C NVMe├┘│
╰─────[PCI Express device 0000:09.00.0 (x4, gen 3.0)]──────────────────────────╯

╭──────[ahci-0 (256Gbps to Southbridge, 12Gbps (4%) demanded)]─────────────[-]─╮
│        sdb┌─────────────────────────────────────────────────────────────────┐│
│✔ removable│me11111111111111111111 btrfs “butters” (12T) 1111111111111111111m││
│30°  no i/o└┤ST12000NM0007-2A SN02  12.00T 4096B gpt   5000c500b408015d SAT3├┘│
│        sdc┌─────────────────────────────────────────────────────────────────┐│
│✔ removable│me1 ext4 at / 1233333333333 ext4 at /home (199.96G) 33333333333em││
│30°  no i/o└┤Samsung SSD 840  6B0Q 256.06G  512B gpt   50025385a02983fa SAT3├┘│
╰─────[PCI Express device 0000:08.00.0 (x16, gen 4.0)]─────────────────────────╯

╭──────[ahci-1 (256Gbps to Southbridge, 6Gbps (2%) demanded)]──────────────[-]─╮
│        sda┌─────────────────────────────────────────────────────────────────┐│
│✔ removable│me11111111111111111 btrfs at /media/store (12T) 1111111111111111m││
│32°  no i/o└┤ST12000NM0007-2A SN02  12.00T 4096B gpt   5000c500b4b8415d SAT3├┘│
╰─────[PCI Express device 0000:07.00.0 (x16, gen 4.0)]─────────────────────────╯

Test with Infiniband hardware

I have no idea if we work with Infiniband junk in the trunk. Get access to some and find out.

Curses UI shows "Already browsing [adapter]" when navigating around

Since approximately the addition of stats in 1.6.0.2 (#18), the curses UI sometimes shows "Already browsing [adapter name]" when one moves between adapters. This is very easy to stimulate on e.g. grimes, my laptop.

Introduce some unit testing

We need some unit testing; it's scandalous that we've gone without it for so long. It would be nice to cover the read_diskstats() functionality being introduced in #18 , actually. But either way, get cunit set up in there.

Import liboutcurses and use it for genprefix()

I've found several bugs in genprefix() after moving it to liboutcurses:

I intend to fix these in liboutcurses. We ought purge the growlight copy of genprefix, dep on liboutcurses, and use those.

NVMe devices show 0 load in details view

This is true for my M.2s on schwarzgerat and my U.2 connected via M.2 ASUS Hyperkit in KillerMike.

╭─press 'v' to dismiss details───────────────────────────────────────────────╮
│Intel Corporation PCIe Data Center SSD                                      │
│Firmware: 1804 BIOS: American Megatrends Inc. Load: 0bps                    │
│nvme0n1: INTEL SSDPE2MW400G4 (372.61GiB) S/N: CVCQ5135007B400CGN            │
│Sectors: 781422768 (512B logical / 512B physical) NVMe connect              │
│Partitioning: gpt I/O scheduler: [none] mq-deadline                         │
│     20GiB P₀₁ 2048→41945087 nvme0n1p1 “schwarzgerat-swap” 8200 1MiB align  │
│           unlabeled swap                                                   │
╰────────────────────────────────────────────────────────────────────────────╯

Demystify numbers with documentation

u/insanemal of reddit's r/datahoarders points out, quite correctly, that our documentation sucks. Write up a gentler introduction to the tool, and some design aspects. In particular, explain what various numbers mean.

Can't compile on Arch due to ncursesw6/panel

We blow up attempting to compile on Arch Linux, due to differences in placement of ncursesw infrastructure in their ncurses6 build.

since fixing #48, optical device shows up as ferromag

│ sr0┌─────────────────────────────────────────────────────────────────┐│
│ ferromag│No media detected in drive ││
│ no i/o└┤iHBS112 2 CL0F 1.07G 512B none n/a PATA├┘

bah. this is since fixing #48 . this is better, since those other ones certainly were not optical and there are a lot fewer optical devices than hotswappable hard drives, but still, get this fixed up.

also need to check growlight-readline about this.

Offer to mount a filesystem after creating it

See https://www.sprezzatech.com/bugs/show_bug.cgi?id=270

Package things up for Arch AUR

Now that we've got #1 resolved, we ought be able to build up an AUR package, as I just did for omphalos.

Support LVM configs

Imported from https://www.sprezzatech.com/bugs/show_bug.cgi?id=291. We've been operating blissfully unaware of LVM (well, aware, but not thinking about it). We'll probably need support it in one form or another, ugh.

Detect UASP

http://linux-sunxi.org/USB/UAS USB Attached Storage Protocol supports faster block transfers over USB. Detect whether it is in use, and make this information available.

Details view sticks WC settings atop border

Look at the details view here. What's up with the spacing? Did we let a tab through or something?

Note that this is also mentioned in #4. Use this bug to track the fix.

configure script hangs looking for docbook

we sat here for a good 30s on a machine that ended up actually having a satisfactory install, weird:

7186 ?        Ss     0:00  \_ sshd: dank [priv]
 7192 ?        S      0:00      \_ sshd: dank@pts/1
 7193 pts/1    Ss     0:00          \_ -bash
 8459 pts/1    S+     0:00              \_ /bin/bash ./configure
 9195 pts/1    S+     0:00                  \_ /usr/bin/xsltproc http://docbook.sourceforge.net/release/xsl/current/xhtml/docbook.xsl

checking for epoll_create1... yes
checking for Docbook XSLT version current... yes
checking for pkg-config... pkg-config

Support nontrivial btrfs configs

Migrated from https://www.sprezzatech.com/bugs/show_bug.cgi?id=331. btrfs can run in one of 4 multidisk modes: raid0, raid1, raid10, and single. Support these.

https://btrfs.wiki.kernel.org/index.php/Using_Btrfs_with_Multiple_Devices

Calculation of decimal part in genprefix() is incorrect

While working on #10, I looked at this, and was like, "that can't be right":

diff --git a/src/growlight.h b/src/growlight.h
index 1671930..19ecf5e 100644
--- a/src/growlight.h
+++ b/src/growlight.h
@@ -454,8 +454,11 @@ genprefix(uintmax_t val,unsigned decimal,char *buf,size_t bsize,
                dv /= mult;
                val /= decimal;
                if((val % dv) / ((dv + 99) / 100) || omitdec == 0){
                   snprintf(buf,bsize,"%ju.%02ju%c%c",val / dv,(val % dv) / ((dv + 99) / 100),
                                   prefixes[consumed - 1],uprefix);
                }else{
                        snprintf(buf,bsize,"%ju%c%c",val / dv,prefixes[consumed - 1],uprefix);
                }

So I'm clearly trying to get a percentage for the mantissa, but...that's not how it's done, son. And indeed:

│  16.46KiB 23437770719→23437770751 partition table metadata

Unless there's something else at work here, 23437770751 - 23437770719 + 1 -> 33 sectors. At 512 each, that's 16896 bytes. 16896 % 1024 is 512, which is exactly half a kibibyte. We instead show 16.46, which is indeed 512 / ((1024 + 99) / 100). The correct display is 16.50.

The formula we want, I'm pretty sure, is (val % dv) * 100 / dv. I'm not sure where that other one came from.

filesystem line allowed to escape box in details view

I saw this today. Note the 'M' sitting on the border, and the 'i' sitting on the next line:

╭─press 'v' to dismiss details───────────────────────────────────────────────╮─╯
│Sandisk Corp WD Black 2018/PC SN720 NVMe SSD                                │
│Firmware: 2101 BIOS: American Megatrends Inc. Load: 0bps                    │─╮
│nvme1n1: WDS100T3X0C-00SJG0 (931.51GiB) S/N: 1908E1801188 WC- WRVx RO-      │┐│
│Sectors: 1953525168 (512B logical / 512B physical) NVMe connect             │││
│Partitioning: gpt I/O scheduler: [none] mq-deadline kyber                   │┘│
│ 831.51GiB P₀₃ 209717248→1953525134 nvme1n1p3 “Solaris /usr & Mac ZFS” 0x6 1M─╯
i           zfs_member “zhomez”                                              │
╰────────────────────────────────────────────────────────────────────────────╯

This is probably very similar to #6

valgrind warning in blkid_do_fullprobe

ead 0/512 of 22 (Input/output error?)
SNLEN 20 END 0 IN 0 iter 20
HEHEEHE [01234567890123456789]
==15964== Warning: noted but unhandled ioctl 0x127a with no size/direction hints.
==15964==    This could cause spurious value errors to appear.
==15964==    See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
==15964== Thread 4:
==15964== Conditional jump or move depends on uninitialised value(s)
==15964==    at 0x4D99883: ??? (in /lib/x86_64-linux-gnu/libblkid.so.1.1.0)
==15964==    by 0x4D9A465: ??? (in /lib/x86_64-linux-gnu/libblkid.so.1.1.0)
==15964==    by 0x4D99A88: ??? (in /lib/x86_64-linux-gnu/libblkid.so.1.1.0)
==15964==    by 0x4D8817A: blkid_do_fullprobe (in /lib/x86_64-linux-gnu/libblkid.so.1.1.0)
==15964==    by 0x114E1E: probe_blkid_superblock (libblkid.c:127)
==15964==    by 0x110CE0: rescan (growlight.c:1014)
==15964==    by 0x110CE0: rescan (growlight.c:905)
==15964==    by 0x1121C5: create_new_device_inner (growlight.c:1137)
==15964==    by 0x1121C5: create_new_device (growlight.c:1179)
==15964==    by 0x112445: lookup_device (growlight.c:1249)
==15964==    by 0x112445: lookup_device (growlight.c:1201)
==15964==    by 0x113C6C: scan_device (growlight.c:1381)
==15964==    by 0x5215FA2: start_thread (pthread_create.c:486)
==15964==    by 0x53284CE: clone (clone.S:95)
==15964== 
==15964== Warning: noted but unhandled ioctl 0x1278 with no size/direction hints.
==15964==    This could cause spurious value errors to appear.
==15964==    See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
==15964== Conditional jump or move depends on uninitialised value(s)
==15964==    at 0x4D99883: ??? (in /lib/x86_64-linux-gnu/libblkid.so.1.1.0)
==15964==    by 0x4D9A4AA: ??? (in /lib/x86_64-linux-gnu/libblkid.so.1.1.0)
==15964==    by 0x4D99A88: ??? (in /lib/x86_64-linux-gnu/libblkid.so.1.1.0)
==15964==    by 0x4D8817A: blkid_do_fullprobe (in /lib/x86_64-linux-gnu/libblkid.so.1.1.0)
==15964==    by 0x114E1E: probe_blkid_superblock (libblkid.c:127)
==15964==    by 0x110CE0: rescan (growlight.c:1014)
==15964==    by 0x110CE0: rescan (growlight.c:905)
==15964==    by 0x1121C5: create_new_device_inner (growlight.c:1137)
==15964==    by 0x1121C5: create_new_device (growlight.c:1179)
==15964==    by 0x112445: lookup_device (growlight.c:1249)
==15964==    by 0x112445: lookup_device (growlight.c:1201)
==15964==    by 0x113C6C: scan_device (growlight.c:1381)
==15964==    by 0x5215FA2: start_thread (pthread_create.c:486)
==15964==    by 0x53284CE: clone (clone.S:95)
==15964== 
==15964== Use of uninitialised value of size 8
==15964==    at 0x527BD1E: _itoa_word (_itoa.c:179)
==15964==    by 0x527F5F3: vfprintf (vfprintf.c:1637)
==15964==    by 0x53390F7: __vasprintf_chk (vasprintf_chk.c:66)
==15964==    by 0x4D866FA: ??? (in /lib/x86_64-linux-gnu/libblkid.so.1.1.0)
==15964==    by 0x4D867CE: ??? (in /lib/x86_64-linux-gnu/libblkid.so.1.1.0)
==15964==    by 0x4D9A4AA: ??? (in /lib/x86_64-linux-gnu/libblkid.so.1.1.0)
==15964==    by 0x4D99A88: ??? (in /lib/x86_64-linux-gnu/libblkid.so.1.1.0)
==15964==    by 0x4D8817A: blkid_do_fullprobe (in /lib/x86_64-linux-gnu/libblkid.so.1.1.0)
==15964==    by 0x114E1E: probe_blkid_superblock (libblkid.c:127)
==15964==    by 0x110CE0: rescan (growlight.c:1014)
==15964==    by 0x110CE0: rescan (growlight.c:905)
==15964==    by 0x1121C5: create_new_device_inner (growlight.c:1137)
==15964==    by 0x1121C5: create_new_device (growlight.c:1179)
==15964==    by 0x112445: lookup_device (growlight.c:1249)
==15964==    by 0x112445: lookup_device (growlight.c:1201)
==15964== 
==15964== Conditional jump or move depends on uninitialised value(s)
==15964==    at 0x527BD29: _itoa_word (_itoa.c:179)
==15964==    by 0x527F5F3: vfprintf (vfprintf.c:1637)
==15964==    by 0x53390F7: __vasprintf_chk (vasprintf_chk.c:66)
==15964==    by 0x4D866FA: ??? (in /lib/x86_64-linux-gnu/libblkid.so.1.1.0)
==15964==    by 0x4D867CE: ??? (in /lib/x86_64-linux-gnu/libblkid.so.1.1.0)
==15964==    by 0x4D9A4AA: ??? (in /lib/x86_64-linux-gnu/libblkid.so.1.1.0)
==15964==    by 0x4D99A88: ??? (in /lib/x86_64-linux-gnu/libblkid.so.1.1.0)
==15964==    by 0x4D8817A: blkid_do_fullprobe (in /lib/x86_64-linux-gnu/libblkid.so.1.1.0)
==15964==    by 0x114E1E: probe_blkid_superblock (libblkid.c:127)
==15964==    by 0x110CE0: rescan (growlight.c:1014)
==15964==    by 0x110CE0: rescan (growlight.c:905)
==15964==    by 0x1121C5: create_new_device_inner (growlight.c:1137)
==15964==    by 0x1121C5: create_new_device (growlight.c:1179)
==15964==    by 0x112445: lookup_device (growlight.c:1249)
==15964==    by 0x112445: lookup_device (growlight.c:1201)
==15964== 
==15964== Conditional jump or move depends on uninitialised value(s)
==15964==    at 0x5280213: vfprintf (vfprintf.c:1637)
==15964==    by 0x53390F7: __vasprintf_chk (vasprintf_chk.c:66)
==15964==    by 0x4D866FA: ??? (in /lib/x86_64-linux-gnu/libblkid.so.1.1.0)
==15964==    by 0x4D867CE: ??? (in /lib/x86_64-linux-gnu/libblkid.so.1.1.0)
==15964==    by 0x4D9A4AA: ??? (in /lib/x86_64-linux-gnu/libblkid.so.1.1.0)
==15964==    by 0x4D99A88: ??? (in /lib/x86_64-linux-gnu/libblkid.so.1.1.0)
==15964==    by 0x4D8817A: blkid_do_fullprobe (in /lib/x86_64-linux-gnu/libblkid.so.1.1.0)
==15964==    by 0x114E1E: probe_blkid_superblock (libblkid.c:127)
==15964==    by 0x110CE0: rescan (growlight.c:1014)
==15964==    by 0x110CE0: rescan (growlight.c:905)
==15964==    by 0x1121C5: create_new_device_inner (growlight.c:1137)
==15964==    by 0x1121C5: create_new_device (growlight.c:1179)
==15964==    by 0x112445: lookup_device (growlight.c:1249)
==15964==    by 0x112445: lookup_device (growlight.c:1201)
==15964==    by 0x113C6C: scan_device (growlight.c:1381)
==15964== 
==15964== Conditional jump or move depends on uninitialised value(s)
==15964==    at 0x527F75D: vfprintf (vfprintf.c:1637)
==15964==    by 0x53390F7: __vasprintf_chk (vasprintf_chk.c:66)
==15964==    by 0x4D866FA: ??? (in /lib/x86_64-linux-gnu/libblkid.so.1.1.0)
==15964==    by 0x4D867CE: ??? (in /lib/x86_64-linux-gnu/libblkid.so.1.1.0)
==15964==    by 0x4D9A4AA: ??? (in /lib/x86_64-linux-gnu/libblkid.so.1.1.0)
==15964==    by 0x4D99A88: ??? (in /lib/x86_64-linux-gnu/libblkid.so.1.1.0)
==15964==    by 0x4D8817A: blkid_do_fullprobe (in /lib/x86_64-linux-gnu/libblkid.so.1.1.0)
==15964==    by 0x114E1E: probe_blkid_superblock (libblkid.c:127)
==15964==    by 0x110CE0: rescan (growlight.c:1014)
==15964==    by 0x110CE0: rescan (growlight.c:905)
==15964==    by 0x1121C5: create_new_device_inner (growlight.c:1137)
==15964==    by 0x1121C5: create_new_device (growlight.c:1179)
==15964==    by 0x112445: lookup_device (growlight.c:1249)
==15964==    by 0x112445: lookup_device (growlight.c:1201)
==15964==    by 0x113C6C: scan_device (growlight.c:1381)
==15964== 
==15964== Warning: noted but unhandled ioctl 0x1279 with no size/direction hints.
==15964==    This could cause spurious value errors to appear.
==15964==    See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
Read 0/512 of 29 (Input/output error?)
Read 0/512 of 9 (Input/output error?)
Read -1/512 of 31 (Input/output error?)

valgrind warning in lex_diskstats

Couldn't read link at /sys/class/block/hugetlbfs (No such file or directory)
Couldn't read link at /sys/class/block/zhomez (No such file or directory)
Couldn't read link at /sys/class/block/chungus (No such file or directory)
Couldn't read link at /sys/class/block/tracefs (No such file or directory)
==26135== Thread 2:
==26135== Conditional jump or move depends on uninitialised value(s)
==26135==    at 0x483D3D6: rawmemchr (vg_replace_strmem.c:1422)
==26135==    by 0x52C09E1: _IO_str_init_static_internal (strops.c:41)
==26135==    by 0x529517E: _IO_strfile_read (strfile.h:95)
==26135==    by 0x529517E: __isoc99_sscanf (isoc99_sscanf.c:28)
==26135==    by 0x1204FA: lex_diskstats (stats.c:150)
==26135==    by 0x1204FA: read_diskstats (stats.c:176)
==26135==    by 0x1141FD: event_posix_thread (growlight.c:1679)
==26135==    by 0x5226FB6: start_thread (pthread_create.c:486)
==26135==    by 0x533949E: clone (clone.S:95)
==26135== 
[growlight](0)>

Lots of "gen unknown" on new x570p AMD pcie4 machine

This is on killermike. We've got one SATA SSD and two SATA spinning disks.

╭──────[ahci-1 (256Gbps to Southbridge, 6Gbps (2%) demanded)]──────────────[-]─╮
│ sda┌─────────────────────────────────────────────────────────────────┐│
│✔ removable│me1 ext4 at / 1233333333333 ext4 at /home (199.96G) 33333333333em││
│31° no i/o└┤Samsung SSD 840 6B0Q 256.06G 512B gpt 50025385a02983fa SAT3├┘│
╰─────[PCI Express device 0000:07.00.0 (x16, gen unknown)]─────────────────────╯

╭──────[ahci-0 (256Gbps to Southbridge, 12Gbps (4%) demanded)]─────────────[-]─╮
│ sdc┌─⇗⇨⇨⇨empty space─────────────────────────────────────────────────┐│
│✔ removable│me11111111111111111 btrfs at /media/store (12T) 1111111111111111m││
│38° no i/o└┤ST12000NM0007-2A SN02 12.00T 4096B gpt 5000c500b408015d SAT3├┘│
│ sdb┌─────────────────────────────────────────────────────────────────┐│
│✔ removable│me1111111111111 zfs_member “pbrstreetgang” (12T) 11111111111119em││
│37° no i/o└┤ST12000NM0007-2A SN02 12.00T 4096B gpt 5000c500b4b8415d SAT3├┘│
╰─────[PCI Express device 0000:08.00.0 (x16, gen unknown)]─────────────────────╯

Passing scaling=0 to genprefix() generates illegal instruction

Glad we moved on #19, since the first test yielded up this:

[schwarzgerat](2) $ ./growlight-test 


     CUnit - A unit testing framework for C - Version 2.1-3
     http://cunit.sourceforge.net/


Suite: test/growlight.c
  Test: genprefix() ...Illegal instruction
[schwarzgerat](132) $

locked_diag() is stripping non-ascii

See https://www.sprezzatech.com/bugs/show_bug.cgi?id=422. locked_diag() is stripping our unicode smartquotes. it ought only be stripping extraneous whitespace.

Error creating a btrfs

We call mkfs.btrfs -h -L "butters" /dev/sdb1, and smartly get back -h is an invalid option. v1.1.0.1.