falconindy / expac Goto Github PK
View Code? Open in Web Editor NEWalpm data extraction utility
alpm data extraction utility
Repository configuration
% cat /etc/pacman.d/custom
[options]
CacheDir = /var/cache/pacman/pkg/
CacheDir = /var/cache/pacman/custom/
CleanMethod = KeepCurrent
[custom]
SigLevel = Optional TrustAll
Server = file:///home/custompkgs
Relevant pacman.conf line:
# An example of a custom package repository. See the pacman manpage for
# tips on creating your own repositories.
Include = /etc/pacman.d/custom
strace shows expac only queries repositories defined direclty in /etc/pacman.conf
:
open("/etc/pacman.conf", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=2990, ...}) = 0
read(3, "#\n# /etc/pacman.conf\n#\n# See the"..., 4096) = 2990
read(3, "", 4096) = 0
close(3) = 0
stat("/", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
stat("/var/lib/pacman", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
lstat("/var", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
lstat("/var/lib", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
lstat("/var/lib/pacman", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
open("/var/lib/pacman/local/", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFDIR|0755, st_size=57344, ...}) = 0
open("/var/lib/pacman/local/ALPM_DB_VERSION", O_RDONLY) = 4
fstat(4, {st_mode=S_IFREG|0644, st_size=2, ...}) = 0
read(4, "9\n", 4096) = 2
close(4) = 0
close(3) = 0
access("/var/lib/pacman/sync/core.db", R_OK) = 0
access("/var/lib/pacman/sync/extra.db", R_OK) = 0
access("/var/lib/pacman/sync/community.db", R_OK) = 0
access("/var/lib/pacman/sync/multilib.db", R_OK) = 0
access("/var/lib/pacman/sync/xyne-x86_64.db", R_OK) = 0
open("/var/lib/pacman/sync/core.db", O_RDONLY|O_CLOEXEC) = 3
pacman.conf: https://ptpb.pw/Z3oc
strace for expac -S '%n %v' powerline-fonts
: https://ptpb.pw/mUce
expac crashes when the size formatter is missing:
[~]% expac -H '%m\t%n'
error: invalid SI size formatter: %m\t%n
[1] 2682 segmentation fault (core dumped) expac -H '%m\t%n'
Specifying the size formatter works as expected:
[~]% expac -H M '%m\t%n'
...
Info:
[~]% pacman -Qi expac
Name : expac
Version : 4-3
Description : pacman database extraction utility
Architecture : x86_64
URL : http://github.com/falconindy/expac
Licenses : GPL
Groups : None
Provides : None
Depends On : pacman
Optional Deps : None
Required By : packer
Optional For : None
Conflicts With : None
Replaces : None
Installed Size : 29,00 KiB
Packager : Dave Reisner <[email protected]>
Build Date : Fr 19 Dez 2014 14:49:04 CET
Install Date : Di 30 Dez 2014 23:33:54 CET
Install Reason : Installed as a dependency for another package
Install Script : No
Validated By : Signature
It would be handy if expac supported a --dbpath option like pacman does. Currently, the only option is to create an alternative config file and then use the --config option. This option would be especially useful for distributions, such as Manjaro, which by default keep a separate up-to-date sync database so that you can view the update-able packages without executing the dreaded 'pacman -Sy' command.
This would be a fairly trivial change. I'd be happy to submit a pull request if you think it might be be a worthwhile addition.
-Jeff
In a script, I'm using expac in a way that mimics cower output:
expac -S -1 ":: %n is available in %r" ${arrayofpackages[@]}
This works great, and I'm doing it the same way as above but with color codes:
expac -S -1 "\e[1;33m::\e[0m \e[1;39m%n\e[0m is available in \e[1;35m%r\e[0m" ${arrayofpackages[@]}
This correctly results in the following colored output:
:: apache-ant is available in extra
:: avrdude is available in community
:: avr-gcc is available in community
:: avr-libc is available in community
...
However, there is a regression when using the latest expac-git, and the colored output doesn't display as expected anymore:
[1;33m::[0m [1;39mapache-ant[0m is available in [1;35mextra[0m
[1;33m::[0m [1;39mavrdude[0m is available in [1;35mcommunity[0m
[1;33m::[0m [1;39mavr-gcc[0m is available in [1;35mcommunity[0m
[1;33m::[0m [1;39mavr-libc[0m is available in [1;35mcommunity[0m
...
Is this an unwanted side effect of the recent changes, or does this seem to be a new normal behavior?
Even if it's left to the default value which is /var/lib/pacman/
E.g.: expac -Ss %n
I have all the standard non-testing repositories, but I also have bohoomil's [infinality-bundle] repo. I added the repository (and made no changes to the defaults as to what signing is required for those packages) followed the instructions
pacman-key -r 962DDE58
pacman-key --lsign-key 962DDE58
However, I noticed when running expac through GNU Parallel with both -S and -Q on non-infinality packages, my internet was spiking dramatically (the effect was amplified to the equivalent of calling expac 16 times a second at points, just based on how the branches of execution lined up). After wiresharking, closer inspection shows me that, even when querying the local sync database, every call to expac has my system calling out to DNS and then to "http://pgp.mit.edu:11371/pks/lookup?op=get&options=mr&search=0x962DDE58".
Is it intentional for a local query to use the internet connection?
Add listing packages in AUR
Since pacman 4.2 now exposes the pkgbase
via .PKGINFO
and desc
(?), would it be possible to add a new formatter for expac to let us determine the pkgbase for a split-package?
Such as %c
or one of AHIJKUWXYZceqyz
perhaps.
[My assumption on pkgbase
being exposed might be wildly wrong.]
Packages in "multilib" do not seem to display.
There exists a format argument for "%N required by" packages, but there is no format argument for listing "optionally required by" packages.
e.g. expac -S "%n - %R" bluez-utils
yields bluez-utils - bluez-hcidump bluez
whereas https://www.archlinux.org/packages/extra/x86_64/bluez-utils/ lists the replacements as bluez<=4.101, bluez-hcidump
, notice the missing <=4.101
for bluez
.
Would it be possible, to append the versioning, too? I would really appreciate that.
According to the manual page cower
should also recognize the --query
option:
-Q, --query
Search the local database for provided targets. This is the default behavior.
However it does not:
$ expac -Q
error: missing format string (use -h for help)
$ expac --query
expac: unrecognized option '--query'
Consider the following:
$ sudo pacman -S "expac>7"
warning: expac-8-1 is up to date -- reinstalling
resolving dependencies...
looking for conflicting packages...
Package (1) Old Version New Version Net Change Download Size
extra/expac 8-1 8-1 0.00 MiB 0.01 MiB
Total Download Size: 0.01 MiB
Total Installed Size: 0.03 MiB
Net Upgrade Size: 0.00 MiB
:: Proceed with installation? [Y/n] n
$ sudo pacman -S "expac>8"
error: target not found: expac>8
Using expac:
$ expac -S '%r/%n-%v' expac
extra/expac-8-1
$ expac -S '%r/%n-%v' "expac>7"
(no result)
It would be useful to have a similar versining support in expac, as this might avoid the need to do some extra check separately before using expac. Hopefully this is in the scope of expac.
In my local fork, I've replaced alpm_db_get_pkg()
with alpm_find_dbs_satisfier()
when searching for exact packages, which gives the following results:
$ ./expac -Q '%r/%n-%v' "expac>8"
(no result, as expected)
$ ./expac -Q '%r/%n-%v' "expac>7"
local/expac-8-1
$ ./expac -S '%r/%n-%v' "expac>8"
(no result, as expected)
$ ./expac -S '%r/%n-%v' "expac>7"
extra/expac-8-1
extra/expac-8-1
It works, but for -S
, the output is doubled - and I cannot find out the reason. Any idea?
I've tried passing r->next
instead of r
, but while the issue is fixed with -S
, this breaks -Q
.
This is related to issue #23.
I would like to limit expac to search only specific repositories when
specifying -S
.
As a work around for this I am instead prefixing the repo name (%r
) and using
that as a marker for awk which then prints the remaining column. E.g.
expac -S '%r %n:%E' -l ':' < <(pacman -Sql "$repo") | awk -v repo="^$repo" '$1 ~ repo { print $2 }'
Ideally I would like to constrain -S
to the defined $repo
above via
something like expac -S --repo=$repo ...
.
Any thoughts?
I can't get the search function to work with the local query. expac -s target
(or expac -Qs target
) goes through all local (installed) packages.
I'm using the latest release 10. Is this a bug or am I doing it wrong?
Using expac-git 3.7.g951f35b-1:
% expac -S '%n is %v...' emacs
emacs is 24.3-6..
Note how the last character is missing. This doesn't happen if it's a format char.
A bisect says: 0ba876f is the first bad commit.
I'd suggest to remove the "- 1" from end in print_pkg.
I haven't been able to reproduce this on other computer but on one of them whenever I run expac (with any arguments) it never returns.
With strace, I see the process keeps doing a 'select' operation which timeout each time:
[...]
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=5322, si_status=0, si_utime=0, si_stime=0} ---
close(7) = 0
close(9) = 0
select(9, [6 8], [], NULL, {1, 0}) = 1 (in [8], left {0, 991841})
select(9, [8], [], NULL, {0, 0}) = 1 (in [8], left {0, 0})
read(8, "tru::1:1338890229:0:3:1:5\n", 1024) = 26
select(9, [6 8], [], NULL, {1, 0}) = 2 (in [6 8], left {0, 999996})
select(7, [6], [], NULL, {0, 0}) = 1 (in [6], left {0, 0})
read(6, "", 1024) = 0
close(6) = 0
select(9, [8], [], NULL, {0, 0}) = 1 (in [8], left {0, 0})
read(8, "", 1024) = 0
close(8) = 0
close(5) = 0
munmap(0x7f5a538f9000, 4096) = 0
close(4) = 0
munmap(0x7f5a538f8000, 4096) = 0
pipe([4, 5]) = 0
fcntl(4, F_SETFD, FD_CLOEXEC) = 0
clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f5a538fc9d0) = 5324
wait4(5324, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 5324
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=5324, si_status=0, si_utime=0, si_stime=0} ---
close(5) = 0
read(4, "gpg (GnuPG) 2.0.19\nlibgcrypt 1.5"..., 79) = 79
close(4) = 0
pipe([4, 5]) = 0
fcntl(4, F_SETFD, FD_CLOEXEC) = 0
ioctl(1, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
ioctl(1, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
fstat(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 1), ...}) = 0
readlink("/proc/self/fd/1", "/dev/pts/1", 63) = 10
stat("/dev/pts/1", {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 1), ...}) = 0
pipe([6, 7]) = 0
fcntl(6, F_SETFD, FD_CLOEXEC) = 0
clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f5a538fc9d0) = 5326
wait4(5326, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 5326
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=5326, si_status=0, si_utime=0, si_stime=0} ---
close(5) = 0
close(7) = 0
select(7, [4 6], [], NULL, {1, 0}) = 1 (in [6], left {0, 991749})
select(7, [6], [], NULL, {0, 0}) = 1 (in [6], left {0, 0})
read(6, "tru::1:1338890229:0:3:1:5\n", 1024) = 26
select(7, [4 6], [], NULL, {1, 0}) = 2 (in [4 6], left {0, 999996})
select(5, [4], [], NULL, {0, 0}) = 1 (in [4], left {0, 0})
read(4, "", 1024) = 0
close(4) = 0
select(7, [6], [], NULL, {0, 0}) = 1 (in [6], left {0, 0})
read(6, "", 1024) = 0
close(6) = 0
pipe([4, 5]) = 0
fcntl(4, F_SETFD, FD_CLOEXEC) = 0
clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f5a538fc9d0) = 5328
wait4(5328, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 5328
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=5328, si_status=0, si_utime=0, si_stime=0} ---
close(5) = 0
read(4, "gpg (GnuPG) 2.0.19\nlibgcrypt 1.5"..., 79) = 79
close(4) = 0
pipe([4, 5]) = 0
fcntl(4, F_SETFD, FD_CLOEXEC) = 0
ioctl(1, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
ioctl(1, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
fstat(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 1), ...}) = 0
readlink("/proc/self/fd/1", "/dev/pts/1", 63) = 10
stat("/dev/pts/1", {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 1), ...}) = 0
pipe([6, 7]) = 0
fcntl(6, F_SETFD, FD_CLOEXEC) = 0
clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f5a538fc9d0) = 5330
wait4(5330, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 5330
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=5330, si_status=0, si_utime=0, si_stime=0} ---
close(5) = 0
close(7) = 0
select(7, [4 6], [], NULL, {1, 0}) = 0 (Timeout)
select(7, [4 6], [], NULL, {1, 0}) = 0 (Timeout)
select(7, [4 6], [], NULL, {1, 0}) = 0 (Timeout)
select(7, [4 6], [], NULL, {1, 0}) = 0 (Timeout)
select(7, [4 6], [], NULL, {1, 0}) = 0 (Timeout)
select(7, [4 6], [], NULL, {1, 0}) = 0 (Timeout)
[...]
I'm running an uptodate Arch x86_64.
Same as %E, see example :
$ expac %D -l'\n' pacman
bash
glibc
libarchive
curl
gpgme
pacman-mirrorlist
archlinux-keyring
Pacman's query operation has a -p
or --file
option, which instead of querying the DB entry, allows querying package files directly:
$ pacman -Qi expac | awk -F '[[:space:]]*:[[:space:]]*' '/Name/{print $2}'
expac
$ expac -Q %n expac
expac
$ cd /var/cache/pacman/pkg
$ pacman -Qip expac-0.07-1-i686.pkg.tar.xz | awk -F '[[:space:]]*:[[:space:]]*' '/Name/{print $2}'
expac
$ expac -Qp %n expac-0.07-1-i686.pkg.tar.xz
expac: invalid option -- 'p'
it would be nice to get the validation field from the local db.
Seen in pacman -Qi output: Validated By: Signature
After patch #37 almost a year passed without any other changes. The problem is still relevant for Raspberry Pi / Arch Linux ARM. Could make a new release incorporating this change?
I use the expac-git from AUR and it is updated to it's latest revision. I have tried the examples and nothing works. All the expac commands produce empty output. The same happens with the stable version too.
I use expac indirectly through the use of pacaur. I regularly see issue 4 as I have boohomil's infinality font bundle repos added to pacman.conf. While I know this is somewhat of a security risk, I have set these repo's to TrustAll. With this setting, pacman is very much faster, but expac is not. I have tested expac locally the same signature levels and I find lags associated to the 3rd party repo to be removed.
Please add support for pacman.conf signature verification levels as a repo-specific setting. Thanks.
printf '' | expac -S '%n' - | wc -l
8814
Compare to pacman --print
:
% printf '' | pacman -Sddp -
error: argument '-' specified with empty stdin
A workaround is expac -S '%n' - ''
or xargs -r expac -S '%n' -
, assuming the user expects this behavior.
If cower-git
is installed on my system, the following command: expac -Q %v cower
outputs the version of cower-git
.
When using the latest expac (version 1-2), I'm unable to output correctly the %C (conflicts) and %R (replace) variables.
Examples: See
expac -S '%n %C'
and
expac -S '%n %R'
All other variables seems to be outputed as expected. Did I miss anything?
I currently have some code which outputs packages along with their dependencies in a format that can be used by tsort
. However this is slow because it appears I have to run expac
for each package in the list. E.g.
for dep in $(expac -S '%E' "$pkg"); do
printf '%s %s\n' "$pkg" "$dep"
done
It would be really nice if I could use expac -Sl '\n%n' '%n %E' "${packages[@]}"
instead where the string -l
accepts is formatter aware, e.g.:
teapot earnest %i master ~ expac -l '\n%n ' -S '%n %E' systemd
systemd acl
%n bash
%n dbus
%n iptables
%n kbd
%n kmod
%n hwids
%n libcap
%n libgcrypt
%n libsystemd
%n libidn
%n lz4
%n pam
%n libseccomp
%n util-linux
%n xz
However as you can see, the %n
is inserted literally.
This would make for example, doing cycle detection really simple:
expac -Sl '\n%n ' '%n %E' "$(pacman -Sql core)" | tsort
The other benefit of this would be to make building associative arrays easier too: (obviously substituting %n
with the appropriate expansion.)
% expac -l ' %n ' -S '%n %E' systemd
systemd acl %n bash %n dbus %n iptables ... # and so on.
The command-line syntax by repurposing -l
is perhaps a little less pretty but it does seem to get pretty close to providing this functionality.
What are your thoughts?
Git clone with latest commit a6e67c3
Traceback:
cc -std=c99 -g -pedantic -Wall -Wextra -Werror -march=x86-64 -mtune=generic -O2 -pipe -fstack-protector --param=ssp-buffer-size=4 -DVERSION="2-2-ga6e67c3" -D_FORTIFY_SOURCE=2 -Wl,-O1,--sort-common,--as-needed,-z,relro expac.c -lalpm -o expac
pod2man --section=1 --center="expac manual" --name="EXPAC" --release="expac 2-2-ga6e67c3" README.pod expac.1
expac.c: In function ‘format_optdep’:
expac.c:143:11: error: ignoring return value of ‘asprintf’, declared with attribute warn_unused_result [-Werror=unused-result]
asprintf(&out, "%s: %s", optdep->name, optdep->desc);
^
cc1: all warnings being treated as errors
make: *** [expac] Error 1
make: *** Waiting for unfinished jobs....
Some request return 2 or more results, and returning only the first one could greatly help in some situation.
I'd like to write a script that show current(outdated) and available version of outdated packages, with expac -Q %v and expac -S %v:
->
"kernel26 2.6.38.6-2 -> 2.6.38.7-1"
This is working fine as long as the testing repositories aren't enabled. However, "expac -S %v kernel26" return 2 results when the testing repo is enabled:
2.6.39-1
2.6.38.7-1
Thus, it is not possible to use expac in scripts with many input targets anymore, as there are no way to know which package gives what result, ie:
expac -S %v kernel26 flex less (all outdated packages)
2.6.39-1 (kernel, testing)
2.6.38.7-1 (kernel, core)
2.5.35-4 (flex, core)
443-2 (less, testing)
443-1 (less, core)
It is always possible to pipe individual package to awk in a loop, but only at a great cost of performance. Implementing a "-1 --just-one" option in order to show the first answer only (similarly to package-query) would be very useful.
Not sure why, but I'm seeing new errors when using a signed repo.
Below is the basic pacman and expac outputs. Pacman doesn't complain about the pipelight repo but expac does.
$ sudo pacman -Syy
:: Synchronizing package databases...
core 111.4 KiB 358K/s 00:00 [######################] 100%
extra 1610.8 KiB 889K/s 00:02 [######################] 100%
community 2.2 MiB 998K/s 00:02 [######################] 100%
pipelight 23.3 KiB 154K/s 00:00 [######################] 100%
pipelight.sig 543.0 B 0.00B/s 00:00 [######################] 100%
multilib 117.0 KiB 867K/s 00:00 [######################] 100%
$ pacman -Ssq cmake
cmake
extra-cmake-modules
$ expac -S -1 -v %n cmake
Failed to register repo pipelight: invalid or corrupted database (PGP signature)
cmake
Here is the relevant pacman.conf sections:
# By default, pacman accepts packages signed by keys that its local keyring
# trusts (see pacman-key and its man page), as well as unsigned packages.
SigLevel = Required DatabaseOptional
LocalFileSigLevel = Optional
#RemoteFileSigLevel = Required
...
[pipelight]
Server = http://repos.fds-team.de/stable/arch/$arch
Output of pacman-key --list-sigs
$ sudo pacman-key --list-sigs | grep Pipelight
uid Pipelight Dev Team (Package Builder) <[email protected]>
sig 3 5DC2D5CA 2014-02-13 Pipelight Dev Team (Package Builder) <[email protected]>
sig 5DC2D5CA 2014-02-13 Pipelight Dev Team (Package Builder) <[email protected]>
and pacman-key --list-keys
$ sudo pacman-key --list-keys | grep Pipelight
uid Pipelight Dev Team (Package Builder) <[email protected]>
Is this expected performance of expac and there's an issue with my setup? Have I missed something? Thanks.
expac 2-1
pacman v4.1.0-2
libalpm v8.0.0
Commit a6e67c3 causes garbled output when using the new optdep format.
$ expac '%O' i3-wm
��� ��� 0�
Reverting solves the issue although all optdeps are displayed on a single line. (Expected newline for each item.)
$ expac '%O' i3-wm
dmenu: As menu. i3lock: For locking your screen. i3status: To display systeminformation with a bar.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.