zstd 支持 about lilac HOT 19 CLOSED

Official devtools moved to [extra] [1], devtools-archlinuxcn updated [2], the build server upgraded to the latest devtools-archlinuxcn, and in the following automatic build starting at 2020/1/7 17:22:53 (UTC+8), no packages failed due to the new zstd compression format. I believe everything has been done 🎉

[1] https://git.archlinux.org/svntogit/packages.git/commit/?h=packages/devtools&id=88c9964e88ac5ba7c1e5dc5a9c167fb640bc0df0
[2] archlinuxcn/repo@3e8b626

from lilac.

有的。当时的邮件列表里有这个

https://lists.archlinux.org/pipermail/arch-dev-public/2019-March/029542.html

是zstd多核和xz的单核对比。

如果是zstd和xz的多核顶级参数对比的话，我做过一次。我记得压缩上空间差不太多，zstd比xz要慢一些(cuda好像是xz 2:00 zstd 2:30这种吧)。解压上可以参考那个列表里的，zstd比xz快太多。

xz多核主要是被喷有不可复现的问题。主要是指xz单核和多核结果不一致（但是单核和多核独立都是可以复现的）。

from lilac.

Update: Arch developers plan to switch official packages to zstd in the end of this month. See https://lists.archlinux.org/pipermail/arch-dev-public/2019-December/029752.html for details.

I guess the switch will happen in devtools (/usr/share/devtools/makepkg-x86_64.conf). I propose to do the same switch in devtools-archlinuxcn then.

from lilac.

nginx config has been updated.

from lilac.

有 pacman 使用 xz 和 zstd 的对比数据么？
我不是很想换，因为 xz 被支持得更好。

from lilac.

好耶～ zstd -c -T0 --ultra -20 - 这个大概是最合适的了。

接下来是测试和实现。archrepo2 和 lilac 都要做相应的修改。

from lilac.

makepkg 看起来不支持呢……

from lilac.

我打了个unzip没问题啊。。。

  -> 正在压缩软件包...
==> 警告： '' 是无效的压缩包扩展名。
==> 正在离开 fakeroot 环境。
==> 完成创建：unzip 6.0-13 (2019年06月20日 星期四 13时33分36秒)

除了会出这么个警告。

我在原来的/etc/makepkg.conf下加了

PKGEXT='.pkg.tar.zst'
COMPRESSZST=(zstd -c -T0 --ultra -21 -)

出来的包我也装了一下，看起来没问题

from lilac.

現在的 makepkg 改了 PKGEXT='.pkg.tar.zst' 之後打出來的文件貌似是沒壓縮的 tar 。
需要等這個 commit 包含在新一版的 pacman https://git.archlinux.org/pacman.git/commit/?id=bae74c8e9e69b4f4e1a235eef21b9b27fb14aff0
以及 repo-add 的支持在 https://git.archlinux.org/pacman.git/commit/?id=7afe51171fe063bf3031cc68fc8c7ac914a01de2

from lilac.

現在的 makepkg 改了 PKGEXT='.pkg.tar.zst' 之後打出來的文件貌似是沒壓縮的 tar 。

原来如此

from lilac.

所以我们是不是装pacman-git就可以搞这个了？

从用户使用的角度有需要等待的么？

from lilac.

我试着用pacman-git打包了一个unzip-6.0-13-x86_64.pkg.tar.zst然后用pacman装，貌似没什么问题？

from lilac.

pyalpm, even pyalpm-git, is not compatible with pacman-git yet, and apparently pyalpm is used in many scripts in archlinuxcn.

EDIT: discussions about pacman-git and pyalpm-git can be found on AUR https://aur.archlinux.org/packages/pyalpm-git

EDIT 2: namcap relies on pyalpm, too. Using pacman-git in chroots may result in flooded lilac logs.

from lilac.

这样。。。

from lilac.

Official devtools has been updated to use zstd as the default compression format for packages. Also, archlinuxcn/emacs-git has been packaged with zstd for more than a month (since [1]), so I'm confident that zstd will work well for [archlinuxcn]. How about doing the switch now?

[1] https://aur.archlinux.org/cgit/aur.git/commit/PKGBUILD?h=emacs-git&id=52b9930b3277ce67627b37b7f8e605e54293c625

from lilac.

What needs to be done for this? Just upgrade devtools?

from lilac.

devtools-archlinuxcn needs to be rebased against the latest devtools first.

/cc @farseerfc

from lilac.

What needs to be done for this? Just upgrade devtools?

Searching 'pkg.tar.xz' in archlinuxcn repos, I found another place to update: https://github.com/archlinuxcn/ansible-playbooks/blob/master/roles/repo/files/nginx-repo.conf - *.pkg.tar.zst should have longer expiration time, too?

from lilac.

Official devtools has been updated to use zstd as the default compression format for packages.

现在还在 [testing]

from lilac.