fluent-package-builder (formerly known as td-agent-builder, it was renamed at Aug, 2021) is a new build system for td-agent which aims to replace the traditional build system omnibus-td-agent since it has several problems due to Omnibus's limitations.
NOTE: Discussed why re-branding is required Rebranding td-agent-builder
td-agentwas renamed tofluent-package- The content of
fluent-packagewas changed to install under/opt/fluent /usr/sbin/td-agentand/usr/sbin/td-agent-gemwas changed to/usr/sbin/fluentdand/usr/sbin/fluent-gem- Changed the path of example default configuration file to
/opt/fluent/share/fluentd.conf
- The content of
- Debian 12 (bookworm) has been supported
- Removed Ubuntu 16.04 (xenial), Ubuntu 18.04 (bionic) support
- Amazon Linux 2023 has been supported
- Introduced new package signing key. The new key will be used in the future release. we still use using old signing key for a while
fluentd-apt-sourcewas renamed tofluent-apt-sourcedeb package for maintaining apt-line and keyring
- Use system libraries: e.g. openssl
- Remove libraries for 3rd party gems: e.g. postgresql
- Remove
embeddeddirectory by omnibus- Use
/opt/td-agent/bin/fluent-catinstead of/opt/td-agent/embedded/bin/fluent-cat
- Use
- Update core components: ruby, jemalloc and more
- Add td-agent-apt-source deb package for maintaining apt-line and keyring
- Download it and install by
apt install, then you can install td-agent viaapt install td-agent. - td-agent-apt-source deb package is created to install td-agent easily. Currently it points to td-agent 4 apt-line.
- Download it and install by
See also this issue for omnibus problems.
- Any host OS that is supported by Docker
- Debian buster or Ubuntu 18.04 are recommended
- Docker
- Ruby 2.5 or later
- Bundler 2.2.0 or later
- Git
- Windows OS (10 Pro or 2019 are verified)
- Docker Desktop for Windows
- You need to switch to "Windows containers" before using it.
- RubyInstaller 2.5 or later.
- Bundler 2.2.0 or later
- Git for Windows
After installed above software, you need to enable additional features from powershell (as admin).
dism.exe /online /enable-feature /featurename:Microsoft-Hyper-V /all /norestart
dism.exe /online /enable-feature /featurename:Containers /all /norestart
Then restart Windows.
- macOS 10.15 (Catalina)
- Ruby 2.5 or later
- Bundler 2.2.0 or later
- Builder gem
- Git
- CMake
- Rust to enable Ruby's YJIT feature
% rake yum:buildBy default, yum repositories for following platforms will be built under fluent-package/yum/repositories/ directory:
- RHEL/CentOS 7 (x86_64)
- RHEL/CentOS 8 (x86_64) - Built on Rocky Linux 8
- RHEL/CentOS 9 (x86_64) - Built on AlmaLinux 9
- Amazon Linux 2 (x86_64)
- Amazon Linux 2023 (x86_64)
You can choose target platforms by YUM_TARGETS environment variable like this:
% rake yum:build YUM_TARGETS="centos-7,rockylinux-8,almalinux-9"You can find other supported platforms under fluent-package/yum directory.
You can also build packages for AArch64 platforms like this:
% rake yum:build YUM_TARGETS="amazonlinux-2023-aarch64"But if you use older GNU/Linux platforms (e.g. Ubuntu 18.04 or before) as your host OS, you need to copy qemu-aarch64-static into the base directory of the target:
% export TARGET_BASE="centos-8"
% sudo apt install qemu-user-static
% cd /path/to/fluent-package-builder
% cp /usr/bin/qemu-aarch64-static fluent-package/yum/${TARGET_BASE}
% rake yum:build YUM_TARGETS="${TARGET_BASE}-aarch64"You can also build packages for ppc64le platform like this:
% rake yum:build YUM_TARGETS="centos-8-ppc64le"But if you use older GNU/Linux platforms (e.g. Ubuntu 18.04 or before) as your host OS, you need to copy qemu-ppc64le-static into the base directory of the target:
% export TARGET_BASE="centos-8"
% sudo apt install qemu-user-static
% cd /path/to/fluent-package-builder
% cp /usr/bin/qemu-ppc64le-static fluent-package/yum/${TARGET_BASE}
% rake yum:build YUM_TARGETS="${TARGET_BASE}-ppc64le"% rake apt:buildBy default, apt repositories for following platforms will be built under fluent-package/apt/repositories/ directory:
- Debian 11 "Bullseye" (x86_64)
- Debian 12 "Bookworm" (x86_64)
- Ubuntu 20.04 LTS "Focal Fossa" (x86_64)
- Ubuntu 22.04 LTS "Jammy Jellyfish" (x86_64)
You can choose target platforms by APT_TARGETS environment variable like this:
% rake apt:build APT_TARGETS="debian-bookworm,ubuntu-jammy"You can find other supported platforms under fluent-package/apt directory.
You can also built packages for AArch64 platforms like this:
% rake apt:build APT_TARGETS="ubuntu-jammy-arm64"But if you use older GNU/Linux platforms (e.g. Ubuntu 18.04 or before) as your host OS, you need to copy qemu-aarch64-static into the base directory of the target:
% export TARGET_BASE="ubuntu-bionic"
% sudo apt install qemu-user-static
% cd /path/to/fluent-package-builder
% cp /usr/bin/qemu-aarch64-static fluent-package/apt/${TARGET_BASE}
% rake apt:build APT_TARGETS="${TARGET_BASE}-arm64"% rake msi:buildA fluent-package-${version}-x64.msi package will be built under fluent-package/msi directory.
You can use with MSYS2 for C extension gem building.
MSI included Ruby can detect MSYS2 environment. So, you can install C extension included gem with MSYS2.
e.g.)
Prepare C extension gem building environment:
cmd> ridk install 2
...
cmd> ridk install 3Install gem via ridk exec fluent-gem install:
cmd> ridk exec fluent-gem install winevt_c% sudo mkdir /opt/fluent
% sudo chown $(whoami) /opt/fluent
% rake dmg:selfbuildA fluent-package-${version}.dmg package will be built under fluent-package/dmg directory.
GitHub Actions' built package is ready to run on macOS 10.15 (Catalina).
Be sure to permit to be authorized for for assistive access. In System Preferences > Security & Privacy > Privacy > Accessibility, you should permit Terminal.app there.
NOTE: Since authorization is at the application level on Terminal.app, it allows any script run from Terminal.app to perform GUI scripting.
- Edit fluent-package/config.rb to choose Ruby & Fluentd versions
- Edit Gemfile and update .lock files
cd fluent-package && rake lockfile:update
- Bump up the versions of rpm & deb packages by the following command:
% cd fluent-package
% rake version:update
% git diff # Check the change log
% git commit -a
- Build packages
% rake apt:build
% rake yum:build
% rake msi:build
It assumes that Gemfile works with Bundler's multiplatform feature, so bundler must be 2.2.0 or later.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent