Code Monkey home page Code Monkey logo

buildbot's Introduction

OpenWrt logo

OpenWrt Project is a Linux operating system targeting embedded devices. Instead of trying to create a single, static firmware, OpenWrt provides a fully writable filesystem with package management. This frees you from the application selection and configuration provided by the vendor and allows you to customize the device through the use of packages to suit any application. For developers, OpenWrt is the framework to build an application without having to build a complete firmware around it; for users this means the ability for full customization, to use the device in ways never envisioned.

Sunshine!

Download

Built firmware images are available for many architectures and come with a package selection to be used as WiFi home router. To quickly find a factory image usable to migrate from a vendor stock firmware to OpenWrt, try the Firmware Selector.

If your device is supported, please follow the Info link to see install instructions or consult the support resources listed below.

An advanced user may require additional or specific package. (Toolchain, SDK, ...) For everything else than simple firmware download, try the wiki download page:

Development

To build your own firmware you need a GNU/Linux, BSD or macOS system (case sensitive filesystem required). Cygwin is unsupported because of the lack of a case sensitive file system.

Requirements

You need the following tools to compile OpenWrt, the package names vary between distributions. A complete list with distribution specific packages is found in the Build System Setup documentation.

binutils bzip2 diff find flex gawk gcc-6+ getopt grep install libc-dev libz-dev
make4.1+ perl python3.7+ rsync subversion unzip which

Quickstart

  1. Run ./scripts/feeds update -a to obtain all the latest package definitions defined in feeds.conf / feeds.conf.default

  2. Run ./scripts/feeds install -a to install symlinks for all obtained packages into package/feeds/

  3. Run make menuconfig to select your preferred configuration for the toolchain, target system & firmware packages.

  4. Run make to build your firmware. This will download all sources, build the cross-compile toolchain and then cross-compile the GNU/Linux kernel & all chosen applications for your target system.

Related Repositories

The main repository uses multiple sub-repositories to manage packages of different categories. All packages are installed via the OpenWrt package manager called opkg. If you're looking to develop the web interface or port packages to OpenWrt, please find the fitting repository below.

  • LuCI Web Interface: Modern and modular interface to control the device via a web browser.

  • OpenWrt Packages: Community repository of ported packages.

  • OpenWrt Routing: Packages specifically focused on (mesh) routing.

  • OpenWrt Video: Packages specifically focused on display servers and clients (Xorg and Wayland).

Support Information

For a list of supported devices see the OpenWrt Hardware Database

Documentation

Support Community

  • Forum: For usage, projects, discussions and hardware advise.
  • Support Chat: Channel #openwrt on oftc.net.

Developer Community

License

OpenWrt is licensed under GPL-2.0

buildbot's People

Contributors

aloisklink avatar ansuel avatar aparcar avatar bwhitten avatar dependabot[bot] avatar f00b4r0 avatar guidosarducci avatar jow- avatar lynxis avatar nbd168 avatar thess avatar ynezz avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

buildbot's Issues

phase1: rework dlprune Step

Note to self, currently fails with:

find dl/ -mindepth 1 -atime +15 -delete -print
 in dir /builder/master_bcm4908_generic/build (timeout 1200 secs)
 watching logfiles {}
 argv: b'find dl/ -mindepth 1 -atime +15 -delete -print'
 using PTY: False
find: cannot delete ‘dl/ath10k-ct-firmware-2020-11-08’: Directory not empty
program finished with exit code 1
elapsedTime=0.012581

Furthermore the -delete -print order is wrong: deleted files aren't printed.

phase1: builds on clean workers always fail on gitverify step

Following build step utilizing Git() stock buildbot build step when performed on a clean worker:

factory.addStep(Git(
		name = "git",
		repourl = repo_url,
		mode = 'full',
		method = 'fresh',
		locks = NetLockDl,
		haltOnFailure = True,
	))

leaves the cloned Git repository in the detached HEAD state and consecutive gitverify check thus always fail:

git rev-parse --abbrev-ref HEAD | grep -vxqF HEAD || git show-ref --tags --dereference 2>/dev/null | sed -ne "/^$(git rev-parse HEAD) / { s|^.*/||; s|\\^.*||; p }" | grep -qE "^v[0-9][0-9]\\."

So every first build on ephemeral VPS results in a failure, each second build then goes fine as Git() checkouts the branch correctly.

Output of the Git() step on a fresh worker, 1st build (BAD case)

git --version
 in dir /builder/master_bcm47xx_mips74k/build (timeout 1200 secs)
...
git version 2.30.2
program finished with exit code 0
...
stat: No such file or directory: /builder/master_bcm47xx_mips74k/build/.buildbot-patched
program finished with exit code 0
git clone --branch master https://git.openwrt.org/openwrt/openwrt.git . --progress
 in dir /builder/master_bcm47xx_mips74k/build (timeout 1200 secs)
...
program finished with exit code 0
elapsedTime=17.474384
git checkout -f 466be0612aaed4034e15095b9fc63ce4cd99b87a
 in dir /builder/master_bcm47xx_mips74k/build (timeout 1200 secs)
...
Note: switching to '466be0612aaed4034e15095b9fc63ce4cd99b87a'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.
...
HEAD is now at 466be0612a e2fsprogs: fix compilation with musl 1.2.4
program finished with exit code 0
elapsedTime=0.453183
git rev-parse HEAD
...
466be0612aaed4034e15095b9fc63ce4cd99b87a
program finished with exit code 0
elapsedTime=0.008105

Output of the Git() step on a fresh worker, 2nd build (GOOD case)

git --version
 in dir /builder/master_malta_be/build (timeout 1200 secs)
...
git version 2.30.2
program finished with exit code 0
elapsedTime=0.007659
program finished with exit code 2
stat: No such file or directory: /builder/master_malta_be/build/.buildbot-patched
program finished with exit code 0
git clean -f -f -d -x
 in dir /builder/master_malta_be/build (timeout 1200 secs)
...
program finished with exit code 0
elapsedTime=0.048432
git cat-file -e 466be0612aaed4034e15095b9fc63ce4cd99b87a
 in dir /builder/master_malta_be/build (timeout 1200 secs)
...
program finished with exit code 0
elapsedTime=0.007559
git checkout -f 466be0612aaed4034e15095b9fc63ce4cd99b87a
 in dir /builder/master_malta_be/build (timeout 1200 secs)
...
HEAD is now at 466be0612a e2fsprogs: fix compilation with musl 1.2.4
program finished with exit code 0
elapsedTime=0.049798
git checkout -B master
 in dir /builder/master_malta_be/build (timeout 1200 secs)
...
Switched to and reset branch 'master'
Your branch is up to date with 'origin/master'.
program finished with exit code 0
elapsedTime=0.052273
git rev-parse HEAD
...
466be0612aaed4034e15095b9fc63ce4cd99b87a
program finished with exit code 0
elapsedTime=0.007634

misc: sync table of hardware to downloads.openwrt.org

This idea is more of a service to downstream projects.

Apart from the various build profile files generated by OpenWrt buildroot, the community-curated Table of Hardware is a super useful resource, which helps downstream projects customize their builds based on device features. Its CSV dump is available at https://openwrt.org/_media/toh_dump_tab_separated.gz

It would be quite neat to have the CSV dump included in downloads.openwrt.org - we run mirrors for that within our mesh network @freifunk-berlin and on little battery-powered SBCs. That helps us having everything needed to build device firmware even in rough/disconnected/broken physical locations of the network, and saves us bandwidth towards the internet gateway locations.

We already started syncing the dump to our own release server firmware.berlin.freifunk.net recently, but I think this could be useful for others as well.

P.S. love your hard buildbot work in the past weeks and months! I've been grinding buildbot too, for our own builds :-)

phase1: IGNORE_ERRORS hides package build issues

Problem

Following IGNORE_ERRORS constructs in https://github.com/openwrt/buildbot/blob/master/phase1/master.cfg#L1172

make package/compile IGNORE_ERRORS=n m
make target/compile IGNORE_ERRORS=n m

are effectively hiding the build issues, leading to hardly spottable issues such as recent kmod-ltql-vmmc build breakage caused by default GCC version bump, where the buildbot step succeeds, but some target's optional kernel module (or package) might fail to build. Contrary to phase2, there are no faillogs generated on phase1 for failing packages, thus we're completely unaware about the breakage.

For a complete overview see the discussion here openwrt/openwrt#15064

Possible solutions

  1. Get rid of IGNORE_ERRORS=n m and make the build fails, thus making sure, that all packages are building
  2. Provide faillogs/images (or faillogs-images) for phase1 package build issues as we've for phase2

I'm more inclined towards 1. as there is quite good CI test coverage on GitHub so I believe, that when something fails to build, like in the recent kmod-ltql-vmmc case, it should be spotted and fixed like in openwrt/openwrt#15065 and openwrt/openwrt#15098.

phase1: 23.05: qualcommax should not be built

qualcommax is a renamed ipq807x target with ipq807x now as a subtarget, and its only present in master but buildbots are trying to build it for 23.05 as well.

Builder: https://buildbot.staging.openwrt.org/images/#/builders/167

This should not be happening as output of ./scripts/dump-target-info.pl targets does not list qualcommax on 23.05:

apm821xx/nand powerpc_464fp
apm821xx/sata powerpc_464fp
armsr/armv7 arm_cortex-a15_neon-vfpv4
armsr/armv8 aarch64_generic
at91/sam9x arm_arm926ej-s
at91/sama5 arm_cortex-a5_vfpv4
at91/sama7 arm_cortex-a7_vfpv4
ath79/generic mips_24kc
ath79/mikrotik mips_24kc
ath79/nand mips_24kc
ath79/tiny mips_24kc
bcm27xx/bcm2708 arm_arm1176jzf-s_vfp
bcm27xx/bcm2709 arm_cortex-a7_neon-vfpv4
bcm27xx/bcm2710 aarch64_cortex-a53
bcm27xx/bcm2711 aarch64_cortex-a72
bcm47xx/generic mipsel_mips32
bcm47xx/legacy mipsel_mips32
bcm47xx/mips74k mipsel_74kc
bcm4908/generic aarch64_cortex-a53
bcm53xx/generic arm_cortex-a9
bcm63xx/generic mips_mips32
bcm63xx/smp mips_mips32
bmips/bcm6318 mips_mips32
bmips/bcm63268 mips_mips32
bmips/bcm6328 mips_mips32
bmips/bcm6358 mips_mips32
bmips/bcm6362 mips_mips32
bmips/bcm6368 mips_mips32
gemini/generic arm_fa526
imx/cortexa7 arm_cortex-a7_neon-vfpv4
imx/cortexa9 arm_cortex-a9_neon
ipq40xx/chromium arm_cortex-a7_neon-vfpv4
ipq40xx/generic arm_cortex-a7_neon-vfpv4
ipq40xx/mikrotik arm_cortex-a7_neon-vfpv4
ipq806x/chromium arm_cortex-a15_neon-vfpv4
ipq806x/generic arm_cortex-a15_neon-vfpv4
ipq807x/generic aarch64_cortex-a53
kirkwood/generic arm_xscale
lantiq/ase mips_mips32
lantiq/xway mips_24kc
lantiq/xway_legacy mips_24kc
layerscape/armv7 arm_cortex-a7_neon-vfpv4
layerscape/armv8_64b aarch64_generic
malta/be mips_24kc
mediatek/filogic aarch64_cortex-a53
mediatek/mt7622 aarch64_cortex-a53
mediatek/mt7623 arm_cortex-a7_neon-vfpv4
mediatek/mt7629 arm_cortex-a7
mpc85xx/p1010 powerpc_8548
mpc85xx/p1020 powerpc_8548
mpc85xx/p2020 powerpc_8548
mvebu/cortexa53 aarch64_cortex-a53
mvebu/cortexa72 aarch64_cortex-a72
mvebu/cortexa9 arm_cortex-a9_vfpv3-d16
mxs/generic arm_arm926ej-s
octeon/generic mips64_octeonplus
octeontx/generic aarch64_generic
oxnas/ox820 arm_mpcore
pistachio/generic mipsel_24kc_24kf
ramips/mt7620 mipsel_24kc
ramips/mt7621 mipsel_24kc
ramips/mt76x8 mipsel_24kc
ramips/rt288x mipsel_24kc
ramips/rt305x mipsel_24kc
ramips/rt3883 mipsel_74kc
realtek/rtl838x mips_4kec
realtek/rtl839x mips_24kc
realtek/rtl930x mips_24kc
realtek/rtl931x mips_24kc
rockchip/armv8 aarch64_generic
sifiveu/generic riscv64_riscv64
sunxi/cortexa53 aarch64_cortex-a53
sunxi/cortexa7 arm_cortex-a7_neon-vfpv4
sunxi/cortexa8 arm_cortex-a8_vfpv3
tegra/generic arm_cortex-a9_vfpv3-d16
x86/64 x86_64
x86/generic i386_pentium4
x86/geode i386_pentium-mmx
x86/legacy i386_pentium-mmx
zynq/generic arm_cortex-a9_neon

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.