Code Monkey home page Code Monkey logo

combirepo's Introduction

Combirepo

Combirepo is a tool for combining several repositories into firmware. It is used two build the image in which some packages are taken from one repository and other packages are taken from another. Some packages mean that combirepo is able to detect forward and backward dependencies between RPM packages and installs the package with its dependencies depending on the parameters that user gives to it.

During the run of combirepo the so-called combined repository is generated. After that it is passed as an argument to the usual mic image creator that builds the firmware.

Intended to be used together with OBS-produced repositories. Refer to man 1 combirepo and this guide for details.

Installation

combirepo has several dependencies. On Ubuntu OS, you can install them as follows:

sudo apt-get install yum python-igraph python-iniparse zlib1g-dev libxml2-dev mic createrepo python-pip qemu-user-static python-configparser python-all dh-make

Use setup.py script for building a deb package:

$ python -m pip install stdeb --user
$ ./setup.py sdist
$ ./setup.py --command-packages=stdeb.command bdist_deb

After that it can be installed as follows:

$ sudo dpkg -i deb_dist/combirepo_*.deb

Another way is to use the Docker build instead:

$ ./setup.py sdist
$ docker build -t combirepo ./
$ docker run -Pti combirepo /bin/bash

Terminology

This guide uses specific terminology:

  • Marked repository means featured repository, for instance, a repository with packages that are built with sanitizers,
  • non-marked repository is an ordinary repository used for the build of production or debugging images.

Usage

combirepo acts as follows:

  1. Download repositories.
  2. Generate (or re-generate) repository metadata in a proper way.
  3. Find kickstart files (with *.ks extension) inside the repositories (they are usually contained in the package image-configurations).
  4. Build the dependency graphs for packages.
  5. Prepare the list of packages based on specified options (forward, backward dependencies, single or excluded packages) that should be taken from marked repository.
  6. Create the combined repository, taking marked packages from marked repository and others from the ordinary one.
  7. Run the mic to build the image.

There are two ways to configure combirepo:

  • config file (recommended),
  • command line options (not recommended), that are described in manpages (man 1 combirepo) and with --help message.

If both command line options and config options with the same names are specified, the command line will have the higher priority.

If the config file is prepared, you can just run the command:

$ combirepo

and it will do everything. Also it's possible to set all parameters from command line as follows:

combirepo \
    -k my_ks_file.ks \
    -f PACKAGE_NAME \
    -A ARCHITECTURE \
    name-repo /path/to/non/sanitized/repo /path/to/sanitized/repo

where name-repo is the name of repository as specified in the kickstart file. Such names are given with repo command in kickstart files:

repo --name=Tizen-base --baseurl=http://...../tizen-rsa/tizen-2.2.1-vd-4.8/standard/latest/repos/base/armv7l/packages/ --ssl_verify=no

Here the Tizen-base is this name. In case when the kickstart file contains several repo commands the user should specify a triplet of {name, path to original repository, path to marked repository} for each such command. This usually happens when the image is built from several repositories which are generated from different OBS projects (usually Base and Main/Mobile). In this case you should specify triplets as follows:

combirepo \
    -k my_ks_file.ks \
    -f PACKAGE_NAME \
    -A ARCHITECTURE \
    name-repo1 /path/to/non/sanitized/repo1 /path/to/sanitized/repo1 \
    name-repo2 /path/to/non/sanitized/repo2 /path/to/sanitized/repo2

Marking packages

Here are combirepo features that control the set of package that must be marked:

Feature Command line option config file option Description
Mark packages explicitly -s --single single_packages Explicitly specify the list of packages that should be marked.
Greedy marking -g --greedy greedy Mark as many packages, as possible.
Mark forward dependencies -f --forward forward_packages Mark forward dependencies of marked packages, i.e. those that they depend on.
Mark backward dependencies -b --backward backward_dependencies Mark backward dependencies (a.k.a. dependees) of marked packages, i.e. those that depend on marked packages.
Unmark packages -e --exclude excluded_packages Explicitly exclude packages from marked set.

config files

Default config file is located in ~/.combirepo.conf, but it's possible to specify another one with -c --config option. Here is an example of config file:

# The general section must always present in the config file:
[general]
# The name of profile to be used:
profile = asan317
# Here you can specify any other name you want.
# Usually combirepo stores its cache and temporaries in /var/tmp/combirepo
# directory. You can change this as follows:
# tmp_dir = /my/tmp/dir
 
# The profile usually looks as follows:
[asan317]
# Credentials to the download server:
user = <your user name>
password = <your password, after 1st run it will replaced with encoded one>
 
# 1. Basic parameters:
# The architecture of the image (see option -A in man page):
architecture = armv7l
# First, list the repositories that are contained in the profile:
repos = repo_mobile, repo_base
# Service packages are packages that are additionally installed to the image
# (see option -S in man page):
service_packages = libasan
# Also supplementary repository is specified as the origin of service packages
# (see option -u in man page):
repo_supplementary = http://...../download/live/devel:/2.4:/Mobile:/ASAN/standard/
# Use special kickstart file for the build (see option -k in man page):
# Usually combirepo finds kickstart file in the repository itself, so you need not
# to specify the kickstart name here.
#kickstart = /path/to/tizen-2.4-mobile_20150831.5_mobile_target-TM1.ks
 
# 2. Sanitizing features:
# Forward-sanitized packages (see option -f in man page) a comma separated string:
forward_packages = browser
# Backward-sanitized packages (see option -b in man page) a comma separated string:
backward_packages = efl, libzypp
# Single-sanitized packages (see option -s in man page) a comma separated string:
single_packages = alarm-server
# Packages that are excluded from sanitizing (see option -e in man page) a comma separated string:
excluded_packages = libjpeg, libpng
# Greedy mode, i. e. sanitize all possible packages (see option -g in man page):
# greedy = 1     # [Uncomment this line to enable greedy build]
 
# 3. Additional features:
# Mirror mode, i. e. use packages from original repository if they have been
# added to the sanitized list, but do not present in the marked repository
# (see option -m in man page):
mirror = 1
 
# 4. Resolving of dependency problems:
# Sometimes repositories contain different versions or even different builds of the
# same package. To handle this case you can specify which of them should be used:
# with higher version/release numbers (big) or with lower ones (small)
preferring_strategy = big
# Also some packages can conflict with each other because of erroneous RPM builds,
# and it is needed to specify which of them should be used:
#preferable_packages = coregl coregl-devel taglib gmime camera-interface-sprd-sc7730 heremaps-engine perl perl-x86-arm python cross-armv7l-gcc-accel-x64-armv7l
 
# Repository descriptions:
 
[repo_mobile]
# The name of repository as specified in the kickstart file:
name = 2.4-mobile-target-TM1
# The URL of the original repository:
url_orig = http://...../download/live/Tizen:/2.4:/Mobile/target-TM1/
# The URL of the sanitized repository:
url_marked = http://...../download/live/devel:/2.4:/Mobile:/ASAN:/Mobile/target-TM1/
 
[repo_base]
# The name of repository as specified in the kickstart file:
name = 2.4-base
# The URL of the original repository:
url_orig = http://...../download/live/Tizen:/2.4:/Base/standard/
# The URL of the sanitized repository:
url_marked = http://....../download/live/devel:/2.4:/Mobile:/ASAN:/Base/standard/

License

The application is distributed under GPLv2 license, refer the LICENSE file.

combirepo's People

Contributors

a-lyupa avatar rayslava avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

combirepo's Issues

Support for customising `build.sub_commands` directly from `distutils` has been deprecated and is about to be removed from `setuptools`

The following feature in setuptools has been deprecated for almost 2 years and is about to be removed:

https://github.com/pypa/setuptools/blob/1ed759173983656734c3606e9c97a348895e5e0c/setuptools/command/build.py#L13-L27

It might be a good idea to import build directly from setuptools for the following code:

build.sub_commands.append(('build_manpage', None))

(build is available directly from setuptools, starting on version v62.4.0)

Versions conflict

There are different versions of libmm-sound in san- and unified- repos:

./combireposQZlEvrepository/armv7l/libmm-sound-sdk-devel-0.12.22-37.1.armv7l.rpm
./combireposQZlEvrepository/armv7l/libmm-sound-devel-0.12.22-37.1.armv7l.rpm
./combireposQZlEvrepository/armv7l/libmm-sound-0.12.22-37.1.armv7l.rpm
./combireposQZlEvrepository/armv7l/libmm-sound-tool-0.12.22-37.1.armv7l.rpm
./combirepo7mmhr5repository/armv7l/libmm-sound-tool-0.12.19-2.9.armv7l.rpm
./combirepo7mmhr5repository/armv7l/libmm-sound-0.12.19-2.9.armv7l.rpm
./combirepo7mmhr5repository/armv7l/libmm-sound-sdk-devel-0.12.19-2.9.armv7l.rpm
./combirepo7mmhr5repository/armv7l/libmm-sound-devel-0.12.19-2.9.armv7l.rpm`

--preferring-strategy big is set.

In case of --mirror option an image is built successfully but in case of --greedy there is an error because of different versions of the same packages:

ERROR:root:Found 7 packages with different version numbers!
ERROR:root: libmm-sound                    0.12.22 0.12.19
...
ERROR:root:Please go and rebuild them!

@ilya-palachev , could you comment this case?

Skip rpmrebuild

@ilya-palachev ,
Is it ok to skip rpmrebuild stage?
What is the correct way to build and install it? I got rpmrebuild.tar with 0 byte size.
Why yum is set instead of zypp with --disable-rpm-patching option enabled?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.