rock-core / autoproj Goto Github PK

It would be great if autoproj could handle different remotes. This is very useful when cloning repos. Not exactly shure how this would work. The problem is that currently I have to change the remote manually when I want to change the clone back to the original repo.

acd is however not currently exportable for all shell types

autoproj update should have a fast path, if the pinned commit is already locally available, not to ask the remote.

Currently autoproj update does not update the remote, but will complain that it cannot find the commit specified in the versions.

It fails e.g. in load_autoprojrc with

/mnt/jenkins/home/jobs/Build Cache/workspace/dev/.gems/gems/autoproj-1.10.2/lib/autoproj/cmdline.rb:192:in `home': couldn't find HOME environment -- expanding `~' (ArgumentError)
    from /mnt/jenkins/home/jobs/Build Cache/workspace/dev/.gems/gems/autoproj-1.10.2/lib/autoproj/cmdline.rb:192:in `load_autoprojrc'
    from /mnt/jenkins/home/jobs/Build Cache/workspace/dev/.gems/gems/autoproj-1.10.2/lib/autoproj/cmdline.rb:1599:in `bootstrap'
    from /mnt/jenkins/home/jobs/Build Cache/workspace/dev/.gems/gems/autoproj-1.10.2/bin/autoproj-bootstrap:6:in `block in <top (required)>'
    from /mnt/jenkins/home/jobs/Build Cache/workspace/dev/.gems/gems/autoproj-1.10.2/lib/autoproj/cmdline.rb:1992:in `block in report'
    from /mnt/jenkins/home/jobs/Build Cache/workspace/dev/.gems/gems/autobuild-1.7.11/lib/autobuild/reporting.rb:278:in `report'
    from /mnt/jenkins/home/jobs/Build Cache/workspace/dev/.gems/gems/autoproj-1.10.2/lib/autoproj/cmdline.rb:1991:in `report'
    from /mnt/jenkins/home/jobs/Build Cache/workspace/dev/.gems/gems/autoproj-1.10.2/bin/autoproj-bootstrap:4:in `<top (required)>'
    from /mnt/jenkins/home/jobs/Build Cache/workspace/dev/.gems/gems/autoproj-1.10.2/bin/autoproj:41:in `load'
    from /mnt/jenkins/home/jobs/Build Cache/workspace/dev/.gems/gems/autoproj-1.10.2/bin/autoproj:41:in `<top (required)>'
    from /mnt/jenkins/home/jobs/Build Cache/workspace/dev/.gems/bin/autoproj:23:in `load'
    from /mnt/jenkins/home/jobs/Build 

calling autoproj update results in the following error message:

autoproj-1.13.1/lib/autoproj/ops/snapshot.rb:169:in current_import_state': undefined methodshow' for nil:NilClass (NoMethodError)

I looked into the code and I think it cannot be assumed that main.importer is returning an object.

The vhs which has no importer is (the git is a clone from another git on the hard disk):
local:/var/lib/jenkins/workspace/ropencv_gem/dev/autoproj

same propblem here:
gems/autoproj-1.13.1/lib/autoproj/ops/snapshot.rb:191:in save_import_state': undefined methodgit_dir' for nil:NilClass (NoMethodError)

In ruby 1.9, the versions file written by autoproj versions has entries like this:

tag: !binary |-
c3RhYmxlLTIwMTQwOA==

instead of

tag: rock1408

same goes for the commit

During autoproj udpate the autoproj gem got updated. The update of the project fails with the following error:

wirkus@MWIRKUS-U:BesMan-orogen_loaders$ autoproj update
/media/data/Development/BesMan-orogen_loaders/.gems/gems/autobuild-1.8.0/lib/autobuild/environment.rb:262: warning: Insecure world writable dir /media/data/Development/BesMan-orogen_loaders/ in PATH, mode 040777
  How should I interact with gitorious.org (git, http or ssh): git
  How should I interact with github.com (git, http or ssh): git
  How should I interact with git.hb.dfki.de (git, http or ssh): ssh
  How should I interact with git.hb.dfki.de (git, http or ssh): ssh
  Which prepackaged software (a.k.a. 'osdeps') should autoproj install automatically (all, none or a comma-separated list of: os gem pip) ? os,gem,pip
  operating system: ubuntu,debian - 14.04,14.04.1,lts,trusty,tahr
  looking for RubyGems updates                                       
  updated autoproj main configuration         
autoproj: updating remote definitions of package sets
  updated rock.core         
  Which flavor of Rock do you want to use ? master
  updated rock         
  updated rock.simulation         
  updated rock.dfki          
  updated compilerspeedup         
  Enable ccache (compatible with icecc)? true
  ccache cache size (system wide, saved in home folder)? 10G
  Enable IceCC? true
  Parallel build processes for IceCC? 15
You need to run source env.sh before changes take effect
  updated besman         
  updated orocos.toolchain         

/media/data/Development/BesMan-orogen_loaders/.gems/gems/autoproj-1.12.0/lib/autoproj/osdeps.rb:1052:in `[]=': string not matched (IndexError)
    from /media/data/Development/BesMan-orogen_loaders/.gems/gems/autoproj-1.12.0/lib/autoproj/osdeps.rb:1052:in `block (2 levels) in merge'
    from /media/data/Development/BesMan-orogen_loaders/.gems/gems/autoproj-1.12.0/lib/autoproj/osdeps.rb:1051:in `each'
    from /media/data/Development/BesMan-orogen_loaders/.gems/gems/autoproj-1.12.0/lib/autoproj/osdeps.rb:1051:in `inject'
    from /media/data/Development/BesMan-orogen_loaders/.gems/gems/autoproj-1.12.0/lib/autoproj/osdeps.rb:1051:in `block in merge'
    from /media/data/Development/BesMan-orogen_loaders/.gems/gems/autoproj-1.12.0/lib/autoproj/osdeps.rb:1043:in `merge'
    from /media/data/Development/BesMan-orogen_loaders/.gems/gems/autoproj-1.12.0/lib/autoproj/osdeps.rb:1043:in `merge'
    from /media/data/Development/BesMan-orogen_loaders/.gems/gems/autoproj-1.12.0/lib/autoproj/manifest.rb:1157:in `block in load_osdeps_from_package_sets'
    from /media/data/Development/BesMan-orogen_loaders/.gems/gems/autoproj-1.12.0/lib/autoproj/manifest.rb:264:in `block (2 levels) in each_osdeps_file'
    from /media/data/Development/BesMan-orogen_loaders/.gems/gems/autoproj-1.12.0/lib/autoproj/manifest.rb:263:in `each'
    from /media/data/Development/BesMan-orogen_loaders/.gems/gems/autoproj-1.12.0/lib/autoproj/manifest.rb:263:in `block in each_osdeps_file'
    from /media/data/Development/BesMan-orogen_loaders/.gems/gems/autoproj-1.12.0/lib/autoproj/manifest.rb:312:in `each'
    from /media/data/Development/BesMan-orogen_loaders/.gems/gems/autoproj-1.12.0/lib/autoproj/manifest.rb:312:in `each_package_set'
    from /media/data/Development/BesMan-orogen_loaders/.gems/gems/autoproj-1.12.0/lib/autoproj/manifest.rb:262:in `each_osdeps_file'
    from /media/data/Development/BesMan-orogen_loaders/.gems/gems/autoproj-1.12.0/lib/autoproj/manifest.rb:1156:in `load_osdeps_from_package_sets'
    from /media/data/Development/BesMan-orogen_loaders/.gems/gems/autoproj-1.12.0/lib/autoproj/manifest.rb:1188:in `load_osdeps_from_package_sets'
    from /media/data/Development/BesMan-orogen_loaders/.gems/gems/autoproj-1.12.0/lib/autoproj/cmdline.rb:214:in `load_configuration'
    from /media/data/Development/BesMan-orogen_loaders/.gems/gems/autoproj-1.12.0/bin/autoproj:89:in `block in <top (required)>'
    from /media/data/Development/BesMan-orogen_loaders/.gems/gems/autoproj-1.12.0/lib/autoproj/cmdline.rb:1624:in `block in report'
    from /media/data/Development/BesMan-orogen_loaders/.gems/gems/autobuild-1.8.0/lib/autobuild/reporting.rb:278:in `report'
    from /media/data/Development/BesMan-orogen_loaders/.gems/gems/autoproj-1.12.0/lib/autoproj/cmdline.rb:1623:in `report'
    from /media/data/Development/BesMan-orogen_loaders/.gems/gems/autoproj-1.12.0/bin/autoproj:43:in `<top (required)>'
    from /media/data/Development/BesMan-orogen_loaders/.gems/bin/autoproj:23:in `load'
    from /media/data/Development/BesMan-orogen_loaders/.gems/bin/autoproj:23:in `<main>'

Is it still possible to build osdep packages from source, like suggested in samples/osdeps.yml?

This would be very useful for Arch support, since some dependencies only exist in the AUR and are installable only from source.

autoproj_boostrap (rock-robotics.org) is installing the non default version automake1.9 on debian jessie. The default version is currently automake1.11 required by some packages like ODE.

workaround:
sudo apt-get install automake

Currently, if e.g. the importer changes from archive to git, the git importer will complain that the source directory is not a git repository. Autoproj/autobuild should have a mechanism where (1) this situation is detected and (2) it asks the user it is OK to delete the source directory.

This is a prerequisite for rock-core/package_set#14

Depending on an environment variable set by jenkins different opencv version shall be built. Older version of autoproj allowed the following:

cmake_package 'external/opencv' do |pkg|
    if ENV["OPENCV_TAG"]
         pkg.importer.branch = nil
         pkg.importer.instance_variable_set(:@tag,ENV["OPENCV_TAG"].to_s)
    end
 end

This is no longer possible because after a successfully run autoproj stops the next time with the following error message:

external/opencv(/var/lib/jenkins/workspace/ropencv2.4.11/dev/external/opencv): failed in import phase
'git checkout -b 2c9547e3147779001811d01936aed38f560929fc' returned status 128

Is there any official way to accomplish the overwrite?

Hi, we need base/orogen/std to be on master (for orogen_metadata), but want the rest on the release.
On aup --all the checkout fails with

Command failed
base/orogen/std(/home/planthaber/dfki/rock/entern/base/orogen/std): failed in import phase
the local branch 'master' and the remote branch master of base/orogen/std have diverged, and I therefore refuse to update automatically. Go into /home/planthaber/dfki/rock/entern/base/orogen/std and either reset the local branch or merge the remote changes

The overrides.d fails if 25-release.yml sets a tag and the override sets a branch. It looks like the .yml files are executed after each other rather than merging them before doing the checkouts.

When I do
autoproj versions --no-deps base/types
I get an error.
tools/autoproj/lib/autoproj/cli/versions.rb:64:in `parse_options': invalid option: --no-deps (OptionParser::InvalidOption)

In case if some standalone package set is imported through source.yml and the same package set is included through autoproj/manifest, it results in endless loop in sorting of package sets by import order.

The following set up was producing this error:

In the project package set the 'rock_simulation', which is a standalone package set, was imported:

name: limes
imports:
    - github: rock-simulation/package_set

At the same time the 'rock_simulation' was added throught the manifest file:

package_sets:
  - github: rock-core/package_set 
  - github: rock-core/rock-package_set 
  - github: rock-simulation/package_set
  - spacegit: rock-dfki/package_set
  - dfkigit: limes/package_set
  - dfkigit: besman-rock/package_set  
  - spacegit: team-learning/package_set

In this case "autoproj update" command resulted in endless loop in gems/autoproj-1.12.4/lib/autoproj/ops/configuration.rb line 296:

def sort_package_sets_by_import_order(package_sets, root_pkg_set)
...
     sorted = Array.new
     while !queue.empty?
          pkg_set = queue.shift
          if pkg_set.imports.any? { |imported_set| !sorted.include?(imported_set) }
               queue.push(pkg_set)
          else
               sorted << pkg_set
          end
     end
...
end

Through the condition (s. configuration.rb line 292) the 'rock_simulation' was not added into 'queue' array. Therefore this package set never appears in the 'sorted' array. That results that the project package set, that imports 'rock_simulation', get pushed to queue all the time.

configuration.rb line 292:

def sort_package_sets_by_import_order(package_sets, root_pkg_set)
...
     queue = package_sets.find_all do |pkg_set|
          (!pkg_set.imports.empty? || !pkg_set.explicit?) && !(pkg_set == root_pkg_set)
     end
...
end

Autoproj does not cleanup, i.e. remove the install folder anymore when doing a rebuild of all packages.

if the environment variable AUTOPROJ_USE_PRERELEASE is set to 1 the boostrap process exits with the following error message:

cannot read file / URL [email protected]:wally/buildconf.git, did you mean 'autoproj bootstrap      VCSTYPE [email protected]:wally/buildconf.git' ?
ERROR: failed to run autoproj bootstrap [email protected]:wally/buildconf.git

It seems that someone is swallowing 'git' because the following command succeeds (2x git):
ruby autoproj_bootstrap git git [email protected]:wally/buildconf.git

If the repository is on another branch, this is not captured in the versioning, and so is not the remote. This leads to problems when doing an autoproj update, because
a) if the commit is available, but on another branch, autoproj refuses to switch
b) the commit might be on another remote

b happens quite regularly, when cloning a repo and adding local changes.

This is related to #40. The problem is that we can't rely on a branch name to persist. E.g. feature gets added on a branch. This is pinned. Branch is merged to main, and removed.

Might be this would be the way forward:

• autoproj commit
  • captures tag/commitid.
  • Has to look which remote that tag it is available in. Override remote when not on default remote.
  • does not capture branch name
• autoproj update
  • updates remotes
  • needs to be more lenient with switching branches. E.g. the check in #40 needs to be fixed.
  • if commit is not on current branch, switch to branch which has that commit
  • if multiple branches have that commit... well then... we're screwed. But I guess just picking one and warning should be ok.

I have an override in the overrides.d for base/types

overriden in main configuration
      branch: update_logging
      commit: 03ed628700601823e42ad3126a614c3b6952902e`

my actual HEAD is on master, but when I do
autoproj versions base/types
I get

- base/types:
    branch: update_logging
    tag: 
    commit: 03ed628700601823e42ad3126a614c3b6952902e

the commitid is correct, but not the branch.

autoproj yiels that it cannot find the given commit id if:

• commit is on another branch than the current one
• commit is on anotther URL than the current one

This happens if you change the branch and try to pin a commit at one time in autoproj.
Same fault occurs if the url of the repositroy is changed too.

It seems autoproj changes FIRST the commit-id and then tries to thange/update the branch or url.

I create a dummy package like the following:

module Autobuild
  6     class  MonsterFakeImporter < Importer
  7         def initialize()
  8         end
  9         def update
 10         end
 11         def checkout
 12         end
 13     end
 14 
 15 
 16 
 17     class MonsterFakePackage < Autobuild::DummyPackage
 18         attr_reader :tracking_files
 19 
 20         def initialize(*args)
 21             super
 22             @tracking_files = Array.new
 23             @importer = MonsterFakeImporter.new
 24             disable
 25             self
 26         end
 27 
 28         def depend_files(files)
 29             if file.instance_of?(String)
 30                 files = [files]
 31             end
 32             files.each do |file|
 33                 if File.exists?(file)
 34                     @tracking_files << file
 35                 else
 36                     STDERR.puts "Could not find file #{file} assumign it's created during bootstrap, creating fake one"
 37                     @tracking_files << File.new("/tmp/fakeAutoprojFile",File::CREAT | File::RDONLY).path
 38                 end
 39             end
 40         end
 41 
 42         def installstamp
 43             return File.new("/tmp/fakeAutoprojFile",File::CREAT | File::RDONLY).path if @tracking_files.empty?
 44             @tracking_files.sort! do |a,b|
 45                 if(!File.exists?(a) or !File.exists?(b))
 46                     return File.new("/tmp/fakeAutoprojFile",File::CREAT | File::RDONLY).path
 47                 end
 48                 File.stat(a).mtime <=> File.stat(b).mtime
 49             end
 50 
 51             @tracking_files.last
 52         end
 53 
 54         def name
 55             "Fake-Package"
 56         end
 57 
 58 
 59     end
 60 
 61     def self.monster_fake(spec)
 62         MonsterFakePackage.new(spec)
 63     end
 64 
 65     def self.monster_fake_importer(spec)
 66         MonsterFakePackage.new(spec)
 67     end
 68 
 69 end

Unforutnalty this results in the error:

[11:21:04 goldhoorn dev-limes]$ autoproj snapshot /tmp/foo
/home/goldhoorn/dev-limes/.gems/gems/autoproj-1.10/lib/autoproj/manifest.rb:698:in `block in load_importers': source rock.dfki defines Fake-Package, but does not provide a version control definition for it (Autoproj::ConfigError)
    from /home/goldhoorn/dev-limes/.gems/gems/autoproj-1.10/lib/autoproj/manifest.rb:689:in `each_value'
    from /home/goldhoorn/dev-limes/.gems/gems/autoproj-1.10/lib/autoproj/manifest.rb:689:in `load_importers'
    from /home/goldhoorn/dev-limes/.gems/gems/autoproj-1.10/lib/autoproj/cmdline.rb:293:in `load_configuration'
    from /home/goldhoorn/dev-limes/.gems/gems/autoproj-1.10/lib/autoproj/cmdline.rb:1916:in `initialize_and_load'
    from /home/goldhoorn/dev-limes/.gems/gems/autoproj-1.10/bin/autoproj-snapshot:9:in `block in <top (required)>'
    from /home/goldhoorn/dev-limes/.gems/gems/autobuild-1.7.11/lib/autobuild/reporting.rb:14:in `silent'
    from /home/goldhoorn/dev-limes/.gems/gems/autoproj-1.10/lib/autoproj/cmdline.rb:23:in `silent'
    from /home/goldhoorn/dev-limes/.gems/gems/autoproj-1.10/bin/autoproj-snapshot:7:in `<top (required)>'
    from /home/goldhoorn/dev-limes/.gems/gems/autoproj-1.10/bin/autoproj:41:in `load'
    from /home/goldhoorn/dev-limes/.gems/gems/autoproj-1.10/bin/autoproj:41:in `<top (required)>'
    from /home/goldhoorn/dev-limes/.gems/bin/autoproj:23:in `load'
    from /home/goldhoorn/dev-limes/.gems/bin/autoproj:23:in `<main>'

I see no way in the API to override this, so non-defines importter should become a weak error, and should not abort the creation of snapshots...

currently when doing an update, the message on the console will say updated for each package. I think it would be nicer, if it could actually only say updated, when it actually updated the package, and say something like "kept ..." otherwise.

when I do autoproj version for a package selection, e.g.
autoproj versions base/types
I get a list of package. However, when I try the same thing for commit
autoproj commit base/types
I get
error: pathspec 'base/types' did not match any file(s) known to git.

It would be greate if there was a way to bootstrap using the latest version of autoproj.

autoproj does not check the dependencies of a package listed in its package/manifest.xml if the package is directly added to the layout section in autoproj/manifest.xml.

Hi,

Our buildserver got the masage "`root_dir': not in a Autoproj installation" in the middle of updating after several updated were executed, no idea why this happend.

updated drivers/camera_firewire
updating drivers/camera_interface
updated drivers/camera_interface
updating drivers/camera_usb
updated drivers/camera_usb
updating drivers/canbus
updated drivers/canbus
/data/jenkins/workspace/workspace/rock-complete-incremental/Flavor/master/node/Ubuntu_current/dev/.gems/gems/autoproj-1.13.2/lib/autoproj/system.rb:48:in root_dir': not in a Autoproj installation (Autoproj::UserError) from /data/jenkins/workspace/workspace/rock-complete-incremental/Flavor/master/node/Ubuntu_current/dev/.gems/gems/autoproj-1.13.2/lib/autoproj/system.rb:66:inconfig_dir'
from /data/jenkins/workspace/workspace/rock-complete-incremental/Flavor/master/node/Ubuntu_current/dev/.gems/gems/autoproj-1.13.2/lib/autoproj/package_set.rb:596:in raw_local_dir' from /data/jenkins/workspace/workspace/rock-complete-incremental/Flavor/master/node/Ubuntu_current/dev/.gems/gems/autoproj-1.13.2/lib/autoproj/package_set.rb:135:increate_autobuild_package'
from /data/jenkins/workspace/workspace/rock-complete-incremental/Flavor/master/node/Ubuntu_current/dev/.gems/gems/autoproj-1.13.2/lib/autoproj/ops/snapshot.rb:155:in import_state_log_package' from /data/jenkins/workspace/workspace/rock-complete-incremental/Flavor/master/node/Ubuntu_current/dev/.gems/gems/autoproj-1.13.2/lib/autoproj/ops/snapshot.rb:34:inupdate_log_available?'
from /data/jenkins/workspace/workspace/rock-complete-incremental/Flavor/master/node/Ubuntu_current/dev/.gems/gems/autoproj-1.13.2/lib/autoproj/cmdline.rb:776:in ensure in import_packages' from /data/jenkins/workspace/workspace/rock-complete-incremental/Flavor/master/node/Ubuntu_current/dev/.gems/gems/autoproj-1.13.2/lib/autoproj/cmdline.rb:782:inimport_packages'
from /data/jenkins/workspace/workspace/rock-complete-incremental/Flavor/master/node/Ubuntu_current/dev/.gems/gems/autoproj-1.13.2/bin/autoproj:198:in block in <top (required)>' from /data/jenkins/workspace/workspace/rock-complete-incremental/Flavor/master/node/Ubuntu_current/dev/.gems/gems/autoproj-1.13.2/lib/autoproj/cmdline.rb:1605:inblock in report'
from /data/jenkins/workspace/workspace/rock-complete-incremental/Flavor/master/node/Ubuntu_current/dev/.gems/gems/autobuild-1.9.0/lib/autobuild/reporting.rb:282:in report' from /data/jenkins/workspace/workspace/rock-complete-incremental/Flavor/master/node/Ubuntu_current/dev/.gems/gems/autoproj-1.13.2/lib/autoproj/cmdline.rb:1604:inreport'
from /data/jenkins/workspace/workspace/rock-complete-incremental/Flavor/master/node/Ubuntu_current/dev/.gems/gems/autoproj-1.13.2/bin/autoproj:47:in <top (required)>' from /data/jenkins/workspace/workspace/rock-complete-incremental/Flavor/master/node/Ubuntu_current/dev/.gems/bin/autoproj:23:inload'
from /data/jenkins/workspace/workspace/rock-complete-incremental/Flavor/master/node/Ubuntu_current/dev/.gems/bin/autoproj:23:in `

'
Build step 'Execute shell' marked build as failure
Finished: FAILURE

We use features that are not available on 1.8.6, so for now validate that the user runs 1.8.7

it would be possible to have a rosmake-like behaviour, where autoproj auto-adds packages that are installed to the build by detecting manifest.xml and guessing the type of package.

If overrides.d is a git submodule, one gets a strange error :
2015-05-07 13:40:36 +0200: running
git --git-dir /home/scotch/entern/autoproj/.git update-index --add --cacheinfo 100644 0a6a539e65f504d2cc808be95a6c4c8a226e896a overrides.d/50-versions.yml
in directory /home/scotch/entern
error: 'overrides.d/50-versions.yml' appears as both a file and as a directory
error: overrides.d/50-versions.yml: cannot add to the index - missing --add option?
fatal: git update-index: --cacheinfo cannot add overrides.d/50-versions.yml

I'm puzzled by this one, it looks like a snapshot is generated, even thou I did not request this.

This error message is also thrown when the asked for commit is not on the branch, even though that branch is actually on a remote:
"branch master of xxx contains commits that do not seem to be present on the remote repository. I can't go on as it would mean losing some stuff. Push your changes or reset to the remote commit manually before trying again"

We should either change the message to be correct, or adapt the test.

In rock.core packages are defines with prefixed like:

00new_packages.autobuild

i added a file called

99metapackages.autobuild

and it was loaded before 04, which results in a unwanted behaviour. The numbering suggests kind of order which is not respected by autoproj

For the versioning fuctionality:

I think it could be a very good feature to be able to version specific package sets and not only the complete version state.

For example in one of my current projects there are some works with very active development in their own package set, which i whould like to keep following on HEAD, but e.g. i don't want the head of rock.core and orocos.toolchain to be on head.

What do you think?

(btw. I am not allowed to set lables: "versioning")

Calling autoproj rebuild on the command line fails because of the following error. What I do not understand in the first place. Why is autoproj restoring gems during rebuild? They are not supposed to be altered. And if they are I would assume the guy knows what he is doing.

As workaround I uninstalled all global gems used by the installation 😒

autoproj: failed in osdeps phase
'/usr/bin/ruby2.1 -S gem2.1 pristine --extensions hoe hoe-yard thin grape rack-cors faye-websocket sprockets rake-compiler websocket activesupport bundler cucumber rice kramdown rbtree binding_of_caller hooks ffi' returned status 1
see /home/aduda/dev/rock_master/install/log/autoproj-osdeps.log for details
last 10 lines are:

Restored cucumber-2.0.0
Building native extensions.  This could take a while...
Restored rice-1.7.0
Restored kramdown-1.7.0
Building native extensions.  This could take a while...
Restored rbtree-0.4.2
Building native extensions.  This could take a while...
ERROR:  While executing gem ... (Gem::FilePermissionError)
    You don't have write permissions for the /var/lib/gems/2.1.0 directory.

In a project there are often multiple deployment targets. E.g. desktop, robot, maybe build server, that have a different manifest structure. It would be great if one could specify multiple manifest files, and tell autoproj as a config variable which one to use.

On the current versioning branch, I get the following error whenever I try to do anything.
The offending commit seems to be a01a61c, the error goes away when I revert it.

/home/jakob/dev/wally/tools/autoproj/lib/autoproj/system.rb:76:in find_in_path': cannot find ruby1.9.1 in PATH (/home/jakob/dev/wally/install/bin:/home/jakob/dev/wally/.gems/bin) (ArgumentError) from /home/jakob/dev/wally/autoproj/remotes/orocos.toolchain/orocos.autobuild:83:inblock in <top (required)>'
from /home/jakob/dev/wally/tools/autoproj/lib/autoproj/autobuild.rb:408:in block in cmake_package' from /home/jakob/dev/wally/tools/autoproj/lib/autoproj/cmdline.rb:318:in[]'
from /home/jakob/dev/wally/tools/autoproj/lib/autoproj/cmdline.rb:318:in block (2 levels) in finalize_package_setup' from /home/jakob/dev/wally/tools/autoproj/lib/autoproj/cmdline.rb:317:ineach'
from /home/jakob/dev/wally/tools/autoproj/lib/autoproj/cmdline.rb:317:in block in finalize_package_setup' from /home/jakob/dev/wally/tools/autoproj/lib/autoproj/cmdline.rb:316:ineach_value'
from /home/jakob/dev/wally/tools/autoproj/lib/autoproj/cmdline.rb:316:in finalize_package_setup' from /home/jakob/dev/wally/tools/autoproj/lib/autoproj/cmdline.rb:1547:ininitialize_and_load'
from /home/jakob/dev/wally/tools/autoproj/bin/autoproj-envsh:8:in block (2 levels) in <top (required)>' from /home/jakob/dev/wally/tools/autobuild/lib/autobuild/reporting.rb:14:insilent'
from /home/jakob/dev/wally/tools/autoproj/lib/autoproj/cmdline.rb:24:in silent' from /home/jakob/dev/wally/tools/autoproj/bin/autoproj-envsh:6:inblock in <top (required)>'
from /home/jakob/dev/wally/tools/autoproj/lib/autoproj/cmdline.rb:1621:in block in report' from /home/jakob/dev/wally/tools/autobuild/lib/autobuild/reporting.rb:282:inreport'
from /home/jakob/dev/wally/tools/autoproj/lib/autoproj/cmdline.rb:1620:in report' from /home/jakob/dev/wally/tools/autoproj/bin/autoproj-envsh:5:in<top (required)>'
from /home/jakob/dev/wally/tools/autoproj/bin/autoproj:35:in load' from /home/jakob/dev/wally/tools/autoproj/bin/autoproj:35:in

By accident I created a gzip tarball with .bz2 extension (using git archive). Using autoproj (aup my_lib), the import fails of course. However, the source file is silently deleted afterwards. I verified this behaviour. The tar -xjf ... command found in the import log does not delete the file if executed independently.

On successful import the file is still there.

I use a source definition like this:

• my_lib:
  type: archive
  url: 'file://$HOME/src/my_lib.tar.bz2'
  archive_dir: my_lib
  update_cached_file: false

It is now more or less possible to bootstrap a new installation by reusing an already existing one, which makes boostrapping much faster. Document it.

for autoproj status,
It would be nice to differentiate in color between an advanced local repository vs advanced remote repository.

Even when repos are not available (as currently gitorious), aup -k reports success with a green message.

Autoproj currently does not generate any error if the osdeps is not defined

I don't really know if the issue arises from autoproj, but to me it looks like some path issues. If not we can at least start tracking down the issue from here.

Using ruby 2.1, orogen fails in post-install:
Orogen "the error is: cannot load such file -- typelib"

Full log (Ubuntu stable, the next log is only missing the information about Hoe, Debian logs are identical):

2014-11-09 23:09:19 +0000: running
/usr/bin/ruby2.1 -S rake default
with environment:
'AUTOPROJ_BOOTSTRAP_IGNORE_NONEMPTY_DIR'='1'
'AUTOPROJ_CACHE_DIR'='/data/jenkins/autoproj_cache/'
'AUTOPROJ_CURRENT_ROOT'='/data/jenkins/workspace/workspace/rock-incremental/Flavor/stable/node/Ubuntu_current/dev'
'AUTOPROJ_OSDEPS_MODE'='all'
'BOOTSTRAP_URL'='https://raw.githubusercontent.com/rock-core/autoproj/master/bin/autoproj_bootstrap'
'BUILD_DISPLAY_NAME'='#698'
'BUILD_ID'='2014-11-09_23-56-16'
'BUILD_NUMBER'='698'
'BUILD_TAG'='jenkins-rock-incremental-Flavor=stable,node=Ubuntu_current-698'
'BUILD_URL'='http://buildsrv01:8080/job/rock-incremental/Flavor=stable,node=Ubuntu_current/698/'
'CCACHE_DIR'='/data/jenkins/ccache'
'CCACHE_PREFIX'='icecc'
'CLEAN_IF_SUCCESSFUL'='false'
'CMAKE_PREFIX_PATH'='/data/jenkins/workspace/workspace/rock-incremental/Flavor/stable/node/Ubuntu_current/dev/install'
'CONFIG_NAME'='rock-incremental/Flavor=stable,node=Ubuntu_current-stable'
'DOCGEN'='false'
'EXECUTOR_NUMBER'='0'
'Flavor'='stable'
'GEM_HOME'='/data/jenkins/workspace/workspace/rock-incremental/Flavor/stable/node/Ubuntu_current/dev/.gems'
'GEM_PATH'='/data/jenkins/workspace/workspace/rock-incremental/Flavor/stable/node/Ubuntu_current/dev/.gems:/var/lib/jenkins/.gem/ruby/2.1.0:/var/lib/gems/2.1.0:/usr/share/rubygems-integration/2.1.0:/usr/share/rubygems-integration/2.1:/usr/share/rubygems-integration/all'
'HOME'='/var/lib/jenkins'
'HUDSON_COOKIE'='c04f78cc-2e67-4329-ae29-67144e299bdd'
'HUDSON_HOME'='/data/jenkins'
'HUDSON_SERVER_COOKIE'='5bfb7c75ae1d8325'
'HUDSON_URL'='http://buildsrv01:8080/'
'JENKINS_HOME'='/data/jenkins'
'JENKINS_SERVER_COOKIE'='5bfb7c75ae1d8325'
'JENKINS_URL'='http://buildsrv01:8080/'
'JOB_NAME'='rock-incremental/Flavor=stable,node=Ubuntu_current'
'JOB_URL'='http://buildsrv01:8080/job/rock-incremental/Flavor=stable,node=Ubuntu_current/'
'KEEP_GOING'='true'
'LANG'='en_US.UTF-8'
'LD_LIBRARY_PATH'='/data/jenkins/workspace/workspace/rock-incremental/Flavor/stable/node/Ubuntu_current/dev/install/lib'
'LOGNAME'='jenkins'
'MODE'='incremental'
'NODE_LABELS'='Ubuntu_current'
'NODE_NAME'='Ubuntu_current'
'OLDPWD'='/data/jenkins/workspace/workspace/rock-incremental/Flavor/stable/node/Ubuntu_current'
'ORBInitRef'='NameService=corbaname::127.0.0.1'
'OROCOS_TARGET'='gnulinux'
'OROGEN_PLUGIN_PATH'='/data/jenkins/workspace/workspace/rock-incremental/Flavor/stable/node/Ubuntu_current/dev/tools/orogen_ros/lib/orogen_ros/plugin.rb:/data/jenkins/workspace/workspace/rock-incremental/Flavor/stable/node/Ubuntu_current/dev/install/share/orogen/plugins'
'OSG_FILE_PATH'='/data/jenkins/workspace/workspace/rock-incremental/Flavor/stable/node/Ubuntu_current/dev/install/share/vizkit'
'PARALLEL'='15'
'PATH'='/data/jenkins/workspace/workspace/rock-incremental/Flavor/stable/node/Ubuntu_current/dev/metaruby/bin:/data/jenkins/workspace/workspace/rock-incremental/Flavor/stable/node/Ubuntu_current/dev/tools/orogen/bin:/data/jenkins/workspace/workspace/rock-incremental/Flavor/stable/node/Ubuntu_current/dev/tools/log_tools/bin:/data/jenkins/workspace/workspace/rock-incremental/Flavor/stable/node/Ubuntu_current/dev/tools/roby/bin:/data/jenkins/workspace/workspace/rock-incremental/Flavor/stable/node/Ubuntu_current/dev/tools/pocolog/bin:/data/jenkins/workspace/workspace/rock-incremental/Flavor/stable/node/Ubuntu_current/dev/tools/orocos.rb/bin:/data/jenkins/workspace/workspace/rock-incremental/Flavor/stable/node/Ubuntu_current/dev/tools/syskit/bin:/data/jenkins/workspace/workspace/rock-incremental/Flavor/stable/node/Ubuntu_current/dev/base/scripts/bin:/data/jenkins/workspace/workspace/rock-incremental/Flavor/stable/node/Ubuntu_current/dev/gui/vizkit/bin:/data/jenkins/workspace/workspace/rock-incremental/Flavor/stable/node/Ubuntu_current/dev/.gems/bin:/data/jenkins/workspace/workspace/rock-incremental/Flavor/stable/node/Ubuntu_current/dev/install/bin:/usr/lib/ccache:/usr/lib/icecc/bin:/usr/local/bin:/usr/bin:/bin:/usr/bin/X11:/usr/games'
'PKG_CONFIG_PATH'='/data/jenkins/workspace/workspace/rock-incremental/Flavor/stable/node/Ubuntu_current/dev/install/lib/pkgconfig:/data/jenkins/workspace/workspace/rock-incremental/Flavor/stable/node/Ubuntu_current/dev/install/lib64/pkgconfig:/data/jenkins/workspace/workspace/rock-incremental/Flavor/stable/node/Ubuntu_current/dev/install/share/pkgconfig'
'PWD'='/data/jenkins/workspace/workspace/rock-incremental/Flavor/stable/node/Ubuntu_current/dev'
'PYTHONPATH'='/data/jenkins/workspace/workspace/rock-incremental/Flavor/stable/node/Ubuntu_current/dev/install/lib/python2.7/site-packages'
'PYTHONUSERBASE'='/data/jenkins/workspace/workspace/rock-incremental/Flavor/stable/node/Ubuntu_current/dev/.pip'
'QT_PLUGIN_PATH'='/data/jenkins/workspace/workspace/rock-incremental/Flavor/stable/node/Ubuntu_current/dev/install/lib/qt'
'ROBY_PLUGIN_PATH'='/data/jenkins/workspace/workspace/rock-incremental/Flavor/stable/node/Ubuntu_current/dev/tools/syskit/lib/syskit/roby_app/register_plugin.rb:/data/jenkins/workspace/workspace/rock-incremental/Flavor/stable/node/Ubuntu_current/dev/base/scripts/lib/rock/roby_plugin.rb'
'ROCK_BUNDLE_PATH'='/data/jenkins/workspace/workspace/rock-incremental/Flavor/stable/node/Ubuntu_current/dev/bundles:/data/jenkins/workspace/workspace/rock-incremental/Flavor/stable/node/Ubuntu_current/dev/install/share/rock:/data/jenkins/workspace/workspace/rock-incremental/Flavor/stable/node/Ubuntu_current/dev/bundles/rock_ugv_nav:/data/jenkins/workspace/workspace/rock-incremental/Flavor/stable/node/Ubuntu_current/dev/bundles/rock_auv:/data/jenkins/workspace/workspace/rock-incremental/Flavor/stable/node/Ubuntu_current/dev/bundles/rock'
'ROCK_PREFIX'='/data/jenkins/workspace/workspace/rock-incremental/Flavor/stable/node/Ubuntu_current/dev/install/share/rock'
'ROCK_TEMPLATE_PREFIX'='/data/jenkins/workspace/workspace/rock-incremental/Flavor/stable/node/Ubuntu_current/dev/base/templates/'
'ROCK_TEMPLATE_SUFFIX'=''
'RUBYLIB'='/data/jenkins/workspace/workspace/rock-incremental/Flavor/stable/node/Ubuntu_current/dev/metaruby/lib:/data/jenkins/workspace/workspace/rock-incremental/Flavor/stable/node/Ubuntu_current/dev/tools/utilrb/lib:/data/jenkins/workspace/workspace/rock-incremental/Flavor/stable/node/Ubuntu_current/dev/tools/orogen/lib:/data/jenkins/workspace/workspace/rock-incremental/Flavor/stable/node/Ubuntu_current/dev/tools/log_tools/lib:/data/jenkins/workspace/workspace/rock-incremental/Flavor/stable/node/Ubuntu_current/dev/tools/roby/lib:/data/jenkins/workspace/workspace/rock-incremental/Flavor/stable/node/Ubuntu_current/dev/tools/pocolog/lib:/data/jenkins/workspace/workspace/rock-incremental/Flavor/stable/node/Ubuntu_current/dev/tools/orocos.rb/lib:/data/jenkins/workspace/workspace/rock-incremental/Flavor/stable/node/Ubuntu_current/dev/tools/syskit/lib:/data/jenkins/workspace/workspace/rock-incremental/Flavor/stable/node/Ubuntu_current/dev/base/scripts/lib:/data/jenkins/workspace/workspace/rock-incremental/Flavor/stable/node/Ubuntu_current/dev/install/lib:/data/jenkins/workspace/workspace/rock-incremental/Flavor/stable/node/Ubuntu_current/dev/slam/envire/ruby:/data/jenkins/workspace/workspace/rock-incremental/Flavor/stable/node/Ubuntu_current/dev/drivers/transformer/ruby/lib:/data/jenkins/workspace/workspace/rock-incremental/Flavor/stable/node/Ubuntu_current/dev/gui/vizkit/build/lib:/data/jenkins/workspace/workspace/rock-incremental/Flavor/stable/node/Ubuntu_current/dev/gui/vizkit/ruby/lib'
'RUBYOPT'='-rubygems'
'SCHROOT_COMMAND'='/data/jenkins/build_scripts/jenkins_slave'
'SCHROOT_GID'='499'
'SCHROOT_GROUP'='jenkins'
'SCHROOT_SESSION_ID'='14.10_amd64-8be82f1c-ff0b-4c2d-976d-f68bdaafb835'
'SCHROOT_UID'='498'
'SCHROOT_USER'='jenkins'
'SHELL'='/bin/false'
'SHLVL'='2'
'SKIP_SUCCESSFUL'='false'
'TERM'='xterm'
'TYPELIB_RUBY_PLUGIN_PATH'='/data/jenkins/workspace/workspace/rock-incremental/Flavor/stable/node/Ubuntu_current/dev/install/share/typelib/ruby'
'USER'='jenkins'
'VIZKIT_PLUGIN_RUBY_PATH'='/data/jenkins/workspace/workspace/rock-incremental/Flavor/stable/node/Ubuntu_current/dev/install/lib/vizkit:/data/jenkins/workspace/workspace/rock-incremental/Flavor/stable/node/Ubuntu_current/dev/install/lib'
'WORKSPACE'='/data/jenkins/workspace/workspace/rock-incremental/Flavor/stable/node/Ubuntu_current'
'_'='/data/jenkins/workspace/workspace/rock-incremental/Flavor/stable/node/Ubuntu_current/dev/.gems/bin/autoproj'
'node'='Ubuntu_current'

2014-11-09 23:09:19 +0000: running
/usr/bin/ruby2.1 -S rake default
in directory /data/jenkins/workspace/workspace/rock-incremental/Flavor/stable/node/Ubuntu_current/dev/tools/orogen
WARN: cannot load the Hoe gem, or Hoe fails. Publishing tasks are disabled
WARN: error message is: No such file or directory @ rb_sysopen - README.txt
cannot load oroGen
did you install Typelib's Ruby bindings and update the RUBYLIB environment variable accordingly ?
did you add /data/jenkins/workspace/workspace/rock-incremental/Flavor/stable/node/Ubuntu_current/dev/tools/orogen/lib to RUBYLIB ?
the error is: cannot load such file -- typelib
/usr/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in require' /usr/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:inrequire'
/data/jenkins/workspace/workspace/rock-incremental/Flavor/stable/node/Ubuntu_current/dev/tools/orogen/Rakefile:6:in block in <top (required)>' /usr/lib/ruby/vendor_ruby/rake/task.rb:236:incall'
/usr/lib/ruby/vendor_ruby/rake/task.rb:236:in block in execute' /usr/lib/ruby/vendor_ruby/rake/task.rb:231:ineach'
/usr/lib/ruby/vendor_ruby/rake/task.rb:231:in execute' /usr/lib/ruby/vendor_ruby/rake/task.rb:175:inblock in invoke_with_call_chain'
/usr/lib/ruby/2.1.0/monitor.rb:211:in mon_synchronize' /usr/lib/ruby/vendor_ruby/rake/task.rb:168:ininvoke_with_call_chain'
/usr/lib/ruby/vendor_ruby/rake/task.rb:197:in block in invoke_prerequisites' /usr/lib/ruby/vendor_ruby/rake/task.rb:195:ineach'
/usr/lib/ruby/vendor_ruby/rake/task.rb:195:in invoke_prerequisites' /usr/lib/ruby/vendor_ruby/rake/task.rb:174:inblock in invoke_with_call_chain'
/usr/lib/ruby/2.1.0/monitor.rb:211:in mon_synchronize' /usr/lib/ruby/vendor_ruby/rake/task.rb:168:ininvoke_with_call_chain'
/usr/lib/ruby/vendor_ruby/rake/task.rb:161:in invoke' /usr/lib/ruby/vendor_ruby/rake/application.rb:149:ininvoke_task'
/usr/lib/ruby/vendor_ruby/rake/application.rb:106:in block (2 levels) in top_level' /usr/lib/ruby/vendor_ruby/rake/application.rb:106:ineach'
/usr/lib/ruby/vendor_ruby/rake/application.rb:106:in block in top_level' /usr/lib/ruby/vendor_ruby/rake/application.rb:115:inrun_with_threads'
/usr/lib/ruby/vendor_ruby/rake/application.rb:100:in top_level' /usr/lib/ruby/vendor_ruby/rake/application.rb:78:inblock in run'
/usr/lib/ruby/vendor_ruby/rake/application.rb:165:in standard_exception_handling' /usr/lib/ruby/vendor_ruby/rake/application.rb:75:inrun'
/usr/bin/rake:27:in `

'

Hey,
we got a new git server, which does not support the git protocol.
As far as I can see, there is no option, to disable the git protocol
in gitorious_server_configuration. Only workaround I would see
for now is to set the git_url to "git@#{base_url}".
By the way, gitorious_server_configuration is a horrible name.
Should be git_server_configuration.
Greetings
Janosch

The current bootstrap script will yell if sudo is not installed

Currently autoproj update will happily revert to the given commit. This is not what one would normally want, but it should only go up in the commits unless explicitely asked.

Some companies will only use gems with a certain license.
The canonical and easy way to check is via the gemspec
via e.g.

spec.license = 'MIT'
# or
spec.licenses = ['MIT', 'GPL-2']

There is even a License Finder to help companies ensure all gems they use
meet their licensing needs. This tool depends on license information being available in the gemspec.
Including a license in your gemspec is a good practice, in any case.

How did I find you?

I'm using a script to collect stats on gems, originally looking for download data, but decided to collect licenses too,
and make issues for missing ones as a public service :)
https://gist.github.com/bf4/5952053#file-license_issue-rb-L13 So far it's going pretty well

If one changes

• gitorious: rock-toolchain/package_set
• gitorious: rock/package_set

to

• github: rock-core/package_set
• github: rock-core/rock-package_set

in the manifest file, the folders in .remotes get deleted, even if they contain
local modifications.

A global default: entry is meant to be useful for any OS and any version (typically, by using a non-os-specific package manager). Autoproj should allow falling back to it instead of requiring to copy/paste the entry in the OS-specific parts.

For instance,:

ubuntu:
   '14.04': a_debian_package
default:
   - gem: a_gem_package

currently requires to do

ubuntu:
   '14.04': a_debian_package
  default:
     - gem: a_gem_package
default:
   - gem: a_gem_package