When starting a job, amount of time passed starts on negative value. Looks very strange.

Also, how is estimate progress calculated? Maybe this is connected.

running laminar behind a proxy and the custom stylesheet isn't taken into account.

Would it be possible to display also running jobs under /jobs?

I could break the display into finished or last job and active jobs

Active Jobs:
hello #10 started ...

Last Jobs:
hello #9 Passed
other #3 Failed
...

when watching progress in specific tag tab the jobs don't refresh with new queued jobs.

are you planning to extend the tool with some notifications? not only browser notifications, but support for external stuff?

Hi I am trying to run laminar in a container (LXD gentoo with OpenRC). And this is what I am getting as output.

The container is named laminar-ci it has internet access,

`

laminard

terminate called after throwing an instance of 'kj::ExceptionImpl'
what(): kj/async-io-unix.c++:748: failed: DNS lookup failed.; params.host = unix-abstract; params.service = laminar; gai_strerror(status) = Servname not supported for ai_socktype
stack: 0x7f1abdbe3063 0x7f1abde311c1 0x7f1abde28e2d 0x7f1abde2963f 0x7f1abde11bb2 0x7f1abde23a10 0x42af70 0x42e99e 0x7f1abde18286 0x7f1abde117cb 0x7f1abde14e75 0x431d5f 0x42b4e2 0x4582eb 0x4297f8 0x7f1abc455f10 0x4299e9
Aborted
`
So there is some problem with dns? Can you please help me? Thank you

I have a couple of failed builds that are covering the passed ones in the Total builds graph. Could these be made semi-transparent?

Thank you

When the laminar web UI is behind a reverse proxy (nginx), the website does not work. It does not show any UI elements:

The means of connecting to the website does not seem to have any impact. I tried connecting via a unix socket and via a network socket. The nginx rule used to forward requests to laminar is simple:

location / {
    #proxy_pass http://unix:/var/lib/laminar/laminard.sock;
    proxy_pass http://example.com:8081;
}

When I connect to http://example.com:8081 directly, the web UI is displayed correctly.

Replace the existing trigger (which queues a job) and start (which runs a job to completion) with queue (queues a job), start (blocks until job starts) and run (blocks until completion)

So I am creating jobs every 10 seconds like this:

for i in `cat $(wc -l pkgcat/*|sort|awk '{ print $2 }'|grep -v total |head -10)` ;do LAMINAR_REASON="$i" laminarc trigger fun-prime PKG=$i;sleep 10;done

The input of i$ is something like this:

cat $(wc -l pkgcat/*|sort|awk '{ print $2 }'|grep -v total |head -10)
app-antivirus/clamav
app-antivirus/clamav-unofficial-sigs
app-antivirus/skyldav
x11-base/xorg-drivers    
x11-base/xorg-server
x11-base/xorg-x11  
....

The scipt fun-prime is starting a container and then building the package inside the LXD container.
Can supply those as well if you need them.

So there is a new run added to the job for a different package.

This is inside the laminard.log:

cat /var/log/laminard 
terminate called after throwing an instance of 'kj::ExceptionImpl'
  what():  laminar.cpp:376: failed: expected siginfo->ssi_signo == SIGCHLD
stack: 0x42be9e 0x42dde7 0x7fa345d8a544 0x42ebec 0x7fa345d918bd 0x7fa345d89ff9 0x7fa345d8e004 0x43284a 0x42b6d9 0x4574b1 0x429877 0x7fa3446866fb 0x429a98

I have:

nodes/default.conf
EXECUTORS=2

nodes/emerge.conf
EXECUTORS=1
TAGS=emerge

but laminar spins up 3 jobs for a job tagged emerge using jobs/job.conf

Am I using it wrong?

Thank you

It should be possible to set a maximum execution time for any job, after which laminar should abort it. Depends on #29

Trying to grasp laminar's power, but something went wrong... I queued a "hello" job, tried to run it 5 times, and received no log output, when I looked into Web UI!

Tried to run it 6th time, received some log output while the job is still running, but after reloading, the job is shown like it never was - everything is NaN or undefined, and no logs are saved.

Maybe it's a bug, maybe it's intentional, but how to make laminar at least save the job logs and stats? By the way, the "Last success" and "Last build" stats on job screen also suffer from this.

Could the running jobs show the current execution time of the job?

So I have 30 executors running they are shown as busy and no idle. After refresh of the page it shows. 30 busy and 30 idle, which is not correct.

Is this expected behaviour?

If I run the command with ENV=env laminarc trigger job the ENV doesn't get passed to the job, but if I do laminarc trigger jobs ENV=env - env gets passed to the job.

If a job run is aborted due to a timeout, the .after script should still run, because that's probably where a notification would be sent

video

Steps to reproduce:

1. navigate to (http://laminar.liguros.net/jobs/fun-prime/)
2. use the next page to skip to page > 1
3. click on jobs within the open window
4. pick another job the page counter is set to the last page counter from fun-prime job

Thanks for looking into it

PS: This doesn't happen if you input new url into the address bar, only if you navigate within the application.

When there are jobs, being added to queue and you are on the jobs page the queued jobs increment and get shown correctly. Once you refresh the page or navigate away and come back the queued jobs are not shown. If there are even more jobs being added to the queue then the queue starts counting from 1 again, which is not correct.

how should I pass variable with space? eg. USE="this that" because every time I use laminarc set USE=$USE it gets truncated to USE='"this'

am I doing something wrong?

The current behaviour of LAMINAR_KEEP_RUNDIR is that if the env variable is set, all job runs will be preserved. This leads to jobs piling up on the disk, which is why this setting is disabled in my setup. However, if I need to inspect the environment of a failed job, I have to change the configuration, restart laminar, and re-run the failing job.

I suggest that LAMINAR_KEEP_RUNDIR supports integer values. The integer represents the number of run directories that will be preserved.
For example: LAMINAR_KEEP_RUNDIR=3 will preserve the three most recent run directories for every job.

This would allow me to enable the setting permanently. It would be easier to inspect and debug the environment in which a job failed without having to re-configure and restart laminar.

I am bringing this idea up, because it seems somewhat related to #18 .

would it be possible to sort the runs by result (success,failure), date started, duration, reason ....?

what are you thoughts on this, please.

it would be nice to have tooltips for most of the graphs. showing the number of runs, or execution time ...

I don't know if this will pass your KISS principle, but would it be possible to start a job from the command interface?

Thanks

A job's current status could be served as a badge, e.g. laminar:8080/jobs/JOB/badge.svg. Remember to set a short cache period.

When I am trying to download a file from laminar's archive I am getting this:

This is even directly downloading

wget http://laminar.lxd:8080/archive/fun-prime/89/dev-db/mariadb-10.2.9.tbz2
--2018-04-12 08:29:42--  http://laminar.lxd:8080/archive/fun-prime/89/dev-db/mariadb-10.2.9.tbz2
Resolving laminar.lxd... 127.0.0.1
Connecting to laminar.lxd|127.0.0.1|:8080... connected.
HTTP request sent, awaiting response... 200 OK
Length: 43209261 (41M)
Saving to: ‘mariadb-10.2.9.tbz2’

mariadb-10.2.9.tbz2                                 8%[========>                                                                                                   ]   3.56M  --.-KB/s    in 0.01s   

2018-04-12 08:29:42 (320 MB/s) - Connection closed at byte 3732424. Retrying.

--2018-04-12 08:29:43--  (try: 2)  http://laminar.lxd:8080/archive/fun-prime/89/dev-db/mariadb-10.2.9.tbz2
Connecting to laminar.lxd|127.0.0.1|:8080... connected.
HTTP request sent, awaiting response... 200 OK
Length: 43209261 (41M)
Saving to: ‘mariadb-10.2.9.tbz2’

mariadb-10.2.9.tbz2                                20%[====================>                                                                                       ]   8.37M  --.-KB/s    in 0.02s   

2018-04-12 08:29:43 (517 MB/s) - Connection closed at byte 8774615. Retrying.

^C

I am running without proxy, so :8080 is directly laminar listening for requests.

But running with proxy doesn't make it better.

2018-04-12 16:39:16 (602 KB/s) - Connection closed at byte 3404021. Retrying.

How can I help to debug it for you some more?

When triggering child jobs, as described by [1], the only returned information about child job's result is exit code ([2]). Having lots of parent and child jobs executed it's easy (at least for me) to lose track which child job "belongs" to the parent one. Thus I think it would be cool for start result to contain also any "pointer" to the child job.

Examples of what I have in mind:

• feedback with job ID only, probably as stdout/stderr. Below Example assumes that job is viewable at URL http://laminar.example/jobs/ajob/75 :

$ laminarc start ajob
75
$ echo $?
0

• feedback with job URL. This probably involves laminard to become self-conscious about its URL, and somehow it would need to became configurable - not sure whether this is cool approach?

$ laminarc start ajob
http://laminar.example/jobs/ajob/75
$ echo $?
0

[1] http://laminar.ohwgiles.documentup.com/#job-chains
[2] https://github.com/ohwgiles/laminar/blob/master/src/laminar.capnp#L26

Hi, I have tried to rebuild last version (commit 9c25681) from git with script from this repo and it failed.

I think, it is due to some old lib or something similar, but I'm not familiar with it enough to find, where issue is. Any kick possible?
Thanks, Valicek1

Error (from command make -j4 in debian docker script, no errors in steps before).

root@95e46937385e:/build/laminar# make -j4
[  8%] Generating laminar.capnp.c++, laminar.capnp.h
[  8%] Generating css/bootstrap.min.css.z
[ 11%] Generating index.html.z
[  8%] Generating laminar.capnp.c++, laminar.capnp.h
[ 13%] Generating js/app.js.z
[ 16%] Generating favicon.ico.z
[ 19%] Generating favicon-152.png.z
[ 25%] Generating icon.png.z
[ 25%] Generating js/vue-router.min.js.z
Scanning dependencies of target laminarc
[ 27%] Generating js/vue.min.js.z
[ 33%] Generating js/ansi_up.js.z
[ 33%] Generating js/Chart.min.js.z
[ 36%] Generating js/Chart.HorizontalBar.js.z
[ 41%] Generating css/bootstrap.min.css.o
[ 41%] Building CXX object CMakeFiles/laminarc.dir/src/client.cpp.o
[ 44%] Building CXX object CMakeFiles/laminarc.dir/laminar.capnp.c++.o
[ 47%] Generating index.html.o
[ 50%] Generating js/app.js.o
[ 52%] Generating favicon.ico.o
[ 55%] Generating favicon-152.png.o
[ 58%] Generating icon.png.o
[ 61%] Generating js/vue-router.min.js.o
[ 63%] Generating js/vue.min.js.o
[ 66%] Generating js/ansi_up.js.o
[ 69%] Generating js/Chart.min.js.o
[ 72%] Generating js/Chart.HorizontalBar.js.o
Scanning dependencies of target laminard
[ 77%] Building CXX object CMakeFiles/laminard.dir/src/database.cpp.o
[ 77%] Building CXX object CMakeFiles/laminard.dir/src/main.cpp.o
[ 80%] Building CXX object CMakeFiles/laminard.dir/src/server.cpp.o
[ 83%] Building CXX object CMakeFiles/laminard.dir/src/laminar.cpp.o
[ 86%] Linking CXX executable laminarc
[ 86%] Built target laminarc
[ 88%] Building CXX object CMakeFiles/laminard.dir/src/conf.cpp.o
[ 91%] Building CXX object CMakeFiles/laminard.dir/src/resources.cpp.o
[ 94%] Building CXX object CMakeFiles/laminard.dir/src/run.cpp.o
[ 97%] Building CXX object CMakeFiles/laminard.dir/laminar.capnp.c++.o
/build/laminar/src/server.cpp: In member function 'void Server::start()':
/build/laminar/src/server.cpp:438:16: error: 'class kj::TaskSet' has no member named 'onEmpty'
     childTasks.onEmpty().wait(ioContext.waitScope);
                ^~~~~~~
/build/laminar/src/server.cpp:440:25: error: 'class kj::WaitScope' has no member named 'poll'
     ioContext.waitScope.poll();
                         ^~~~
CMakeFiles/laminard.dir/build.make:205: recipe for target 'CMakeFiles/laminard.dir/src/server.cpp.o' failed
make[2]: *** [CMakeFiles/laminard.dir/src/server.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
CMakeFiles/Makefile2:67: recipe for target 'CMakeFiles/laminard.dir/all' failed
make[1]: *** [CMakeFiles/laminard.dir/all] Error 2
Makefile:127: recipe for target 'all' failed
make: *** [all] Error 2

If the connection from the browser to laminard is interrupted (e.g. server dies, or reverse proxy times out the websocket link), the browser should attempt to reconnect, and display a notification if this fails

Would be nice if an open Laminar tab in browser could ping you with a browser notification when something finishes cooking up.

I hope it doesn't break your minimalistic approach (which is cool, because Laminar is blazing fast on a low-power Raspberry Pi!), but would be probably useful for me, as my job notifier doesn't work for some reason...

It should be possible to display each tab group on the jobs page in a high-visibility mode for displaying on large screens in the office

The current Bootstrap 3 layout is starting to look a bit dated

When I set LAMINAR_ARCHVE_URL, my artefact suddenly got a strange URL: http://192.168.1.33:8080/jobs/hello/https://ci.<mydomain.tld>/archive/hello/1/hello-20170922.

I'm accessing Laminar via HTTP because I cannot proxy websockets with my lighttpd server - it lacks mod_websocket to do it, and Laminar is trying to access unsecured socket anyway - but the reason is not important as different URL schemes probably are.

When asked to shutdown, laminar should terminate all child processes and mark their result as "aborted"

Screenshot
https://postimg.org/image/4l12ra9tgr/

Currently the executor seems to be set to 8 - at least on my machine. Can you tweak this setting?

So this happens only with run number 1 of a job all subsequent runs behave correctly.

I wanted to do a screencast, but failing on some ffmpeg (x11grab deprecated) so will add this later if needed.

I have noticed following:

I have couple of jobs queued and there are already let's say 4 jobs running. If I edit cfg/nodes/TAG.conf while jobs running, after I save laminar will launch another 4 jobs.

It should check if the limit of jobs is not already running and launch jobs only if the current running jobs < jobs defined in .conf

So currently if you stay on the jobs page and let jobs run, you are able to see all jobs that started after you have opened the jobs page. If you follow a link to details about single job and come back you only see approx 30 jobs.

It would be nice if you scroll to the bottom of the page that the page would dynamically load more finished jobs (it can be 30 at a time).

Do you know what I am trying to say?

for "Builds per job in the last 24 hours" and "Average build time per job this week" graphs if a lot of jobs are running the graph is unusable.

Hi Oliver,

it is me again.

I am getting this on my box, again:

terminate called after throwing an instance of 'kj::ExceptionImpl'
  what():  kj/async-io-unix.c++:297: disconnected: ::read(fd, buffer, maxBytes): Connection reset by peer
stack: 0x7f7ca9133236 0x7f7ca91093f4 0x4456c0 0x42fb0c 0x7f7ca911076d 0x7f7ca9108ea9 0x7f7ca910ceb4 0x4338ba 0x42c08c 0x458274 0x42a157 0x7f7ca79e672c 0x42a368

What info do you need to debug?

Hi,
it would be nice if job/node (at least job) config was parsed before job run (trigger event) - in more complex scenarios with nodes, there is problem with jobs queued forever.
Example: Create job hello2.run and it's config hello2.conf, then execute laminarc trigger hello2 - job si queued forever. (no hello2.conf parsed until daemon restart).

By studying sources, I didn't found any simple workaround - should I create patch/pullrequest? I'd appraise you to handle this simple fix by yourself - I won't like to mess your clean code by my hand :-D

• Consider whether boost::filesystem and websocketpp can be replaced with new functionality in capnproto
• Investigate suitability of capnproto http features for serving files (see #37)
• Replace manual SIGCHLD handling with the new functionality from capnproto
• Much tighter code could be written by relaxing the strict separation between capnp code as used in Server.cpp as compared to other files. It already leaked into Run in a few recent commits.
• Reduce duplication in client.cpp and RpcImpl

Hi Oliver,

so I have laminar crashing upon me under load. And if I restart it there are stale run directories, which don't allow to start new runs or jobs.

It would be good to solve the crashing problem of course, but also if a crash happens enable laminar to recover from it:

• deleting stale directories in ~laminar/run/job_name/
  or
• mark the last run number in the database so the new jobs get the next higher run number

Let me know what you think.

Pavol

When packaging Laminar, I got this warning from rpmlint:

laminar.x86_64: W: executable-stack /usr/sbin/laminard
The binary declares the stack as executable.  Executable stack is usually an
error as it is only needed if the code contains GCC trampolines or similar
constructs which uses code on the stack.  One common source for needlessly
executable stack cases are object files built from assembler files which don't
define a proper .note.GNU-stack section.

That’s a CVE waiting to happen, so it should be fixed.

When I configure the environment variable LAMINAR_KEEP_WORKDIR, the directories of the individual job runs are not preserved.

Steps to reproduce

• Run as laminar user:
  $ LAMINAR_BIND_HTTP=unix:/var/lib/laminar/laminard.sock LAMINAR_KEEP_WORKDIR=1 /usr/bin/laminard
• Trigger any job using laminarc

Expected results
The directory run/my-job/1/ is visible after the job crashed or finished.

Actual results
The directory run/my-job/1/ is temporarily visible. As soon as the job execution stops, the directory is removed.

BTW: I do like the project idea very much! Despite being a very young project, I chose to work with Laminar, because everything looks so tidy and polished :) Thanks for your efforts.

Tried to compile on Gentoo on my Raspberry Pi. Got this instead:

/home/kisik21/laminar/src/laminar.cpp: In member function ‘void Laminar::runFinished(Run*)’:
/home/kisik21/laminar/src/laminar.cpp:689:48: error: invalid conversion from ‘size_t* {aka unsigned int*}’ to ‘uLongf* {aka long unsigned int*}’ [-fpermissive]
             (unsigned char*)&r->log[0], logsize) == Z_OK) {
                                                ^

additionally, I had to install rapidjson for build to process, it happens to be included somewhere.

I am trying to execute a script over ssh, but get different results when compared to starting a local script.

Executing a script on the remote machine:

This is the pkg-builder-remote.run file on laminar server

#!/bin/bash
echo Running on robor ...
ssh [email protected] /bin/bash /root/test.sh
echo Done

And this is the test.sh file on the remote server:

#!/bin/bash
C_NAME=pkg-builder-1
echo $C_NAME
lxc launch -e -p prf-funtoo -p default fun-generic "$C_NAME"
sleep 3
lxc exec "$C_NAME" ip a add 10.214.101.2/24 dev eth0
lxc exec "$C_NAME" -- ping -c3 10.214.101.1
echo "Waiting 3 seconds"
sleep 3
lxc exec "$C_NAME" -- poweroff

Results from the run as displayed on laminar:
pkg-builder-remote #1

Console output

[laminar] Executing /var/lib/laminar/cfg/jobs/pkg-builder-remote.run
Running on robor ...
pkg-builder-1
Creating pkg-builder-1


Starting pkg-builder-1
PING 10.214.101.1 (10.214.101.1) 56(84) bytes of data.
64 bytes from 10.214.101.1: icmp_seq=1 ttl=64 time=0.152 ms
64 bytes from 10.214.101.1: icmp_seq=2 ttl=64 time=0.056 ms
64 bytes from 10.214.101.1: icmp_seq=3 ttl=64 time=0.054 ms

--- 10.214.101.1 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2118ms
rtt min/avg/max/mdev = 0.054/0.087/0.152/0.046 ms
Waiting 3 seconds
Done

Executing the script over ssh connection

This is the pkg-builder.run file on laminar server:

#!/bin/bash
C_NAME=pkg-builder-$RUN

echo Connecting to remote host and starting container $C_NAME
ssh [email protected] "env C_NAME=$C_NAME /bin/bash -xe"<<"EOF"
        echo $C_NAME
        lxc launch -e -p prf-funtoo -p default fun-generic "$C_NAME"
        sleep 3
        lxc exec "$C_NAME" ip a add 10.214.101.2/24 dev eth0
        lxc exec "$C_NAME" -- ping -c3 10.214.101.1
        echo "Waiting 3 seconds"
        sleep 3
        lxc exec "$C_NAME" -- poweroff
EOF

This is the output from laminar:

Console output

[laminar] Executing /var/lib/laminar/cfg/jobs/pkg-builder.run
We are going to start container:
Connecting to remote host and starting container pkg-builder-37
+ lxc launch -e -p prf-funtoo -p default fun-generic pkg-builder-37
Creating pkg-builder-37


Starting pkg-builder-37
+ sleep 3
+ lxc exec pkg-builder-37 ip a add 10.214.101.2/24 dev eth0

So the ping command and sleep and poweroff commands never get executed.

Am I doing something wrong?

Thanks

• It would be nice to spot which jobs suffered regression or recovered, but that might be too specific to my use-case. So like last failed job where previous job succeeded or vice versa show jobs that failed before but are ok after last run.
• pass rate of the package/job sorted
• how many jobs from a group failed%.
• show jobs that are taking longer/shorter then before (significantly)
• show average job run