coreos / go-systemd Goto Github PK
View Code? Open in Web Editor NEWGo bindings to systemd socket activation, journal, D-Bus, and unit files
License: Apache License 2.0
Go bindings to systemd socket activation, journal, D-Bus, and unit files
License: Apache License 2.0
Hi,
Currently github.com/coreos/pkg/capnslog/journald_formatter imports github.com/coreos/go-systemd/journal, but github.com/coreos/go-systemd/journal and github.com/coreos/go-systemd/sdjournal/journal import github.com/coreos/pkg/dlopen. This is a pain for distro developers: we want to package both github.com/coreos/go-systemd and github.com/coreos/pkg but cycles in the dependency graph are a pain. In this case it looks as if .../pkg/dlopen could move to ../go-systemd but I've no idea if that would really be appropriate.
Splitting things out into separate repos would also always be a fix for this.
Cheers,
mwh
Right now, sdjournal
only lets you access fields you already know to expect. The C library has sd_journal_enumerate_data
for accessing all fields of a single message, and sd_journal_enumerate_fields
for listing all fields that all messages in the journal contain.
https://www.freedesktop.org/software/systemd/man/sd_journal_get_data.html
https://www.freedesktop.org/software/systemd/man/sd_journal_enumerate_fields.html
I am working on enabling systemd-cgroup support in runc by adding a flag and seeing this error.
Here is the branch
https://github.com/opencontainers/runc/compare/master...mrunalp:systemd_cgroups?expand=1
[root@dhcp-16-129 testroot]# runc --systemd-cgroup start 1234
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x0 pc=0x68839f]
goroutine 1 [running]:
panic(0x7a5460, 0xc820012130)
/usr/lib/golang/src/runtime/panic.go:464 +0x3e6
github.com/coreos/go-systemd/dbus.(*Conn).startJob(0x0, 0x0, 0x8a26e0, 0x33, 0xc820136100, 0x4, 0x4, 0x0, 0x0, 0x0)
/root/gosrc/src/github.com/opencontainers/runc/Godeps/_workspace/src/github.com/coreos/go-systemd/dbus/methods.go:47 +0x32f
github.com/coreos/go-systemd/dbus.(*Conn).StartTransientUnit(0x0, 0xc820122210, 0xf, 0x81f300, 0x7, 0xc820148000, 0x7, 0x8, 0x0, 0xc820122245, ...)
/root/gosrc/src/github.com/opencontainers/runc/Godeps/_workspace/src/github.com/coreos/go-systemd/dbus/methods.go:137 +0x2d5
github.com/opencontainers/runc/libcontainer/cgroups/systemd.(*Manager).Apply(0xc82000fac0, 0x27a3, 0x0, 0x0)
/root/gosrc/src/github.com/opencontainers/runc/Godeps/_workspace/src/github.com/opencontainers/runc/libcontainer/cgroups/systemd/apply_systemd.go:229 +0x1086
github.com/opencontainers/runc/libcontainer.(*initProcess).start(0xc820132000, 0x0, 0x0)
/root/gosrc/src/github.com/opencontainers/runc/Godeps/_workspace/src/github.com/opencontainers/runc/libcontainer/process_linux.go:247 +0x41e
github.com/opencontainers/runc/libcontainer.(*linuxContainer).Start(0xc8200ce180, 0xc8200e6d80, 0x0, 0x0)
/root/gosrc/src/github.com/opencontainers/runc/Godeps/_workspace/src/github.com/opencontainers/runc/libcontainer/container_linux.go:189 +0x1d2
main.(*runner).run(0xc8200d6f90, 0xc820090930, 0x8, 0x0, 0x0)
/root/gosrc/src/github.com/opencontainers/runc/utils.go:342 +0x630
main.startContainer(0xc82008ec60, 0xc820090900, 0x0, 0x0, 0x0)
/root/gosrc/src/github.com/opencontainers/runc/start.go:122 +0x406
main.glob.func11(0xc82008ec60)
/root/gosrc/src/github.com/opencontainers/runc/start.go:72 +0x17b
github.com/codegangsta/cli.Command.Run(0x81f838, 0x5, 0x0, 0x0, 0x0, 0x0, 0x0, 0x8703a0, 0x1a, 0x8d16a0, ...)
/root/gosrc/src/github.com/opencontainers/runc/Godeps/_workspace/src/github.com/codegangsta/cli/command.go:137 +0x1081
github.com/codegangsta/cli.(*App).Run(0xc82008ea20, 0xc82000a200, 0x4, 0x4, 0x0, 0x0)
/root/gosrc/src/github.com/opencontainers/runc/Godeps/_workspace/src/github.com/codegangsta/cli/app.go:176 +0xffa
main.main()
/root/gosrc/src/github.com/opencontainers/runc/main.go:123 +0xc8e
Perhaps I should have clued into this more when c224373 went by but we need the ability to control jobs without waiting for them on all job control methods. This is required for coreos-cloudinit which should never call the blocking API since that can result in deadlocks.
This is a tracking issue for the upstream Golang bug that keeps the large journal entry stuff from working right now. Large journal entries won't work until this is fixed upstream in Golang:
We encountered this issue today twice on a production system. We haven't yet identified the root cause, however this appeared to be an ancillary issue.
The host was running out of memory, likely from journald. It attempted to compress and rotate logs, but was unable to allocate the memory. Journald died but was restarted. However' we're also using the docker journald log driver along with journald's syslog transport. After the restart, all docker containers failed to continue logging with EPIPE
errors to stdout/stderr.
In looking at the journal
code, it connects to journald on init, however doesn't have any error handling where it might reconnect if journald was restarted.
The log output we had was:
May 26th 2017, 05:34:03.204 System journal (/var/log/journal/) is 1.7G, max 2.6G, 931.8M free.
May 26th 2017, 05:34:04.220 Failed to initialize XZ encoder: code 5
May 26th 2017, 05:34:04.231 systemd-journald.service: Main process exited, code=dumped, status=6/ABRT
May 26th 2017, 05:34:04.232 Failed to compress (unnamed temporary file): Invalid argument
May 26th 2017, 05:34:04.242 Detected coredump of the journal daemon or PID 1, diverted to /var/lib/systemd/coredump/core.systemd-journal.0.9b634f7d87464833a67ca9124f25ab86.14979.1495802022000000.
May 26th 2017, 05:34:04.242 systemd-journald.service: Unit entered failed state.
May 26th 2017, 05:34:04.245 systemd-journald.service: Service has no hold-off time, scheduling restart.
May 26th 2017, 05:34:04.245 systemd-journald.service: Failed with result 'core-dump'.
May 26th 2017, 05:34:04.259 Stopped Flush Journal to Persistent Storage.
May 26th 2017, 05:34:04.259 Stopped Journal Service.
May 26th 2017, 05:34:04.259 Stopping Flush Journal to Persistent Storage...
May 26th 2017, 05:34:04.268 Starting Journal Service...
After that, we only get huge error spikes to New Relic with EPIPE
.
Here in JournalReader.Read(), the returned length can be larger than what's actually been read when the buffer provided is not big enough.
Which will cause a panic in https://golang.org/src/bytes/buffer.go#L177. (go 1.6.2 linux/amd64)
I found this when doing kubectl logs on some pods
.
Here is the panic trace from the rkt api service:
Jun 11 01:05:48 yifan-test-rkt-minion-group-fc7i rkt[1212]: panic: runtime error: slice bounds out of range [recovered]
Jun 11 01:05:48 yifan-test-rkt-minion-group-fc7i rkt[1212]: panic: runtime error: slice bounds out of range
Jun 11 01:05:48 yifan-test-rkt-minion-group-fc7i rkt[1212]: goroutine 6580 [running]:
Jun 11 01:05:48 yifan-test-rkt-minion-group-fc7i rkt[1212]: panic(0xdbd0c0, 0xc82000c030)
Jun 11 01:05:48 yifan-test-rkt-minion-group-fc7i rkt[1212]: /usr/lib/go-1.6/src/runtime/panic.go:481 +0x3e6
Jun 11 01:05:48 yifan-test-rkt-minion-group-fc7i rkt[1212]: io/ioutil.readAll.func1(0xc8201b9950)
Jun 11 01:05:48 yifan-test-rkt-minion-group-fc7i rkt[1212]: /usr/lib/go-1.6/src/io/ioutil/ioutil.go:30 +0x11e
Jun 11 01:05:48 yifan-test-rkt-minion-group-fc7i rkt[1212]: panic(0xdbd0c0, 0xc82000c030)
Jun 11 01:05:48 yifan-test-rkt-minion-group-fc7i rkt[1212]: /usr/lib/go-1.6/src/runtime/panic.go:443 +0x4e9
Jun 11 01:05:48 yifan-test-rkt-minion-group-fc7i rkt[1212]: bytes.(*Buffer).ReadFrom(0xc8201b98a8, 0x7f9d80c0b780, 0xc82012c428, 0x0, 0x0, 0x0)
Jun 11 01:05:48 yifan-test-rkt-minion-group-fc7i rkt[1212]: /usr/lib/go-1.6/src/bytes/buffer.go:177 +0x322
Jun 11 01:05:48 yifan-test-rkt-minion-group-fc7i rkt[1212]: io/ioutil.readAll(0x7f9d80c0b780, 0xc82012c428, 0x200, 0x0, 0x0, 0x0, 0x0, 0x0)
Jun 11 01:05:48 yifan-test-rkt-minion-group-fc7i rkt[1212]: /usr/lib/go-1.6/src/io/ioutil/ioutil.go:33 +0x156
Jun 11 01:05:48 yifan-test-rkt-minion-group-fc7i rkt[1212]: io/ioutil.ReadAll(0x7f9d80c0b780, 0xc82012c428, 0x0, 0x0, 0x0, 0x0, 0x0)
Jun 11 01:05:48 yifan-test-rkt-minion-group-fc7i rkt[1212]: /usr/lib/go-1.6/src/io/ioutil/ioutil.go:42 +0x51
Jun 11 01:05:48 yifan-test-rkt-minion-group-fc7i rkt[1212]: main.(*v1AlphaAPIServer).GetLogs(0xc82012c418, 0xc8200485c0, 0x7f9d80c0b730, 0xc8203de760, 0x0, 0x0)
Jun 11 01:05:48 yifan-test-rkt-minion-group-fc7i rkt[1212]: /opt/build-rkt/gopath/src/github.com/coreos/rkt/rkt/api_service.go:767 +0x734
Jun 11 01:05:48 yifan-test-rkt-minion-group-fc7i rkt[1212]: github.com/coreos/rkt/api/v1alpha._PublicAPI_GetLogs_Handler(0xe69940, 0xc82012c418, 0x7f9d80c0b698, 0xc82013c960, 0x0, 0x0)
Jun 11 01:05:48 yifan-test-rkt-minion-group-fc7i rkt[1212]: /opt/build-rkt/gopath/src/github.com/coreos/rkt/api/v1alpha/api.pb.go:1088 +0x175
Jun 11 01:05:48 yifan-test-rkt-minion-group-fc7i rkt[1212]: google.golang.org/grpc.(*Server).processStreamingRPC(0xc8202d9740, 0x7f9d80bf6158, 0xc82019abd0, 0xc820372540, 0xc820312040, 0x1580720, 0xc820199230, 0x0, 0x0)
Jun 11 01:05:48 yifan-test-rkt-minion-group-fc7i rkt[1212]: /opt/build-rkt/gopath/src/github.com/coreos/rkt/Godeps/_workspace/src/google.golang.org/grpc/server.go:421 +0x2dc
Jun 11 01:05:48 yifan-test-rkt-minion-group-fc7i rkt[1212]: google.golang.org/grpc.(*Server).handleStream(0xc8202d9740, 0x7f9d80bf6158, 0xc82019abd0, 0xc820372540, 0xc820199230)
Jun 11 01:05:48 yifan-test-rkt-minion-group-fc7i rkt[1212]: /opt/build-rkt/gopath/src/github.com/coreos/rkt/Godeps/_workspace/src/google.golang.org/grpc/server.go:493 +0x114e
Jun 11 01:05:48 yifan-test-rkt-minion-group-fc7i rkt[1212]: google.golang.org/grpc.(*Server).Serve.func2.1.1(0xc8202d9740, 0x7f9d80bf6158, 0xc82019abd0, 0xc820372540, 0xc820199230, 0xc8202d2420)
Jun 11 01:05:48 yifan-test-rkt-minion-group-fc7i rkt[1212]: /opt/build-rkt/gopath/src/github.com/coreos/rkt/Godeps/_workspace/src/google.golang.org/grpc/server.go:278 +0x49
Jun 11 01:05:48 yifan-test-rkt-minion-group-fc7i rkt[1212]: created by google.golang.org/grpc.(*Server).Serve.func2.1
Jun 11 01:05:48 yifan-test-rkt-minion-group-fc7i rkt[1212]: /opt/build-rkt/gopath/src/github.com/coreos/rkt/Godeps/_workspace/src/google.golang.org/grpc/server.go:280 +0x5ac
Jun 11 01:05:48 yifan-test-rkt-minion-group-fc7i systemd[1]: rkt-api-service.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
Jun 11 01:05:48 yifan-test-rkt-minion-group-fc7i systemd[1]: rkt-api-service.service: Unit entered failed state.
Jun 11 01:05:48 yifan-test-rkt-minion-group-fc7i systemd[1]: rkt-api-service.service: Failed with result 'exit-code'.
Currently the util
module is not covered by any tests. We should start adding at least some basic ones, which would help avoiding breaking things (e.g. #192) during refactoring.
Hi everyone. I was wondering if a tag could be made for a release of go-systemd? I've been working on updating the Debian packaging for go-systemd and would like to package a release instead of merely a snapshot.
I was going to go out on a limb and say that since there haven't been many commits recently maybe a release tag is in order but there have been a few fixes and merges since I last updated my local copy. Obviously it's up to you guys.
Thanks,
Tim.
My usual first step of looking at a go repo is to go install ./... from the repo root, to make sure everything compiles. But you can't do that with go-systemd, because you get this:
# github.com/coreos/go-systemd/examples/activation
examples/activation/listen.go:25: fixListenPid redeclared in this block
previous declaration at examples/activation/activation.go:25
examples/activation/listen.go:34: main redeclared in this block
previous declaration at examples/activation/activation.go:34
examples/activation/udpconn.go:26: fixListenPid redeclared in this block
previous declaration at examples/activation/listen.go:25
examples/activation/udpconn.go:35: main redeclared in this block
previous declaration at examples/activation/listen.go:34
Maybe it would be prudent to move each example into their own directory?
dbus.Conn.startJob() adds the job to the joblisteners map:
c.jobListener.jobs[path] = ch
But while jobComplete reads this it doesn't actually remove it from the map.
sudo ./test
Running tests...
ok github.com/coreos/go-systemd/activation 4.550s coverage: 0.0% of statements
? github.com/coreos/go-systemd/journal [no test files]
ok github.com/coreos/go-systemd/login1 0.003s coverage: 41.9% of statements
ok github.com/coreos/go-systemd/machine1 0.003s coverage: 65.0% of statements
ok github.com/coreos/go-systemd/unit 0.002s coverage: 93.7% of statements
ok github.com/coreos/go-systemd/sdjournal 5.279s coverage: 36.7% of statements
--- FAIL: TestEnableDisableUnit (0.00s)
methods_test.go:353: Invalid argument
--- FAIL: TestGetUnitProperties (0.00s)
methods_test.go:382: / is unwanted
--- FAIL: TestSetUnitProperties (0.00s)
methods_test.go:454: Unit tmp.mount not found.
FAIL
coverage: 63.1% of statements
FAIL github.com/coreos/go-systemd/dbus 2.143s
sd_journal_wait appears from the man page to take a (relative) timeout value but the code passes in an absolute wall-clock time, which will presumably be massively larger than what we meant
In sdjournal.Wait():
to = uint64(time.Now().Add(timeout).Unix() / 1000)
should be
to = uint64(timeout / 1000)
It certainly works better with this change at my end.
forgive me if there's an obvious solution, but the journal code looks unusable since one can't initialize the socket connection from outside the library.
Hi!
I have a problem with
Line 218 in e97b35f
I'm using a slightly modified version of (r *JournalReader) Follow()
returning a channel of *JournalEntry
s but the waiting logic is preserved so I'm writing here.
I noticed that after some time the overall number of active goroutines grows slowly and differs from one host to another:
goroutine profile: total 29
17 @ 0x42ef3a 0x42f02e 0x406038 0x405dfd 0x4fa712 0x45fc51
# 0x4fa711 github.com/mheese/journalbeat/journal.Follow.func2.1+0x91 /github.com/mheese/journalbeat/journal/follow.go:86
Here is an example of a single goroutine dump:
goroutine 180549 [chan send, 16014 minutes]:
github.com/mheese/journalbeat/journal.Follow.func2.1(0xc4215d8230, 0xc4215d81c0, 0xc420177920)
/github.com/mheese/journalbeat/journal/follow.go:86 +0x92
created by github.com/mheese/journalbeat/journal.Follow.func2
/github.com/mheese/journalbeat/journal/follow.go:89 +0x29e
Here is a rough repro: https://play.golang.org/p/PNGLjD_S5m
When using a buffered channel of size 1 you are likely to get one event already sent to channel buffer, and one event pending to be sent to the channel before you get a pollDone
signal. This causes the aforementioned issue with orphaned goroutines being stuck.
A couple of questions in this regard:
select
?default: events <- ...
? Both of them being reasons for a leak.Hello, I'm trying to connect to systemd --user, using dbus.NewUserConnection()
dbus.service and dbus.socket are both running
by default DBUS_SESSION_ADDRESS is set to
unix:abstract=/tmp/dbus-RdwKN0BXaL,guid=6b95481a972dfb9cbd5654b557024934
when trying to connect it shows this error
Process org.freedesktop.systemd1 exited with status 1
I tried changing it to the socket in the dbus.socket
unix:path=/run/user/10561/dbus/user_bus_socket
but then it fails with
Activation of org.freedesktop.systemd1 timed out
looking at the journald logs I see
dbus-daemon[910]: Failed to activate service 'org.freedesktop.systemd1': timed out
any ideas what might be wrong?
Thanks,
Kiril
#164 (comment) seems to have highlighted a race between Match/Send/Wait/Next, where Wait
returns after seeing the matching entry but Next
reports and EOF. Increasing the Wait timeout doesn't seem to help, so there may be a race somewhere.
After #178 and #179, this is the only remaining testsuite failure. It doesn't happen always, but seem very frequent on debian/ubuntu systems (I've seen it often locally and on containerized travis).
We originally discovered in fleet [0] that systemd silently truncates lines at 2000 characters. This has been confirmed upstream [1]. Unfortunately, I believe we need to add this check to our unit file parser so we can give a reasonable experience to users rather than making them chase down the same rabbit hole that I did.
[0] coreos/fleet#992
[1] https://bugs.freedesktop.org/show_bug.cgi?id=85308
Hi there,
I was taking a looking at go-systemd's machined functionality and I was wondering how come there's only RegisterMachine()
?
If adding additional functionality in there is something desired, I'd be happy to submit a pull request adding features such as GetMachine, GetImage, ListMachines, TerminateMachine, etc. And if it'd be preferred that I add each one in a separate Pull Request, I can do that as well.
Thanks!
It looks like someone refactored the serialization of the unit directives to group them, but then failed to update the tests.
Repro steps
Result:
The process will hang
Expected:
Return an error telling me I've asked to do the impossible
Here's a script to illustrate the problem:
package main
import (
systemdDbus "github.com/coreos/go-systemd/dbus"
"github.com/guelfey/go.dbus"
"os"
)
func main() {
systemd := systemdDbus.New()
path := dbus.ObjectPath(os.Args[1])
info, _ := systemd.GetUnitInfo(path)
println(info["ActiveState"].Value().(string))
}
Build this and run something like
./test ./test /org/freedesktop/systemd1/unit/ping.service
vs.
./test /org/freedesktop/systemd1/unit/ping_2eservice
This describes valid dbus paths: http://dbus.freedesktop.org/doc/dbus-specification.html#message-protocol-marshaling-object-path
This should illustrate what encoding systemd expects: http://cgit.freedesktop.org/systemd/systemd/tree/src/shared/util.c#n1360
The methods are documented here:
FYI I'm currently working on implementing these methods in the notify_fds
branch of my fork, and I have a basic POC working (but without proper tests at the moment). I plan to contribute back my work under the CoreOS contribution guidelines. The use case is that we (Uber) have services where a single process needs to listen on multiple well-known ports, and we'd like to implement graceful restarts of these services.
Sometimes it is helpful to look at an enabled unit file. I'd rather not go look up the file in /etc/systemd, etc., which is kind of the point of using the Go bindings in the first place. :) At the commandline I can just run "systemctl cat ...". Having an equivalent method on dbus.Conn would really helpful. I expect it would dovetail onto work for issue #77.
As a temporary workaround I am reading the dbus properties I need (after hunting down the mapping from the conf directives) and manually converting them into unit.UnitOption values. From there I can use them or call unit.Serialize to get the unit file. However that's a lot of work, full of risk of getting something wrong (or out of date), for something that I would expect is not an out-of-the-ordinary need.
Since the unit directives were grouped into sections using a map, when iterating over the map to serialize the data, the ordering is no longer deterministic due to the randomisation of map iteration that was added into golang.
FYI. When I test sdjournal, I got this error:
systemd/sd-journal.h: No such file or directory
compilation terminated
Disclaimer: I'm not normally a Go developer, so some of my terms may not be quite right.
I'm trying to build a Docker plugin that use the go-plugins-helpers
from Docker and I wanted the binary to run in Docker alpine
, but alpine
doesn't have the linked dynamic binaries or whatever, so I was reading up that I need to set CGO_ENABLED=0
when building. I've tried a few different build commands, including:
CGO_ENABLED=0 go build -a -installsuffix cgo -ldflags '-s' -o bin/linux/amd64/local-persist -v
CGO_ENABLED=0 go build -a -tags netgo -a -o bin/linux/amd64/local-persist -v
(Tried that last one with and without CGO_ENABLED
flag. No error, but still won't run in alpine
.)
I keep getting this error:
../github.com/docker/go-plugins-helpers/sdk/unix_listener.go:12:2: no buildable Go source files in /go/src/github.com/coreos/go-systemd/util
make: *** [binary-linux-amd64] Error 1
Steps to reproduce yourself:
git clone https://github.com/CWSpear/local-persist
cd local-persist
docker run -it -v `pwd`:/go/src/app golang:onbuild bash
# now you'll be inside the docker container, next run:
export CGO_ENABLED=0
go get -v -d
go build [whatever flags] .
When attempting to compile an application that depends on go-systemd, I get error messages from pkg-config failing to find lsystemd
. I've produced this on images derived from golang:1.5
and ubuntu:14.04
with dockerfiles along these lines:
FROM golang:1.5
RUN apt-get update
RUN apt-get -y install build-essentials pkg-config libsystemd-journal-dev
# compile go application here
I've gotten a range of different errors by playing with the combination of the many transitional libsystemd packages that I install:
# pkg-config --cflags libsystemd
Package libsystemd was not found in the pkg-config search path.
Perhaps you should add the directory containing `libsystemd.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libsystemd' found
pkg-config: exit status 1
# pkg-config --cflags libsystemd-journal
Package libsystemd-journal was not found in the pkg-config search path.
Perhaps you should add the directory containing `libsystemd-journal.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libsystemd-journal' found
pkg-config: exit status 1
# github.com/coreos-inc/quay-builder
/usr/local/go/pkg/tool/linux_amd64/link: running gcc failed: exit status 1
/usr/bin/ld: cannot find -lsystemd
collect2: error: ld returned 1 exit status
go-systemd currently relies on an existing dbus system bus in order to invoke systemd methods. However, this may be not strictly needed as systemd itself exposes a UNIX socket for one-to-one dbus calling at /var/run/systemd/private
.
It may be possible to implement a fallback mechanism, in order to retry connecting to the p2p endpoint if the system bus doesn't exist. However this needs a bit more investigation to determine if this works in a transparent way, and the same set of methods/properties are available.
Hi go-systemd team!
Many thanks for all the work you do!
I'm packaging go-systemd for Ubuntu (as part of the juju project).
We do license checks for all the code we package as part of our packaging pipeline.
It seems to me that one file in your repo violates requirements of the Apache-2.0 license.
Could you please help me to understand if it's true or not and (if so) fix the issue in your repo?
File in question: https://github.com/coreos/go-systemd/blob/master/daemon/sdnotify.go
File states that it is 'forked from Docker project' which is correct.
Original file from Docker project can be found here:
https://github.com/docker/docker/blob/master/pkg/systemd/sd_notify.go
In terms of Apache-2.0 license forked version of the file is called Derivative Work.
Docker project is licensed under Apache-2.0 license (as well as go-systemd project).
This license allows you to redistribute Derivative Works only if you obey the following rules:
(a) You must give any other recipients of the Work or Derivative Works a copy of this License;
While both projects are license under Apache-2.0 license, text is a bit different between the licenses.
Go-systemd license: https://github.com/coreos/go-systemd/blob/master/LICENSE
Docker license: https://github.com/docker/docker/blob/master/LICENSE
You may see that at least formatting of paragraph 4 is different. It means that you need to include Docker's license into your project (maybe put it inside daemon/ folder?)
and put a reference to it from sdnotify.go file.
(b) You must cause any modified files to carry prominent notices stating that You changed the files;
While 'forked from Docker project' looks okay in the sense of this requirement I'd recommend you to specify which exact file you've modified (link to Docker's github repo may work).
(c) You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices <...>
Docker's sd_notify.go file doesn't contain explicit copyright notices but they are available in Docker's LICENSE file:
https://github.com/docker/docker/blob/master/LICENSE#L179
You may find there: 'Copyright 2013-2015 Docker, Inc.'
This copyright information needs to be included into daemon/sdnotify.go.
You may add your own copyright notice (for your changes) but you shouldn't remove Docker's copyright.
(d) If the Work includes a "NOTICE" text file <...> then any Derivative Works <...> must include a readable copy of the attribution notices contained within such NOTICE file <...>
Docker project has such NOTICE file: https://github.com/docker/docker/blob/master/NOTICE
This file contains some valuable information and I'd recomment you to put it into daemon/ folder as well.
It may be linked from daemon/sd_notify.go as well as the LICENSE file.
Thanks A LOT for helping!
Oleg
Hello there,
I am trying to improve the performance of dbus
methods and I realized that the linking operation is quite expensive. This operation becomes a bottleneck whenever you need to link-start
many systemd units. In fact, the response time increases over the time whenever you want to link multiple units.
As a result of my investigation, I found that the force (force=true)
property of the dbus
function doesn't bring any benefit (when using the latest version of dbus). It is true that the force
property set to true
could bring some consistency in case of race condition. However, I think it should be configured to be false
to offer a better performance.
func (c *Conn) LinkUnitFiles(files []string, runtime bool, force bool) ([]LinkUnitFileChange, error) {
result := make([][]interface{}, 0)
err := c.sysobj.Call("org.freedesktop.systemd1.Manager.LinkUnitFiles", 0, files, runtime, force).Store(&result)
if err != nil {
return nil, err
}
What is it your impression ? The improvements in terms of performance are in the order of seconds and could change the complexity of this operation.
Why did you decide to define force=true
as default ?
Thanks.
I think it will be better if this function returns 0 in the case that $NOTIFY_SOCKET is not set as stated in sd_notify() documentation.
Basically this TODO needs to be implemented:
func (s *SubscriptionSet) Subscribe() (<-chan map[string]*UnitStatus, <-chan error) {
// TODO: Make fully evented by using systemd 209 with properties changed values
return s.conn.SubscribeUnitsCustom(time.Second, 0,
mismatchUnitStatus,
func(unit string) bool { return s.filter(unit) },
)
}
can't accurately monitor units being restarted without calling SubscribeUnitsCustom with a very short delay. which wastes a bunch of cpu.
There should be a isSystemd helper function to help services determine if they are running under systemd. This will aid services in support more systemd features such as sdnotify, without breaking ignoring errors when not running on systemd systems.
Since #110 go-systemd now uses Cgo, and depends on both libsystemd-journal
(at build and runtime) and pkg-config
(at build). These new dependencies are only used by the code for reading from the systemd journal.
This is moderately inconvenient, as previously go-systemd only used native Go packages to communicate with the journal or systemd.
Ideally native Go could be used, but maybe hiding the journal-reading feature behind a build flag hides the pain.
Instead of using a timer in SubscribeUnits react to PropertiesChanged event:
http://lists.freedesktop.org/archives/systemd-devel/2013-October/013546.html
Looks like an upstream dependency changed:
$ go get github.com/coreos/go-systemd/dbus
src/github.com/coreos/go-systemd/dbus/dbus.go:186: cannot use conn.Object("org.freedesktop.systemd1", dbus.ObjectPath("/org/freedesktop/systemd1")) (type dbus.BusObject) as type *dbus.Object in return argument: need type assertion
~/.../gopath/src/github.com/coreos/go-systemd/dbus$ go test
# github.com/coreos/go-systemd/dbus
./dbus.go:67: undefined: dbus.BusObject
./dbus.go:71: undefined: dbus.BusObject
FAIL github.com/coreos/go-systemd/dbus [build failed]
Not sure when it broke, but it broke.
I'm filing this because I want SkyDNS on windows, but I won't resent a "wont fix" close. Obviously there is no hope of systemd on windows....
C:\Users\rober_000\Documents>go build github.com/skynetservices/skydns
github.com/coreos/go-systemd/activation
C:\Projects\Go\src\github.com\coreos\go-systemd\activation\files.go:47: cannot use fd (type int) as type syscall.Handle in argument to syscall.CloseOnExec
ParseProperties(props []Property, unitType string) ([]_unit.UnitOption, error)
ParseOptions(opts []_unit.UnitOption) ([]Property, string, error)
From what I can tell, all the unit options [1] map into dbus properties [2](though the reverse is not true, of course). I've done something like this for a limited sub-set, but it would be helpful for a number of reasons if this functionality existed upstream (here).
[1] http://www.freedesktop.org/software/systemd/man/systemd.directives.html
[2] http://www.freedesktop.org/wiki/Software/systemd/dbus/
The value of the ExecStart
in the following unit file ...
[Unit]
Description=PING
[Service]
ExecStart=/bin/bash -c "while true; do echo \"ping\"; sleep 1; done"
...is being parsed /bin/bash -c "while true; do echo \"ping
. I expect it to be /bin/bash -c "while true; do echo \"ping\"; sleep 1; done"
.
It will be better if provide API for read journal like python
http://www.freedesktop.org/software/systemd/python-systemd/journal.html#accessing-the-journal
core@coreos-test ~ $ journalctl
Error was encountered while opening journal files: Invalid argument
core@coreos-test ~ $ cat /etc/os-release
NAME=CoreOS
ID=coreos
VERSION=723.3.0
VERSION_ID=723.3.0
BUILD_ID=
PRETTY_NAME="CoreOS 723.3.0"
ANSI_COLOR="1;32"
HOME_URL="https://coreos.com/"
BUG_REPORT_URL="https://github.com/coreos/bugs/issues"
Unfortunately the ListUnits
method does not fill my needs. An enabled (but inactive) unit may not show up under some conditions. I've run into this consistently.
The implementation should be virtually identical to ListUnits
. ListUnitFiles
is a dbus method on the manager object (per the docs).
What Happens:
Calling the ListUnitsByNames
function on *dbus.Conn
results in an unexpected error: Unknown method 'ListUnitsByNames' or interface 'org.freedesktop.systemd1.Manager'
What I expected to happen:
A slice of UnitStatus
and no error, or a more descriptive error message.
How to reproduce:
Sample Code:
package main
import (
"fmt"
"github.com/coreos/go-systemd/dbus"
)
func main() {
conn, err := dbus.New()
if err != nil {
fmt.Println("error getting connection: ", err)
}
res, err := conn.ListUnitsByNames([]string{"acpid.service"})
if err != nil {
fmt.Println("error with ListUnitsByNames: ", err)
return
}
fmt.Printf("got %d results\n", len(res))
}
Test system:
uname -a; lsb_release -a
Linux rebecca-vboxdev 4.4.0-62-generic #83-Ubuntu SMP Wed Jan 18 14:10:15 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.2 LTS
Release: 16.04
Codename: xenial
This should be an extension of the existing GetUnitProperties(unit string)
interface. Probably a prototype like GetUnitProperty(unit string, propertyName string) systemd.Property
https://github.com/coreos/go-systemd/blob/master/dbus/methods.go#L164
Hi there,
Would there be any opposition to implementing support for importd? importd docs found here.
I'd be happy to implement this and submit a pull request is this is something desired.
Thanks
In coreos/go-systemd/activation/listeners.go, line 34 method net.FileListener() is used to create net.Listener from file. For unix domain sockets it returns UnixListener, whose method Close() unlinks it, which leads to deletion of the file in the file system. Here is method Close() from unixsock_posix.go:
// Close stops listening on the Unix address. Already accepted
// connections are not closed.
func (l *UnixListener) Close() error {
if l == nil || l.fd == nil {
return syscall.EINVAL
}
// The operating system doesn't clean up
// the file that announcing created, so
// we have to clean it up ourselves.
// There's a race here--we can't know for
// sure whether someone else has come along
// and replaced our socket name already--
// but this sequence (remove then close)
// is at least compatible with the auto-remove
// sequence in ListenUnix. It's only non-Go
// programs that can mess us up.
if l.path[0] != '@' {
syscall.Unlink(l.path)
}
return l.fd.Close()
}
I think this behavior conflicts with socket activation, because systemd doesn't anticipate this and doesn't recreate file or replace socket. So, when I kill/reload a socket activated service (for test), systemd restarts it with the same socket. but without correspondent file, which leads to immediate crash of service with "file not found" error. In my opinion, this disable important use case of socket activation.
I tried to use abstract names (starting with '@') and they work perfectly (see 'if' block in the code above). Unfortunately, server, which I use as a proxy, doesn't support them.
I am about to implement somewhat cumbersome solution in my own code, but may be you have a better idea how to deal with this issue (may be in your lib, or in it's client code). My idea is to wrap net.Listener, returned from activation.Listeners() with my implementation of net.Listener interface and replace method Close() so, that for UnixListener it do nothing, but with preceding invocation of activation.Files() I am going to obtain and remember file descriptors to be able to close them before exit. A bit better, I can store file descriptor in the wrapper as well and close it in my Close() method.
We're using journald-cloudwatch-logs, which is a Go application that sends our servers' Journald logs to Amazon Web Services' CloudWatch Logs service.
Up until very recently โ perhaps the last couple of days โ I was happily installing journald-cloudwatch-logs via go get
as part of a Chef run. However, it's started bombing, and it seems to be failing while installing prerequisites, one of which is go-systemd-sdjournal. I tracked down the failure and was able to reproduce it while installing go-systemd-sdjournal on its own. Here's how the failure looks:
[root@server ~]# go get github.com/coreos/go-systemd/sdjournal
# github.com/coreos/go-systemd/sdjournal
/usr/local/src/go/src/github.com/coreos/go-systemd/sdjournal/read.go:164: multiple-value writer.Write() in single-value context
The system's running Red Hat Enterprise Linux 7.2.
I don't have any knowledge of Go and largely am just a monkey installing packages, so I'm not comfortable debugging this. I thought I'd bring this up here to see if anyone has any idea what might be going wrong, or if anyone else is experiencing the same problem.
I'm happy to provide any further details. Just let me know what you need.
Thanks!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.