• create basic cli outline for multiple flag support
• add functionality for ps flag
• add functionality for mems flag
• add functionality for rss flag
• add functionality for fds flag

We need to describe how do contributors submit patches for go-criu?

GitHub pull requests, mailing list, both?

I tried to run make test for this repo and it's not working

[kir@kir-rhat go-criu]$ make test
mkdir -p image
test/piggie
Child forked, pid -1
test/test dump `pidof piggie` image
CRIU version 31500
panic: runtime error: index out of range [3] with length 3

goroutine 1 [running]:
main.main()
	/home/kir/go/src/github.com/checkpoint-restore/go-criu/test/main.go:79 +0xc88
make: *** [Makefile:28: test] Error 2

One thing I can see is test/piggie is run w/o arguments, while it requires a log file as an argument. The other (secondary) issue is lack of error checking.

I have a feeling I am missing something very obvious here; please help.

Steps to reproduce:

• Run crit x <imgs_directory> rss

Cause:

• Panic on line https://github.com/checkpoint-restore/go-criu/blob/master/crit/explore.go#L329 due to vmaIndex exceeding the total length of virtual memory addresses

Traceback:

panic: runtime error: index out of range [53] with length 53

goroutine 1 [running]:
github.com/checkpoint-restore/go-criu/v6/crit.(*crit).ExploreRss(0x140001b7900)
        /Users/rahul/go/src/criu-try/go-criu/crit/explore.go:329 +0xa50
main.glob..func5(0x103453ec0?, {0x14000216300, 0x2, 0x2?})
        /Users/rahul/go/src/criu-try/go-criu/crit/cmd/cli.go:172 +0x290
github.com/spf13/cobra.(*Command).execute(0x103453ec0, {0x140002162c0, 0x2, 0x2})
        /Users/rahul/go/src/criu-try/go-criu/vendor/github.com/spf13/cobra/command.go:920 +0x5b0
github.com/spf13/cobra.(*Command).ExecuteC(0x103453060)
        /Users/rahul/go/src/criu-try/go-criu/vendor/github.com/spf13/cobra/command.go:1044 +0x35c
github.com/spf13/cobra.(*Command).Execute(...)
        /Users/rahul/go/src/criu-try/go-criu/vendor/github.com/spf13/cobra/command.go:968
main.main()
        /Users/rahul/go/src/criu-try/go-criu/crit/cmd/cli.go:216 +0x28

If this seems a valid issue, I'd be glad to take this up!

@snprajwal One of your previous PRs removed

package stats

const (
	StatsDump    = "stats-dump"
	StatsRestore = "stats-restore"
)

Can you re-create those constants and also use it in your stats wrapper functions. Those constants are used in CRI-O and Podman.

@rst0git this would require a new release. 6.2.0 probably

Evaluate between using something from the standard lib like ioutil or os.open for .img file reading as binary and similar or using go-diskfs

• port inf()
• port outf()
• port dinf()

rpc/rpc.pb.go is a generated file but its source is missing...

Would it be possible to commit it to this repository please?

I am trying to figure out, how to go about restoring shelljobs in go-criu. Inside my Go code, I'm launching a child process and dumping it. When I go to restore it, I get the following error every time.
restore failed: operation failed (msg:Error (criu/tty.c:991): tty: Don't have tty to inherit session from, aborting err:0)
As far as I understand it is trying to inherit the shell session that the Go program is running it, so it fails. Am I missing a simple way to do this? I am setting the shelljob paremeter on both dump and restore, and have also tried setting setsid on the child process, but that isn't doing anything. Thanks!

magic-gen need not be a module by itself, it can easily be rewritten into a standalone script that can be run directly without building. The logic can been simplified and logging made more verbose, to enable better debugging. The test in the makefile also uses a samplemagic file which need not be retained.

@snprajwal Do you think you have time to create a 7.1.0 release?

Requires #16 to move ahead with this.

• port image.load
• port image.dump

Hello,

I'm trying to run make but I have the following error:

mkdir -p image
sudo test/piggie
Child forked, pid 10466
sudo test/test dump `pidof piggie` image
CRIU version 31300
Dumping
can't open image dir (open 9991: no such file or directory)Makefile:26: recipe for target 'test' failed
make: *** [test] Error 1

The 'image' directory is there but I don't know why go-criu cannot see it.

Any suggestions?

Thanks,

Copyright statement appears to be missing...

README for https://github.com/golang/protobuf says:

It has been superseded by the google.golang.org/protobuf module, which contains an updated and simplified API, support for protobuf reflection, and many other improvements. We recommend that new code use the google.golang.org/protobuf module.

Versions v1.4 and later of github.com/golang/protobuf are implemented in terms of google.golang.org/protobuf. Programs which use both modules must use at least version v1.4 of this one.

Meaning we need to either switch to google.golang.org/protobuf, or at least use github.com/golang/protobuf v.1.4+.

There will be an error when resuming the frozen program
Here are the mistakes
(criu/tty.c:1014): tty: Don't have tty to inherit session from, aborting
err:0

New features

• crit library

Breaking changes

• Deprecate stats and rpc in favour of crit

Related projects that will need PRs

• runc
• checkpointctl
• podman
• LXD

I'm not sure if go-criu is following SemVer semantics, but if so, it would be great to tag the current (v3.11) release as v3.11.0, so that Go modules can refer to the tag (instead of by commit).

/cc @kolyshkin

I think we should have stricter merge rules.

Currently:

• some PRs are merged with merge commit, some without;
• some PRs are merged by its author without any reviews/LGTMs.

I propose:

1. Require at least one LGTM (both being not from the author) to be able to merge PRs. I would say 2 LGTMs but there are not enough maintainers here.
2. Require merge commits from PR (i.e. disable "rebase and merge" and "squash and merge" options). This is important as github PRs often contains additional information and discussions about the commits. Without a merge PR, it is much harder to track a commit back to the PR it originated from, and thus find that additional information;
3. Require CI jobs to be green in order to merge (just as a precaution mechanism).

If everyone from @checkpoint-restore/go-criu-maintainers agrees, I'll implement the changes. Please vote in comments.

• @adrianreber
• @avagin
• @kolyshkin
• @rst0git

root@test:/home/admin/go-criu/test# rm -rf image/ && go run phaul-main.go 16956
Make server part (socket 4)
Make client part (socket 3)
Migrate
S: start iter
`- grow iter (163) reached
S: start iter
Final stage
Do dump
Failed: operation failed (msg:Error (criu/util.c:981): Unable to restore TCP_CORK (1): Operation not supported
err:0)
exit status 1

My current development environment go 1.16
package main

import (
"github.com/checkpoint-restore/go-criu/v5"
"log"
)

func main() {
c := criu.MakeCriu()
version, err := c.GetCriuVersion()
if err != nil {
log.Fatalln(err)
}
log.Println(version)
}

The following error occurs

github.com/checkpoint-restore/go-criu/v5

github.com\checkpoint-restore\go-criu\[email protected]\main.go:37:14: undefined: syscall.Socketpair
github.com\checkpoint-restore\go-criu\[email protected]\main.go:37:33: undefined: syscall.AF_LOCAL

Currently, the CRIT package serialises protobuf to JSON with camelCase for the field names. This was chosen as a design decision to prevent having to annotate every protobuf field manually with the actual snake_case name in the marshaling/unmarshaling logic. Incidentally, I discovered that the protojson package now allows configuring this particular case via protojson.MarshalOptions (UseProtoNames). Enabling this will reduce the parity between the Go and Python CRIT libraries, since the Python library uses the snake_case names from the proto definitions for the fields. Is this something we should enable? While it will not break API, it will definitely break the JSON output compared to before, and if anyone is using custom logic to parse the JSON (although the shouldn't), it will cause their package to break.

cc @rst0git @adrianreber

The Go version of CRIT currently outputs raw data decoded from files.img. However, it would be more user-friendly if we map constants used in certain fields to human readable strings and decode the value used for IP address. The following example shows a comparison between the output of the Go and Python versions of CRIT:

• Go version of CRIT

        {
            "type": "INETSK",
            "id": 36,
            "isk": {
                "id": 36,
                "ino": 106886,
                "family": 2,
                "type": 1,
                "proto": 6,
                "state": 7,
                "srcPort": 0,
                "dstPort": 0,
                "flags": 2,
                "backlog": 0,
                "srcAddr": [
                    0
                ],
                "dstAddr": [
                    0 
                ],
                "fown": {
                    "uid": 0,
                    "euid": 0,
                    "signum": 0,
                    "pidType": 0,
                    "pid": 0
                },

• Python version of CRIT

        {
            "type": "INETSK",
            "id": 36,
            "isk": {
                "id": 36,
                "ino": 106886,
                "family": "INET",
                "type": "STREAM",
                "proto": "TCP",
                "state": "CLOSE",
                "src_port": 0,
                "dst_port": 0,
                "flags": "0x2",
                "backlog": 0,
                "src_addr": [
                    "0.0.0.0"
                ],
                "dst_addr": [
                    "0.0.0.0"
                ],
                "fown": {
                    "uid": 0,
                    "euid": 0,
                    "signum": 0,
                    "pid_type": 0,
                    "pid": 0
                },

Unfortunately #90 did not resolve the problem with conflicting proto file names. Since protobuf v1.26.0 a hard error will be reported when the compiled Go program starts up that has multiple conflicting protobuf names linked into it.

Steps to reproduce the issue:

1. git clone -b go-criu/v6 [email protected]:rst0git/libpod.git
2. cd libpod && make -j `nproc` && sudo make test

Describe the results you received:
The tests fail with the following error:

panic: proto: file "sa.proto" is already registered
	previously from: "github.com/letsencrypt/boulder/sa/proto"
	currently from:  "github.com/checkpoint-restore/go-criu/v6/crit/images"
See https://developers.google.com/protocol-buffers/docs/reference/go/faq#namespace-conflict


goroutine 1 [running]:
google.golang.org/protobuf/reflect/protoregistry.glob..func1({0x1b0b380?, 0xc0004ce240?}, {0x1b0b380?, 0xc0004ce280})
	/home/rst/tmp/libpod/vendor/google.golang.org/protobuf/reflect/protoregistry/registry.go:56 +0x1ee
google.golang.org/protobuf/reflect/protoregistry.(*Files).RegisterFile(0xc0001a8678, {0x1b204c8?, 0xc0000fc000?})
	/home/rst/tmp/libpod/vendor/google.golang.org/protobuf/reflect/protoregistry/registry.go:130 +0x39c
google.golang.org/protobuf/internal/filedesc.Builder.Build({{0x153c2e0, 0x34}, {0x25a1a80, 0x100, 0x100}, 0x0, 0x1, 0x0, 0x0, {0x1b10fd0, ...}, ...})
	/home/rst/tmp/libpod/vendor/google.golang.org/protobuf/internal/filedesc/build.go:112 +0x1d6
google.golang.org/protobuf/internal/filetype.Builder.Build({{{0x153c2e0, 0x34}, {0x25a1a80, 0x100, 0x100}, 0x0, 0x1, 0x0, 0x0, {0x0, ...}, ...}, ...})
	/home/rst/tmp/libpod/vendor/google.golang.org/protobuf/internal/filetype/build.go:138 +0x19d
github.com/checkpoint-restore/go-criu/v6/crit/images.file_sa_proto_init()
	/home/rst/tmp/libpod/vendor/github.com/checkpoint-restore/go-criu/v6/crit/images/sa.pb.go:190 +0x198
github.com/checkpoint-restore/go-criu/v6/crit/images.file_core_proto_init()
	/home/rst/tmp/libpod/vendor/github.com/checkpoint-restore/go-criu/v6/crit/images/core.pb.go:1109 +0x6f
github.com/checkpoint-restore/go-criu/v6/crit/images.init.12()
	/home/rst/tmp/libpod/vendor/github.com/checkpoint-restore/go-criu/v6/crit/images/core.pb.go:1095 +0x17
path is /home/rst/tmp/libpod/.coverage
Unable to read coverage file cmd/podman/coverprofile to combine, open cmd/podman/coverprofile: no such file or directory

Ginkgo ran 1 suite in 2m10.452270337s
Test Suite Failed

I can confirm that the initial solution renaming core.proto / so.proto files to criu-core.proto / criu-so.proto works as expected.

• check flags for in or default to stdin
• check flags for out or default to stdout
• port image.dump

Hello,

Is there any examples showing how to do a live migration of a simple app from machine A to machine B with go-criu?

The example given in the go-criu repository (phaul-main.go) only shows a very simplistic scenario where the machine A (source) = the machine B (destination).

If the source machine and the destination machine are two separate machines, what is the setup needed in order to realize a simple live migration?

I have looked into the code and here is my understanding:

• Need to setup a shared storage between the two machines for storing image files (to avoid moving/copying the image files).
• Create 2 go programs that will be executed on the two machines (currently in the phaul-main.go example, these two programs are mixed up togother)
  • A phaul client (MakePhaulClient) running on machine A and a phaul server (MakePhaulServer) running on machine B.
  • Use TCP/IP socket instead of unix socket (AF_LOCAL) to setup memfd
  • What I don't understand is that in the phaul-main.go example, the server handler is passed to MakePhaulClient as an argument. How can we do that if there are now two different programs?

Thanks,
Bao.

This issue is being opened to discuss the possibility of removing the vendor dir from the repo. Vendoring enables us to have reproducible builds, but at the cost of updates and security fixes in newer versions of the modules.

Including the upcoming crit library, we will be using three modules - proto, x/sys and protojson. All three are hardened packages with proper testing before every release. Go automatically fetches them when running go build based on the info in go.mod. We can use go get -u to update the deps manually when necessary.

• check flags for in or default to stdin
• check flags for out or default to stdout
• create pretty formatting.

I am currently going through Podman's dependencies (updating the vendor.conf) trying to pin them to releases when possible and available. Some distribution, Debian in particular, are very conservative regarding go packaging as many libraries don't have any release (or no semantic versioning).

Would you consider creating releases of go-criu?

Trying to enable the golangci-lint linter 'stylecheck' I see a couple of reports in crit/.

@snprajwal maybe you could enable the linter and have a look at the reports and fix it if appropriate.

go-criu and runc are causing issues at runtime. I noticed it while updating dependencies over at CRI-O (cri-o/cri-o#2124). go-criu is pulled in via libpod.

Is this a known issue? Do you know how we could workaround that?

[valentin@nebuchadnezzar cri-o]$ ./bin/crio
2019/03/12 15:02:37 proto: duplicate proto type registered: criu_page_server_info
2019/03/12 15:02:37 proto: duplicate proto type registered: criu_veth_pair
2019/03/12 15:02:37 proto: duplicate proto type registered: ext_mount_map
2019/03/12 15:02:37 proto: duplicate proto type registered: join_namespace
2019/03/12 15:02:37 proto: duplicate proto type registered: inherit_fd
2019/03/12 15:02:37 proto: duplicate proto type registered: cgroup_root
2019/03/12 15:02:37 proto: duplicate proto type registered: unix_sk
2019/03/12 15:02:37 proto: duplicate proto type registered: criu_opts
2019/03/12 15:02:37 proto: duplicate proto type registered: criu_dump_resp
2019/03/12 15:02:37 proto: duplicate proto type registered: criu_restore_resp
2019/03/12 15:02:37 proto: duplicate proto type registered: criu_notify
2019/03/12 15:02:37 proto: duplicate proto type registered: criu_features
2019/03/12 15:02:37 proto: duplicate proto type registered: criu_req
2019/03/12 15:02:37 proto: duplicate proto type registered: criu_resp
2019/03/12 15:02:37 proto: duplicate proto type registered: criu_version
panic: proto: duplicate enum registered: CriuCgMode

goroutine 1 [running]:
github.com/kubernetes-sigs/cri-o/vendor/github.com/golang/protobuf/proto.RegisterEnum(...)
        /home/valentin/go/src/github.com/kubernetes-sigs/cri-o/vendor/github.com/golang/protobuf/
proto/properties.go:458
github.com/kubernetes-sigs/cri-o/vendor/github.com/opencontainers/runc/libcontainer/criurpc.init.
0()
        /home/valentin/go/src/github.com/kubernetes-sigs/cri-o/vendor/github.com/opencontainers/r
unc/libcontainer/criurpc/criurpc.pb.go:1058 +0x549

so my gsoc proposal mentions two approaches
A - to create a similar script in go
B - acess the .h file directly.

I think A would simply be easier and less messy.