Code Monkey home page Code Monkey logo

Comments (12)

aoxn avatar aoxn commented on June 1, 2024 1

golang lib use epoll to implement synchronized Read and Write. This requires a non-block socket. see golang implementation for more details, https://github.com/golang/go/blob/4f04e1d99fac7abf067b6bd3a299f1fbc9a59414/src/net/sock_cloexec.go#L45

from vz.

Code-Hex avatar Code-Hex commented on June 1, 2024

Guest disk information

$ df -h
Filesystem      Size  Used Avail Use% Mounted on
udev            952M     0  952M   0% /dev
tmpfs           197M  740K  197M   1% /run
/dev/vda         11G  3.0G  8.1G  27% /
tmpfs           985M     0  985M   0% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           985M     0  985M   0% /sys/fs/cgroup
/dev/loop1       58M   58M     0 100% /snap/core20/1171
/dev/loop3       49M   49M     0 100% /snap/core18/2127
/dev/loop0       49M   49M     0 100% /snap/core18/2252
/dev/loop2       58M   58M     0 100% /snap/core20/1274
/dev/loop5       61M   61M     0 100% /snap/lxd/21544
/dev/loop4       38M   38M     0 100% /snap/snapd/14296
/dev/loop6       29M   29M     0 100% /snap/snapd/13643
/dev/loop7       61M   61M     0 100% /snap/lxd/21843
tmpfs           197M     0  197M   0% /run/user/0

from vz.

Code-Hex avatar Code-Hex commented on June 1, 2024

current my opinion
#12 (comment)

I think this is caused by the thread between libdispatch (which is handled by Objective-C) and Go, or a problem on the VM side.

from vz.

cfergeau avatar cfergeau commented on June 1, 2024

Would be worth building/running with go build -race ... to see if the go runtime has anything to report. With go routines and macOS dispatch queues involved, it's going to be more or less required to protect these from concurrent access:

socket.go:var connectionHandlers = map[string]func(conn *VirtioSocketConnection, err error){}
socket.go:var shouldAcceptNewConnectionHandlers = map[unsafe.Pointer]func(conn *VirtioSocketConnection) bool{}

Had done something like this in an experimental branch which is now obsolete :)
cfergeau@1aa2c8c

from vz.

aoxn avatar aoxn commented on June 1, 2024

I am also not able to close the VirtioSocketConnection, close return nil, but io.Copy does not return.

func NewProxyVSOCK(p *Publish, vso *vz.VirtioSocketDevice, port uint32) func(conn net.Conn) {
	return func(conn net.Conn) {
		klog.Infof("[MAC_OS]prepare to connect to vm port: %d", port)
		onConnect := func(vzconn *vz.VirtioSocketConnection, err error) {
			defer Close(conn, "MAC_OS: unix  --> vsock")
			klog.Infof("[MAC_OS][%s]remote connection to vm established: %s",p,vzconn)
			//defer Close(vzconn, "MAC_OS: vsock --> unix")
		        	
			//stop := make(chan struct{}, 1)
			grp := &sync.WaitGroup{}
			grp.Add(2)
			//go copyStream(grp,vzconn, conn, "MAC_OS: unix  --> vsock",false)

			go func() {
				msg := "MAC_OS: unix  --> vsock"
				defer Close(conn, msg)
				n, err := io.Copy(vzconn, conn)
				if err != nil {
					klog.Errorf("[%s]copy to vm: %d bytes copied, %s",msg, n, err.Error())
				}
				klog.Infof("local conn done")
				grp.Done()
				Close(vzconn,"")
			}()
			//go copyStream(nil,conn, vzconn, "MAC_OS: vsock  --> unix",false)
			go func() {
				msg := "MAC_OS: vsock  --> unix"
				n, err := io.Copy(conn, vzconn)
				if err != nil {
					klog.Errorf("[%s]copy to vm: %d bytes copied, %s",msg, n, err.Error())
				}
				klog.Infof("vzconn: done")
				grp.Done()
			}()
			klog.Infof("[MAC_OS][%s]unix --> vsock started, wait done: ", p)
			grp.Wait()
			klog.Infof("[MAC_OS][%s]receive goroutine sig 1",p)
			//<-closer
		}
		if vso == nil {
			klog.Errorf("[MAC_OS]Attention, nil vso:  [%v]", vso)
		}
		klog.Infof("[MAC_OS] do connect to port")
		vso.ConnectToPort(port, onConnect)

		klog.Infof("[MAC_OS][%s]connection complete: %s",p, conn.RemoteAddr())
	}
}

here is the output


podman login: I0113 21:34:45.277146   25120 publish.go:112] new connection:
I0113 21:34:45.277207   25120 proxy_darwin.go:16] [MAC_OS]prepare to connect to vm port: 2375
I0113 21:34:45.277216   25120 proxy_darwin.go:56] [MAC_OS] do connect to port
I0113 21:34:45.278198   25120 proxy_darwin.go:59] [MAC_OS][unix@/tmp/aoxn.sock:vsock@2375]connection complete:
I0113 21:34:45.279323   25120 proxy_darwin.go:19] [MAC_OS][unix@/tmp/aoxn.sock:vsock@2375]remote connection to vm established: &{c7g2ktdmk1u64864pt80 %!s(uint32=3283772544) %!s(uint32=2375) %!s(uintptr=10) %!s(*os.File=&{0xc0004ce120}) %!s(*vz.Addr=&{2 2375}) %!s(*vz.Addr=&{0 3283772544})}
I0113 21:34:45.279654   25120 proxy_darwin.go:48] [MAC_OS][unix@/tmp/aoxn.sock:vsock@2375]unix --> vsock started, wait done:
I0113 21:34:49.783454   25120 proxy_darwin.go:34] local conn done
I0113 21:34:49.783544   25120 publish.go:185] []close connection
I0113 21:34:49.783559   25120 publish.go:185] [MAC_OS: unix  --> vsock]close connection

from vz.

aoxn avatar aoxn commented on June 1, 2024

vzconn.Read() does not return after vzconn.Close().

the sequence to reproduce.

       vzconn := Connect2Port()
        grp := sync.WaitGroup{}
	grp.Add(1)
	go func() {
		bb := make([]byte,1024)
		n, err := vzconn.Read(bb)
		if err != nil {
			t.Logf("read error: %s", err.Error())
		}
		t.Logf("read %d byte: xxx", n)
		grp.Done()
	}()
	t.Logf("sleep 5s")
	time.Sleep(5*time.Second)

	vzconn.Close()

	t.Logf("sleep 3")
	time.Sleep(3*time.Second)
	t.Logf("wait")
	grp.Wait()
	t.Logf("finishe")

from vz.

aoxn avatar aoxn commented on June 1, 2024

need to setNoneBlock mode for the descriptor.

func newVirtioSocketConnection(ptr unsafe.Pointer) *VirtioSocketConnection {
	id := xid.New().String()
	vzVirtioSocketConnection := C.convertVZVirtioSocketConnection2Flat(ptr)
	err := unix.SetNonblock(int(vzVirtioSocketConnection.fileDescriptor), true)
	if err != nil {
		panic(fmt.Sprintf("set nonblock: %s", err.Error()))
	}
	conn := &VirtioSocketConnection{
		id:              id,
		sourcePort:      (uint32)(vzVirtioSocketConnection.sourcePort),
		destinationPort: (uint32)(vzVirtioSocketConnection.destinationPort),
		fileDescriptor:  (uintptr)(vzVirtioSocketConnection.fileDescriptor),
		file:            os.NewFile((uintptr)(vzVirtioSocketConnection.fileDescriptor), id),
		laddr: &Addr{
			CID:  unix.VMADDR_CID_HOST,
			Port: (uint32)(vzVirtioSocketConnection.destinationPort),
		},
		raddr: &Addr{
			CID:  unix.VMADDR_CID_HYPERVISOR,
			Port: (uint32)(vzVirtioSocketConnection.sourcePort),
		},
	}
	return conn
}

from vz.

aoxn avatar aoxn commented on June 1, 2024

fixed.

from vz.

Code-Hex avatar Code-Hex commented on June 1, 2024

I don't know why need to setNoneBlock mode for the descriptor?
could you tell me if possible. thanks
relates: #23

from vz.

Code-Hex avatar Code-Hex commented on June 1, 2024

#23 does not seem to be fixed #13 (comment).
I don't think non blocking is related

from vz.

aoxn avatar aoxn commented on June 1, 2024

@Code-Hex Sorry for the miss-leading, #23 does not intended to solve #13, it fix the problem of copyStream blocks on reading forever.

from vz.

aoxn avatar aoxn commented on June 1, 2024

issue #24 might related with the crash scenario. remove code runtime.SetFinalizer(socketDevice, func(self *VirtioSocketDevice) { self.Release() }) seems fixed.

from vz.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.