Eliminating NaCl FD's about lind_project HOT 9 CLOSED

Hammered out a bunch of this over the last 24 hours. Got a proof of concept to compile.

I'm now like 95% sure this is going to work. Will need to do a few days of debugging to make sure everything is interfacing correctly and need to solve a small implementation problem dealing with vmmap protection checks.

from lind_project.

At this point I have everything running and all tests passing. Will have a PR up in the next day or two once I triple check everything.

from lind_project.

This issue is finally revived. This looks like the cleanest way to get rid of the open() overhead.

NaCl descs are also used in the lind_syscall.c interface which seems to be handling most netcalls, so I'll have to rip it out of there as well.

from lind_project.

My initial attempt here to just rush in and rip out the NaClDesc system hit some roadblocks. It seems the Desc's run up against the validator and the Vmmap, though I'm still convinced this is possible. Mmap in particular is messy, which we knew.

i'm going to brainstorm a bit and plan this out in writing before I start re-arranging the code. I think there's a chance to make this much more elegant and easy to pass through, and that's probably worth it at this point.

from lind_project.

Mmap seems to have 3 pathways that eventually lead to a map 1) Map anon 2) Prot-exec & map-fixed 3) everything else.

From tracing, scenario 2 is never actually allowed to completed (get to the actual mmap call) because protections that are set up. This is great news because that's the annoying scenario that interacts with the validator/accesses this metadata field.

from lind_project.

NaClVmmapEntryMaxProt assigns protections in the vmmap based on desc's that are allocated to them. I'm not sure if this is necessary for us or can be moved to SafePOSIX.

from lind_project.

So this became overly complicated, to the point where it felt like I was re-writing a majority of native client and the file dependencies became a total nightmare. If this is going to happen it needs to be part of a system wide overhaul.

The great news here is by combing over this stuff in detail, I was able to figure out how to eliminate the excess system calls (fstat, fstat) from open and mmap, which was the original goal to reduce total system call time. So though this long term goal is still open, the short term goal was achieved.

from lind_project.

Well, I've found myself back here again on this > 3 year old issue. This time possibly with a solution.

While reviewing #235 I found out a bug thats existed since the beginning of Lind, where the close syscall in the library OS is only called when all refs to a NaCl desc are removed. This is a bug because refs are added when a desc is mapped, causing close not to be called even if the application calls the close syscall.

Looking at this made me review my last attempt at removing NaCl Descs. I realized 2 things. One, I was trying to completely remove NaCl Descs from NaCl as opposed to just removing them from the syscall pathways. Two, a ton of headaches were being caused by trying to fix Desc references being included in the vmmap entries.

As far as I can tell, the only reason NaCl coupled descs and map entries together is for some windows edge cases.

Anyways, I'm like 80% sure this can be done at this point.

Trying to do this again now would:

• remove the last performance cost for I/O that I think is possible to fix (this is ~10-15% of the per syscall cost in the small buffer pipe scenarios)
• fix an issue we've been trying to address for a long time and greatly simplify things
• fix the vmmap dtor issue

I think this would take me ~ a week to figure out

from lind_project.

Finally closed with latest PR. Hallelujah!

from lind_project.