[RFC] Development plan about enclave-cc HOT 29 CLOSED

I added "Components" and "Follow-up Story".

from enclave-cc.

@jiazhang0 thanks!
I'm thinking we could also start moving some of these tasks to "Projects" soon to have better tracking for status.

In IC project, I use "projects" to maintain components. So do you also mean this?

Yes, that was my thinking but instead of doing it for components, we could track milestones.

I created the first milestone 2022-04 as example. Let me know whether it makes sense to you.

from enclave-cc.

Update： Currently we have completed the first stage of POC that we could use k8s to deploy user app hello-world successfully.
We test the startup time of POD from creating to ready is about 11 seconds, and will optimize it in the future to be less than 10s.

from enclave-cc.

shim-rune launch enclave-agent container：
time="2022-06-06T19:01:51.255097194+08:00" level=debug msg="createAgentContainer() end" end time="2022-06-06 19:01:51" name=containerd-shim-v2 source=containerd-rune-shim-v2 start time="2022-06-06 19:01:49" total time cost=1.45895407s

shim-rune send request to pull image:
time="2022-06-06T19:01:55.848000611+08:00" level=debug msg="PullImage() end" end time="2022-06-06 19:01:55" name=containerd-shim-v2 source=containerd-rune-shim-v2 start time="2022-06-06 19:01:51" total time cost=4.56953493s

shim-rune launch app-container：
time="2022-06-06T19:01:56.611420379+08:00" level=debug msg="create App Container() end" end time="2022-06-06 19:01:56" name=containerd-shim-v2 source=containerd-rune-shim-v2 start time="2022-06-06 19:01:55" total time cost=753.127063ms

from enclave-cc.

@jiazhang0, can you add following name to the task list for components and Milestone 1?

• rune and PAL API v4 @YangLiang3, @gordon
• Gramine (including PAL adaption), @gordon, @ying

Milestone 1

• designs the PAL API v4 changes. @YangLiang3
• Gramine PAL API adaption, @gordon, @ying

from enclave-cc.

• Move "Gramine PAL API adaption" from the first milestone to the second one.
• It is confirmed that ying2liu is the dev and contact from Gramine team.
• It is confirmed that qzheng527 is the dev and contact from Occlum team.
• The first milestone will use Occlum to show the demo. Gramine adaption (decoupling design) needs to be discussed further.

from enclave-cc.

@jiazhang0 is this issue still relevant or can be closed?

@jiazhang0 I think most of the items originally covered by this plan are now complete. Maybe we can close this and track any remaining items in their own issues?

from enclave-cc.

"enclave-cc v0 PoC"

Just thinking out loud but I wonder if it would make sense to get this work tied to the master project "CCvX" milestones somehow rather than using it's own versioning which is behind the master project. After all there are many components we share.

import shim-rune and rune components (without commit history) from IC project directly.

Is there some functionality in these we don't need for enclave-CC? IIRC you mentioned the carrier framework stuff isn't necessary. I was wondering would it make sense to clean-up things a bit first?

the LibOS in app enclave can mount the OCI bundle from unprotected storage as app container rootfs.
Gramine PAL API adaptation.

We should probably have a task that designs the PAL API changes. Also, this suggests that the milestone criteria is set so that both Occlum and Gramine are supported and validated for each feature. Should that target be made more explicit or we pick only either one for the POCs..

For the "v0" milestone I'd also suggest: all design opens closed.

For a later milestone we need a task to implement that API limiting.

from enclave-cc.

Just thinking out loud but I wonder if it would make sense to get this work tied to the master project "CCvX" milestones somehow rather than using it's own versioning which is behind the master project. After all there are many components we share.

I try to rename the milestone names but not sure if they are appropriate.

Is there some functionality in these we don't need for enclave-CC? IIRC you mentioned the carrier framework stuff isn't necessary. I was wondering would it make sense to clean-up things a bit first?
We should probably have a task that designs the PAL API changes.
For the "v0" milestone I'd also suggest: all design opens closed.
For a later milestone we need confidential-containers/documentation#21 (comment).

Done.

from enclave-cc.

@jiazhang0 thanks!

I'm thinking we could also start moving some of these tasks to "Projects" soon to have better tracking for status.

from enclave-cc.

@jiazhang0 thanks!

I'm thinking we could also start moving some of these tasks to "Projects" soon to have better tracking for status.

In IC project, I use "projects" to maintain components. So do you also mean this?

from enclave-cc.

@jiazhang0 thanks!
I'm thinking we could also start moving some of these tasks to "Projects" soon to have better tracking for status.

In IC project, I use "projects" to maintain components. So do you also mean this?

Yes, that was my thinking but instead of doing it for components, we could track milestones.

from enclave-cc.

I created the first milestone 2022-04 as example. Let me know whether it makes sense to you.

@jiazhang0 looks good!

from enclave-cc.

@bigdata-memory Update the detail info for the task "Gramine PAL API adaption".

from enclave-cc.

Add "enclave-agent as a single process needs to integrate attestation-agent as internal module/service. " to the second milestone.

from enclave-cc.

The containerd 1.7 plans, including the new APIs, were discussed in last week's community call. Since we are still in the very early stages of the implementation, I think we should (re-)think which approach is best for enclave-cc. I'm planning to take a closer look how the 1.7 design for enclave-cc would look. Thoughts on this topic?

from enclave-cc.

@mythi Thank you for raising up this. The containerd changes would happen and affect kata-cc as well so eventually we will switch to new approach of containerd if kata-cc decides to follow up it. I guess offloading image pulling to TEE is still a basic principle of CC community so the possible impact is shim api change, which is controllable for enclave-cc.

from enclave-cc.

Currently, we have a good progress on agent enclave and app enclave based on occlum to unpack and mount container image with image-rs, but it is still insufficient to construct the enclave-cc to work out the complete flow.

In next stage, shim-rune needs to complete the interaction with agent enclave during pod creation. In addition, we need to define the last piece of PAL API v4 about the flow of the creation of LA channel. After the above approaches applied, we will get the initial PoC in May, and try to accomplish Gramine PAL API adaption in June.

So the plan needs to move forward as a whole for one month.

from enclave-cc.

(Occlum as basic Libos)
What Have Done:
According to the architecture’s steps

1. Completed create and exec agent enclave container based on rune and Occlum(Step 1)
2. Completed pulling the common image(Step 3)
3. Completed unpacking image to bundle(Part of Step 5)
4. Completed mount rootfs and exec User Application(Hardcode fuse key)(Step 8)
5. Rune PAL API V4 has been submitted to CC community

What Have Left:

1. Ttrpc communication between shim-rune and image management (Intel & Alibaba Co-work) (Step 2)
2. Setup RA and get the decryption key between Verdictd(KBS) and Attestation Agent (Alibaba) (Step 4)
3. After getting the decryption key, decrypt the image(Intel) (Part of Step 5)
4. Get fuse key by LA between agent enclave and App enclave(Intel & Alibaba) (Step 7)
5. Rune attest command in Rune PAL API V4 is still under discussion.

from enclave-cc.

Plan update:
Postpone the following 5 items one month, because it is insufficient to submit the PR on time. Most of efforts are occupied to implement the initial PoC. The design doc of shim-rune would be submitted in the end of this month.

• import shim-rune and rune components with a clean base from IC project. (May to June)
• all design opens closed. (May to June)
• Occlum PAL API adaption. (May to June)
• implement that API limiting. (June to July)
• add CC operator support to deploy enclave-cc. (June to July)

from enclave-cc.

Update: shim-rune already supports to launch agent-enclave during Pod creation.

from enclave-cc.

Update: shim-rune already supports to launch agent-enclave during Pod creation.

@jiazhang0 is this using the pre-created bundle format?

from enclave-cc.

Update: shim-rune already supports to launch agent-enclave during Pod creation.

@jiazhang0 is this using the pre-created bundle format?

Yes. A pre-created oci bundle for agent-container/enclave is needed instead of the form of container image, because agent-container/enclave is the first component along with pod creation and there is no direct support for it to pull and unpack it in a TEE.

from enclave-cc.

@zhiwei-intel-h - Would it be possible to break down that 11 seconds to show where that time is being spent? Presumably most of the time is spent pulling the image, but it would be interesting to understand how the remaining time is divided between the remaining steps (if possible - do the current logs provide sufficient detail I wonder?).

from enclave-cc.

@zhiwei-intel-h - Would it be possible to break down that 11 seconds to show where that time is being spent?

Also, what's the "baseline" value without enclave-cc in the same environment.

from enclave-cc.

Update： Currently we have completed the first stage of POC

Let's document the key takeaways from the first milestone: were there unexpected issues that require proper fixing in subsequent milestones. any changes in the design or milestone plans based the learnings?

from enclave-cc.

Milestone updates:

• Move the initial PoC demo in cc meeting to June, because the boot time is still long a bit.
• Keep the original goal of June, supporting image protections and remote attestation.

from enclave-cc.

@mythi @jodh-intel we are preparing the time statistics of launch pod and making some optimizations. Due to the China Dragon Boat Festival, the data will be posted after the holiday. Thanks

from enclave-cc.

@jiazhang0 is this issue still relevant or can be closed?
If it's still relevant to what release do you think we should map it to (mid-November, end-December, mid-February etc...)?

from enclave-cc.