Design topic collections about enclave-cc HOT 31 CLOSED

Add a new item "Local Attestation protocol for FUSE key transmission" assigned to @bigdata-memory

from enclave-cc.

Add a new item "Local Attestation protocol for FUSE key transmission" assigned to @bigdata-memory

Do you think a separate key is needed? I was wondering would a seal key with the mrsigner policy be sufficient.

from enclave-cc.

Add a new item "Local Attestation protocol for FUSE key transmission" assigned to @bigdata-memory

Do you think a separate key is needed? I was wondering would a seal key with the mrsigner policy be sufficient.

The so-called Local Attestation protocol include several request/response interactions to exchange parameters bound to local report for authentication against the possible MITM attack. For example, inspiring ECDH used by sgx sdk LA sample would be a choice. Of course, sealing key is another option. However, the implementation of negotiation process for sealing key is complex than ECDH and it is not hardware-agnostic. This is the details of EGETKEY(key_id="SEAL_KEY") according to SDM vol 3:

To ensure the generated sealing key by agent enclave is generatible for peer app-enclaves which possibly host different LibOS, there are several constraints to be satisfied at least.

• unified ISVSVN and CONFIGSVN versioning scheme.
• same minimal-and-allowed-attributions

If EGETKEY() for FUSE key generation only occurs in agent-enclave side and then transmitted to app-enclave, the benefits of EGETKEY() are degenerate. We would rather use ECDH directly for easy life.

Please correct me if I have something wrong. @bigdata-memory

from enclave-cc.

From what I understand, the seal key of mrsigner was designed to be tied to the code which the ISV signed on.
However, in our case, there is no guarantee that the code of workloads encrypted by FUSE key which comes from the same signer who signed the agent enclave.
Moreover, with that seal key, we ensure that the data can be decrypted by the code signed by the same signer, which means that it is only used for secure data saving and the data could be anything including untrusted code. So if we use it for decrypting workload which will be running within the same signer's enclave, it would not serve the same purpose as the seal key.

from enclave-cc.

in the enclave-cc design the agent enclave and app enclave are signed by the same signer and that made me think the possibility to use sealing keys. In this context, the "data" is the content from the encrypted container layers. The LA based approach is fine too.

from enclave-cc.

in the enclave-cc design the agent enclave and app enclave are signed by the same signer and that made me think the possibility to use sealing keys. In this context, the "data" is the content from the encrypted container layers. The LA based approach is fine too.

Yes. Both enclaves are signed by the same signer in enclave-cc. And if both enclaves use the same version (ISVSVN) of LibOS, using sealing key is appropriate. I suggest we leave the space in the code implementation in such a way: if ttrpc service for LA channel is unavailable or unsupported, or the parameter passed to app enclave enforces using sealing key scheme, app-enclave can generate a sealing key with mrsigner and isvsvn as policy. Does it make sense?

from enclave-cc.

Looking at how the LA design is unfolding I'm thinking sealing may be much simpler.

Agent and App enclaves both have the same MRSIGNER.
The agent generates the FUSE key.
The agent seals the fuse key at a known location using MRSIGNER policy.
The app enclave is started and reads the known location.
The app enclave unseals the key using the MRSIGNER policy.

No need to implement the sigma protocol in the LA example, and wrap that with any signaling between the two containers.
We may want a more elaborate policy than MRSIGNER, but this would also be the case with LA.
cc @bigdata-memory

from enclave-cc.

@dcmiddle In essence, the goal is to securely and efficiently deliver the filesystem encryption key to app. enclave from agent enclave, both of them running in the same Pod. The way which leverages MRsigner to seal and deliver the FSenc key is possible and looks much simpler to implement.
However, there are some implications which we may need to take into account, for example:

• The location or file path needs to get pre-configured on both sides which may collide with existing file
• Any enclave instance with the same MRsigner can reveal this FSenc key unconditionally no matter when and how
• The sealing operation happens before the fetching operation, so the app. enclave instance has no chance to provide specific credentials to agent enclave for further authentication or customization.

from enclave-cc.

Yes, these are important implications for us to consider @bigdata-memory

• The location or file path needs to get pre-configured on both sides which may collide with existing file

Writing the sealed key to a path relative to the encrypted bundle (currently based on container ID) seems to be the right gist, but I agree this would need to be fully thought through and specified. Seems like a tractable problem though.

• Any enclave instance with the same MRsigner can reveal this FSenc key unconditionally no matter when and how

I also agree this is a limitation, however local attestation has similar limitations unless we open the door to a sophisticated MRENCLAVE policy wherein the container decryption key is conjoined with an MRENCLAVE policy.
I believe an MRSIGNER policy is also no less secure than the VM side of the CoCo architecture where attestation policies of the guest are no more restrictive.

A slightly better policy would be to additionally use ISVEXTPRODID or ISVFAMILYID, such that Agent and App Enclaves have those same values but no other enclave produced with MRSIGNER has those ISVEXTPRODID or ISVFAMILYID values.

• The sealing operation happens before the fetching operation, so the app. enclave instance has no chance to provide specific credentials to Agent Enclave for further authentication or customization.

I think this is a restatement of the same problem. The only way to authenticate the app enclave is with its MRENCLAVE which implies that the Agent knows how to discriminate trusted vs untrusted MRENCLAVEs.

from enclave-cc.

@dcmiddle Yes, I think that we need to consider the ISVEXTPRODID or ISVFAMILYID as well for finer grained security protection. However, it is static approach and the decision is made at building/deployment time, actually it is similar to what report key does in LA. For stronger security protection, we may further take account of some runtime factors, such as freshness, workload identity/provenance, location and some other security attributes. With those information, the Agent enclave can make better decision based on runtime security context of enclave instance.

from enclave-cc.

Let's proceed with Sealing and move Local Attestation to future exploration.

Based on this discussion and observation that Occlum and Gramine both support sealing as well as a similar use case described in the SGX Developer reference[1] I believe the first version of enclave-cc should use sealing for secret exchange from the Agent Enclave to the App Enclave.

[1] See figure 3 and surrounding text in the SGX Developer Reference

from enclave-cc.

@dcmiddle @bigdata-memory As I wrote, sealing key is an another option to implement the transmission of FUSE key.

After I read the new discussions, I recognize that:

1. Sealing key is light-weight than what I expected, because it doesn't have a negotiation process. More specifically, I originally thought sealing key scheme will also negotiate the parameters affecting the generation of sealing key as LA protocol. That is fine.

2. If enclave-cc assumes that the peer enclaves must be same, using MRENCLAVE policy (optionally with other policies) is possible and is more secure than pure MRSIGNER policy.

3. How about directly using sealing key as fuse key instead of using sealing key as session key to wrap the fuse key? It looks much simpler without the concerns about the store location.

4. Let LA protocol go on its development and merge the design doc with exploratory heading. It is still a good option with the benefit of TEE HW-agnostic, but complex than simple sealing key scheme. Additionally, it is a common approach to establish an attested channel between TEEs (not limit to sgx enclave I think) and it should be implemented an option to serve FUSE key transmission. Therefore, the definitions of generate_fuse_key() and get_fuse_key() interfaces should be modularized.

from enclave-cc.

2. If enclave-cc assumes that the peer enclaves must be same, using MRENCLAVE policy (optionally with other policies) is possible and is more secure than pure MRSIGNER policy.

This is an interesting idea. Are you suggesting building the Agent and App as one binary (and then launching separate instances, perhaps using a flag)?

from enclave-cc.

If enclave-cc assumes that the peer enclaves must be same, using MRENCLAVE policy (optionally with other policies) is
possible and is more secure than pure MRSIGNER policy.

This is an interesting idea. Are you suggesting building the Agent and App as one binary (and then launching separate instances, perhaps using a flag)?

Can you please clarify this?

How about directly using sealing key as fuse key instead of using sealing key as session key to wrap the fuse key? It looks much simpler without the concerns about the store location.

What will the sealing key be derived from?

from enclave-cc.

3. How about directly using sealing key as fuse key instead of using sealing key as session key to wrap the fuse key? It looks much simpler without the concerns about the store location.

Occlum does not support setting mrsigner key policy currently.

from enclave-cc.

If enclave-cc assumes that the peer enclaves must be same, using MRENCLAVE policy (optionally with other policies) is
possible and is more secure than pure MRSIGNER policy.

This is an interesting idea. Are you suggesting building the Agent and App as one binary (and then launching separate instances, perhaps using a flag)?

Can you please clarify this?

Let me clarify this.

Enclave-CC may be used in 2 sorts of deployment scenario. The first one is that the workload provider purchases the host machine, so he/she is responsible for deploying enclave-cc by self, including K8s, LibOS, rune, shim-rune and so on. In this case, the LibOS should be signed by self and thus MRSIGNER reflects his/her identity. This is why we are talking about MRSIGNER policy can be used in sealing key generation.

The second one is that the workload provider doesn't purchases the host machine, so he/she is responsible for deploying container image, without paying for the maintenance cost of host machine. In this case, K8s, LibOS, rune, shim-rune and so on are deployed by machine owner or CSP. This fact also implies the fact that the LibOS is not signed by the workload provider, and thus MRSIGNER cannot reflect his/her identity. This is why I'm talking about MRENCLAVE policy should be used in sealing key generation.

If MRENCLAVE policy is used, the LibOS used to run app-enclave and agent-enclave must be same.

Dan is trying to understand this idea and mentions this idea plans to build the Agent and App as one binary. More specifically, all artifacts involving the calculation of MRENCLAVE should exist in both app-enclave and agent-enclave. I don't think about it in deep.

How about directly using sealing key as fuse key instead of using sealing key as session key to wrap the fuse key? It looks much simpler without the concerns about the store location.

What will the sealing key be derived from?

See the picture in #3 (comment) for the reference. It shows the factors affecting the generation of sealing key.

No matter which policy is used to generate sealing key, I think the approach of using sealing key between app-enclave and agent-enclave can be simpler than the approach mentioned in #3 (comment)

In other word, the LibOS used to run app-enclave and agent-enclave can use the sample logic to generate sealing key with the consistent input factors. This approach can effectively avoid a complex procedure of fuse key protection.

from enclave-cc.

If MRENCLAVE policy is used, the LibOS used to run app-enclave and agent-enclave must be same.

Dan is trying to understand this idea and mentions this idea plans to build the Agent and App as one binary. More specifically, all artifacts involving the calculation of MRENCLAVE should exist in both app-enclave and agent-enclave. I don't think about it in deep.

To Occlum, it can support (not in current master branch yet) same MRENCLAVE for both Agent and APP enclaves if they are built with the same version libos, even the application on top of the libos are differenent. The application is mounted during the boot process which is not involving the calculation of MRENCLAVE. That is to say, there is no need to build the Agent and App as one binary for the same MRENCLAVE.

from enclave-cc.

3. How about directly using sealing key as fuse key instead of using sealing key as session key to wrap the fuse key? It looks much simpler without the concerns about the store location.

Occlum does not support setting mrsigner key policy currently.

The default sealing key in Occlum is derived from the mrsigner.
But in current Occlum based Agent enclave, the fuse key is provided by the app, has nothing to do with the sealing key.

from enclave-cc.

3. How about directly using sealing key as fuse key instead of using sealing key as session key to wrap the fuse key? It looks much simpler without the concerns about the store location.

Occlum does not support setting mrsigner key policy currently.

The default sealing key in Occlum is derived from the mrsigner. But in current Occlum based Agent enclave, the fuse key is provided by the app, has nothing to do with the sealing key.

"provided by the app" is ambiguous here. it is specified (hard coded key) in the agent enclave code. if what you say about the default sealing key is true, then leaving key= parameter out should just work.

from enclave-cc.

Let me clarify this

There are 2 cases,
Case 1: The workload provider is also the infrastructure owner and signs the workload. So, MRsigner based solution is proposed for this case.
Case 2: The Workload provider depends on the CSP to run the workload. And it looks like the proposal is to use MRenclave.

For case 1, if everything is owned and operated by the workload provider, then why even use MRsigner, can he/she measure their own app-enclave and agent-enclave. Also, in this case is attestation really needed? Seems like all the pieces of software/hardware are controlled by the workload provider.

For case 2, based on the below,

To Occlum, it can support (not in current master branch yet) same MRENCLAVE for both Agent and APP enclaves if they are built with the same version libos, even the application on top of the libos are differenent.

IIUC, this means only measurement is done for the LibOS and both enclave-agent and app are out of the picture. I think this doesn't add much value and can cause a security risk if the app is malicious or the agent-enclave is compromised.

What will the sealing key be derived from?

See the picture in #3 (comment) for the reference. It shows the factors affecting the generation of sealing key.

This shows all the key policies from the SDM right? My question was specific to what policies are planned to be used for case 1.

from enclave-cc.

But in current Occlum based Agent enclave, the fuse key is provided by the app, has nothing to do with the sealing key.

My understanding is that we plan to use the sealing key to wrap the fuse key and pass it to the app-enclave as the communication channel is not secure.

from enclave-cc.

For case 1, if everything is owned and operated by the workload provider, then why even use MRsigner, can he/she measure their own app-enclave and agent-enclave. Also, in this case is attestation really needed? Seems like all the pieces of software/hardware are controlled by the workload provider.

In this case, the local attestation is not needed. Instead, using sealing key as wrapping key to protect FUSE key is appropriate. In addition, the decryption key for app-container image needs to be remotely provisioned by relying party through remote attestation (see the step 2 and 3 described in https://github.com/confidential-containers/enclave-cc/blob/main/docs/design.md#architecture ). This happens before agent-enclave starts to launch app-container image.

See the picture in #3 (comment) for the reference. It shows the factors affecting the generation of sealing key.

This shows all the key policies from the SDM right? My question was specific to what policies are planned to be used for case 1.

Yes. I draw it according to SDM. In case 1, sealing key approach uses MRSIGNER policy in the input parameter of Keyrequest. Maybe you are talking other "policy" rather than the MRSIGNER policy field in Keyrequest?

from enclave-cc.

But in current Occlum based Agent enclave, the fuse key is provided by the app, has nothing to do with the sealing key.

My understanding is that we plan to use the sealing key to wrap the fuse key and pass it to the app-enclave as the communication channel is not secure.

Yes it is.

In addition, I talked using MENCLAVE policy for sealing key generation is about case 2. Sorry for the confusion I made.

I recognized we don't need to seek one unified approach to cover both cases. Your understanding can cover case 1 perfectly, but we need a solution for case 2. I have already thought of it in deep. Tomorrow I will describe it.

from enclave-cc.

To Occlum, it can support (not in current master branch yet) same MRENCLAVE for both Agent and APP enclaves if they are built with the same version libos, even the application on top of the libos are differenent.

IIUC, this means only measurement is done for the LibOS and both enclave-agent and app are out of the picture. I think this doesn't add much value and can cause a security risk if the app is malicious or the agent-enclave is compromised.

I think @qzheng527 is talking about KSS. With KSS support, the manifest or configuration file will be not reflected in MRENCLAVE. Instead, the digest of manifest or configuration file is represent by SECS.CONFIGID. KSS can decouple the configurable configuration info and enclave binary, allowing the digest of manifest or configuration file not be reflected in MRENCLAVE. In occlum, the digest of app is recorded in manifest, and thus manifest digest doesn't affect MRENCLAVE if KSS is used. I think this is the background info inside @qzheng527 's comments.

from enclave-cc.

I recognized we don't need to seek one unified approach to cover both cases. Your understanding can cover case 1 perfectly, but we need a solution for case 2. I have already thought of it in deep. Tomorrow I will describe it.

What's in case 2 that it won't work like in case 1? I could not find the answer from the thread above.

from enclave-cc.

I recognized we don't need to seek one unified approach to cover both cases. Your understanding can cover case 1 perfectly, but we need a solution for case 2. I have already thought of it in deep. Tomorrow I will describe it.

What's in case 2 that it won't work like in case 1? I could not find the answer from the thread above.

In case 2, the tenant or workload provider doesn't have access to the host or platform. CSP controls it and schedules the workloads from different tenants or workload providers to maximize the utility of platform resources. Case 2 is the typical container usage in the practical cloud PaaS production. The tenant or workload provider can only purchase a Pod (See Azure ACI https://azure.microsoft.com/en-us/services/container-instances/). More specifically, the K8s nodes and container runtime are invisible to the tenant or workload provider.

Therefore, how do you deploy enclave-cc and signed LibOS by self in case 2? There is no delivery / deployment interface available to tenant or workload provider to set up them at all.

In fact, we would recognize the case 1 is not economical at all. The tenant or workload provider pays for platform and maintenance cost to just run confidential containers. Why not simply run sgx LibOS or TEE VM? The benefits of enclave-cc and kata-cc are to reduce the prime cost of platform payment and platform maintenance efforts. They are fit for case 2, and thus we cannot assume the tenant or workload provider can control the platform.

from enclave-cc.

In case 2, the tenant or workload provider doesn't have access to the host or platform. CSP controls it and schedules the workloads from different tenants or workload providers to maximize the utility of platform resources.

My question was why a sealing based approach would not be able to cover both cases and I cannot find the answer in #20 either.

I think in both cases the agent-enclave needs to provide the evidence also for the part that what libOS env (e.g., container entrypoint/cmd and env) is going to be used by the app enclave to ensure only the tenant allowed input is run.

from enclave-cc.

In case 2, the tenant or workload provider doesn't have access to the host or platform. CSP controls it and schedules the workloads from different tenants or workload providers to maximize the utility of platform resources.

My question was why a sealing based approach would not be able to cover both cases and I cannot find the answer in #20 either.

I think in both cases the agent-enclave needs to provide the evidence also for the part that what libOS env (e.g., container entrypoint/cmd and env) is going to be used by the app enclave to ensure only the tenant allowed input is run.

In case 2, sealing key with MRSIGNER policy is not useful because the LibOS signer is CSP. In this case, the production delivery and deployment interface doesn't allow tenant to deploy LibOS signed by self. This results in the LibOS binary is signed by CSP.

Therefore, the possible identity authentication method is the sealing key with MRENCLAVE policy. In practice, it is difficulty to create the same LibOS binary plus manifest (containing detailed security policies and configurations specific to LibOS). KSS can decouple the connection between MRENCLAVE and manifest, but LibOS binary itself is still hard to be same for hosting both app-enclave and agent-enclave. For example, Gramine has an unique design to support enclave-cc. The "Payload Receiver" as app is hashed into MRENCLAVE, and it is not used by the Gramine LibOS hosting agent-enclave. This means Gramine will use 2 different MRENCLAVEs to run enclave-cc.

from enclave-cc.

@jiazhang0 is this issue still relevant or can be closed?
If it's still relevant to what release do you think we should map it to (mid-November, end-December, mid-February etc...)?

from enclave-cc.

Any disagreement with closing this issue? The one remaining task has it's own issue now in #20.

from enclave-cc.

Any disagreement with closing this issue? The one remaining task has it's own issue now in #20.

+1 for closing

from enclave-cc.