11. October 2022

[dimakuv]

Agenda

(please write your proposed agenda items in comments under this discussion)

• Ying discussing UNIX Domain Sockets (UDS) between independent Gramine instances.

UDS between independent Gramine instances

Ying presented a tiny PoC-level patch to Gramine that proves that it is possible to have an encrypted UDS between two independent Gramine instances. There are two changes:

1. Removing instance_id in the host-level UDS name in the get_gramine_unix_socket_addr() function: https://github.com/gramineproject/gramine/blob/master/pal/src/host/linux-common/gramine_unix_socket_addr.c.

   • Currently Ying uses a trick of strlen(name) > 64 to distinguish between app pipes and internal Gramine pipes/UDSes and app UDSes. The former have randomly-generated rather-long names, the latter typically have names that are less than 64 chars. This is a hack, and will not be admitted as a PR.
   • Borys reminded the architectural concepts in Gramine: PAL pipes actually create UDSes on the Linux host, and LibOS uses PAL pipes to create several objects: application pipes, internal Gramine pipes, internal Gramine UDSes, and application UDSes. In Ying's work, she is only interested in making the last kind of objects (application UDSes) shared; the rest objects must stay private to one Gramine instance.
   • Borys and Dmitrii have two equally reasonable proposals:
     • Add a new boolean function argument to all functions (e.g. bool is_shared_pipe) in LibOS and PAL. This boolean will be false for application pipes, internal Gramine pipes, internal Gramine UDSes. And it will be true only for application UDSes.
     • Expose instance_id to the LibOS component and move the creation of the host-socket name to the LibOS (as a concatenation of instance_id/random_name for application pipes, internal Gramine pipes, internal Gramine UDSes and as simply uds_name for application UDSes).
     • Borys or Dmitrii can implement one of the above proposals quickly. So Ying can consider this part solved and should concentrate on other parts of her patch.

2. Using a sealing SGX key based on MRSIGNER to set up g_master_key here: https://github.com/gramineproject/gramine/blob/5fd2145632d81ff3d4af40899eb8f7fb2135e663/pal/src/host/linux-sgx/pal_main.c#L655?plain=1. This forces creation of the same pipes-encryption key for all Gramine instances with the same MRSIGNER, and so two independent Gramine instances are able to talk to each other.

   • Michal reiterated the problem of two instances of the same application (e.g. Redis) starting, and then our app will connect to any of them, and there is no way to distuingish with each of the two Redis instances our app communicates, which may lead to some attack scenarios.
   • Vijay mentioned that this tiny patch is a PoC. The problem above should be addressed by some local-SGX-attestation scheme that Ying and Vijay will develop.

3. Gramine (the app) needs to talk to Occlum (remote server). Such deployment is part of the Enclave-CC project, and one more thing to implement/finalize.

   • This requires standardization of the RA-TLS protocol between different SGX runtimes. The work on this is happenning.
   • Dmitrii should ask corresponding leaders from different SGX runtimes:
     • what is the current status?
     • should the implementation efforts start, based on the currently agreed spec?

[ Rather independent thought from Borys: Two Gramine instances have different FS views. So the same in-Gramine path of a UDS in one Gramine instance (on which the first enclave listens) may not be the same on the host as the same in-Gramine path in another Gramine instance (at which the second enclave connects). Generally, this problem concerns any files in independent Gramine instances (with their own manifest files). Even worse, this problem concerns the same Gramine instance, if the FS mount points start to alias. ]