User Tools

Site Tools


SGX Migration


The recent commercial availability of Intel SGX (Software Guard eXtensions) provides a hardware-enabled building block for secure execution of software modules in an untrusted cloud. As an untrusted hypervisor/OS has no access to an enclave’s running states, a VM (virtual machine) with enclaves running inside loses the capability of live migration, a key feature of VMs in the cloud. This paper presents the first study on the support for live migration of SGX-capable VMs.


  • Challenge-1: Enclave states are sealed in hardware. Traditionally, the hypervisor can access all the running states of a guest VM. On SGX platform, the hardware has many new states for each running enclave. However, neither the hypervisor nor the guest OS can access an enclave’s memory (EPC) directly. Moreover, some of the enclave states, e.g., the CSSA, cannot be accessed by any software (even the enclave itself).
  • Challenge-2: Neither the hypervisor nor the guest OS is trusted. The migration process requires that the enclave states are dumped out of the enclave where there is no hardware protection. A malicious hypervisor may steal or tamper with the states during the process. A malicious guest OS may violate the consistency of states of a multi-threaded enclave by controlling the scheduling.
  • Challenge-3: Cloning or rolling back an enclave could be malicious. Traditionally, a cloud operator is allowed to clone a guest VM to get multiple instances or rollback a guest VM to some previous checkpoint. Cloning or rolling back an enclave might be considered as a malicious behavior. On SGX platform, if enclave migration is enabled, a malicious cloud operator might issue fork attacks or rollback attacks.


We identify the security properties that a secure enclave migration process should meet and propose a software-based solution. We leverage several techniques such as two-phase checkpointing and self-destroy to implement our design on a real SGX machine. Security analysis confirms the security of our proposed design and performance evaluation shows that it incurs negligible performance overhead. Besides, we give suggestions on the future hardware design for supporting transparent enclave migration.


Briefly, the enclave migration process, as shown in the figure, includes the following three operations: first, the source machine dumps one enclave’s running states out. Second, the dumped states are transferred to the target machine through the network. Third, the target machine creates a new enclave and restores the running states to resume execution.

On the source machine, the control thread is responsible for generating a checkpoint that contains all the enclave’s memory and execution context, including the software-unreadable states maintained by hardware: the CSSA field in TCS. Specifically, we have designed a software mechanism running in the enclave that can track CSSA by monitoring all the entry and exit events of the enclave, without any dependency on the untrusted guest OS or hypervisor.

On the target machine, the restore process contains the following four steps.

  • Step-1: the target machine creates and initializes a virgin enclave using the same image of the migrated enclave.
  • Step-2: the source control thread will remotely attestate the newly created enclave. If the attestation succeeds, it will establish a secure channel with the target control thread to deliver the key encrypting the checkpoint.
  • Step-3: the target control thread will restore all the memory using the checkpoint, and utilize the SGX library (out-of-enclave) to restore CSSA. In the meantime, the target control thread will also track the CSSA using the same method mentioned above.
  • Step-4: before resuming execution, the target control thread will check whether the tracked CSSA is the same as the one in the checkpoint.


The evaluation results show that our system’s performance overhead is negligible: to migrate a VM with 64 enclaves running inside, the total time of migration grows by 4.7%, and the downtime increases by only 3 milliseconds. More evaluation results can be found in the paper.


  • [DSN] Jinyu Gu, Zhichao Hua, Yubin Xia, Haibo Chen, Binyu Zang, Haibing Guan, Jinming Li, Secure Live Migration of SGX Enclaves on Untrusted Cloud, Proceedings of the 47th IEEE/IFIP International Conference on Dependable Systems and Networks (DSN'17). Denver, CO, June, 2017. [pdf]
projects/sgx-migration.txt · Last modified: 2017/12/26 10:30 by root