Mercury: Combining Performance with Dependability Using Self-virtualization

Virtualization is becoming a powerful and enabling technology for both research and daily use. However, the performance degradation it brings is not negligible. Evaluations on the two most popular virtual machine monitors (VMM), VMware and Xen, shows performance overhead from several percents to tens of percents for various workloads. IO intensive workloads suffer much more serve performance degradation than CPU intensive workloads.

Meanwhile, many virtualization-enabled features such as online maintenance are not performed frequently. For those who doesn’t need a virtualization layer underneath all the time, the performance loss is rather annoying. This is especially true for performance critical computing.

It is noticed that if just one VM is enough for regular work for most of the time and virtualization features comes to help infrequently, it’s not necessary to have virtualization standby all the time. This project proposes a technique, called self-virtualization, which could dynamically activate a VMM and insert it under the operating system. This procedure could be done rather swiftly (about 0.2ms), without being noticed by user. Reverse procedure could be done for the operating system to back to regular run. By providing self-virtualization technique, an system administrator could choose to enable/disable virtualization at will.

To present self-virtualization technique, we need both static alternations and runtime supports.

• Narrow the gap of runtime difference between an operating system on bare hardware and an operating system on VMM. Here we refer the former native operating system, and the later virtualized operating system.
• Provide a mechanism to dynamically activate a VMM and insert it under the operating system. This means the operating system should be dynamically lifted to a lower privilege level, leaving the most privileged level to the VMM. Reverse procedure should also be available.

By leveraging para-virtualization (this is a virtualization technology for x86 architecture, OS is modified to co-operate with VMM for performance and correctness), we made some alternations to the operating system so that it could be more alike a virtualized operating system. These alternations include:

• virtual address space layout: 64MB virtual address space is reserved for VMM
• memory management: provide native operating system the knowledge of pseudo physical address and real physical address

To dynamically activate and insert a VMM under the operating system within a tiny time span, Mercury warm up the VMM at machine boot time and deactivate it. In this way, most of the data structures are ready and in memory. What should be done when virtualizing the operating system is to re-fill the VMM with correct runtime data, and all of which could be calculated efficiently.

A “virtualize my operating system” command is issued by a user to a /proc file, which further issues an interrupt on a dedicated interrupt line. In the interrupt handler, several procedures are done to switch the operating system to the virtualized mode.

• De-privilege of operating system: lift the operating system to ring-1, install VMM’s GDT/IDT to hardware, write-protect all page tables
• Re-calculate VMM states

Here is the figure of the overall architecture of Mercury.

The “VO” in the figure refers to virtual object. A virtual object is regarded as an operation function set. One virtual object is specific for one machine interface. For instance, “Native VO” is the function set that operates bare hardware; “Virtual VO” is the function set that operates the virtual machine interface abstracted by the VMM.

Our performance measurement shows that Mercury incurs very little overhead: about 0.2 ms to complete a mode switch, and negligible performance degradation compared to Linux.

Here is the table for micro-benchmarks and application benchmarks.

We have proposed a technique, called self-virtualization, which enables an operating system to dynamically activate and insert a full-fledged VMM under the operating system. This approach is completely software-based. It effectively eliminates unnecessary performance overhead that virtualization incurs at regular running time and thus combines performance and dependability for performance critical computings. To demonstrate the technique, we implemented a prototype named Mercury. Performance measurements show that such an implementation incurs negligible performance overhead and allows fast switches among different execution modes.

• Haibo Chen, Rong Chen, Fengzhe Zhang, Binyu Zang and Pen-chung Yew. Mercury: Combining Performance with Dependability Using Self-virtualization. (Best Paper Award) In Proceedings of 36th International Conference on Parallel Processing (ICPP-2007), XiAn China, September, 2007. [pdf]

We thank the members of system research group in Parallel Processing Institute. This work was funded by China National 973 Plan and Intel University Research Grant.