Paravirtualization (as an approach to run virtual machines) has been getting a lot of attention lately. The primary hypothesis is that paravirtualization makes the job of virtualization easier and more efficient than pure hardware virtualization.

There was a time and place where paravirtualization made sense, but that time has now passed. Let me explain why it is no longer a viable approach.

Paravirtualization (an idea orginally introduced by the Denali Project) proposes to modify a guest OS in order to redirect virtualization sensitive operations directly to the virtual machine monitor instead of trapping to it as done in pure hardware virtualization. Pure hardware virtualization is where hardware is completely virtualized and supports unmodified guest OSs.

Paravirtualization requires substantial engineering efforts in modifying and maintaining an operating system. However, these heroic efforts are inevitably losing the battle against Moore's Law and hardware advances being made in the x86 space. By the time the first product with paravirtualization appears on the market, more than 80% of the shipping x86 server processors from Intel and AMD will have hardware-based virtualization acceleration integrated into the chips (Intel-VT and AMD-V or "Rev-F"). This hardware-based acceleration is designed to optimize pure virtualization performance, primarily the virtualization of CPU, and it renders OS paravirtualization efforts as completely unnecessaryand behind the technology curve. Perhaps more important, hardware-based virtualization doesn't require users to upgrade their operating systems, or their middleware and application stacks - a very complex and resource-intensive undertaking.

The issue with paravirtualization is that once you break the strict hardware interface between the hypervisor and the guest OS, one might as well call the "hypervisor" a microkernel and a guest OS a process and move on. And we all know how "well" an OS works with a microkernel. Linus Torvalds posted some comments on the recently flamed up (again) debate on the merits of a microkernel, stating, "It's ludicrous how microkernel proponents claim that their system is 'simpler' than a traditional kernel. It's not. It's much, much more complicated, exactly because of the barriers that it has raised between data structures." I concur with Linus.

Virtual Iron has decided against paravirtualization in favor of "native virtualization.". With hardware advances coming out of Intel and AMD, we see native virtualization capable of matching physical hardware performance without any of the complexity and engineering efforts involved in paravirtualizing an OS. From our discussions with a broad range of users, they simply do not want to roll out modified OSs unless the trade-off is heavily in their favor. This Faustian trade-off is no longer necessary.

The current batch of chips offering hardware-based virtualization acceleration from Intel and AMD, primarily helps with virtualization of CPUs and very little for virtualizing an I/O subsystem. To improve the I/O performance of unmodified guest OSs, we are developing accelerated drivers. The early performance numbers are encouraging. Some numbers are many times faster than emulated I/O and close to native hardware performance numbers.

Just to give people a flavor of the performance numbers that we are getting, below are some preliminary results on Intel Woodcrest (51xx series) with a Gigabit network, SAN storage and all of the VMs at 1 CPU. These numbers are very early. Disk performance is very good and we are just beginning to tune performance.

Bonnie-SAN - bigger is better             Native                  VI-accel
Write, KB/sec                                    52,106                 49,500 
Read, KB/sec                                    59,392                 57,186 

netperf - bigger is better              
tcp req/resp (t/sec)                             6,831                   5,648

SPECjbb2000 - bigger is better 
JRockit JVM thruput                            43,061                 40,364

One we complete the tuning, we expect guest OSs to compare very favorably to running on native hardware, but without requiring users to upgrade their OSs. This should open up users to applying virtualization to a whole new sets of applications and workloads.

I welcome your comments.

Edited: 07/07/2006 at 03:20 PM by Virtualization Blog Moderator

Paravirtualization is a dead-end approach is perhaps a bit sensational, don't you think?

The first thing to point out is that I believe your accelerated drivers are paravirtual right? (this is what was mentioned on xen-devel). Clearly, paravirtualization is not a dead-end :-)

I think a more accurate title may be that CPU paravirtualization is a dead-end. I understand this argument being based on engineering trade-off. I believe this is really just a Xen argument though. Our interfaces are way too complicated.

If you look at Rusty Russell's proposal on osdl-virtualization, or even VMware's VMI proposal, it's clear that CPU paravirtualization can be done with very little engineering overhead. Once you've got the basic paravirtualization (which is essentially just software traps instead of world exits), you can then selective explore other paravirtualization optimizations.

I won't defend shadow paging, but certainly there are useful paravirtual optimizations that aren't IO related. For instance, page table update batching is really useful for minimizing exits. Also, it's definitely worthwhile to paravirtualize the platform timer (so the OS can know about "stolen" time).

Another important point to make is that this paravirtual optimizations don't have to be mutually exclusive with VT/SVM. They would still be useful in this scenario.

In my mind, the right way to do paravirtualization is to treat it as software VT/SVM. Doing this in an OS like Linux is pretty trivial (the patches are only a couple kslocs).

Another point to make is that I'm not convinced you really need paravirtual IO drivers to obtain this level of performance :-) I suspect that this will know this for sure in the near future but I thought I'd at least put it out there :-)

I applaud your efforts.

I did have a question and I can't seem to be able to find an answer on VI's website.

VI keeps claiming that they can make a VM scale across multiple physical boxes for performance.

I fail to see how that's possible unless dedicated cluster interconnects are used.

A hypercube topology with multiple interconnects can typically provide good results.

How does VI3 do it? Especially since it's now using Xen, and Xen doesn't have this capability I believe.

Thanks,

Dimitris

Virtual Iron doesn't scale a single VM across multiple boxes. With the advent of multi-cores, this feature is no longer necessary.

Hi, I would like to use a I/O benchmark tool which can truly saturate your disks. Bonnie does not support setting the number of outstanding I/Os, nor does it support AIO. if you use Bonnie, it will be probably not to get your SAN disks saturated. Then disk I/O data measured this way may not be accurate. Liang