VirtualIron.com | Join | Contact Us

Blog
Virtualization Blog
Virtualization Blog - Virtualization Clinic: Benchmarks
Decrease font size
Increase font size
January 26, 2007
  Virtualization Clinic: Benchmarks

One of the questions we were asked after our recent webcast with IDC (recorded version coming soon) was what our thoughts were on what virtualization benchmarks should be used and when these metrics would be released.


It's a great question and we strongly believe that virtualization performance depends on the application characteristics (for example, CPU, disk, and ethernet IO).


Currently, there are two benchmarks being developed by the industry - one by VMware and the other by Intel. You have to carefully look at what the benchmark measures to determine its applicability to your task. Both VMware and Intel are using a blend of many benchmarks to approximate a "standard" consolidation environment and are being submitted to Spec.org. We expect they will be in committee for some time.


There are a number of benchmarks that measure different attribute types. For example, SpecINT (also from Spec.org) measures CPU performance. If you are running CPU intensive workloads, then this may be the most applicable benchmark. There are disk IO benchmarks such as bonnie, or application benchmarks such as Exchange. See http://www.microsoft.com/technet/prodtechnol/exchange/2003/performance.mspx for more details.


The bottom line is that everyone should evaluate the types of workloads that they plan to virtualize and run your own benchmarks, as your environment (network configuration, hardware, operating systems, etc.) will influence the results greatly.


Post your questions in our forums and we'll answer them!


    Posted By: Alex V @ 01/26/2007 02:32 PM     Virtualization  

February 5, 2007

Comments


 
What do you mean "there are two benchmarks being developed by the industry"? For performance benchmarking, there are all kinds of levels (CPU, memory, I/O etc). For disk I/O benchmark tooks, I know there are bunch of them available and most of them are open source tools. I would be very interested to see if Virtual Iron can publish I/O performance data as I'm doing my own I/O performance analysis on VI now.

 Posted By: Liang Yang @ 02/05/2007 02:04 PM   :  Post a reply

 
I would not surprise to see VI and Xen have very good CPU performance in guest domains. But for a mature product, I/O (disk I/O and network I/O) performance in guest domains are much more useful and meaningful than CPU performance.

 Posted By: Liang Yang @ 02/05/2007 02:11 PM   :  Post a reply

February 9, 2007
 
I was speaking of benchmarks specifically for virtualization, though we agree. There are many benchmarks that should be looked at and what should be looked at can be very different for different organizations and cases.

 Posted By: Alex V @ 02/09/2007 08:25 AM   :  Post a reply

 
But SPECint you mentioned in your post is never intended to be a tool for benchmarking of any virtualization technology. It appears long before VT-x and it is used to test how fast the CPU could be. With dual-core or quad-core platform, i think we would like never worry about the CPU in the guest domain. Instead, I would think more about I/O.

 Posted By: Liang Yang @ 02/09/2007 11:52 AM   :  Post a reply

 
BTW, do we really need to a benchmark tool which is related to virtualization techbology. The purpose of VT is to let you know there is no VT there. So any native tools should be the best for the performance purpose.

 Posted By: Liang Yang @ 02/09/2007 12:13 PM   :  Post a reply

FuseTalk Standard Edition - © 1999-2007 FuseTalk Inc. All rights reserved.


Copyright © 2003-2007 Virtual Iron Software, Inc. | Privacy Statement | Terms of Use | Site Map