Metrics to measure the success of the cloud: A report from VMworld 2012

Metrics to measure the success of the cloud: A report from VMworld 2012

Although we were happy to see Michael Dell and Paul Maritz in attendance at VMworld 2012 (the latter receiving a “standing-O” after his Tuesday afternoon lunchtime address), we’ll leave the reporting on their activities to the bigger trade publications.

Instead, we’ll share some stats presented by VMware’s Sanjay Mirchandani and Craig Stanley during a standing-room-only breakout session titled, “Planning and Measuring the Impact of Cloud: IT Metrics that Matter.”

The goal of Sanjay and Craig’s session was to provide IT leaders with some baseline metrics for measuring the success of moving to the cloud, while also emphasizing the value that the IT organization added to the cloud transition.

Sanjay and Craig want IT leaders to be able to convince the end users in all business units that cloud initiatives should fall under the jurisdiction of IT. To help them do so, the pair established benchmark metrics in three categories — efficiency, agility, and reliability — that all IT departments should aim to achieve.

As your organization moves to the cloud, see if you can clear the bar set by Sanjay and Craig:


  • Total annual cost per end user: $1,800
  • Percentage of IT labor cost spent on application development: 18 percent


  • Average time required to provision new virtual machine (VM): 1 ½ days
  • Percentage of overall capacity procured on a variable basis (monthly or more frequently): 15 percent
  • Percentage of capacity provided to tenants on variable basis (monthly or more frequently): 60 percent
  • Average time to release a non-legacy application or functionality: 156 days


  • Percentage of time SLAs are met: 99.9 percent
  • Total planned annual downtime per application: 360 minutes
  • Total unplanned annual downtime per application: 60 minutes
  • Number of external failed audits: 0
  • Total external audits: 16

Craig augmented his 99.9 percent stat by stating that he preferred to know HOW OFTEN and under what circumstances data was unavailable. He said, “You can be down 10 minutes every week and still deliver a 99-percent availability rate. But if you’re an end user who needs the data right now, to you it’s ALWAYS down!”

How does your organization’s performance stack up against their numbers? If you’d like to talk about ways to improve your move to the cloud, check out the available Cloud Migration Workshops available from the consultants at SHI Labs.