Over the course of the next few weeks, I’m going to be building out a few vFoglight servers running the latest version of our product, 5.2.6, and the next release 6.0.  We will be looking at performance as we load up VMs from 10 to 100 to 1000 to beyond … plus it’s all going to be running in a VM.

Before we begin to install the product, the typical thing to do is to size out your environment and architect a solution based on what you know, and some of what you don’t know.

 There are a series of questions you have to ask:

·         How large is my current environment?

·         How long/big do I want to plan growth for?

·         What are the current and expect total number of vCenters, Data Centers, Clusters, Resource Pools, Folders and Virtual Machines?

·         How much data do I want to keep?

The first three are all up to your environment, but the third question plays in to the last one as well, around how much data do you want to keep?  It’s important because you want to size your database appropriately for growth.  By default, vFoglight doesn’t expire data.  We do, however, age data, while maintaining its historical importance.  I’ll get into that in a minute, but let’s look at data growth and why we want to be sure we plan appropriately.

Consider that for a single VM, there are over 200 statistics to collect.  Most, if not all, have some importance, so as your environment grows there are going to be a lot of data points.  For example, a Resource Pool has its own CPU/Memory limits, reservations and values.  Those are on top of the rollups you need for things like CPU %Used for the Resource Pool, which would be the sum of all the VMs inside it.

So back to the number, we want to ensure that we have accurate numbers and plan for growth so we can deal with the large volume of data coming in.  To help control the data flowing in we age data gracefully, which is configurable as well.

The system defaults to aging data as follows:

·         Store raw data in memory for 15 minutes

·         Roll-up raw data to 15 minute averages and store for 3 days

·         Roll-up data to hourly averages and store for 2 weeks

·         Roll-up hourly data to 4-hour averages and store indefinitely

But wait!  If you roll up data, it loses its historical importance and becomes worthless over time, right?  After all if you average data, it will lose peaks and valleys and turn into a gentle curve!  Normally, that is true in most cases.  However for vFoglight we do something quite different for metrics, which is one of the really powerful portions of the tool.

For virtually every metrics, we store:

·         Minimum value

·         Maximum value

·         Average value

·         # of samples

·         Standard Deviation

·         Sum of squares

This means that while we roll up data, you may not be able to determine the peak in CPU utilization was at 2:24 or 2:30, but you could see that the Maximum value from 2:15 - 2:30 was 95% vs. the average which may just be 30%.  This ensures that even if the data is rolled up time and time again, you still have the same importance without having to take up all of that space and affect performance.

So how big is a VM, ESX host, etc…?  On average, each object is going to take up about 1MB a day of data over the course of a year.  This grows at first rapidly, due to the raw data highlighted above, to around 30MB after 3 days and then gradually climbs to roughly 356MB over 365 days.  This means, that if you add up all the objects you can easily find the size of a database & plan for growth.

For example, with 100 ESX Hosts running 2000 VMs you can expect around 15G of data in the first 3 days and growing up to about 146G over the course of a year.  For this I would then want to add in growth rates and a buffer to ensure to ensure plenty of room, 10% is a typical number that customers use, and you have your database.

The next part of puzzle is how BIG does the vFoglight Management Server have to be?  I’ll cover that next as we start testing out different size environments.

To be continued…

 

Thomas Bryant III

Vizioncore Senior Product Architect

At the recent VMworld show in San Francisco, we had the honor to win the Gold award for vRanger 4.0 in the Data Protection and Business Continuity category.  This is recognition that is loved by the business, but carried with a badge of honor by the entire cast of people that work on a product.  Everyone puts a little piece of themselves in every product they work on.

 

It’s for these reasons that we celebrate our award and those of the finalists and winners in all categories.   It’s in Vizioncore’s interests to have a strong, growing virtualization industry with a robust ecosystem – it’s reflective of the overall opportunity and business value that we can create in the market.

 

In the data protection and business continuity category, we won after a comparison was made by a set of judges.  At the show, they compared products against a long list of criteria and also did hands-on walkthroughs.    The judges commented that the winner, “provides a cleaner interface and all-around faster tool, speed being crucial in this category.”    This category was filled with a long list of entrants – so the competition was tough. 

 

We are now focusing our efforts on what it will take to win next year’s award in this category.

 

Gartner states that there are 50 million x86 server sockets in the world.  We believe that a large majority will be virtualized in the coming years.  Today – with generous estimates – there is somewhere between 5-8M x86 sockets virtualized across Windows and Linux.  And the bulk of the remaining sockets will be virtualized over the next three years.  With an average of 7.5 VMs on each socket, each with about 100 GB of allocated space, all running workloads spanning applications, databases and desktops, the size of the data protection problem is only going to increase. 

 

We saw an opportunity to innovate in the market space by deploying a strategy based upon these tenets:

 

1)    Simple, affordable for the SMB

2)    Scalable (data, num VMs, num sockets) with an easy to articulate ROI for the enterprise

3)    A solution that provides a complete – and integrated - lifecycle management approach to virtualization from analysis through automation. 

4)    A solution that merges the application, database and virtualization worlds by leveraging the core assets of Quest and Vizioncore.  Examples of this opportunity include moving beyond object-recovery of images into application, farm, and environmental recovery.  Another aspect is our ability to drill into the VM to monitor application internals – not just processes - such as Oracle, SQL Server, and Java. 

5)    A solution that combines the VDI with server virtualization management enabling companies to deploy, manage and control their VDI environments with the same processes they are using to manage their data center infrastructure.

 

To execute on this strategy, we saw a need to re-invent the vRanger platform to prepare for a world where legacy approaches would be stretched thin.  The requirements for the platform included:

 

1)    It must have a direct-to-target architecture between sources and targets.  This would make backup proxies optional and introduce massive parallelism that can be configured in a central management console.  This would reduce overall backup windows.

2)    It would be a platform that would enable the consolidation of conversions, backup, replication, eDiscovery, and archiving functions in a unified, common way.  It would acknowledge that all of these functions can be handled through a common image management approach, in an integrated way.

3)    It would provide management flexibility – not just in the jobs that could be scheduled – but in its entire control.  It would need to provide PowerShell and Web Services APIs to make the system integrate with administer and corporate automation tools such as RBA.  (See www.thevesi.org as one example.)

4)    It would enable multiple source options to deal with heterogeneous virtualization environments – so enabling backups through proxies, consoles, or virtual appliances. 

 

We began engineering this new platform two years ago and delivered the first version of this technology with vRanger 4.0.  The introduction of vRanger 4 also enables us to be more agile in the market with faster releases that are extensions of the core engine that has been built.    Our near term 2009 releases include releasing a new form of global deduplication that is patent pending, Amazon S3 / FTP targets, gateway appliances, change block tracking to eliminate scan times, vStorage support, ESXi, replication integration and reading from vRanger 3 repositories. 

 

As of Monday 9/14/09, we’ve hit RTM of vRanger 4.1 that includes new features around interoperability with vRanger 3 backups.  This release is being distributed to customers now and to the general public shortly.

 

For those customers that require ESXi and VCB support today – we are continuing to provide Full Support to vRanger 3.x giving you a technology and support option. 

 

The longer term releases of this platform include a focus on Sharepoint, SQL Server, and Exchange object, application, and farm recoveries; multi-repository search and indexing; integrated backups with VDI; and multi-hypervisor support starting with HyperV. 

 

An important part of any data protection product is its ability to be reliable.  And there was some blog chatter about a potential issue where vCenter could crash if it accepts API calls too early: http://laez.nl/vizioncore-vranger-4-0-pro-crashes-vcenter-2-5-update-4-update-5/.  While most software issues are contained within support and KB articles – this one was discussed in the community – and should be addressed in the community as well.  This issue was reported toward the end of August.  Since that time, we’ve created a resolution that is part of vRanger 4.1 now being delivered to customers.   Like all software products, we encounter bugs – and it’s the professionalism and dedication of our dev teams that enable us to resolve these issues quickly.

 

We look forward to servicing the virtualization market in the coming years with the new platform.  And we look forward to competing for next year’s Gold award by showing how we will be bringing together the worlds of applications, VDI, and virtualization.  

 

Tyler Jewell

VP, Products, Virtualization

Vizioncore and Quest

Tyler.Jewell@Quest.com

978-884-5355

 

 Hi, James Kahn here again, Systems Engineer for Vizioncore in Australia, talking about automation.

 

Automation: What Is It?

 

As the name implies, automation is taking time-consuming, manual tasks, and making them automatic.  Automation itself has been present in IT for years in many forms, ranging from administrator scripts to single-purpose toolset applications.

 

While automation has previously been a nice-to-have, and used in an ad-hoc fashion for many IT departments, with IDC’s estimated 40% year-on-year VM growth (Source: IDC, Virtualization and Multicore Innovations Disrupt the WW Server Market, Doc #206035) – virtual machine sprawl – automation is fast becoming a necessity for IT department productivity with an ever-growing number of VMs in the enterprise.

 

Automation Frameworks

 

An automation framework should be easy to use, so that automation is easy to implement in the business.  For IT automation to be effective, it needs to be accessible to the administrators and engineers that will be using it.

 

The Virtualization EcoShell (link) – the brainchild of Scott Herold (link) – is an administrator level automation tool to quickly and easily improve individual productivity when managing a virtualized environment.  Out of the box – or straight from the download, if we’re being pedantic – you can do tasks like automatically build a Visio diagram of your virtualized environment, or have all your ESX hosts instantly scan for new storage, tasks that have traditionally taken a lot of time.  You can also use EcoShell to build PowerShell scripts to automate environment-specific tasks.

 

Most of the benefits of automation are realized when it is fully embraced by IT as a whole, rather than by individual administrators.  vControl (link) is a virtualization automation framework that allows the automation philosophy to be integrated into IT operations – the entire IT department, and therefore the business, can benefit from automation, with systems knowledge and automation processes contained centrally rather than in disparate scripts spread throughout the enterprise.   IT Subject Matter Experts can embed routine operational tasks in the framework, freeing time to focus on pro-active projects, rather than operating reactively.  vControl uses drag-and-drop and a diverse set of options for building automation workflows – including integrating standard script languages like PowerShell and batch files that you are probably already using.

 

So What Could I Do?

 

All this talk of automation and efficiency is great, but I’m a concrete example kind of guy, and I like to hear of specific examples.  I’m guessing you do too.  I’ve put together three possible examples below of how you could benefit from automation. 

 

Example 1:

A misbehaving multi-tier application crashes relatively often, and needs frequent rebooting in a specific order, otherwise it doesn’t start.  Rather than waiting for users to open a helpdesk ticket, create an automation workflow that executes every Saturday at 3am to:

·         Shut down the application cleanly;

·         Take a vRanger Pro backup of the VMs;

·         Start all the application’s VMs in the correct order;

·         Run a test script on the application; and,

·         Send an email to the administrator if the test was successful or open a helpdesk ticket, restore from the latest vRanger Pro backup and page the on-call engineer if the application test failed.

 

Example 2:

IT is doing a network migration project and needs to migrate hundreds, or even thousands of virtual machines to a different virtual network overnight.  Create a workflow to switch the virtual machine’s networks to the new network, and execute it to quickly cutover to the new network.

 

Example 3:

The company’s main website has unpredictable periods of very high traffic.  Create a workflow to measure load and response time of a web application, and automatically start up new web servers and reconfigure a network load balancer as demand increases.  Shut them down as demand decreases.

 

I’m only just scratching the surface here with these examples.  One of the great things about automation is that you can tune it to your business requirements.  The potential is huge.

 

- JK.