In the first two parts of the series we’ve been talking about the basics of vFoglight and how the system components operate.  In the previous post we were testing about 30 VMs, so now we’ve added another 70VMs bringing the total up to 100.  Again, we can look at the JVM Memory Usage and Load Estimator to see how taxed the system is. 

 

Figure 1 - JVM Memory Usage

Figure 2 - Load Estimator

As you can see in the figures, there is data growth over time and periodic spikes in load.  What is happening to cause the spikes?  Garbage Collection, GC for short, is running on the vFoglight Management Server during those spikes.  Essentially, as the JVM creates new objects, it uses memory to store the objects.  Over time these objects become unused and the system could reclaim that bit of memory space to reuse for other purposes.  That is what is happening during these spikes, GC is finding unused memory, clearing it out and allowing it to be reallocated.  Garbage collection is akin to Memory reclamation in many senses.

Since the load is acceptable for 100 VMs, we will now go ahead and add in approximately 650 additional VMs.  In total we’re now going to be importing 809 objects.  Objects are comprised of: Virtual Machines, ESX Hosts, Resource Pools, Folders, Clusters, Data Centers & vCenters.

Looking at the new load after adding the additional objects, we can observe that the load is maxing out the 2G of memory during GC periods.  As the host is well undersized for a 64-bit application, it will be shutdown and more memory will be added.

Figure 3 - JVM Memory Usage & Load Estimator

After adding memory to the system, which is also highlighted in Figure 3, we can see load is once again reasonable.  Essentially doubling the resources the vFoglight host, the load of the system has significantly dropped and performance has returned to normal.

vFoglight VM - Updated
OS - Windows 2003 R2 x86_64
vCPUs - 2
Memory - 4G (Changed from 2G)
Hard Drives - 3 VMDKs on Raid Group 2 (only VMDKs on the Datastore)
        10G C:\ (OS Partition - 64k aligned)
        5G D:\ (Swap Partition 4G Fixed size - 64k aligned)
        30G F:\ (Application Partition - 64k aligned)
vFoglight 5.2.6 x86_64

In the next post I’ll be talking about how we start to scale to something much bigger.  We’re already at 809 VMs/objects!  Next we will do over 2500 & 5000 VMs/objects on our way up to over 10,000 objects!

Stay tuned,
Thomas

 

 

Continuing from the last post, we are now going to explore installing vFoglight 5.2.6 into a VM and start pointing vCenters at it.

For the purpose of the test I’m using the following configurations to start and we will grow it as we add more vCenters and thus more VMs or as performance dictates.

 vFoglight VM
OS - Windows 2003 R2 x86_64
vCPUs - 2
Memory - 2G
Hard Drives - 3 VMDKs on Raid Group 2 (only VMDKs on the Datastore)
        10G C:\ (OS Partition - 64k aligned)
        5G D:\ (Swap Partition 4G Fixed size - 64k aligned)
        30G F:\ (Application Partition - 64k aligned)
vFoglight 5.2.6 x86_64

Physical Hardware
Dell T610
OS - vSphere 4.0
CPUs - 2x Intel Nehalem L5520 @ 2.27Ghz (8 Cores + 8 HT = 16 Logical CPUs)
Memory - 32G
Disks - 3 RAID1 Groups
        Group1 = 2x 146G 15K RPM SAS
        Group1 = 2x 146G 15K RPM SAS
        Group3 = 2x 1.5TB 5400 RPM SATA

A little house keeping before we get into some data.  I’ve chosen to go with the x86_64 version of vFoglight, and intentionally I’ve undersized the requirements to allow us to show a single host and how it can scale to large environments.  As with many 64-bit applications, the memory requirements essentially double vs. 32-bit applications as they require a larger foot print right away to handle more memory pages.  The default behavior of a JVM is to allow for utilization of 75% of total system memory.  For example, below you will see figures around the JVM with top at 1.5G for possible use in the JVM when the total system memory is 2G.

After downloading vFoglight 5.2.6, I simply followed the installation process doing a custom install and changing the paths to F: instead of C: as that is where I want to place the vFoglight application.  This is good for many reasons, but one of the main reasons is it allows me to easily see the Disk IO for the entire application.  

FIrst up, a single vCenter with 1 ESX host and 30 Virtual Machines.  After configuring the collector to talk to this vCenter, it does the heavy lifting of getting the data over to the vFoglight Management Server.  Below in Figure 1 & Figure 2 you will see the memory required to run the Management Server, collect the metrics and then the load of the FMS as we see from within the application.

Figure 1 - JVM Memory Usage

 

 

 

 

 

 

 

 

Figure 2 - Load Estimator

 

 To be continued…

Thomas Bryant III

Vizioncore Senior Product Architect