So you've decided to implement virtual workloads in your environment and reap the benefits that consolidation can provide—flexible resource allocation, improved utilization factors, and better use of IT resources. But with this consolidation come a realization and a consideration: With consolidation we have multiple workloads sharing the same hardware, often with one workload hosting resources for another. What steps can we take to help ensure the availability of these consolidated workloads? That is, how can we avoid single points of failure that the consolidation approach could impose on the configuration?

I'll focus on several items that you should consider when configuring a consolidated approach to your workloads. You'll need to keep in mind several factors, including enabling high availability to storage, network communications, and even processor and memory resources of the partition itself. We'll look at ways you can make access to virtual storage, network access, and even the partitions themselves highly available. In this article I'll concentrate solely on keeping the resource environment highly available. The subject of keeping applications within the various hosted OSs highly available is beyond the scope of this article.

Virtual Storage Considerations
At a high level, the idea of virtual storage is that one partition owns the actual, physical, disk controller and disk drives and abstracts a portion of the physical storage to other partitions. In the Power Systems environment, we have two possibilities for the "hosting" partition: an IBM i partition or a Virtual I/O Server (VIOS) partition. These partitions can host I/O resources for IBM I 6.1, Linux, and AIX partitions, and an IBM i hosting partition can also virtualize I/O resources for an integrated Windows server. Consider the configuration that Figure 1 [2] illustrates.

What will occur if the hosting i partition is taken down—whether because of an unlikely system crash or for system maintenance? If the hosting partition isn't active, it cannot provide I/O resources for other partitions. Put another way, in this configuration you could view the IBM i partition as a single point of failure for the other partitions. Two methods that you could consider for avoiding, or at least mitigating, this potential single point of failure are to mirror the virtualized storage between two physical systems or two hosting partitions on the same managed system, providing storage that the hosted (or guest) partition can then mirror. Let's examine both these options in a little more detail.

Mirrored Hosted Storage
With mirrored hosted storage, the objects representing the virtual storage are located in a physical storage device that can be mirrored between two physical systems. In the configuration that Figure 2 [3] shows, an IBM i partition is providing storage for hosted workloads. The network server storage spaces (i.e., the virtual disks) are located in the independent auxiliary storage pool (IASP) on both systems. IBM i mirroring technology, such as geographic mirroring, is used to mirror the virtual storage between two IASPs (one on the production system and one on the backup.) So at this point, both systems have a copy of the virtual disk.

Additionally, the second or backup system will also need to have defined the logical partition for the workload(s) as well as the network server description. If this solution is being implemented for the x86 integrated solutions, the additional configuration objects required for those solutions will also need to be defined on both systems. When a shutdown of a workload occurs on the production system, that same workload can be brought up on the backup system. In fact, there are exit points available in the vary operation of the network server that can be used to automate the bring-up of the workload on the backup system. You can set up a similar configuration by using two SAN devices (one connected to each system), then using SAN replication mechanisms, such as remote mirror and copy, to keep the virtual disks in sync across the two devices.

Keep in mind that the solution I've just described will keep the object or objects that represent the virtual disk in sync between storage devices connected to two disparate systems. You'll need to ensure that both systems have LPARs defined for the workloads being made highly available as well as enough processor and memory resources to support the workload(s) if failed over to the backup system.

Another less elegant way to achieve a similar solution would be to manually copy the virtual storage object between the production and backup systems. Using IBM i as the example hosting partition, you could use the SAV command to save the network server storage space (NWSSTG), then use the RST command to restore it on the backup system. Prior to i6.1, this solution requires downtime for the hosted workload as the storage space cannot be copied while active. With release i6.1, you can take advantage of the save-while-active support to save the storage space even when the hosted workload is active—thus making this a more manageable solution. Keep in mind that the storage space on the backup system will only be as current as the last time it was backed up from the production system. Thus, this is a good solution for infrastructure-type workloads that have little change in data but may not be the best solution for workloads that see frequent data change.

Hosted Storage Spread Between Hosting Partitions
Another method that can be used to provide high availability of virtualized storage is to provide virtualized storage from more than one source, then allow the workload to mirror the storage. In the configuration that Figure 3 [4] illustrates, the guest partition (the partition that's benefiting from virtualized storage) is being provided storage from two hosting partitions (the hosting partitions can be either IBM i partitions, VIOS partitions, or a mixture of the two). This configuration requires a virtual SCSI adapter relationship between the guest partition and each of the hosting partitions.

Now, since the guest workload has multiple disks, it can implement its own software disk protection scheme across those disks. As an example, if the guest workload is a Linux partition, the software RAID support in Linux could be used to mirror the storage across the multiple "disks" that Linux sees. When one of the hosting partitions is taken down, the hosted OS will see this as a "failed" disk and continue to run on the mirrored disk. Then when the hosting partition is brought back up, the "disk" will again be available to the hosted OS, and it can re-mirror, or re-sync the storage. Note that the ability to support this type of configuration for a hosted IBM i partition is dependent on Power 6 hardware and both the IBM i guest and host partitions being at release level i6.1

Network Considerations
Let's explore what you can do to make virtual network connections highly available. In the configuration that Figure 4 [5] shows, a virtual LAN has been established between the partitions, and that virtual LAN is being bridged (or routed) to the external network through the network interface on the IBM i partition. As shown, if the physical network adapter on the IBM i partition becomes inoperable, network traffic to all the partitions on the virtual LAN will be lost. You can mitigate this problem by taking advantage of IBM i's support for proxy across multiple physical network cards. To understand this approach, first you need a basic understanding of how Proxy Address Resolution Protocol (ARP) allows traffic to flow between the physical and virtual networks.

In a Proxy ARP environment, a subnet is created for the virtual network connections; this subnet is a subset of the larger network that the physical IBM i network adapter is connected to. The addresses on the virtual Ethernet adapters and the subnet mask indicate the network size and starting and ending point of the subnet range. Additionally, the network configurations of the workloads on the virtual LAN use the address of the IBM i partition's virtual LAN connection as their router. When a workload on the virtual LAN needs to communicate outside of the virtual LAN address range, it does so through the gateway, which then bridges to the external interface on the IBM i partition. The final piece of the puzzle is the bridge between the virtual and physical networks on the IBM i partition. You configure this bridge through the Associated Local Interface setting of the virtual LAN's TCP/IP interface in IBM i. If we leave the story there, we still have the single point of failure; that is, if the associated local interface is down, the bridge is down, and we cannot get network communications to the virtual LAN.

The solution is to allow the virtual Ethernet interface on the IBM i partition to proxy across multiple physical interfaces. You set this configuration though iSeries Navigator, as Figure 4 shows. With this configuration, you create the TCP/IP interface for the virtual LAN connection in iSeries Navigator, then adjust the properties. Notice that Proxy ARP has been enabled and that multiple interfaces are listed in the preferred interfaces list.

Here's how it works. Assuming that both preferred interfaces are active, when the interface on the virtual LAN is started, it will proxy across the first physical interface in the list. Only if that interface goes down will traffic then be proxied across the second interface in the list. If/when the first interface becomes active again, traffic will immediately be proxied again across the first interface because our configuration has indicated that is the most preferred interface. All of this occurs without loss of network connectivity; thus, we've established a highly available network configuration. Keep in mind that this approach doesn't provide load balancing; subsequent addresses in the preferred interface list will be used only if the first interface becomes inactive. When this capability is implemented for the IBM i interface on the virtual LAN (i.e., the interface that will be the gateway for the other workloads), a highly available network structure for all the workloads on that virtual LAN is the result, as Figure 5 [6] shows.

Of course, this solution requires multiple physical network connections to be available to the IBM i partition performing the routing. Something to keep in mind is the availability of the Integrated Virtual Ethernet (IVE) adapter on the Power6 systems, which allows for up to 32 separate/distinct network connections to be established through up to four physical ports on the IVE. Mapping one of the network connections from the IVE along with a second non–IVE network connection to the IBM i partition would be a good way to provide the multiple physical network connections required for this solution.

In an upcoming article, I'll provide more details about a variety of methods available for networking in a virtualized environment.

Live Partition Mobility
So far we've looked at how to make virtual storage and virtual networking for a workload highly available. The final item we'll examine is how to make the partition itself highly available. For Linux and AIX partitions, Live Partition Mobility, a component of PowerVM, facilitates taking a running partition along with its hosted applications and moving it from one physical server to another without disruption to the infrastructure services the partition is providing. Typically the process of moving a running partition from one physical system to another takes only a few seconds, and the entire state of the OS is maintained—including processor state, memory, attached virtual devices, and connected users. Live Partition Mobility is intended as a proactive process—that is, as a solution to a planned outage. It can be a great aid in helping to meeting SLAs since running partitions and applications can be moved from one server to another. A workload no longer needs to be unavailable when maintenance or some other event is causing a planned outage to a production system.

A number of requirements must be satisfied for partition mobility, including both systems being Power6 models and all I/O for the partition being hosted by a VIOS server. Live Partition Mobility can be a good solution to help reduce planned downtime by supporting the ability to dynamically move applications from one server to another. Additionally, Live Partition Mobility can help meet the need to respond to changing workloads and business requirements by moving workloads from heavily loaded servers to servers with spare capacity. Live Partition Mobility can also help meet the need to reduce energy consumption by allowing workloads to be consolidated during nonpeak times and servers to be turned off. For more information about Live Partition Mobility, see the IBM Redbook "IBM PowerVM Live Partition Mobility." [7]

Virtualized Availability Benefits
I hope you've gained some additional insights into the flexibility that virtualization can provide in both implementation of workloads and ensuring the high availability of those workloads. Virtualization can be a great aid in maximizing an enterprise's IT investment while reducing the resource expense. Properly implemented, PowerVM and virtualization capabilities can help ensure the availability of your virtualized workloads.

About the Author
Erwin Earley (erwin.earley@us.ibm.com [8]) is a managing consultant at IBM who has worked with the Rochester, Minnesota, development lab since 1996. Erwin currently heads up the Open Community Center of Competency in the IBM i Technology Center. He has worked in the IT industry since 1980 and has experience with several UNIX variants as well as Linux and IBM i.