You know the feeling — you've just left the grocery store and need to make one stop at the post office or gas station before heading home. You figure it won't take long, so the ice cream, deli meat, and gourmet cheese you just bought should be safe and sound in the trunk. But before you can reach your stop, your car starts to sputter. Now you're in a panic — not only will your car be out of commission, but you're also about to throw away about a hundred dollars worth of food. Terrible, right? Now, just imagine that it's your only duty to get that food where it's going, and instead of having a hundred dollars worth, you've got millions of dollars worth and your reputation on the line. Would you take the chance of your car breaking down? Didn't think so. And so, I bring you the story of Tony's Fine Foods and its quest to eliminate downtime in order to get that gourmet cheese to your table.

With more than 400 employees, the West Sacramento-based Tony's Fine Foods (see Figure 1 [2]) is a food distribution specialist serving the western U.S. since 1934. Tony's specializes in distributing fine-quality deli, bakery, food service, and perishable products. The company's four warehouses provide storage of more than 12,000 items in three different temperature zones, and state-of-the-art, center-divided climate controlled trailers allow Tony's to deliver an assortment of products on a single truck. Also, the company provides warehouse storage for other food distributors. In addition to facilitating Tony's Fine Foods' famous personal service, the company system supports online ordering used by customers, partners, and Tony's sales representatives. The company runs a production 520 and D/R 520 with V5R4. About 200 employees are connected to the server through client machines, and Tony's has an IT staff of eight hard-working folks.

Trucks move in and out of Tony's Fine Foods' warehouses nearly 24 hours a day, so you can imagine the consequences of a system outage. As the company grew, the IT team looked closely at the ability to recover from a critical failure. In addition to the main data center at the company headquarters, Tony's has three additional warehouses within California. The company needed to avoid downtime at the main center that could affect the other sites. "Whether it's keeping processes moving at our warehouses, having websites up for online orders, or dozens of other business requirements, we want our systems to be seamless from the perspective of our customers and partners and our own sales representatives," says Mark Geery, director of IT at Tony's.

The Pangs of HA Hunger

After attending an education seminar in 2006 at an IBM users group, members of the IT staff started to grow more concerned about high availability (HA). With curiosity piqued, the IT team started to dig a bit deeper, talking with vendors at conferences and researching the importance of HA online and in tech magazines. The team agreed: It was time for Tony's to safeguard its business with an HA solution. The team investigated several possible solutions and talked with existing users of each package. Major concerns were cost, support, company infrastructure, and customer satisfaction. After lining up a few potential products, the team watched realtime testing of the multiple options. This was a pivotal moment, says Geery: "Some of the other product's customers hadn't done live switch-testing. This gave us some apprehension at the time. We wanted a solution that was field tested and really worked. We didn't know how complex our configuration was compared to other companies. We felt more comfortable with the diverse experiences that we heard about from talking with Vision Solutions and its customers using the company's MIMIX product." So, the decision was made — Tony's would implement Vision's MIMIX ha as the foundation of its disaster recovery infrastructure.

MIMIX ha replicates a production environment to a backup server in realtime without data loss. This hot-standby server can be used for disaster recovery or to take over when a company needs to perform maintenance on the primary server. It supports any application environment and network topology, including intra-server replication, two- and three-tiered ERP environments, bi-directional replication, and cascading replication. The solution also includes a slew of helpful features to get the job done smoothly: IntelliStart is a first-of-its-kind HA innovation that plans, configures, and deploys the MIMIX ha solution. AutoGuard technology automatically seeks out and eliminates any issue that can affect HA integrity. Dynamic Apply intelligently determines whether to use a multi-threaded apply process for some data and a direct apply process for other data. CDP makes it possible for customers to quickly and easily recover data from any point in time. Lastly, MIMIX's Switch Assistant is an automated system based on HA best practices to ensure successful switches. The solution is offered in two versions: MIMIX ha1, for large companies, and ha Lite for SMBs.

Soup's On

Tony's acquired MIMIX directly from Vision Solutions and got to work right away. Some of the major milestones included synching the ERP software with the new system while running on two different OS levels, running the first switch test, switching over live to the new system, and running the first switch test back to the D/R system. With six IT members from Tony's and two from Vision, everything went very smoothly. Says Geery, "Vision Solutions' team helped us all along this process — consulting on the configuration of the upgrade system, OS upgrade questions, extensive library setup for Mimix synchronization, on-site switch testing and cutover, creation and ongoing changes to runbook, and ongoing switch tests." With excellent support, the team needed only two days of training to be ready to handle the new solution.

No modifications of Tony's existing applications were required for integration with MIMIX, and phase one of the rollout was complete in March 2007. This phase involved cutting over from the old 810 to the new 520 and upgrading the OS to V5R4. Tony's then created a second LPAR on the 520 and turned on MIMIX to synchronize between the two partitions. "We purchased another D/R system (a 520 in January 2008), and then installed MIMIX and got ha Lite running. We now are synchronizing to the second LPAR and H/R 520 with ha Lite. We have done two initial switch tests. The last step will be to physically move this D/R 520 to our collocation site," explains Geery.

Most of the problems encountered by Tony's were related to the way the host applications worked. A large number of temporary file members were created and deleted, which caused issues with the MIMIX audits by reporting false alarms. With help from Vision, Tony's has been able to completely eliminate this issue.

When asked about the greatest benefits of installing MIMIX ha, Geery answers, "For day-to-day operations, the ability to perform offline backups without affecting production will be extremely valuable. It will allow our warehouse operations to continue uninterrupted saves up to 30 minutes each night. The ability to upgrade from our 810 to the 520 in less than a day was also valuable. And of course, the peace of mind that comes with having contingency plans is priceless."

But Tony's isn't done reaping the benefits of MIMIX just yet: "We still are looking to take advantage of the second LPAR or D/R iSeries for our tape backups instead of using our production system. This will cut our downtime for day-end processing and help make our employees more productive. We also look forward to testing OS upgrades on a secondary partition before installing on our live systems. This will make these upgrades less challenging to schedule and help us avoid unneeded risks," says Geery.

A full HA plan with peace of mind on the side. Delicious.

Erin Bradford is an associate editor for System iNEWS.

Vendor Contact Information

Vision Solutions

800-957-4511

visionsolutions.com

MIMIX