Krispy Kreme Doughnuts

Critical Issue

The key to Krispy Kreme Doughnut’s brand building effort is their ability to serve hot, fresh doughnuts at convenient outlets any time of the day.

To ensure that customers get the freshest baked goods possible Krispy Kreme maintains a complex supply chain that is managed with sophisticated technology. 100 percent system reliability is required for these core systems. An interruption in system availability could have a seriously negative impact on Krispy Kreme.

Challenge

Krispy Kreme Doughnuts has an aggressive inventory control process that involves replacing products that are just a few hours old. If any of its computing processes fail, the delivery of fresh inventory could be disrupted.

Krispy Kreme manages its wholesale operation with an EDI solution from EXTOL that runs on an IBM i server. This system, along with remittance processing, accounts receivable, accounts payable, sales reporting and other applications serves 400 users.

In addition to supporting a demanding supply chain initiative, Krispy Kreme also relies on an electronic billing system to invoice its customers. This system can generate thousands of invoices each month for big retail outlets and delayed billings can seriously impacted cash flow.

Krispy Kreme sells its products in 293 store-factories, 90 satellite locations, and countless supermarkets throughout the US and in eight other countries. According to Greig Radford, manager of System i Technology, “We have trading partners sending information to our EDI system all the time and if it’s not available then we could have problems. This could cause a serious ripple in our supply chain that would hurt our wholesale operation.”

From its retail manufacturing facilities Krispy Kreme delivers to other outlets only what each one can sell within a four-hour shift, and then it follows up with fresh shipments a few hours later as demand requires.

Solution

Krispy Kreme needed to build additional resiliency into its data systems so problems of varying magnitudes would not interrupt their finely tuned manufacturing and distribution operation. With only twenty-eight people to manage it’s entire IT operation, Krispy Kreme needed to move forward with an additional measure of care. “We just don’t have the manpower that companies with big IT staffs have,” says Radford. “We didn’t have resources to put a person on an evaluation project for weeks or months on end.”

Radford decided to implement Assure iTERA HA from Precisely based on advice given to him by friends who were familiar with the product. Technicians at Precisely’ development and training facility in Salt Lake City, Utah, helped install the product on Krispy Kreme’s production IBM i server, and remotely on a backup system. “The installation procedure is very straightforward because all you really have to do is provide access to your system and they handle it for you,” says Radford.

Two technicians from Krispy Kreme then flew to Salt Lake City, for three days of training. Precisely takes an approach to training that gives Assure iTERA HA users a thorough understanding of the entire HA environment. Participants don’t spend days configuring a hypothetical training system, and then try to remember what they learned when they get back to their own systems. They configure their own machines remotely from Precisely’s training center, with the help of a knowledgeable mentor.

According to Radford, “Precisely’s approach to training is really forward-thinking. We have been able to manage this system ever since then with a minimal amount of effort.” Krispy Kreme’s backup machine resides in a separate building several miles away from the production computer. These systems are connected using 100 megabytes of a one-gigabyte Ethernet link.

At one point, Radford says, the data communications line between the primary and backup system was down for 12 hours. “We had some PTF issues that were completely unrelated to the high availability environment. When the line came back up we had over 12 million transactions to replicate— lots of save files and IFS-type objects. Surprisingly, it only took three hours to get caught up and we didn’t have a single error.”

Krispy Kreme has had Assure iTERA HA for only a short while but they are committed to continually testing their HA environment and reducing the amount of time it takes to swap roles from the production system to the backup. “This is just a matter of good housekeeping. Each time we execute a switchover test from the production system to the backup, it takes less time. I now know exactly how it works and I have a high level of confidence in Assure iTERA HA.”