owards the end of an otherwise typical business day in September, Kurt Sundling, Technical Operations Manager for Monaco Coach Corporation, got word that the Monaco staff’s attempts to access Help Desk data were coming up empty. It seems the 1GB help desk database had disappeared. Sundling and his team immediately investigated, working into the night to locate the missing data.

They researched the server on which the database resided to confirm that the data really was gone. The transaction log revealed some encouraging news: the data was not truncated?so the records were still in tact wherever they were. During a check of the backup tapes, they discovered something more alarming: the tapes had not been removed since March 2000 when the database administrator was replaced. All the data gathered since then had not been backed up.

The 1GB help desk database had disappeared.

Monaco Coach, which produces more than 10,000 vehicles each year and sells them through more than 350 dealers worldwide, has 1,000 employees who rely on the help desk for rapid responses to their hardware and software queries. Without it, not only would they be unable to resolve their own internal IT issues but they also face more than 100 daily customer calls with no IT support. With a two-domain network of 40+ dual-processor PCs, 50+ servers, and Monaco’s own ERP software to support, Sundling was looking at a crisis if the lost data wasn’t quickly restored.

Sundling was looking at a crisis if the lost data wasn’t quickly restored.

Luckily for Sundling and Monaco, Lumigent Technologies had a solution. Sundling acquired Lumigent’s Log Explorer data recovery software, and within four hours he had it tested and installed and had recovered all the lost data. Log Explorer used Monaco Coach’s Microsoft SQL Server transaction log to audit its database activity and selectively recover the data.

Sundling concluded that the lost data was the result of an administrative error, not a system error. So the story has a happy ending, but Sundling realizes Monaco Coach dodged a bullet on two fronts: the missing data disappeared from a database that serves internal Monaco staff rather than customers and it vanished at the close of the day rather than before or during peak hours. He learned a very important lesson that day, however: preparedness is crucial?have a plan.

Since the incident, he has been trying to develop a disaster recovery or business continuance plan for his staff while still fulfilling his normal duties. Sundling believes that because IT managers and developers are the ones who take the responsibility when servers go down, data vanishes, or some other technical catastrophe occurs, it’s their responsibility to establish disaster recovery plans for their systems. However, he acknowledges that he can’t establish a plan by himself. He needs support from the CIO and the other business units in the company. The IT manager must accomplish the tough task of convincing upper management that the time and money necessary to create and implement a plan, and then train the rest of the IT staff, as well as maintain the plan, is a viable and critical business venture.

If [a disaster recovery plan] is not number one [on the priority list], it should be number two at least.

When considering the proper ranking for a disaster recovery plan on an IT department’s priority list, Sundling said, “if it’s not number one, it should be number two at least.”

Sundling summed up the goal of a disaster recovery plan. “As a department, we’re reactive rather than proactive,” said Sundling. “When you have a backup strategy in place you can take more of a proactive stance.” According to Sundling, a backup strategy answers several questions in the face of crisis: Who does what? Which data goes where? What’s important to backup? Can the lost data be recovered? An IT team is much better off knowing the answers to these questions before a crisis forces it to answers them unprepared.