Disaster Recovery Experts Speak Out (continued) 5

Re: Horror stories and lessons learned
by karen (07-Nov-01 6:01 PM GMT)

Our company had a disaster recovery plan, offsite storage, offsite hardware and software, and even offside copies of the plan. But the plan was incomplete. It didn't cover one disaster, which in light of Sept. 11, is one that all companies should have. Ours was a bit slower.

Our CEO was killed in an accident and the new management brought in a new manager over IT. He was smooth and dishonest, and in less than a year, our IT department went from 6 to 0. Our disaster recovery plan could handle losing software or hardware or even the building itself. But it required knowledgeable people. Years of software and data went out the window with the people, and they've essentially started over.

I was the second to last to go, by a week, and it still makes me sad.

Re: Horror stories and lessons learned
by Martin Garvey (08-Nov-01 0:50 AM GMT)

I'm very sorry about your situation. The voice at the top has to sing the song, or the best BC plans can go to waste. At least there is much more awareness of the benefits now, and your experience should serve you well during these times. I wish you the best.

Re: Horror stories and lessons learned
By Dwight Walker (07-Nov-01 9:27 PM GMT)

Same thing happened with a customer of mine. They had SirCam virus go through their LAN. One PC had vital Windows software files deleted and would not boot past a DOS prompt. A thief had stolen their Microsoft Office 97 CD and they had aging Pentiums. When I tried to reformat their hard drive and reinstall Windows, the old PC's CDROM drive was hard to detect. We bought StarOffice to replace MS Office and I spent some hours getting the cables right in the PC to get the CDROM to be detected. Moral: lockup your software CDs as well as your Windows CD so you can rebuild a PC.

Re: Horror stories and lessons learned
by Jason Buffington (08-Nov-01 3:37 AM GMT)

I once decided to wipe a server clean - only after ensuring that the administrator had a clean backup. Being thorough, I actually did a test restore of a file that was backed up the night before. So, I was good to go. I reformatted the server and after reconfiguring everything, restored the data - no problems!!!

BUT - the user account that was driving the backup software was not an Administrative account (translation: it couldn't "see" everything). So, the backups (for six months worth of stuff) didn't have any data from any user other than the LAN Admin's. Everything was gone...

Re: Horror stories and lessons learned
by Charalynn (08-Nov-01 4:31 PM GMT)

How do you solve the age old problem of being responsible for the IT development (i.e. database applications pushed to the desktop) but yet your back up and recovery responsibility exists with the Server and Network Admins?

Two weeks ago our development team lost an entire application source code and data stored on the server. No problem, we called our Server Admin and asked for a restore from the night before. Wrong!

Apparently their backups had been failing for over 2 months due to a memory problem. Although messaging is turned on and log files are checked, the error went undetected. Two months of application changes and two months of data are gone. I'm responsible for trying to explain to the end users and their management what went wrong. And, yes I'm trying to be politically correct.

So now what? How do I trust that the backups will work tomorrow. They are supposedly the experts on their side of the fence. I'm proposing a audit be done by someone completely outside of the organization and possibly going to a better backup software. When I asked to see their backup and recovery plan I got deer in the headlight eyes! I'm completely frustrated! As we all know the end users don't understand the difference between our roles and why should they! We together (SND Admins and DB Application programmers) should be working together.

I need a solution but don't have the keys to the kingdom!

Copy the Integer Hard Disk
By Michel Merlin (09-Nov-01 8:25 AM GMT)

I wittingly use the "integer" word: Using the excellent Second Copy (http://www.centered.com), I keep updated a copy of my whole Hard Disk on a second HD, generally of same brand, type and capacity. The first copy is long (7 GB in 6 partitions), but the next ones are just 1 to 5 minutes (depending on the amount of files changed).

I do this for all the PCs in my family; my own desktop has 3 racks installed, each other PC just needs to have its main HD in a rack/drawer combo. I keep my own HD in rack #1, my wife's HD in rack #2, her backup HD in rack #3, and I copy the whole HD.

To backup my own HD, I just swap my wife's and my HDs:
- now in rack #1 is my wife's HD;
- now in rack #2 is my own HD;
- now in rack #3 is my own backup HD.

(Moving a HD from a box to another is not a problem, as know all those having actually tried: first time, Windows will have a lot of "New Hardware found" - you just install them; next times, starting Windows shows no difference at all with your old good Windows.)

This way, I don't need anything else than the backup HD and a PC box to retrieve my goods: no software, nothing else.

In addition, the backup is very easy to test: I simply insert the backup HD in a box and start it.

The only (but big) drawback is that I have just one backup. What I would need is Second Copy should be able to make *differential* backups (not "incremental"). So on my backup HD I would have, say:

- an initial backup (say 7 GB, dated Sun 4 Nov, partitions C:, D:, E:, F:, G:, H:);

- 5 diff backups, in partition I:, dated Mon 5, Tue 6, Wed 7, Thu 8, Fri 9 Nov; so if I want to restore my PC in its state of Thu 8, I just copy the partitions C-H, then I use the software (needed in this case) to apply the differential backup of Thu 8 - no need of applying 4 incremental restores.

Paris, Fri 9 Nov 2001 09:25:05 +0100