Undo is great, right? You can do pretty much anything, hit Ctrl+Z and it’s restored to the way it was before. It would be nice if life had an undo button.
Until they come up with one, we have to create our own.
Mike Plunkett was my first professional mentor. He taught me a lot of things, but the most valuable thing I ever learned was to always leave myself an out. Before I was allowed to do anything, he would ask me, “if this doesn’t work, what are you going to do to undo what you’ve done?”
Most of the time the answer is to make a backup. Editing a text file? Make a copy. Upgrading some software? Back up your database. Getting ready to run a query against a database? Wrap it in a transaction.
It’s a good idea, right? It didn’t really hit home for me until one fateful day as a fledgling sysadmin.
I had a directory full of interface files. They were what allowed the system to dial up the modems in these little Okidata printers and print out the customer’s freshly merged credit report. I had the bright idea one afternoon of creating one big text file with all the smaller files in it. Y’know, as a backup.
The files either all started with the same string or had a common extension; I don’t remember. Let’s say they all ended with .modem. The first command seemed innocuous:
cat *.modem > all-my-modem-files.txt
Boom, I had a big text file with all my modem files in it. Wait, how do I tell where one ends and the next begins. Surely there’s something else I could do. I remember at this point I was a real big fan of using -> as a heading in my plain-text documentation.
What about putting “-> myfilename.modem” in the file just before the text itself?
***DO NOT RUN THIS***
for filename in *.modem; do echo -> $filename; cat $filename; done > all-my-modem-files.txt
***DO NOT RUN THIS***
The command ran silently (no news is good news in unix) and I was rather proud of myself. I opened the resulting file and was a perplexed when I was greeted by pages and pages of dashes, one per line. What the hell?
I listed the files in the directory:
-rw-r--r-- 1 xinu users 2 2010-06-16 00:28 atlanta.modem
-rw-r--r-- 1 xinu users 2 2010-06-16 00:28 boston.modem
-rw-r--r-- 1 xinu users 2 2010-06-16 00:28 lafayette.modem
-rw-r--r-- 1 xinu users 2 2010-06-16 00:28 miami.modem
Two bytes each? TWO BYTES? These files were easily 250Kb each and they weren’t all the same size. My heart sank into my stomach. I printed one of the files.
xinu@xv:~/tmp$ cat atlanta.modem
-
xinu@xv:~/tmp$
Time stopped. I looked at my for-loop carefully. Oh no. No no no no. I just overwrote every file in this directory (100s of them) with a single dash. The ‘-> $filename’ bit trashed them, albeit efficiently.
My phone lit up. Every branch office was getting errors back. They were getting calls from furious customers saying that they were no longer able to get their credit reports. I was never so scared in my entire life. I wish I could say that I took a deep breath, assessed the damage and worked it out. I floundered. I called the backup tape company in a panic. I worked for hours trying to find a good tape with the data I needed. It wasn’t pretty.
Every day since, I’ve never done anything unless I had thoroughly contemplated the consequences. That isn’t to say I’m a pessimist now, but I have a healthy respect for Murphy and his law. Have you given Murphy a professional nod lately?
