Comedy. Tragedy. PEBCAK. Systems administrators know them all -- as evidenced by these tales from IT pros recognized on National SysAdmin Day.
Today, July 26, is National SysAdmin Appreciation Day. So take a moment and thank the unsung heroes at your company who manage software deployments, keep networks safe and speedy, handle bizarre help desk tickets, and politely remind you -- for the tenth time -- that you must be connected to the VPN to use that app.
To honor these "overlooked network ninjas," OpenDNS, the vendor that sells the Umbrella Web Security service, launched the 2013 OpenDNS SysAdmin Awards. Self-selected candidates submitted stories for awards such as the "Hand-Me-Down Hacker," for creating your own solution because the budget wouldn't cover buying one; "Flying Solo," for solving an IT crisis alone; and "PEBCAK" (problem exists between chair and keyboard), for the weirdest IT support request and how it was resolved.
The theme of this year's SysAdmin awards program is "resourcefulness under fire amid tight IT budgets" -- an all-too-common state of affairs. The contest includes tales of genuine creativity and courage, with some comedy thrown in. Here's a summary of some of the winning award submissions.
Doing More With Less Award: Erik Sheldon, with KSBJ, a Christian music radio station in Texas.
After being called upon by management to install Microsoft's Hyper-V without much working knowledge of the virtualization software and no training, Sheldon somehow made it all work. Here's his award entry:
Last year I was working on trying to put together a business case to take us from separate free ESXi VM servers to a VM Cluster for disaster recovery and high availability. Being a non-profit, any time we need to spend money it's a nightmare if it's not budgeted.
That being said, most of the purchases we have each year were not budgeted for because that's just how things go. Anyway, we went with a Hyper-V solution because in my old director's words, "It was the cheapest." Since Microsoft has amazing deals for non-profits, I get it, but we had never used the Hyper-V environment, and I was very comfortable in the VMWare world after using it for a few years at that point. Being in control of all I was (100+ older workstations that always have issues, 30+ servers at the time that were also old and had cranky issues but were moving to VMs to ease that pain, a network that was ever-evolving, a flat network to all of a sudden the need for 20+ VLANS per my director, a complete overhaul of our radio station's control rooms, boards and software), I was ready to snap if I had to handle one more thing.
Luckily, my director found some vendor who would give us a cheap deal on the setup, configure and training of the Hyper-V environment. "Cheap deal" meant that they wouldn't come every day, all day, until the project was done from start to finish. It meant they would start it, then come out when other clients didn't need them at other locations.
So fast-forward two weeks later. My boss is upset with how things are going with the project, and up and fires the consulting company without letting me know. When they don't show for a few days, I call them to get a timeframe and they tell me to speak with my boss, who then in turn tells me that he fired them "because they were too slow, and since I've been watching them, I should be able to take the project and run with it." He didn't care to find out that they were only about 30% done, that they hadn't done any training yet, and that all I saw while watching them was just trying to figure out what exactly I was looking at, since I had seen a SAN setup, a cluster setup or … oh yeah … a Hyper-V environment!
So, add the next week of working crazier hours than I normally do, an unhealthy amount of caffeine (quad-shot grande white mocha from Starbucks will put hair on anyone's chest), and every article I could find to help me, and I came away with a 90% completed Hyper-V cluster. The reason I say 90% is that over the next few months, I'd find little things here and there that I had to tweak, or that I wish I would have understood more to set it up better.
The experience was rough, but I know a ton more than I did, am able to work in Hyper-V or VMware, and have an amazing private cloud to slap on the resume. The best part of the whole thing was that my director at least listened to me on what hardware to purchase, even though we didn't have the funds budgeted for entirely. Over a year later, things are still working fantastically. The projects still come like that -- even more so now, though, since he and my new director have seen what I can do and want to get every last drop of sweat out of me.