I remember my first year in my first real IT job. I was the network guy (mostly) and so I spent most of my days cleaning up cables, replacing equipment, building VPN’s, etc. I moved into System Administration work and started migrating old Solaris systems to FreeBSD. We had 50 or so servers and plenty of time, so things were manual but worked well. After 5 years of that things ran like a top, and I got bored.

I remember my first year in a real SaaS startup. It was crazy. I was writing scripts to fix problems and cramming them into systems as quick as I could – whipping up SQL in my sleep, integrating with partners when I wasn’t try to get some new piece of infrastructure in place, and losing a lot of sleep. Things were manual and it didn’t work too well. After 2.5 years of that I was done – time to do something else – I needed a serious change of pace.

Today I’m somewhere between those two, which means I’m busy as hell but I have hope because things are getting better. Mostly things are getting better because I have better tools at my disposal, and the freedom to use them to make work easier for me and my co-workers. Those tools, however, take time to put in place and that time is usually borrowed (or stolen) from time budgeted to short-term fixes. You put off a short-term fix so that you can spend the time and put in place a long-term fix. It may not be the perfect, end all fix, but it’s a lot better than the short-term fix. And most importantly – it frees your time – it MUST do this – so that you can invest that time in further long-term improvement.

Then I see folks still being me at that SaaS startup – fighting the endless wave of problem after problem, the type of problems that only get solved faster by typing and thinking faster. These problems lead to other problems because people are people and people make mistakes. These problems never go away if you never stop and think about how to make them go away. Most importantly, as a wise man once said “We cannot solve our problems with the same thinking we used when we created them.”

If you are that person who maintains the status quo – who wants to just keep things “simple” and not push for change – you are about to become a Dinosaur. Maybe not overnight, because there are plenty of folks who honestly don’t get it, they don’t understand that we’re getting to the point where the role of a sysadmin isn’t to sit and fight problems all day – it’s to understand the system and program an automation framework to do that old job that we all know so well. It’s to work with Engineering to design systems that don’t fail in the way those old systems did, and to build in monitoring and response systems to let you get a good nights sleep. The System Administrator has become the Engineer – but not building features customers see – building features you hope the Customer never notices because it all “just works”.

All those servers in corporate computer rooms are being replaced by services in datacenters. Those computers are run on a scale that is similar to large grids that have been around for a while, but with an exponentially faster rate of environment change. Daily code pushes, new machine builds, unstable software that requires constant tuning to keep up. With those important issues to deal with on a scale of hundreds or thousands of machines, you have to be able to focus on what matters.

So, go pick up a book on ruby. Deploy puppet, or chef, or cfengine if you are a masochist, and start to understand the power of these tools. Start focusing on how you can write code to run your systems instead of finding tools to make your commands run faster on many machines. This is the skill of a growing number of todays Sysadmins, so get on the bus.