Operation Bootstrap

Web Operations, Culture, Security & Startups.

Introducing Change Control to a Startup (Part 1)

| Comments

About 2 years ago now, I had the opportunity to put together my organizations first change control process. The company at the time was a very typical startup with many smart people hacking away at the system throughout the day. At the time, this was perceived as a competitive advantage for our company. We were “agile” and able to out-maneuver our larger competitors. This opinion was strengthened by the fact that we were in a new industry which was rapidly evolving and we needed to change quickly.

With that environment in mind, you can imagine that introducing change control to the organization didn’t exactly get folks excited about the impact this might have. This short series is intended to outline what I saw work and what didn’t work. Change control is ever-evolving and our process today is no different. Even though it is celebrating its 2nd birthday, it is changing on a regular basis to be optimized and meet changing business requirements.

First, a few simple rules and then in future posts I’ll chronicle the process that I saw our organization go through.

Be Flexible & Introduce it slowly If I could give only one piece of advice (Hopefully I get more of you attention than that), flexibility would be it. Do not expect to put the perfect process in place from the start and rigorously adhere to it. Eventually this will fail and more likely than not, it isn’t going to take very long to do so. New process introduced into a new organizations – especially startups – take a while to get everyone on the bus. Change control seems completely opposed to the objectives of most startups so gently introducing the process will go far better than alternative approaches.

Start simple Get something out there and do not wait for it to be perfect. The process is going to need to change anyways but changing the process is quick and relatively easy compared to changing peoples behavior. Get a head start on the behavior part and start with a simple process that at least gets people thinking about it. I started with a simple word document emailed to a ticket queue. We would take the document out of the queue – store it on a fileserver so that modifications could be made and approval could be added to the doc. People complained about the process – but nobody built anything better. Eventually the word form was recreated in a ticket system and that is now used. It’s better, yes, but wasn’t available when I started this. Regardless, I still got moving in changing peoples mindset.

Get support in advance of enforcement This probably seems obvious but it means more than just having an executive state “We need change control – go do it”. There are many stakeholders who are impacted by a change control process including QA, Engineering, Project Management and Operations to name a few in our organization. Within each of these organizations there are likely multiple individuals with a significant voice in the organization. Time needs to be spent reviewing the proposed process with them and getting their support. By doing this, you engage them in the process, insure that they have an opportunity to offer improvements, and make them a stakeholder in the success of the change control program.

Be an evangelist Don’t take this to mean you should be a zealot. You need to be the constant voice reminding everyone that change control should be thought about and embraced. Try to make up memorable principles for questions you get asked over and over. My favorite was alway “if you have to ask [then the answer is yes]” which I used anytime anyone asked me “Do I need to write up a change control for this?”. I had fun with it and if someone had an argument for why they shouldn’t write one up then we would talk about it; Sometimes I’d give in and sometimes I wouldn’t. Most of all, keep everyone on your side. There may be some very resistant individuals; leave them for the end when everyone else is conforming – don’t take them on when your fledgling policy could be shot out of the sky for being “too heavyweight” or “too invasive”.

Being the evangelist also means training people on the program. Create a training program, it doesn’t need to be terribly formal, where you walk through the process on a regular basis and get feedback!. Feedback and training are key to keeping yourself exposed to changes that need to happen to the process. You need to understand how everyone is using the program and interface with them regularly – put yourself in the line of fire when things aren’t working so that you can understand the problems.

Track metrics – understand the impact This is something I did a very poor job of. When it came time to explain to people why change control was valuable I could point to a few cases where having the documentation made it easier to identify and reverse a problem but I couldn’t quantify the overall impact on our service. Track the changes that are made and track the success of those changes. It’s even better if you have some of the metrics from before change control. Unfortunately, if an organization doesn’t have change control it probably isn’t tracking changes either which generally means outages are not always attributed to a change made. Tracking changes and attributing service problems to specific changes allows for analysis and improvement which is what the program is all about.

Promote change to the process where needed and do not give up You’re building a new process without many guidelines for what will work. Even if you select some standard that exists or something that worked at a previous company, this company will be different. Expect and embrace change. Everyone needs to understand that even though you put a provision in the policy which says they must sign their name in blood, if that process just isn’t working out for everyone it can be changed. Being open to change is critical to others being willing to work with your broken process. Believe me, at the beginning it’ll be broken (or at least that’s the feedback you will get) and will require some tuning, maybe even an overhaul, to get running smoothly.

Don’t give up. This is a hard thing to do; it’ll take years off your life. Unless your organization is begging for change control there will be resistance, whining, gnashing of teeth, etc. Do not give up. Take more walks during the day, take people to lunch, take people to breakfast, go have beer after work, whatever. You need people on your side for this because they can help you tremendously in shaping a successful process. Think of it as a great opportunity to build a stronger relationship with the other managers in the company.

That’s all for now… more to come.