Who do you educate about how to make good security decisions? How do you educate them? What is your goal in educating them? How often do you think about these questions?

I’ve stepped foot into two organizations now who were start-ups or late stage start-ups with some pretty significant security problems. The amusing part? The smaller of the two, the one with fewer resources, the one with fewer folks making all the security decisions, the one with really less at stake in a security breach – is magnitudes more secure than the other.

In both of these organizations most of this structure was established by individuals. The decisions about how to architect your perimeter, your internal restrictions, your process & policies, these things are built by a few overworked system or network admins. Were these people making these decisions because management insisted that things be secure? Absolutely not – I guarantee it. In the later case in fact, many people see the current architecture as too restrictive to allow business to function, too difficult to navigate… a business that can’t operate as fast as it would like because it has to work around security measures. They have found ways to manage though – things work fine for the most part.

Lots of you are reading this thinking the later company is in better shape and in general I would agree with you. There are a few cultural and psychological advantages to making things “too secure” to start. First, have you ever tried to convince an organization without any security standards that they need to enact standards that will require resources from people to change things, require people change the way they do things and will initially be observed to slow things down? If you have, you know what a pain in the ass this is. Sure – there are plenty of techniques to produce evidence to help in this process but at the end of the day the people making those decisions need to understand the benefit and to see the benefit they need to understand that where they stand now isn’t ideal. Second, it’s easier to convince someone to keep something restrictive than it is to convince them to make it more restrictive.

Now, take the new organization with a sysadmin who sets things up in a restrictive way from the start. Lets just pretend they’re overly restrictive & difficult to work with. Yeah, we’re pretending because that never happens in the real world. So, in this organization, after 2-3 years of operation, people get used to the fact that they can’t get to the Internet from certain machines, that they don’t have direct access to systems in the datacenter without going through a bastion host, that if they go plug their computer into the Accounting network, that they don’t have access to the Datacenter at all. Someone new comes along and asks “why is this so hard? Why all these restrictions?” and the answer is “It’s always been that way”. Now, some people will challenge this, but most people will not. Most people will accept that there’s some security benefit to it, and that at some point someone made the decision based on analysis and discussion and all the things you expect. The reality is, one person said “I’m doing it this way” and provided everyone else with education about how to work with that.

Where am I going with this? In those early stages, when those decisions are being made, whether it be in a new start up or a large organization starting a new project, many of those decisions are made by individuals implementing something. These individuals aren’t on the security team – if you are lucky enough to have one. The decisions might be based on policies, if you are lucky enough to have good ones that people know about and understand. Most of the decisions are made by a small group doing their best to get it done and who have varied experience and interpretation when it comes to security.

You are starting to see more focus on this from organizations like SANS offering “Security Architecture for Systems Administrators” and other courses geared toward helping individuals make these decisions better. Believe me, system administrators want to make things more secure, they want to make things better, but very often they have learned what being secure means from information handed down from another admin, a mentor, or otherwise. This leads to decisions like implementing SSL on all your internal sites with self-signed, unverifiable certificates. Changing passwords every 30 days but allowing non-complex passwords & using “Password1” as your new employee password. Requiring the use of ssh public keys, but allowing passwordless private keys to be stored on hosts in the datacenter.

I think it’s important to educate security experts who focus on security but I don’t think enough effort is being made to educate the individuals making these decisions. The SANS mentor and @Work programs are good steps toward making this easier for organizations but the community as well needs to work on this. I think rather than talk about it any more I’ll try to do something about it – if you have suggestions, let me know.

If you are a System Administrator, consider getting some formal security training – go hang out with your local security groups, go to Defcon and meet some folks. You gain an amazing amount of perspective just from being around people with the correct mindset and that mindset helps you more in making decisions than any amount of training can do.