Social engineering attacks as part of penetration tests are risky

Most penetration tests target systems, but social engineering attacks target people. This is an important difference: attacks on people feel personal in a way that attacks on systems do not. All too often, the individuals who fall for a social engineering attack feel terrible. We know that “human error” is a tremendously unsatisfying and unproductive root cause, but it’s hard for people not to internalize a social engineering attack as a personal failure.

Worse, only the most robust blameless cultures can withstand creating a situation where others blame the person who fell for an attack. I’ve seen employees who clicked on spearphishing emails be formally reprimanded, have privileges revoked, and in one case publicly chewed out at an all hands. How is that a useful outcome of a penetration test?

But even worse than that are the opportunities for unprofessional and downright inappropriate behavior when social engineering is allowed. Sadly, there’s a distinct subset of the security community with a complete lack of understanding of appropriate professional behavior.

Most of the time, this comes up in the form of pentesters engaging in behavior that I’d put somewhere in the grey area: compromising staff members’ personal devices or personal email accounts (as opposed to work accounts); breaking into office buildings to steal equipment or plant network monitoring devices; compromising social media accounts to perform recon; etc. Some people might consider these activities appropriate. I don’t, but I understand there’s some grey area here.

However, that’s the small stuff. I’ve seen much worse:

More than once, I’ve seen pentesters threaten support staff when softer methods don’t work. I know of a situation where a Red Team member broke an engineer’s car window to steal their laptop. I know of at least two situations where a pentester followed staff members home to attempt to compromise their home networks. In one of those cases, the pentester did this repeatedly (about a half-dozen times), to the point that the staffer thought she was being stalked and called the police. She later quit; the company’s lucky she didn’t sue.

I wish I could say that these situations were the exception, but I’ve seen this sort of thing happen repeatedly and consistently. In my experience, easily half of the penetration tests or Red Team exercises that have included social engineering components have seen some form of borderline or outright inappropriate behavior.

Even if I’ve got a sampling bias, and it’s more like 5 or 10% – is that really a risk you want to take? Especially when the results of social engineering in penetration tests are so useless?

Social engineering attacks don’t produce useful results

What’s the purpose of a penetration test?

If you said, “to see if a system can be compromised” – sorry, that’s incorrect. There’s no such thing as a totally secure system: any system can be compromised, given sufficient effort. No, the purpose of a penetration test is to generate remediation work. That is, penetration tests expose risks, which teams evaluate like they would a real-world failure, and that feeds into remediation work that improves the system’s security over time.

In order for this to effectively improve an organization’s security, the exposed risk needs to be something previously unknown, and the generated remediation work needs to effective and useful. But when we analyze social engineering attacks, all too often we produce entirely known results, and entirely useless “remediation”.

Take spearphishing. An employee bites on to a spearphishing attack. What does this reveal? That users can be effectively spearphished. In what world is this a surprising result?

And, what’s the remediation action after this attack? Usually, it’s “more phishing training”, as if that’s ever been shown to reduce the likelihood that and organization falls to a phishing attack.

These are just useless and uninteresting results. I don’t have to pay for a pentest to know that my organization is vulnerable to phishing attacks, because all organizations are vulnerable to phishing attacks. And if “employees can be phished” is the finding I’m trying to remediate, well, that’s not a risk that can really be effectively addressed.

Now: sometimes, really good postmortem/retrospective work on phishing attacks can find useful remediation work: ways to change authentication systems to be less vulnerable to phishing (e.g. U2F), techniques to remove access that certain types of users should have, better logging/monitoring/alerting, etc.

These are useful results, but there’s a better way to get to them.

Simulating a social engineering compromise

If we think carefully about the issues we’re trying to unearth with social engineering attacks, we realize it’s these types of systemic fixes that we’re after (and not: “can employees be tricked into clicking on emails?”). And there’s a better way of getting to these results without the downsides of social engineering attacks: simulations.

Here’s how it works: as part of the pentesting engagement, you pick someone on staff to be the “compromised” person. You keep this a secret, but you tell this person and get their permission. Now they’re on the inside, and there isn’t any risk of blame or inappropriate behavior. At some point during the exercise, this person hands their credentials over to the pentest team, and they proceed as if they’d compromised the staffer in question. If you want, you can have this staff member notify the team that they’ve been “compromised” at some pre-determined point in the future. For example, you could give your detection team 24 hours to detect the rogue activity, then have the staffer send a “I think I’ve been phished” email to the security team.

This approach is especially effective when it comes time to diagnose and do root-cause work. You’ll be unable to get lazy and just say “training”, because the event you’re diagnosing starts later in the attack chain. You’ll be forced to talk about detection, response, and systemic protection. When you treat the compromised account as a given, the next steps you generate are a lot more useful.

(As a bonus, this technique also simulates insider threats, and the remediation you’ll do helps protect against them, too.)

Don’t social engineer; simulate

So, the next time you have a penetration test, don’t allow social engineering attacks. Instead, use a simulation, and focus your remediation work on preventing, detecting, and responding to attacker’s activities after the account is compromised. This will produce better results, and not open the door to hurt feelings, bad outcomes, or inappropriate behavior.