Tag Archives: CISO

The Ian Amit Spectrum of Pentesting Efficacy

It’s been a while since I posted (duh), but recently I’ve had something brewing in my mind that appeared to not have been clearly discussed before, so here goes.

I’ve been seeing some discussions and ambiguity around pentesting, vulnerability assessment, and red teaming (again – no huge shocker for those of us in the industry). However, as much as the “look at our shiny new red team” marketing BS coming from big companies (read as “we do pentesting, we have a new name for it so you can pay more”), pisses me off, what bugged me even more is the lack of clarity as to where and when pentesting can/should be used, and through which means.

I offer you this – my simplified spectrum of pentesting efficacy.

In short, here’s how this works: first identify the actual need for the test. There should be three categories as follows:

  1. Testing because you have to (i.e. compliance). PCI is a good example here. It’s something you can’t avoid, and doesn’t really provide any real value to you (because of the way it is structured, and as we all know, compliance/regulation has nothing to do with security so you might as well ignore it.
  2. Testing because you want to make sure that your controls are effective, and that your applications are built properly. This is where the “meat” of your pentesting should come into play. This is where you see direct value in identifying gaps and fixing them to adjust your risk exposure and tolerance (based on your threat model and risk management, which you should have, or if you don’t, just go ahead and quit your job).
  3. Testing to see how you fare up against an actual adversary. Bucket 2 above was fairly technical in its scope and nature. This bucket is threat focused. More specifically – threat actor focused. Your risk management strategy should identify the adversaries you are concerned about, and here you want to see how you fare up against them in a “live fire” scenario.

Once you have these, you are almost done. Here’s how you approach “solving” for each category:

  1. Compliance: find the cheapest vendor that’ll check off the box for you. Don’t be tempted for cheap marketing tricks (we red team, we thoroughly test your controls, we bring in the heavy hitters who have spoken at BlackHat and DEFCON and the super-underground conferences). Remember – you are getting no security value from here, so shop around and see who will tick the box for you. Remember to be heavy handed on the reporting and remediation, as if you are doing things correctly, the scope should be very minimal (remember – just enough to cover compliance requirements) and you should easily have these covered as part of your standard security practice.
    Also – no point of putting any internal resources into here since it won’t challenge them and is menial work that should be outsourced.
  2. Controls and Applications: this is where you should invest in your own resources. Send your security engineers to training. Build up an SDLC that includes security champions and involves the dev teams (DevSecOps anyone?). This is where you should see the most value out of your investment and where your own resources are better equipped to operate as they are more familiar with your operating environment, threats, and overall risk prioritization. In this category you’ll also sometimes include testing of 3rd parties – from supply chain to potential M&As. Use your discretion in choosing whether to execute internally or engage with a trusted pentest company (make sure you utilize the Penetration Testing Execution Standard when you do).
  3. Adversarial Simulation: This is where you shift from pentesting to red teaming. The distinction is clear: pentesting focuses on one domain (technical), and sometimes claims to be a red team when phishing is involved (social). A red team engagement covers three (technical, social, physical), and more importantly, the convergence of two or three domains (social/physical, technical/social, technical/physical, or all three). This is where you should engage with a company that can actually deliver on a red team (again – use the PTES sparingly in helping you scope and set expectations for the engagement), and can work with you to identify what kind of adversary they should be simulating, how open or closed does the intelligence gathering works, how engaged will they get with targets, and to which degree are you ok with potential disruptions. This is where you’ll see value in affirming your security maturity across several domains, and how these fare up against your threat communities. This will also enable you to more clearly align your investments in reducing exposure and increasing your controls, monitoring, and mitigations for actual loss scenarios.

I did mention vulnerability assessments initially, and if you made it through here you noticed it’s not on the spectrum. Well, it kind of is – it should be part of all of the engagement types, and is not considered an independent member of an engagement. Hint – never outsource this. It’s extremely cheap to engage in VA by yourself, and there are plenty of tools/programs/services that are highly effective in continuously performing VA for you that range from free to very cheap. Don’t be tempted to pay any premium for it.

That’s all for now – hope this made some sense, and helped you in prioritizing and understanding how pentesting (and red teaming) are best applied through your security program!

So, There’s this new (for me) LinkedIn “publishing” thing, that prompted me to try it as I was posting a semi-rant there.

Let’s see how well that works out:

https://www.linkedin.com/today/post/article/20140531211959-1510435-security-and-maturity-beating-the-averages?trk=prof-post

Ambulance chasing or DNA research?

I am fortunate enough that some of the new topics that I have discusd lately have generated interest in the community and the industry. As such, there are obviously  voices that do not agree with the approach (I still like to call is SexyDefense, although the more adult part of me agreed to SDES – Strategic Defense Execution Standard).

More pointedly – there is the argument of “what would an offensive player know about defense”, and “defense is hard, we’ve done it [for our customers] forever, and people are fairly happy”. I’d like to tackle these two head on:
Yes, I’m mostly an offensive security person. I cannot deny my passion for Red-Teaming (heck – I’m good at it, and I enjoy it. Deal with it), nor my past research on finding issues with systems and organizations. Nevertheless, as we all know – practicing offensive security is done in order to boost defenses. Its main role is to find flaws in the defensive mechanisms and then amend them. Here comes the tricky part – amending them is also something that I do. I know, a shocker! But fortunately I’ve had a chance to work not only with small businesses, enterprises, and F-100 companies, but also with nation states and multi-national organizations… So yes – I know how hard defense is, and I have also practiced it and can say that I actually enjoyed it – especially since I was able to “sign off” of some great improvements in the defensive posture of such organizations. Last but not least – guess what happens after a Red-Team engagement is over? Right – a long, hard look at the systematic failures and vulnerabilities of the organization. And how to fix them, and how to prepare for another attack such as the one that the res-team simulated. (reality reminder – red-team is essentially adversarial modeling – probably the only true test of how an ACTUAL attack is going to look like on your organization. And guess what? It doesn’t look like a Nessus scan or a Metasploit autopwn…).

Second – yes, defense is hard. And this “newfangled” approach is something that has not only been tested in the real world, but it also makes sense <gasp>. Our old approaches of detection and “prevention”, using the same old tools (spell Anti-Virus, Firewall, Intrusion Detection/Prevention, DLP, and what-not) are not working. Let me say that again:

It’s not working!

Why? Simply – we keep chasing our tails with the same old issues. We are really good at Incident Response (some of us are making a nice chunk of money off it), but we really suck at actually improving the security posture over time. Hence my reference to ambulance chasing (i.e. incident response), vs. DNA research (actually changing the defensive strategy and posture to cut the number of incidents).

Personally – I have enjoyed some really tricky incident response engagements that challenged me and my customers (and sometimes led to the satisfying “gotcha” moment when coordinated with LE). Nevertheless, organizations do not really learn from such incidents. They have a short memory span, and get back to their old “look at the blinky lights on the firewall appliance” approach. However, changing the DNA is something waaay more interesting and rewarding. And that’s what we are trying to do here folks…

So – are you going to stay an ambulance chaser and keep rejecting the idea that your revenue stream may be affected if organizations take defensive security more seriously, or are you going to help the change and actually make an impact?

Security Awareness and Security Context – Aitel and Krypt3ia are both wrong?

It was pretty obvious that after an Information Security persona such as Dave Aitel has posted his “Why you shouldn’t train employees for security awareness” article, there would be a lot of flak from the industry. A lot has been said about training employees to be somewhat more savvy users when dealing with corporate equipment and data (i.e. “stop clicking shit”). And even one of my favorite and outspoken Information Security personal had a great rebuttal on the matter – Krypt3ia’s “Throwing out the baby with the bathwater: Dave Aitel’s approach to INFOSEC“.

While I really appreciate both opinions, and while Dave’s might have been a little self-serving (aren’t all of our statements online?), I find myself in a very “Zen” place – saying, yes – you are both right, and wrong at the same time.

Krypt3ia points out that dismissing the human factor is going to lead to failures beyond what we can imagine as an industry. The reason here lies back in the fact that when we approach “Information Security” we focus too much on the “Information” part, and less on the more holistic meaning of the “Security” part. Trying to solve infosec issues through technological means is a guaranteed recipe for failure. No one, no technology, or software can account for every threat scenario possible, and this is exactly why we layer our defenses. And layering shouldn’t just be done from a network or software perspective – security layers also include access control, monitoring, tracking, analysis, and yes – human awareness. Without the human factor you are doomed. And that’s a personal promise from someone who’s been abusing the lack of layering and dismissal of such human factor for quite some time now running red-team engagements with high-profile, high-security clients (see – I can be self-serving too!).

On the other hand, Dave is also right – you can’t just throw everything on the employee and expect them to magically turn into “APT detectors” just because they clicked through some CBT program for a few minutes (or hours for that matter). You have to get the basics first, and Dave’s list is just as good as anyone else’s:

  • Audit periphery
  • Perimeter defense and monitoring
  • Isolate & protect critical data
  • Network segmentation
  • Access creep
  • Incident response
  • Strong security leadership

In no particular order, one should establish a consistent and solid implementation of all of these aspects for their organization.

Having said that, saying that employee awareness should be out of this list is where Dave went a little too far. Strong security leadership, access creep, and data protection are not technical feats by themselves. These are exactly the areas where employee awareness turns what could be useless (but very expensive) pieces of software or appliances to something that would actually work under an attack on your information assets. The point is not to _divert_ the spending on awareness, but to _combine_ them into your security strategy.

Which brings me back to my first (and only) point – stop thinking of information security as an industry of blinkenlights and snazzy software solutions. It’s about hacking, and hacking as we all know never stops at gadgets and code. Think of information security like an ATTACKER. Think about _their_ scope, and realize how your organization looks from that perspective. Now, take your budget and spend it on the areas where attackers could have compromised your informational integrity (HEY! Don’t touch that Nessus scan result! I told you to THINK goddamnit!).

And with that, I’ll leave you to your wonderful weekend before Vegas (one last self-serving statement – go check out “Sexy Defense” if you are really interested in an effective defensive strategy that goes beyond blogging and writing articles 🙂 ).

Happy hacking!