Tag Archives: consulting

An obituary to pentesting?

I just saw a blog post in which Mike Kemp discovers the realities of 2010 (linkedin). (disclaimer – I know Mike and love him as a person, and this is my way of poking at him a bit – no disrespect here, but pretty much the opposite)

Now, go read that post (yes, I know, it’s long, but trust me).
This isn’t new (albeit very honest, direct and true),but here are a couple of comments I have:

  1. Penetration Testing is dead. Overrated, and abused by fancy vulnerability scanning, it died a few years ago. If you are still paying for one – check carefully what you are actually getting…
  2. Automation is king. I actually argue that 80% of what’s sold as a pentest by the major providers can/should be automated. All those scanner monkeys should be fired or forced to step up their game and actually do some work.
  3. Compliance? Really? Do you really want to go there? It’s got nothing to do with security, and if you thought so for a second I want to have what you were on when you did.
  4. Standards. This is where Mike touches on a sensitive topic for me (yes, PTES…). I’d actually challenge Mike to show me how PTES (which he mentions in the post – but you already know that because you read it, right?!) restricts providers by providing the engagement steps – which they should follow. There’s no restriction to scope, and I have personally used PTES in red team engagements. Full scope, no bars held. But still with a standard to follow, and something the client can also keep track of and know what to expect (and demand).
  5. I fully agree on the “pass the wealth” point where you should call in someone else who’s an expert to deal with a specific client request. Done that many times, and have never lost a customer that way.

Last but not least – yes, I do think that most pentesters can be replaced with a script. As they should. I do however have a solid advice to Mike and others who are still valuable professionals that have skills which are not replaceable by automation: demand a proper engagement model. And yes – I’m referring to the PTES again. You’d notice that threat modeling is part of it. Done properly threat modeling achieves multiple goals:

  • Forces the discussion to be around security rather than compliance, price or other factors that have nothing to do with security.
  • Scope goes out the window as threat models focus on the BUSINESS and not the TECHNOLOGY.
  • Enables the organization to test itself against its adversaries (threat actors/communities) rather than against pentesters. Much more rewarding, and correct.
  • Enables the provider (if it can muster to perform a decent threat model with the client) to charge decent rates for its services. You can clearly show how this isn’t some automated software running and spitting out reports, but skills and experience playing. It’s then your responsibility to follow through on it and make sure the final deliverable also looks like that (otherwise you are looking at a very short success rate for trying to adopt only part of this approach).

I actually welcome the hordes of scanner monkeys and tool-jockeys. They make the real professionals look even better. And although professionals don’t often have the marketing/sales power of the big-[number], trust me – they are busy, and doing work that the “big” and “trusted” suppliers can’t even start to put on their canned proposal templates.

The realistic cost of a web application pen-test

So I was having some really interesting conversations over the last couple of days with some of the best people I know in the security industry (yeah, I’m looking at you guys…), and one topic came up on which we all agreed and shared mutual frustrations about: the ability to evaluate the quality of a “standard” web application pen-test (web app pen test was taken just as an example for one of the many security assessments and management services we all provide).

It started off from a benign question of how much would you say that an average engagement would cost? After making sure we all had about the same numbers in place, we started comparing notes on quotes that we have seen in the field – which ranged from the ridiculous low to the obscenely high. What makes these really absurd is that on both ends you usually find the lowest quality of deliverables.

For a lack of proper time to discuss this I’m going to leave this open and post a followup quick rant on what to actually look for from your security provider and how to gauge them in terms of efficiency, quality of deliverables and overall value.