Pentesters and businessman are doing it wrong

Following my last post on the realistic cost of a pen-test (which as I mentioned was derived from long conversations on the topic with a couple of friends from the industry), I’d like to review one of the best presentations I have seen lately – Chris Nickerson’s Brucon talk.

I’ve had the opportunity to see this talk shape up to be what it ended up like in the week or so that we have been hanging out together. And let me tell you – it was one hell of a week. There were some reactions to the talk (no wonder – Chris was on stage) and I’d like to put things in perspective (at least mine, if you want more go talk to Chris…).

The first point which is directly derived from the talk is that we, as an industry, have been doing the wrong thing for a long time. Pentesting has become a glorified minion work, and we just kept it behind for such a long time. What the talk tries to say is open your mind, and DO YOUR HOMEWORK. Chris calls it “do work”, but I’m saying that before we do work we need to do homework. Learn. Inspect. Absorb. See beyond the technical aspects of a pentest. Understand what is the environment in which the business operates, who are the key players, partners and customers. How does the business make money? What would hurt the business the most? Only then, we can approach the pentest with a clear goal in mind (and no – it’s not getting root/shell on a box).

The second point that I’d like this talk to provoke is that we are not the only ones at fault. It’s also the customers (yeah – I said that the customer is wrong. Sue me). They have been trained to ask for technicalities. Be it a pentest, a product or even a service. Most of the times they can’t really explain the methodology behind what they are asking for and the business relevance of it. Instead of asking for a pentest for a new web application, they should be asking for a security assessment of what makes their business “tick” which may be related to the web application. Small difference in wording, HUGE difference in scope and ROI from such an engagement. And yes, this all comes back to us as we have been offering “off the shelf” pentests that have no actual relevance to the business side, and have “technofied” our services and products to fit checkboxes of some obscure regulatory compliance. We need to retrain our customers (i.e. the industry) and get ourselves trained on the business aspects as well.

This topic is just one of many more that were conceived during the security-on-steroids-week which was Source Barcelona and Brucon. I’d rather post these side-effect ideas that were generated from discussions around the talks than the actual talk contents (you should be able to download these anyway in the near future from the conference websites anyway).

6 thoughts on “Pentesters and businessman are doing it wrong

  1. I’ve spoken to testing companies who will only quote on a per-server basis as all the competition does the same and if they went in offering this new style of testing the client would reject it straight away as there is no way for them to compare a holistic business test to a test of 50 servers and 100 desktops.

    As most businesses have had it drilled in to them by compliance and other sources that they need server pen-tests they aren’t going to bother trying to do the comparison and will just ignore the business test.

    I think it will take a lot of user re-education and until that happens I think a lot of firms aren’t going to want to take the risk of quoting for business tests because they know they will lose and in the current economic climate they can’t afford to lose.

    We managed to get companies to start asking for pen-tests in the first place so it is possible to educate them we just need to find a way to do the re-education in a way that won’t harm our businesses while we are doing it.

    1. I’m %100 with you on how the market looks like NOW. I do believe though that WHEN we start improving our reports to include the business impact of what we do, we can continue the customer education and have them start to demand business impact work as well.

      This also brings up another question – who in the company is issuing the request for the pentest? If it’s the IT/technical guys than you can’t really expect much. Surprisingly enough, from my experience once you start working with Audit/CFO/Business risk management you can more easily get to the right discussions.

      We have our work cut out for us that’s for sure!

  2. the re-education is going to be the hard part.
    i didnt attend the conference, but i did sit with chris in a classroom for a few days, so i can pretty well educatedly extrapolate the contents of the talk. this is a reflection of his philosophy on the more holistic approach to what we do, in that there should be more to it than reconaissance, exploit, pivot, exploit some more, write report, profit. actually being able to expand into the larger non-technical domain of the enterprise takes a greater amount of creativity, savvy and knowledge about the world in general than a lot of folks may realize. the intellectual property of an enterprise are its crown jewels. technical shortcomings included, failure to properly assess access to the crown jewels by any additional method is a largely unaddressed problem.

    its just way too easy to just simply fire up the framework, point it, and fire away.
    its an easy result to ask for and resultingly provide.

Leave a Reply