PCI for Pen Testers

the besides DC 2017 videos are brought to you by threat quotient introducing the industry's first threat intelligence platform designed to enable threat operations and management and data tribe a new kind of startup studio Co building the next generation of commercial cyber security analytics and big data product companies well we're here today to talk about PCI and more specifically PCI for pentesters my name is Joseph Perini I am vice president of technical services for PSC the PSC is a qsa company which means we help organizations become compliant with the data security standard now I've run the pen test team for about nine years or so but I actually started as a qualified security adviser that means I

was an auditor and if I got to be honest I suck at being an auditor because frankly I get really bored sitting in a room eight hours a day people telling you that they're compliant and I my eyes kind of glaze over and I just don't trust it I like being a pen tester because I don't have to take people's word for whether or not they're compliant I get to check now my team does about 400 to 500 pen tests a year now we don't do all of the pen tests for our customers so over the last nine years that means I've been able to see quite a number of third-party pen tests and some of them been pretty good some

of them have been really bad but mostly it why is this flipping back and forth there we go but ultimately doing it this long I become really passionate about pentesting and PCI so much so that I co-sponsored the special interest group on penetration testing back in 2015 and I wrote great big sections of the guidance that came out so I hope I know what I'm talking about at this point so let's talk about some background here what is PCI all right back in April 1993 CERN went ahead and released the World Wide Web into the public domain that's when the internet was born now it's been said that two things drive the internet porn and

e-commerce and there's a lot of both now this ecommerce on the internet spawns a whole new avenue of fraud and the card brands are just getting hammered on a regular basis so in the fall of 1999 Visa says enough is enough and they create their first security program for merchants and it was called the the cardholder information security program or the C ISP so great all the other card brands go yes me too me too when they go out and they create their unique security programs for merchants visa says all right everybody you've got to be compliant in 2001 and that date comes and goes and nobody's compliant but why well think about it for a second you've

got if you're a merchant you've got five different security programs that you're trying to be compliant with and the card brands you know they don't necessarily get along together so you could be compliant for discover but that doesn't necessarily make you compliant say for Visa or for master cards so it's a real mess now if you're a hacker this is awesome because we've got all these new e-commerce sites out there and they're all running on iis with asp and there's sequel injection everywhere and you can't swing a dead cat in any direction without hitting a poorly protected e-commerce site now cyber source says back in 2000 that there will be one point five billion dollars in

online revenue lost to fraud and that number is gonna triple over the next decade so in 2004 the five card brands to get together and they say we're gonna create this thing called the payment card industry data security standard and they release version one now in 2006 due to feedback they create this thing called the payment card industry security data standard Council to help the standard and to help guide merchants become compliant so since then this council has given us eight different versions of the standard so what is the council supposed to do well the council is - supposed to oversee the standard now they don't do it by themselves they take input from participating

organizations which are really large merchants and service providers and they take input from the queue SAS and they take input from the card brands now they're also responsible for certifying the qualified security advisors making sure they know what they're doing when they go in the field to help merchants become compliant they also manage this thing called an ASV program or in an approved scanning vendor which you need to have scan you quarterly in order to be PCI compliant now counsel does a whole bunch of other stuff too they do an internal security Assessor program they do a payment application qsa program they do a PCI forensic investigator for when your security didn't work out the way you thought it

was going to they do a PCI professional Program pin transaction security point-to-point encryption they even have a program for qualified integrators and resellers the people who are responsible for installing payment applications in point of sales so they they cover quite a bit now who has to be compliant basically anybody that takes a credit card regardless of size has to be PCI compliant and you're gonna fall into one of two buckets right you're either gonna be a merchant or you're gonna be a service provider now a merchant is defined as anyone who takes a payment card with one of the with the logo of one of the five card brands great so if I have a swipe I've got a

cafe if I'm amazon.com I'm a merchant now the other group is a service provider and they're defined as a business entity who isn't a card brand but they're directly involved in the processing storage or transmission of cardholder data so these are companies that provide services that could impact the security of the cardholder data environment so a good example of that would be say Iron Mountain now Iron Mountain's got a bunch of tapes that they're taking back and forth and they contain card data so to some extent they need to be PCI compliant if I'm a managed services provider and I manage the firewalls for a merchant I have some responsibilities of being PCI compliant because I could impact the security of

that merchant if I handle the point-of-sale systems for a bunch of restaurants again I'm a service provider and I have to be PCI compliant now compliant with what well in total there are 416 individual requirements which are you know broken into twelve or basically six different control objectives and they are build and maintain a secure network protect cardholder data maintain a vulnerability management program implement strong access control measures regularly test and monitor networks that's interesting and then maintain an information security policy so which bits do we as pen testers care about so there are four requirements that get our attention the first one is requirement 6.5 which is to address common coding vulnerabilities well this basically

tells us that we need to use OAuth 10 and the cwe 25 when we are assessing web applications and then 6.6 requires that you review public-facing web applications and there are three ways that you can do this you can as a merchant you can either do a static source code review or you could use a vulnerability of web application vulnerability assessment tool or you could just skip your code altogether and you can just put a web application firewall in front of your web app and that will help you meet that requirement then requirement 11.2 is about vulnerability scanning and this is for both your internal and your external now anybody can do your internal vulnerability scanning but your external

has to be done by an approved scanning vendor now 11.3 the requirement for internal and external penetration testing can be done by anyone who's qualified so there isn't a certification for them so now we've got this PCI standard right we've got four hundred and sixteen requirements now everything's hunky-dory it came out in 2006 and we're good right no boom I mean the first one was TJX and it was you know multi-million dollar compromise and if ya look it's like every year since the PCI standard came out there's been some major hack what the hell this was supposed to fix this why are we still having this issue well merchants will tell you it's because the standards are ineffective

it's too hard to implement and it cost too much money and then security professionals will argue that it doesn't go far enough and it's all about checking the box ok great so who's at fault well I've given this some thought I've been doing this since PCI came out and I think I've got the answer if you think PCI is crap it's because you're doing it wrong I believe that were to blame so I believe the claim by some that PCI is nothing more than a bunch of kids with clipboards running around merchants checking off check boxes is probably pretty accurate for a lot of Q essays and pen testers aren't blameless either I've read some

tragically bad pen test reports I mean the report was pretty oh it was gorgeous it had great graphics it had you know professional fonts it had pie charts galore the whole thing screamed professional but when you read it it was nothing more than an esse skin in a dress so a PCI isn't easy and if merchants you know could figure this out on their own they wouldn't need us they wouldn't need Assessors I think some of the problem is that the security firms in pen testers individual in particular haven't really looked at the standard I mean looked at it dug into it and pulled some guidance out of it the other problem is being an Assessor of

any type is a really awkward position to be in so if you do your job too well your client hates your guts you'll hear things like oh he asked too many questions he digs too deep he's too hard on us great if you do a crappy job though your client gets breached and then everybody wants to know who the Assessor was because they did a crappy job pen testing is kind of the same raise your hands if you've ever done a great job on a pen test and somebody got fired yeah awkward so you know at the same time if you don't do a good job there are five other companies just waiting in the wings for you to screw up

and take your customer away from you so I honestly think that there to some degree there many companies are just afraid of rocking the boat for fear of losing a client and I think that needs to stop we need to as pen testers we've got to understand the standard and then developing an accurate and effective approach to performing our pen tests we also need to be willing to stand our ground and educate the customer when maybe we don't all agree on what the outcome of the pen test means so let's talk about some of the common challenges for both clients and for pen testers think the biggest one for clients is that this wasn't their idea I

mean think about it nobody wakes up in the morning and goes damn we should be PCI compliant let's go out and get a PCI pen test no they don't they have to they are required to do this stuff and I honestly think that affects their willingness to cooperate and engage the other thing is that PCI can get really expensive so you've got to get if you're a merchant or a service provider you've got to go out and get an Assessor then you've got to get your ASV scans and then you've got to get your pen tests internal and external web application and then if you've got a payment application you might have to have that

looked at to then they're gonna find things and those things are going to cost money you're going to have to upgrade software you might have to buy Hardware you're gonna have to implement certain security controls it's going to be inconvenient and it's gonna get expensive really quick the other problem is that pen testing has a real tough time demonstrating value and when a customer has a lot of demands for their budget I think pen testing is a great place for them to cut corners now the other thing is that the standard doesn't always scale it doesn't always work for smaller customers PCI there can be technical and financial barriers for smaller companies to go ahead and really embrace PCI in

some cases it takes a full-time person and if you all you've got is a system admin who's already wearing multiple hats then you are probably not going to do a really good job of PCI now what are some of the the challenges for pen testers well there aren't very clear instructions on how we're supposed to do this there are no cut-and-dry rules about how a pen test is supposed to be done so that's why we're here today I'm gonna call out where we can get that information and then there's no PCI certification for pen testers now for the quarterly external scans you have to get an ASV an approved scanning vendor and these companies go through an annual

baseline testing which means that the council can be assured of a relative level of consistency well there's no such consistency with pen testers so pen test companies really don't know what they're getting or whether or not the pen test company they hired is even qualified to do the job so and then lastly I think for us and I've said it before as I think we're afraid to push too hard a PCI really is supposed to have teeth if a merchant or a service provider is not compliant they're acquirer will start to levy fines and these fines can quickly add up and in some cases you can put a smaller company out of business well I don't think we

want to be the bad guys and like I said if we hold them accountable or were too good there's that fear that they're just gonna go find another company that's just easier and softer and we'll just go ahead and let them pass but again I think if we educate ourselves and we educate our customers and we can start looking for ways to bring value beyond just checking the box on a list of requirements we could make PCI real security so how are we supposed to pen test well there are four primary documents that we're going to get some information out of the PCI data security standard testing procedures has a testing procedures 139 page framework

for assessing merchants and inside of it it includes testing procedures and additional guidance around the specific requirements so we're going to want to look at that and then the 2017 penetration testing guidance I got to say I'm quite proud of this one because I spent you know over a year working with other security companies and a merchants and service providers and participating organizations and we took input from the card brands and you know we put together a document that was published in 2015 if you ever get a chance to a pci-sig I say do it just jump into it it's gonna take some time but you'll be amazed it's quite enlightening what it takes to get you know everybody with

who's got a different agenda and different ideas - all finally come together and agree on something it's a lot of work and it's pretty amazing now the ASV program guide covers pretty much the mechanics of being in a ASV but we're going to glean a few ideas out of that as well and then lastly there's this guidance for PCI DSS scoping and segmentation and this introduces the concept of a connected system so a connected system is anything that talks into the cardholder data environment that could impact the security if compromised that's a connected system so we're gonna look at those documents now let's dig into these a little bit and see what information we can glean from them so

from the testing procedures and guidance this is a good piece of information this is the key takeaway we are required to simulate a real-world attack situation and identify how far we can penetrate into the environment that's awesome because that kind of changes things and kind of sets the whole tone for how the pen test should be performed this does not mean we get to run a vulnerability scan slap our logo on it and call the color to pentest anymore this could be real fun but this document also tells us that we need to target the CDE and the connected systems cool we've now we know exactly what we're going for it gives us permission to exploit

identified vulnerabilities I don't know if you've ever had a pen test where the customer said no you can't do exploitation a post exploitation it's like well then what am I doing here this isn't a pen test this gives us permission to do that it also says though we need to use a defined accepted third-party methodology so you can't use Bob's House of pen testing as a way of doing it you've got to use something that's known the NIST 800 115 or the open-source security testing methodology use something like that your approach on that and then it says that we have to do this at least annually and after any significant change there's always a question of

what's a significant change and the answer is always well it depends a significant change is traditionally anything that could impact the security of the cardholder data environment so for example now let's say the customer changes a firewall rule is that a significant change probably not you could just you know run nmap again and make sure that all the firewall rules look or a nipper or something to just make sure the the firewall rules look okay or let's say you had cross-site scripting in a web application and you fixed that alright well we'll just check for that we don't have to test the whole lap again but maybe you add a whole new network or you

re architect your application and add new functionality those would be significant changes that should trigger another pen test and that will require for you to come out and again work with your customer so I've been talking about this thing cardholder data environment and CDE so we should probably define this and the council defines this as the people processes and technology that store process or transmit cardholder data which is the the pan the primary account numbers and or sensitive authentication data which in part is the three numbers on the back of your credit card so it includes this thing called people so that kind of means that we're not just attacking a static set of IPS

or applications this changes things now we have to go after the people who are part of the CDE and these are traditionally defined as the privileged users who have access to or a responsible for managing the cardholder data environment or they use in scope applications we'll think about it for a second we get to hack them so now that means maybe we can look at other stuff that they use in their day-to-day job their internal corporate domain now we're beginning to take a look at a bigger picture now if you just said okay all I'm going to do is hack the CDE you just point your tools at it you look at the perimeter you'd walk away

fine but now if we look at what are our CDE requirements are we've got these people that we can start to go after let's dig into the pentesting guidance so you would think that pen test companies would know the difference between a pen test and vulnerability scan but I have seen some reports that demonstrate quite the opposite now we originally did this for merchants because a lot of merchants really don't know the difference but it was good for some pen test companies too we also talked about in this document of black box versus grey box testing now I am quite the advocate of grey box testing I think it's a better value proposition for customers than going in blind a lot

of what we're going to do in a black box test is just look for you know your architecture your IPS we got to figure all that stuff out and we will get there but we will spend a couple of extra weeks doing it and we're gonna have to go through all your emails and we're gonna have to find your Visio diagrams tell you what we could save a lot of time and money if you just give it to us upfront and that's again better value for the customer this document also talks about what to look for in a penetration tester if you're a merchant trying to hire them or if you're a qsa looking at somebody's report trying to

figure out whether or not they knew what they were doing so we talked about some of the key certifications things like the OS CP we did not say that having a cissp makes you a pen tester so we're trying to offer some real guidance on it then we also added this consideration for social engineering and I wrote this myself because I'm I was quite insistent that we bring up social engineering one of the things I don't understand is that in the standard and in PCI we don't really talk about it but if you look at all the major breaches there has always been some element of social engineering even if all it was was a phishing attack

and why we don't test for it or educate or you know have that conversation about it blows my mind but this year I went ahead and proposed another sig specifically just on social engineering now it's not a sexy topic and that it's got a snowball's chance in hell of actually getting selected but I'm hoping it does because then if it does we can start to put together some sort of guidance about how to do this effectively without demoralizing your users with and getting some real benefit from it and then we also put together some report content guidelines because a lot of the reports that that we were all seeing or just all over the map you

couldn't really tell what the report was about and in some case you couldn't even tell at the end of it whether or not the customer was compliant so the ASV program the third document that we're gonna pull information out like I said before it's mostly about the mechanics of being in ASV but there's a couple of things that we can pull out of here that are important to us number one it tells the ASVs that they must use either NVD or CVS s for setting their severity levels cool because please do me a favor stop making up your own severity levels I know what you're trying to do I've seen the reports you're trying to stand

out you're trying to do something that's unique and you know put your stamp on it but really all you're doing is confusing people you're rarely more effective than the existing third-party standards and what happens is is you end up really messing up the remediation process because then the customer doesn't know what to prioritize they've got one document that says one thing and then they've got another document that says something else and they never quite match because you were coming up and calling it by different things let's stop we can use this to put together some sort of common severity language great then we're all on the same page again let's make it easier on the

customers and then the ASV program called out these things called automatic failures it's it's kind of cool so you automatically fail no argument we're not going to discuss it you fix it if we find out of date or obsolete operating systems in the CDE you run in Windows 2003 not unless you've got an extended Microsoft contract you got 2000 doesn't matter you're fixing it other things were if I find a database open from the internet guess what you automatically fail for God's sake go fix that now if you're using default credentials Tomcat Tomcat on anything it's an automatic fail and then then we talked about other things like web applications securities cross-site scripting sequel injection directory traversals these

were all automatic fails cool we didn't have to have any discussion negotiation or argument and then some other things that are kind of interesting were information disclosures that they consider an automatic fail if you reveal the internal IP address scheme through you know you're a web application call or something like that you expose it to the Internet that's an automatic fail you've got to go fix it for both error messages you got to fix them revealing source code you got to fix it so the ASB program kind of gave us some things that we could arm ourselves when we're doing a test with so that when we communicate to the customer it's very cut and dry and very clear

what they have to do we want that we want to make sure that the customers know what they have to do so that they're not confused all right the guidance for PCI DSS scoping and segmentation is another document that we can kind of pull some info over it I hear a lot of times why are you looking at that that's not in the scope for PCI r-ark USA didn't want to have anything to do with that like well you know what maybe maybe not just because it's not in scope doesn't mean we shouldn't maybe look at it so this guidance kind of gave me a little justification for that behavior it also talked about the connected systems that

need to be in scope the other thing it talks about too it's a good reminder is connections from third parties now let's say you're a service provider you provide a banking application and you've got a whole bunch of credit unions across the United States that connect into you well have you ever done a pen test from one of your customers you know like well no they're our customers we would never do a pen test from there well why not you don't know what that risk is you don't know what that connectivity looks like you don't know if somebody's screwed it up for example until you actually test it and this document says hey you know what you need

to consider it and the other thing it tells us is that we need to do segmentation testing so a cuss demur who has using segmentation to keep certain parts of their network separate from the CDE well we got a we got to test those segmentation then I'll talk more about that later in the presentation so there's always an argument around scoping the queue SAS argue it you go to the PCI council meetings you hear it between the council members and queue SAS merchants want to argue it and I've got the answer it's pretty simple it's not out of scope if it can be used against you let's face it I mean hackers do not have scoping rules why do we let

the merchants put limits on it we've seen organizations get nailed out of left field right we've target we don't know where it's coming from now if we were just focusing on the systems that the qsa has to look at we're gonna miss a whole bunch so if we increase our scope so that it's basically everything then we have a greater chance of protecting our customer now it does mean that you're gonna have to stop and educate your customer explain the risks and that's okay it's an extra step in our process but we're experts and clients do want us to guide them so let's do that now does this mean we get to go all medieval all over their networks no we

do have to act responsibly and reasonably and we do that using a set of guidelines called the rules of engagement we're all familiar with that right have you ever had a pen test with a really whacked out set of rules of engagement where I had a bank they told us if you find a password you can't reuse it because somebody might get offended okay well you couldn't do anything exploitive god forbid you look in somebody's email by the time we were done and we were like okay you know you could have done all of this yourself but we're happy to do this I mean we like your money but it's not really valuable for you so the best part of PCI is that

it sets the rules of engagement for us what we do is we go back to that original primary objective the intent of the penetration test is to simulate a real-world attack with the goal of identifying how far an attacker can it penetrate into the environment now we get to be exploitive we get to chain attacks together there isn't anything here that says well you can't do that no you can't do that it basically says we can do everything we get to do a damn good pen test for a change this is awesome now does this mean we get to do crazy stuff no how do we meet this intent and do it safely there are some common

things that we need to discuss first off don't bother with any denial of service attacks think about it if you knock over a server with a whole bunch of card data in it well that's probably going to be about as secure as its ever gonna be because if you can't get to the card data right it's good so there's that the other thing is you need to have a discussion up front with your customer about how you're gonna handle risky and fragile infrastructure so if you've got a customer let's say financial institutions a lot of these will have mainframes right if you scan a mainframe it locks up and then you've got this really uncomfortable conversation with

the stakeholders and your customer and you're probably not going to be invited back so get a hold of them first talk to them about this what do you have in the environment that could be affected ask them the question if you have you ever done a scan or a pen test before and something went south and then talk about that and decide how are we going to handle this we can't just not test it but we can come up with better ways maybe rather than doing automated tools we're gonna have to do it with some more manual stuff maybe if it's a web application what we could do is we could test it in staging so testing and

staging versus production can you do that sure why not but you need to have some criteria that you set with your customer first off the code base has to be the same can't be a version before it can't be a version after it's got to be dead-on and also make sure that whatever environment they give you a stable a lot of time that staging environment is like the old rickety box that's underneath somebody's desk make sure that it they're not changing it that it's not a dynamic environment that it's stable and the code is static and if you can do that then great you can go ahead and test in staging if you can't maybe you should

talk about when to test maybe during testing during off-peak hours is the way to do this I prefer testing during the day I prefer testing with realistic traffic you know real-world attacks happen at all hours they don't happen you know at two o'clock in the morning necessarily so I like it during the day because a couple of reasons if I have a question I can pick up the phone and somebody's going to answer if there's a problem there are resources that can immediately respond to it so this is good now I do have some customers that'll say well you've got to test it at night that's fine we'll test it at night but you've got a promise that you're gonna pick up

the phone and you're gonna answer me the call when I call because I've got a question or when you didn't give me something or something's not responding right so you kind of have to negotiate this with your customer to make sure that you get everybody on board and be careful about customers that do things like say well you can test but you can only test in a two hour maintenance window knock yourselves out guys if you want me to do this in a two hour maintenance window that I'm gonna have to double the amount of time that I'm gonna spend on this because this stop and start thing really screws with the efficiency and it's also increases the

likelihood that something you know bad could happen inside a system so usually I push back and shoot for at least six hour windows for testing depends upon the customer but most the time if you take the time to explain what the risks are you can get some agreement now when do you stop well that's the other cool thing about PCI is that it actually sets the success criteria pretty clear for us we knew what our targets were we had to go after the CDE and we have to go after connected systems well cool having an established success criteria prevents us from going too far and it also makes it easier for everybody to know when we're doing a

good job so maybe just compromising the perimeter of the CDE is success criteria and we're done great or maybe they've got this set up where they use a jump box ssh into an RSA key whatever to get into the CDE and if you can compromise that intermediary box you're good great that's the success criteria I've had some customers that have said hey look if you compromised our corporate domain we're good that's that's we're done we know it's just a matter of time at that point I love those customers those guys are engaged they know what their risks are or even something like access to source code it's gonna be different for every customer you just want to make sure that

you sit down and document it with them so that you don't go too far and if you do get into the CDE stop don't get cocky don't try and prove the point go well we're gonna get credit card data out well no you're not cuz things go south really quick and the next thing you know havoc alarms go off got to remember the CDE is a money-making vehicle for the customer you do not want to interfere with that because you do want to be invited back okay I'm not tell you how to do pen testing you know how to do pen testing but let's talk about some unique characteristics of PCI pen testing so for the external pen tests they're

actually probably some of the easier ones because they they're like anything else HIPPA a regular pen test and every web applicator every application pen test is the same they follow the typical pen test approach of discovery and numeration vulnerability scanning exploitation post exploitation there's there's no magic and pci for that one but be real clear that most companies have no idea what they have so asset management especially for some reason in PCI is going to be a real mess so what you can do is take whatever I piece or think targets they give you and compare that with the ASV scans because they should be the same and if they're not bring it up some one of them is wrong

and if it's wrong you want to make sure that you're making your customer or aware and that you come to some sort the conclusion the other thing is try to expand the scope as much as possible take what they've given you do some OSINT on it and then take that back to the customer go hey we found all this other stuff and asked the question this way don't ask them if it's in scope because they'll go no ask them if any of this stuff could impact the security of the cardholder data environment because that's key it's not about scope it's about impacting the CDE or getting in so try and poke at that a little bit if you can

do that with the externals then you're doing okay the internals are a little bit more complicated so the internals are definitely harder to scope and they're definitely harder to price because usually a lot of your scoping and pricing is just based on static IPS number of Web Apps and that's not how the internal pen tests are going to work because everything's in scope it's not out of scope if it can be used against you you need to have a pretty damn good understanding of that customers environment now ask them a question not just what do you sell but ask them how do you sell it so a lot of customers are going to have you know multiple channels

so they might have e-commerce that might have brick and mortar they might have a call center all of these places are gonna be where card data is going to be coming in so get an understanding of that up front before you scope it because you don't want to under scope you don't want to put too little resource and too little time on this and then when you're there remember you're attacking the privileged users yes you're gonna have some opportunity to go right at the cardholder data environment and there might be some instances where it's wide open and you're in those are boring those are 20 minute pen tests but as your customer matures it's going to

get harder now you're gonna have to go after those privileged users and look at non-traditional networks don't just thank Corp and CDE go after the voice go after climate control go after the guest networks guest networks rapidly become de facto Corp networks because everybody's too lazy to get on the the Corp Network printer networks I mean these things it's weird they don't get the same attention I know in a pen test before I've gone ahead plugged into avoidant Network and add access into this other system and then I could pivot through that right into the CDE so peek at it take a look at it don't trust the scope and I said that before in the externals and this is

kind of a theme customers don't know what they have they'll give you this list of IP addresses let's say they've got you know for class C's and a 10 dot network well do they have 4 class C's do they have 20 class C's you know it doesn't hurt to run math scan real quick across a slash 8 and say hey what are all these other networks that that came up you know pull all the machines out of the domain and convert those to IP addresses look at the the networks that they're on poke at this thing a little bit see if you can get more information out of the customer and finally automate as much as you can because you are

fighting time that is the downside and the limitation dependent of PCI pentesting it's all about efficiency and a trade-off of time so if you can automate and be more efficient you're gonna do a better job for your customer all right reporting no one likes doing the reports everybody loves doing the pen test the pen tests are cool but doing the reports suck and it shows in a lot of pen test reports that I read it really shows that nobody's into them but these are some of the things that you should have in the pen test report for PCI all right the executive summary we all know that a statement of scope this is important you

need to document this what was in scope what did you test where did you test from what was included and then you need to put in your statement of the methodology that's kind of required so that the qsa looks at it and goes ok they didn't they weren't winging this they didn't make this up and then your statement of limitations now if you had some limitations well we couldn't test during these time frames or you know this system we we had to do manually great put it in the document that way the customer knows that there might have been some limitations and they didn't get full coverage that's ok but document it and then testing narrative now this

is more me than PCI the way I want my pen testers to write the reports is I want them to do a chronological narrative with the exact tools and techniques and the syntax for what they did I've had customers come to me and go you know we really don't want to fix any of this because we think you're magic and we don't think normal people could do this and we don't think it's really can't happen there isn't really a risk okay you know I'm management I'm not that good if I can get in it's a problem so in our reports if we can put a testing narrative in there that is real clear and that so that even a junior

windows administrator could do what we do they can replicate our our results okay then we have made this real this is now something that isn't imaginary or theoretical this is a real attack that's good we've brought value we've demonstrated that this has some tangible value to the organization and then finally some segmentation results need to be in your pen test reports and then your findings need to be clearly documented let's talk about findings so in your findings what I'll and the first one is again me less more me less PCI but you're gonna run into situations where you're doing a PCI pen test and the internal corporate network is Swiss cheese they really have paid no

attention to it but they did a good job of keeping you out of the CDE so now what do you do how do you how do you write this up well I like to write it up just as normal just as if it were the same severity levels that for anything else but instead now we just indicate whether or not we could use it to get into the CDE and if we couldn't they don't have to fix it but we brought it to their attention and we brought it to their attention with the real severity level why is this important well now they can use our report to handle or to affect other regulatory regimes I mean

if you could have gotten into HR and Finance that's important they should know about that sort of thing and now you've created a report that they used for ci but now has value in other areas so that makes the penetration test report more valuable you need to put in the risk and severity and again not a risk in severity you make up on your own use something that's established I know that CVS s does not work for every finding but try and and do it as closely as possible to some third-party methodology put in the targets affected references if you can find blog postings about the vulnerability great do that put in your description another thing it's not

required but it's a good idea is put in this the steps the customer can take to test remediation this is going to help them you're gonna find remediation is is always a challenge and if you can show them how to test that they fixed it this will make retesting a lot easier okay a word about segmentation testing I promised I was going to discuss it segmentation testing is required by requirement 11.3 4 which says you must test all out of scope networks for all segmentation methods to make sure that they are effective cool well that can also be a lot of work well how are you supposed to do it well the penetration testing guidance says that you can use

the first two phases of a penetration test which is discovery and enumeration which is nmap right so you could run masks an or nmap from an out of scope network towards the CDE and that would satisfy segmentation testing the problem you may run into is that you've got customers with hundreds of out of scope networks the nice thing was is that this requirement came out at the exact time we were doing the SIG's so we were able to draw to everybody's attention that this is nuts you're gonna have people spending days doing segmentation testing costing hundreds of thousands of dollars it's not worth it so we came out and in the guidance we do have sampling is okay

what that sample size is I can't tell you needs to be reasonable it could be ten percent could be twenty five percent you'll you'll know your customer better it depends upon the the number of networks if it's you know 100 networks maybe you do segmentation testing from 10 or 15 if it's 10 networks maybe you do all of them you're just gonna have to kind of take a look at that on a case-by-case basis now something that's changing is service providers have a greater responsibility they have more cards and therefore they are required now to do segmentation testing every or twice a year now they may not necessarily ask you to do it and that's okay

teach your customer how to do it on their own because the requirement doesn't say that it has to be done by a pen test company or an ASP or anything it just needs to be done so you're not going to run back out there just to run a scan or nmap and you're not gonna send out an appliance because you can't move that from network to network to network teach them how to do it themselves teach them how to document it again you're bringing value to your customer and that's gonna get you invited back it's gonna let you do a better pen test okay issues on remediation follow me along with this one so let's say we hack a

client like this I've got they've got wpa2 I grab the handshake I pull it down I crack it and I jump on their network then I do L LM n R and n BN s with responder and I grab some more hashes and you know what summer 2017 I've got a I've got a credential fantastic so now I Kerberos their domain and grab their service provider named hashes and I throw those up in the GPU cracker and give it you know for six hours and hey I've got a domain admin cred cool so now I take that and I attack those privileged users and with that I get on their box and I wait for them

to connect to the CDE and then I pivot through and I've got my success criteria I present this to the customer and they come back to me and they say you know what we're just gonna fix that first thing we're just gonna fix the wpa2 we're not because the way we figure it is if you couldn't get that you wouldn't have been able to get all this other stuff all right well this is what I tell my customers pen testing is really about doing two things it's about finding initial vectors about finding escalation paths now let's go back to the the biggest limitation we have with PCI which is time frankly I may only be able to get one initial

vector during the time that I'm there and then I got to find the the escalation paths so yeah there's a chance I could have missed one or tomorrow there could be a new vulnerability which opens up another initial vector which means that if you only fix that one there's a chance somebody's going to come around find the other and all those escalation paths are still going to be there and you have accomplished really nothing in terms of securing your environment so we tell them you've got to remediate everything in the attack chain you can't just do that first one again we're pushing back a little bit on the customer but if you explain the risk you increase your value

so the other thing I get is you know what we've decided as an organization that we're going to accept the risk on that vulnerability no you're not there is no risk assessment or at risk acceptance in PCI the only people that get to accept the risk are the card brands because it's not your risk to accept its cardholder data and only the card brands get to decide now I've got a pretty good idea that they're not gonna let you get away with that so yeah no nice try the other thing is planned for the client to maybe not always come up with the best solution for remediation let's say you've you got in and they're

decided that they're gonna spend a million dollars implementing some sort of I don't nak solution to keep people from getting in and if you spent maybe if you've been on the phone with them for 15 minutes you could have pointed out that hey nak is not everything it's supposed to be and we can bypass that with this and you know save them some money so it's important don't just deliver the report and walk away you can do that with a lot of other pen tests because you know they don't have to fix anything with PCI stick around advise them tell them hey before you spend any time or money or effort in remediation let's sit down and talk about it let's

do a tabletop discussion because you know I'll bring a couple of my smart guys and they can probably you know tell us right up front whether or not your solution has a chance of being effective or if we've seen it before and we've already bypassed it we're gonna save you some time and some energy so retesting is important with PCI in fact it's one of the the few regulatory regimes that requires that you come back and make sure that notable exploited vulnerabilities have been corrected so there are two ways that you can do this you can either do direct observation you do the testing yourself you the customer has cross-site scripting and they fix it

and then you go in and you retest it well that's that's direct observation you've you've done it the other way you can do this is through review of documents where the customer sends you screenshots or you know output from something let's say they had LL M&R in the environment and you made them fix it well send me evidence that you applied the GPU across all the systems and send me a pcap file well I'll review that and that's sufficient so sometimes I don't need to go back out on site to do the test I can just rely on the customer coming in and doing it but what's important is that there is no sampling of evidence in PCI

so if you've got 60 boxes with eternal blue then you need to have evidence showing that all 60 boxes have been fixed you can't go and with 10 you've got to have everything in order to make sure that they are effective this regulatory regime could come back on the customer and you need to show that they've done everything that they could all right we were at 51 minutes so in summary it's this you know what the standard will drive your pentesting engagement if you look at it and pull out those pieces it actually gives you the information it gives you permission to do a really really good Bend test and exploitation is required this is

fantastic so take it Chane different vulnerabilities together go further and deeper into the network then you might be doing otherwise remember it's not us against them we're there to bring value we're there to make sure that they're secure we have this then it stops being a checkbox and we get a better job we also develop a relationship with the customer where we become a trusted partner and again don't be afraid to be the expert and being afraid or being the expert sometimes means you have to say no you have to push back you have to say this is how it is I'm the expert and then be able to tell them why pointing back to the PCI

documentation and because if you do this correctly I honestly believe that this could be one of the meant best pen tests that your client has ever had all right guys thank you it's nine o'clock you came in I much appreciate it if you have any questions you can get some answers on the PCI security standards council this is where you can find these four documents you can download them you can find me at japery nyet pay us w.com or I am je Perini on the twitters thank you oh yeah you got a question come talk to me oh sure why not [Music] okay no the whole is devices support ipv6 by default it's a table so it's a

PC I've still valid validated it's just another vulnerability man it's not there's nothing special about ipv6 if it's not being used and properly set up then what have we got we've got man in the middle attacks against ipv6 that we can use it's if it's not appropriately configured then it shouldn't be in the environment so you would put that down as a finding

well the testers taking some risk but you gotta you know you gotta ask did he miss it because he didn't look for it did he miss it because he didn't have time there are things that you miss in a pen test that is the reality of pen testing but it should be in your methodology to test it to look for it

yeah it's not again it's it's it's like saying you know is it patched or you know does a vulnerability exist in a system it should be something you're checking there's a lot of stuff in in you know ipv6 that heck we don't even have to it at this point do because there's all this other sort of stuff but the fact that it exists there and it's not configured properly is a vulnerability that should be listed in a report yeah does that satisfy your question yeah me too but it's up to us so you know again if we see issues like that we need to document it and educate the customer on it

again your your you're approaching ipv6 as though it's something special it's just another vulnerability so they don't have to call it out they don't call out eternal blue they don't call out art poisoning it's just another vulnerability if if the pen testers aren't looking for it well that's not the PCI counsels fault or responsibility make sense yep any other questions doing all right again thanks guys [Applause]