Pentest Fails — Revisited

hit every b-sides ever including both virtual ones [Applause] and today he's speaking twice so big thanks to adam uh enjoy he's gonna you're gonna talk about pen yeah sounds right from iot fails to pentest fails hey that works all right everybody hear me okay all right so sorry no slide deck don't have one prepared for this um actually i'm in the process of updating it for upcoming conferences so you might get a little bit of a verbal preview of it here so i don't know if any of y'all have caught any of my previous talks on pentest fails but the idea is that well first of all let me take a step back i'm adam compton nice to meet you i

work for a trusted sec and um today we're going to be talking about fantastiles now what is a pintest fail a pen test felt is where the pen tester themselves causes some issue on an engagement or they do something to themselves or something like that the concept is around this was that i hear it too many times at conferences and excuse me i hear too many times at conferences and online and other places where you see somebody get up and say here's this awesome thing i just did or here's this great new attack or this great compromise or great research i'm like that's great but i want to hear about the times you screwed up or the times that you messed

up or you didn't get what you wanted or something else happened like that that helps show the humanity of the nature of what we're doing because i don't know who you are if you are getting it perfect the first time every time congrats to you i don't know anybody like that we all have a learning and growing experience as we go through well so i started a number of years back

i was doing a quick little talks with my friends and what have you about just talking about old funny stories about pentest fails and what have you they suggested getting up doing talks about it i've been doing this ever since so some of these well all of these are either ones that i experienced myself i did myself or worked closely with the people who did these i mean a pinterest fail can be anything from the nature of let's say we're doing a fishing exercise we're sending out a phishing email to somebody you set it all up you get ready to send it out it's not a big deal you've done several of these over the last couple months

you're feeling confident about it you're good to go not a big deal so you take an old template you set it up and you send out the phishing email well that's great and all until you're sitting back and wondering why am i not getting any hits off of this why isn't anything working then you start looking at it looking into it and well yeah you did set up the web page you did set up the email you sent out the emails you just forgot to restart the web server or you forgot to restart something so you're saying that you wasted all that time not a big deal so you decide you're going to restart it fine i double check

everything the servers are up and going the email list is right i send it out you start getting some hits nothing amazing but you start getting some start looking into it and um you're going down through there and you're looking at it and well i finally figured out why or you start realizing why you're not getting so many hits as you would expect that's because you've been doing several of these over the last couple months and you didn't copy over your base template when you started this one you copied over your previous engagement when you started this so you have the wrong logo or the wrong company name things like that so that is an example of a type of

a fail there's other types where you're writing a report and you have this great attack path you really enjoyed it and you did it on this engagement again and it's awesome attack beginning middle end so you copy and paste it over to a previous one because you had it go through uh peer review you had it written up so good you don't want to risk it because it's been approved and on it copy it over and lo and behold you forgot to change up one of the uh screenshots or you forgot to change out some of the wording just stupid little things like that now those are the more non-humorous versions of this but there's always the

funny ones where during covet we're all sitting at home we're doing our engagements we're doing zoom matings or microsoft meet or whatever the case is and you're sitting down you're doing the talk with the customer talk with your boss or whatever and you have your nice shirt on you're sitting there in front of the screen and everybody calls for a lunch break or something like that or needs to go to a restaurant and you get up and go to the restroom and i can't say that this is necessarily personal experience maybe maybe not and you forgot that you didn't pause the video i don't know about you but i don't always wear dress pants when i'm at home

on a zoom meeting might be unknown what you are wearing but to say make sure you always cover that video and you turn disable it you cover up the screen you shut your monitor whatever because that's the last thing everybody in your company needs to see is your butts get up and walk away from the computer regardless of what clothing is on it i'm just saying another one that happened a lot here recently was um you're doing a presentation you're having to do a screen share or something like that you're doing a virtual conference something like that you have your screen set up you're doing the share session and you're clicking through the slides and you have some co-workers or someone

else who likes to play games with you so they start sending you messages innocuous ones at first just to see if it gets through to you and you happen to have it and you happen to have forgotten to disable notifications on your computer so you get a little notification pop up in the window while you're giving your presentation like okay you okay okay you ignore it hoping it doesn't happen again and then you start getting a host line of notifications popping up getting more and more risque more and more offensive whatever the case is it just keeps going down through there this all goes to knowing your audience knowing your co-workers and being prepared for the situation there's steps

you could have done to prevent this to prevent the egg on your face so to speak but you just forgot to do it in this case but there's also the other funny ones out there like you're going to break into a building you're doing a physical pen test let's say you're going through you got your tool kit ready with you you have your drop box or your tap device you've got your clipboard whatever it is that you have that is part of your kit to break into a building you get up to the building you do your host spill you get through into the building you pass the front desk you're going along you find a comms

closet you hook all your stuff up and you set your bag down because that's now your home base while you're doing the spin test or physical assessment start looking around for other little things you see a back door right there um this particular story happened to be there's two of us doing this so there's two of us going through this whole scenario working off of each other we found this back door we're like let's see where this leads we open the door we walk through next thing you hear is as the door closes you're standing outside in a smoking area or some other outdoor area you turn to your co-worker and say i thought you had the door you both say

that realizing that neither one of you had the door the door is closed and there's now a badge reader to get back into the building you don't have any of your gear with you because you dropped it off in the comms closet because you were going to use that as home base so now you have to find another way to break into the building without any of your kit live and learn always leave yourself a back door in make sure not both of you walk out the building at the same time make sure you have another way back in things of that nature speaking of physical pin test there's occasionally the situation where you're going to be doing a physical pen test

you get the address from the customer where to go it's in the statement of work you show up you're doing the pen test and you succeed you get into the building you're going through and you're noticing none of the logos look right none of the name that then you start looking at the letterhead and it doesn't match with your customer this point you have walked in the front door walked past the guard got through the turnstile went up the elevator got off on the right floor and got through the security door into the company and it's the wrong company so you back yourself back out you go downstairs you look at the big directory and your

company is not listed there you call up your point of contact or you call up your sales team call upon contact turns out that the company moved like a month ago or two months ago after the signing of the statement of work but before the engagement and no one chose to inform us so you just broke into the wrong facility that is one scenario there's always there's also the scenario that you're doing the whole package i'm head down carrying a bunch of packages trying to get into a building and it's a shared facility again multiple different doors one going off to a data center one going off to a credit union stuff like that you're

walking in there and you're keeping your head down so they don't recognize your face anything like that you walk up somebody is grateful they open the door you go in you walk it in you turn around and you're not in a bank or in a credit union or whatever you're in some architectural engineering firm you say excuse me you back back out and you go and realize you went in this door and not this door not a big deal you figure it out you work through it the very next year you get the contract for that customer again your co-worker does the exact same thing walks into the wrong door again walks into the wrong company

you have a good laugh about it realizing that the customer or whoever that architectural firm is thinks that we're complete idiots and we'll probably never get their job but no big deal there i mean this is just an example of some of the various things i've encountered over my years there's some really doozy ones dealing with interns and being sleep deprived and all kind of other issues out there but this is just an idea of some of those um i'll go ahead and take a few moments here since i don't have my slide deck or anything prepared just to see does anybody have anything they would like to ask a story about i have plenty in my repertoire of my

brain that i can pull up from previous experiences or if anybody has any of their own experience that they would like to share i'd be happy to facilitate either way anybody have any questions or requests anyone yes

breaking into what building after hours

yeah i was not part of that um if you want that story you can present that story that is not me i will share a story i'll go ahead

the most fun story i get to use the most fun uh one i performed myself was i broke into a senior citizen living facility by walking across a snow covered yard in complete dress pants button-up shirt jacket with a big backpack going through a construction site up a set of stairs through a construction area into the upstairs uh level of this building the fact that no one stopped a short fat hillbilly in clothing walking through the snow doing ages

while i mean it was a large facility i walked through the credit union the credit union individual that's at the front desk there look someone says can i help you i come now i'm good walk right by them to another door open it which leads me into the actual building which happens to be their new uh data center portion that they were constructing which happened to have live servers sitting right there with no protection on them i did what i needed to and walked on and walked out past the front desk same facility i broke into this place like five different ways walked into the front area i had a big giant big gulp with me

walked up to uh the front desk she's like or the secretary or office manager whatever's like can i help you i'm like i'm trying to find xyz location i'm supposed to have an interview there um where am i how far away am i and she goes oh you're just right down the road i'm like do you have to have a restroom i can borrow i have this giant i just drank no worries it's it's behind me to that wall up those stairs down that walkway at the very end i'm like okay i'll go do that about an hour later i walk back out after i've set tabs and broken into doors all that thank you very much not

once this woman's bad and i that i've been gone for an hour to the bathroom but yeah there's other odd stories as well yes most valuable thing i was able to exfiltrate from any place um a probably a database of about a thousand to two thousand credit cards with social security numbers with um other pii that's all that's necessary but yes probably that um i've also been president for one where we walked out with a server just a server out past the front guard and he's like okay people will let you get away with pretty much anything if you act like you're supposed to have you exploited a high visibility stepladder no vest yes um

again just go and say i'm here that was one of the fails by the way walk in with the vest clipboard uh we're with the local power company whatever it was in this location we're here to test xyz like okay um come with me they took me through the door stand right here okay i'm waiting i'm waiting i'm like i'm starting to get suspicious no one's coming i turned to open the door to the lobby to leave and i see cops walking i'm like shut the door apparently this facility has had multiple attempted break-ins in the past of people pretending to be local utility companies busted sorry so he got me in but not the way i want it

in so did you want to say something come on up i don't have a mic for you yeah i don't want a mic can you all hear me okay yeah i'm good my name is russell mantel i do penetration testing well i actually used to i don't anymore but i got some good some good fails uh like he mentioned uh one time i did a physical security assessment and i just printed a fake work order for the the phone company and i walked into a daycare and the daycare just literally escorted me right through the building for the actual server area and i just plugged in my cable and they just let me do my thing

there i really didn't know what's going on but that's not a fail one good fail that i did uh was you can with command and control traffic we usually use some internet facing redirector that points to the customer's internal network uh we set up a sox proxy which is essentially a tunnel into the client's network except for when we set it up we didn't bind it to the loopback adapter we bound it to the actual public internet ip address so we literally expose the customer's network to the internet on a fox proxy which just gives you a tunnel into their network for like three days and because we were using like i think we're using a tool

called almond rocks and it basically has no logging so not only did i have to call the customer turn them i left them exposed to the internet for three days i also told them i also can't tell you what happened in those three days so that was not harder too yeah that's a good one they never did business with the skin but i don't blame them i would not either yeah that's a good one i like that one uh a different one anyone who used noticeable framework before yeah a couple people there's a smb scanner on there and one of the options says something to the effective use credential and i was trying to do a password spray which is

one password a lot of accounts except i didn't understand how the option worked and what the option really meant was every password that's in the database but you also want me to try that and i walked out every single account in the domain yep that was not great i'm trying to think of any other good good ones physical security story like you mentioned i was in a big high-rise tower in seattle and they there's like a whole i.t department you could walk into and i walked through that multiple times and never challenged me to ask like what i was doing but i did kind of like can i steal a server i sold a computer and

walked out with it nice i got the elevators actually did chase me down and uh that the debrief was why did you come out why'd you like look after me they're like oh because you've got a desktop if you can grab the laptop off the rack nobody would have thought anything about it another physical one that i wasn't present for but many of my co-workers were they planned it out they cloned badges they had it all set up they did all their leg work to get it right they're deciding that since this building is open 24 7 there's not a lot of people there after hours they're going to come back after hours so they

pull up they get by the guard the guard doesn't even come out of the booth they badge at the door it opens they get in they park they don't want to go right in the front door because they'll be seen right away and they know that they don't work there so they go around they find a door on the window in the lower level on the first level that is open so they open it and they climb in and they get down and they start walking through the building and here again they do a number of different things all awesome they each grab a laptop off a desk and they walk out the front door

they get out the front door they get in the car and they leave next day they come in and the point of contact's like you broke in didn't you yes how did you know oh there's several large muddy footprints on alice's desk like a little bit of awareness here people come on they didn't catch the lost laptops mind you but just that there was big muddy footprints where they climbed in but yes go ahead so one time i was trying to do a telephone pre-texting which i really hate doing you call someone you try to convince them to give you their password or download or run something on their computer and the way you can do that sometimes is

like you spoof your phone number you can just download an app just through phone numbers you can spoof your phone number like it's coming from within the company so you can pretend to be from the company this particular company you couldn't spoof internally because everything was done off of extensions and it wasn't working so i just picked the number in the local area so i was just trying to at least come from the same area code and i called them and i said something like hey this is mark from i.t notice the problem with your account i'm gonna need you to go to this website and change your password really quick and he's he's like hey why are you why are

you calling me from mcdonald's and that's the phone number that i had nice i didn't think that would actually show up as like mcdonald's on the actual call caller i need thing i tried to play it off and said oh yeah i'm working remote for mcdonald's but they didn't believe me similar one with uh the phone calls trying this i just googled it just picked one i just look for any phone number in the local area

so i was doing one same sort of thing you call in you're pretending to be somebody i'm like i am alice or tom or whatever from uh hr and i'm trying to or i t and i'm asking oh hold on a second ryan goes pause for a second then i hear hi i'm tom from from i.t i'm like click that happened to me that happened to me once too so what i started doing was i was trying to impersonate something legitimately from my team so i'll just what i started doing is starting off with my first name and then when they say it just doesn't sound like mark i'll i'll switch up a different last name

right i'm a consultant that was hired to come home yeah hoping that my first request to get them to except but uh i don't think very fast when it comes to like when i'm being challenged on the phone i called one company and i said something like hi i'm michael mark from i.t and they said you don't sound like mark and i couldn't think very fast and my little sole argument was well it is i have a code maybe i believe he had a question

okay with the police there's actually some fail stories with that as well but typically on a physical pen test you have what's called a get out of jail free card it's a memo that writes up your name what you're doing who you work for what dates you're supposed to be xyz locations and then your point of contact the director or whatever that company signs off on it sometimes they'll include photos on there that's great and there'll be contact phone numbers on there as well for the directors everyone like that so that's typically you show that they call the first one and you're done they acknowledge that you are who you say you are you're doing what you're contracted

to do and it's all done now there is one where somebody walks into a building decides to leave a a gift basket at the front desk full of goodies including thumb drives and squeezy balls like stress balls so i can adjust a gift basket from they pick some random i.t company or the corporate office or whatever i forget they set it up there and they go to leave and somebody comes in and says oh i'm sorry what are you doing here and they're like okay wait right here and i'll take this to someone and then they're stuck there a little bit and somebody comes in and starts talking to them so they're trying to get

out of there but they can't really get out of there real quick so somebody finally gets suspicious and they call the cops because apparently here again there's been bomb threats in this area so we run into i've ran into our co-workers i've run into this situation many times it's weird so cops come question him he says i'm here for an i.t engagement pen test here's my get out your free card they're like all right here and they handcuff them and set them down on the chair right there they call the first name on there doesn't answer doesn't go to voicemail they call the second one same thing they call like three of the numbers that

are on there the three that are provided like this the president the ceo and the ciso or cio right here no one answers and it either goes to a full voicemail or something i don't know but either way no one answers they're escorting him out of the building at this point to the patrol car when somebody tries one last time and they get through to somebody turns out that they all three are at a off site having some sort of meeting and they didn't want to be disturbed so the learning lesson here is if you're going to be breaking into about into a building or facility and you have some points of contact that are supposed to

be your way of not getting imprisoned make sure that they're not otherwise occupied and are able to answer your phone it's all very important

then you get out as soon as you realize it and hope it's not recorded anywhere and if you do get questioned that's when you call your employer and possibly your employer's lawyer because you may have some explaining to do early early in my career i made a mistake with a with a vulnerability port scanner stuff like that in that i provided a very long target list on the command line this particular tool truncated or the command up the shell truncated after a certain point due to building rules of this tool and all that it turned what would have been a partial ip address into a subnet range and scanned it happened to be a large motor company a

vehicle company and or i think it was and we got a phone call from them saying why are you scanning us like we're not we're looking we were the end result of that was not them being mad at us they were asking us did we find anything could it turn out much worse yes [Music]

i have done well during like that sort of exercise it may go for a week may go for a couple of weeks it all depends on what the back door is what the call out time is if it only calls back to you in case it fails like once a day so like that it's low enough that they might not detect but i've also had pen tests where we'll do a pen test we create a domain admin account and they terminate us from the network before we can clean that up and so we tell the customer make sure you disable and remove this account we created it as part of the engagement we come back next year it is still there

so potentially years

yeah i've done one where not necessarily a back door but well yes worked all day long on an engagement trying to get into a rectangular web app pretty sure we could we finally worked our way through it sql injection command execution i was blind we did the whole nine yards we got access into a the underlying operating system we were able to execute commands but we weren't able to see the response back we knew we were executing commands because based on time delays and we could see certain activities like pings and other things we knew we were doing stuff and we're trying to figure out what to do there but some reason it wasn't reporting out to us

and we had an intern who suggested that maybe wait keep in mind we've been working all day we're already like at seven eight o'clock at night we're tired we didn't eat much we're really just running on steam here intern said well you know if that's a windows box uh the windows firewall will tell you in its logs what the errors are if you can access that if you turn that on we're tired we're not paying attention we enable the windows firewall we lost our connection we turned it off we went home for the day and we said that we could not compromise them i mean you do something like if you're working so long on something and you get

access and then you lose access it's generally better just to walk away for a little bit clear your mind come at it fresh the next day or what or somebody else talk over it all that um but it happens and you just deal with it typically our access is for a type of won't produce this all authorized work it's only a couple weeks and we by design will terminate the connections after a couple weeks right we won't typically maintain access to customers network for multiple weeks right a lot of times payloads are configured with kill dates so that wouldn't matter what happens they'll quit working after a certain date but right i don't know i don't know about adam i don't

typically use hardware devices i think they'll plug a device in it'll give me access to the computer i can take my usb stick with me it stays with software backwards i've not done that myself if we do a physical assessment that's part of a larger operation we have cheap devices and more expensive devices that we plug in and at the end of the engagement if they do go missing we have as part of the contract that the customer is to return any that was up that it was uh confiscated and i'm sure there's legalese in there as to what happens if they can't find those like somebody walked home with it i'm not sure i've not been part of that

scenario yet but i'm sure it has happened and i don't know what they do with that honestly consultants typically run 200 to 350 an hour and you can buy a raspberry pi for one hour yes honestly it's not a big deal correct that's right generally speaking yeah now if they have like a a sale card or something like that you can add up pretty quickly it all depends i think we're almost out of time anybody have any last questions okay go for it

it's generally a good idea to start um

personally in my experience i think it ultimately depends on the customer if they have had a large number of external pen tests or assessments already maybe you should go with an internal for you if they have not had anything done yet i would probably do an external and internal combination assessment something like that if you're wanting to go full red team it doesn't really matter because you're going to go for a point of entry whatever it is whether it's external it's physical whatever the case is but i can see arguments any given direction on that depending on the environment that you're testing but i mean we've gained access internally through an external web app many times

but once you do that you can go ahead and start on the internal or you can say okay we gained access here and we'll continue the external and once it's done then we'll jump over to the internal and assuming we already have this access what could we do so you can piggyback off of each other things of that nature you got any ideas yeah there's multiple different approaches have you heard of the critical security controls there was like 20 last time i woke up number one is like have an inventory of your asset your control number 20 is a pen test right and the reason is because if you've not done the other 19 controls you're

getting a penetration test you're basically a waste of time you can already answer the question um am i doing this thing right yeah no because you don't have the control implemented so i typically ask the customer what value are you trying to get there is value to looking from the outside and then also values working from the inside we will typically where i work we will typically start with what we call the assume breach methodology we're going to focus less on how to get in because we know attackers are pretty good at that we're going to focus on what happens after you get in correct doesn't mean it's not valuable from the outside either if you have no

external presence like i've been asked to do an external test for a law firm and they have no websites literally just their building id address and like there's nothing there right services what's the point yeah i usually ask what value do you want that's a good point too as well as the first one you brought up is if they've done nothing to test their own environment to go through the checklist what have we done what have we done do we have these controls in place if they don't have that you're going to know that as soon as you start because you're going to gain domain admin you're going to compromise all their database all that and at that

point you're not doing them much good and you're not doing yourself much good because they have a long ways to go before they even get to the need for doing a pen test really they have a lot of things they got all the baby stuff ish the checklist stuff and all that to get done first so all right i think that's about time for us thank you all for this impromptu little talk here that kind of meandered around the place a bit thank you very much [Applause]