Bryson Bort - Adversarial Emulation

or today to give you a talk about a material emulation so who see me speak before Wow and you came back the rest of you don't know what you're in for so your traditional InfoSec presentation is typically pretty dry its monotone its technical PowerPoint that's not how I like to play so I prefer audience participation now the end of the day you know this is America so you have your choice of how you want to participate but if you don't participate correctly we'll get you there you know I mean good see it's easy

ry s so in anyway all right so fundamentally what we're going to talk about is that the challenge when we look at today's red team when we look at pentest is a primarily win centric aka ego focused approach right I get in own domain drop Mike you so I won yeah GG of all our children so we're gonna we're gonna keep this G rated I've never actually given a G rated talk before it's gonna be hard that please make icon to keep me like on pace it's hard for me alright I get a little out there and I forget it so fundamentally we're gonna talk about is how do we make it business centric how do we make it matter how do

we make it results focused and so my contention is that today's red theme falls short of achieving this which is why I call this adversarial emulation the idea is that the purpose of the red team is not to do red teaming the purpose of the red team is not to win the purpose of the red team is how do I Drive through a business centric result how do I understand my business with the threats that are against it so Who am I I founded scythe I'm a co-founder of the ICS village nonprofit and I founded Grimm I'm clearly a clown so I like to play around and have fun I say the one thing I think it's pretty

cool up here so is everyone familiar with the hack the planet CTF that the village brought yeah so for second year in a row our winner at DEFCON is won a black badge so that's just that's actually what that picture is right here is that's he's he's Israeli so just pointed out here at army cyber and Israeli beat off the Americans so y'all better show up next year and win that crown back so anyway that's him up on the closing ceremonies at Def Con getting a black badge for our CTF so real quick I'm not going to give you a vendor shell but two things that are irrelevant here so in 2016 a fortune 50

who you all know and shop at came to us and said hey we've reached the end of the rainbow because there are two kinds of companies and it's not the companies that have been hacked or not act or don't know it yet it's the only two companies I've ever seen in cybersecurity of those where the leadership takes cybersecurity seriously and those that don't it doesn't matter how good your technical staff is right like she looks really smart but if hurt well you look smart how about that I'm sure you're also very talented and capable and your organization loves that you work there but if your boss is a that's g-rated right this is hard for me

I'm working on it solely so you will not be effective no matter how much you try or whatever you try to do or your organization your work clothes or whatever you're trying to do will not succeed it's the leadership that makes security not the technical side that's where it starts right now you need to be technically competent to actually achieve that vision but if you do not have the leadership then you will have what a lot of you have probably at some point in your life which working at an organization that doesn't take what you do seriously and you get frustrated right and keep being surprised that you run into the same issues over and over again so those

folks went through that and after a years they had hit the end of the rainbow where they're like hey we've used everything on the market we can't find any way to test our defenses further and that's where they came to us and that's that's where sight came from from I realized that I came with this idea that called the hypothesis unbounded attacks and it's that we keep chasing attackers by looking at what we see right I am really good and we as an industry are really good at solving yesterday wow we're so smart when we look at what happened why can't we ever figure out tomorrow why can't we get to what's gonna go tomorrow

why is the threat continuing to confuse us and the reason for this is because like anything we're looking to big we're looking to broad the whole space hi Liz there's a seat right here

I really didn't expect you to do that so the the funny part of that bit actually was who was that besides Las Vegas so if you saw the picture of the ladies shooting the unicorn on the unicorn that's the lady who shoot the unicorn we had a panel and I was leaked to our panel like I'm a speaker and I was several minutes late to it so she reminded me of that earlier which is why she was so kind to play ball showing up late to my talk so back on the bounded attack space does anybody in here know how to measure security seriously like measure security it's a trick question I it is a trap yes

nobody can measure security we have no way to measure security I have no way to quantify or empirically measure security I can look in a lab and say hey I've seen these things and if I run these things in the lab then I get that test but what does that do in the real world we keep getting surprised by whatever happens tomorrow again we solve yesterday to get to tomorrow we never figure it out ahead because we have no way to measure security and so when I was looking at this I was like whoa let's take access off the so exploitation is actually mathematically infinite is my first statement there are so many possible combinations which is

why vulnerabilities and exploits continue to happen and oh by the way that's just technical access do I need exploitation to get on your computer what's another option been humans people people who stick their passwords on post-it notes that's not technical that's just Scott being Scott

so let's take access off the table and let's figure out we can constrain it then at post exploitation because here's the other thing once I gained access to your computer if I'd done anything yet I got shell I'm looking at um bathed in green light the black background have I done anything yet nothing I haven't done anything yet I'm there but I have not accomplished a single thing you actually care about yet you don't like that I'm there but it's the same thing as like the burglar coming up and picking the lock to your house they haven't stolen anything yet nothing has happened so that enthusiasm I'm like a Disney character now it's alright your dad your dad's not

following either so once I gain access I can constrain it across two vectors one cuz this goes to this other question I'd always had in my mind is why is this the only discipline in the world that I know of where somebody else seems to know my own space better than I do that doesn't seem fair communications that's not true the defense the organization chooses how you communicate right you don't get to show up and bring something else to the table we have a certain number of communication protocols one those are the only things we allow in the environment and to pattern of life as an have to blend in with what's there so I

can only talk with the way things are already being used in that environment I have to blend in with that so how many different communication protocols are there in the world aren't that many are there alright like fingers and toes number finite the limited number and then if we look at the capabilities what do attackers do what are the behaviors that they do that's also constrained there's only a certain number of things that they do and where they do them and so if we can find a way to combine here are the ways that I can talk over these are the things I can do we've now created the ability to measure security because we can replicate the entire attack space

because it's now bounded enough for us to be a metric that survives to tomorrow this is this is this whole hypothesis I've been working on for two years with research and it bounds a lot of the different ideas and the things that I talk about so all right so I gave part of the answer here what is the value of post exploitation you made eye contact

he was panicking seeing what you can do that what does that mean seeing what you can do next to be proactive in your estate and your environment to define the first question you posed to us with the future instead of the past the future instead of the past okay oh now you want me to come back all right you wait wait wait sir is it the post visit the pose that did it for you this is what you would sorry Q react we actually do need it in the microphones for the recordings of your company so business right business business value business context business risk this is where cbss falls flat this is where our

industry falls flat because we get a bunch of nerds going to the business or going to the mission and going like Oh nerd things and is this goes well how did who let him in the room right it needs to be in a business context and what's the largest surface area in any business in any organization any mission largest surface area guaranteed you that I need another pose

hey I didn't make creepy eye contact what I did it that's the line of sexual harassment people users there you go users people are the largest surface area of any organization right we talked about this earlier when we talk about the limitations of exploitation we talk the limits of technical people have to be one a part of the equation they are the largest factor in your risk how they use your computers have any of you and this is off the record no seriously how many of you who have circumvented defensive controls to get your job done right don't worry I'm not going to challenge your sf-86 people use computers to do their jobs they will do

whatever they need to do as best as they understand it because not all of them are technical wunderkind like this electric group most of them are not that technical he's not that technical that's why he was late so the point here is the value in the post exploitation space is I'm truly understanding the contours in a business context for understanding that business value because I'm bringing people into it gave the engineer he's a person so Red Team versus adversary inflation so we talked the very big thing when explaining that and this is mostly for you because you relate so I just want to catch you up to speed is we are moving from the way today's red team

works which is primarily a surgical win focused project to where could we make it more business centric where can we get actually like a results-focused process out of it so we'll start with some definitions so we're all clear right all the way on one hand from vulnerability scanning to assessment to penetration testing that's where a lot of folks think about this stuff that's the access question that's where I'm looking at system hardening that's where I'm looking at how do I get into this application this system to a next step which is a red team and then I'm setting up a friend of mine who I'll call out later he's actually here in this room or else I'd be picking on him

more then is this concept of the continuous purple team so purple team has everyone heard of purple team's they haven't seen a purple team you have once or twice I'm gonna pick on him first a minute or two a minute or two did they'd like throw you out of the room Oh so back to that leadership problem all right you said you've seen it more than a minute or two though you've seen it once or twice okay talk into my chest I don't think those work they don't work no they don't you get to talk in the mic another rescue do none of your bikes work okay so you've seen a purple team what'd you see

so you're saying they were there but they didn't learn anything show hands yes I did know and that's that's one of the frustrating things from the technical side of this right like who's done pen testing here who's come back and seen the same thing the next year you run the pen test and you're after you run the pen test right why does that happen

so we go back to the leadership assertion you had it do it something you definitely don't have a microphone you have to talk into my chest you think I'm kidding you were late so you missed that now sit down so when we're speaking to them we have to speak them in their language right they hired nerds for a reason because we have the technical depth to understand what this is what this requires we have to tie it back into the business that we can bring into the room so when we tie that back into like an operational workflow purple team is I have an offensive assessment an adversarial emulation a red team that's connected in with the business

that's tied into blue team workflow and operations right that's a full-piece we're defense becomes a part of the assessment and it's shared so oh lots of words okay so red team attributes what we see mostly today I mean can you all read those yeah you don't need me to read those I mean I'll do it funnier than you but you can still do it quicker in your head any anything you would just up there

don't be afraid or do I mean it either way but say something

you're affiliated with the prestigious university RIT what do you think decorate

everyone else agree ever Arry damn it sorry darn it shoot shucks Oh golly God sorry this is my that was a joke Jesus come on some of the jokes are subtle all right so what is adversary emulation one it needs to be customizable I have to be able to change my command and control I have to be able to change what kind of actions I can put on objective you go back to my original hypothesis around the bounded attack space right I'm working across two vectors of how do I change my communications and how do I change what I'm doing what behaviors I'm emulating what TTP's do I want to do it needs to be repeatable right one I need

to be able to repeat this exactly so that I can use that as a specific metric to understand how things have improved or not the other advantage of it being repeatable is it makes it much easier for that purple team goal where now I have something I can easily recreate back on when the blue team or DevOps is looking at remediation or building something else this is now something they can bring in as a part of their test matrix kill chain insight going back to the fact that we don't want our red team to just focus on winning we want them to understand the contours and the choke points of our business you don't need to solve everything through

technical solutions right if I determine that let's say I have white listing in my environment so first thing I'm going to do is I want to create an adversary that tries to come in and run an XE and it's not white listed so I validate that control that white list works that payload did not succeed does that mean I should just stop or what if I then go okay we have validated that whitelisting works throughout our environment now let's whitelist it and see what happens next let's go further down the kill chain with understanding what that adversary might do because this is the point of where we're trying to emulate an adversary nobody in here has enough

budget or time or resource to truly put together a full nation-state capability you don't that's why we're trying to emulate it we need to understand the specific points that are relevant and find the cheapest economics to emulating them in our environment to understand them we don't have to do exactly what they do to do that right I don't need to defeat white listing I just need to validate it to a point and then defeat it by white listing does that make sense and then automatable all right so white box versus black box this follows the same concept when I talk to clients I tell them that most most folks like the black box pen test like you don't know

anything and just like see what you can do why right going back to do you really expect me to bring nation state level of capabilities within a two-week time frame you laugh but that's the industry expectation that's how most people think yeah I know it's crazy but they do I don't know why what they do so where I say is like again let's cut to the chase give us all of the information because at the end of the day it determine adversary will get that information that's what you really want to understand your pretended opacity not true and you having me do a two-week assessment that gets stuck on that doesn't prove anything so white blocks

versus black box defense validation

talked about a lot of these this part we sort of cut a little bit so executives this is this is where we start tying it from all right this is an interesting concept Bryson let's really take it to what's the business case the start of the business case again is one if I can really do emulation of adversary and again there's a lot of cuts that make that easier to do that which we just discussed so if I can find something that's economical repeatable and scalable I can now validate the security spent how many people think that's what a see so actually does today what do you see so is do today get fired well that's why they're on to your

turnover they tend to quit before they get fired it's like it's just smart enough it's like alright the cons almost up time to go to the next town with that the three envelope story I don't know the three envelope story ok so here's here's the advice that I can that I can give you and then you're on your own and he gives you three envelopes let me hit of crisis open an envelope it was the first envelope because things got bad after a few months there's like an ounce a reorg all right now by at least another year while everybody tries to forget what the heck's going on and forgive it a second ago applause I think it was something

like visit projects are at product strategy or something renamed every changed the names so my joke my joke on this is that like literally being a CSO is just being a great con man prove me wrong I mean first of all who in here actually knows everything about security what none of us nobody does that's the problem none of us are actually experts we know like slivers really well but nobody is a whole expert because this whole space has been unconstrained it's impossible so the seaso is in charge of all of that what else are they supposed to do whatever Gartner tells them and whatever their peers are buying done yeah you you got all but that's the game

that's it that's all we got at least I'm the same as everybody else that's the best I can do that's where we get into the value versus snake-oil since the threat is in fact the threat that's what we're worried about maybe we can now start to get to what really works for us right that's your interest in adversarial emulation where we can take red teaming beyond where it is today and then of course the people side who works for Nemesis being here thank you for being brave you did know you actually you're current why would I talk to old news you're new news he like shrinks away I'm just coming over you just for that embracing nice to meet you

you really like ran away how long have you been married

[Applause] okay adversary Emily she seems like this was not the time to do in real life he's clearly apt he's in here screwin thing eight years eight years of wedded bliss right this way enthusiasm yes there you go so you were kidding FSSP it's been good for your marriage you can't script these things right this is real life some time talking about the people component you need to just set him free this is post exploitation children post exploitation see I can do the G rated humor it's like Disney where there's a subtle context the adults are gonna get it the kids are just like Simba okay so how confident are you as an MSSP that you really are on top of

everything that's going on in your customer environments why not because it's the customer people again

what's your ability to put those vulnerabilities you are aware into their business context like again you know a server being a lot of different things right like one might contain everyone's social security numbers the other one might be just you know the Eventbrite registration for tonight's awesome barbecue at 6:30 p.m. that was a that was a shameless plug there is in fact a barbecue free barbecue tonight 7:30 Scott who was over here earlier is making 50 pounds of smoked meat personally like personal and made barbecue homemade homemade again children stay away from hand and meat jokes

so I see you circled something else for this time period I am NOT Paul Jeremy or Addams Wanda how do you feel about that now okay now that's all you got to do is tell a man he's funny and we're good oh that no you got to take potshots over the microphone sir no not that microphone that's convenient it's like he's over here playing on his phone what's going on you sure

hi or are you guys saying oh are you also one of those just caught in the crossfire here yeah and now you know you're here how do you feel good so far good so far what what who said well at 15 minutes really have I been talking that much alright threat intelligence today one it's driven by static identifies everyone's seen the Pyramid of pain before correct was the Pyramid of pain summarize the two sentences golfclap comes up yes starting with trivial easy simple annoying not really CTI mostly is the bottom three right again i figured out yesterday what is yesterday well yesterday is here's this hash coming from this IP address tied to this domain

what do you thinks gonna be tomorrow something else done game over that's that's the challenge with today's cyber threat intelligence is these things change and how do you consume like real threat intelligence if i'm going beyond that now going up the pyramid of pain where do i get those really cool tough TTP's in the red research forty page reports that's how you get it today here is a 40-page pork operationalize it it's not machine readable there's no way for me to ingest that stuff i'm actually submitting a talk to miter attack on where i'm going to attack the whole cgi industry for the entire talk it's gonna be good my career has been just fantastic i have

i have a shorter i have a shorter time life in a sea is oh alright analyst reports this we're talking about you have to read them their lengthy would you actually do that neutered malware has anybody ever worked with new term malware you know what that is yes you have yeah what are you really inviting me back up with the microphone

well we're all jocks so nerd thinks I'm funny yeah add software now where found in the wild we take out it's dangerous components and then we play with that right that is like real adversarial emulation right because I'm taking a real thing and I'm gonna play with it but it's costly and it's difficult it's resource intensive right I have to reverse engineer it I have to take it apart I have to recode it I after then feel confident I've done that well enough where I haven't just shot myself in the foot and there's also the potential risk that the defense gets focused on the signature aspects of that neutered malware versus the behavioral components of what that malware is doing

so this is a talk that's going on right now as well I don't know about right now because West Coast time but today at beside Sacramento who knows what impasses see you are smart you just throw the confidence like I am gosh darn it I'm smart I'm good I'm great at this stuff no you gotta repeat it you're great I do see you're late this is why you don't know this what I do yes super nerdy response that how many people the room actually got that you're right you're absolutely right it was a concise no that was you're you're you are right I just said fortunately look what we're dealing with I look smart to

me here I'll go back with you slides over there we go that's the CIA they don't let me talk to see sweets so anybody else were to take another shot at that in laymen terms I think you raised your hands okay what it is is this one of the primary ways that I ingest smell where I turn it into a hash right this is a fingerprint fingerprint every time I see it is that bad thing and cut it out of the environment so we talked about the downsides of that and we can throw stone at the weakness of that approach but it is kind of a fundamental building block of defense isn't it well what if it was

flawed yeah so we we put together we did research we found a way where you can have the same thing have multiple finger so it doesn't even have to be polymorphic to break the hash that underpins a lot of how defense is done and there's a link to the paper so this is threat intelligence from an emulation perspective up giving credit the specific content is not from Katie but this approach of highlighting this kind of thing did come from Katie Nichols it lighter for those of you who have not seen my der beek on Fox to the one who when I asked what's the 13 emulation and simulation gave like just drop Mike no answer which I was not

expecting so she's really smart so our lady so threat intelligence right here is where I take specifics of a threat actor and I start breaking them down in this case I'm using miter attack who hates miter attack who doesn't like miter attack I don't work for miter I don't in fact they complained to me when I did not properly show called Dara at Derby con it doesn't take what as or paths so software is a service platform as a service the fact that it doesn't have that what is the okay what it does do well is it is the first periodic table that we've had to combining elements for understanding attacks right there are it is still it

is still fleshing out but before this we had no common vernacular on how to say this is what a bad person does now we do and here's how we would do something like that right I can look at this go this payload was an auto IT all right well Auto IT is s 0 1 through 9 they retrieve an executed PowerShell our shell is a particular technique from that domain well there's the IP address I can see so we start to pull this and we now have what if we call attack a periodic table I've now developed a chemical equation from that periodic table to describe a particular attack so this is where I recommend using attack not to get

focused on where I think it breaks down more than the gaps that it has is the way that industry is respond to it has been like okay I know have a checklist to go against hey show me I'm good against PowerShell it doesn't work that way but that's how business is trying to use it today but from an emulation perspective this is my way of describing a chemical compound aka particular adversary what they can do and how they do it so pre attack Enterprise attack a whole bunch of attack this is the latest addition so this is from Blake Strom who is actually his name is mister attack so is the lead over attack they've created

now sub TTP's so you can go even further down the matrix of understanding like well here's high level like PowerShell and they're now trying to actually bring out all the sub components of that so this is a living breathing thing mitre is very open to criticism there they actually are so if you if there's things you don't could do like that's Blake I'll give you his personal cellphone number

Blake I hope you're watching this so these are the good things these are the bad things open source options so this is something when I first came up with the idea of this talk and I'll be blunt right always acknowledge your mistakes when you kid there's the homeroom mess-up when I gave this talk at Derby Khan I wanted to give a much more detailed component of using open source tools to then show you technically how to conduct adverse or emulation across particular vectors turns out that was a bigger project than I imagined so what we have done is this is a collaborative project that a few of us are working on where we are tracking 24 tools 19 of

which are open source those the ones we're actually going to consider in scope and by November we are going to release a high level summary matrix of all of them like so like what kinds of things they can do across all of that and then after that the way this project is going to morph is then where I talked about how to specific adversaries and emulate them we're gonna we're going to be publishing that for each of those frameworks so like we'll take a PT 3 and we'll try it with all 19 and we'll show the pros and cons of like trying to emulate with one of those open source frameworks so this is just an open

source project that we're been working on for a couple of months and we hope to have the initial matrix completed by November so essentially all the real depth of what I'm talking about philosophically here we will share out

so these next few slides which we're going to kind of skip through are fundamentally when we're looking at adversarial emulation the two elements that we need to look at our host behaviors and network behaviors right what can I do on a host what is visible on a network how long is the average dwell time of an attacker before they're discovered it's a lot right the number goes up every year I've steady good I think it's three hundred days right now I think I've seen to two hundred days in the past why is that the case because where are most of your eyes they're on your hosts around your boxes as an attacker a real attacker that's

just their test matrix that's where you try to see me on a box where I operating I'm gonna beat you every time but I cannot defeat the laws of physics the laws of physics are everything that leaves that box goes down to layer one ones and zeros I can't do anything special about ones and zeros right it's not like I'm like well I'm twos no I'm one zeroes just like the rest of you now you have to find me in that soup of ones and zeroes but I can't hide from you right I can't prevent you from ever knowing I'm there because I can't defeat the laws of physics

my buddy Jorge he teaches a class sans and right now this is a two-day class on purple teaming and he also does want to adversarial emulation brand-new class I think their first when I put this together was before he even taught it in september/october and they're looking at putting together a 5/6 a class on full purple teaming in adversarial emulation so I usually don't like throwing a lot at sans but Jorge is a good guy and he's this is one of the cutting edge pieces of somebody really trying to put a lot of in this as opposed to you know being told at 10:30 last night you're giving a talk at besides Augusta that's why you

had that other thing circled and why my name is taped up there was I was a last-minute dad I can't read either questions

what's gonna wake up businesses

what's the purpose of business what does that have to do with computer security what's the largest budget in any business the budget to run the business to do the business marketing sales operations largest budget every business ever right we created this company to make the best cookies of all time computer stuff it's just what I need to do to be to scale to be a modern business so now all right so a fraction of my operational budget was very spirited was it interesting the IT budget is a fraction of that business budget security is a fraction of the IT budget those relationships will never change ever stuff like this is a niche inside IT security so if that is always

the way things are then we have to work within the context of just like let's make it a little bit better incremental ISM I got into an argument a few weeks ago or some like actually with a straight face and this is somebody who I think is smart said in 10 years this problem will be solved I I asked if I could take what he was taking but I didn't get a good answer to it any other questions then I bid you adieu [Applause]