Keynote

we're excited to be here like super super super excited cooks my entire security and career started in the city so for those of you that don't know that it is like here's me goosebumps telling other people that requiring around comfortable telling people that when I got here I was on the phone with some work stuff it was making me like really depressed and so I was just doing the like depressed talk the whole year and I got to my hotel rooms and it just blew me away when I looked out the window and I thought about all of the different times that I've had living here in the ten plus years that I lived here and it just completely goof

up thinking about it right now it makes me really happy so thank you thank you guys for just being here and it's starting and having a community here because when I was here I think it was something that all of us in the community really wanted but nobody had really gone out and done and so the best we could do would be like pitch of each other at work and you know like be happy about it kind of dish until we got wasted at the end of the night to try and figure out how many times it wasn't the viral walls hole this is the top so yeah I want a clerk by just thanking all

you guys heard for being here this is it's something that's near and dear to me because if any of you understand the history of working side came from it really came from us being pissed that it cost so much money to get access to information so much money when we looked at blackhat and I was like these pieces I swear I'm going anyway so feel that these pieces should people cost $4,000 to go listen to one of my own [ __ ] friend pop like the band oh I'll be [ __ ] I'm not going to do this and then I thought about all of you Blake didn't know we're like scrapping together to try and get money

to go to this in $4,000 they know like [ __ ] house party we're throwing a house party I'm calling everybody that you want to talk you know one here their car I'm going to be like you know give the same talk at our house we're not going to have any cameras so put some dirt in it and they were like done so all these people came over to this giant [ __ ] out that I rented that we almost burn to the ground literally because some people aren't really good with like yeah they computer well but they don't electronic well so there is an AC unit that we bought from Sam's that we took back because it

wasn't defective but whenever we get any money for that [ __ ] somebody Johnny had put an extension cord to the AC unit and left it curled wrapped like pronto how many E's are in here but like that how you start fired and so the funniest part about this damn thing was that we're leaving the house and we're wrapping up were like okay we're going to get burgers to punch the thicker and also there's people who eventually be by don't burn the house down we said the word soul burns or [ __ ] house down and we came back and the carpet was black to be [ __ ] kidding me Jarell and he just looked at me and you

stood gone we have anything about forget and hit because was born it was it was weird it was weird to know that we had all these things happen in all the sudden everybody kind of came together and was at the house and having everything in time we didn't have a motto we didn't have a so we had you know our goal was throw out Friday before people could learn about you and anymore like manna that's really cool Jack Jack was an attendee right Jack came over to the house and it was really obvious from the beginning exactly who was going to do things and which people were there most people were just kind of tourists

the funny part is that even the tour is converted to working and so we had a hundred people in this big-ass house party that had talks and everyone was doing something if something was messed up it would just get fixed if there was a cooler that was leaking someone would stop it and clean it up if there was a whole game going on where people being too loud it was like self-police and the thoughts will go on if people wanted to talk about something they would just bounce in the middle of the talking to Phil outside and start talking about [ __ ] there were that house are everywhere there was [ __ ] getting

done there was no rule set about it it just happened we were so hot in this stupid room that this prick bill like there's supposed to be a wedding chapel all the best night could have these used skylights so I knew Jack was wanted to get get [ __ ] done pickles because were like it's wicked hot in here the whole air-conditioning fire thing isn't really working for us what do we do and Jack's like let's get a target so then Jack and I climbed a [ __ ] roof of the house and we're like nailing a car down to the top of this group to try and give it some shape and people inside are like

clapping because they're dying from the like Vegas heat and we just were like okay so we don't really have to make a will set and tell people these are all people that want to be here they want to be here so badly that they're going to make sure that it stays comfortable for them to stay that to me from that first moment of doing it all the way up through building it to levels where we were renting out hotels and doing weird stuff to now where it's like filling the tub painting I am more than honored to know that it is kept the same tradition of people who are here actually want to be here it's not because they have to be

it's not because you know hey just as better than work or his it's great that I get to go on the Blue Devil today and not show up for the Box people are here because they want to be it makes me so so happy to see that in where my career started so I thank you guys for that more than you'll ever all right the fun part of not so on this point where you know that I swear still to really get the different partial if you talk he actually did that so that's my history of work as you can see get started here it started with people that are in this room that I haven't seen in ages that

I'm so excited to catch up with you know I run a company called Juarez we're known for doing red teaming in full scale type adversarial exercises you just make up whatever [ __ ] words breaking stuff so we wanted you know we're we're part of the people and I can start claiming fun things like for the reason IBM says they do red teaming because they I don't after doing until they were just like [ __ ] they keep bumping into the little company of n people and they're beating us and deals what we do is do I know I'll put it on our like on Tom if you were awesome welcome glad you count application we get to realize

we've done some community efforts these sides thing trying to make the world a better place in any ways we can try to make it a little bit funnier with a less serious we're trying really really hard to fix ten testing because people just use that term and then you know you have somebody in the corner it was like barreled away like hitting the vanessa's button it on her home and then it just makes me want to murder them so you've got a whole bunch of really interesting and smart people from all world together to try and create a standard around pen testing because at least if we wrote it down which key quote that I have ever

heard from a friend of mine the lies we told today are the best practices of tomorrow I took that's a heart and we thought it's just write all this [ __ ] down and now it's in it it's in PC eyes and always often places and every day that I hear it being used somewhere only hot it's looking where do it and then the showing the rest of the way you're like I seen it on the news only why is this crazy [ __ ] we got we didn't work together put it out until trying to progress some things we do some stuff like code review at work we do some incident response work we do if

you rip assessments we do physical security we do some social engineering type stuff from pops moment and testing everything else I'm very sure Amanda would kill me if I ever did that so I don't specifically do it that way I try and use a little more tact right that's straight horribly okay I want to talk a little bit about vulnerabilities because it's starting to get like really stupid out of control in my opinion how cool is it second from the bottom it's high totally where I feel like we're getting to a point out of forwards so it's going to be like only your face falls off you know is the foot it's the way out there

right now that it's driving me nuts so I've been trying to figure out how do we get away from it and get a little bit more towards measurement versus people like just freaking out all the time because it is eventually you get there's like too many drunk chips explosions pipes whatever in the movie and you're like it's not enough and then you end up which is like back in the Furious for you you know everybody 75 years old it's just nothing but you're blowing up the whole time and you're like man this movie doesn't have a lot of action in it I feel like that's the worst place for us to go so I just have been trying

really hard to figure out like what time to change it so I look kind of it where we started right we started with equation that completely don't make sense for your security there XE ISPs in here yes he has a team hand up CIA's ISO order Thai people okay anyway you remember this one this the single loss expectancy times annual rate of Earth right give me any light loss expectancy what yes how rumors probably yeah hey guys do you think you're going to break indoors here like three or five times this year hold on let me check the list you know and then they're like great till the order is solely three guards then we let

money and only let me help it this doesn't work that's not real it's just it it's this right it is I don't know it it's this many big how much different if rich ultra high versus medium ultra high stupid and we're like ok so that's kind of crazy we realize that some actuarial person you know no fault to them you know they were trying to calculate life expectancy or someone who smoked and with a bunch of data set to really figure out actually where you're going and we just made [ __ ] up but it sounded cool then we tested people on an episode of I so what it's real and best practices lies remember so want a guy or

maybe we need to visualize this because that will get more people enroll so we make create dashboards right so now we have a dashboard of stuff and I'm not bagging on rapid7 just it's just a big dashboard but you know hey hey one up what do you please crap are we're four percent okay wait a second the sourcing do they have to look at her like four percent of our operating systems are fun oh yeah what about those oh those are purple are now so that's supposed to be happy or not happy about 4% now like a whatever least it was an Oliver is not a good way to manage progress of the environment by

like the amount or lack of stuff ready but we do it we thought all right cool now we have this we can Thai words that people really get afraid of like wrists you know that it took like three thousand years for Japanese to make a word for risk because their opinion of that word with that well being specific enough for real three thousand years to make a word we just throw out all over the place so I look at these no oh we made a risk for this is right context-driven risk score 2742 what chicken dreadful coats van Rycke yeah that's it i drowned ten thousand times of the I'll just settle for not telling one time I

was like let's talk don't really freaking out like I don't need to be so paranoid then I'm like throw a cup of water on the dolly but it just keeps going so that didn't really work well right like that approach does not fix anything because we just took those boner hilary's and we push the [ __ ] out of people underneath us and when I fix the vulnerabilities it's your fault and they're like I can't get the medical device somebody cooped up to the thing write down like they're gonna die if I catch it and they're like I don't care no more 4% so what we do that we outsource risk we said hey let somebody

else determine how risky it is for us because then we make a bad move we just blame them we'll see BFS said it was hard to watch it and you're like the best assistance miss person diet and they're like not my problem so we help decide that for me that also where I don't want to be I don't want to be in a point where like you know hey guys let's jump off this bridge that Paris you know doesn't sound good they're like not your decision like bill do you think Oh yesterday you know that not how I should be doing those things what any difference between like Steve ESS 2 & 3 recently because all I did was

add critical right they're like hey let's make [ __ ] worse and they say any of the things that were low and rose the medium because whatever maybe they'll fix it mark there's like no actual science it just it just made [ __ ] more risky which really really helps us right like boom as soon as we move to this next version of scoring me some more oh how would we move to the scoring ever like now we're just going to use two three go so awesome how Isis bolstered our three days so that that's not helping either because we're managing based on type of motor abilities and type of risk knowing full well that there's always going to

be something new that comes out we chase our deal it never stops we continue to chase our tail over and over and over and over again and we will never ever ever succeed you can't it is impossible to succeed managing your program based on abilities because they will never go away even if they do you are managing towards zero like the end of the day your most effective security program is zero not the number that I want to hear as an executive a man we're going to be zero some day but we're really far as negative right now how our kid thousand nine hundred forty six for the knee I don't know okay so so then

this year we get together early okay we gotta get out of this motor abilities I'm having a real forum right so we figure out like somebody was probably sitting down on one day and just started looking at a crayon box and early I don't know how to describe my job and execs they're idiots so we'll just use colors and pretend with the second grade if you like do you know what that is bill oh yeah she know the red is bad yeah okay we're going to do red you don't hear like the work of never going to show you what are you going to show you that stuff still okay you have to yet you have to okay cool

and then they're like can we do something collaborative into lecture would you have a blue team and a rich we'll call it purple teaming Lester a put the woods where we like attaching to his Barney complex like hey men and we love you it's all good perfect eyes where comes in Industrial Security it doesn't seem good so we created a pentesting thing to try and start whittling down the lack of real with reality that were in vulnerabilities and created is like ice cube to thing the problem is that none of us could agree how to look at how that meant notice I mean like there's a lot of people things there which is life flexibly scan I'm

doing a parent tab click go straight coffee go like search YouTube for things right so the value of that came out the window right in the beginning because we didn't have a good definition of what we were doing and everybody was like this will be sweet I can make a bunch of money doing the thing all I have to do is go crack a copy of a kinetic so I'm with that guy this crack copy the time let it work out I like it it's great not good because it doesn't really provide metrics back and what we're trying to do if not make decisions for people right our job is not to make a decision for them our job is to

empowered stems make it as in my system a really really long period of my life to do some weird [ __ ] when I was trying to figure out social engineering it's like early two-thousands I went all over the place I studied an ashram in India that I went to you know some real weird hippie it's not [ __ ] and I took a bunch of classes in conjoint Family Therapy because I was like cool if they're this can make you people who are hating on each other like each other again they must really go out and manipulate people so like ash to learn how to do all include I know but whatever they've done

at the time right like reverse engineering permeated with that so I did these courses in one big propellant thing that I came away from was that day the conjoined Family Therapist never made a decision for the couple ever that was the one thing they did not big all sorts of other crazy [ __ ] fire love investing cry the passion to things regret them do all social but they couldn't make the decision for them and it hit me really profoundly is that that was our job our job is to give people the data that they need to feel like they made the right decision and feel good about it at the end of the day when

we put the entomology to work security it is not something that is measurable it is not a construct or honor off it's not something that you can put your hand on or anything else the word security is only a feeling that's it if our job as security professional if you call it feelings professional it is a little bit more accurate I mean how many of you do consulting anybody consultants cake you realize that you're a therapist right like all you're doing is brokering like the shooting time of their day like when you feel like it's okay you're going to big progress I promise just one step in front of the other put all of us do that in the

enterprise - right so it's about empowering people in all of these ways that we tried to start we've never given them the data to really empower them but just force them into the same right so here's what epic looks like for us with our zero final what [ __ ] is wrong and then they fix some of them to the hina fix all of it because environment history and politics servitor so they paid money to prove that their security program was worse than they thought it's somehow we're still a growing industry which just really fascinates me you know like I guess you know do grading as the service is where we sit in a software development world they

call those things tech debt right if anybody's gone through this silly little chart for right like it has negative value and it's totally invisible to you it's technical debt so what I think is that the way that we've been testing for a really long time it's a fantastic debt every time we go we test me finding that [ __ ] they only there certain amount of it BAM we have debt I don't think that is a progressive way and they can see regressive way for the environment so I just figured okay we're doing at home I pulled that off I would do it but I feel like the like learning curve something that would be really hard for me to get

over okay so started thinking about like adversaries right and then the realistic adversaries that were actually there and then the theoretical APIs trees right the Maelstrom rightly the whole internet is going to kill you what do we do now okay we could show them all of the bad and say these are all ways that the Internet can you kill you and the people are just going to be afraid of water preferably I thought about all the ways I can die in the ocean and we'd never get mail ship I mean the more times I watch Shark Week the more times I don't even get a cool thing it's up there in the water nearly hot Bobby it's tapered

so so I I do some skill trade right like I always try and see if I can trade with people outside of my industry to figure out what I'm doing wrong I know I do a lot of wrong stuff and so I was working with the guys that united in their flexibility their clothes and it was awesome and what a cool machine and if I could con them into like getting me into a place where actually by a giant plane simulator that's a great deal and what I learned through talking with some of those cats is this quick i personal instance looking up pretty much my whole career in doing testing because they said something so simple and so profound

they're generally cool the reason that we have these is because we want to put the pilots in a situation that's really really risky but not lose our pilot or really expensive airplane and I'm like I don't think about that that was what we're really for not emulating attackers we're reducing risk by testing in a fashion is controllable like completely different story now nobody thinks about of the hood is able to [ __ ] lock oh they how much more respect what I've got in my career had I not been like on the criminal instead of like I'm the scientist it would've been a completely different life that I would have had it kids my budget so like okay cool how do I bring

in and decide to this so you first start off and you're like let's just take all the crap that we are to do in reposition it so okay get people who can do the hack thing complain about school pack all the [ __ ] right up stuff tell people if they suck you know add an advancement like being able to meet with the defense team tell the defense that they sup tell me the only guys this up and then you know okay we're close but Nasser probably try not are not a simulation so we wanted to get to a point where we can move into a place where we're simulating and not harming the environment now

being treated like thugs with unease on it actually proving that the work that we do at the end of the day every single day the company is better not worse work so started looking at like how how are things repeatable good start - Alex repeatable but I think they're [ __ ] there's no opinion great ideas on the top you know from reconnaissance all the way down to actions on objectives to see - excellent agree with some of that stuff probably why a lot of those steps are really close to what we wrote in pencil independent dinner right being able to go and step through that possibly every single way that we're going to be

attacking the environments that somebody knows what's coming at them and exactly those how they're supposed to defend against it and if they don't do the job in between them we have so fundamental part of it and really have a problem with his the absolutely [ __ ] ridiculous chain on blue team son right like I was a blue Timur I would just laugh at this okay first detect that's the first part detect where I see them like doing bad things I know I didn't I knew from doing the bad thing and they're like okay there tonight attacks over right like you're be denied them more likely go back to doing that things Thank You detective it's denied them

again create closed loop over with why disrupt like what like we block there but now we're to mess with their ready so for this that we're going to first disrupt them which I know that means with whatever we'll find some with his mom then we're going to degrade then we'd like get on a honeypot etosha to them through like the message again happy bro you know and then okay Jim scrub is that you degrade them and fit them all more valuing the patina like J hey I'm actually Sears call this is what Hardwick who all that [ __ ] and people use this for real and talk to executives who aren't in the same like community is up with odds we

we understand what this really means but they look at this and they're just like what the hell was the other [ __ ] that you're trying to do do you seriously stop an attacker and the talk [ __ ] or like we got them in like really soon makes fun of them and then people whoever you like to see them by like wear a clown suit lesbian like haha I'm not even from defender and then destroy what like you send people to the canal to kill okay lucky really good go back to making like weapons and [ __ ] stay out of our cyber space like we not not how it works the real world so I like I like how

David started looking at these things anybody seen a whole pyramid of pain right it's a great idea it's really good in the thing that you start to see quickly in an environment into identifying hash values is the bottom right if you can't identify a hash value in your environment across all your [ __ ] you have not met that one hard to think about because people are like what I can see interesting but like what you still out there the last I you look back at step one right but if we manage things all the way up to the top what is it do for us just like a flight simulator does for the people in the pilots in the

airplane if we get to do the toughest exercises it starts to strengthen us right we're not doing things that are basic run on achieving basic levels of success we're extending the program enforcing its progress through training through exercise through development into experience alright the people in the world where though I hope someone here is wearing it's like crushing on you the who we're the like there's no packs for human stupidity t-shirts those people to be morons why oh because what's the patch what's the pastor stupidity my needs aside from death educational experience right right stupid hot ah that's hot you're an idiot okay I'm just not going to touch that again BAM learn game over package install right

but if we can do this at the highest level this up underneath here is going to be the easy stuff this is the software we're going to be banging at them so hard to be like really really you're just in the same like attacker at my company ding calm like come on man and like step it up a little bit we're a little bit better than that now like I want the defender since all [ __ ] inserting it deceive and degrade and destroy and all that subdue it these clickers always play like the adapters are probably just going to get more mad and then your thing is going to be in the bees nest into one white stone so

I've been working with mitre and a couple other organizations to try and start quantifying and qualifying the different types of attacks this is nothing so the anybody incest yes okay awesome yes you we used it yet okay good so the TAC is out there trying to say can each one of these persistent to bring us all the way down to command control just like we had to be deaf what are the actual techniques that are there the oil injection crossword incredibly decentered centers being able to make a centralised knowledge repository of what those attacks are so then if we can start testing those types of things we're actually to the point of that tough that testing PDP's we're not just

down at this bottom sign and bones and whatever else we're finding [ __ ] you can actually do you can respond to and you can move forward so our Charter when we are working on the next way of trying to manage these things and not have any more debt with creative talk of badasses who were treated like fighter pilots and not like [ __ ] and then to automate as much of the defense and offense as we could so that then the smart people are training the machines not trying to keep the machine running which i think is our biggest problem right now on the defense side that people are trying to use baling wire duct tape keepo and thing

together when it should be a giant machine that all we're doing is tuning right we should be like formula 1 status we shouldn't be like you know ground in my overall goal was if I could get enough metrics just like the actuarial to predict death predict lifespan I could predict attacks not just an attack I could predict the entire attack chain how successful it would be how fast we would text them how good our defenses are and they'll give you all of those things before it ever happens sound cool okay quick way to do it just just you know look easy this is the workflow that's how you do a family might drop okay so you guys you gotta make a

strategy this this particular strategy the beginning was talking through all right what does it look like when we were doing a pen test right so from engagement then going through intelligence on assessment target acquisition exploitation prevent lateral movement impact persistence XO then we report stuff and you're usually going to hand the customer the report they go back they cry their fists certain tech debt during testing test we come back we'll like see you next year we'll create more debt the next time so like thanks for giving us valuable I wish you probably started working with the teams of this stuff so what are we going to do well I don't want to hand them a report

the reports useless for them doesn't give them any experience all I want to say I was successful with gathering Intel these different ways simply down with somebody in the socks sitting down with somebody who's on the offensive side I'm going to redo those things and we're going to simulate with that pilot this particular exercise they're going to train and tune their defenses just on me in that moment and that exercise and we have a rule set where it has some defenses built that increase our capability that name right kind of report any other shift that day shift in the chain right so then all we have to do is continue around a circle and see all the

other places do we had some success in it and by the time we get to the end of this project [ __ ] will be different in that environment in fact it will see different the doorstopper report won't matter because we will have changed the environment instead of going into depth we're going to show positive increase because all of this stuff that I found even if I couldn't protect against it it was now attack item for me to do something about it I no longer am I just handing my report on disability every charge eating bat your [ __ ] and I like back to the talk so I can watch you do it was hey you didn't pass your stuff

I'm going to assume you can do nothing about it hopefully you can help me out I need to go back and figure out how I see what people are doing bad stuff to your day I'll let you know if I figure that out and if I do figure it out let me know who to call or what buzzer - right right so then you can get on the job and in between get it as close as you can now I haven't group it I have to look at coverage right I have to see what do I have in the environment that's actually going to be able to get me there this is one of those [ __ ] eye charts that

I've made but at the same time as soon as you do things of this if you say all right I've got ids/ips right and it's going to cover these areas of the adapt chain instead of looking at it is you know right now what do people do like take the Gartner charge overlay if you like justice how many magic quadrants I have and look I only have six out of the nine magic waters doing a flick my security program needs more money that doesn't help but if I can do it based on everything the attack chain and say do I have capabilities with this thing binding see - do I have capability stopping see - if my fire i-gos hang of

uniform and I screw is that the only control that I have that stops it should I be telling my team hey we need to get redundancy in that box because it's the sole control that gives us visibility into this phase of the attack wouldn't that be a more valuable finding than like yeah WordPress is vulnerable against you no good let it be vulnerable at least on this topic don't they understand [ __ ] like that's what I want to see anyone you'll find really quickly is that you're going to find you that you have products that are tripled up in one area and complete utter fail total gap in another area and you can be able

to look at the program with always we should probably do something that does this here like why would we do that it's not here on the coast of water I feel like we need to get away from that [ __ ] we need to just make sure that works like I don't give it who's the leader is I can be capable with some piece of technology that's 25 years old okay we'll go with it when I was talking about this last night blew my mind thinking about it my Sidewinder firewall that I installed extra guarding bacon for our first internet connection would have stopped every single one of the [ __ ] DNS tunnels is used every

single one of them my 20 year old viral right how come I knew [ __ ] doesn't do that like I just go back to that work so now we had a lot of any hopin we got shared information please really have to share put on all that I carry so that's part of the reason that we started with attack you know Rob and a bunch of other people started with poem wiki if anybody seen that I internal a little wiki that has a bunch of attacks stood up already in it they're like what tools you use and how you use those tools and where to use those tools and all that stuff really cool because now if I can give my

attacker tools techniques what switches I use the ways that I got around things act by apps whatever else any of the scripts that I have any of their like commercial tools that exist there or the open source tools going to write all that out that means that when my defenders have time they want to go to rule writing by the way I'm calling it bull writing because I have a really big problem with nothing but anytime they want to go Google writing they can find something to write it again there you go and run all the same [ __ ] that I love they don't have to call when I can you they can go run it themselves and then

the next time I keep doing a campaign we're going to get caught they were like BAM that's their stupid odd you know like if they're we also my biggest problem with the whole hunting order maybe one of you guys cancer and found some ones I can answer again so please educate me let us be one thing anyone serious I'm seriously asking does anyone know when they say that they have a fun team what in the [ __ ] are they hunted please complex anomalies they are pressing for mysteries they're like [ __ ] Scooby Doo crush it but off the right [ __ ] right more unknowns what's authorized was not right I don't know do you have a callus everything

that's authorized we have and that's what land alone no idea that's what is this is a developer okay I won't offer out that is because all they do is make up new [ __ ] that we've never seen delete them from the network like oh damn it our product says right it just killed you need to hear it I'm not bagging on people who are on front teams you're just in shitty marketing positions you know like you're probably well better off doing real defense stuff and being the engineer that you actually are versus the marketing part right so we're trying to make it to a point where there is something to hunt if I have a

repeatable process and I'm attacking you in a measurable way you have something to hunt me or my scripts or my functions and if I fire 25 attacks that day and you're looking and you find six attacks that day you did not kind of chickeny fine right measure if it's like the difference between like a hunter in the woods and like proposal serviceworker we're like Dondre the woods like take aim at something a postal service wherever we just like rolls in and they're like half of this mall because it's a really bad look for them so we've also made car the minor has been working on this car is the the analytic repository so these are evolved in

different types of analytics that exist out there currently to do determination against particular types of attacks so please you can find it what it is what the hypothesis of that particular attack type is the attack methods the different items that you use anybody seen unfetter yet if you haven't get ready for the awesome so unfit er was a project that broke out of some minor things that got picked up by the NSA purple SiO guys and started becoming a community development projects so that you can a big giant really awesome elfin cluster and start feeding all of your attack data and all of your defense data into it and start coordinating those things so that when I

say I released for rabbits you can say I shall for rabbits and we know that it's right when you say our these poor rabbits you know like I got 95 rabbits to like well maybe you shot a little too much you know like the thing that was six five and 260 wasn't a rabbit they're like blue being like oh I this is a like 10 hourglass by itself but I just want to show you that some of these things are getting made so that you're not paper where it's stuff that we've able to POC all the way from back down to did the defenders see it how well did they capture it at one time to capture

it so we date examples right so now discovery doing that commands you know the Jaypee search with an amazing article about all the different attacker attacks days investigated in 2016 okay and out of those attacker attacks in 2016 they found that more than 90 percent of them use these necklace so gates and I started making the script together so just any late running the next minute because if Jimmy's earth and every fact they had had 90 plus percent of people using these documents then I sure as hell better be able to detect when these things are being run because even if all my other should fails I had a 90% chance catch the example before

they run an acumen awesome okay that there's some noise that happens here but if you start in aggregate count of like Ramnath command then ram this one then ran this one then ran another one the alert goes up really really really bad super-high resolution right simple topics like this are things that create huge amounts of effectiveness in the deep approach then what we can do after that is worth paying how many times begin to count so I take my next command I put it in the GPO I send it to 15 different parts of my network to different machines I have it auto runs of thing and then I look back at my selection I go oh I rated 15 times but I

only got or alerts and weirdly enough my poor alerts like one of them came from fire I one who came from like ask want rule that I had one who came from like some random you know like all I had see a gets involved you have had I don't know but it was still there you know one of them was like of error because we texted to a mainframe you know we start learning all these things about your environment how it actually works serving a task in this [ __ ] gets solved instantly right but you know exactly how to solve it this is the tool that I use this is how I saw it into the coverage

gap or did I not implement this rule set someone directly right so we have stimulate things once we stimulate them we are so low so scouring our attacks will get maturity on the detection will get maturity and protection we get confidence confidence is it just like I built that morning if things like well I ran the thing five times and only got alerted once so I'm not very confident that if this plan is some random part that you picked of our network once it at the time it would catch it so I'm a very low-level confident even though I know what my maturity is will give it sophistication anyway used sticks so things is pretty good for general

specifications for life elite innovator you're never gonna see our cool [ __ ] down to like it's subscription nobody really cares I'll you know it's called this pirates at the bottom like I want to be a hacker and that's before and often like the shell of the hacker so I split this load button over here we get the detection rates then we can start finding really cool things like how sophisticated was that attack how well is it we detect against is how easy could be protect against it how fast do we respond so these are some ice chart so and when you get five particulars looking at them later it'll make way more sense but

effectively it's data techniques like you know a fast recovery figure out what the vets or protection are right catalog those things so that even unlike defense even though I think that one of these things happen go to this sheet and go look in these different products X right I can use unfettered if you back tickle we didn't find this thing click on this here thing if you find one specification is in terms of level of maturity right based on these scores right like oh it's a it's a 3 we have centralized logs you know it's functional but it's not to the point where it's like stepped off the red bar timing is really important here this

helps us predict future okay if I can figure out all these simulations in time I can use iterative math to figure out what it's going to look like a few make this kind of certain so now to take these things that are remapped amount across all of the attachment so each one of these able to stimulate these different things process information you know this games buffer overflows we're going to do a lateral movement once may your love services and all of these different areas right so what am i doing doing unit right basic unit so you gotta get that out a little bit more and try and get an idea of what the attaching octopus death this is what

happens when you stimulate all the active pads inside of that chain and use my Carlo iterations to generate the top 1 million abt dress another way that you do that is just a we have all these potential ways of attacking that we've simulated give me a million different iterations of people using different techniques each time and then bubble up to the top the ones that are the most risky I hate that [ __ ] order the ones that work the best I saw none trade myself and when I find the ones that are going to work the best if my the ones that are the most successful my final ones it would have the least ability to

defend ourselves again that's aligned to simulate first because that's the [ __ ] that's going to hurt us we know it's going to hurt it the fact we don't need some intention to come in to be like ah GPP your domain got your ass man password what's up come here like you know like got it thank you for charging me take your hands do [ __ ] I came with two minutes like that I need those part of the configuration cool partly because we can also other then you got stuff somebody is going to get this before I go to the next block so I'm just here for a second ah really cool thing carry well the

other minor frame arts little cataract music at started suggesting different intelligence reports this is how you can make threat intelligence operation a meaning in I took the maybe an apt 95 or for weapons by acting fish out or whatever they're gone now if I took that report and broke that down to TT pieces not all the [ __ ] wording the clog is their tactic so what you need to show you the ship just what exactly happened from that attacker group I could break their own exactly one of the sites to be attacked by a BG platter and if I simulated all those things before I go into my simulation charts and say T TV

this is this this this and this and go home we've never simulated that one quick let's go simulated go simulated go back and I can tell you probabilities of that group attacking us the effect of this level of that group attacking us and what time it would take for us to find that particular adversary groups Wednesday attacking my virus how cool would that be to tell an executive when they say how vulnerable are we to a p228 you're like I have this level of confidence that we are at this vulnerable but we will catch them at this one of their - they'd be like damn you actually knew that oh [ __ ] you know

Lee let me go tell my other detective or say all right I got my dog boy if you got this anybody figure out what this means of life oh yes apt so life that's organs words we want to get to a point where it's so gangster that they can't mess with them or when they do it hurts really bad alright lastly defensive measurement once you get all this up and if you can predict the future well you need to do we need to figure out how to make our defenders better we need to make it less drag we need to make them improve in offenders we need them to be able to measure themselves if you gamify

a certain set invitation thing if they're just trying to get up to their hundred percent score or whatever she needs to be able to see how well they're doing against these hackers you know not only am I going to tell them hey if you successfully hundred five out of five rabbits today I'm going to say if you kill five out of five rabbits faster than anyone else in the world and there it was like she said so hard like I had so much time I give you something else can you get a given extra step so that they understand how to improve their own process we can do that by searching just for what they're searching so person

eyes or gums for a while and implementing these things live both at Boober and Exim organizations where we're providing them a full service Red Team all day every day building these models for them and it's cool because we can now go in to one of their agent engines and saying I want to see all of your offenders and how many of them searched for this string or this IP address or this artifact their bio see that I know I left there and set up all my search capabilities and wait and see in a real-time battle board how fast each defender is responding to me and where they're at in the syncing process so the ticketed be able to look at the

ticket timestamp be able to look at Wednesday me to search for my thing that was bad and show them meantime the detection without them ever having to give you a report so cool because I can actually tell them how well they're doing is that they can look back in it and be like damn it that was when I went to lunch and you're like my problem like when it's just launched maybe the next time that you get the attacks you'll stay in it to the next level or fix it when you see these things you could start measuring them over time and then as they get all of these different pieces of and your analytics you now can

create the thing that every executive in the world walks which is the dashboard people [ __ ] love that's not back to the portability thing because they love it so much you got to give it to them but this one is different because now we're talking about every term we run one of these simulations all of these metrics go positive they don't go next right I can say that because we're staying in it and they want focus on oh we think we're going to have more nations they crave our focus on increasing the ability inside of this give it Academy right I can do things but show the maturity over time in the break down I can show them how many

different techniques we had in it what part of the attack chain are we the most capable right like if we're focusing everything on a situ like okay well by then we probably lost all our [ __ ] if we smooth it a little earlier what if we have stuff really early and have soft after that then we're hoping that you know our magic M&M firewall stops everything and the half of that were like well if they still should at least be Drive you know but all these things show them they show them coverage they show them really what tools are relying on because we know what to opens up that particular alert or what analysts is

using what's role and how well can effectively but we have this great simulation but really to me it solves the problems that I have had my entire career at communicating probably at this charge no the problem is to me you have this idea of total protection detection response right fully capable kicked in we win concept right whether that's an app whether that's a campaign whatever else you have the potential the potential is like well in that coverage matrix we have everything covered versus we have some gaps in coverage maybe we can't find you know maybe we won't have neckla and maybe I have this kind of data that data so there's some attacks that we're just

not going to see because we don't have the potential to see it but the real thing that I wanted to see was what our actual capability was even though he has a potential to see all this stuff do we wait did we not did we not see it we not execute our full potential in the environment because we were at large because we didn't give a graph to the alert broke because there was some logging engine things that look whatever all these intrinsic problem things are did we actually execute at our capability level or do we have coverage problems the thing that I found really really awesome about this is it made my case for something that I have not been

able to measure my entire career which is my constant velocity of police stop buying [ __ ] just please stop mind stuff if you buy more [ __ ] it's not going to help you so I tried to walk on the path of if you buy more [ __ ] you have a larger attack surface if I put antivirus on a machine technically I have more ways to attack the machine not less they added a new vector attack right simple but they're like don't book you okay bye let me I've gotten at this point we were simulating these things that I can prove to them without how about that if this was 40% and this

was 20% if I added something new into the program increasing the potential I actually made the program works I approved to them on a piece of paper a graph they can see and they can read the actual states here and our execution gap group because you Bob your chip that to me was one of those moments in my career so far whereas I felt like I found the friggin rosetta stone but being able to find the explaining to them there are ways to fix this that will actually work that is not in you going and buying more crap I know that sucks for every vendor in the world but don't worry they'll come back and

buy [ __ ] right I'm almost done waiting so the future of this to me is that I don't want to continue to play this front that like where hackers and hackers and and we we do all these things we simulate the criminal because it's [ __ ] we don't I mean I don't know how many of you have gotten permission in a scope so I grabbed an executive and like thrown a flight of stairs like could go to their mouth like I get your [ __ ] cuz that's all I do the crew I wouldn't do all the fancy hack and things that beat the hell out of them and like throw my trunk and let

them sit there for like a day and then pull out be like you want to your kiss-cam - I'm sweet so you're ours they took a dog killed a kid just like I'll tell you what the types you did right that's criminal like we're trying to simulate things and we need to expand our scope of simulation and get the level of respect in that skills Convention so that we can be an instrument for making defense better we can be a flight simulator for the fighter pilots that need to be up there protecting us all day every day and we need to lower the level of risk that those people have in their training scenarios Oh questions we were to stop with that

it's just I feel like feel like we have a big opportunity to change the message and the more we change the message the more respect we're going to get and the more respect we get the more fun [ __ ] we get to do thank you