Rockie Brockway, Rick Yocum - Att&ck Path Effectiveness

all right again to do that coming you're pretty right guys our next talk drachmas all right I'm rocky I'm the practice legal offices CSO at trusted center and I mean you know Tyrone Taylor with a monkey I've been uh I've been doing a contributor security risk penetration testing etc for 26 years I make a lot of

hay I would give them doing insecurity like stuff for about 16 years six of those and consulting about ten of them running security or programs worked in practically every industry of shapes and sizes and for trusting something and at any point infinitive my parrot oh yeah let us know yep cool all right kind of a high level high level agenda here right up it who who's familiar with both the attack sorry but is all about basically now we have put together a frequent in systems report using them under attack recommended so talk about what is really at the end of the day of this illustration here the top growth that represents tactics and the way we're

using the formula is basically you know and again at a high level this component of Margaret a query work is documenting all the known attacker techniques that are out there today but no attackers engines and they're broken down into individual tactic who makes those initial access execution persistence the second so that's that's one component of the framework that we're using today we're gonna kind of you know do some credit where credit is due from inception respect and then we're going to talk about the evolution of you know we really started with is where we got to pay coverage what coverage actually means and that were effectively sexually it's because they're two very very different things

we're going to talk about some Santa Claus in same outputs of our model we're going to discuss some considerations that we went through over other new over a number of glasses of wine when we were really building a lot of this stuff together and situational client and reporting construction and then where work where we think we're taking this mess strap in okay mostly gum no lighters not proper organization came out of MIT they run the evidence or policy entries and manage federal R&D centers and the attack framework they developed in 2014 in addition precocity so what it is is my base referential index of techniques to spy item series and in all those techniques that are mapped to tactics

and data sources and platforms data sources is going to be really one of the key elements here for the what powered using the framework for this kind of system our assessment area there are also additional components of the attack framework which include mappings of no threat active groups what industries they are actually targeting and what techniques each of those groups have been known to actually exit in the wire all right so that also gives us a whole other kind of really interesting said they didn't play with from from an adversary's if anybody okay it doesn't know what it is take out so anybody can do since this is an open source free market everyone you can use

it to understand gaps in your current pulses right in relation to no attacker techniques you can also use it to illustrate redundancies and controls right so as an example we did we did one of these assessments four or five bars and to do the size of that company and there were other you know three different business units that actually all invested at different al beach systems and you know we that would experience rate that there's a significant amount of redundancy in terms of their control sense just by walking through this the attack for Amanda mattered all those those tools to those techniques you could better entertain the like the go the technique to be addressed like tools and processes

right I'll changes some people process technologies will impact effectiveness and again what what are we defining is coverage and we're really defining as effectiveness because the fact that this is really people right and that's something that they're just better protect for immigrants home isn't taking a look and then the most efficient ways to services ok so he's at the inception here the 2017 the attack framework to added profiles like I said Fred a caboose and that's what kind of triggered my interest in it in the attack crater itself because we do risk assessments and when we do full-on risk assessments we're taking a look at really or for face was a risk assessment approach the first basin is where good

leadership to understand what critical business systems are in the organization and under state law specialness right so look at what financial impact let's say the severe financial impact what number would that mean to your organization based on your annual revenue and business model that would put you out of business now let's work backwards from there and trying to see what's a significant if I mean laws over the local I came to Alaska so that's phase one phase two is our analysis right what are the motivated threat actors out there that are what are the products out there that are motivated enough to try to take your stuff right and so that's that's when I started looking at me from

the threat activity of the document threat everything is now this is a resources unknowingly source but it's a resource words I can say okay I'm working with their organization they're eating let's say the oil extraction is so how many documented threat actor groups are have been known to date to be targeting the oil extraction industry now what are the techniques that they they seem to actually use in the wild now that's make sure phase three of our risk assessment which is penetration testing is targeting those business critical systems that we define in leadership you see those active techniques by those and threat actors and that we could be capable of moving impact the space or

electricity systems so that's where I kind of drawn into this attack from that same year earlier Vedas warlock host pair articles on the threat level this is really the lightbulb moment because what he did was basically use he wrote a an API in PowerShell that can connected into the attack McKee and just download the entire thing into like an accelerant right so then he took a look at and I'll probably be the next lot but basically there was people goes and Excel to map data sources that are associated with the individual techniques he got put into the framework and in doing so then created away from a threatening perspective where he did to kind of test

the teams to ensure to see how well they're cutting their threats well that's really awesome and you know we took that idea and applied it to this spring so this is a sample technique in the attack prevalence up right in this particular instance it's tol cycling what of it also there's a description about one of the really cool things here is data sources now basically what that is is if your current controls are looking at those three data sources processes of network process monitoring for the DLLs they ignore her controls should be able to detect dll sun excessively so that's that's where that's really where this you know kind of started with all of the

place of us now we can basically work with an organization and say okay let's be tutorial on your controls for each control let's figure out which of the we did the attacks right I think they're our favorite they never something data sources right so the lower levels processes a network responder his fifties some of the news and it back those data sources to your actual tools but there's a pal out the firewall CrowdStrike EGR whatever right and that should then give us an idea of how well your tool sets in a perfect world scenario should map against each of the 223 individual node attack techniques to make sense every question is cool this is a example of that kind

of that motor side of the framework which is documented for an active groups so this is inspector 3390 they have some aliases over there they go to office 2010 and has targeted organizations in aerospace government defense technology energy and manufacture right so that's really interesting data going to use them mm-hmm it also lists each of the known attack techniques that they have been seen to actually execute a lot so we can use that to talk about racism

okay so that's kind of cool but again it's really kind of a perfect world that's a perfect world snapshot right that's what we're calling coverage and coverage assesses is the tool set has the capabilities to address each of those no document enemies and during these assessments Plaxo so here the sitter's of snares you know the example we were doing a covered assessment for one of our clients and they're pretty sophisticated we're pretty advanced remote one position security perspective and we gave them kind of like okay there's a book you spend a week or so companies working with their to you citizen but all there's just a ridiculous amount of control and at the end of the day when the Shogun kind of

like like nap hug like they're just a it sells you right here's a 223 attack techniques and here's a heat map they're just like excuse that doesn't tell me doesn't tell me how good right so that's why in our blog and started thinking about the school so I'm gonna take it so so we had a little thing right we the uh so we have no thing to figure that out so in our hat beam up if we have all these tools in next we index these tools against these data sources and the mitered for anymore as these data sources of next attack needs great and get a binary one to zero as to whether or not we have the right

to protect and you can see some cool stuff like that like whether need to buy more tools who's enough gas but what they didn't know was yeah we have DVD arson we don't know how troubles less big difference whether or not you're ET arson is going to catch something it's different than whether you're solution completely person whether not something we're attached something is based on a bunch of environmental barriers so those variables are is it deployed up if it's deployed everywhere okay a higher likelihood it'll catch them if it's not deployed everywhere what's the liquid time from an event happening to alert true if that's a really long time frame your likelihood of protecting something in the time

frames you want what's your how are you Monday and all these things what's your staff so we had to build something to assess these variables for each tool to figure out how not only a tool if you have to cover charisma but whether or not you're using that to the web from an effectiveness person so the term we went on obviously our coverage this and just to help illustrate we have some of the differences between this whoa you're right so cover just by its wonders here you have to cover technically not back to this is qualitative it's a one to five attendant you cmmi sword so we're currently going with one to five on this but it's how

good are aggregators so if you have a sim for instance a place where all these logs are aggregate should that play the coverage well probably because in aggravators are not actually detecting wanted a source its consolidated right but should play go back in this palm of the cert because it's definitely gonna that how will your correlating day instead of to get an end time how quickly things are happening and you're trading on a set or a certain staff and restraint lesson could be very different that they're training long the tools that are actually doing the detection so aggregators don't impact results of potential versus actual likelihood we cover that a little bit already so what

do you need to do these super cover assessment huge blessing right you know this thing the tools you need to not it's dancers it's perfect this you need all those barriers you need those variables for each tool that your clothing and you know coverage really focuses it's a total bases sense gaps are about some tools effectiveness is to a lot more strategic in terms of coding the staff understand that we need to change our policies around how are managing so some of these pictures for example you know this is what our total spits out there you can see what ones and zeroes on one side for specific techniques have been affected they both address protected and detective tools

but we do that separately we'll run up run assessment for protective controls and assessment groups we tended to control the Schism you try to switch that into one assessment probably waters know if the active liquid smooth words together so you can see where there's true gasps nothing protecting with defect against but from a tools assessments respectable brother said mom this is also really kind of look at the

new can figure out how to communicate Hey so what is it so this is how that works who looks like Excel perspective you got your systems down the far left side got mother is the top whether the detective or protective about platforms the study the yellows tell me for poverty system so the bottom then tool mapping fun severs have to all these individual data sources for the fifty days but beg again forged effectiveness assessment you can see something whose the additional stuff you're gonna want to love things like what's the scope of implementation here update what's the overall available all that good stuff and then if there's an aggregate or everything as well sure this one score there were schools as

well vii because if you have firewalls that are you know covering Auto Center or is Asian but coming in under the spiral for going to your cell you're looking for your scent for all your alerts okay when I touch so blend all that stuff in in the really the meat of the assessment is for our perspective talking to subject matter experts what tools do you have in how you have been using right different organizations we use the same tools and loudly differently sometimes so the same tool even though it can do ten days versus twenty eight so it's every data source but doesn't necessarily mean that they're using it to seem all those days you have to do that assess that analysis

for every organization of those so we come in knowing by and large works how what they're actually able to do with the stuff so now we're spits out this amount of stuff this is a heat map that Robbie was talking about fortunately be balanced again it's a but specifically disease without much of these services does active effectiveness for lens which is different than protective effect in this room it's different than preparedness for related so ultimately set up a bunch of graphs across platforms across protection interventions that can be done together and you can see this a whole bunch of different ways we also spit out a bunch of metrics just do marital metrics or executives that it might think just

India know are we failing are we great eighty one number two tell me about security people this room probably though isn't really a thing but you've been start to approximate some stuff some in a high-level anyway to understand well if we want to be at least of back at this 105 scale we want to be at least at 3 and the very times the initial access key that's maybe 8 or 10 tactics are technical techniques and initial access to the marker your a top-grade where 80% of where we wanted to be you can start to get to 1 there and these are really high low you can talk about how far the gap is from where

you are need to be and start to have those conversations don't see what we can do so much it's just this beautiful go back one of the one of the other things you know that the media are working on is putting together a database of all of these different tools and one thing the sources going back into those tools right because right now it's a bathing process we have to actually you know you don't know inherently what that particular tool is and kind of like go across all the data sources we have to actually do some research kind of particular out but we're you know as we are doing were actually building that database out so

they have a lot of this stuff and it's easier these layers helping us out of that because they're publishing tool assessments themselves now the way we get specific reference to it thread group a only uses 50 of the 230 techniques then when they partner how Alto or whatever say ok how many you know if these actually - in that context that our group so we don't necessarily know across all 230 you know across the that they're slowly building it up but it's it's starting to come where that's going to be just

so in spirit

organizations and this seems like the sister spatial are driving a yeah

you know they've got maybe our walls they're just out of date so my question is is or will there ever be application for readers back to small organization ya know a lot of sure a lot of air in water yeah that's a really good question I'll preface my answer with you're totally right but this is really kind of a more mature world conditions for small organizations coverage is probably we're on a start if you know that your work was slow it's like the organization's and they don't whatever but very light to gauge that you know what's a little dribble so yeah I mean baseness covers probably because you know they don't even have a secure on TV right it's an

IT diamond all those spirity that's doing money right realistic because we're we're talking with students to get all this data they're strapped with all this data about do you have enough people looking at their stuff

absolutely yeah and the thing that I've had the Rockies answer is so the way we approach this particular small medium-sized authorizations and the way competitive set the spreading of we have it was not for us is don't talk to small IT shops about deal outside they're not chances are that's not their day job they're not gonna understand talk to them about the tools they have because they're gonna understand the tools are place and you as a security practitioner make that translation between oh you have this tool okay you can cover DL outside living right you can use those data map then the data source is a shortcut to quickly figure out you know what tools they have a place in how

about maps of how it's going to cover to the 230 sometimes

all right so what else for you so this is something we've just started so the car infected this stuff over here another question that tends to get asked a lot by leadership is hey I'm in the financial industry what are you worried and we all know the answers everything right I was all agree that's what am i spending but not for too much all these things right so we started saying okay this all the information is must begin out of the threat or they're an actor who've an industry stuff a mugger because it is not the tenants and let's look at it my industry my industry basis to spy my awesome it's right if you're

the financial burden what are the threat actor groups it might or what are they tend to target if you're in the extractor extractions what are they then funnily enough these techniques are easy technique so there's a heck of a lot of overlap between the various industries but it does allow us to ultimately sort of start to answer that question particularly for organizations law oh my gosh I have not been where should I start okay well everything's important but maybe you start here this is known to be used in the why against organizations and again actually reticent here is a huge part of how eat people is people it was partly together this stuff is not

actively quite tied together in a framework form that you can just pull up right so Rick actually took a lot of time and remember our tourism the slide will protect verbal description this Fred actor group is targeting manufacturing comma extraction comic that's all I had work with so yeah so so long my in the wintertime the stuff around techniques and tactics and day so it's pretty sure they've been banging away at four by six years it's a pretty good place like the last update six months ago I'd like 15 times it but it's pretty good it's very consistent the stuff on threat actor groups the industries they work in what techniques they use directly versus what techniques

are associated with software that they they use that's a lot harder to pull out on in that in some cases it's it's buried in there but it's buried in for these next two deals with some of them so you can't pull this stuff out of my ear we have nine it will take some lab work to do today that actually becomes a funny story because we were gonna build one of these assessments and attack company which was annual first inaugural attack conference was back in October and that's when they decided to update the employers birth room and add like 50 or 60 more attack techniques so we were in the middle of an assessment and I

went to go look up datums like Oh

without the big wine this but it makes something like this where you know if you're working with an organization that don't know where to start okay let's start with tools that have to look at your data sources that are at least looking at these major things from your industry or even in a higher level say okay well let's start with the tap okay let's start it let it work concern this discovery here we got a lot more red than for the destination although there but you cannot start to have those conversations and figure out okay me to start somewhere let's start with tools that generally do these types of things oh so we thought about a lot of steps

and wonder that's - then don't make are saying these days so some these were talked about you should protect it tools map through data sources one of these things would be scope us over detective tools obviously media navigational if you're detecting against certain types of data and so clearly mess it's a protective tool however the their assessment is you probably can use data sources and include you may as a shortcut right so if you need to get this assessment done to start the ball rolling on something probably get away if you're already fairly light on that maturity curve if you have executives that are going to drill into every detail however you might want to map

protective tools directly attacks it's going to be a lot more work 230 some mappings for every tool as opposed to fitness but you're gonna get more accurate information right what variables are meaningful indicators of effectiveness we went through a bunch of things here today we ended up settling out the ones that you know very show that blue area right so in terms of like updates and as matt has right how well are how well are these your team members all your team members act actually we like specific tools right you guys really good on the panel out of a baby or carter black you know maybe training what about town constraints you have another people

right because one of the things that is just you know across the board heavens and every industry is hey boss I want some budget for this new tool oh I forgot many people did one was 200 yeah you know and if you decide that this is something to build on your own right this is our framework you know we kind of like me appropriately we thought we were about to make decisions that made sense to us but we don't make sense together you know is a tool only as effective isms we just very yeah so the mechanics behind this start to get in Houston right so if you're doing an effectiveness assessment and you care

about your do we have enough step or I rate that one the file on apartment staff training I'm going to write that one five can we do updates it did we do this now all right so now I have you eight variables across her over Karbala and there's some ones and some reasons the box what's the aggregate score for that tool isn't it an a breaking he said Louis of all that is it the highest of all mechanically we landed on it's coming right if you have a camera in your house but it's not watching all the places you wanted to watch it that's great if it's on all the time that's great if you

thought necessary if you can see underground everywhere it's not watching it so it's going to lower the overall score for that so the way that we treat the scores across every individual total we say well we're gonna take the lowest score all the barriers but now when you take a step back and I have multiple tools looking at the same thing does the same you can even plot because now I have a carbon black and a lot so they look at the same technique and one of them is a 3-1 was to five and I still only agree actually I probably was five because it's a carbon black sub I love this then I'm gonna be able to see as I

have multiple things alerting it's gonna take advantage of the best to learn right so your mileage may vary on those mechanics and when the assessments that you're performing a political environment that you working you may not want to show wise across the board for everything just that's how the mechanics work everything that you know I think the best scored for all take the average there's potentially something you said the defense in depth or you can say the average score between the three to five might bring down your score because there's a lot more noisy because of our staff maybe you know there's a lower back multiple tools at different levels detective people might actually make us

a lesson so the mechanics are going to matter you're gonna think about that when you're older generators black power guitar that early and static it's very interesting one for us so so one of the things that I think it's stumbled additionally very soon there are fifty some data sources we we initially started asking the questions okay how well was here handle how well pump up goose your team around that's a process on the outside work right well the cycle that would be individual take not marry but but the data sources that could detect T a little side of the power you guys have those bases and that was a long question all right because you can't be like that

everybody's like okay what was that to find to find the status to find that businesses so we realized that they already had Matt the tools to the data sources so I got just asked about the tools right how how whether you guys enter Palo Alto fire to pretty good little sweet and you know forward and then we already have that data source manner to that tool already so now we do come reverse engineer this little day and be like okay here's the here's the effectiveness from a team training and talent perspective for this tool and that means that these data sources are pretty good and then therefore all of these techniques in terms of easy later it was

such an aha for making these assessments go faster because again they talk to each one a language they understand to talk to them about the tools that they work with every day you don't have to educate them on data sources or techniques or tactics you got the security guys to know now if you get some really interested practitioners Bible means evangelist right but for the most part they're not I do something because they just want to help you

so as far as reporting goes touched on some of these already as well so once you've run detective and protective sort assessments when you overlay them and see some really cool stuff particularly where you might have complete gaps in techniques that are being watched or deputies present that perspective you can see where you have double coverage from up protected and detective perspective or you're just detecting or this perfect and you can really start to see regardless of whether or not they pass get your strategy you can see where you can happen in a raid eg results by the tactic can actually get some cool yeah you know so so once you kind of have an understanding of the tools and

data sources map them to kind of like tactics but gets great then you know maybe your controls are really let's say good enough across all of the tactics but I mean we know I'm gonna have a safe place to kill chain so well you know because of that kind of a concept of the tactical kill chain right the attack chain why don't you just maybe may be either resource constrained you know for a lot of tools and therefore data source perspective maybe you just get really good a few move attackers right because theythey arete eclis if you kind of detective and prevent that chain that's pretty good if you've prevented like through that entire attack so this gives

you insight into where and it gives the options right maybe we want to be pretty good at all of the things but maybe we want to be really good at specific tactics and kind of make those decisions and because of the later attack framework is platform specific with respect to that piece you can actually start to prepare some indicated you can actually start to see in some cases the cost of security solutions by plot at least the supporting and security description in that kidding you have drive some interesting students around we want to continue to maintain the spot I'm Shamim we're ghostly window-shop we want to start reduce some Linux what does that mean plus you can search well

the rest with any of these things you can start to map that out and do some of those what-if scenarios we have less lot more to get more platforms and drive some strategic students so what else we do yeah so there's this whole bowl for our temper we have so that's one of the things we're gonna start with and the mobile attack very much automatically highlight that whisky standings based on the history of Congress so maybe a little flavor of that right but there's a lot more that I think we can do this and that can provide value you know really kind of moral leadership yeah I believe affecting your Devon tools is one of

those things that it sounds really good it's on the slide it's mechanically extraordinary this might be one that we had a not doing all week but the idea is right that will our organization with just automatically spitting out options well these are only this tool is what we are done as long as these free tools you don't eat this as long as there's some stuff you can do mechanically to get close to that doing it that's of challenges and another thing we want to do is overlay you know our penetration testing to go straight likely attachments right so you can look at any technique sort of in my solution but the reality is usually they're part of a longer kind

of chain and and so once you start to overlay the data about how our pen testers have got into networks you can start to see well you know you might have to start from a chain there may be these three entry points but they all consolidate to this this you can stop that point effectively that's now the technique that people rigged the tactic that you care about a little before you might not care as much about and the tactics earlier change so overlaying that would give us a lot of good information around how best to prevent or detect certain tactics by using Botox yeah where we're sitting on a goldmine of information of trust itself

six or seven years of really advanced penetration test angles so you know it's it's metadata that it's not easily accessible right now because of you all PDF reports so I'm trying to see if I can hire an intern for the summer just read through five years of penetration testing results and basically get like the top three to five like you know ways that they have been breached you know but now we have things like okay it's this size organization their maturity is X the revenue is you know this and so we can begin really Gary or interesting different leadership as well and you know a company is financial with a two billion dollar you know annual

revenue that kind of like that that no companies in that little area here's all the ways that they were in your secure so tranches all the ways that they were actually able to increase and things like that so that's that's calling will see see my father

you talk about the

fools is just like you said gotta love them toll but

Oh

great

all right so so in our experience right it's kind of like to size balance some organizations are like that word bomb worse other organizations typically larger and more mature or be able to labor criticized like we work for an organization that looked at our final results and just like that that's too so so however to answer your question it's super super difficult to get you know it's almost like you need social engineering skills to really kind of figure out where [Music]

yeah there's a number of different things that you can check but I mean even if you get better treatment you're checking that box I'll are with a vendor drainage sauce so and I would say that that question is ecology that question actually illustrates of one of the big reasons why to us this is not a replacement for a penetration test this is there's a scent that gives you a lot of good information of the strategic Club but to your point you might go to organization says yeah our teams are great I could talk about manager oh yeah yeah we haven't worn enough people than we need and they all know everything about tools that we have for whatever

political or just sort reasons they think that what you do can test me test that press you might find something they did so these kinds of assessments are fantastic for you strategically where you are where you want to be where you might want to focus resources but it's not our placement time tests that will actually show you what went wrong all right so yeah you know for anybody here is interesting right the biter attacked the frameworks out there is a pretty to use it take a look at it see what a little like this week we've built it interesting as well you know again there are there are tools out there that ABI

and this may be how many questions so my so suitable for so I didn't hear any where do you covered like if you have like prospect or something like that I'm like is it employed throughout the organization no word that's partly that is just like the better that's perfect evolution not about like degrees you know so you had on yes and it's not relations we got on the fire you know let's say functions or something like that right kind of yeah but yeah so if it's about how you're using the tool right in terms of collecting data sources and you get that as part of the mappings but if it's about you know how much the tools implemented in the

organization you get that spark necessary anybody else questions Oh