Discover Analytic Gaps with Unfetter

the besides DC 2017 videos are brought to you by threat quotient introducing the industry's first threat intelligence platform designed to enable threat operations and management and data tribe a new kind of startup studio co building the next generation of commercial cyber security analytics and big data product companies all right thank you guys so much for coming my name is Shawn McCullough I first of all I want to say you guys had a lot of other opportunities for things to do today with the CTFs and happy hours and stuff like that going on so I really do appreciate it that you're here today for the talk on the pamphlets there were two people supposed to be talking my buddy

Mike actually is in Italy right now on vacation so he's a giant jerk he's he watching this later so I wanted to know he's a jerk for just not showing up but he told me beforehand it wasn't like I just showed up today first of all I want to thank the AV guys I caused some trouble wanted to use the mic and so I want to thank you guys so much they've been working really hard all week with all the other volunteers so if you guys just to give him a ground applause because they're doing such a great job I would thank you so much so my name is Shawn McCullough and I do work at the

National Security Agency I don't know if you've ever heard of it yeah I know I know it sucks a little bit but this is actually a talk about what I'm doing in my day job part of my research so this is a National Security Agency project and everything they looked at it improved it and say yeah let's go and talk about it I'm super excited to be able to come here and talk to you guys about it today I promise you I'm not going to go over I understand I'm the last one there's a happy hour I'm between you and happy hour so I promise you I will not go over I'm going to ask

you guys we're not about five minutes from the talk if you could just hold a finger let me know all right let me clarify which finger let's go huh yeah I don't want yet flicked me off and then I'll be like oh my god why I'm so pissed at me so my name is Shelby cool I've been 18 years the National Security Agency I spent the first half of my career and what's called signals intelligence that's the kind of stuff that you see on TV dramatized and the movies where you know the people are in the van like hey turn the satellite on and also this satellite shows up and looks through the car x-ray visions that is

all 100% true that's exactly what's like no it's not it but that is one half of what we do the other part of my career the second half has been an information assurance we are responsible for helping provide security for some of the nation's most critical infrastructure especially community communications for the military and one of the ways that we do that is through something called CAPTCHA red teams blue team and hunt a few years ago when I used to do talks I would have to explain what red team's blue teams are hunts but how many of you all have a general idea of blue team red team hunt teams you've seen in the talks yeah so it you know we have we've been

doing that for I don't know over a decade I don't know how long those teams have been around and I love seeing those concepts coming out into commercial world and people taking advantage of it and learning and understanding and changing it to fit their model it's it's really cool to see that I I was lucky enough at one point in my career to be the technical director for the blue team red team and hunt teams and technical directors like I don't know like strategist or something I don't know went to a lot of meetings but one of my Jobs was to try and figure out do we have the right tools in place to do the

missions because we we would basically ride all own tools you know that's what we was pretty cool we could do that and as a software developer I was usually called on the kind of help figure out do we have the right things in place and my boss would come to me and would say hey over the last month in our hunt and blue team operations what new things that we've been able to do and I'd be like oh we were eight we wrote two new analytics that was the answer like is that good should we have done five analytics were they the right analytics were you know what are we not doing that we should be

doing I really had no good answer for those questions and you know the boss goes away is kind of kind of sad so I thought about it for a while and I said there's got to be something better for us to figure out where we should be and where we're going to go in terms of being able to try and find bad guys that are operating on networks so I got lucky enough I got to take basically a three year sabbatical and during that time I went out and I basically researched what companies organizations government organizations research are doing in advancing the technology of intrusion detection so I looked at machine learning I looked at behavioral

analytics statistical heuristic --kw throw all the kind of stuff at it and I I kind of went into it thinking I was gonna find something that was going to be so much better that it fixes everything we're doing now I I was naive like I thought I was gonna find something and I did not find I found a lot of really cool stuff I saw a lot of really new things that were actually going that would push the envelope but I still wasn't finding the waste how do I know I'm we're doing the right thing how do I know what we aren't doing correctly that was the biggest question we can kind of figure out what we're looking

for but I don't know what we should be looking for next and then I happened upon some research that mitre was doing called the miter attack model and I say you know what that's not what I was looking for but this is perfect so I spent the last two years of my career working with minor understanding the mitre attack model and building tools open-source tools for people to try and make use of the mitre time model so today I'm going to go over what minor attack model is I'm going to talk about the tools that we are building and giving away and hopefully getting some great feedback on and then I'll tell you about what we're hoping to do in future

and give you an idea of how we are able to could interact and get people that to try and talk to us and tell us what they need in their environment see what we're doing is that sound good does sound like a good idea all right you guys ready all right before we can look at the future I always feel like we have to go look in the past so I want to introduce you to a friend of mine this guy's so cool this is Carl so Carl was live back in the 1800s and Carl was a doctor and he was a botanist really and Carl is known and he's thought of as the father of modern taxonomy technologies

an idea of how do you classify and order things so that scientists and other people can communicate about that that's what Carlton and he had a sweet hairstyle it was really cool so Carl Carl was really good at kind of trying to identify every little aspect of everything that that that he was seeing riding him down categorized in building and what he did is he figured out a model for naming and categorizing every little living thing right so you've heard about this in biology class you probably have forgotten it but this is how modern-day everything that is living is categorized and in this taxonomy so this was very important I mean if you think about the previous slide where I

had discussion I'll pull it back up so you can see the limits of my language are the limits of my world this idea that back in that time scientists would come up with or would I discover something new a new plant a new animal and they had no really good way of sharing it so we're gonna use an example so it's kind of silly example but I think this proves the point so nicholas and mike are at a at a party with karl here right so we're all in a party we're at the happy hour with with karl and Nicolson like two gentlemen and they're like Nicholas comes up I discovered a new animal all right this new animal it

flies it has wings it's black it's got claws it's a brand new animal so cool and Mike is like hey I saw the same animal it's got wings it's black it flies it's got claws we must be discover this name animal and they've mean the animals and that science is a sign they were drawn right the problem is Nicholas discovered a crow and Mike discovered a bat you know they only had a small amount of ways to describe the characteristics and it wasn't matching and they weren't able to define it so Karl figured out how to kind of create these layers of abstraction so that as you get lower and lower you can identify the proper characteristics and be able

to name something and then be able to share it with the rest of the community so you've probably seen something like this right so there's a difference between Homo sapiens and are humans and ostriches they get down to a level at certain point there's a there's identification there's some sort of characteristics that makes them different right that it you know you look at the order one's a primate the other one is a I don't know what that it's a bird I don't know it I have no idea what that is I should have picked something else err AV sometimes it was that bad off from there um so but there's definitely there's characteristics at this layer that says

these are different things and they're in no matter what animal or plant or whatever is named you can categorize in these levels alright so that's you know the kind of history of taxonomy how does this apply bats and crows how does that apply to us let's fast forward we're in the present day we're at the happy hour and Mike and Nicholas are having a conversation but with a set of Carl they're talking to ed SCOTUS right so they're having conversation and they want to describe to each other a threat or an attack or something that's going on inside their networks we have the capability of describing some things in a taxonomy that we can in a lexicon that

we can understand so I want you can talk about an IP address of the sea to know that's that we seek beaconing out to write we can we can describe that if I say IP address you probably can read it understand what it is same thing with a domain name right there's a taxonomy there that we all understand it file hashes this is another one the problem with those things is they change very quickly it's very cheap and easy for an attacker to change an IP address or a hash or file or domain right so the but there's an entire industry built around coming up with new IPs domains and hashes and sending them out people buy

these feeds your antivirus your other your their monitoring system that you might have at your network which do a really great job I'm not saying they're not great but they're identifying things that are known in a taxonomy that's easy to understand you're able to automatically take advantage of it and block and it's awesome right it does not help if you're trying to figure out what is the behavior that I should be looking for what is the things that an attackers doing that look different or I should be watching out for so think about this let's assume that you all are the smartest analysts in your and your company that's supposed to be monitoring your network right the the boss reads

this new threat report come up with some by some company that's put it out or a researcher or Simmons put it on Twitter there's a ton of new stuff coming out on Twitter of the new TAC techniques and they take that and you're the smartest analyst so they're gonna give it to you and they say can you tell me if we are protected from this and usually what you have to do and we we kind of know this because we went out and talked to over a hundred different companies and government groups and individuals who have to do this kind of work and this is kind of how they described it they take that research

they reanalyze it right so every single person that's gonna make me take advantage of this research reanalyze is the whole thing and then they have to figure out what it means to them and their environment analyze their environment because usually it's really hard to tell and then they write a report in an email and send it up to the boss at the end of the day tired a they've analyzed that it's in an email that's the results or some of the really good ones we're putting it in Excel write the info sex toolboxes Excel and which is fine it's great it's document but it's not repeatable it's really a waste there's got to be something better

and so we saw the The MITRE attack model and we said I think there's something new here that we can use to try and help bridge this gap between the taxonomy and the lexicon so we can describe more complex problems than ip's domains and hashes and so here's the fancy slide but the other problem is is it's a lot about it there's a lot of communication well that was weird all right okay it's a lot about communication how can you communicate between the net defenders of the professionals we say professionals we're thinking like the researchers you know that coming out of the research these are people are actually trying to figure out how to protect their own

networks or the boss you know that has to figure out where to put the money to protect your network right these three groups are always having to figure out how to talk and they don't have a common language or common lexicon we think there's a better way we are we are building a set of experiments to try and figure out if we're correct in there art is a better way all right mitre if you go to mitre I'm sorry attack might org this is what's on the page threat modeling methodology for various phases of the ASHRAE life for life cycle and platforms that are known to be targeted by cyber threats this is what might have puts in

the basement I have a different spin on and I don't work for mitre good friends with those guys I love them but I kind of have a different spin on it so let me ask a question this is mandatory funtime just so I can understand we're at how many people in this room have seen this slide either you've gone to the webpage or you've been to a brief can you keep and keep your hands up if you would so much for something all right so I don't have maybe a little less than half of the people have their hands raised keep your if you're actually using this in your environment somehow a lot of hands from

there something right in the back there's a good group all right great that's awesome all right so if you have never seen a mitre attacked now you can raise your hand because we're gonna go through it I have a different spin for those of you seen the brief probably having a little bit something different about it if you don't know how if you like it and you're interested in but you're not using it in your environment I'd love to talk to you like to hear what your needs are and how you can how what how you could think you could use this to put in your environment but the miter attack model is a taxonomy for describing attacker

behaviors so this on this page for those have you seen it this is actually every one of these boxes is a description of an attacker behavior this entire sheet is actually just post exploitation on a Windows box there is one of these pages that look similar to this for mobile phones Mac OS Linux Enterprise Services and also that something to call pre attack which is like scanning and initial exploitation and all that kind of stuff so you're thinking about the different layers there's the description of behaviors and then I have my domain right so kind of going back to that original lexicon and then you have this idea that I'm gonna group this by the

things I care about from my particular interest for this case I like to use the windows one because it's been around the longest and I know it the best and then there's different layers so I know you can't read it you're it's fine there you can go on the page it's a lot of data but it's not submit to be read right now but I wanted to subscribe what you're looking at this across row this blue each of those is a what's called a tactic this is the thing the adversary wants to do all right I'm gonna blow it up so you can see a little better it's still not great is it you got in the

back you can't really read it right somebody okay I'll read so this is they want to gain persistence privilege escalation they wanted to fade the defense's credential access which is you know I'm logged into some user but I want to be a better user discovery of what others what else is on the system or maybe in the network moving from one box to another getting gaining execution of my really cool malware collecting data the of interest of course you got exfiltrated out why else would you be on box and then command-and-control so each of these blue things is what's called a tactic this is the thing that the adversary wants to do on your computer

all right now for each of those blue things all the boxes down or all the techniques that have been documented or reported or out in news or research that some actor has done this before and then they're cataloged or categorized under those blue headings right you guys follow me so far I know you can't read it but you kind of get the I'm trying to give the map a little bit I'm figure you can go back and look at later or look on your phones now if you want you probably see it um so let's take I'm gonna dive into one of those and really kind of try and show you how this kind of walks

through so this is credential access I love credential access this is my best friend I love credential dummies this mini cats right so if you go onto the web page attack that miner at work you click the windows exploitation one I think it's called windows exploitation you'll get that box I showed you four you'll get this and then you click on any one of these boxes right here and you're gonna get this all right you still can't read it I realize that but I'll walk you through what it is so credential dumping mini cats here's a very detailed description of exactly what it is and how come it works so well they've identified an ID for it they

have a list of all of the platforms it works on right so it's been observed to be worked on this is a small my screen but like all the different Windows versions that has been observed to work on this is the privileges you need in order for this to work so what do you have to have in order for this to work administrator and system level access right and that here's the person who wrote it if you're familiar with Capek and maps back to the Capek where it makes sense it's another another thing of describing like vulnerabilities and reactions and then down here it gives you examples these are actually the intrusions the intrusion sets that had

been recorded or reported to have used that particular technique all right we're gonna keep going this is where it starts getting really cool this is where I love this here is a really good write-up in description of how you would mitigate this technique here's a really good description of how you might detect this technique and then here is a whole bunch of this is a research organization - so they they've really give it here's all the references this is how we know what we wrote is real right so I love this case you think about it you have the fall you cool alright so you have this behavior right so you have this detail described

behavior that has been documented enta fide now you start putting some context around it here's the attackers that have used it before which is interesting it's probably not that useful for a lot of organizations but if you're a big company in a vertical you might want to know what my other buddies have been attacked with this model maybe I should go look at it right so you know here's your intrusion says here is your mitigations here is your analytics or intra way you might block it or detect it and then here's the platform's right that's a lot of information what's great about it is they've done that for every single one of those blocks so it has

that not all of them are using much details mini cats because in many cats is a ton of reporting but you've got a lot of everyone in these blocks and this is one of five graphs so there's five hundred so there's a lot of detail and information it's that context around the description of the behaviors that I found most interesting I I felt like that's what we miss when we start talking about a particular technique when you know over email or you read the report there's a description of this technique but where is the so what how do I kind of gain that I have to reanalyze it this kind of lays it out for you

but it gets better this is where it gets really cool not only do they have something called minor tech but miter has something called car cyber analytic repository you'll notice these are acronyms there are also words I don't know how they figure out to come up with these things but the cyber antipas story is a list of analytics really is what they are right so it's this this list of and yeah this list of analytics here's like a name and then there's a description of it right here but here is actually the list of all the attack techniques that this each analytic relates to so before we had the minor attack technique and there was a

description of how you would detect it here's an actual way in which you could implement that analytic in your environment or at a generic environment each one of these analytics has description with the analytic is in this case in your environment you know it might have cmd.exe be fired but you probably want to make sure that's coming from explorer or you know something else that's known and it's not somewhere unknown file right so that's the idea look for a command dot exe being run from a really weird binary here's the description of why this thing is actually something you might want to look for and then here is the the attack techniques that it relates to so this

actually Maps back to it so now we have some more fidelity and those analytics right you kind of start seeing this map created but this is where I love getting cold here's pseudocode there's a description of the actual analytic in a very generic sense and then here is a data model of the data and the action that you need to collect so in order to figure out if command XE is being launched from something weird I need to know when a process is created when the exe is created and who its dad is or mom astray so the parent and that's all I need and I've listed it right here and they built a data model they also have a

listing of sensors and what sensors can pull up and you start putting this in those boxes now you can start making decisions based off of this data if I enter at one time so in case the point I was having a conversation with them and a new version of system on had come out or what version was and they were looking at hey this is they identified all the things that system on collects and they said hey I love out of all the LAN oolitic sync are what's your coverage was system on and they were able to use MediaWiki to actually pull that information up and they can say 80% of all our analytics used system on that's

probably a good thing to suggest for these analytics right you can start making actual defined decisions about your environment because you had the context around it that was the idea and it's for every one of these and they just you could just add to the list so we really liked minor attack we thought it was very interesting it's a little blighter on my screen but we decided we looked at and we said you know this is really cool but I how do you use it that was kind of hard a lot of people had their hands up they had seen it but they're not used in the environment sometimes it's hard to figure out how to use something like

that so we decided to try and come up with a set of experiments to build tools open-source tools that are free to the community that takes a person through workflow so that they can get answers about their environment they couldn't before with this kind of information that they might be able to make a decision about their environment so they can better protect it or they can try and mitigate something they weren't doing before that's a set of experiments we have hypothesis we try and find people that are interested in trying it out we gain information from about how they work in their environment what would work for them and then we build a set of tools

for people to try out and they tell us whether it works or not it is really a set of experiments III would I will tell you that what we're providing if you want to take and you put in your environment that's awesome tell us how it how it works we and how you want to use it and how it might be changed because that's constantly what we're trying to do I I'm not selling you a product right so do we are it's a set of open-source tools and I'll kind of walk through it's kind of not as interesting that I I guess the main focus of what we're trying to do is take that context of my tertex the

framework with all of the related data somebody adds their own information about their own environment to their an instantiation of it and can they make a decision that's that's the big thing that we're trying to ask so the first experiment that I key I started creating with something called my Durant I'm sorry unfetter analytic and that is the github repo that you can go to just so you know everything is on WWF Federer oh so it's all right there unfit er analytic was really this idea let's take a core set of data beta basically web application or some other kind of tool let's make the relationships we all saw in the miner attack so every one of those specific

attack techniques the adversaries are the throw actors that have used them the tools that you might use to actually perpetuate that attack this is could be malware something like that the analytics and then of courses of action or sometimes we'll called mitigations create these relationships and then see if we can make decisions off of it the first thing might our analytic the whole idea was can I quickly stand up something in an environment run analytics through them right these car analytics that we saw in the previous page get an alert but instead of just seeing this alert alert 25 hi you know that's you know sometimes that's a lot of times with socks are seeing but put the

context of why that analytic is important map that analytic to the attack pattern it's trying to attack or the the mitre attack a technique that is trying to detect and then from there you can quickly look at maybe how I mitigate or other people that it's been reporting provide those kind of relationships to it that was the experiment that's what we wanted to first try out so I basically built standard elf stack on the right hand side here is actually the server so built a basic elk stack on top of it I'm running Apache spark and then we converted a whole bunch of the car analytics into high spark to run the analytics implementation is important

less you're actually interesting going to play with it but the idea was everyone of this analytics when they fire they fire and they bring back to information but from the miter attack model and they describe it to you in the Cabana display so you can kind of see it you get the so what and that was the idea we also set it up with it online you can it's all in docker it's two commands to run the whole thing and then we provide the index logs to move the windows area reports and the system on i'm not windows airports i'm sorry the windows event logs and the system unlocks over to the lock sash with the

Analects and everything's tagged correctly and it's all kind of done for you the idea is for people to try it out and say does this make sense in you for you would you use this in a sock this kind of thing not not this particular one but would you use this kind of thing does this help you that's the kind of questions we started asking people and we got some good feedback especially from people who are into analytic development they say hey that's great this is actually additional context they could divide but we went and talked to those hundred and something different organizations over a course of seven weeks I think is what we did and they

were like this is interesting but we actually don't know what we're running like we heard that a lot like we're not quite sure what's where our gaps are we're not at this point of doing analytics so although experiment was interesting it wasn't the most important problem so this is just a displayed which doesn't show up anyway so doesn't matter so the next thing we did is we built a second set of experiments this one's called unfettered discover and this is a web app look and the web application it's all open source the same as the other one here's the github repo right for it the the whole point of this one was to say alright you know something about your

environment or what you care about so put that into the system and let me be clear this system is something you download and run your own environment we're not hosting a version of it for anybody so you run in your environment and you put in things that you care about in your environment that you know about and then you start seeing where your gaps are compared to the attacker techniques that's the basic idea so whereas before with unfetter analytic we were really looking at that whole indicator and so what with the indicator we started moving around the shield for the other parts of I hit the run button sorry so the other parts for unfetter

discover we started trying to take advantage of these other relationships alright so if you're trying to figure out where your gaps are in your environment you probably know what analytics you're running or maybe you know what your mitigations are you know in our case we use the top 20 critical controls is anybody familiar with the top 20 critical controls a couple of few of you okay so we use those because they're free available for anybody to go and look at so we implemented those in the system and we create a relationships and of course we use the rest of stuff from attack the miner stack model and so we built all that relationship into the

backend system and so our idea is this is our hypothesis our hypothesis is that if you know about your mitigations that you have in place and you can rate how whether you're doing a really good job a really bad job with them you can see which mitre attack technique you're at most your risk app that's the that's the hypothesis and we believe that if you you rate your stuff that you know and you see your problems based off of what the attacker can do and you can get see what the history of that technique is and how it's used and how you detect it and mitigate it who's used it before that would be more meaningful to maybe

you or your leadership who has to figure out what to do next that's our hypothesis so we built experiment around it experiment and and I kind of scribed it but it's a little weird when I show people how so I want to make sure that I show make it good do a good job here so and I'll say this I probably won't do questions at the end of this because it's the last one and people you know kinda probably ought to go but I'll stick around afterwards and I'll probably hang out and happier so if you have any other questions please come up talk to me I'd love to talk about this stuff obviously so I'd love to talk

to you and hear your thoughts of where we can improve it or what you think works really well too we like we like to hear that kind of stuff so if you are doing a pretty good job of patching your software right then you're at less of a risk for exploitation right I mean that I think there can be a debate on exactly what what this weighting is but if you're patching really well then you're you're probably not getting hit by a lot of exploits if you're doing a really bad job of patching then that's that's a risky area right that is something you probably want to focus on all right we'll look at the converse of that you you are doing a

really great job of monitoring your network security traffic if you're monitoring your network security traffic at your boundary you'll probably catch really weird remote desktop coming into your environment right or you at least have the tools to do that maybe you're not looking at the right dashboards but you're you at least have the ability and that's probably a lower risk that's the kind idea does that make sense does that make sense is that confusing anybody right now it's okay that's really no good okay it confuses me sometimes when I would talk about it so we're trying to figure out the best way to explain it cuz it gets a little weird when you start looking at it so what we do is is

not rocket science on how to do this is it's not designed for some way to plug in their environment scrape a bunch of data and get like the the gold nugget of exactly what I need to do you know that the golden piece of thing that says this is the most important problem that's not the idea we want it we have to do a little bit of work and the first thing we do is we actually go through and we do do a survey this is all of the analytics I believe yeah this shows all the analytics that are in minor and a mitre car right so you go through and you basically say whether you are not

using that analytic at all or maybe it's logging but only at a local system it's not being sent back anywhere your centralized logging but there's no alerting off of it so it's just collecting a bunch of data right or you're alerting but there's a tons of false positive so it's probably useless in your sock environment anyways or yes it's awesome it works every time right so you kind of rate this and you go through and you rate all your analytics when you're done we start showing you not based off of the analytics you I'm clicking to make sure I don't follow but you're not getting shown what analytics look like you're actually getting shown which of the mitre tack techniques

you're most at risk at you know before we looked at with the Stars you answer one question about one problem you see the answers by the techniques that's the idea so to walk through because it's unreadable this is actually the blue boxes across the top of the mitre attack chain right so this is the group's and we're only using low windows right now the windows exploitation ones this is your overall risk Green is good red is bad and then these are if you hover over you can see it kind of see a breakdown of your questions if you click any one of these blue groups you get a list of all the attack techniques those white

boxes going down the column and you get the rating for those this is all of the tech techniques that there are no analytics associated with that's probably one place you would want to look right so it kind of start telling you not just what you did on your analytics but the so what that was the idea you can click any one of these minor attack models and now you get a deep dive of exactly what that minor attack technique described we basically pull all the information and then the answers you the quite they all the answers to all the analytics we do this for the mitigations the top-20 critical controls if you ever go look at the top

2000 controls each control has between 5 and 15 sub controls and we ask very detailed questions about do you have a policy for this control have you implemented the control has a control automated and is it reporting back and there's like a 1 through 5 rating for each of those so it takes a long time to fill out but you get a much richer information about your instruction infrastructure if you do that and then we also do it for sensor so it is kind of generic because we don't have any products in here I except for like 6 months this Montour some free stuff but you can create an add your own stuff and you can say I'm running

this kind of sensor or this kind of detection system and it's running on all my critical systems and that thing detects this attack patterns it'll understand that I don't tell you what you're doing how you're doing the other thing what you do is you start rolling up remember we are very interested in trying to figure out how to take all this data all this context will information and be able to share it between different groups not just people in the same team but now we're trying to figure out how do we take that and roll it up to the people who have money right who are who are you're going to and asking for and say we need help in this

area and they're like show me why so we're trying to figure out what's the best way to get you to show so that you can show why so therefore each one of those boxes and I'm not doing a demo I thought about the demo I just try today but when I was a new hire I would had this big demo for like the boss like three levels up and I had the system running and I see here coming down the the cubicle aisles and I turned back in my computer blue screen like right and so I'm freaked out about demo so I don't I don't have the demo but it does run on my laptop pretty well but you can click

each one of those assessments however you ask questions and you start getting roll up information here's the top things you probably should look at because they're the way you're the weakest these are the alerts that tell you basically which of the attack things across the blue your your should be most worried about you're doing pretty good here this one maybe you want to look at we also raided the attack patterns based out of a nice a tact planners but the tech techniques the miner taxi and the white boxes we rated them by how hard or easy they were for an attacker to use and we kind of looked at it based off of some some other people that weren't

using this and how they did in their environment but we also kind of looked at like if if you have to be like super leet to do it maybe at a company that's not the first thing you care about but if it's in Metasploit you probably should care about it because a lot of people can run it right so it allows you to kind of flip you can actually determine what your threshold of pain you're looking at and it just as adjusts everything so you can kind of see what you care about for your environment so that's the first set of experiments this is this is for people who are trying to figure out and assess their own

environment we built another dashboard this was another experiment we had heard and we had talked to people that were interested in trying to figure out if I care about these intrusion sets because maybe they're attacking my industry or they've been reported tacked my industry what are they doing and what's similar and what's different like if there's all these intrusion sets and they all do the same thing maybe I should focus on that right that was the kind idea so we built a thread dashboard with all that data that miter miter tack provided and remember we are pulling all of it almost all of this information from the minor attack thing so this was an mitre Tech open source

all the data and we're pulling their data and that's how it built it we expect that as people if they take advantage of this they're entering their own data for their own environment because it's going to be different for everybody's environment their care abouts will be different but over here is actually a list of the intrusion sets that are in there you can click up to five and you start seeing highlights of the attack techniques that they have been described using in in the research that mentor had it the colors don't kind of quiet come out here but you can start seeing overlaps too so it'll actually tell you by multiple colors if five all

five of them you know use this particular technique it'll highlight all this you can kind of look at it from one shot but remember what we care about is not so much what are they doing but what do I need to do to stop it so you can click a tab at the top and you actually get a view of the mitigations you should probably put in place first because it is most used or these mitigations will block more of the attack patterns that were described before right so the idea is these four people I care about these are the attack techniques they like the most these are the mitigations that will attack that will stop those attack

techniques that make sense that makes sense all right so as I said this was a set of experiments we do this using something called the Lean Startup model that we're basically a really small start-up inside of NSA which is huge so it's really weird intz government to like saying oh we're like trying to be a start-up inside a government is really odd especially when you look at like how much approvals I had to get just to bring these slides here you know and so but what we do is unlike a lot of startups which you know says I have an idea I'm gonna go spend a year trying to build something and then I'm gonna come

out and try and sell it we actually try and do the opposite we come up with an idea usually based off a talk in other people but or because we haven't seen them for me and my partner who is having a great vacation right now in Italy um we are we are both come from the blue team red team hunt team so we had seen this kind of stuff these problems before inside of our customers we kind of thought it would be really different we thought the commercial world would be like a lot better but dead end to be the case they just had different scales of problems so we decided to say instead of

trying to build something and then try and convince people that it's a good idea we decided everything that we build is based off of people who said I'm interested in this this is how I like to I want to use miter attack a lot of cases the people we talk to are doing it in an Excel spreadsheet and at some point it becomes unwieldly and so we start building and every month we do a full deliverable to github for people to try out and to demo it so it's constantly changes constant getting new features based off of the information we get from people who are interested in it and then we go to them we say all right

try it out and then we measure how effective it is for them but we ask them it says did you use this was it important did it give you a new insight you didn't have before and if they say yes then we say great we'll keep going it they say no we'll say all right maybe we will stop this feature it doesn't help until we find somebody that works just like the the unfetter analytic which is the first one thing I built I love it so much but we stopped it because we weren't finding the people who could take advantage of it love to restart that again but and we moved to this because we found this is where the

people had the most problem that we talked to just about the product self I mentioned what was built for unfetter analytic what Elks tack Apache all that it's everything we have runs and docker we try to build it so you only need two commands get clone and then docker compose up basically builds the whole things really nice we built the backend in something called sticks does anybody familiar with sticks can I just a chance a lot of people that's pretty good are you using sticks in your environment a couple it's something oh really good alright so we built everything is six 2.0 I think we're not quite at the next version but we needed to pick something

we're all JSON and rest so he said all sticks is 2.0 is great so I'll JSON rest but what this is is a standard for communicating threat information between processes or groups that don't really know each other right that's the idea of it so if someone is a is coming up with new they're doing research and they're discovering new threat information they can publish this Styx data any other product that is sticks compliant can receive it and add it to their system so we built everything in sticks mitre is actually publishing the attack matrix and all of the relationship stuff in sticks so when you bring it up it goes and pulls it from their github and loads

everything so it's kind of new every time you're loaded and you run it we I I had to say I was a little bit oh we were I think we were the first open-source product that actually used sticks to to that capacity to that level I put one of because I'm not quite sure but I didn't see anything else we're I'm actually part of the the Oasis group that oversees the sticks so I trying to always figure out what's the new thing that people are talking about and working on to make sure our product supports it because we want these unfetter products to be able to communicate with each other and with repositories of people who are starting

to build sticks data so that as threat data is being created and published it this will bring it in and take advantage of it so yeah let's get that it's not too much work so just if just real quick on sticks this is all of these sticks objects that we take advantage of so attack patterns campaigns courses of action or like mitigations indicators or analytics same kind of thing identities it's like you know Frank created this new thing so that Frank's ID is on it that's all that is the malware I'll skip over here threat actors and intrusion sets which there is a difference and I don't understand the difference so I'm not gonna explain it it's really

confusing to me but and then reports and sensors the other thing that we do is sighting so in the unfetter analytic when when alert fires and says this is interesting and it brings up in Cabana it will also send the data to a local version of unfettered discover and you can kind of see it in your list so we thought about what if people event take advantage of that and like a little bit higher level the dashboard then the sock is using to look at all the events the way we do that is we actually created a rest interface for all of our objects if you download this and run this you can go to an API page which actually

generates this really great detailed example of every single API you can click any one of those and you'll actually show an example of how you would edit and delete that data and then you can actually run it so if you're doing this production I would turn this off but you know per development for experiments this actually works really nice I will say that at an upcoming conference there's a couple folks that are actually trying to figure out how to use this this API in their environment so the unfetter is describing the data that they care about the most and I think they're gonna brief it at wild-west hack and fest so I'm pretty excited to see how it what comes along

with that alright so just kind of in summary and a challenge so if you have never you seen miner attack you have now really quickly I can't I asked you to go take a look at I think it's a really great resource think about how you might be able to take advantage of that that detail information in your environment if you have looked at it and you've been thinking about how they take advantage of it I would I would love to get some input on how you think you would use it and the miner type even if I'm not selling you on unfetter this is an experiment to try new things I would really love to

see people who are building their own tools or companies go and grab this say this is a great idea and implement in there and product that'd be great this is MIT and government license yeah I think you can do whatever I'm not a lawyer so I don't want to say that on camera but I think you do whatever you want with it go book go read you know what MIT license means but um I would be happy for people to use it and tell me let me know how it works and and but we're also looking for people to provide detailed feedback about the products say if I have this environment we have this problem I need this answered I have this

question I need an answer this is the data I have I would love to hear that kind of because maybe we can put it into the product a couple things that we are getting ready to start working on now we are going to create a more detailed model for analytics and sensors so that when you create an analytic you'll actually put in that pseudocode that we saw with the miner type model it will know the data model that you have just that you need to run that analytic it will have the sensors that will know the data model it will start recommending which analytic you could use in your environment based off of the sensors you

assess before that's cool I'd love to see that work I'd like to see that work and not be so cumbersome that no one's going to enter and we're hoping to make it easy to enter the data but that's one of the things we're doing we're also doing some stuff to do text analysis so that you can feed it a report and it'll tell you which miner attack techniques were described in that report and the intrusion aside to describe so instead of that member that that's you guys are the smartest analysts you're given that report you have to figure out what's in it we're gonna try and provide some tools that helps you figure that out on

your own that's called unfetter insight and then we got somebody that's playing around trying to figure out it's an experiment may not work the other thing that we're working on is we're trying to create these unfetter antis systems so they talk to each other so you can have a hosted system that has publicly available information that's being updated whenever and then you can have your own privately hosted one that's only for your environment and it can go out and it could pull that data we're probably will have that working and when I hope to say by the end by January I thinks probably have all that done but being able to exchange the information and we're also talking about because we

really care about that relate that exchange of information so our the analytics that are put in here you could share with other unfetter systems or you can have a bunch of people logging in and entering analytics and we're gonna actually do a rating system in there so that you can say yes this analytic works for me or no it didn't work and this is why and you kind of kind of crowdsource that so that's another one thinks is we have large groups of organizations that maybe don't work together but they're all working together on a link so we want to try and help those folks we're also going to improve the threat dashboard so that you get better

reporting out of it so that's the other thing they're working on that's that we're doing right now so between now and in December so we're always looking for new challenges for the new year and so if you have some really great idea we'd love to hear about it I appreciate your time thank you so much I know this last one so thinking besides for having me and you guys have a great day thank you thank you [Applause]