What Did the SIEM Say?

[Music] check check hey it works because i don't want to be hunched over the whole time so i want to start with did everybody have a good b-side so far i know we're last i know you're all tired we're very appreciative for everybody who came out to listen to us wax poetic about sims so let's jump right into what did the sims see start with some introductions jr you can kick it off hi everybody i'm jr presby i'm a splunk architect at the u.s census i work work for t-rex uh i'm also part of the maryland air national guard so when i'm not but that's one so so yeah uh currently national guard so when i'm not doing stuff with splunk

over at census i'm doing more splunk stuff or with the air force i've been using sims for quite a while if anyone remembers cisco mars yes i used it i've configured it it sucked it's end of life i'm also a pretty avid golfer uh actually a couple weeks ago i got caught by my sizzle during lunch it was a nice day went hit the driving range came back had my glove on still he's also an avid golfer he was kind of mad i didn't bite him so and that's me in a nutshell thank you and i'm sean or understudy i am a paranoid and the head of security operations at verizon media i was a breach consultant i ran some

managed services for a while i'm a general sim user and program builder my job and what i'm going to be talking about throughout the course of this talk is the how do we use this to find bad guys that's really what i'm here for he's way smarter than me i also really really suck at twitter i am trying to get better though but that's enough about us so if we kick right into it let's start jr what is a sim anyway well what a sim is security information event management system the sim gives you okay the sim gives you different uh effect functionalities one big thing is uh log retention it's a good place to store all your logs

into the sim and then once you have all your logs in that one location you can also get do correlated searches across different data sets with your sim uh then you can help so hold on because i want to make sure i get this right because i thought it was a thing that pissed off security analysts regularly well that's what that's later on we talk about alert for t but we're not we're not there yet okay but yes they do piss off security analysts um because sims also give you alerts and when alerts fire off analysts need to do something now there's a lot of different kinds of sims too right just about everybody's kind of in this game

you want to go through some of that yeah there's plenty of play different sims out there uh the elk stack's a popular one uh i'm in a splunk shop currently also in a splunk shop this is not a splunk talk we promise uh ignore my splunk

but yes um there's also another great thing to do with sims is again you're getting dashboards and visualizations now yes most of us here can go on a system grab and find anomalies in the logs which is fine but that doesn't scale you're not going to do that across an enterprise so you need a synth of a single place to kind of look for everything and find find what's not known will find your anomalies using your sim so fun fact i did actually spend about six months as an engineer much like you and i found out two things one i'm not very good at it and two i hated it so i quit doing that and i

started doing other stuff that was way more interesting but i very much rely on guys like jr to make the guys that i work with better and able to do their job so as we kind of walk into this talk one of the most important things to start with is the importance of knowing your use case with a sim that becomes so important because that use case really says how you're going to configure it how you're going to set it up what you're going to look for all of that so let's go through some of those yeah so the main driver of sims in the space now is compliance it's the you shall to meet to be

compliant with your compliance you'll have your audits your periodic review to make sure you're staying compliant and those are one of the biggest drivers of sims you're seeing in the market and when sims are built to those standards a lot sometimes they do end up with issues so compliance sims when i go to a customer site i can quickly tell that this sim was built strictly just for compliance because it is a check the box sim and they can't really hunt or find anything in it it's just you see what they need to have as far as yes we have these types of blogs there's some dashboards and there's nothing else next we have security operations

pissing off your analysts send lots of alerts to keep the analysts busy but that's not actually what the point of a sim for security operations it's to bolster security posture of your operation i i click too fast i do feel like i'm on a bit of a power trip with the clicker like swinging every time um threat hunting and now threat hunting now an actual usable sim we can actually find bad guys and uh protect protect your network and look look look for notables and uh so that because we're come from splunk shop a notable is essentially a security event and everything else so if we say notable sorry we screwed up because we're trying to be not specific

but what it means is a security alert or event for anybody not familiar with the splunk site right um and incident response and last but not least instead of response with your sim therefore as long as you have the logs if you can you know afford to keep those logs for a length of time instead of response can come in and figure out what happened and you're there so now that we've gone through some of that really the first thing that you do you you pick your product you figured out what you want to do you know your use case first thing you have to do is say what is coming in what are we gonna do with this and

that's our log source problem um first one we really talk about here we go back to compliance because it's such a prevalent use case yeah so the compliance driven sims they tell you what what to put into your sim how long to keep it and those are those those drive your requirements other things that can drive your requirements is your customers and their use cases well they'll drive your requirements of how you design your solution any specifics um examples or thoughts or things that people might want to come at if we're compliance what do we have to log yeah youtube we want to look at security relevant events firewall logs after hours logins all logins admins

admin usage in your sim uh proxy logs i mean there's a gamut of things that needs to be inside of a sim for an effective hunt it ultimately depends pci hipaa what you're looking at depends on what you need to log this however is not a compliance talk because we want to keep everybody awake so we're not going to go too far into that next we come into the intense importance of knowing your environment what tools do you have and what do you use what's your detection suite and package look like is that going into your sim why or why not we're going to talk a lot more about that later um what's your server and endpoint

deployment are you primarily mac windows what do you have from syslog servers or os query uh and then what are your crown jewels and this is a really important one that a lot of people don't do threat modeling across crown jewels and just identifying their most important company assets in the first place because if you're not logging your most important company assets then you're not going to know when you're losing a bunch of money because they just got taken down or breached and you define your detection strategy which we'll also talk about quite a bit later so are you going to go right out of the box splunk has splunk security essentials um qradar has their own

nitro has their own they all do you can turn them on you can tune them you can play with them most of the shops that i've been at prior didn't really understand what they were looking for and just kind of turned them on and we'll get to that um are you going to use a framework we're only going to talk about one framework today and that's attack so if anybody's not familiar with attack you should be it's by mitre it's great that guy works with them [Laughter] fantastic but i don't really think there's another one worth talking about so that's the only one we're going to talk about or are you going to take a hybrid

approach and i feel like that one we're going to cover a whole whole lot but first let's let's go through kind of defining the strategy and let's use miter as an example because that's really interesting and a little bit overwhelming right so that is miter attack in a nutshell and i'm not going to go into each individual piece of that or through all of that yeah there's our nutshell if i'm trying to figure out detections across this and not even detections but i'm trying to figure out what log sources i want that's overwhelming as hell to look at but there's kind of a saving grace to this pardon the dad joke point too as overwhelming as it may be

netflow traffic alone can net you a bunch of different stuff across mitre so it's all about understanding your log sources and then knowing what you can get from them and that becomes the important part of scoping so you're not looking at this and saying oh i need all this stuff you're looking at this and saying oh this log source that i brought in can cover these things but perhaps you're a new shop and you want to be a little bit more specific is anybody familiar with this report this is red canary's 2019 threat report fantastic report they basically took all the attacker stuff that they have and they said this is how attackers attack if you're building out for operations

detection or hunting this is a really great thing to have because you can say hey attackers are attacking us this way am i looking at that am i logging that do i really know what that means and as a nice little cheat sheet we've included some things in there different event ids different things that you can monitor and pull in that can kind of help you throughout that process so i'll leave that up for a minute just in case and we can probably try to make the slides public so you're saying if you're not logging powershell you're failing realistically at this point if you're not logging powershell you're probably failing if if you want to find bad guys

and i want to find bad guys that's my thing so i would strongly suggest that because a lot of this other stuff so maybe cats down there most of the time now it runs in powershell so you log powershell you get mimikats you get a lot of stuff when you run powershell it's fantastic so we're going to step away from this for a little bit and we're going to come back to it later with the use case as we talk about detection strategy we're going to move on to the importance of data throughput and for that i'm going to pass it off to you right so more logs does not equal better sim that is not how you measure the

effectiveness of a sin there's you got to be selective well what gets kind of gets into your sim um i've been to customers where i see them ingesting tons of voip logs i don't know why they have void blocks they're also uh ingesting blocks and tear down block tear down packets doesn't give you any affirmative there is a caveat to this though and that caveat all depends on your use case because i i'm sure everybody here especially if they're in blue team has heard the phrase garbage and garbage out not true the goal of the sim is to take garbage in and make value out that's what it's there for if i ingest all telemetry data from

command line interface i'm not going to get a lot of value just from that if i pair that with a pretty lively amount of thread intel or a bunch of other stuff all of a sudden that becomes incredibly valuable because i've got command line interface talking to a known threat actor that i'm tracking so you could say that that's garbage in value out which not saying pipe everything into your sim because one it's expensive and two it's not realistic but know what you want and don't just assume like oh well i can't alert just off this so i don't want it anyway sorry continue and also when you when you're when you understand what's going into your sim

understand the path understand where your syslog servers are what firewalls are between them if you have s3 buckets understandable security groups that go into those s3 buckets um if anything changes that can prevent that data from flowing that needs to be treated as a security incident these things should not be tampered with [Music] and also integrate with all your security tools your scriptures need to play nicely so if you haven't if you have an av solution the av solution should not be um scanning the the data in your in your syslog index you don't want that's just that's just gonna hurt performance and it's just a waste of a resource so don't do it make have your security

tools all play nice together if you have firepower you're using different managing rest apis if you can have some automated actions where your sim can actually block things um have them work together within the sim now for your dad joke oh and event time this is kind of one of my pet peeves yes i know we're on the east coast and we think the world revolves around our time but when you're logging those these are your three time choices and actually just to draw the same time so yes use use a standard timing format to ingest your logs once it gets to your site then you can always convert it to your eastern time to keep your analysts happy but they

need they need to come in the right way from the from the beginning you want to tee up the next one okay so now tell me about retention actually yeah this is yours yeah so data storage and retention so you gotta understand those requirements for data storage and retention uh i remember we were one customer and uh they typically currently had about six months worth of work to storage logs um and then one of their security leads told me that according to the stig that they read they're supposed to have five years worth of logs and i explained to him you barely have five months uh instead of arguing over this i said you know give me give me some time i'll

try to figure out a solution and then i priced out i priced out a seven figure solution that could contain that could hold six fighters with the data logs and that was queryable that conversation never got brought up again [Music] so moving on also data protection data resiliency um now you can't encrypt everything typical syslog of udp port 514 is not going to be encrypted you have to understand that but once it hit your syslog server you can encrypt encrypt that data and then data should be encrypted at rest for your sim and also data resiliency you have a sim it's a it's an integral part security process don't have your sim in one rack so when

we lose power in that one building you have no sim i've seen this before it sucks don't do it [Music] and also ensure solution can scale um year after year you're going to get more data the data is not going to your data is not going to go down it's going to go up yes splunk loves this and others vendors but this is the reality of the situation and sometimes you have to scale out so you have to scale up and just add more add more devices or just maybe just add more storage or more capacity to the existing infrastructure so what's your recommendation because there's a lot of differences between scale up and scale

out so like what's your approach how would you go about that i typically like going to scale out because you can you scale out um out gives you also gives you the data resiliency and through your load you know load balancing and better performance i typically if you can afford it you can i'll go for scaling out the other upside of that is it kind of hits some of bullet point number two you get a little bit of resiliency when you scale out [Music] and lastly for this one all right so now we're going to talk about understanding your login requirements oh oh yeah faster all right yeah so fast no talking about storage now if you have

this data you have all your alerts you have your rules you have correlations but if you don't have the iops to actually search the data what how useful is it many times i've been with customers and the main thing i've been told is hey don't worry storage is cheap you can buy more you don't need storage you need iops data in your sim you you take a tiered storage approach to it most current data needs to be on your best performing storage then your second tier data for uh things that's been a little bit older and things that you really don't care about you know stick it in glacier or put in a vault somewhere it's not going to get

stored but you have to look at that tiered approach but things that's going to be queried consistently needs to be needs to perform so there's a little bit more to this line actually too than just how fast your storage is and that's how good are your are your analysts at pulling up the data and what does our architecture look like i've been in situations where an analyst would try to look for something over an hour and it would take two hours to run not particularly efficient especially if you want to run that relatively regularly right one hour turns into a two-hour search now i have data now i do it again just doesn't work great um but now as

we're starting to move into some of the analyst problems so let's keep going down that path so alerting seeing the bad guys i like seeing bad guys i think that's great what can keep you from seeing bad guys lack of visibility if we don't have it we can alert on it i'll tell an old ir story that i think is really relevant to this case so we went on a case for a customer who had a server that was hacked we were trying to find the entry point of said server we strongly believe very specifically where that entry point is still was on a thin client network and they went from there and then pivoted into the server environment

however they had zero logging on their thin client environment and if anybody's familiar with inclines once you restart them it blows everything away so if you're not sending anything out you got nothing and they had nothing so we can strongly believe that and we made the strong recommendation that they start logging in but it doesn't necessarily mean that you're always going to have that visibility into the things that you want you can't alert on it and a lot of times you can't get some of the contextual information so lack of context it's a really big one if i'm monitoring a bunch of threat indicators and i'm worried specifically about apt targeted threat indicators and i'll use this example probably a couple

times because it's a really easy example to use and i'm lazy um and i'm getting hits from dns but i have no necessary link or dns logging to tell me where it's going to i have no other necessary network logs to tell me where it's going to or what generated that alert or what generated that connection what value is that to me i can't track that down so it could be bad it could be not bad but i can't really do anything about it and that doesn't help me as an analyst to find bad guys so having that context and having that ability to dig a little bit deeper some redundant log sources covering multiple parts of network etc tend to be

really beneficial in this use case um next we go into non-actual alerting um does anybody just out of curiosity anybody here like work in a sock type environment couple so anybody deal with non-actionable alerting stuff that comes in you can't do anything about it couple so what's your group's mission what are you there for um i work at a place that has multiple programs so we have different missions and different actions so you know we might have an alert that says disabled user account attempting login that's great i don't care maybe insider threat cares but i don't i don't want that in my queue or we might have you know potential brute force and we got to be real picky

because if it's a hundred i don't care what can i do about it nobody got access they haven't gotten in it doesn't matter to me and i don't wanna burn cycles on my people going man i can't really do anything like i looked into it but there's nothing i can do not really a good use of resources so next as we move into that we move into probably the big one for everybody involved in this and that's alert fatigue alert fatigue is a [ __ ] um the whole concept of alert fatigue for anybody who's not familiar is this process that i've looked at this type of thing over and over again and every time it was this

so every time going forward it's going to be this it basically removes the part in our brain that goes this is interesting and goes i'm bored it's nothing move on it's a pain and it's a very very real thing in most operations settings so how do you kind of fix that and that's not always on the analyst that often stems on content and if a content team doesn't have a good relationship with your analyst team you're probably screwed because i use a formula and this is my thing take it if you want intent fidelity and frequency because everybody talks about fidelity i want the highest fidelity alerts in the world i don't give a [ __ ] about just fidelity

i really don't fidelity is great but if everything that i had was 100 true positive i don't really need an analyst at that point i'm done i know it i don't need that person to make the distinction and an analyst role really is to say hey i have determined that this is bad or i've determined that this is not that's the job so you take a couple things into consideration instead of just one you take intent so intent is what is this thing looking for what does this mean what is the possibility that this is going to be particularly bad and if it's a true positive how bad is that bad gonna be so is this a breach is this denial of

service or is this oh just somebody was you know web application pen testing and we blocked it so that's where intent comes in fidelity i think everybody knows likelihood of true positive to false positive and then frequency and frequency doesn't get talked about a lot but frequency is a really interesting one because if i have something that has a very high intent and for for the record frequency is how often does this fire on a regular basis so i have if i have something that says if this fires and it's bad it's a breach which is a big deal and it fires once a day to once every couple days but most of the time it's false positive

maybe it's a command line argument or something that sometimes happens in the company but i can't tune it because you can't tune everything you should be able to tune everything but you can't tune everything maybe i want to keep that there because it doesn't come in a whole lot it's relatively easy to verify and we can move on and as we kind of talk later about moving past the basics we can talk about some ways to fix that but for now that's where it is the one thing that didn't make it to this deck i realized was tuning tuning and more tuning um and this concept of the the constant back and forth in a sim from an alerting

perspective where you have to be on top of it so you have to have processes to tune out intel if you're tracking intel you have to have processes to tune out alerts you have to have processes to say i've determined this is false positive i have determined that i can always say this is false positive based on these specific considerations i never want to see this again i want it gone you got to be able to do that it's like blowing something away it's fantastic and also with tuning you need to remember when your environment changes your symbol has to change with it when things are updated you need to understand that if you're in the sim

game you're in the ccp or your cab change boards you need to understand what's changing that environment so you can properly log that environment so then throughout that we kind of come into this concept of knowing what you're looking for and that becomes a very important part of this because if you don't know what you're looking for you're not going to find it i'm going to talk specifically about a very about a type of a learning strategy called hybrid alerting strategy sounds fancy it's really not what that means is we're taking a bunch of tools taking ids we're taking edr anything that we can we're sending that into our sim now there's a bunch of reasons why we might

want to say so we have crowdstrike i'll use crowdstrike because the use case is using crowdstrike um there's a bunch of reasons that we might want to take all of our crowdstrike telemetry data and alerting and put it into our sim so if our sim is our single pane of glass for alerting and we have a great tool that has great alerts from an endpoint perspective i want those alerts to come into the channel that my people work out of i don't want to put them in two panes of class two panes of glass means that there's less of a response time that's not great so the more you can get into one place the better more importantly

having all the additional telemetry data and logging that a tool like crowdstrike or carbon black can do adds some other interesting benefits that we'll get to in a little bit secondly intel whatever you use be it free be it paid be it through a threat intelligence platform be it not intel is valuable intel does a good job of filling gaps in a non-specific way with intel i don't need to know how the attacker is coming i just need to know what he uses what's his domain set what's the ip set and intel changes a lot and also much like any alert needs to be constantly tuned but intel can give you visibility into things that you would

never have visibility into because if you look at something like mitre attack specifically you can make use cases out of quite a bit of that from an alerting perspective but it will take a long time because i don't know how many people have alert like the guys who generate alerts they're usually not that many of them and they're usually way overworked and helping out a lot of people um so intel can fill in those gaps plus i can't necessarily get i can't make an alert off every piece of attack it would never work there's too many things that are too close to normal user behavior that you just can't do it so we add our

intel pset so now we're logging intel as a feed we're logging our tool sets our ids all those alerting as a feed and now we're going to start to try to fill in some gaps with the tag so we're going to pick a framework and we're going to use that framework and we're going to look at it from that perspective before i go into the use case you want to talk about i say anything about this i also need a drink [Music] oh no about the yeah well using miter is basically is a standard there are other frameworks to use but most most people use miter for that reason it's it's most complete it covers most

of the use cases i am strongly of the opinion that none of the other ones matter now that attack exists because it's easily the most comprehensive and it's basically what everybody is using so standardization is good and we as an industry really suck at standardization but we should be better at it because then we can kind of be like oh how did you find this thing and how does that look and we can talk to each other and present ideas in a way that's like oh i could totally implement that exactly like this it's great so we good for the use case so let's kind of do a use case and i'm not going to lie let me prep this a

little bit i stole this use case bold face right from miter and i did it for a very specific reason because one of these sections is talking about tool detections and what tools can see and miter's testing tools and i didn't really want to go through the the pain and suffering of trying to test some of these more expensive tools and be like what will detect where the miter's already done i'll use theirs so we're going to use their apt-3 emulation plan and i stole this right from their website there's a link to the report there if you want to go through it so you can take a look so they set up their c2s they set up all their software

packing compromising hosts defense evasion they did a lot of stuff i'm not going to go super into detail on the technical side of just this but i will tell you that this is what they were testing for so out of that big list that i showed earlier this is specifically what they were looking for so it's a relatively small subset they already did the work for me from the tool perspective so i use crowdstrike as an example if i have crowdstrike that's what we had an alert on now it's important to say before somebody goes oh that's not much because that's not necessarily true this is what we had a specific generated alert on in near real time

that does not include detections via telemetry data or detections from the overwatch team that was also involved in the study and i just want to throw out this disclaimer because it's important and i have no problem with the product but it's important to note that we'll talk about the importance of some of that other data a little bit later but for now this is what generated an alert so if i'm piping alerts directly into my sim i would have been alerted at these stages i can map that i can know that i can know exactly where i am so now we have a bunch of intel i stole this from a 2014 threat report on apt3 from fireeye

um they have a bunch of information on apt3 and there's a bunch of open source intel and normal intel on it in general so if i'm looking for this specific attacker group i've got some things that i can look for i can look for ips i can look for domains i can look for hashes if i have the ability to look for hashes i can do that with crowdstrike i can do that with osquery i can do that with any windows system just mine this one so i have a lot of ability to kind of use some of this to go well i don't have a specific detection for this but i can find it like this

which is great so how does intel fit so intel kind of doesn't map to a specific piece and that's okay intel kind of fits over top so intel is going to be a little bit more kill chain detection we're going to be looking at execution we're going to be looking at command and control and exfiltration and those items so intel fits a little bit more kind of over top because it's broad and at any point that we see that indicator we're going to see it we go from there into hey this is what we detected we detected throughout here so what kind of gaps might we have what didn't we see could we caught this

earlier could we caught this different because a lot of attackers are going to probably attack just like apt3 did and a lot of the same techniques happen they get repeated they get reused so the intel helps us with apt3 specifically crowdstrike we're testing them they have good detections they're finding stuff but they might not find everything so what are our gaps so this is by no means an all-inclusive list not even a little bit this is just some of the things kind of off the top of my head that i was like oh we could probably potentially try to build some use cases off this and some of them are kind of specific i don't know how hard this is

for everybody to read it's very hard for me to read right now um so command line interface command line interface might not be one of those things that we can we talked about this but it might not be one of those things that we can say oh yeah we can do this but we can look for certain operators we can look for like no no exact on powershell um we can tie it to intel and see if there's any intel hits so there's a lot of very specific things and when you start to alert on these you get specific so i can tell you that per technique it's not uncommon for me to have made or

seen about six to ten individual use cases per technique that we're hitting down a lot that's how you're going to get your visibility because you're going to need different things for different log sources and then different things for different things detecting power shell is power shell now i need 15 ways to find bad power shell so we color in a little bit and we see that we now have the ability to fill in a lot of this stuff and this is also where you find one of the one things about attack you're not always going to find everything i challenge anybody and if i i very well could be wrong i'm not an expert by any

means

i would love to see somebody make good detections for discovery that's pretty normal like moving through folders i i'm looking at you joe we're gonna talk about this now you made a face um it's normal user activity for the most part it might not be a bad hunt but it's not a great detection unless you're very very specific in what you're looking for which you have to address from a risk perspective at that point i'm very very specific here there's a really good thread on twitter before i came here actually about the same thing use attack no attack but understand the limitations don't just color it in because you have one rule because that doesn't mean

you're protected there's a lot of things that come into consideration we tend to pick a technique stick with that technique until we've mapped every possible thing we can think of for that technique and then move on to the next one um and then we're going to have to go back when we're done and readdress all of them because everything will have changed because yay we love this so we'll move away from this use case for for a minute here and we'll move into a little bit on the hunting and ir side which oddly enough is not a whole lot different what you need some of the problems that keep you from finding the bad guys there that are

retention um we talked a little bit about um you know not logging stuff but data retention is a big one here because most breaches what's the current data i haven't been an ir consultant in a little while but a year and a half is it a year and a half we find people after a year and a half if attack if a good attacker's in a system i feel like that's right i feel like that's right but please don't quote me but it's astronomical so if we're at a year or even a half a year what happens if we don't have data and we detected the tail end of a breach how do we find out what they did so if

you're worried about that i hope that you're logging stuff on the things that are important to you at least searching through big data we could do a whole talk on searching through big data so we're going to glance off of it in like a minute making sure that it's indexed and parsing so that you can search and you have specific fields and using every training resource humanly possible to learn the tool and its specific query language and i'm sure you have something to say here definitely with your queries practice your tool know your tools uh be efficient with your query language don't search every index don't search all time for everything underneath i can't just do star index

equals star no i want my lobster you're not doing sorry to start and don't run everything in real time you don't need everything in real time i genuinely suggest not to run anything if you're searching you're looking for something that happened in the past therefore you don't really need real time that's from purely a searching perspective but we're this isn't alerting this is searching this is looking through all data right anything else you want to add there [Music] all right so here's the here's the big one so we talked a bit about lack of visibility from like a sock and alerting perspective oh my god hunting in ir people need way more all the visibility

so important because every little tiny detail every bit of command line interface every service that was created every process that started every network connection that ever happened becomes so important in a breach situation because in a breach situation you essentially have a couple goals what happened and what did they do you need everything to figure that out and you pull on the weirdest of threads like just the absolute weirdest of threats going through this you could be anywhere from email to network traffic to endpoint you want it all it's a big deal in that um same thing with hunting i mean hunting is essentially alerting without alerting it's going through a huge minutiae of data everybody talks

about hunting like it's so sexy right it's really not it's basically going let me look at every executable that exists on a system and let me see if any of them stand out or oh let's look at every instance or a command line was run and see if anything's weird and you get a little bit better let me see you know every child process pair and any weird ones or let me uh look at every executable under seven thousand bytes like it gets to that point like it's very tedious work but it's very exciting when you find stuff and that's why everybody loves it um great when you find stuff and i did go did some hunting over a client

and we assumed that since we had mcafee on our systems that no one was burning removable media that that wasn't supposed to and i couldn't get logs from the khaki and so they said that we're good on this at this point but we did have sysmon data so i i simply just uh launched this mod on this on the system that i knew had burner so i was not running any of my thin clients and then we found i made a list of everyone who's burnt and everyone who's authorized to burn took that delta with a simple powershell i mean with a simple python script in this instance and then reported quite quite a few

people burning that weren't supposed to uh the mccappy folks were thrilled [Laughter] so as we talk about this let's kind of go back to this again this is exactly the same one that was there before but now the sock has kind of evaluated this situation and they've determined that no command line interface new uh you know uh scheduled tasks those aren't really things that we can find a way to make a decent alert off of it's not something we can do they are great hunts though wonderful data to have for a hunt if i'm hunting i want to look at every scheduled task i want to look through command line i want to look at every new

service although you could potentially make a case to make that an alert depending on where you are and more than just from the hunt perspective as great of hunts as these make you better have them if it's an ir if you're in a breach situation this stuff becomes intensely more valuable one thing that i do want to cover on about this though is say my hunt team or my guys who are hunting are going through all scheduled tasks or all command line they're going to take that analytic and in general hunting they're not just going to poke in the dark they're going to refine it they're going to say okay determine this false positive remove

this remove this and they're going to try to pair that down to as small of a piece of data as they possibly can when they do that they're going to make a determination that oh maybe at the end of this hunt this analytic could be useful for the sock so very important to have a general like between sock in between hunt if those are two separate teams to have conversations between them because some of those analytics that they're doing they might be able to pair down enough looking through this huge minutiae of data to turn it into a very valuable alert and then i swear this is the last time i just wanted to give a general example outside

of just the use case of some things that might be beneficial for hunting because i know crowdstrike detected powershell but it might not detect every bad piece of powershell so powershell makes a pretty good hunt um so just kind of a thing about some other things that you could hunt that could be very useful um and lastly another thing that i have found very useful throughout my career and i've kept this up for a very long time is a list of really cool windows event ids for hunting and alerting purposes i've got about a three page list i've paired this down to about 20 that i think are relatively important and relatively easy to implement and manage

power shells in there any clearing of other security logs rdp connections and terminal services different account creation and password reset stuff those all become very beneficial when you're working through a breach or ir which is when i used to use this data or even as building out a detection and a learning strategy so we've done all this stuff we've got this stuff built how do we move past those basics what comes next in the world of all of this and as a general person who doesn't actually like this one very much i'm going to let him talk about it first well yes and in our world you need to look at to automate some the routine processes your maintenance

your patching of these systems the the role of your sim is to support your stock support your analysts and if you're spending lots of time doing maintenance uh it's time wasted look to automate these look to automate these things so as much as i say an analyst is by a person and i do because i strongly believe that analysis is done by a person there is something to be said for taking away the really crappy tests and i've resisted this for like five or six years there's something to be said for taking away the crappy little tasks that analysts have to do day in and day out every single time my fear with automation has always been

that they're gonna try to use it to replace people and then we're not gonna get the instinct because analysis is an instinct it's a gut feeling it's this concept of like this feels wrong anyway next we move into ai and machine learning and that's kind of a big one too um and it's happening a lot in some of these tool suites crowdstrike's employing it it's buzzwordy as hell but everybody's talking now about ai and machine learning but there's some cool stuff you can do with it if you really think about it from a project so like data exfiltration wicked difficult to really catch but when you start adding some machine learning algorithms into exfiltration and netflow data and what

that looks like you can do some kind of really cool stuff with it and i've seen some really cool stuff done with that so as buzzwordy as it can be and maybe it's not where everybody promises that it is it does do some cool stuff you got anything to add there [Music] all right i don't know behavioral analytics that you want you know that yes that's more definitely more in the lane um when you look at your users you see where they're logging in what times after hours a few uh even in certain environments where folks could be logged on certain systems now they're logging on different systems and different types of commands they're doing

uh look into that and try to figure out what they're trying to do and put that piece of puzzle together so you know what i'm realizing is we kind of get towards the end of this we have gotten through basically this whole thing and only said correlation engine like one time that's amazing um we're pretty much wrapping up we do want to take a chance to give a special thanks i'd like to thank william josh and jeff three people that i work with who proofread this slide deck and told me all the things that sucked so that i could fix them you've got a couple yeah thinking antoine and craig good mentors to me throughout my career

and my team at t-rex my team at verizon media and most importantly our spouses and children who put up with all of our [ __ ] and let us do these kinds of things um so questions thoughts comments concerns gripes does everybody want to go home [Music] anything all right hold on hold on we got a question

it had to be a machine learning [Music]

[Music] you're not wrong and i think we're definitely going to see it i will say one thing about my time in the consulting world there are a couple adversaries who are really really good at what they do most of them are really really lazy because it works so they don't really have to be really really good they just have to send you a phishing email and you'll click it however every time we make a technological advancement to find bad guys we have to expect that they're going to make the same technological advancement to not let us find them anything else

[Music] absolutely so um talk about like miss which is a free threat intelligence platform so if you manage on a threat intelligence platform like misp or something of that nature um i believe most things like splunk have apps um otherwise you can ingest out into like a stix taxi file and then have your sim automatically pull it the downside of ingesting into a file is you have to replace it every day and it's not going to be near as real time in case you isolate out like as you remove iocs you want to make that reflective as well and it's really nice to be able to manage that in one place rather than going like well let me pull this out of

splunk and then let me pull this out a minute no ideally you want to have it integrated in such a way even if that's sending it up to a damn s3 bucket and then having it pulled down um so that you're doing the whole list not just addition so that anything that's been phased out is also going to because phasing out intel is an incredibly important part of using intel for detections anything else