Detection as Code: The Engineering-Focused Future of Detection and Response

welcome to detection as code the engineering future of detection and response I'm Jackie Beau and I will be your panel moderator today I'm really excited to get to talk to you all I'm really excited to be back at b-sides my favorite conference don't tell the other ones don't tell them all right so what are we going to talk about today we're going to talk about how applying engineering focused Concepts and like a software engineering mentality to kind of the classic world that is pretty Manual of detection and response actually unlocks so many possibilities reduces manual toil and just makes everyone's lives better so in particular what I'm talking about in applying these kinds of Concepts is

and how it makes us better is it helps us detect actual serious threats to our environments rather than what tools or things that we have no control over tell us is a threat it helps us automate a lot of the noise out of our day-to-day it helps us get faster turnaround time from detection to actual response so you can do the work that you need to do to keep your companies and organization safe and also it just means you get to focus on the things that matter and the interesting parts I have strong feelings about this topic I started my career as a malware Analyst at a government cyber security operations center or we'll refer to them

as socks doing manual cue based work and since then I have done detection and response at companies as big as Facebook and as small as 100 person startups what I've always found is that the way that we approach threat detection and response we're Treading Water we have so many alerts so little of them are High Fidelity the things that are actually incidents aren't usually coming through our tooling we separate out analysts and engineers and we expect them to operate on outputs of tools that they haven't written and they don't have any ownership over uh and so I want to share one of my favorite quotes which is by Cliff Stoll if you don't know about him I highly

recommend reading the Cuckoo's egg he's pretty much the OG blue teamer and he has a quote that goes the first time I do something I'm a scientist the second time I do something I'm an engineer and the third time I do something I'm a technician I want to be a scientist so that's kind of the mindset that we are applying to to this uh this space so the good news about all of this manual toil and work that we do is there's a shift that's been happening and I've witnessed it over the 12 or so years I've been in this field to apply engineering focused Concepts and that's what we're going to talk about today and so for those of you who

don't know me I currently am at Asana and we have started up our detection and response program which if you couldn't guess is engineering focused and so we focus on a lot of the concepts that we're going to talk about today things like detection as code infrastructure as code automation having really high quality data Pipelines and yeah so obviously I love talking about this area but I decided what would be better is to get people who are also doing this work to talk about it with me and share their perspectives so without further Ado I'm going to turn it over to our panelists so they can introduce themselves tell you a little bit about their backgrounds and why they're

excited about this topic so first I'll go to you Louie yeah hey how's it going folks I'm Louis Barrett for the past 10 years or so I've been working in both detection response as well as instant response these days I work in the AI space in product security as well as Cloud security I'm basically trying to marry the two disciplines such that we get better at building detections using Ai and get better at security securing AI using the detection work I've done in the past really excited to talk to you today where are you currently I'm currently a scale AI leading both product security and Cloud security and we're hiring yes awesome Jessica oh I gotta get closer to the mic I'm

being told usually everyone tells me I'm too loud um yes uh Jessica rezine I currently work at Brax as a security operations manager which includes the security detection response function as well as corporate security um and I find my I found my way to the wonderful world of security and scenario spawns I started in tech support and then I moved into a knock kind of a junior sres type tile style role and I just loved incidents like I'm that crazy person I love a good crisis the world's on fire and I'm having fun finding out like where the burning pile of garbage is but that pivot and I realized like my favorite incidents were the security

incidents and that's how I came to the detection response space it's like hopping one crisis to another um and I'm particularly excited about this because I've been really fortunate in my career that when I started out at in a knock I got to witness the transition to being less of a knock and trying to be more engineering focused and then in my security roles I've been at smaller companies and had the opportunity to help build a program program which like lets me do more engineering focused stuff so I'm like super passionate like I have experienced the benefit of detection engineering like engineering first rather than the sock model so that's why I was really stoked to talk about that here today

awesome thank you Julie yeah I'm Julie Sparks I'm a security engineer at Brooks I currently work with Jess and you can tell we align on a lot of things but I've been in security for about seven years started in GRC and then transitioned over to detection and response what really interests me is the fact that you can apply these things that we like learn about from other Engineers software Engineers SRE and we can take that and we can use those tools to make detection and response more efficient and give us more time to work on threat detection and work on security incidents some of the things that I'm passionate about something else I really love is that these engineering first

detection response teams we tend to help each other we love to open source things we love talking to each other about these engineering problems and how we can work together to solve them so that maybe we all have the same size team but the end of the day we're able to work on more exciting problems and really optimize what we're doing and that's just been a Love of Mine love it thank you all right let's get into the nitty-gritty what are we talking about when we're talking about engineering first detection and response so first we're going to start with detection as code which I'm sure many of you are familiar with this is a switch

to really owning your detection signatures writing them into a code repository using classic CI CD pipelines and the the benefit here is that you don't have to deal with signatures that you have no idea who wrote them was it a tool was it an analyst was it someone who left your company six years ago and put six random IPS into a deduction signature we've all been there um so there's a record um and so Louis I'm wondering if you could talk a little bit more about like detection as code and why it has been revolutionary yeah so we lost that segment we had a pretty small team and we had to be detect um engineering first otherwise it

really wouldn't scale so for us we've really benefited from having our detections in Version Control allow the engineers to comment on detections learn from what people are building and have more visibility across the team and what was actually being done while we're trying to detect it also let us know at a point in time what were we able to detect so you could actually go back and say hey we would have caught x-ray wouldn't have caught X based on the detections we'd actually submit the source control and you know even with that being in place we could say like hey if we have an incident come up we know that we had a gap and we know how

to fill that Gap because we can literally see you know what that Gap was written into Source control additionally it kind of like helps to deal with crunch as well a lot of teams are under this kind of crunch work style when you have small teams but if you're able to kind of push things through CI and test whether that your detections actually work you're less likely to rent an issue in production where you're then having to say okay this detection is failing and we don't know it's failing yeah alerts that were just generated exactly why like knowing why is crucial to detection so like we should definitely be better at that when we're building

alerts yeah another point I think is really important is when you are making pull requests for your detections and you're storing everything as code you get the collaboration aspect that other software Engineers benefit from we're able to provide feedback work together become a better detection engineer because everyone on the team is looking at it or at least a subset and you guys are working together on what makes a good detection and then all of that is like historical like saved in the code like the repo request and can be referenced by people who are onboarding or who want to learn more about what you're doing yeah and I think like the big thing about detection is code is it offers

consistency repeatability collaboration and things are just better when you have that like you you know exactly what's in your environment it's like just the next step to being more mature in the space is like you're not you're not expected to write something sit look at all of the data in the future be like oh what would fire and in a traditional model that's like kind of what you do you don't have the tools to test what it's going to do in sort of your seem prod and still until you do it um and it's just like a massive step forward in maturity to be able to store detections in a repository anywhere and not just in whatever logging system is

running them for you totally and speaking about maturity I think I would be remiss to not talk about infrastructure as code kind of this concept where we're actually building all of our Cloud infrastructure our data pipelines our product in code so we have again this like source of truth of what infrastructure is set up what is it going configurations and I mean one of the big benefits of that is disaster recovery right like things get blown away you can send them back up but I also think for us as Defenders it gives us so many steps forward in um like our processes because if we know what infrastructure should look like when it's drifted and also like we can

accurately threat models so much better and so just yeah this is this is something that's also like a little near and dear to my heart is like infrastructure is code it's just what you do now and if you as a team are not doing it you're behind your team's not technically mature you're not really sitting to play at the table with the big dogs if you're talking to another team that does infrastructure as code and you don't it's like this is just a core table Stakes for an engineering team to do and like of course a detection Response Team should be doing it yeah but there's and there's so many ways you can do it

depends on what works best for you but you need to be doing it be an adult put on your big boy pants yeah it also comes out like getting the respect of the engineering teams you're going to ask to make meaningful change if you're not kind of like you know being toe-to-toe with them and their capabilities let's be frank like they're not going to respect you as fellow engineers and we can't really call ourselves Engineers within security unless we're building something contributing the way a software engineer or infrastructure engineer would right yeah I think there's also like the ability to scale up and down um depending on like your pipelines whatever alert alert payload your handling

um there's a lot of these benefits that come with having infrastructures code and being able to add and subtract resources as needed yeah it makes makes your life is an engineer easy too because you don't you you have an idea of what's going on in your environment you have those yeah absolutely I remember one Friday evening of course because you should make breaking changes on Friday evenings always I blew away all of our IM configurations in AWS but thankfully terraform so it's really a savior oh yeah I mean I don't work for Hershey court but um so when you mentioned pipelines and so something that we have to talk about when we talk about detection and any

kind of threat detection is we're operating on so much data so many different logs from different places and so there's this concept too that Julie we were talking about about data engineering and having like a really clear process on how you're processing all this information and storing it and how you use it I wonder if you can yeah so maybe this is my hot take but I think that detection and response should have more data engineering skills whether that's having a data engineer on your team or just having detection response people that are familiar with data and can work on these pipelines but at the end of the day you're not going to have good quality detections if you don't

have the data to write those detections on so data quality where you're able to ingest logs from understanding what those logs are and for a lot of teams that also means understanding the load of data you're going to be bringing in and whether it's cost efficient for what you're going to need in an incident or whether you're going to need for a detection further on so I think focusing on data engineering as a problem and something that can really up uplift your detection work and making that like a foundational piece is really important at Brooks we we open source substation which is a way to deploy serverless data pipelines to collect logs we are able to

collect logs store them pre-process and post-process after they're enriched and so that gives us the benefit of being able to store the data have it raw but also be able to perform any Transformations we want on the data so maybe we want to add certain role enrichments we want to like make custom hashes based on certain patterns and then we can utilize those in our detections but all of that can only exist because we have control over our logging pipelines and what happens to our data and does anyone have a story to share of like what it would have looked like trying to do that if you had a scene instead of like controlling your

own data pipelines it wouldn't have happened because you just don't have access to like some vendor has sort of given you this box and they go here you go it does all your logs it does all your detections but they don't give you that control to get in and actually see how well they're handling your data and they might not be doing it very well and you could just be stuck with whatever their slos slis are yeah and I feel I feel like we've talked a bit about kind of this like build versus buy um mentality and I personally feel like there is a balance between sometimes you do want to buy vendor Solutions but you

should have a framework or a way that you're thinking about what you're what you're building versus what you're buying and so I'm wondering for any of you have like stories about where you made that trade-off how you think about it yeah I think there's this underlying concept that you have way more flexibility to build if you have teams who have also engineering like like skills otherwise if you're just if you have a team of analysts and they're working within a tool you're kind of just going to buy a new tool for them to work on but if you also have Engineers on your team who are building the tools that you use to do the detections like that's kind of where

the magic happens and build isn't even possible in a situation where you're not focusing on engineering first no 100 true I think we also need to like kind of nudge vendors to build tools that support the ability to extend and integrate because without that capability we're never going to be able to build anything that's hybrid and we'll always be in this situation where we're you know at the mercy of these vendors to give us you know data fields like strings where they should be IPS and everything we know that should be right and they're just not doing this for us so they need to kind of move in the direction of detection engineering and you can just fix it yourself if you

have yeah not a string should be but yeah let's do it I'm going to give like a big plus one to that extensibility the ability to work with a product and have it be able to be molded to your environment and have control over like what's going in and coming out is just revolutionary rather than I mean I'm not going to name any names but just like buying you know your classic big name Sims and you have to learn their proprietary you know query language and you have to do everything in this very specific way that probably doesn't really have anything to do with your environment and additionally like those skills don't transfer right like exactly

you get this like lock-in or where you as a professional now can't grow because you've only used vendor X and vendor X's skills are not transferable to vendor why critical think we're starting a revolution I think we're like looking at fictional response engineering teams are going to take back the power from vendors because if they're not giving us what we want we can build hybrid solutions that take over that and take note vendors in the room yeah get scared or work with us or open up your platforms so we can really yeah really customize them yeah I think that's one of the successes I've seen a lot with like our open source like toolkit is it

doesn't do what we want let's change it like oh we have a cool new idea like we have uh checks like you can't ship a bad config for this environment anymore but we could only do that because we built it ourselves not because a vendor would have allowed us to do that yeah I did want to go back to your comment about how sometimes we have kind of this ceiling that we put on when we have analysts come in and they're only working on these proprietary tools and they become experts in one you know query I'm really trying hard not to say a vendor name right now but as a hiring manager and I know we

have like other like people involved in this when I see a resume and your experience is nine years working with one proprietary scripting like query language that just has to do with this one tool I can't see how I can slot you in to the like the current kind of like engineering focused approach and I would say that is not dunking on the analyst that's dunking on the companies that don't give the opportunities for you to have extensible skills [Laughter] word all right I want to talk a bit about the noise and prioritization and so we know that working and threat detection response where our scope is so uh it's so large and I mean Jessica you

talked about you own corpsec as well so it's not just incident response it's not just threat detection it's also corporate security and asset control and the the issue too is right we're not working on systems that are static or not changing we're not working with threat actors who aren't changing what they're doing every day and so there's so much that we could focus on and there's so many things that we could focus on we have vendors we have organizations like miter telling us what we should be focused on we have Frameworks that were written 10 15 years ago that tell us we need to be focused on certain things how do you operationally know what to focus on when

you're building out your detections and I'm gonna throw it to you Louie so I think it really comes down to threat modeling your environment and using Frameworks internally that makes sense um one that I really like is kind of this High exposure versus low risk to low exposure high-risk type of map and what that basically looks like is if you have uh say an external system that's customer facing right you have a lot of control of that system typically because you designed it you can put security controls in place but it shouldn't really be something that's high risk versus the other side of the spectrum there where you have your cloud system is that might have very very low

exposure no one really sees this except injuries with admin access but the risk is very high of something were to go wrong so focusing on either side of that actual diagram tends to be really impactful because you get the swath of people who are trying to attack you while also focusing on the systems that are most critical in terms of things actually going wrong in your environment so with that in mind like I've always worked on very small security teams and we've had to prioritize because we can't bring it we can't log everything I never believe we should log everything else it's expensive it's so expensive an insane Paradigm that only your vendor wants you to do for more money right

vendors I see you in the room um yeah so like basically focusing on the things that you believe as your organization have the highest risk put your logs there right and then really you get to know those logs and learn them yeah so rather than like that breadth first let's do everything let's log everything let's actually look at what the threats are to our organization and like every company every team is going to have different threat models like there are different attackers there's different threat vectors and trying to apply a one-size-fits all it's just gonna it's going to give you too many alerts it's just it's not going to be a good time this is something I would love

Julie to comment on because she because she actually did something which has been really an incredible prioritization tool for the team that I reference constantly yeah hoping to actually probably open source us but it's a way of thinking about log prioritization so went back to this whole thing like my hot take is the data engineering um but what logs you ingest how you store them and what is in those logs um should all be prioritized basically for incidents for detections you need these logs but you only have a team of three people so how do you choose which logs to ingest and what data to keep and how long you keep it for and for us we have

like more frequent storage where we can like access it immediately in frequent where it's a more expensive to query and so we have to make decisions um based on our budget and how much data we're bringing in where do we put each and when do we spend time ingesting those logs and I think having that conversation internally looking at um there's a couple different ways you can look at it from like the risk of the the logs the platform itself or the logs are coming from you can look at like how is that platform secured how often are you having incidents with that and use all these different factors to determine what is the priority of ingesting those

logs how do we store them and then making sure those are available and like planning your quarters around that so that you keep getting logs in you're making sure that you're moving towards the progress of having everything you need to do those incident response actions those threat detection actions yeah and so so what we have as a team now is we have Internal Documentation the log prioritization framework it's there it's documented logs come out to high medium low risk depending on like the context of our environment the framework is very specific to like what we are where our customer data is and because of this we have a prioritized backlog of high medium low log sources

and we're like cool what are we doing next we just pick them up off the top of the pile and then those conversations for me as a manager become a lot easier because I have conversations about how much our tools cost and I get to write documents that justify the cost of our tools and when we have a frame where it's like look we think about this in a really mature way it also makes all of those managery like conversations like talking about budgeting and like what the real value is because we've been prioritizing like so consistently over time like Finance is like okay you're actually doing a pretty good job and how how do you leverage these to talk about

the things that you're not prioritizing because I know as security teams I joke with my team that a lot of what we do is emotional triage because people come to us and they're like I'm scared there's a there's something weird that is happening and you have to talk to them in a reasonable way about what you're prioritizing why you're prioritizing it and so I'm wondering how you Leverage kind of these systems are these like processes to do that yeah it's I think what happens most commonly is someone who has a little bit of knowledge about the space makes a suggestion that at the surface level sounds incredibly reasonable so like as part of these conversations talking about where we

want to log this Source not this Source we think this is more important a regular one that comes up is like oh what about Network things what about VPC flow logs and I was like do you know the volume um but it becomes like out of a place of a lack of knowledge from them and because we have just like this is how we think about it this is how we prioritize it's much much easier to have a conversation about a framework in place to do this rather than like no you're dumb or like like pushing someone off and making them feel bad you get to read them into how you do it rather than them

feeling like you ignore them or blow them off because you didn't listen to them and it becomes like self-service too like Louie I'm wondering yeah yeah I find that super helpful in the past because like just pointing people to the document station you have as to how you think about logging tends to alleviate their fears and usually have compensating logs they weren't aware of like we think about VPC flow logs and VPC Vlogs will have DNS logs are useless so like is there a better way of getting that context there often is right and as long as people can see that they're less focused or hyper focused on the thing they think is scary like an additional

kind I want to make on this was around like documentation in general and kind of back to our original infrastructures code detection is code questions here these things are self-documenting once you actually put them in place so it becomes a lot easier to have these discussions after the fact because you're codifying these things so people can go back and reference them that's a really good call out because the documentation also makes it easier for you to onboard people to your teams as well as stakeholders and compliance efforts our jrc team loves us like working with us like you're on top of it and we're like oh it's because we just we write down how we do the thing and that's like

a huge amount of like what compliance and grp yeah infras code infras code makes um the GRC audits like very happy times yeah it doesn't have to be very hard to make GRC happy it's actually like can be quite simple when you focus on the things that matter yeah and I'll give like a little shout out to someone on our GRC team Alan you're amazing um but he'll he'll he will look at something or be pulling evidence from a ticket and he'll go find the ticket from last year where it's a specific place in a GitHub repo that I linked that shows like yes this is this is the connection of the logging for this thing and then

he just goes and he's like wait this is what you showed me last year is this the same page like am I right and I was like yes you're right just show to the auditor again and the consistency of having it documented in the code repo which is also like self updating essentially is massively helpful for like my peace of mind when it comes like audit season I try to keep the team out of it and I can usually because they've documented the work awesome I want to talk a little bit about Automation and especially like automating manual processes and I think here I I think a lot about how we saw this revolution when it started to adopt engineering

practices and devops so I'm wondering and I'm going to say it are we in like a sec Dev detection Ops Revolution is that what we're fighting for here yeah okay I hate acronyms too but yes it pains me a little bit to say yes to that and I mentioned earlier like I started out my career in a knock in a knock that moved from like into a more devops approach so I feel super super passionately about this um yes devops is the future and I think all this comes down to a very Central like human behavioral idea of ownership if you own a system you own it um which means you have the skills to

maintain it you have the ability to say no when other people want to touch your system and you can like make changes when you need it and if you feel the pain when that system breaks you're going to build an excellent system and if you have any of those things split to other teams you can't control your own destiny things are going to get worse but when Engineering Systems break if you can fix them and you feel the acute terrible 3 A.M this is the worst pain when they break you better believe you're going to fix that the next day but if it's an analyst that feels that pain and some engineer over there hears

about it like theoretically they're not going to make any effort like really emotionally to fix that thing you just have misaligned incentives so in my mind like devops helps bring the right incentive structure and sense of ownership like to work and I don't do you can really have an excellent like system if those incentives aren't aligned correctly and directly the reason vendors aren't really here to help you right they're not living your life they're not feeling the pain points that you're feeling and until they are we won't really get better products so we do have to be the ones pushing them to make these things extensible to give us some more of that control back yeah and I think when we

talk about automation something that I hear often when I talk to people like especially people who are in analyst roles now or have spent a lot of time in like classically more analyst type roles they are like well you can never automate out the people from threat section response like and it's something that I think is very like uh gets spicy quickly and I want to say I don't think any of us are thinking that we can automate subject matter experts out of this process but what I'm talking about instead is instead of having someone on the factory line dealing with things that are coming out of a black box giving them the ability to actually modify the factory and

change what's coming out like Jessica said it's the ability to control your own destiny and the ability to apply your specific knowledge and threat detection and threat actors and how to protect companies and actually be able to do the cool work of like hunting and red teaming without having to be stuck under well I have 5 000 alerts to look at today so um and I have no idea what this IP address is yeah I think it's like time allocation if you think about it you only have so many hours in the day detection Response Team handle a lot of things if they're engineering first they're engineering building new systems they might be building incident response

automation they're going to be writing detections they're doing threat research you might be doing threat hunting like you're doing a lot of different things and you really want to free up time so that people can be doing those interesting things and they're not burning themselves out looking at alerts or handling the same thing over and over again and so like I really like this like way of viewing automation as just a tool to make your team better not a tool to remove people yeah I think a pretty key example there is kind of in just the alert and enrichment type pipelines if it looks building out and you know these things should serve to make every one of our

analysts a bit faster and also kind of spread the knowledge of our senior analysts more widely saying here's a process I would follow this process is now codified as a script or an enrichment Runner such that anyone who's more Junior picking this up can immediately benefit from the experience of others yeah and when when you're like looking at a thing in front of you and you need to do it faster if you can control the system that's emitting the alert you can control the system that's providing you with the enrichment like yeah I'm gonna take that extra five ten minutes out of my day so that I'm one minute quicker every other time just like crops up in

the future yeah potential gains yeah yeah and I think like the alert management system that uh you and Josh built for the team is like a really good example of trying to take all that stuff and put it all in one place all in one spot that gives someone the ability to do all the information they need like just go like why stop yeah I think a key part is also like looking at how alerts are coming in like the point where you're making the detection to where like an analyst or an engineer is responding to it how can you make that process like faster and easier for people whether that's like incident response automation for us it was also

like looking at the context of every alert before it gets to the um end on call so that we can suppress and de-duplicate alerts more efficiently so we're not getting if the system spikes we're not getting a thousand alerts because something that it did download a script on every endpoint and we want like an alert from every single one of those like it's going to see that it's the same script that's running and we're going to get one alert to triage um and looking at ways to do that so that you're spending less time doing this like repetitive work where you're doing these like investigations that are probably benign but you still need to do

them and you're able to like free up that time to like hey there's like a new threat research article that came out and I want to write a detection and so just like giving the power back to the individual yeah so it's like once you see something and when you see it for the first time you want the context there to not have to go to a bunch of other systems or like and have like you know 40 tabs open as you're trying to figure out where this came from and what it means and also once you see something getting all the contacts there great but then do you need to see it again ever

like if I'm triaging an alert and it's not relevant or it needs to be tuned or something I never want to have to process it again um so yeah yeah I think you touch a little bit like on burnout incident response roles detection response humerals are really burnout prone because you have the high pressure you're doing an incident response like any given you could just have a terrible week um and yeah like it's like death by a Thousand Cuts I would see it in your like your day-to-day life that alert's really annoying and you like sling it away like oh it's fine um but the automation really gives people the opportunity to make their life better

yeah I sleep in hand since they have a place basically with automation yeah as possible it makes your life better but it also speeds up if you're in a real incident where there's that like pressure like there's like a million bricks on your shoulders and you're like struggling to breathe and you're like I have a million things to do right now well like you can automate some of that away whether that's like the incident like process handling writing the documentation whether that's isolating laptops all of these things make it easier so that like it's not as much pressure on a small tune that's really well said um I have one last question for each of

you which is about what you're excited about for the future or kind of things that are on your mind and then I want to open it up to any of you who want to ask us questions and kind of pick the brains of the panelists here so I'll start with you Louie what are you excited about yeah so I'm definitely most excited about like the using AI to make our jobs easier security Engineers I think there's an opportunity in basically every part of security to do this if you're an application security you can use language models to essentially look over a block of code and see if this is risky or includes any like known

vulnerabilities on the instant response side you can use the summarize instant Communications basically removing the need for instant Commander

but like really being able to summarize this especially when you're dropped into it like as a newcomer it can be like really you know challenging to scroll back up through that instant and figure out what's going on it's really nice if you can just say hey AI summarize the last comms what items are open what items are still in progress who are things assigned to so there's a lot of opportunity in this space and we as security people need to like actually look at what's possible for these technology and like feed this back into our work sorry do you work for an AI company [Music] all right Jessica how about you um I so I've like recently actually got

a really excited about like what ways we can take detection or response like skill sets and move them like elsewhere in the organization and like very specifically for me I right now am fascinated with the financial fraud space um so at brex we have we have like a fraud compliance like org that is separate from us but we collaborate um when we do have like incident level stuff when like fraud has stuff and I'm so fascinated like the work is so analogous but using different tools you're detecting slightly different things but also there's the security aspect too so I'm increasingly excited about like how can we take the cool stuff detections response does and like

apply it to other places within the company because I think that we also like our team specifically has a lot of data engineering like chops and skills that we can also share too so it's like that's the part I'm also excited about like I love textual response and being able to like take what we do to other spaces that are really adjacent is like that's my my new my new thing I'm on right now it really is applicable across domains I think sometimes we Silo ourselves and we're like we do this very specific thing but what we're actually doing like the meat of it is so applicable to others one thing that also keeps me too is um like alert management

and cue management like a support cue like we kind of do that like we actually have a lot of ability to tell people like why are you doing that just turn that off there's no value and other people don't think to do that in some ways yeah so I think we have a lot to share all right Julie yeah so one thing I've been thinking a lot about is how we're writing detections and how they exist in the system and I think that the traditional way is like you write like a query and proprietary language of some sort and then you store it and it's looking for a specific behavior and then you match on it but it doesn't have this

versatility to it um I think like a more modular approach where you're maybe writing logic that's behaviors and then you chain those behaviors together to represent a technique or an attack that you've seen and then that way um when you're as a detection you only tune a small part or you're able to add all these parts together that you've already built into a new detection and I think that versatility and modularity is going to be really important and actually like showcasing showcasing the skills of the deduction engineers and actually modeling the attacker's behaviors in a more sustainable way so you're not constantly editing or writing new detections when one little aspect of their technique has changed

what if you told us what what AI you're getting kicked off the panel sorry excuse me I want to say quickly what I'm excited about is just people sharing what they're working on I want to just give a shout out to Brax like part of why I was excited to have y'all on the panel is you're both creating new systems but you're writing about it and you're sharing what you're doing and I think a lot of us don't realize how interesting what we're doing is and how interesting it is for other people so like please you know do talks write blogs it really helps other people and open source your stuff and AI and Ai and AI yeah yeah all right do we

have questions yeah before we open up the questions I just want everybody to give a round of applause for these oh thank you [Applause] [Music] all right so um they have a lot of enduring fans it's just me so if if anybody has a question just raise your hand and I'll try to pass you to Mike because they can't hear you from up there especially I also really can't see you yeah they can't see you either the lights are bright so if you could come down a little foreign I'm an old man

uh I think it was Louis you said but this is a question for any of y'all I'm curious what you think but curious how you feel detection as code has enabled you or not to validate whether now your detection is actually catching what you're looking for I think Louis you mentioned like looking back and saying oh we should have caught that or not is that something that detectionist code has enabled you to kind of program in events plant flags that you're looking for or uh yeah what's the deal there yeah so we actually along with the Texans code actually have the adversaries as code as a part of our testing pipeline such that when we write

the detection we also have an adversarial test that should trigger that event right so if we're not essentially testing our detections we're not really doing the the full job of knowing that this works we're just throwing something over the fence and hoping it will actually detect something so definitely big enough like codifying attacker behavior and making that the testing mechanism for actual detections no next question

raise your hand though I think someone's just walking around we can yeah we can we can talk more about um kind of others question cool what are some of the hardest parts about making this revolution dream a reality for each of you hiring yep okay tell me more about that what is it about hiring that makes it I I feel really fortunate that um the I was hired in to be a manager of a team that already had like an engineering Focus to it and I got like lucky there are people who are already there and and hired before I arrived um so yeah but hiring like was a challenge like trying to find another

person finding someone who has a mix of the software development like skill sets and then also the operational ability to do an actual like incident response and run the incident is like a really challenging overlap to find because you you're typically finding people who have more of an analyst background like they don't they haven't been giving an opportunity to do both those things yeah hiring is very hard yeah I think another thing is like leadership buy-in so if you're at a company where you have security leadership that are behind this idea of having like hiring predominantly engineer people with engineering skill sets and doing this like detection and response um engineering first style then you have a lot of support and that's

great but there's a lot of companies that are maybe a little bit older school that see security response as having to be a sock and so being able to push change for that I think is going to be really really difficult like I think that's why you see a lot of detection and response teams at newer tech companies newer startups because they were able to start the team from scratch with engineering first principles rather than having to go back and change everyone's perceptions and get buy-in yeah absolutely we have another question here okay okay non-hiring challenge I mean the only thing I've seen was like try and break folks off that uh reactive mindset into a proactive one because

like you could have the like a brand new engineering program we're like yes we won't be engineering first but then you have analysts who are only used to being reactive yeah and kind of like breaking them out of that mindset of hey I have to react right and I'm like no yes spend two hours build a good detection and it will save you time later it's getting people like into that bought into that idea basically for sure I call it the reactive wheel of hell where you are so stuck in operational work that everything is coming in at the same volume and severity and you're just running around with your hands on fire setting everything else on fire

um no one relates to that [Laughter] hi um this is kind of a question for a comment that both Jess and Louis made so you mentioned kind of seeing the gap between how financial fraud detection Works versus sort of like security incident in response do you see some patterns potentially like going towards the security side so for instance things like synthetic data sets and using those or is this something that people already do in industry I think it depends a lot on like what your fraud team looks like um but typically from my perspective like from what I'm seeing the fraud team operates a lot like a sock um there's uh there's like levels within

it like there's Specialists for escalation some people who do more of the the data the actual data modeling for fraud and detection rules is backed by a data like an actual like team of Engineers but then the analysts who are actually operating on the work um you know they have good knowledge about what they're doing but they don't have that engineering level knowledge so that that's what I see the in the parallels the most is from what I the way I watch this the fraud teams operate it's a lot more analogous to a sock cert than it is to like an engineering team so that's what like makes me really excited but it'd be really hard to push

that model like that would be a really challenging one I think that's why I'm also like fascinated by looking at the fraud space yeah I think for me what I've seen is like the kyc type of type of motion in both spaces is really important and if you just have take away that singular piece and focus on it both sides can really benefit right because when it comes to like identity in your applications or within your environment if you have a really strong signal for that you can solve for fraud and you can solve for most security issues yeah 100 plus one here hi so you basically talk about how you want to use AI to do a lot of proactive

security response and I think actually that's pretty ingenious coming from an SRE background because firefighting [ __ ] sucks because that's my language um but I wanted to know for the AI side are you actually building custom llms or are you using like chat gpt's professional license as an exposed API and then using some of those agents that they have to build those models and those tools so this might be a bit of a hot take maybe some folks in the room from some of these companies but I'm so ready I don't believe we should really be leaning too heavily on third-party llms it's not actually really complicated to have an in-house llm that's a lot smaller to serve our

specific use cases and that's basically what we're doing right using these smaller more fine-tuned models instead of relying on sending our you know really confidential data out to a third party wait you mean sending all of your logs and customer information to a third party that doesn't have strong privacy or security policies is not a good idea oh no no who hurt you all right we have uh last question

there you go you're someone's really oh so um so great talk thanks for the panel so so uh for the vendors in the room [Laughter] you just outed yourself so uh I understand having having worked for both open and closed Source uh tooling but when it comes to detectionist code how would you like us to when it when it comes to say we're different customer bases different user bases want to make different changes at the same time how would you like us to manage that would you like to say maintain a fork and we offer you changes and you decide to accept them or not or have you thought about how that might work yes yes

yeah that's exactly what we do right now it's like yeah let us work but also we want to benefit from what the community and what the company because you all have in-house subject matter experts and if you don't please get people who do security working in your company yeah like the the model of like this is our official stuff we offer you this is what you do and here's the community stuff you might find helpful those are kind of the three buckets I think of the detections falling into and remember sponsors paid for Visa so that's why we didn't trash them directly no we said no did I say any names no no no no all right all right thank you guys

we'll be around thank you everybody thank you [Applause]