Tim Schulz - Automating Adaptive Adversaries

thanks everybody that's hung around this uh long i know uh doing conferences all day can be a lot going from speaker to speaker let me swap this up all right so uh as was mentioned before we're going to be talking about adversary emulation and automating adaptive adversaries a little bit about who i am i'm the adversary emulation lead at scythe now that's only been my role for the past year or so prior to that worked at sandia national labs i was on the information design assurance red team there for a number of years probably about five years total and i also worked at mitre on the mitre attack team worked on caldera and the attack

evaluations so uh if if you know me and you know some of my talks attack is going to pop up so just warning you in advance now our word of the day is scale and when it comes to talking about adversaries especially adaptive adversaries they're looking at having bigger impacts so automating their actions those types of things that's really what they're trying to do day to day and ultimately what i'm looking to give you all in this talk is not only just sort of a view in how you can do it but also just how to think about it and give you some examples and sort of a modular framework to work through so that you can

pick an adversary that you care about now everyone's going to have different adversaries that target their either industry or that you might be worried about so that's something that you can sub in whatever adversary you want as we walk through these examples and the idea behind this is that this is something that you can hopefully take to your day-to-day job and use as well as something that you can sort of follow along with here so automation really allows consistency at scale this is something that's really important uh sort of a simple concept but overall is why we want to automate things getting a bunch of experts together in a room is something that's great but

trying to get them to hack everybody on the planet if you're a nation state is something that's much more difficult to do you need tools you need ways to scale that expertise and so that's where when we're talking about adaptability it's really capability what can they do consistently what are they able to build and what is the objectives they're able to consistently basically fulfill for whatever their mission is now that's why i sort of like to sub this here is we're actually not automating the adaptive adversaries what does that mean it's mostly the capability their own development how they do research and how they transition that research into something that's scalable now with something when it comes to red

teamers pen testers things like that is that they like it's easy to think of yourself as adaptable because you can get rip past anything if you're giving enough time money uh you've got something like kali linux and that's sort of how you gear up now this is not how adversaries gear up they don't get a special version of kali linux necessarily they have a couple different tools and that's where we talk about scale individual people with kali linux frankly does not scale it's not something that by itself works at scaling up to the nation state levels and so that's that's really the main message here when we're talking about scales that individual expertise it doesn't scale it's not something that uh

it's good for trying things out it's good for learning uh lessons and maybe for business impact and the types of tests you're trying to do that that can work but when we're talking about adversaries that are constantly adapting to a changing defensive environment that have to target different organizations they can't afford to have new people every time try and relearn something and so that's why we want to talk about what are some of these tools what are some of these ways adversaries do it now this is how adversaries gear up so if you're familiar with the left the screenshot on the left side here this is cobalt strike and lots of adversaries especially ransomware threat actors have been using

cracked versions of cobalt strike and like we have seen a couple i think it was about a year ago now uh somebody didn't get paid in the ransomware world and as such they leaked out a playbook and so this is the kanti ransomware that was leaked and of course it was in russian and then it was translated and so what you're looking at is the translation there on the right now this is better documentation than any offensive security tool or playbook run book whatever you want to call it that i've ever seen or anyone really had seen and the idea behind this is that the operators the people that are going to be using this can copy and

paste that is the only skill they need to know they can copy and paste everything in fact they do it so well that if someone did a typo in the run book they will copy paste that too and it won't work but that's the level that they are trying to go to they are looking for scale they want anybody they want to give them a cobalt strike terminal and say just copy and paste these commands that's all it takes and so that is scalable because if you can get anybody that can copy and paste that's a much easier skill to teach than somebody that needs to be a red team or penetration testing expert so how do we measure this capability and

how adversaries are looking at it the overall concept of this comes from something called worldly maps it's simon wardley does value chain mapping and the idea behind this was that it was something for looking at how to map businesses in competition and it really breaks down to three i sort of took it butchered it a little bit and broke it down into sort of infosec terms and i've broken it down into three major bins when you're measuring capability the first thing is research it's everything starts as an idea you've heard lots of talks here that talk about some of the different research ideas as people are sort of pushing the boundaries of their field and this is where things are unproven

thing a theoretical attack may or may not work and in this example i'm going to give before diving into sort of the cyber security example is wishing to fasten two things together now the next thing after we have research is we have custom this is where you have your proof of concept something that is custom built that takes an expert to operate it and it may not even work every time and so this is where if you have fasteners wanna fasten two things together uh and same with like nuts and bolts used to be that artisans made these uh nuts and bolts individually only one worked with the the only artisan that built it was the

only one that knew exactly how to carve it out correctly and that's great but it again doesn't scale because when you need to build a ship or you need to build uh you know these days anything from ikea for instance you need a lot of these and so that's where the product part comes in is that this is really a maturation of any sort of capability you started as research but as you're trying to scale it you're really trying to get to product and so that's what we're going to do for talking about cyber security capabilities now of course let's dive in and look at reframing this challenge with our new three bins that we talked about and this

is where i warned you before mitre attacks coming in because this is a way it's an open source framework it's a really good way of charting how real adversaries are performing their actions now if you're unfamiliar about how mitre attack works or haven't or you've heard about it but don't use it it has basically three layers of sort of abstractions uh the broadest one is tactics at the bottom and this is going to be the objectives adversaries are trying to achieve it's sort of the 10 000 foot view it's things like initial access resources that kind of thing now under each of those tactics you have specific techniques and this is where we start to dive a little bit deeper into

each of these things techniques are going to be the technical means by which adversaries achieve those goals and then we have our procedures which are our most granular this is going to be the exact things that the adversary ran on a system for instance in order to achieve their goal so let's look at starting from the broadest drilling down deep and this is where credential access is an example of a tactic it has this little t number t a and then t is for technique and then procedures really don't have a mapping so that's where we'll talk a little bit about some of the challenges of that a little bit later but when we're talking about mapping

your different tactics across these this is where you can either map your team you can map the tools you're using you can map a specific adversary now defense evasion is something that tends to be something that is sort of cutting edge but a lot of adversaries and red teamers they all have this as custom right they all have to build this each and every time give credit to the edr vendors they do a pretty good job and or at least spend a lot of money on people that do good jobs on building out really good signatures based on specific tools for instance meme cats it would be one of those but defensive asian to make sure

those things run tends to be something that you have to do over and over again and continually do so that is custom now discovery this is something looking at assets on a network adversary's trying to discover things for lateral movement things like that this is something that does tend to be more of a product and map for instance and so this is how you would begin to map out the different tactics that you're concerned about either about the adversary you're concerned about or you can map your own capability now once you've gone through all this you've got this nice sort of stacks and you'll notice in this case we don't have a bunch of research partially because

research is so resource intensive and not every team has people that folk are focused on research most of the time it is focused on what are the products we use and also what are things that we can supplement whether it's integration stuff like that to sort of speed those up and that's where we get to attack chains so this is where you can start to build things together to identify what are those specific techniques are sorry tactics uh and which ones are we do we rely on specific people for this is something i always remind everybody with adversaries too adversaries have deadlines uh that was another thing lots of leaks out of the the conti ransomware camp they had

some of their emails and chats and all of those things got spilled and this was over their support of russia in the for the war in ukraine and so somebody spilled all of those things they went through uh they have everything from reviews with managers they have some of the same problems i'm sure you all are going through at organizations where someone new got hired and they're making more with than the previous people and birthdays all of that stuff and so that's something you have to remember they have expertise too they have to chart this stuff out too and so they don't have it's not just a blob of expertise that somehow just can go after anything they have to choose

both the targets based on the expertise they have as well as try and map and build new capabilities so that's why it's really important to be able to map all of these things is that you can understand where where they're going to shift now not all ttps are equal right and this is something where i give the example of if i'm going to start you and say you're starting externally uh from an engagement and you have to do discovery and that's that's where you're starting versus you have a persistent agent on in a network what are you going to choose if your objective is to get in the network you're going to choose persistence every time but that's

because it's harder right and this is why you can identify those resource heavy points and especially if you have specific things in research that's where that is super resource heavy it is going to require experts and expertise and that's why when you're looking at this you want to shift things to the right and so that's really what we're looking at is shifting all of these things as much as we can to the right now every ttp is not going to be able to be shifted and your team might not have the budget for instance to buy a tool or a product that puts puts your k your capability in most of the product but adversaries can't either

ttps and just general adversary operations they do have a lot of difficulties for instance users users always do different things and while we most of the time talk about how users make things difficult for defenders which they tend to it also means that attackers have to discover what's happening in an environment before they can do things like lateral movement unless they have an exploit they tend to leverage whatever is in the environment which requires waiting and learning about where they've landed and so this is where you really want to see where are single points of failure too there's another thing if you have a team full of people and you rely on one person to get

you initial access or to do defense evasion each time what happens if they leave right we're in the middle or towards the end or i don't know how they're charting it of the great resignation right i've talked to many security companies that are all like hey if you know anyone good send them my way and so this is something that's increasingly uh something that more and more organizations are looking at is how do we bake expertise as we get it even for a short time into either a specific tool or something that can be transitioned to to someone new now let's talk about how we can potentially shift right one is automating technique changes so

this is something that mitre attack is updated uh they do a big update twice a year adverse areas are really active they actually uh you have all these new cyber threat intel reports that come out on a pretty regular basis sometimes it's monthly sometimes it's more than that and then you have big vulnerabilities that pop up and everybody publishes something on it and so if you're following some of this stuff you need you need to make sure that if any zero day pops up you can deal with it but you still have to deal with everything else too and that's where automation comes in now one of the challenges with automation is that this is this top one is i think one of

the biggest challenges in this space is that we tend to really focus on the research and custom parts of what we're building we researchers want to come up with the research identify something that they're doing build a proof of concept to showcase that it is an idea that works and then that's where we leave it but the challenge is that that is still something that's important and needs to be brought into a product eventually uh for a lot of those it's it's vulnerabilities for instance that are still there that are still going to be deployed in systems i mean legacy systems there was a talk earlier that sort of talked about all the legacy devices that are still across networks

and so this is why making sure you're able to identify old things through automation and scale is really important not all techniques are scalable this is something else that is is good in some terms and is bad in the others uh it's good in that if every technique was scalable it means ever adversaries could create worms or easier and faster which that luckily is not necessarily been the case but it also means that it requires more expertise or people to work through and make sure that something is tested in your environment on a on a more recurring basis and then this last one here is something that a lot of businesses struggle with is that not all techniques

are relevant especially when it comes to mitre attack and people are hearing about it more and more and more they think we need to check every box we need to make sure that we have 100 miter attack coverage that is not how miter attack is supposed to be used from the attack team themselves they will tell you that it is not a check box exercise and so that's something that you that is hard to sort of instill in leadership when you're showing graphics and they're like why do we have all these boxes that aren't covered and saying it doesn't matter to us is not something they always understand so those are challenges here with automation is that trying to figure out

what do we want to automate and what is going to and we have to prioritize now let's give an example of why automation is important when it comes to adapting to adversaries who also can make a shift so let's start with powershell um powershell is something that continues to be a problem uh let's see over a decade i don't even remember when it was first introduced but as far as security talks addressing powershell dave kennedy gave one i think back in 2012 2013 uh that talked about holy moly like powershell is a huge attack surface and if you're a pen tester like you can just run away with it now there's been better detections and

things like that that have come around powershell but there have been other research techniques that have been built into products that people aren't as aware of and that's unmanaged powershell it's powershell without executing powershell.exe you call the dll directly and this is something that requires a different set of detections and so this is really key as an adversary can switch from powershell to unmanaged powershell and previous detections could be useless so you need a way to do that now the question might be how difficult would that be for adversaries the top here is a little screenshot and a snippet from that conti playbook they have this powershell invoked nightmare which is print nightmare and then they have

basically adding a new new user and this second one is a quick cheat sheet from cobalt strike that shows exactly how hard it would be for the adversaries that are building this to make this switch in their playbooks it would be literally switching out the word powershell to power pick remember i said earlier they're copy and pasting everything they don't they're not going to think about it they will copy and paste it and this is something that's very easy for adversaries to change it should also be very easy for you to change in your environment because this is something that's potentially coming at literally any time i don't actually know why adversaries haven't picked up on this

yet but sort of thankful that they have it because when it happens they will have unfortunately more success unless more and more people are testing against these so this is where you really need some sort of tool that can do this if you haven't worked with some of the red teaming tools before there is something called the c2 matrix this is an open source community-led project and it basically has all of these c2 platforms and for a while there every week a new red teamer was releasing a new command to control framework on twitter and so they took all of those and they've charted out the capabilities so that you can see what you need to

test it has open source solutions on there as well as paid ones so you can do comparisons so it also if you don't want to look at the google sheet it does have like a questionnaire where you can ask about the different types of features that you want to test and it'll tell you which ones you can use now adversary emulation is another way that we can tackle some of the challenges let's talk about some challenges with that though is that it is still a very expert driven space uh and so this is that's really the biggest issue with it some of the breach and attack simulation vendors are trying to address some of it

but largely right now the pairing of cyber threat intelligence with uh with red teamers that can essentially execute it either in a defanged or benign way is something that is still again very expert driven and so that's that's something that is an issue here now it can also solve some really hard problems for us so let's talk about an adversary's capability shift initial access shifts from custom to product what's the way that they would do this how do you buy initial access well you can buy a zero day now this is something that previously sort of the economics of ransomware uh sort of changed the equation around it zero days were expensive and they have continued to get

more expensive but for instance uh the kante ransomware group made two billion dollars i think last year so that's a lot of money and they can probably buy a lot of researchers and a lot and they can turn that money and a zero day into money really easily and really quickly right and that's another thing with like uh the f5 exploit that came out most of the time if something hits really quickly most adversaries especially when we're talking about ransomware they do the same thing they're just they're doing everything else the same they're just shifting how they do initial access now if you're not familiar with zerodium they're a exploit vendor company and if you can't see all of these for instance

a windows remote code execution with zero click so no user interaction if you have one of those to sell to them you'll make a cool mill just like that so my question is is this in your budget right if you want to do a zero day to make sure that you're keeping pace with adversaries then this is what it potentially is going to cost you and remember this is if you're a researcher that has one not if you're trying to buy it which i'm guessing has a slight markup so this is something just to keep in mind and i still have to have conversations with people when they say we want zero day exploit testing and i

pull up this chart and ask them if they want to add that to the quote so that's something that there's an easier way and if we want to do this from sort of the blue team perspective it's assume breach testing assume the adversary gets in all right what would they do next and so this is this covers a lot of different scenarios even beyond the zero day we have insider threats we have uh whether someone clicks an email that's something that always it is with all the security training that we have these days there are still spear phishing emails that will land every single time there are people in your organizations that are paid to open

emails and that's something i'll give my example here if the head of hr gets a proton mail email from someone that it contains an attachment and says this is an accusation of sexual harassment of a vp or the ceo of the company and you need to enable macros to see it they're going to click it every time because they should it is their job right to make sure that they're checking that the company is would be covered and that's potentially a plausible scenario so they're going to click on that and they will do it probably every time there's other scenarios too so you need to assume that's going to happen and adversaries don't get very creative

after that these the same things over and over and over again so that's where assume breach testing allows you while it's a little bit of an extract an abstraction it allows you to tackle a lot of those scenarios all at once same with insider threats now moving up a little bit into something a bit more granular this is a technique a specific technique id and in this case it's actually a sub technique which attack couldn't just keep it to ttp they had to make it a little bit more complicated and so that's why they added sub techniques a few about two years ago and so under credential access we have this os credential dumping lsas memory uh the

tool that made this popular is mimi cats so you're going to have to do this with all of your specific tactics that we've mapped out here and so this is what it might look like now even assume breach scenario user execution is a general technique that they have that includes a lot of those things that i mentioned it's got somebody clicks a link somebody goes to our url somebody opens an email and then we're going to talk about process discovery a little bit more in depth and then we just have all of these now notice we can't execute anything yet but we're at least well on our way to understanding how we can test against

this adversary and how what parts of this that we can automate and then this is the final one which is the procedures and so techniques and procedures really go hand in hand uh they of course have layers of abstraction because you can have multiple procedures that map to a single technique and this is that next step in understanding can i automate this procedure or is there a different one that i can so and this is the case where you can use proc dump to dump lsas and that's just some syntax there that you can run this command and you will get an lsat on now a lot of questions come when you're operationalizing attack or trying to

figure out how do i what are those procedures like how do i make that map if you go into mitre attack you'll get mixed reviews procedures are listed for a lot of things but sometimes it talks about this adversary uses a specific piece of malware and that's all it'll say and so now you have to go and figure out like maybe you don't have reverse engineering or you have to go and figure out somebody that says this piece of malware does something this way because you remember human operators are still driving that malware right and so a tool that was meant to sort of address this problem was red canary's atomic red team project it's a great project it's

sort of how people get started when they're looking at mitre attack and they want to execute it within their environment now a quick disclaimer about it it's a great place to start but it is not complete it is a community driven project and it is really really good uh if you're interested in getting started with it they've got a really like i said really welcoming community you can easily contribute it's it's something i always recommend people go and check out and see uh see some of what the community has built out around it but it has sort of become a check box exercise as i mentioned before mitre attack has become something that everyone wants that check box at the

technique level but we still have to go with procedures and testing of individual techniques is great for checking your logs but adversaries never execute a single technique not in a vacuum they tend to execute them as a chain and so this is where atomic red team tests come from unit testing or atomic unit testing in software engineering that's meant to test a very specific thing and so that's where it comes from it's really good for for attacking or getting started but once you're trying to automate or emulate adversaries it sort of falls short uh let's see the last thing here is that the sort of secret with attack is i've shown you a really clear example of

going through a tactic a technique and a procedure but almost every single procedure requires two attack techniques in order to work and so that's not always clear so let's talk about process discovery one of our this was in the discovery tactic process discovery's the technique running task list on command line is one way of doing it here's the thing command line is its own technique this is where i said before you typically have a goal that you want to achieve and then you have an execution method to achieve it but those are two different miter attack techniques now there's other ways though to achieve this too and we're we're talking about adaptive adversaries they want to

achieve the same goal oftentimes they want to understand what are the processes that are running they're not going to invent new ways to do it but they might look for ways that might not be either logged or alerted on and so that's where there's a bunch of different ways to do this we have windows command line we have powershell we have wimmick or wmi is all different ways all three of these commands give you the same info in different ways now there are atomic red team tests for all three of these and that's great because you can either use the command line or powershell in order to execute them however there's another way that is the windows api now api calls

are something that are not well uh detected on right now most edrs actually fail to detect these types of things that's got some history behind it some of it is because uh with if you're intercepting api calls you can see basically everything that's going on within the system it's hard to say uh you're you're basically saying i need to moder literally everything happening on the system for the greater good of security and so that was a privacy concern when people were first looking at it with edrs and they said no we don't want that level of introspection we don't need it but more and more adversaries can do this every single one of these is listed

as a miter attack procedure which means that real adversaries have used these exact commands in these in this case this exact function in real compromises and the reason that the native api doesn't work with atomic red team is because it requires more than just running this it's a function you have to write a program that's going to call it directly and so that's not something that's necessarily easy to do but it is something that you still need to be able to test against so that's some of the challenge with atomic red team is that you're not necessarily going to have something for each of those tests you're going to have to figure out all right what are these different

processes that we can use in the case of this one persistence it's a long uh a long scheduled task creation and so that's the the granular level you have here is what you can use to figure out whether you can swap things out deliberately in order to emulate specific adversaries and so that's where as you build this out you're potentially going to use other tools and other cyber threat intelligence in order to figure out these if you don't want to do it yourself there are some resources that we have here mitre has miter ingenuity they have their center for threat informed defense is what they call it and so they have blogs they have a

github that has all of their emulation plans they also have the attack evaluations which actually walks through of all of the different edr vendors they have like it's like 20 or 30 of them at this point that they run through these tests and they do these comparisons so that you can see what are the specific techniques what are the procedures that these edr's are able to detect or are not able to detect now scythe we also have something called threat thursdays we release these uh once a month and uh and the idea is that they are open to everyone in the community mitre has much more detailed plans i gotta give them credit like that's the gold standard for it but it

takes them eight to twelve months to release these things because they do a ton of work with them so i'd recommend checking those out same thing with cyber threat intelligence reporting the d4 report is another one i like to give a shout out to they cover a lot of ransomware engagements and they really go down to the procedure level and so that's something when you are looking at cyber threat intel that allows you to build out this type of this sort of level of test that you're going to need because otherwise you're going to be filling in the stories with holes yourself so just wrapping up here i think the the key thing that i really want to get

across beyond just giving you some of the ways to think about things as well as some techniques to do that is that automating techniques and capabilities is just as important as researching new techniques this is something that i think in in information security it's really easy to get swept up in all the cool new cves people come out with but there is a lot of really good research that's done that needs to be built into a tool built into a product and that will overall be something that helps the greater community and so that's something that i just want to highlight and sort of call out to everyone that's doing that thank you because i don't think

people get enough credit all right so wrapping it up here's for uh since i'm the last talk i've got a couple you know if people have questions happy to stick around otherwise you can catch me afterward or around [Applause] thank you very much cool questions comments question um this whole lecture seems to be telling me that we're talking about evolution on the red team side that we're going from like what we would call a pilot in an aircraft to just an operator sitting at a terminal like the old teletype operators is that the true so the question was around let's try to paraphrase that is the is around are are we is this sort of a paradigm shift from

having a specific like expert that is driving everything versus someone that's that people that are doing things sort of let's see more more remote like scaled out uh and i think i think the answer for a lot of it is yes i mean i i think from a red team uh red team perspective that area has sort of shifted even over the past three to five years uh red teams used to do basically a lot of the things that i just talked about up here but they are increasingly being asked to get around edrs and those types of things and so they have to dive deeper and deeper and deeper into understanding how those things work

and and i think that's something that has required a level of expertise that continues to go up and so not quite sure if i'm answering your question here on this but i think we we are seeing a shift in the types of expertise that is required of red teams and it's also meaning that you have tools that are sort of back filling what they used to do because there's such a need for testing across technology right even five ten years ago you had a lot of businesses weren't quite forced yet to use too much technology in their day-to-day but i mean we've seen ransomware uh actors hit um hair salons as well as of course you know fortune 50

companies and so from that uh standpoint uh you probably are not going to have a red team come and do a test for a hair salon right um you know that they they just aren't going to be able to afford necessarily that level of expert and that i think is actually the majority of companies won't necessarily be able to afford a red team to come in and do something so all right yeah well your question yeah um so i was just thinking about the concepts of blackboard it's called a small security team

the guy who um security and just thinking how important is it for people to be able to test your environment to me it seems

they're upset because we're going to be pulling back on their 30 at short fleet

so the question was around uh whether you should have people internally that are uh attacking your own infrastructure providing the red team uh and my answer to that is yes i would call it a purple team though um and and to address part of what you mentioned is that animosity sometimes between teams sometimes when it comes to red teaming and penetration testing there's a win-lose mentality and that's something that tends to tends to drive tensions high right and so when it comes to purple teaming it's sort of taking one of those specific things so i'll jump back in my slides here uh so let's pick take this page for example you can anything that you can

link to mitre attack you can say this this has been used by a real adversary that's how they scope down attack is that they want to make sure that only things that have been used by real adversaries and that goes back to what i was saying about not every technique matters and uh because if you're just following red team researchers on twitter like they're brilliant people but they're releasing new stuff like every couple hours essentially where if you went to your one of your teams like your sock for instance and you were like we're going to test task lists and then we're going to test this and we're going to test this and see you could be

transparent about it everybody wins regardless and i think when it comes to talking and building relationships with other teams right that is something that takes a lot of time and being able to try and make the rest of your processes more efficient to allow for that sort of human interaction team building time is important i think that's where you know the short answer to your question is yes you should do more things internally if you can but as you mentioned teams are stressed these days from so many different things they're asked to do and so that's where if you have tools like some of the ones i mentioned from like the c2 matrix and other things like that

caldera is another automation tool that is open source and free uh those are the things that you can do that hopefully to help you free up time and provide you uh sort of a quicker way to address some of those things but yeah i mean that's a tough problem but i also think that's a super common problem right now across teams and you know that's where red teams are i think being asked more and more to try and be more collaborative to try and help uh help build bridges within organizations because of some of those okay good yeah another question um

so yeah the question comment was uh was that skeptical that that companies that hire red teams to bypass edr leverage that and i would agree i think the the understanding of what it takes and also that's the type of thing like red teams are are full of experts that's a lot of people don't like see sort of behind the curtain of all the things it takes for red teams to do these days to be successful i would say the money would be better spent making sure edr is running everything because they probably got it yes and and to repeat the question for everyone is uh is that making sure money is better spent with with more

collaborative efforts like making sure edr is running everywhere that it's tuned those types of things and i think yes we're seeing more of an embrace of those types of uh approaches in organizations and that because the organizations that really do benefit from a red team that have you know that that can bring in that expertise is a very small subset realistically and so trying to make sure that there is sort of a broader more collaborative nature around security can help address some of these problems that i know a lot of organizations and i've been in several that have had teams that are butting heads constantly because of actually past incidents between teams and so there's that sort

of history within the organization that makes it really difficult to yeah achieve anything so yeah good good point all right i've got three things here first question uh goes to you here and then i have two more uh backdoors and breeches all right oh all right who doesn't have one there we go all right come on down what's that ah that's all right you gotta get your prize can't be shy for prize one more let me just throw it all right all right you ready all right i'll underhand it we'll see how this goes oh it's all right that's right luckily i'm in cyber security not athletics anything else cool thanks everyone thanks again