Does This Thing Actually Work? Testing Defensive Tools

okay we're live hi everybody i am this guy stuart mcmurray i am a lead engineer on klarna the payment company's ever-growing security offensive security team contesting red teaming that kind of things also just a gold fashion neckbeard unix nerd um these slides should be tweeted now onto twitter so feel free to grab twitter if you want copy the slides also welcome worth mentioning that i'm not affiliated with any company organization or cartel or anything of the sort which does defensive type things that we'll be talking about so this is my own my own opinions um and even more so it's not the opinions necessarily of my employer past present future and so on um also

typical disclaimer please like don't hack things if you're not allowed to um if you do my name is george washington keep that in mind when somebody asks so back to the talk um quick deal they are of the talk uh we have this problem that we don't really know if defensive tools work until the tools are tested um point number two is you have to figure out how like who knows um vendors are probably not the best to ask and in fact in my opinion they're like the worst people to ask is i don't know they're selling something um and and the whole thesis behind the talk is the best people to ask for the blue teamers to to

figure out if their tools are working um and point three it's you know doing the testing itself is actually quite fun and it's pretty easy and it's an enjoyable thing to do and that's sort of what we'll be talking about so over the next 40 odd minutes um we'll go we'll talk about sort of the idea behind this talk uh and then it's a little weird to say yeah blue teamers should doing security testing it's kind of like the opposite to what they normally do then we'll look at sort of my red ish approach to a blue problem and how to test things and we'll take a quick case study that i was originally going to like use

a tool i've gone against so to speak uh we won't do that because that wouldn't be very nice so the idea behind the talk is we have this problem um every every large organization with the network needs some sort of security tooling in place we need the ability to find bad guys stock bad guys make reactions so on and so forth we have to figure out though if they actually work and it turns out a lot of blue teams that i've talked to don't really know they're like yeah we know what it should do we know it has done we don't know if in the future it will keep doing um also these tools are

ridiculously expensive so like um back before i moved over this way of course the us had a car and it would not occur to me to make like a massive purchase that's like multiple times my income uh without test driving and i i feel the same way about security tools like you know multi-million dollars or euros of tooling um you probably should test it before you buy it there is a counter argument to that and that is that it's somebody else's budget really like you know it shouldn't be on the budget of the organization buying the thing to figure out if the thing works like do dev work basically for the vendor so here's the solution

let's just see if this thing that we may be hoping to buy or may have bought already is actually worth the money we'll just dig in and find out if it works um in the interest of a 40-minute talk scope it down though um so when i talk about a thing a defensive tool i really am talking about like a single defensive thing i'm talking about an edr box appliance agent what have you a network traffic analysis thing a firewall thing maybe even like a dns server that's configured to be particularly hacker unfriendly um the same principles apply to what you know in the modern world you get oh it's a cyber analytics a defense platform is

using ai and ml and threat intelligence and all that stuff it's same thing applies um tweaked a little of course and and every so often you'll find yourself or find a situation where it's not just like hey i'm testing a thing it's like hey we've hired this this other team to you know defend a part of the network sort of an mssp type thing it works more or less the same um with some minor adjustments and and we're going to again scope it down to one question to answer and the question is does it work which in real life something turns sometimes turns into like okay does it just work enough um which actually realize in real life often turns into

like okay does this thing do anything um and then sort of as a side quest we like to also answer and just how hard is it to get past it i mean like you know most most of these tools are not uh particularly impossible to bypass so that's that and that's that's just a fun bit anyways so it's like kind of your reward for doing all the work um and then the question is when you would do this and hopefully you're doing this pre-purchase hopefully you know somebody is like hey we need a new edr product for whatever reason um let's let's get a bunch of trials and test them and see what happens i've done that it's

it's enjoyable for sure and it's it's really helpful to see like hey the vendor you know we're being exactly forthcoming for this one and this one works better um probably more often it's post purchase it's like hey we have this tool maybe you hired a red team or maybe you set up a new red team or whatever and you can see like oh you know just doesn't actually do what it says um and and i say red team it's actually probably better for the blue teams um the other time i've seen it is post compromise like bad thing happened bad thing was done to the company really it doesn't just happen and the company's like all right we missed that one

what else are we going to miss or what else can we catch at least to know where to fix things a bit better which gets us then to the who's going to do it um the thesis behind this whole talk is that i think the blue teamers are the best not just like because i have other things i want to do but blue teamers in general are the best suited to do this testing um start from the other end like who is not good managers stock managers sizzos that kind some of them are awesome um i have to work for one some of them you know they have different priorities they're more concerned with kpis closing

tickets what not uh sometimes they're a little less technical than would be expedient to know exactly how their tools they're meant to have work um the other obvious answer is usually the red team the fantaster so on and so forth uh quick pull of the audience who's on the blue side of things so about a third maybe how many blue tumors cringed when i was like hey yeah we'll just get the red teamers to test the tools one one is honest in the crowd all right so um we have one person who's honest anyways but yeah um that having been said ask us for planning ask us for like you know hey how else could you get through this

how else could you would you attack this what you know what other tricks you have up your sleeve um besides we also don't really always know the entire defensive scope of the company or the organization as it turns as it may be um and then then the other you know and actually probably the most often asked if the thing works is whoever's selling it you're like oh can you map this to whatever framework or matrix or whatnot and they'll have a nice list for you of test cases they've done and um there's the benefit that it's free hopefully uh it should be anyways there's the downside that um you know i have cat pictures and that's

a cool cat picture so that that's who you wouldn't ask um who you would ask or the blue team uh ask them because they they know all that or hopefully know anyways all the defenses that are there they know probably the gaps i i used to work with a fellow who went from blue to red and um actually it's running the red team i think the organization i was at now and um we hit this amazing thing you would just rock into a company we were working with and you just go to their sock and be like hey it was a sock analyst how do i hack you and they'd be like oh dude all you gotta

do is like use this creds there and jump to the system so yeah the blue team knows um and they'll know they've used these tools every day so they're you know pretty pretty set even just from that point of view um more importantly blue knows the environment that you're operating in they know that hey that you know we block whatever out we've got high cmp outbound say so no we don't need to worry about that even though the vendors may be iffy on that um they know some things are definitely possible like man we know that who is gets out happened once we know who is gets out all you have to do is ssh over who is and it

works um and they know you know where they may have a little less visibility which is what's kind of testing is good um and they also know like hey we don't just need like one trick to win on the red side and certainly i would never do this um but some red teamers anyways are like hey i used https and it won hey i used https and i got out of your network again hey by the way hd yeah so it's you know you get a broader scope of things um and so from that no it's missing uh the red side does know that too a few more side benefits you get from this um when i've done this in the past

working red blue you know the whole chain of people uh communication gets a little weird so you know you find yourself like i think oh i need to test this you know one set of activities one set of behaviors and the blue team you know analyst or whatnot i'm working with stock analyst is like not quite and then he goes talk to his it's like whoa wrong operating system something along those lines um also get a sort of more honest look at what needs improvement especially if you're hiring a external team to do this uh they like repeat business and if if your report that you hand a client is like yo everything's broken it's it's really hard to say and please

hire us again it happens but not that i would ever do this again um cheaper of course and i've also found when i've worked sort of on the purplish not like in the modern sense of purple team you're just like hey you see this kind of thing hey you know come come look at this cool thing i'm doing um i find that oftentimes especially like junior soccer analysts and it guys really love like the learning like oh how you do this one trick and such i'm usually smaller scoped so that is sort of a thesis the why behind it um here is sort of an approach my own approach to testing these sorts of things

for a bit of context mostly when i'm thinking you know tool bypass i'm i'm really thinking about two things i have to get past and thinking about um automated tools so like something making an alert and i'm thinking about whoever's doing the triage tier one sock something along those lines um sort of a counter example i know young lad who he had a file on his desktop i think lol.pie and because he just thought it was funny and uh sock was like oh my god what are you doing you know deconfliction happened and he's like i don't know what he was doing but he's like not playing league of legends because the soccer stock because you know the one the sock

was like oh my gosh file names are credible we'll believe this so you know flip that on his head like hey if i just rename a file and it works then it works so you know that's kind of like thought number one is okay just it just has to work enough um somebody asked me yesterday like how do you pack linux binaries and i'm like why and the answer is because nothing looks so uh upx i guess but yeah so that's the kind of thing um and then so that you know dependency to figure that out is okay what what can i get away with to get to that point uh so i need to know

what's not blocked uh because if something's blocked but not caught because i also need to know it's not cod something's blocked and not caught it's like well let's try again okay you know respond go again likewise if something's caught but not blocked well it still works i mean i'll get a socket manager mad at me probably but um that's kind of par for the course with red teaming so that's kind of the two things when we're formulating these tests we're talking about to think what what can we do to avoid getting blocked what can you do to avoid getting caught um and then it's always good to keep in the back of your mind what else can you

get away with uh you know it's it's sort of the red team mentality um context from the red side but like hey not not just like yeah i won yeah i got ceos you know dropped a report on the ceo's desktop or something but like what else can i do um this all turns into like fooling the system at five ish places um sometimes on the sensing side so uh network traffic analysis you know it's sniffing taking pcap and whatnot it can't be doing that for everything so i'll just like turn things off um i know of three different tools that don't look at icp so okay i can fool the sensing because it's not sensing icp processing

so um if something is taking a stream of events data whatever has to like turn that from some raw sensor output to something useful um so if i can break it there i that that's another win another way to break a tool which is then informs these test cases we're gonna be talking about later um good example of that we had something a tool i was going against it was parsing http sure um so i put at the beginning of my ssh stream like an http request header just like the one line get something something and then just ssh after that and then likewise from the server i just put like an http 200 okay and then like

ssh after that and and that poor tool then it was not expecting gigs and gigs of http requests and response simultaneously that broke um likewise decisioning is the next sort of step in this i hate to call it a chain it's overused but chain uh if you can break the decision if you can say you know hey you you sensed this thing on the wire you looked at it but you decided it's probably not that bad then that's a win too um i don't have a canned example off the top of my head but you know that is like fooling it where it's like yeah it looks okay um alerting as well so every so often i'll

do something that you know what else can i get away with i'll do like something like that and i'll usually say to the stock guys hey dudes do you uh do you see that and there is a tool without naming names of course that you know they'll say yeah it made an alert i was like cool would you have seen that and you know most of the time yes but every once in a blue moon it's like well the tool was like nah it's not that not that awesome it's not that important so if you can break it there and then of course the response is the other one um you know if you know even if you're like

oh my gosh the sky is falling but you don't actually kill the whatever needs to be killed well i still win cool so back to this this uh rambly philosophy step one what what are we actually meant to do what is this tool really meant to do um and sometimes it's not obvious if you if you get one of these uh enterprise grade cloud native ai ml low calorie intel driven makes your t in the morning thing does not tell you what it's actually supposed to do it tells you that like some some ceo's gonna buy it but um so you have to sort of sometimes really dig and figure out okay what does this thing actually do and sometimes

it's just as simple as like regex um sometimes it's as simple as like matching domain names uh sometimes it's complicated neural nets are involved and whatever else and then also how it does it so if you can figure out what source of data it gets in um that much you know the closer you get to like truth for lack of a better word the closer you get to uh figuring out how things work oh boy my timer just locked itself there we go uh so like the network tool i was talking about it it used a packet socket or something snapped snarfs traffic and pcapped it and did some math over that um the linux edr thing that actually did

catch me once and it just it parsed on it d output i was like oh so i just have to stay out of audit d no problem um things like that and so where do we find this well poker on the ui um which is something that blue teamers are way better at doing than red teamers um speaking only for myself i poke around like edr ai uis and i'm like i could do doing other things uh false positives as well so if if the tool whatever it is says hey this is a bad thing like the ll.pi like oh my god he's running a forbidden game okay we know it's looking for games let's see if

let's let's validate that let's try another game let's try something pretending to be a game so on and so forth um in a similar situation i found out it was like mac app like dot app directory things that was just a name matchy thing um spotify was not allowed by technical policy but allowed by like hr policy um and it turned out splattified that app worked really good uh and i had music and the sock was not happy uh documentation also um documentation can be misleading by the way but you know just like if it says hey turn this thing on to catch this thing but there is it can't be misleading there is a

tool out there i i um my name is not on it and hopefully it never will be but there is a like bug report in that the documentation says it blocks x and allows y and it actually allows x and blocks y so yeah keep it take it for what it's worth um previous incidents as well if you had a thing that happened and it did not get caught you're like maybe that should get caught or you had a thing that happened and i mean it's an incidence you somehow you had happened um that's a good way of saying hey this you get caught um so a call i had once and by call like i

was doing something it sounds like what are you doing and i was like hey yos made up yes credits don't you and i was like no they're like yeah you do it's you're coming from the wrong network for those creds okay so i know that whatever tool that was i don't know what tool it is because don't tell me but whatever tool that was i know like it catches aws creds from a weird place that's good it's good um less good sources that still may give you good information about what our tool does uh the sales pitch itself um if especially if you get like a sales engineer that will give you good information um

and also false negatives if if you have a team that does a thing and you're like they're like hey you catch this and they're like no uh yeah take that um and then let's take these you know what's this tool meant to do and compile it into sort of a list of use cases um this can be all the way from really specific things like i would like to match dns names that are longer than 18 characters and catch dns tunneling just to like i want x-fil blocked and then we should also ask what should this tool do what we like this tool do especially if we're looking for a replacement for a tool um if it's something like hey we have

this thing it's not working all that good let's find another thing you know kind of a wish list um it does happen and i've i've seen this before vendors are just like oh yeah that would be a good thing to block um so like i mentioned the 18 character dns thing it was a friend of mine was going against a dns something and the client specifically said hey could you could you do dns tunneling we want to see how this looks in our tools and i just happened to have some code to hand him and i handed it to him and that the blue guys came back and were like you didn't do dns tunneling and he's like

well i did and they're like well your domain names were too short so okay lesson learned for them maybe we we shortened the like length of domain names just because the vendor was like oh yeah i guess i guess you can't like tunnel out with three character sub domains um regulatory requirements as well especially if you work in like pci dss fintechy type things um finding credit card numbers is always like you know if something can identify credit card numbers and three cpv numbers and whatnot that's that's an excellent ask it's next to impossible but it's a good thing to think about um et cetera uh trade magazines i find come from above usually um

did this thing and i was like hey i found a power strip several jobs ago i found a power strip on you know let you know loose uh i could log in and like shut down a server rack and they're like cool story bro and then like six months later uh some trademark or some sizzle magazine or whatever had like oh my god you can get hacked by iot um just happen to have a report ready for that it's had a funny date on it but you know all right so we kind of have our like wish list high level wish list of use cases um in the terminology use case test case it's everybody's

going to use different terms that's cool um so let's then come up with kind of a list of test cases more technical like things we can put on the wire it's it's a good idea to keep these small like a pass fail or a like a severity of alert or something um it's also a good idea to keep it part of like notionally part of some larger scenario part of something that's uh either like a cool hack that happened um so when electronic arts lost something like source code and it came out it was a slack cookie that was owned uh team came to me and was like hey could you show us talk to us about slack

cookies neat little demo and some learnings happened and some fixes happened it was good um but like the the the test if you wanted to call it that was just like can i put this slack cookie in a browser oh yeah of course um i could anyways so you know make it make it part of a larger scenario that gives you a bit of impact uh gives you a bit of credibility and makes it easier for the people who are reading these like results to kind of figure out oh yeah this is this is actually useful um you can also take it just like from knowledge of how things could happen like i said the guy i knew was just like

hey sock how do we own you and they're like yeah sure so that sort of like chain of events and they knew like okay we should probably worry about this then there's like little bit like oh yeah this one little thing it does actually work and you know helpful tests um and then sometimes you know bad things are done uh and you know you can take your tests and go on that there was an organization um i heard of and it was had some crypto minor stuff and some you know comms out and uh the question was like hey can we test like just the comms that were similar to what happened and i could see that actually um until

we did and we just i mean it was http requests in a loop there's nothing big but it was a useful thing um and a thing and a question like hey just do our tools that are supposed to catch c2 actually catch this um those years ago i have no idea if they did or not also when you're putting together these test cases keep as few confounding factors around as possible um a because it's good science uh but b more importantly it's it's like 100 chance somebody's going to say something to the tune of well yeah it worked when you did it but you know you you had a an easier answer you know it worked when

you did it but like a real bad guy so the less of that the better um also when you have a stopwatch make sure it stays on cool so then then as you're coming up with these test cases it's good to figure out like where to start it's really tempting to be like i'm just going to do a whole hack from beginning to end um sales dudes love that if you ever gonna sell a product like having a scenario and scenario like uh is is a very valued thing that's not what we're doing here we're just testing like does this thing work so a better idea is just start start right before the thing that your testing

is supposed to come into play uh so if you're gonna say hey does this thing catch some execution okay just ssh in star dp in just something drop it to the desktop double click see if it works um it does work if you're going to look at something that blocks c2 you start with code execution ssh and drop your thing don't worry about whether that gets caught just see like hey does you know my http out to something weird get caught um it is it is highly likely that at some point when you're doing this you're like yeah we assumed access to whatever someone will come back and say yeah but if somebody has access to whatever we have

bigger problems um you've done two things there yes that's what the tools for and then you've also identified sort of a bigger problem in the organization as like the security mindset um so anyways once you have this list of test cases like single testable things make a list you know so you have a list of use cases test cases and then last things try and get caught like do it um which is uh very situationally dependent it's very like way outside the scope of the talk how to like test all the things but essentially it's hacking light it's all like the fun bits of hacking without like the setting up infrastructure and really boring stuff you'll need some

sort of hacker tools hacker tools i the vast majority of the hacker tools i use involve ssh and like cat and grip um but you'll need something to test it especially if you're on the network side of things a great place to find this is search results favorite search engine um some search engines are less great about getting things on the first page but the second page is some forget the fella's name i'll never remember but he says you know the one thing i'm worried about blue team one thing i'm worried about is like a monkey armed with github i think is what he said so yes google proof see if the uh not supposed to name things apologies

yeah see if it's google proof um for and postpoit host endpoint based things um the gtfo bins and the lol bins projects are great sources they'll just give you a list of like commands you can run to do bad things um by the way if you're interviewing somebody and you're like ah find the one command that does weird things and you let like open internet ability totally go to the website um more more you know to that point also roll your own like if there's nothing out there that tests the exact use case you want that's cool um it's it's always surprising to me like how like eight lines of python were discovered five years later

um or how like you know really like terribly written implants work um there's this one that was uh iranian malware i think it was dns tunneling and um like i i re-implemented it as best i could from reporting i took it to a client site was like hey let's test and see you know this kind of thing and it like broke immediately because like they were blocking the ip address scheme type thing that this tool used i'm like how did this actually work i i wrote the thing i think during a meeting because i was bored that day uh so yeah to take it it's totally there's the argument like hey is this realistic yeah

i could do whatever it works um as you're doing this think about impact you don't necessarily have to like drop something it works and then you know get domain admin but it it does make a better story when you can say hey this thing happened um it's good you know like hey this you know whatever we did made an alert so it's obviously bad or we hope it is anyways that's a start but some human visible-ish something is better if you can you know make the test such that somebody sees something add yourself to an ldap group um i know a fellow and that's like he just gets bored and and he ends up in ldap

groups and and then we end up with things to fix um less and less these days which is quite nice uh and the more the more real you make these tests the more real you execute the tests uh the more credible it is and by this i mostly mean where you're doing them so user endpoints i mean don't just like go up to somebody and say hey can i hack you please um but you know something configured reasonably similarly it's not great to say like oh yeah we just got domain admin but the box didn't have the normal set of defensive tools on it because you know there's the question like would that have blocked it um prod

network as well uh with care um if you don't have care i get this message one day and and i i got the message after i figured out how to reconnect to the network and uh the message was our ceo would really like you to stop breaking the wi-fi so test and fraud carefully on the blue side another benefit is you you know how things look in security tools nothing's looking defensive tools so you don't have to worry about like false positives i mean you're going to hopefully have them but you also have hopefully a much better idea than red side does how things are laid out and how to not break things anyway so you have all this you know

test cases and list updated listed with results uh so now let's take a few minutes and look at sort of a kind of real-ish case study and like i said in the beginning originally i was gonna you know think of something i've gone against tested and be like all right here just change the names to protect the innocent um that wouldn't be very kind so let's not do that instead the tool we're going to go against is kind of a kind of a mishmash of several other real-life things all of the all of what we're going to talk about has happened just not in one tool recap what we'll do in the case study um

we're going to work out what this tool is meant to do and also what we'd like it to do we'll figure out some test cases and then and we won't actually do it but notionally get caught also please note when you're doing this there's no need for this all to be this formal like it can totally just be like you know got some time in one day and write it i i don't know how many times i've sat in meetings that i wasn't really part of like i was there because i don't know they wanted somebody with long hair and a beard and i was like i'll just write this thing and then like a month later i'm

like let's use this thing and see what happens so every situation of course is different so here's our super duper effective tool which is really not one tool but it's a network-based bad guy catcher and maybe bad guy sometimes blocker um and and in the sales pitch again notional they were like this thing catches c2 this thing catches x-fil this thing catches lateral movement so on um and and what we've sort of figured out is it catches everything and it's meant to block obviously bad like high confidence malicious activity um totally not a contrived example that works for a 40-minute talk okay so step one let's figure out what what this tool is meant to do

so from the vendor's description we know it catches c2x fill lateral movement cool we need a little more to go on than that though so we know from previous alerts notional alerts that dns tunneling gets caught okay that's something we can go on we know from a friend that you know high volume ssh is bad and that a tool like this that catches x-fil can reasonably be assumed to catch high volume ssh true story actually a lot i know network admin went to him security guy and it's like hey why does our network traffic go like this and he said that doesn't happen our network traffic doesn't go like this and and it goes like this if you scp out the

entire company's code base as it turns out uh then it goes like this really fast so does somebody's paycheck um likewise the thing i alluded to earlier uh we know that you know again somebody else uh quite a bit ago was compromise cryptocurrency dropper persistent steel was a cron job and uh all it did was curl in a crown job that reached out to whatever so we'll assume that that's probably something that should get caught that's well within the scope of this tool um on the catch side on the block side from the vendor description we know it's high confidence malicious activity that's great uh what does that mean so let's poke into the documentation ssh

tunnels are meant to be blocked and there is a tool that does this um and it's like hey we will not allow ssh tunnels to be made your only point-to-point shell type things sure that's cool um and then we just like take a first page of search results like high confidence malicious activity or just known malware and see what's there so you'll get some pretty old stuff that hopefully is caught you'll get some maybe modern stuff that hopefully is caught there are a few tools out there that um without naming names that are really really common to see in the wild uh if you can convince somebody to spend a couple thousand bucks for a license

that would be an excellent test case for almost everything um and then we'll say from a previous job another true story we'll just comms to a malicious ip um we were using this network scanner tool and uh and the guy who was not actually on our team who was using the tool from our infrastructure for his own purposes like work purposes not that kind of own purposes uh scott called him and was like hey dude um what you got against russia he's supposed to be like 10. network right like internal he's like cool dudes i guess and uh turns out there was a bug in the tool but one of the defensive systems was like one of

the defensive tools was like hey yeah coms to known c2 address comes to like some ip that is known to harbor bad things nowadays everyone this was before everything went to cloud um it's a little less high fidelity but we'll just assume for the sake of a case study in 10 minutes that uh that's a thing and then sort of like stretch goals like hey what else what else can we get this to catch so let's ask the red team um there is a red teamer that at least one will say try icmp i know at least three tools that block it add that to the list um another person to ask is the people who deal with

regulators like hey if you know if we could use defensive tools to help with regulatory oddity things then it's a conversation i've had before what what would it be um and the answer i got being working for fintech is credit card numbers like yeah cool if if you know that happens sure so we have a list of use cases like i said make a list here's our list uh we will go against dns tunneling high volume ssh periodic http requests on the catching side we'll try and test ssh tunneling known malware google search engine results and some malicious ip on the blocking side and we'll just throw some icmp tool at it and maybe some fake

but you know past one check credit card numbers see what happens that then turns into a bunch of test cases which look something like this um do i have a little pointer thing no sometimes google gives you pointers and sometimes it doesn't it's my yeah the crystal's visible yeah so we'll just lump them sort of into three things that are our imaginary tools meant to do it's meant to catch c2 protocols we'll have a bunch of test cases meant to catch well known malware bunch of test cases meant to catch x fill bunch of test cases um i will not read these all out i promise i would like to point out a couple that

are sort of interesting um it doesn't have to be elegant necessarily uh this is actually a way to get dns on twitter um you can't actually get dns feel this way we're testing a network thing it doesn't really matter that you know the likelihood of finding something like that in your logs is low um what wrote something similar to this and like the guy next to me was like hugging his tummy and rocking back and forth and uh and at that point i realized pearl is an awesome language to write things in uh because i can't yeah i would have trouble reading that somebody looking at that by catching it the same thing gives me

another hour um and then similar on the the simple side here's that curl and loop i was talking about that's girl in a loop that's all that is so that works um and and it can be really simple or you can go complicated or so uh we were talking about you know first search first page of i literally typed in like dns tunneling entered and this there is a tool i changed the name um but there's a tool written in ruby that i've used before it's a pretty good tool um yeah throw on there see if it works that kind of thing um anything else of note no so also also worth mentioning it doesn't

have to be exactly faithful so this was scpus the high volume ssh thing um scp on the wire is going to look like ssh because it's tunnel over ssh right so that that is sort of that a few tips that i've gotten just learned over the years when you're going to do this use long format options in syntax so things like curl here um i probably should just use hyphen hyphen silent there much like down here form uh that way when you're reading it and more importantly when you hand this spreadsheet something somebody's gonna ask for the spreadsheet and the spreadsheet to somebody with like red boxes everywhere or whatever it'll be a lot more obvious what it is

um speaking of the spreadsheet make a copy uh it has happened in the past years ago that i had this nice spreadsheet color coded and all sorts of good notes and whatnot and and we were handing it to somebody i forget who and it it turns out that my spreadsheet changed and things that i was like hey this is important for whoever it was to know were deemed not as important uh by probably a sales guy uh so yeah copy your spreadsheet you might need it i i did not and then i had to go like reconstruct it um also make sure your tools are fairly easy to use there's without getting any details there's a

tool set that i wrote to do this sort of thing uh that is still in use at an organization that i have not been a part of for a few years now um but you know i i turned it into a just sort of like a a draggy double clicky thing that worked pretty good um keeping logs is also important pcap is good as a log uh and i've had many people say hey can you give me the p cap for that i am not entirely sure what somebody's going to do with 100 gig of ssh p cap but somebody will ask um and likewise just logs of like what you're doing uh screen recordings are also not a bad

thing um that when i was talking about the icmp thing the vendor was like hey can you send us a screen recording of you abusing icmp and i did and i hit up arrow and enter and it was like done it happened um and then last last but not least uh and it will happen come prepared with answers to the hand wavy like no that could never happen there are no droids here move along uh you'll find yourself in these meetings where like you'll start the meeting and you'll be like yeah so i tested i don't know just to pull one of these things out yeah we wrote a tool to do icmp tunneling we were you know had some

time we wrote it and uh you know it worked and then somebody would be like yeah but you know you can sit and write tools the bad guy class we're worried we're not really worried about the you know fsb or something we're worried about like you know our lower level hackers they're not gonna be able to write their tools only focus on like known malware um and then you'll sit in the meeting and and you'll be like yeah so we also did this thing where we um you know we use curl and loop like you know kind of the lower level guys you're talking about and and somebody will invariably tell you yeah but you know

we're looking at more sophisticated adversaries it's true story i mean these things happen all the time in meetings um and and you're sitting there thinking like all right either i'm a wizard or i'm a knucklehead i don't know which i am and then sometimes you can be mathematical and possibilities and i've been there too um i was i was working with a group of mathematicians who were using math uh that that is not sort of math i understand to catch bad guys and like take various data points and and turn that into a score basically and one of them one of them was convinced that i think it was like a one minute beacon interval or something was not a

malicious beacon interval i had a shell with a one minute beacon interval at that point but you know these things happen uh so yeah be prepared to be you know you'll have all sorts of um sort of resolution of cognitive dissonance things um they happen or you know to be mathematically impossible all right quick recap so um tools it's really hard to know if they actually work when you're using them before you just test them um and even if you test them you'll only know if they work for what you've tested but it's still quite a bit better than probably um i think blue teamers sort of the thesis of the talk uh the best people to test

are blue teamers you have the most knowledge um you have the most uh skin in the game really so you know you're the best and then when you do the testing itself figure out what the tool should do uh come up with some test cases and do it uh thank you for watching uh happy to take questions if there are any and again that will link you to the slides which makes you all the fun stuff none it's like pub is open now oh all right non-drinker there well thank you for the amazing talk was very interesting just you let me think in it is true that the red red team can help but you you know also

when you are testing that the tool because at the end that is the message you you gave us that you we need to test in order to check that the tool actually work we need to think like the attacker so so in a way we need to think like like the red red team is thinking you're just curious what what do you think i have a slide for you okay yeah so things i'm thinking with regards to one tool these are the things on my on my brain on my mind um you know mostly i'm thinking i have to full tier one or whatever i was automated whatever the first step that makes the decision because if

whatever the first step that makes that decision uh decisioning right here um whatever that is if that's like yeah it's cool then yeah that's cool so uh that's that's the first thing and then figure out what what i can do that won't get blocked maybe it blends in maybe there's just no capability against it and things i do that won't just not get blocked but won't get caught they're not i mean there's subtle differences there so that's kind of in one half of a slide sort of the mentality there yeah um but happy to talk to you afterwards much longer conversation if you'd like yeah good question though um any other questions all right cool thank you very much