BSidesCharm - 2019 - Mathieu Saulnier - BloodHound From Red to Blue

[Music]

okay hello and welcome to this talk called blow down from red to blue my name is meteor sony uh you can find me on twitter at scooby mtl uh just a little bit about me so i work in infosec since 2000 i've been a blue team member for the last seven years working in different soc i'm a security architect at bell canada which is one of canada's biggest isp and telephone company my roles are adversary detection team lead and tread hunting team lead i'm also a defcon blue team volunteer village since its creation last year at this point of the presentation you might wonder if i'm mentally challenged or if i just have an accent according to my

doctor it's because i'm french canadian that i speak like this um so french canadian or canadian in general were nice people until we start talking about hockey and if you're not from our team you better stfu a little bit of advice as well i do not pronounce h in front of words and i don't pronounce s's at the end of word a little bit of the agenda here so we're going to do a quick overview of blood down some basic usage of blowdown we're going to start customizing our blood down mainly using a cipher language then we're going to focus on destroying effect paths and finally we're going to do some reporting and automation so in the room here who has worked with

bloodhound before okay and who has used cipher before built their own cipher query okay perfect defender thinks in list attacker thinking graph as long as this is true affect her win very famous quote that you probably heard before from john lambert from microsoft but what does it actually mean or actually well bloodhound is one of those tools that can help you shift that and see your network as a graph so when we talk about list what do we mean what we mean these kinds of lists so the list of asset a list of server name a list of group some serial numbers all things are very useful then when you go inside this you have yet more lists about install

software open ports maybe compliancy check vulnerability list so on and so forth when we're talking about a graph what are we talking about well first of all graph is not only for security it's for anything um so well i'm going to give you an example in the real world so this is you and alexis and you're in a restaurant come along taylor and jordan now you become infatuated with jordan so you want to get you want to get something so you're going to leverage different relationship in order to get what you want so you're going to get your family relationship to alexis which is a co-worker of taylor which is a friend of jordan and with

that you will get jordan phone's number and by the icon i choose for phone number you get an idea of how old that can be now i hope you also like the fact that i'm using gender-neutral names and relationship make a big effort into this now a graph for an attacker or for defender as ourself would look a little bit like this so you have a user that has a relationship or actually the attacker will land somewhere usually impersonating a user through password spray phishing or any other means and then that user will have an admin right for example to his own machine a very bad infosec practice but it does append more than we want then from that

machine you might be able to rdp to a server such as a terminal server onto which there will be a lot of user having sessions uh depending on the size of your organization it can go from hundreds to thousands and among all of those user one has a little crown here so that user has a session and why does it have a crown well because it is member of a high value group such as domain admin so this is a little bit what a blowdown shows you when you use it so a little bit of overview so what is blood down this is the only slide with so much word but bloodhound use graph theory to

reveal the hidden and often unintended relationship within active directory environment so both attacker and defenders can use those paths to identify very complex attack path and defender can use them to actually eliminate them a little bit about the story of blowdown it was introduced at defcon 24 in a talk called six degree of domain admin here's the link if you haven't seen it and in that talk you can see rowan actually unlock his password manager and make the repo public it was actually demote a few days before that at b-side las vegas as well [Music] i became in touch with the bloodhound at blackout 2017 in a talk called the industrial revolution of lateral movement from tal mahara and talbury

where they explain how to automate all of this process of querying and exploiting the path very interesting talk if you haven't seen it i recommend it last year at black cat 2018 arsenal blowdown 2.0 was released and blood down is developed by waldo captain jesus and arm joy people i'm very sure you're familiar with so what does blood down do exactly well it does three little very simple thing first of all it queries your active directory then it imports the data into a neo4j backend and then with the gui it shows relationship between the objects [Music] why should we use blood now well for the red team you can build attack path offline that reduces a lot the noise on the

network for example when an attacker lands e-scan or he queries that active directory instead of port scanning all the subnets he has access to and then jumping on a machine stealing all their credentials jumping on another machine that then come back try another machine steal all the password all the ashes so on and so forth so we can know exactly which machine's target which account to target where he's going to go next and exactly where it's going to land for blue team we can use the same type of query to find the busiest attack pad so the paths that give our most of our user access to high privilege accounts and then we can destroy those paths

before they are exploited or we can actually even monitor them if we are advanced enough so we have we have all the chains in json so we can ingest that and then we can follow the trail so we can see if a user authenticate if user x authenticate on machine b and then steal this credential and go somewhere else we can follow the trail that's a little bit more advanced now we're going to talk about the basic or the first steps first of all we need ingestors so there's three main ingester for blood down there sharpon which is a c-sharp executable there's invoke blood ion which is a python script that load reflectively blow down

and then sharpen and then there is the blood on python that was developed by dirk chen from foxhill so you can leverage this if you land in the linux box for example you can still get all the goodness from blowdown then on the right side you see some collection some switches that you can use so collection method or dash c for short all is usually what we're going to use and you can add logged on because all does not include logged on because in windows server 2016 and windows 10 10th anniversary you need to be local admin in order to get that information then if you want to be a bit more stealthy you can use

dc only this is usually not something that we as defender are really we don't really care about that but i just put it here to explain it then a very interesting one is max loop time that goes with the collection at the session loop by default oneon will try to get sessions for two hours in our in our case or the case of the defender usually we will want to extend that time frame because as you probably know there's some admin tasks that are done only at night but if you run it two hours at night then you won't get your user data during the day so it's a good idea to run it to two 24

hours or more maybe two days over the weekend and shift when you collect your data then there's a search forest switch that's interesting if you have more than one forest so it will query all your different forests for all the other switches and information you can use sharpbound-h for help or you can go directly in the source code sometimes there are some options that are not documented in help or the switch change a little bit and the help file is not always up to date now the gui well this is a screenshot of the gui but there's nothing like a live demo so let's jump into the real ui so this is the screen that you get when you launch

the tool so you have domain admin on your right and all the members on the left okay that's great i'm not seeing my mouse that's going to be great

ew you feel a bit dumb now can i see it here okay cool okay so here you see the database information so you see that we have a roughly 5 000 users with roughly 5 000 computers uh here you have nut info so when you click on something you'll get all the properties and here you have the pre-built query most of the time the query that probably you're almost familiar with is find shortest path to domain admin so you click here you select your domain and off the little dog go while those is a nice thing we're going to look into the just continue our tour so here we have your custom query to edit

custom queries you can click on the little pane here it's going to bring up a text editor and you can add queries i'm going to share a link a bit later in the presentation where i give all of my cipher queries that are in this list [Music] now if we click on any of the edge like any user here as i said brings up the information about it what we can do also we can right click on one and then we can have we can go to edit node so we're going to see the information if it's on the domain and all the properties of the node and we can add any property that we want if we need to

another thing that we can do with the right click is mark as owned so if we know that this credential this object was owned by an attacker we can mark it as own and we can mark it as high value as well last thing we can set it as a starting node and you'll see that it appears here on top and then we can use a little road here and do exactly like in google map and type our destination in this case we want to go to domain admins and it's gonna create the path from that user to domain admin other um things that are interesting here is you can right click any edge and get help about

what it does so hair very bad render but anyway uh help on airjp so you have all the information how you can abuse some upset uh consideration and then some reference that you can go read further um you can use this as you saw it was a bit long to generate the shortest attack path graph so if you have a more like a very big database it's going to it can take hours to generate so you can export the graph either in png so an image or in json file that you can load after using this little button here and it's going to be way faster to generate your graph here you can upload data but you can

basically just drag and drop in the ui it works as well this change layout and here we have the settings so in the settings uh two things i want to point out first of all the query debug mode which works with the raw query bar down here so every if this is enabled every time you click somewhere it's going to update the query at the bottom and this is very helpful when you want to learn cipher and you want to modify your own query you start with that and then you modify as you want and then the most important feature i guess of blow down is the dark mode that was introduced in blood down 2.

every time i say dark mode in this presentation there's a fairy that is porn now a few oh yeah one last thing here we have some filters so if we're not interested in all of the uh links or edges we can remove one uh the ones we don't need and then we can rerun the query and then there's a few shortcuts key that are not documented that i want to share with you so for example we can click on control we can click on control and it's going to toggle on and off the labels on our graph so it might be useful to show or not show depending on who you're sharing your data with

then you can use shift ctrl and i to bring a console here where you can see errors if there's any and some useful information now okay there's a space bar also that brings spotlight and you can see any object in your graph and then you can click on it and you can use here in the search bar as you see you can see you can search for user but something that a bit less known you can search for gpo like this so you see all the gpo that the tool gathered and you can use ou like this and it's going to show you all the ou that it gathered last key that i want to share is a

comment r that's going to just reload and refresh the whole thing and brings you back to where you started back to the presentation so this is just a quick reminder of all the comments i said they were undocumented so i documented them so now they are documented at least somewhere now we're going to talk about the graph database that is behind this so the backend neo4j another freeware so you can download it there and you started like you would start any other services by invoking it with start you can stop or restart and then you can access the control on localhost force 7474. you might wonder why use the web console instead of the you the

gui to build your query well there's two good reason one it has a dark team fairy and it points you the error if you make any uh typos or error in your cipher query so here it gently point to me that the equal sign here is not good so when i change my equal for the semicolon it'll actually return me the information that i ask and here i just ask i want the group that i have the property high value set to true and then return me all the names so those are the four groups that are high value by default when you launch blood down [Music] now we're going to do some customization or learning to run

so some basic cipher all cipher query starts with a match and then you need to specify some objects so in blood down we have user computers groups and gpo and ou i think and then you have you can use a dot to access their properties in this case username then you have relationship which is a bracket with a little arrow pointing in the direction and the relation type inside then you have the path finding which is shortest path from one variable towards another one and in this case we want all the path with all the level that's why we start with one one up until unlimited up then you have wear where you can do some

filtering and then you return the value that you want to your console there's two ways of filtering there's the explicit way and there's the using the where clause so this is um explicit so you say i want a group where the name is domain user at testlab.local i want another group where the name is domain admin at testlab.local and then i want to have the shortest path between my object one and my object n and object m and then return the path when we do it with a where it looks a little bit like this so we declare our variable n group m group again and then our shortest path from n to m and then we will say where

name and name starts with domain user and and name contains domain adm it could be starts with ends with just to show you that some versus versatility and then again you return the path so here are the queries side by side when you run them you get the exact same result so why should you use one versus the other reason number one is the time it takes to gather so it's way faster to use explicit than it is to use where on the other end if you're a consultant or if you have multiple domain where is a bit more useful because you don't need to recreate your query for every customer or for every domain that you have

using the console we can also improve our query so when we do not optimal query it's going to show like this with the exclamation mark when you click on the exclamation mark it's going to tell you how you can improve your query and then you can change it so here basically what we're doing we're looking for a user with a shortest path from the user to high value group and we want to return the number of user that have a path [Music] so the two you see that the exclamation marks are deleted and this is the new query where you actually define your variable right inside of your shortest path function so this is the proper way of

doing it so yeah pro tip number one explain and profile so to get to help you um improve your query you can use explain you can append explain explain the explain or profile in front of your match query if you run explain it's going to execute plan but not run the statement whereas profile with that will actually run the statement and you'll see exactly which operator is doing most of the work so here's a query that show you again any object towards any object with high value because we're not specifying the type we need to say that n the starting node and the endnote must not be the same and then we return a name and the label

and we count them when we run explain it's going to give us this when we run profile it's going to give us this so it's pretty much the same thing but the numbers are not exactly the same and this will also give you the time here's a quick slide to show you how complex cipher query can become so here you have some optional match some collect uh you count things you extract you filter you unwind so lots of more advanced features those are queries that were shared in the blood ion slack and i believe they were both created by waldo originally here are some useful query that you can run so first of all about domain user so this is usually one

of the thing that i like to start with is to find every rights that domain user have on on the network so we're going to start with where domain user our local admin or where domain user uh the shortest path from domain user to high value targets where does the domain user can rdp to and we're going to remove those link this low hanging fruit then we can look at all the other bad rights that they have so all of those query are in the my github page that i'm going to share then you can look for kerberos sting or kerberostable account if you don't know what kerberos thing is it is a technique where as you request a

weak cipher in rc4 of a password and you crack it offline for a service principal name so for functional ids basically if you want more information about kerberosting i suggest you go see the excellent website adsecurity.org by sean mccall and then i also bring back the top 10 x that used to be in version one but they're removed in version two because i think it's a good way to start hunting in your database and give you some nice information about where to start here's the link to the repo and at the end i'll have the last slide we'll have a link to the whole presentation so you don't need to take pictures of everything it's all going to be there

if you want to know more about cypher here's a cheat sheet made by neo4j where you see the main keywords what they do and there's lots of documentation online now that we know how to build cipher queries we're going to start destroying paths

so we're going to start in a control environment and we're going to start by creating a problem ourselves so we're going to use the merge comment to create a link between domain user and a computer 673 and we're going to give and we're going to create an admin link an admin admin to edge from the two so we're going to merge all that when we run this we get one relationship was created now we're going to test that the new relation was actually created and so we're going to do a path where groups domain user is admin to computer 673 and we're going to return the path not surprisingly here is our relation now i'm going to show you two ways to

actually uh test your media your remediation first of all we're going to filter out relation so here we have a match on our domain user group towards our admin link towards our computer 673 and we're going to remove all relationships where the type of the relationship is admin 2 and then we're going to return the path this is method number one filtering out second method is deleting the relationship so same beginning we have our domain user admin to computer 673 and we're going to delete the relation we need an extra command to actually print so exact same beginning but instead of delete r we're going to print and not surprisingly when you run those commands you get

no data now that we have something that works we're going to test it against live data so here's again our basic query about finding all the domain user path to domain admin the shortest path so when we run this we get this this path sorry so now you might wonder when you should use filter out versus deleting if you have only one edge of a type it's it's it's a good place to use filter out if you have multiple you're better to delete it so here in this example we're going to tackle execute decom on the top here and as you can see in the path there is no other execute decom so it's a prime

target for filtering out so here's how it looks so we are same the beginning is the same and then we're going to filter out the relationship execute dcom and we're going to return the path if our mitigation was right we should not have any path to domain admin anymore houston we have a problem there's still a path but now instead of six ups it's nine up away so the mitigation that we want to apply or that we want to push to our sysadmin is not effective it's not going to fix our problem so this is one of the reason why you want to test before you ask people to do something if you ask them to do things and in the

end you have the same number of vulnerable or the same number of path they're going to lose faith in the product they're going to lose faith in the process and most importantly they will lose faith in you in your process and your team maybe pro tip number two now we have five groups that are high value by default in your organization you must probably have more than that so here's a query to find groups that have a name that contain admin and that don't have the i value property set and then you're going to return those name so here for example we have asia admins europe admins and north america admin so three more groups that are that the

user are admin so what we're going to do is we're going to use the same beginning but instead of returning the value we're going to set the value the attributes to true so this will change our five group from the beginning when we run the high value groups to eight groups now pro tip number three it's it's nice to have the group that are high value but what about the member inside of those group those are also high value targets so here is a query that will actually set all of the user inside those groups to true for those who have a good eye you might have noticed a little something different about this query

the relationship is reverse so it goes from right to left this is just to show you how flexible the query the cipher language is so what it looks like it looks like this so by default this is domain admin and the user so you see that there's no diamond for the user so that means that they are not high value when you run that query it's going to change to this so all the user inside of all of the groups that you have tagged as high value will become high value target when we do the shortest path it's going to start from looking like this from domain user to high value groups to this so you see that you have a lot

more pads that you need to take care of if you really want to remove problems from a domain user pro tip number four i know i know it's not christmas and i keep on giving so here is the the shell version of it and i'm gonna talk about it a bit more later but here i use it just to show you um the time so yeah so here yeah there's a small difference between the two and if you can spot it it's right here so in the first one we return a path and in the second one we return an attribute it's much faster to return an attribute than the whole path so if you don't need the actual path and

you just need some attributes use that instead it's gonna you're gonna save lots of time [Music] again another tip to to make your query more performant there's a small difference between those two query and it's right here so if you don't need to do anything with the relationship you do not need to assign a variable to it it's going to save some time so here the query basically maybe i should just explain you're looking for domain admins shortest path to anything basically if you want to learn more about cypher and how to use blood down defensively there's a good webinar from spectre ops called operationalizing blood down attack graph for defense here's the link you can also read sad

processor year of the blue dog post or is excellent whisperer and book then of course there's the blood down slack here's the link if you want to join especially the cipher queries there's lots of people there including myself that can help you with your queries now we're going to talk about reporting attacker thinks in graph management needs metrics so i want to thank sad processor for that one first query we're going to run is the percentage of user with a path to domain admin so we're going to start with our query where we're going to look at the user any user that has a path to a group called domain admin and we're gonna count also we're gonna

let's do a second match where we're gonna count all the user that we have in our database so distance user as user total and then we can account the user that has a path and we're going to assign that to a variable that you as a path so user as a path and then we're going to do some math directly in the return function where we're going to say user that has a path divided by the total of user multiplied by 100 give us the percentage so right now in our database we have 100 of user that have a path to da normal we created links earlier in order to do that now you can show in in table like this

month over month your progression so percentage of user with d8 can go from 100 57 12 and all your other queries all your other metrics can be there if this is not visual enough for your management you can use gauges and honestly if they don't understand these ones i cannot help you anymore those ones were built very easily with google docs for example now this is a query that i'm extremely proud of i worked very very hard and i actually found i made it work just last week so i just put it in uh what it does actually is it gives you domain admins with session on non-dc machines so your domain admins should only

log in into domain admins and not on any other computers otherwise they leave their ashes there and they can be stolen so the query is here says that you want a computer that is member of a group where the group is not domain controller and you're going to assign that to a variable called non-dc then you're going to use your non-dc machines and you're going to get all the sessions on that machine so all the users that have a session on that machine and all the of those user all the user that are in the group domain admin and then you can account you're going to print the username all the distinct username and you're gonna count

their connection just want to highlight here that again cipher pretty powerful language you can nest as many relationship as you want in your path when you run that query you're going to get something like this where you get the name of all of the the name of all of the admins or the domain admin account sorry that you have and the number of computer they are connected to that are not in domain controller if this information is not enough maybe you have some username that are hard to identify or to match with user you can use you can display more attributes than only one so here you can see that using brackets you can use you can refer to multiple

properties of the object now is a here's a little log graph of all the other queries um this is not probably the best way to display it you're bright people i'm sure you can come up with better ways of showing this data but at least i get some points i guess for the dart team now we're going to talk about some automation [Music] so we're back in the shell and new for j ship with a comment that actually let you talk directly to the database so we're going to do we go to the bin directory we export our username we export our password and then we can paste our query cipher query right there with

the cipher shell one important note is that you need to enclose your query in brackets so it's a good habit to always use double quotes in your queries and then you can use single quote when you're using that or the other way around doesn't matter you can use single quote when you build your query if you're more used with single quotes and then use double quotes in the shell but not try not to mix and match because you're going to have lots of problems so here the query uh basically we're looking for a kerberostable account in high value groups so that's why you have groups high value equal true and the property as spn equal true then

we're going to return the name as username and the group name as group as you can see in the return function here so in the shell is going to look like this so it's pretty readable but it's even more readable if you send that information into a csv now we're going to cap the csv output and we're going to get something like this so q8 is just because the query that i've run is my eighth query in my list but you can add maybe the month the date on your query so you can keep track now from this output in csv if you open in a spreadsheet it gives you something like this so that's very easy to share

with your management with your admins and everybody i mean who cannot open a spreadsheet right now we can also do some alerting so month over month we run the same query or week over week or whatever the time frame you do so you run the query you compare with the last result and then you alert if the number increase for example this month we have uh one kerberostable account in high value group next month we have three we want to investigate why we have three wanna maybe create a ticket for that maybe it's legit maybe not but then we can adjust and we can make sure it's already time for the conclusion so i think i went way faster than usual

um so using a tool like blowdown defender can think in graph too i think i've demonstrated that cypher is a very flexible language and it's important i want to reinface the fact that it's very important to test the real impact of a remediation before you contact your sys admins um or your uh your administrators so they they keep on believing in the process and in the tool and in the mechanics also another little takeaway is that not all query are worth automating with blood down i'm going to give you an example it's easy as you saw to see all the domain admins from your domain with blood down but you probably want to you will run blood down maybe once a

quarter once a month once a week but you probably want to be alerted in real time when there's a new domain admin in your group so you better use windows event id 4728 instead of using blood down for that now it's staying a time it's time for the thank you so i want to thank beside charm for giving me my first opportunity to talk here in the us i want to thank uh sean mcauliffe talberry and daniel bohannon for all the talks that they're doing the way they're sharing things the energy they put they really inspired me to submit the talk here today and a very special thanks to waldo and captain jesus for being very

supportive of the whole community giving free tools and just being really nice sports in general as promised here are the links for the all the slides if you have question i think we have plenty of time [Music]

any questions yes what's the learning curve on the cipher query language is it something that's easily picked up or is there online training okay so the question was how how's the learning curve for cypher query software query is a language that is used a lot so it's easy if you want to do something to google the function you want and there's generally a open stack discussion or some discussion some forums that will give you an answer it won't be related to security but usually you'll be able to adapt so if you're if you're familiar with database language in general i think it's similar i have zero programming experience like zero zero and i could pick it up and start

building interesting things with it any other questions thank you very much for your time