Velociraptor: Digging Deeper

thanks uh thanks tess for coming today and uh giving me the opportunity to uh talk to you about velociraptor which is our uh our new open source uh endpoint visibility tool um and i'm actually really really excited about it um and um because i get to call myself a digital paleontologist so um uh yeah i guess that's kind of cool um and um yeah so so what is velociraptor uh you might have heard about it i mean uh we've we've uh sort of been uh uh it's been around for a little while uh and um and you might have heard about uh about us so so here's me and uh and nick klein is my

business partner and we are the people behind velociraptor um so what is the need why do we need velociraptor so the thing about velociraptor is that it provides deep visibility of your endpoint and you you might already have an edr tool or a virus scanner uh on your endpoint and that gives you a lot of visibility and as things become more and more uh encrypted on the network we need to increase our visibility of the of the endpoint we could do threat hunting you need to be able to look for something that is you know unique uh something that's different and it needs to be very flexible at the same time you you don't really want to be pushing new

updates to end points uh because you know there's process management there's deployment management and you know and and so uh this is kind of uh it's kind of a trade-off on one hand you want to be really flexible to be able to essentially uh respond really quickly to new threads to collect new kinds of evidence really quickly uh but at the end on the other hand you can't really deploy new software uh to the endpoint so you can't just uh you know there are other open source uh endpoint tools uh and you might you know be able to write a patch or whatever to the uh to the 10.2 but then you have to push it out so

uh a goal is actually to develop a tool that is really flexible really powerful um and doesn't require a lot of um a lot of updates uh to be able to respond to rapidly changing um threats so let me just give you a quick technical overview and it's kind of a very basic uh very typical architecture on the left side we see all the endpoints that that we support you know uh windows uh linux macos uh and we we deploy a velociraptor server that can be on premise or in the cloud whatever um with uh gui front-end it's got some data storage and and file store um and then we have encrypted communications to the

endpoint so we can essentially query the endpoints for whatever we want you know instantly so there is a gui um and users the uh investigators so analysts you know can uh use the gui to essentially uh ask various questions of the endpoints and trying to uh collect uh artifacts about what's happening on the endpoints and and and that sort of thing so that's basically the the architecture it's a fairly typical architecture the only difference uh from uh maybe um other architectures like ger is that we don't really do any polling uh there is a persistent tcp connection that we can use to stream data constantly and so that means that if you've ever used ger

and and you had had to task a client then you had to wait for the next poll and it's like 10 minutes and it's kind of kind of annoying but so we don't have that we just have him so it's immediate right so very very quick um okay so the other thing that we thought of we wanted to make velociraptor really easy to use and easy to deploy so it's just a single binary it doesn't really need any dlls or any dependencies so you can just push it out and run it statically compiled without any libraries it's the same binary for clients and servers of just one binary um and it just runs being a server or a client depending on

command line options uh we don't really use a database at this point so that makes it really easy because we just collect files velociraptor is really just a collector um so it just it doesn't necessarily want to do any of the the you know analysis like the splunk or the the you know el elk stack or else tag that that you want to do it just collects those files and if you want to then go ahead and put that into another uh stack that's that's that's totally fine um and so that means that we basically just kind of produce files like csv files and uh and we just can grab files um so we don't really need a database so we

are all just uh file based right now um and you know you can customize the configuration and so forth but what is really the main uh difference between velociraptor and other tools is the implementation of a query language now it's not the first tool that uses a query language and you might have heard of os query which was uh you know the first tool that really just kind of hit on this idea that we could write a query um and and then query the endpoint for its state uh using a query language and oscar uses sql but velociraptor uses vql which is an extension it's an extension to sql that allows it to do much more powerful

things and so in reality the old velociraptor is is just a vql execution engine so that's how you talk to the endpoint is you just send queries and then the endpoint writes the queries and then you basically collect them right so a query is just something that that produces a table of results with rows and columns and so essentially uh we just send the queries um to the to the uh to the endpoint but nobody really wants to write queries because it's kind of complicated and long and you know tedious so we have this concept of artifacts um which i'll go into later which is just a way of kind of uh giving the query a name and taking parameters

and sort of writing it in a yaml file so it can be reused so that query can be reused and so we call that artifacts so artifacts are typically collecting a specific type of thing that we are after okay what we are actually really want to do with artifacts is to kind of encapsulate the knowledge that dfir person would have about what what they are looking for and what they are expecting to find inside a yaml file so that the user doesn't have to know uh all the details about like where should i look for these files where you know where is the internet history files stored you know where are the event logs stored that

kind of thing that that is all encapsulated inside the artifact file the user just says i want to just go and grab all the internet history files in there [Music] so so so when you write an artifact the artifact is really about answering a question so it's kind of a high level thing and and the actual mechanics of how to answer this question is the more lower level so we have different levels of artifacts uh very high level and and you know we might just say what programs were executed in order to answer that question we might need to look at a number of different evidence of execution places and that's where the forensic kind of

knowledge comes in and a forensic expert might say well you know you might look at shim cache you might look at prefetch you might look at executables on this modification times timelining that kind of thing and so but each of those will be an artifact so if i then said okay you know what's a shim cage oh well there's a sheep cage artifact how do i find it and then that artifact goes and finds it for you and so that mechanics of how that works is also encapsulated in artifact uh and but on the on the bottom level everything is really just running vql queries at the end it's just all queries and it all goes to the endpoint so you

can implement a new artifact yourself without having to redeploy a new client just by itself so here's an example of what an artifact looks like and i don't really expect you to read it but you can see these big blocks and so the top level we have a name and an artifact just has a name and a description so we can search on it in the in the gui it's a yaml sort of file and then it takes maybe a bunch of parameters and the parameters could be you know um anything that the artifact wants to take like in this case this particular artifact um searches for uh parents execution so like uh you know if this

process is called you know svc host then it has to have a parent it's called you know um so you can't have a parent called cmd.dxe right it has to have a parent that is called system and so on so you have so in this here is the parameter which is just a lookup list of you know the parent and what's the expected parent name kind of thing so we can just so if someone tries to install malware on machine and then just try and call it like system or svc host or lsas or something like this so that it doesn't look odd when you look at the task manager um then we could pick it up

because we will say oh actually that's not right because it wasn't launched by the correct parent so this is this is one of the attack techniques in the attack matrix but this is just an example so over here i'm not going to go into it in to too much detail but that's the vql that is being run right and then over here we have the parameters so if so if i wanted to add another another such relationship to check for then you know just add another another uh entry in this list and off we go right i don't really need to understand the query uh but you know you can um i'm not going to go into it in too much

detail but uh people always ask us what's the difference then between vql and os query or velociraptor knows query and on the left this is the same query from a project to map the attack framework for os query and so it's in sql but each of those uh each of those sections here is is its own query and you can see that this one just looks for services.exe so it's just one right so each line here oops is is like a paragraph of queries and if you look at that query it's actually really complicated it's got a drawing and it's actually really quite quite a complex query and you have to copy and paste it

for each one of those so i'm not saying that uh so that's fine right but it's just that the reason that we can uh work in a much more concise way is because vql is so much more powerful because we can pass parameters and we have plugins that pause csv files so we can give the parameters of csv files and so on so let's have a look at how how we can extend that idea so this particular artifact is looking for event logs so this is a triage artifact which is just collecting files and you might run it in order to say well give me all the event logs on this machine i want to investigate them

you know the forensic investigation and you know we can do the same thing we just basically give the possible locations for where the event logs might be in in a kind of uh um csv file so that the user can say oh there's another location you guys forgot about let's add that in it's very easy you don't really need to understand vql to add that in uh and off you go so so i hope i kind of picked your interest here with the power that having a very powerful query language gives you as a user i'm just going to do a bit of a segue here to show you how to set it up and

you might think oh you know it's kind of complicated to set up but i'll just show you exactly all the steps that are required to set up a velociraptor instance so the first thing you need to do is create some configuration files and you just ask velociraptor create configuration for you and this will generate keys and basically generate your initial configuration for you so it asks us a couple of questions here um we answer the questions and it basically builds a client config and a server config and that's really all there is to it i just yaml files the server config is you know for the server client config is for the client um it

contains all the key material that you generated for the authentication and cryptography and no it's not blockchain it's just normal um and um all right and and then to start the server we basically just say it's the same binary again we just give it another command we say you know just the front end start it up um and off we go we can log into the ui and same with the client so the client we have an msi that we can ship that you can deploy and then you just drop the client config file or you can just run it interactively like this and this is kind of what it looks like that's the the dashboard on the front

you know we can see a number of machines connected this is a very small deployment but you typically see this kind of wave you know when people go home and come come back and so the number of clients just typically grows over the day goes up goes down that sort of thing so i just wanted to talk about uh the first instance so let's say that you had velocirapted in the first instance deployed everywhere and then you wanted to investigate a particular machine so you would say well okay like let's let's uh the first thing i would want to do is go and navigate the file system of that machine and just look at the user's

files so we have a virtual file system view in the gui that you can just go and and essentially interactively browse the the endpoints right and so you can see over here we've got several headers like file ntfs registry and artifacts so these are the different things about the endpoint that we can pause we can do ntfs so raw and tfs access we can we can use that to bypass any file locking things like that so let's say that we we tried to copy the user's anti-user.dat hive which is where the user settings are so typically if a user gets compromised you know the malware might actually insert itself into the user data to the user's part of the registry not

the system part of the registry and so we'll need to go we might want to go and copy that out right so we want to go and navigate that and it's normally stored in c users slash username slash into user.ads but how do you know that i know that but you know you just need to know right um but then if you actually try to fetch it you'll see that you can't actually do that because that file is locked when the user's logged in so when the user is logged in windows locks the file so you can't just copy it out so now you actually need to you need to access the raw ntfs bypass file locking and

actually grab the file from the file system so you need to know that for that file typically you you would need to use the ntfs driver and you can see here that we're using uh row ntfs uh with c c column and you know uses nic and user. so this so this is how you would you know and you can see that you can fetch that file when you download it you see this little [Music] icon that tells you that you fetch that file and you can you can then grab it from the server um but you sort of need to know what you're looking for and exactly how to get it and this is this

is this kind of like example of specialized dfir forensic sort of knowledge to do that and what we try to do um yeah and this is how you you just basically can click download to to get the file so so this is great but you know it's very manual it takes a while and then you also need to know what you're doing and we wanted to just basically solve that for everyone right because we don't believe that people should need to uh you know we want to commoditize this kind of uh forensic knowledge right so what we so we created an artifact an artifact basically encapsulates that knowledge of where you would expect to find and

use it.that and how you would expect to to grab it to be able to download it so that you can see here the it already encodes the fact that we have to use the ntfs driver to uh wtfs access to to grab it and the location of where we expect it to be for each user and it automatically grabs the entity user that from all the users right not just from that one and so this is basically the next level of automation and this gives you an idea of how you can use vql to try to try and kind of encapsulate that knowledge right because now the old user has to say is oh yes i just want to grab

my ent user upload just just upload it into user.dad and it will just fetch it so they don't need to really think about you know where they're going to get it from that's what the vql looks like and then like imagine what happens next is that um you you find that a user was compromised on the domain and then uh the bad guys use that compromise account to uh literally move that often happens uh across the domain right and so you want to know which machine uh did this particular user you know look log into right and so you might want to be more specific about more surgical about getting the end users of that of just a compromised

account so what you would do is you would take that upload artifact and then customize it and just add a simple condition where the name matches so that's the regex match operator backup user account so that's enough now you've just taken the artifact and changed it to be more surgical more specific for your needs so you don't really need to you know understand vql at a deep level to be able to just tweak things a little bit and then just get uh the right the right artifact so so that's kind of level up so so we've seen that we could now encode this kind of idea we've got an artifact we've written the artifact we've changed it

uh to suit us and now what we actually want to do is is just run it everywhere we don't before we saw how we could just collect from one machine but actually we want to level it up we want to just go grab it from all the machines at once and in the velociraptor terminology we call this a hunt right so a hunt is basically where you automatically run the same artifact collection across the entire network um in this at the same time so you basically just go to the ui and you hunt select the artifacts um and then just run it so the hunt is basically something that's always running and if a new machine joins

uh and it hasn't run the hunt yet then it will pick it up and collect it so all the time you'll get more and more and more machines joining and so if you like run the hunt for or say a week then you get a pretty good coverage of most of the machines that have been online at least a little bit during that week yeah so you can just select different ones so you know you might in this case we might just grab the enter user evidence of execution uh or all the internet explorer cache for example like all at once all right so so we can do that uh now imagine that someone tells you oh you

know there is this new malware um and it's actually kind of a bad one and this is maybe a thread feed might say well you might find that the malware leaves its configuration file in a certain location uh maybe it's in the user's users uh slash star it might have a couple of dots and it might have some random letters and you know these kind of things so so you might say well i could look for that kind of file on all the machines and see maybe that would be the evidence that this malware is running and so this is where we might use the file finder file finder is kind of a more generic artifact that is kind of

very powerful and you can see that it has a number of different options here we can search for particular globes we can search for keywords we can use run dfs and we can limit it by you know recent times modification times and over here we have this ops per second setting that actually allows uh requests that the the endpoint uh lowers the rate it's a rate limiting kind of thing so it doesn't kind of hammer the endpoint with you know trying to search for this file it kind of goes low and slow right trying to find it so it minimizes impact on the endpoint um yeah so so with this this is kind of a swiss army knife of uh

of of file finding and you can you can actually achieve quite a lot but all of it is written in vql as well so you can just modify it as well but you can run that hunt everywhere and see which machines have been infected another another example of some of such a thing is a hunt for sysinternal tools so for example like you you guys all know that sysinternal tools are great you know you can use them and they're really super useful a lot of admins use them but also hackers love to use them as well because you know we have psx we have handles we have secure deletes you know we have a lot of

tools which are very useful for hackers so what you'd like to do is find evidence that these tools have been run on a machine and and then that helps you to hunt for unexpected tools like that so uh of course cis intel uh cis internal microsoft they already know that these tools are used by hackers so they have uh they have left a special kind of artifact for us to find uh called the eula so when you run the sysinternal tool you have to accept the euler and it leaves a registry entry for for that right a lot of the time the hackers forget to clean that up so you can basically search for that and

that tells you because of the time timestamp of the registry key then you know when it was created so we have an artifact to just find that so again the artifact is similar it's just a registry finding globe it just finds we know the location of the registry key um and we just get the timestamp and present it in this way and so we know like okay psx and s delete were run you know by the user nick on this date so if this was like a uh you know a um an admin person like not not a system admin but like an office admin or a accountant or something like that that would be very strange they probably

don't really have a reason to run peace exec but if it's a sys admin maybe it's not so useful um so we have the the sounds dfir uh poster has a lot of examples of basically these artifacts that you should you can look for to find evidence of so this is kind of the forensic artifacts and uh there's a big list of them you know user rc steam cage and so on and uh what we try and do is write artifacts uh velociraptor artifacts to try and basically automate all of these things right so we don't have to know where to look necessarily uh but but we we can then present them all in the same

the same uh artifact so for example in this more or less higher level artifact evidence of execution then we actually do all these things we get the user assist stuff we get the timeline we get the recent apps all these different ones uh automatically so all the user has to do is click that one and and get all the other ones this is an example what that might look like there's the user assist uh hive and we can see you know when it was executed uh and and the particular execution um the thing that was executed so you know if we're looking for a malware uh when was it executed where was it executed from this is the kind of evidence we can

look for the same thing for lateral movement for wmic involved methods all these kind of techniques that hackers love to use all right so we can also do the same thing for specific ioc so for instance if you get a threat intel to say oh you know there's a particular apt and they use a particular threat a particular technique and these are the artifacts that they might leave behind then same same thing we could write a velociraptor artifact to find those things and then do a hunt across the network and see if any of our machines are being owned so this is actually super powerful um because you can you can essentially respond a lot of the time it's like a

whack-a-mole if you find that one machine was compromised uh then then the question is okay you know i i know that they use this kind of tool or this kind of technique on that machine where else have they been and being able to hunt for it across all the machines at once means that basically you can you can find all the other lateral movement and how that group has moved through it um uh hunt for shadow id so this one is actually kind of uh interesting um uh you know in this case you know we might hunt for dropbox usage who's using dropbox in our um in our company but there's a bunch of different ways you

know that they might use uh dropbox over the web they might use the application uh they might have a chrome extension and so you can basically use all of these generic artifacts chrome history chrome extensions you know to basically look for specific evidence that people have actually used dropbox so you know if in history if you see dropbox website being accessed or if they have cookies that means that they've logged into dropbox at some point or if they actually have the dropbox application installed you can tell also when it was installed uh so this could be useful for even just you know just checking up to see that people are complying with policy and that kind of thing

all right so so we uh so far we were able to hunt everywhere and find evidence of bad stuff right but it's very reactive you know like we we suspect that something bad has happened and we have to go back and uh and and try and find that evidence um but what about if we change that and turn it into a way to be alerted when something bad happens so we turn it into monitoring instead of just going to look for the badness let's let's have velociraptor tell us when something bad happens and this is the idea of turning hunting into monetary and uh before we can talk about monitoring i just wanted to uh talk a little bit about event

queries and even event artifacts and so you before i showed you how a vql query uh basically returns uh a rose and and columns and you might think of it as just sort of like a database query you might say oh you know it's just like querying a database and if you ever create a database you write your query and the table comes back and and there you go right and in in no uh in many ways that's the case but velociraptor queries are are asynchronous it's not really a database it's you're basically querying the endpoint and you're running uh code on the endpoint so uh the query doesn't actually have to terminate so imagine if

you were running a query that never terminates like so imagine like it's a really slow database right so instead of but instead of just waiting there forever like watching the thing spin and not giving you results imagine that it can just give you results like one row at a time kind of thing right this is kind of what vql is like it's asynchronous so so if you have a query that never terminates and let's say the query is just sitting there watching for particular events to occur then as those events occur the the query will return one row at the time stream into the server and so that basically becomes a monitoring framework because the query

itself can just inform you whenever an interesting event happens and then you can process that event in the rest of the query so so this is really what event artifacts are are all about and uh so they are actually streamed from the endpoint uh and we have a a buffer file on disk so so they're actually streamed even when the machine's offline uh they're actually uh collected locally and then and then pushed out uh to the server um and and then we can actually operate on those and the query kind of runs on its own right just sending us rows uh as it goes along so so this so this is called an event query and what can we

do with an event query it's kind of interesting let's let's have a look so an example is to monitor dns on the endpoints dns lookups are actually a really good indicator for for malware right because if you have information like threat information about bad cnc domains and you can go back through your dns logs and see whether any of your machines have hit those bad dns cnc's you can you can use dns logs to to look for patterns for repeating queries for periodic queries it's quite useful you can log dns on network hubs but then you won't see any of your endpoints that are not on the network if people are working from home or from starbucks you won't see

that if you're only logging dns on the on the network so you're better off to log dns on the endpoint and then have the endpoint push those events to the server so you get better visibility right so velociraptor can monitor the dns and you know and you can essentially see that you know you can group it by a number of requests and hopefully if it's a cnc maybe you'll see a lot of a lot of requests or you can look for you know whether the domains are you know like random and so on the domain names are random and so on but but the interesting thing is that the dns query is an event query because essentially

it's like running a query that never terminates and every time the machine makes a dns lookup there's another row being returned and streamed to the server so it just keeps going like that right so this is a type of essentially event event monitoring but let's level that up and do something a little bit more complicated uh let's look for monitoring usb devices so a usb device is actually kind of an interesting thread from uh many in many ways people can plug them in and they can introduce malware but also you know people can copy files from your you know file server put them on your usb drive and go home right and then exfiltrate it that way

and so they can still um files can steal documents and so on and you know it might be against policy but you need a way of knowing what's actually being copied to the to the thumb drive you know did they just plug their mp3 collection in and listen to it or did they copy stuff to it you know from the business documents so you know just just as an example we can write an um an artifact that does that that looks for thumb drives and essentially what it does is it can look for the event of inserting a thumb drive and then in this particular one it just lists the directory right so it just gives you like a file listing of

that thumb drive when it was inserted and if any new files were added to it so like users copied files to it um it shows you those new files is you know just a listing of them so you can use that to essentially just kind of keep across all usb activity just logging it right so so and that query runs all the time on the end point even the machine is offline he will just queue up the results and then flush them to the server later but basically you can see that this one query now allows us to to do something that's kind of interesting right like it's we just thought of it and wrote it in vql right and

that query can can implement something useful um now imagine that in this particular case we think oh well you know we don't really want to just see the listing uh we're more interested to see whether that stealing documents our office documents from like a confidential document maybe they're copying that to the thumb drive so what what that requires is we might want to see when a new file appears on that if it's an office file i want to i want to do a keyword scan of that and see whether it has the word confidential or secret or whatever your classification marking is uh and so in order to do that of course you need to have a special kind of scanner

because most you know office documents are compressed and you have to actually scan the the content not just the zip file um but anyway uh if you download velociraptor and have a look that's all implemented in vql that's scanning right so in this case uh but the interesting thing is that this artifact actually uses the previous artifact so the one the one from before it simply lists all the the documents and this one scans it and this is a similar one uh just grabs the office macros i just want i'm just worried about having macros on my thumb drive and so it grabs the office macros and uploads them to the server and you can see how it uses

the other artifact to list from the list right and then it restricts it by office extensions and then it grabs the ole vba files from that so you can see that artifacts can sort of build on top of one another and even event artifacts can build on one another and actually use use each other okay so now we want to level up again so we can we can monitor stuff we can get more information what about doing something about it what about responding well it's actually not very much different than monitoring we can turn monitoring into responding because the uh the event artifacts on the endpoint are running all the time then then you can say

just as easily as i could have said uh to collect the vba file i could have said well if you find a vba file kill word or shut down or send me an email some kind of response act take action right but the interesting thing is that essentially uh that happens autonomously so the endpoint already knows what it's doing instead of going to the server and then the server saying oh this is bad do this the the endpoint already knows what the query is supposed to be doing so it actually just so changes the the pla changes the game a little bit because um we now have a response plan we have a plan the endpoint knows what it's

supposed to be doing and it's autonomous it just does it by itself uh an example blockpiece exec right so ps exec is uh you know in some environments maybe it's forbidden maybe it's not allowed so this particular artifact detects ps exec by uh looking for a service installation from the wmi and then doing a yara scan of the service so to check if it's the psx service and then we use that artifact inside of this artifact to to do it to kill it right to tusk killing so you can build on that so detection the first artifact was about detection and then the second artifact is about taking action so response um and and then there we go there is

actually a bit of a bit of a race condition here because you you detect it then you you kill it so you might not kill it fast enough right but all right so so that's an example of uh uh events uh running on the client on the endpoint uh what about the server we can also run event queries on the server and when we run them on the server it's uh it's the same thing as running on the client but we can enrich the information on the server so here's an example where we want to monitor for powershell encoded powershell and as you know powershell has the option of giving it an encoded command and it's actually

really hard to see that i mean you see the encoder but it's hard to see past it when you look at process execution logs uh you know you you really need to sort of decode that so what we actually want to have is some system to automatically decode that as it happens so in this case what we want to do is receive all the process execution logs as we normally do but then if if we see a process execution log that is a powershell with encoded command then we want to just decode it and here here's what we would see from the just the normal process creation the process execution log that velociraptor does you know out of the

box that's what's sent from the client from the end point uh but if you see that uh you know that's not very helpful right because you don't know if it's good or bad or anything so we can write a short artifact uh on the server and you can see what it does is it watches that other artifact so it watches for all process uh execution logs uh and then when the command line matches a certain the you know command then it goes and decodes it and so you know we might have and it actually creates another artifact another sort of table with just the decoded scripts right so you can see when something is bad

straight away here right so this this this is a typical like download cradle for metasploit or whatever okay so that's cool now the last topic i wanted to to mention is velociraptor's automation possibilities because it is a very open platform the whole point is to be very open and so we actually have an api that you can use to write python scripts and so on with python or any other language to communicate with it so i just wanted to give an example of how we would level that up through the api how would we use velociraptor inside our own script so this is a scenario what we want to do is look for certain binaries that are sensitive for example

a service creation event and we want to take those binaries if someone created the service and we want to upload them to a sandbox like joe sandbox or aristotle or something like that so how do we actually implement something like that in velociraptor well here's the plan this is what we want to what we're planning on doing so the client is going to basically sitting here and then when a new service is installed we're going to have an event log an event log message written it's a 7045 event log the client is going to watch that event log for this message when the message occurs then the client or the velociraptor client knows that a new

service was installed and then it can extract the binary and then it will upload the binary to the server right then on the server we're going to have a python script that we're going to watch for those binaries being uploaded and then in python upload it to to the sandbox so that's kind of the plan right so this is what we're going to use this artifact on the client windows detection service upload which essentially looks for a service creation event and just uploads the file then we're going to write some pythons code here and i'll just go through that that's actually that's pretty much all the code there is it's not very big this is the function to upload to the

sandbox and you could do anything else with with that file you know once it's been uploaded you can archive it take the hash whatever this particular i think is like five six lines of code that's all you need to connect to the api with python and then you can see that what we're doing is we are actually all the api is it's just issuing a vql query so it's just not connecting to the database and issuing a query so we can issue this query and i'll just go back one step this query says watch for any new events system upload completion when an upload is completed watch for that event and then grab it grab the file

and then for each reported one we're going to print out these kind of uh information and also call the submit file function up here and then submit it so so that's really all there is you can now set up essentially watchers that you know you you can set up when as if event happens uh you can push something to splunk or we can push something to elastic or we can do whatever so that's how we integrate velociraptor into other tools you can also use that to issue other artifact collections so you can say you know if something happens then collect those other artifacts as well so this is where we basically interface with uh external systems so this is how

it looks like in practice the bad guy says sc create fake and then the binary right and that immediately triggers the action and it uploads that binary uh and in our python script we grab the binary we submit it to joe sandbox and then a little while later sandbox tells us malicious yeah so that's that's our feedback there so this is just an example of of how you would use the api to integrate into more of a of a complete pipeline but i think the the most interesting thing for me is this uh this possibility uh for people to just kind of implement uh their own ideas their own approaches uh by either implementing their own um

artifacts or uh or or implementing their own uh scripts around it using the api just being able to use it like a swiss army knife in different contexts and different uh different ideas so where to from here so it is first of all i have to say again that that velociraptor is open source anyone can use it uh and it's on github we we have a development roadmap you know including this is some of those uh things with uh you know this is kind of what we thought about but again we really appreciate feedback from users to tell us what's important you know and what features people want uh and we appreciate you know prs as well uh pull

requests uh there may be bugs um it's still kind of work in progress um but it's really exciting because i think what we're trying to do here is build a community and try and build a whole bunch of community sheds artifacts and knowledge you know try and liberate that knowledge from people people uh that have a specific forensic knowledge and have that being accessible to everyone uh through different artifacts so yeah so please try it um download the latest release read the manual and and send us an email and contribute back so thank you thanks

any questions like

yeah so uh you mentioned that communications between server and clients are encrypted yeah can you elaborate more on the authentication model between clients on the server because obviously it seems like uh considering that they bought the same binary and so forth the client itself can be used as a nebula or effector so it's targeted for the attackers so so the same binary in the sense that it's the same code but of course when a client runs as a client as a service it's being a client right it's not a it's not a peer-to-peer network if that's what you mean so the communication protocol is encrypted over tls um and then we have uh we it's basically using the same um

encryption or authentication uh protocol as ger um with uh it's elaborated in a lot of detail in the in the google documentation but it's essentially a a key based pki based system right uh to to authenticate clients to servers um and then we have tls on top of that again so thank you

thanks for the awesome presentation thank you and i have a question for the youth experience is that possible to create some markers and instead it's instead of the writing required so the macro will expand it to the query when you say macro what do you mean by macro like for the query for example if i only want to provide the the hard the hash of file and make a query of uh whether this file uh existing actually okay uh local store servers and then instead of writing the whole query statement by just using a macro and to abstract a lot of uh frequently used activities yeah yeah that's a good idea so let me just

uh let me just sorry actually i think you can just jump to the right thing so uh that's so i guess this is what you are getting at here so this parameter list right um for example if you in your case that you mentioned like a hash list and your query might say does it match any of these hashes right but you don't want to hard code your hashes in the artifact because then you'd have to change your artifact all the time you know when the hashes change right what you really want to do is to put your hash list maybe on the on the web somewhere and then say go fetch that and then

compare it against you know today's hash list right that's where i'm what you're getting to right so that parameter what you do is if you the same technique right you put your hash list which might be a csv file or whatever right but in this case you see how it's inline in as a per is given as a parameter to the artifact but you can actually host that let's say on a web page somewhere else and then we can uh you can write your vql artifact to say fetch that and pause it and it's actually the same thing right if you look at the you see what we have here uh it pauses the csv file and that's the

lookup that the lookup table right that it uses so you can basically instead of pausing it from a parameter you can pause it from a web page that you get so and then when the artifact runs it will fetch the csv file pause it and then and and then you you can then just go ahead and manage that separately that's what you're getting at right like yeah so you can make it really complex and in fact we in the in the in this car release which was done yesterday there is uh there is ability to incorporate third-party binaries or files uh that are versioned and it's exactly the same thing like like you described there's an

inventory file with hashes and then and then it just fetches like the latest one um and and so you can push new new versions of hashes or whatever it is that that you want yeah you just mentioned the latest one it's impossible to make query based on the time range like only once big query and for the last three days for example so the query so the vql query doesn't create a database there's no database right it actually runs uh so this is the query right select star from and this is a plugin which is a bit of code right and then if you wanted to limit that by time you can go where you

know time is greater than this or whatever right but yeah so you can you can do but it's it's not uh it doesn't really have the data it's not like a database where you can query over a range except for the server where you're collecting the data so yeah you can do it on the server but not on the end point is that what you mean thanks any other questions i'd just like to eastbound on what he asked so for file to be able to search them based on the you know file metadata you will be able to do that yeah yeah absolutely so you can uh so we actually have a full and ntfs parser so you can

do things like search for uh the different uh attributes and we're gonna add more and more of the ntfs type forensic analysis so you can do detect things like time stomping and things like that but as it is now you can look for file metadata like timestamps and uh you know size and these kind of things as well as you can run yara sigs on the content of the file um you know and so and then and not only can you do that but you can also do you know process memory and you know that kind of thing um and another question sorry so uh is this um sorry i missed the first five minutes of the presentation is this

ex actually running on the host itself yeah so it's an agent so the the agent here is running the client we call it a client it's running on the host so it can be installed as an agent so as a service or you we also actually support an agent list mode where you can just push it with group policy and it just runs and then you can just shut it down after a day or two which is useful for doing a number of hunts in an instant response situation so in a for like traditional forensics perspective you need to maintain the timestamp of all files so when you're extracting about how how are you maintaining the original test stamp of

the files so you think that you don't want to change the access time when when you access it right so if if you do if you do that so there's a number of ways of accessing the files obviously if you access them through the os apis they might change access times but we also have a raw ntfs browser that allows us to essentially do ntfs file system analysis on the endpoint directly which doesn't really go through the the normal file system so it doesn't touch it's a read-only you can't change any of the timestamps but you know changing the access times i mean that's that's uh that that's a good point but you you have to be careful uh sometimes

but usually it's not that big a deal on what we're trying to do here is sort of like triage an incident it's not so yeah you can think about it but it's not so critical in many cases to not change the system so much right yeah i guess this is uh it's yeah it's for both because i think the problem is that if you go around taking forensic images of everything then it's just gonna take a really long time and if you have a big case with hundreds of machines then you actually can't really do that in in this way right so depending on the case you know it may be uh acceptable to to say yes it's been modified a little

bit we know how by us interacting with the live system

there's a possibility of changing the time stamp on these disciple files yeah i mean i'm not a lawyer so i can't talk about you know code admissibility but i mean i think that as long as you you know what's happening it should be fine okay thanks so much

any questions more questions oh there's one at the back there

uh you mentioned that there was a slight delay between detect and respond can you elaborate a little bit yeah yeah so basically what's happening is we are not in line like we're not a kernel module that you know that we have to consult before uh the process is allowed to continue right we are simply monitoring it so what happens is uh if you look at um i'll just shift back to this particular case that uh that it worked the ps exec one yeah this one right so what's happening is that we are watching for the wmi events you see there's a within one which means the wmi is going to essentially check every second so we have at least one second delay

since the service was installed and we get nullified and then we actually go through and uh and yara scan it so we can decide like you know there might be a second or two there and so if so if you run the so you see how in this command i ran um ps exec with uh sleep so it will work but if i run like a dir it's too fast right because it just starts and stops we don't actually get told about it quickly enough because we are not in line here in this case so it's not like it's starting the service then waiting for us to tell if it's okay you know like we're not really

um like a yeah we're only nine so there's a race condition here right um so so that this is kind of the the down part of not being in line is you can never actually stop something completely you know but yeah this is an example of of a response type activity any other questions oh here's my name

first off great tool i love it whenever we see automation that makes blue team's lives easier how are you planning or have you had considerations around protecting the agent or having a server artifact that detects when it gets killed um because one of the first things you do as an offense is enumerate yeah cool i need to work yeah yeah so so yeah that's that's the never-ending ending problem um so at the moment it's installed as a system level agent so uh service so if you you know you need to be system to stop it uh or an admin to stop it um yeah we don't really have any kernel module or anything like that that we

could you know it's it's kind of an arms race right uh it is on the roadmap to to do that sort of thing um at the moment your best bet is to rename the binary i mean sometimes we do that it's kind of lame but [Music] because it's open source you can sort of rebuild it but you know it's not it's not great um yeah if you have any ideas of how we should do that i would love to know