BSides Iowa 2018: "Threat Hunting Windows Event Logs w/ Powershell"

thanks everybody for waiting through my technical difficulties trying to get a display up and working today I'm going to be talking about threat hunting with event logs in PowerShell we're pretty much not what I'm looking at doing with this talk is looking at expanding into newer solutions if you don't have like a seam in place that you're already using or we're looking to how to build upon our existing seams and put more tools into place to be able to kind of tweak those rule sets that we're using and give better logic and better things out of it and then the last thing is kind of looking at ways that we can maybe move beyond our critical

infrastructure because a lot of our monitoring and things like that we're only focusing on the critical infrastructure we're not really focusing on the entire big picture so before we go down this hole a little bit about me my name is Justin Williams I am an information security professional prior to that I've worked in numerous different areas in IT have been in helpdesk database administration primarily development I'm an absolute gigantic fanboy of Microsoft products in PowerShell and as a whole and then currently I just I'm actually three weeks into a new managers gig and an information security role so the other thing I want to touch on is I am part of the Omaha wasp chapter leadership so if

any of you want to make a trip over to Omaha and talk about app sec with our group just reach out to me all right so kind of some of the goals for this talk is first off and foremost I wanted to talk about where do we look for events in Windows and how do we get at that information what files and things like that in logs might be useful to find information where can you look to extract data from those logs kind of do a little bit of an introduction into sis mine not really a deep dive but more so working with the output that it gives us after install it and we're running it and then

the last thing is is I want to give everybody an introduction into PowerShell to put everything together because as I found when administration and those rules just adding the tiniest little bit of scripting like that into your arsenal kind of makes everything more powerful and makes you a lot more useful and being able to determine what's going on in your environment so first and foremost we're going to look at what our typical seem scenario is most of the time we're gonna be logging servers work and that that's pretty much it so nothing else workstations are going to be what's spending most of your time actually accessing unprotected areas of the internet not many people are going to be surfing the internet on

their servers not many people are going to be downloading malicious tools onto their servers but your end-users are going to so at most with the workstations we might be looking at logs and event logs and things like that but what we're really getting is is we're getting alerted by our seems that our AVS are catching things right so we're not getting anything useful out of that information other than the system that we're running is working and then lastly like just touched on before with the critical infrastructure you could in some instances we're only looking at our PCI compliance areas right we're looking at that's where the money is that's where our biggest fines are so that's

where the organization wants to invest that's where they want to protect it so when you start looking at your seams and the licensing costs and things like that that are getting out of control you're looking at putting your all of your costs into that and then kind of ignoring everything else unless it starts to hit that environment and so with that this kind of sum it adds up to just pretty much a gigantic failure for everybody if you try to catch everything from the start you end up with too much data and you're not getting anything any value out of that if you're only looking at PCI environments and things like that you're not getting enough data to actually get

a big picture of your environment and the last thing is is where this kind of comes in all together is is the data that we're getting into if we're getting too much or too little how actionable is that data we said we're getting notified that our next-gen antivirus is taught is stopping something but we have users on our network that our front end users they are admin users that are explicitly calling wmic or PS exec and moving laterally across the network when they really shouldn't be using those and we're not seeing things like that necessarily through those products so kind of starting out with this want to look at the windows event logs and some

of the some of the ways to look at what would be built-in functionality that would exist without having to install any additional components to be able to maybe find some information first and foremost is the system the system event log that's typically going to give you things like service information service installation information what status it is and you might get some windows update information out of that as well the next one is the application log we'll see things like install events and then you'll see some of your security events and window and antivirus events will actually log into the application lock system log and then the last one is the security log is pretty much the next

where next place to look at for all your all your information it's where pretty much the bread-and-butter for everything on your own that system is going to be that's going to give you things like your logged on events who's logging into in and out of the system it'll give you auditing process creation process termination things like that are all going to be contained within within the security log so kind of when I first started out in IT I started looking at my window or at my event logs and everybody probably recognizes Event Viewer right so the thing about this is it's hard to get actionable data out of Event Viewer when it's pretty much going through each

thing step by step and then each message things like that are gonna be kind of kind of different within the environment between depending upon which log you're looking at so it's hard to get good things it's scale to do your collaborate your event correlation and things like that so where that's changed with this

is you can start getting at that information using PowerShell a lot easier if you look at there's one command there's actually two commandlets one we'll talk about right now get event log is going to be problem for looking at your basic system information you can use this to get to get at that stuff directly through PowerShell the you can use different filters you can you can query things by type using it and then you can look by source log and then I'll talk about it a little bit later more in-depth but you can take the output of it with the thing that's nice about PowerShell is the output of everything is a custom object or it's an object and

then you can format that to create a custom object so when you start wanting to analyze and move the data around you can take the message if you like with the event messages you can take if you know the specific ID you can enumerate through that to create an object that you can put into someplace that you can then have better analytics against the only thing with that is is when we talk about the original system logs that kind of going over now the discrepancies between the formatting and between the types of messages do exist so

so what you'll see is the message itself is this right there's gonna be right here each new line and the typical message is terminated just with the with a new line terminator but when you look at the message between the application log between the system log between the security log each one's going donessa not necessarily going to be formatted exactly the same so you can't just take a generic and build the object out that way so lame

the best example is gonna be when you start getting into your security events and as you can see this is the message here it's gonna be mine itself is going to be a new event so when you start getting into some of the formatting things like that that's where it starts to become a little bit more difficult to work with one of the things that I've found is if you take that output and you do a split on the new line and build an array out of it and then look through that knowing where the end from which part of the array the information is that you want to get at you'll start seeing you'll you can start querying the

data a little bit eat a little bit more easy and that's where doing the things like building the custom objects and things like that will really start to help you

so going back to a slideshow

and then the last thing and the other reason why I like working with PowerShell becomes a lot better than just doing things like within the Event Viewer is because of the commute the computer name switch you start being able to see your correlation that way if I want to run a command on once if I see something like PS exec being ran on one system if anybody's familiar with a Japanese cert they did a manual about lateral movement techniques and you can and get event correlation so you can see things like if PS exec is ran on one system what's it going to look like what's it going to do on the other host so you can start you see it on one

system you can use the computer name switch to then crawl your network look for all systems on your domain and see where it was executed and on the other system so you can do that with the with we said this Japanese cert manual or minor attacks framework you can use known indicators through that as well to be able to detect lateral movement that you may be missing through your seam and other tools that you have so the last thing within the windows logs is the windows event forwarding that if you have the ability to it's actually a built-in centralized logging solution and windows that not a lot of people are using just a campaign put together an

article in 2015 if you search for her and monitoring what matters it'll give you you'll be able to find that article on the TechNet site and in summary it talks about even if you are using a scene that this information could be bent this could be beneficial to your environments because of the systems that you're not catching it gives you a centralized area to look at so instead of having to see that it's see something in the event and then crawl your network looking for that data that's all in a centralized location so you see the event fire and in the log you can also look for the corresponding event for and the different system that it ran on at that

point and then the other benefit to the windows event forwarding is is that when you're first setting it up when we talk about that information overload you can select what you want to be able to put into the law so if you want to only focus on your user tracking events you can do that if you only want to look at the process events and look at what's being what processes are running on your networks you can do that as well and it's a great way for you to be able to look at everything without being overloaded on the amount of data that's coming through and trying to make sense of all of that at the exact same time so

with that that kind of leads me into a little bit more talking about sis Mon and adding that into this type of framework it's been around for quite a while it's part of the SIS internal suite of apps and when you start getting into vlogging specific types of events it's incredibly useful for that there's quite a bit of information out there in and around it and in my experience working with it the general purpose of like the events are all standardized into the same type of a message output so we don't start running into the same type of scenario that I was talking about with the system and event logs where every message is different you can

start parsing those messages as they come in and they're in the same format so once you kind of have your object output to work with it's the same format pretty much for all the events that are firing there might be each event is standardized by it but the way that the type is you can build objects on the fly and then some of that may seem like duplication with the system and security logs and while it might be there's additional it more advanced information that you can get out of it that you might not necessarily catch things like process hooking and then you can even take it one step further by putting out so there

some great articles in and around filtering it Swiss alone security puts out the decent security on their site for decent security comm they put out a XML file that you can load in for the configuration that will kind of filter out some of the duplicate things Mathieu Graber puts out a lot or he did a great write-up on the spectra office blog talking about how he uses or how he actually builds custom configurations within system on as well and then last one would be Carlos Perez is constantly writing new and updated articles in and around it as well so when you install it you're just going to download it from this internal site on Microsoft site and

then you'll just run it with a - I switch and that will install it runs as a service on the system that you're running it on and you have to be an administrator to be able to install it or or maaan it modify it on the system I'm starting out and then like so you can from the configuration standpoint you can centralized or decentralized the configuration files and that's kind of a choose-your-own-adventure on how you want to do that you can there's pluses and minuses to both either way like one I've seen it pushed out there doop through group policy to each individual system and then I've seen it where people talk about just having a file share that all

systems access with the config file loaded within that

so

doing the same thing with PowerShell running through you can still get to the same law the same way that you would have you would interact with the system application logs in that you'd still you're still able to get at the syslog events the difference is is if you look at the that's actually there's I don't know why they do it but there's two there's two different ways that you can interact with the system logs and one was the get event log like I showed you earlier but you can't work with any of the advanced more modern of system logs with get event log that's only for like your basic five event logs to look at

where as get win event like if you you know they have Microsoft's the like 50 60 different Microsoft event logs underneath in their advanced system that's how you would look at and be able to query and that is taking the output and converting it to an an actual custom PS object by doing that it now takes the message it formats it is an object so then I can now if I want to put it into a CSV file to load into a database so that I can get additional collection data there if I want to just load it into Excel and look at events there I can do that so it's one of the it's a

that's one of the things that you can do that you can't do that you can do a sis mind that you can't to Sara Lee do with the output from all of the windows logs and the thing is is if I change my event ID which I think three no matter what my event ID is the same format of the message the way that it formats the output of the message it doesn't matter how many lines or what's in there it will be able to format the same on each line so when you split like I was talking about being able to split into a newline build an array and then split off of that string it's the same

type of concept so that you're able to then parse the data a lot easier and be able to see get at things that are going on on your network a little bit better it's not like it's not something that you necessarily have to use and there's benefits to it but it's also not necessarily meant to scale the way that people have been using it but it is it is an additional tool that you can use in in your arsenal to be able to get at the information that you have

so pretty much from there working you what can you do you can do pretty much if you can log it you can this start building logic off of it you don't necessarily need a seam to be able to do it you can also use it use this information as a way to tweak your your existing scene without actually making modifications to it while it's running in production and so you're not messing with any of the data logic that you're already getting so it's a way to kind of see what's happening you can mess with your audit policies and then just take the information from from these law from those logs off of your audit policies to

make sure that you're getting actionable data out of that as well and then in my case like probably where it clicked the most with me for this type of scenario was I didn't really have anything monitoring my Active Directory environment at the time but I wanted to be able to see who was mod any type of anytime a modification on any type of special event on my domain I wanted to be able to see that that happen so I had a scheduled task that ran periodically that would go and get all the events that occurred within since the last time the scheduled tasks ran and it would look at those and then see if there were

any group additions and look at see things like if a user modified my domain admins group for example and it would tell me that that group got modified and it would telling me who modified it and it would tell me the time and then I could go back to whoever did that to make sure that it was actually there was a legitimate reason why it was happening and so if you don't have the tools it's a way that you can start getting some of that insight and then you can start using that as even potentially like is a proof of concept to get you those tools later on down the road so more information we said you can look at

Carlos Perez and Jessica Payne are two people that kind of talked about this a little bit more in depth jessica actually just wrote one back in December that was really interesting about doing something similar with the event forward and loading it loading the data into bi to get more analytics and and some of the Microsoft Azure automation behind it to be able to parse your events and alert off of that and the specter ops group puts out quite a bit of information and normally when they put out a tool they'll talk a little bit about this mine and configurations like that as well and then last is the Swift land security log Twitter handle there's

quite a bit of information under there as well if you want to talk to me more about it I'm on Twitter at eff pieces you can email me at my email address and then I'm on numerous different slack channels but these ones here the primary ones at online so with that I'll kind of open it up for any questions anyone it might have

I'm not sure on the things that aren't being caught by default and when I'm starting out looking at the things that are running in my environments and that I'm looking at the types of processes that are being created and where they're being ran from and that type of information and the thing is is like I want to talk about things like that there's a lot of legit there's a lot of new tool tools that are being that are used for legitimate purposes that aren't being used legitimately anymore and so making sure or checking finding out if those types of things are running on your and where they're running from and if they're supposed to be running from

there

I'm sorry not in my current neighborhood in the current environment no so I I won't use their PS exact is now one of those things that it's disabled I'll push people towards using PowerShell remoting things like that versus using PS exec so that's just something that I know that I can I can I can alert on because I know that it's not going it should not be legitimately being used in my environment

all right thank you everybody [Applause]