BSidesIowa 2019 Reviewing PCAPS with Security Onion

all right everybody welcome to this presentation almost kind of a lab but we're not going to be setting up security onion although I'll be going through some of the steps but this this presentation will basically be about me how I will use security onion as a way to generate alerts on pcaps of malicious network traffic and I've got the link up here it's where the it's got the three peak apps that we're going to be using today and they're there in zip password protected zip archives the password is infected and there's also a PDF file that has the PowerPoint slides I'm going through right now so for this hour we'll look at a lab environment pcaps ins

security onion how I review peak apps using security onion a little bit of the way that I set it up and three examples of malicious traffic and how security onion can shed some light on exactly what's going on through the use of alerts and what-have-you so this is a really simplified diagram of a real-world environment where you've got clients and you got servers that are talking through each other to each other through switches and routers most likely before we get to the firewall which is routed to the Internet and then usually before the firewall you have a security appliance that is detecting anything malicious that's going on in the network and depending on the type of appliance

it could be just monitoring or it could be some sort of intrusion protection system so I talked about security onion because security onion is free people have asked me why I don't talk about my employers security systems and those are really for an enterprise environment and it cost money so my mandate is part of unit 42 so network's is to help inform the community and just in overall just better prepare people to be able to in my case to help people be able to understand militias network traffic so for a lab environment physical lab setup that I have at home looks a little bit like this if we're gonna draw a flowchart I've got Windows clients I'll

set up a domain controller I'll set up a client and then that will feed into a switch that's monitored by what I have is a gate computer which is recording the traffic and then it goes out and is routed through the internet and when I have this set up I'm using intentionally vulnerable machines and no protection whatsoever because I want a full infection chain of events all right I want to see you know from the very beginning of affection to anything that could possibly happen on that infected Windows host now what happens in the real world when we're out there anybody who's responsible for near real-time detection of malicious activity in their network you're going to see bits and

pieces of these things almost always I've never seen in a real-world environment a full infection chain usually somebody takes their work-based laptop home they get it in fact that they come back and you start seeing the post infection callback traffic or if you get an alert at work it it gets stopped by your protective mechanisms in place so you may see say like an exploit landing page and that said everything gets cut off or you may see an attempt to download of malware but it gets blocked so you don't see a fall infection chain but in my lab environment what I want to see this stuff I'm curious I want to see what would a full infection chain look like

because normally the places that we work at we're not going to see that it generally is unprotected home networks or the types of companies and organizations that do not take security as seriously as they should so this is a representation of a physical setup and here's a representation of virtual environment now you notice I don't have security onion in there security onion is kind of what's the right word for it resource-intensive the minimum ram recommendation is 8 gigs of ram for for a security onion vm so if you've got any of the for example like a MacBook up until just this most recent model 16 gigs was the max amount of RAM that you could have so it'd be kind of hard to

set up a Windows client and a Windows server and security onion to monitor the whole thing is it's going on as it's happening so what I will do is I will use security onion to play back the peek app and see what would have happened and that's what this next section is about I'm going to look at how I use security onion to play back and review peek apps that I've recorded in these lab type environments so the first thing you have to do for security onion is you go to security onion net there's the Downloads tab it takes you to a github page where you can download an ISO and then you can boot the ISO on

a physical host or a VM and then you install security onion 16:04 by clicking the icon on the desktop now once again you go to security onion net go to the Downloads tab takes you to the github page the github page you can download the ISO you boot the ISO on a physical or a virtual host and as you boot it it will go to a desktop and then you would install security onion when you install security onion it'll take a little while and once you get done the first desktop it you'll see you'll you'll have to configure your system so you please double click on the setup icon as it says and you will need to

configure network interfaces now what I'll do is I'll use DHCP because I'm using the standalone advice device for security onion and I'm not using it in a production type environment so I don't have a static address that I need I'll just take whatever address DHCP will have because it's irrelevant to what I'm using security onion for now notice where it says ENS 33 which is the with more recent versions of Ubuntu which is what a security onion is based on the the distro instead of Ian zero a year and one it's you you'll generally get some sort of a different type of name for the for the interface for the network interface you'll set it up and

you only have one interface generally because you I'm using it as a standalone device and our reboot and once I reboot I could have to go back into setting it up again and I'll have to else get the network configuration because of our configured it now there are a lot of steps to to doing this it's pretty clear it's pretty straightforward when you're setting up security onion which is one of the reasons when I discovered it back in 2013 that I really liked it because I tried to set up snort based on I think it was FreeBSD at the time and I failed miserably at it but security onion was the first time that there was a distres

like hey you just follow the instructions and boom you're good to go so there's the in this one the VM that I used as I was setting it up specifically for this presentation I just used four gigs of RAM well once again I'm not using it in a production system I'm not using it for a full near real-time monitoring I'm using it just to playback peak apps and see what types of alerts I can generate on them and I am going to use they'll have an evaluation mode I'll use production mode do the new set up and instead of best practices which is what you think you should use I'm going to go custom because I want to I'll explain it

here in just a little bit with best practices you don't get to pick the signature set that you're going to use and the the IDS engine so you've got different signature sets I want to say that it uses a defaults to the emerging threats open rule set and we'll use snort but as the ideas but Cercado is also an option for the IDS what what I suggest to people is that you can use the snort subscriber rule set the subscription itself cost like 30 bucks a month but you don't have to subscribe you can just register and get rules that are only 30 days out of date and this one this third one here with a smart

subscriber tell us rule set and emerging threats no GPL rule set is is the one where you will get both rule sets that will come in at the same time now you'll notice when you do that it will ask you for your snort subscriber or tell us code so in this case I want to go to snort org sign in when you sign in there's a little area where you can sign up and I'm going to use my email in this case I've already done this but you know make a user email make a password agree to the agree to the conditions and then there you will have your profile you can go to your profile and then click where

it says point code and you'll have your own code your own personal line code and you can use that copy and paste that into the security onion for your own code so you can get the registered rule set from Talos which is the snort based rules that they have as opposed to the emerging threats rules which in this case we're using the open rule set which is free as well and I'm just gonna leave it on snort although you can do sericata as well in my personal home lab I'll tend to use just the I've got access to the emerging threats pro rule set to use that I don't have the option to combine that easily at least using this setup

with the snort rule set so because I have access to the et pro rule set I will use the et pro or sorata a couple of other things as you're setting it up yes you want to enable the IDS engine so you can actually see the alerts on the traffic when you're playing back the peek apps and that's and that's pretty much it everything else when you're setting it up you should be able to use the default value or whatever it has and then once you get done we can start squill which is a graphical interface that that shows the alerts there are other ways that you can look at it on security onion squill is the one that I

tend to use that I refer to when I set up the account when I do the setup I use the easiest possible username and password which is the username as a and the password is six a's which is as simple as i could get it and in this case once again we've got OS sec the the name of this virtual machine is Esso user vm and you'll notice that the e and s 33 here which is the interface that i'm going to be using to monitor so when i use TCP replay to replay the pcaps in security onion I'm going to use ans 33 and then there's quill right there it's opened up it doesn't have anything in it

I'm gonna minimize that for a minute and at this point I've got my three P caps that we'll use today that are available on the website or on the USB keys here and if you're using a Windows laptop there is malware in these p caps and especially if you use in Windows 10 Windows 10 would tend to delete those P caps because it detects malware now I want a terminal so I can do the command line commands to do the TCP replay command so you can basically just right-click in the background and open terminal when you open the terminal this is the command that you would use in this case to playback that first example

of the pcap sudo TCP replay - - int f1 equals ENS 33 and then the name of the P cap now the only problem when you do this is that P cap is traffic that takes place over to space about two hours so if I'm going to play this back I'll have to wait two hours to see all the alerts on this so you can use a speed multiplier and I'll generally use 30 in this case so 30 you know divided two hours by 30 and you've got what 160 divided by 30 yeah I don't know it's if it's a few minutes as opposed to two hours that you'll have to wait so this is the

command that I used when I was generating alerts for this presentation so if you got everything set up and we know the command to use and we have three P caps that we can look at right now so let's take a look three examples of malicious traffic once again that malware - traffic - analysis 9 / 2019 / b-sides I a I've forgot to capitalize the s and B sides so I don't know in some cases the capitalization is important so we have those three P caps and they are all set up in an Active Directory environment so this Active Directory environment the domain is Papa sunlight comm where the network segment is ten dot 4 dot 17 the

domain controller is on dot 4 papasan line - DC and then you see your segment gateway in the broadcast address we basically have our network segment here for this Active Directory environment that we're going to see three examples of Windows malware infection traffic so use that TCP replay command and I get a bunch of stuff I get a bunch of alerts you know you'll notice on the left-hand side you have read and you have orange and orange of course are not as serious apparently as red alerts and every once in a while I'll see a yellow which usually is usually some sort of protocol based thing or very very informational it's not an alert it just kind of gives

you a little more context on what's going on on for the traffic now we'll say in this I didn't see anything that was on the snort tells register ruleset that that showed up so I don't know if it was how how I set it up or if I just wasn't receiving I don't know why I wasn't receiving anything on the snort owls rule set but I was receiving plenty on the emerging threats open rule set now one of the alerts don't want to point your attention to is where it says et Policy DNS update from external source but if you look both of the IP addresses this DNS was coming from within the network so there's

obviously some sort of miss configuration here so I actually had to go into the snort dot-com file that's not a configuration file and where I found that the external net showed up as any IP address which would be any of the internal IP addresses I don't know why that happened so I basically had to change that in the snort configuration file to show that external net is not home net sericata has their configuration file in the same directory and it was the same thing with sura cotta as well it they had commented out the line that shows the external net is not home net and said it was any so I had basically had to switch that so yeah

and I found this consistently when I was going through security onion setting it up this the same issue where it showed external net as anything and which also included whatever my home that variable was so that was one thing I had to fix right away but the interesting thing in this one is I could ID the infection based on one of the alerts which I have highlighted here the second from the bottom which is iced ID which is an information stealer also known as bock pot and the great thing about security onion is I could go to that alert I can highlight it and I can look at the the alert itself what are the parameters

what's the pcre the Pro compatible regular expression that was used to catch this particular malicious traffic and I could also see the actual TCP stream of what the traffic was that triggered it us for a security onion if you've got the the options for a bro turned out now what's bro called now because then they renamed it Zeke so it still it still called bro in this version but Zeke there's an extracted directory where it will automatically extract any Windows executable files or dll files that it finds in the pcap as as we're playing it back and extracted it out now if there are other things in this case there's also a word document a Word document

with macros that kicked off the infection chain that was downloaded it was the initial thing that kicked off the infection chain that's not going to show up in here so if I want to see to be able to use Wireshark to look at the pcap and find the both the malicious Windows executable file and the malicious Word document I can export HTTP objects from that pcap and Wireshark and in this case I've got two objects that I can export one says letter underscore tj8 three doc that's the word document with a macro you enable the macro and then it immediately calls to team focused comm PL the URL there which returns a Windows executable file which is nice tidy so that's

example one another thing which wasn't apparently clear to me when I started fiddling around with security inand was how in the heck do I clear the alerts right all these alerts were piling up and I didn't know how to get rid of them it's like well you just select them and then hit f3 and f3 will delete the alerts f9 will escalate the alerts and put it in the escalated tab now the and I'll get to the reason why you'd want to escalate them here in a minute let's look at our second peak app now our second peak app we're doing the same type of command and we're seeing far little in this case we only see we only see basically

Windows executable being downloaded so we don't see anything else now the interesting thing here is you'll notice in the in the count section that that CNT column all right the CNT column shows five events for that DNS update from an external net but 34 events for each of those other alerts so they're grouped by source IP so if it's triggering on traffic coming back from that public IP address that that server then you know that all of the alerts that are triggering on that traffic came from that one server but if your source IP is your internal IP address and it's going to an external server its triggering on like a get request to some public server then and

you have 34 alerts it only shows the very first information right it's only going to show that TCP and TC the the IP address and TCP port pair so if you escalate those if I were to click on this second or third or any of those lines and hit f9 to escalate them instead of f3 to delete them it will all have a list of 34 alerts if I do everything there I would have you know 70 something alerts well let's get back to the this particular matter at hand so there is an infection on this computer I can guarantee you that I know because I infected it but all I'm seeing here is that there are policy alerts and the

fact that there was a Windows executable downloaded from 180 3.1 77 238 dot 19 so what else is on here well you could you get to explore the pcap but in this case there is email traffic all right there's email traffic coming from this Windows client SMTP traffic if you filter on SMTP for this particular pcap you'll see plenty of plenty of information there and if you followed the TCP stream so basically left click to select any of the frames that are listed there and then right click to bring up a menu follow TCP stream you will see the SMTP traffic that has an email that is exfiltrating information about this infected Windows host this is

a key logger and you can tell where it's being sent to the subject line is Constance Neumann / Neumann PC keystrokes which is a clue right there that this is something a little shady going on and if we filter in this case on SMTP defragment that should show us how many times an email was sent out within the the recording time of this pcap and what what I found out was it didn't send in anything periodically it was only when you typed so you can see I went about 22 minutes going I'm not seeing anything else come up maybe I'll just start you know typing up documents and saving him and doing stuff and once

I did that then I still you know I started I got three in a row really quickly so it's sending that stuff out there so I want to say this is a Hawkeye key logger but whatever it is it's it's an info stealer it's a key logger and you'll notice that with this open rule set the stuff that's really available I did not have anything to tell me within security onion that there was anything wrong other than that initial executable which was more of an informational alert let's look at the third pcap and we do have some interesting things here so and this is another case where the paid rule set definitely gives you more additional

insight than the free rule set all right so the emerging threats as a pro rule set from what I understand last time I remember hearing about it was like 750 bucks per year per sensor right so if you're if you're a if you're an enterprise you got sensors all over the place you're you're you know it's a that's a good bit of money if you're a single person like me I'm married but a an individual like myself who you know 750 bucks a year that's not someone I'm going to pay I'm lucky in that because I do the website the malware traffic analysis dotnet website I was contacted by the emerging threats guys before they

got bought out by Proofpoint and it was able to work out a deal where I could I have access to the emerging threats Pro rule set for purposes of the blog but what I can tell you is that if you look at the alerts that are highlighted here in red the very first one you'll see that it shows a fee odo tracker command control channel Fierro Doe is another name for a motet I mean people have heard of a motet okay so a few hands you motet eat motet is a malware that's an information stealer slash banking Trojan and it also acts as a platform to load other malware so as I'm as I'm testing samples v motet

I can almost always generate some sort of follow up malware in this case the follow up malware is trick pot you can't see it here but on the last one you'll see an SSL blacklist as far as the sir the the ssl/tls cert certificate where it says it's dried X slash trick bot and in this particular case that's oversee and we're seeing HTTPS SSL TLS traffic on TCP ports four four nine four four seven which is typical for a trick bot infection so what happened in this case is you had a JavaScript single J's file so it was a it was a fake invoice is what it was so you had an initial download of a zip archive it contained

its you know something that is something invoice J s the unwitting victim on a vulnerable computer would double click that and then it reaches and then it grabs the the follow-up executable in this case in C motet the e motet malware binary and then you start seeing some a motet callback traffic and then a little while later you start seeing the trick bots now e motet communications are encoded or otherwise encrypted so when it emote EDD grabs its follow-up malware it's not grabbing it in the open like we'll see in the very beginning so you're not going to see any alerts for follow-up executables come over the net because they're all encoded as they come

over the wire so if I were to look at this particular P cap the third p cap using this specific expression I want to look at the HTTP requests that are listed and I want to look at SSL handshake type equals one and not SSDP traffic because there's some UDP stuff that comes across with HTTP requests now what I will say is this setup that I have here for Wireshark of column displays as something that I've talked about before I do traffic analysis workshops where the whole first hour is here's how you set up wireshark to get a better view of the traffic that's more applicable to some that's looking at malicious traffic infections because Wireshark is an

amazing tool but by default it's set up to be a jack-of-all-trades to everything to everybody right so it is set up to appeal to as much somebody that's trying to diagnose a network for connectivity issues or some some sort of network problems than it is for actually you know having the mindset of a security person that is looking at traffic and trying to determine what happens especially with a lot of web traffic now one last trick that we can do is we can use in security onion we can enable this pulled-pork configuration file pulled-pork is the program that is used to update the rule sets for snort or in this case snort or sericata so it's setup but one of the things that

I figured out early on is there are some rules because I'm just playing it back and want to see every possible thing that that is listed I want to do enable all of the signature IDs which is the SID here so I'll go into that and I'll add at the very end pcre : alert so I want it to actually trigger on everything that shows as an alert even if it's disabled by default in that in the normal configuration you'll get a lot more alerts and it can be a lot of noise so this definitely went a little quicker than I thought it would because I have like 63 slides and definitely went through them in approximately 30

minutes let me do something here real quick as we're taking questions I'll put that URL back up so if anybody has any questions now is the time to ask please yes ma'am how did I know how to do the changes in the files the configuration files and whatnot trial and error I first discovered security onion in 2013 and one of the things that actually did happen is as I'm doing these blog posts and showing what I have and you know there would be certain things that would be in the traffic that I wouldn't be able to show on the blog post and I'd correspondent via email with Doug Burks who is the security onion maintainer and

he'd given me some helpful tips on some of this stuff and that's how I picked up some of it the the part with the HomeNet issue that was just me figuring out why is this happening I got to look into the configuration files and they that that was just something after a while you kind of pick up on so I mean you really have to know in order to be able to do this effectively there are three things that you know you got to have a basic knowledge of network traffic fundamentals right to understand that UDP port 53 DP port 53 for examples what you normally see a lot of DNS traffic on all right there are just

certain things that that you have to be used to and it requires a good familiarity with network traffic fundamentals because we're dealing with network traffic now another thing to use security onion to get good with that you have to have a good solid knowledge of linux fundamentals right so in order to be able to you know just open up a text editor you know knowing that you have to use sudo - - - VI a system file you know that's owned by Roots you know certain things that you have to be able to do that that implies that you you've got to know a good solid Linux fundamentals oh you're welcome yes sir yes I do I don't have a video

the question was that I'd mentioned how I have preferred setup for Wireshark for the column display on the malware traffic analysis that net blog I've got an older one that I did and I recently did one at the last half of last year for Palo Alto Networks for the blog if you look Wireshark configuration Duncan and then maybe Palo Alto Networks to Google that you should be able to find that one pretty quickly right now at this point in time I've done three blogs that are tutorials on Wireshark for Palo Alto Networks for the unit 42 blog so I've got the one for setting up Wireshark I got one for identifying hosting users I got one for using

wireshark filters any other questions yes sir

good question the question is how often do I see that initial download and the well now I could say so I've been working in Palo Alto Networks as a researcher not as for you know any sort of customer support just straight research so I don't have prior to that I was working at Rackspace at a socket Rackspace hosting provider right so I would have some indications that you know they're on this stuff coming in but what I'm doing a lot of these mountain where our samples I'm getting from virustotal URL Hass other things that people are reporting and these are stuff that generally get caught by their usual email spam based so malicious spam that

generally gets caught by the your organization's spam filters so this stuff isn't even seen now every once in a while you'll see something because it's like a it's like a firehose the criminals are using a fire hose and just spraying out whatever they can write every once in a while an email will get through for one reason or another it's that kind of a cat-and-mouse game right there they're constantly tweaking their emails just trying to get that that that initial word document or you know whatever to get somebody to double click so it's rare and then like I had mentioned earlier a lot of times if you've got so you if you have decent spam filters in place if you got a good

firewall and web security gateway filtering type stuff it's you're not gonna see a full infection chain but so the answer to this stuff this is all commodity malware which is out there every day as the criminals are just trying to push it out firehose like fashion to get that 1% worldwide that makes it somehow profitable for them so realistically we don't see that much now what I find what I found in in my personal experience with a few crews that I've worked with is that the better your security systems are your security architecture is the less experienced that the the people that are looking for the bad activity will have because you don't see it you start getting

complacent so it's kind of rambling answer to your question but I rarely see this stuff in a decently protected enterprise environment if at all but these are the same tactics that you know targeted attacks will use right so I mean it you may not see these particular examples but some of the stuff that we run into hopefully as I'm providing these to pcaps in my blog or the types of traffic that you might see if your organization is you know the the target of a successful attack you'll see some of this stuff any other questions well if not oh sorry yes

do I have a preferred preferred OS for my infected lab host Windows 7 right now Windows 7 64 bit the Service Pack 1 or whatever it definitely is much much easier to infect a Windows 7 host than a Windows 10 host so say what you will about Windows 10 as far as usability you know spyware callback type stuff all the other complaints that people have about Windows 10 Windows 10 is that much much much more secure operating system than Windows 7 is but they are still they are still close enough that a Windows 7 host could get infected you know it will the file directory structure and some of the things that you will see artifacts from an actual

infection you would seem in the same places on Windows 10 knows it's a it's a real pain to actually get a Windows 10 host set up to where you're disabling as much as you can in an almost unnatural manner in order to get it infected for Windows 7 default settings for Windows 10 I would have to basically turn off the firewall and make sure and turn off any of the levels of protection and even then there's probably some registry tweaks that I could do to ensure that certain things are also not going on that would make it even easier to kind of set it up to where I can infect a Windows 10 host like a good a Windows 7

host set of default any other questions if not then thank you very much for coming to my presentation you