BSides Delaware 2017 - Day 2

First of all, welcome. My name is Alex Rubin. I'm talking a little bit about post-exploitation. So this is kind of a quick, high-level overview, rundown on common techniques and tools that you may see in the environment if you're conducting a pen test, if there's a current attack happening. So a little bit about myself. I finished my Associates in Information Security from Delaware Tech Terry Campus last December. Since then, I enrolled at Wilmington University in Computer Network Security. I've got another year left there. Anybody interested, we'll be posting the slides and the demo videos, public Dropbox afterwards, and there will be a link on the Twitter account. So anybody who's worked in a Windows domain environment, this will be a little

bit of a review, but these are some things that I really want to cover before we get into it. So the big point with Windows domain environments is centralized authentication and security. Everything managed through Active Directory Domain Controller. to log in to your domain account, you're actually authenticating to another server. You're not authenticating to your local machine. That brings us to the domain users versus local users. That's your domain user that you're authenticating to when you log in. So as a local user, you'd be authenticating to your local machine, and the important part about that is the password hashes are stored on the local machine. Domain environment, those hashes are actually stored on the domain controller. And user access tokens and single sign-on, I don't want

to get too deep into that. be aware of access tokens. That's how Active Directory keeps track of what resources you have access to, have permissions for, and what systems you're logged onto. So if you log into your workstation on your domain, there's actually going to be an access security token stored in memory for that computer. So I'm going to kind of tell the story of a pen test that's going on here. Use it as background for the attack that I'm going to demonstrate. So you have been hired to pen test a business with a moderately sized Windows environment. You've already compromised a single workstation for a phishing email. You've got somebody to download an executable and run it. You have a shell. That's all

you have. So you have a shell. Now what do you do? That's where the heart of this presentation is. So after you get a shell, what can you do to move around, to escalate, to secure your foothold in that Windows domain environment? Before you do any of this, you have to identify what you're going to be targeting. Anybody just shout it out. What is the main target in a domain environment? Go for it. But there are also secondary targets that you can hit along the way. So what would be something else? Maybe network administrative workstation, good stuff on there. You have file share, everybody logs into a lot. You need to plan out what's going to

happen ahead of time before you just start attacking everything. Have your foothold on that one workstation. From there, you need to escalate. Maybe you don't have the permissions, maybe you only have a local account. Get your foothold on the domain. So one of the tools that's really good for that, once you have what's called a meterpreter shell, is Hashcat. You can dump the password hashes, use Hashcat to crack those hashes into plain text passwords. Actually, I have a video demo that I recorded earlier. How to do that. So you have your meterpreter shell hash dump. You can see right there, account name, Landman hashes, LM hashes. So I already have those hashes on my Windows system. I'm running

Windows Hashcat so that, there they are. so that I can use my GPU to track these hashes. So regular hashcat, it works with your CPU. It utilizes that resource to track the hashes. A thing called CUDA hashcat that utilizes CUDA architecture allows you to be much faster at this kind of task, track those hashes. Like for instance, in a Kali Linux VM, the ones I use at school, this process may take about 30, 40 minutes, maybe an hour. You can see here, it takes about a second. So a little bit about what I'm typing in here, -A0, dictionary tag. So I'm trying all the passwords that rock you.txt. It has how many passwords? Millions of passwords. Actually taking the hash of each of those, comparing it

to the hash in the file, if they match the password. -M1000, that's the mode you're using, the hash type. So NTLM, hashcat, that's mode 1000. Have the output file, target hashes, and the word list that you're using. Enter, and it's done. Not a long process. So there are all the plain text passwords for all the users on that system. So you may be thinking in a domain environment, why do I need the local user? So think about you have a domain environment that has two, three hundred computers. You're not going to want to image those one by one by one by one. In an image server, you take one image, push it across the network to

all of them. that local administrator account most likely isn't going to have a different password for every single workstation. So if you get the local administrator password, you probably have the local administrator password for every single system on that domain. Back to my PowerPoint real quick. As I was saying, the local administrator account is important. Keyloggers. It's a fun one. So you may not have the password hashes that you can crack for the domain user since they're stored on the domain controller. but since the user that you've compromised logging into that system with their domain account, drop a keylogger. You already have full access. You can actually use Metrepeter. They have a built-in keylogger module. You just run a command, it dumps it

on there, and start recording keys. We'll be coming back to that later. You want to expand. You want to move laterally. You don't have sufficient permissions on that one workstation that you compromised. You want to move out into the network. Maybe to a file share, the network administrator's computer if you can. So like I was saying, local administrator logins are often reused on many, if not all, the workstations. Just because it would be incredibly difficult and you need to go around and set a different one. File shares. The reason file shares are such a big target goes back to that single sign on the access tokens. So when a user authenticates their domain user, their domain account, creates that token. That's stored in memory. When they access

the file share, since it's single sign-on, they don't need to log into that again. It passes the token. So just think of all these users that are accessing the same system, just dumping all their tokens in there. You can use Mimikatch, you can use Metrepeter, there are a few other tools that you can use to grab those tokens and actually impersonate them. So specifically, if the domain administrator had logged onto the file share, impersonate the domain admin, job's done right there. But let's say for our purposes, we're not long on the file share. But we're able to access the domain administrator's workstation. Actually, I want to take a second and do the Mimikatz demo real

quick. This is also another way to get passwords. Mimikatz, a RAM saving tool. So it pulls information out of running memory, plays on the screen for you. So for instance, local user accounts that are logged in on the system, Mimikatz to actually dump those plain text passwords out of memory. Kind of scary. So this one actually has the initial attack in it. I set up a listener in Metasploit. Find notabackdoor.exe. Really safe. There's a shell. So right now I am the student user. Escalate the system. I'm now the system itself. So the Mimikatz module, there's also a standalone version you can use. There's some plain text passwords. So in newer versions of Windows, you actually have to migrate

into the local security process, the lsas.exe process to be able to do that. But in Windows 7, it doesn't care. They kind of got smarter about it in Windows 10, actually virtualized and isolated the lsas process. So this still works in very specific conditions, but it's a lot harder to do in Windows 10. Plenty of other tools you can use. This specific technique does not work nearly as well. You are now on the local network administrators' local admin account. Something you might do. Keylogger. Stop the keylogger. I'm going to type in his domain password. He types it in. You have an administrator shell on the domain. Typically, this is where your job as a pen tester would be. You may need to have

vulnerability testing on other things, but as far as the domain goes, you have the keys to the kingdom, you have everything. You can access anything you want at this point. If you're an attacker of the malicious variety, because you're not getting paid to hack this network, You want to get your money's worth. So you want to stick around for a while. So you want to persist in the network. So Netcat is a fantastic tool for that. There's also a persistence module. So I have demos for both of them. Netcat, it actually creates a listening port that hosts a computer. So this is me configuring the registry on the target so that on startup, this process starts running.

I'm uploading the Netcat executable, and then I restart the computer. I reboot it. And when it comes back up, I'll also open the firewall port for that one. I think it's port 9000. And when it comes back up, it now has a listening service on 9000. If I connect to, it drops me a shell. There's a registry entry. The backdoor is running. I just use the same tool, Netcat, for listening and reading connections. Connect to my target system on port 7000. It drops a shell. So that'll work as many times as you want it to. And then there's also... another method with the Meterpreter. There's also another part at the beginning of this just to show some of the fun stuff you

can do with the Meterpreter. I just took a screenshot of the victim's display. Hey, there's some secret stuff right there. So it doesn't seem too bad, but imagine if you have some important spreadsheet up on your screen. So this is the Persistence module in Meterpreter. So run persistence. Dash X. This service starts up when you boot the computer. You can also use dash U for when a user logs in. - I 5 it actually beacons out a every five seconds doesn't create a listening port tries to initiate a reverse TC so think about how firewalls work they don't trust things on the outside they don't want things coming into the network we're using target machine it's connection it trust

the firewall trust things from the inside port 9000 my IP address that connects back to and then - a it'll automatically create a listener to that connection runs it's a VB VB script that manages this whole process and I have another shell now. So at this point if I want to connect back in all I have to do is set up a listener on that port on that IP address, wait about five seconds and I'll get it. Further learning. So there's all kinds of additional functionality in Netcat, Hashcat, Mimikatz, Netasploit, Meterpreter, all the tools we went over. There's so much more they can do that we can't talk about in 20 minutes. So much fun stuff. Like for instance, Meterpreter, you can take screenshots,

you can turn on the webcam, the microphone, loggers whatever you want you have full access to my recommendation we to try yourself in a lab please in a lab okay so that that's how I simulate all this I had two VMs I had Cali and I had Windows 7 I was using one to attack the other it all took place inside this one laptop that's a really easy setup you can spin up a couple different versions of Windows Cali Linux it's been up one at a time and attack them even see what holes you can find and then go learn about how to fix them. Honestly, that's the point of all this. We need to fix these holes so these tools don't

work anymore. There you go. Any questions? So the persistence module, I have not done it that way. I believe the script that it downloads is only configured to do reverse TCP, but there are plenty of other tools. With that one persistence module, it's very good at what it does, but it only does that. But there are plenty of other tools. Netcat, you can configure to use on any port. Very versatile. Question? Domain passwords are not actually stored on the system. That's what the token's for. To authenticate to the domain controller, it gives you a ticket or a token that you use to then authenticate to any other resources. So that security token is what's sitting in memory. So you

can actually use a interpreter to impersonate that token and then forward any traffic through that system. It looks like it's coming from the token you're impersonating to the domain controller. use that as a pivot point. But you can also use Mimikat, what's called a get attack, and you can actually create your own Kerberos ticket. Active directory uses for authentication. So if you do get the domain administrator hash, you can actually create your own ticket signing ticket and create your own access to it. Mimikat is also very versatile. Not just for dumping passwords at it. Okay. So I know you said it's actually upgrading, but Windows 10. Mimikatz, they got smart. So the entire LSAS process, the local security manager, they virtualized and

isolated the entire thing in memory. It is extremely difficult, I won't say impossible, but extremely difficult to get Mimikatz to work in that way on Windows 10. So specifically in Windows 10, there's a feature called Credential Guard, which is what virtualizes that process. You want to make sure that's running, make sure that that process is...

specifically for the local administrators if your your domain is so big that you think you still have to have same image for everything the same local administrator password make it something read it make it something that hash guy is gonna crack in a second because that that password is only valuable there are other attacks but for that purpose of course it does yeah I didn't know that's no

Kali Linux? Get off the website? It's free, open source. Yeah, that's what I was getting to. There's also whatever Microsoft DreamSpark is called now. Microsoft's been getting a lot better of allowing people to test their operations. There are resources out there. Yeah, that'd be fantastic. Off the top of my head now. Okay. You absolutely could. You absolutely could. It just doesn't happen very often. It doesn't happen very often. It's more... The administrator gets in that mindset that this is a domain environment. They don't need to worry about local accounts. And then they push out that image and the same password for every single one. So there are ways to mitigate it like that, just you

don't see it very often. Does anybody know that? Question? Question? It's my understanding that it's in all Windows 10. My understanding. Thank you for coming. Good morning. If you could please have your seats. I have Well, I had the most awesome thing and awful thing happen to me, which is my slide deck kind of exploded last night. So I spent some time rebuilding it. I have a lot of information to go through. And hopefully, by the time I'm done, you will want to try this at home, unlike what my t-shirt may imply. And there is a prize. If you find something interesting and approach me and show it to me, you could win, or rather receive, this shirt. One moment.

It's an out-of-use t-shirt. Lightly used. Only partially worn. You could win a nice holiday shirt, very festive for the coming holiday. So, as I said, my talk is called Ninja Looting Like a Pirate. Standard disclaimer, no enforcement to be here, no gun in my head, no offers of valuable, valuable, valuable items. There unfortunately will be bad language in the presentation, because normally I don't get to speak to people, and I'm very happy when I get out and get a little over myself. Me, I come from here, lovely city known for technology, lovely Baltimore, not really. And I'm a pre-1986 kid, so boom, that was my gateway drug. I also do like to hack the human body, throw people around, choke people out. It's amazing that only three things

can stop a conflict. And if you remove any one of those three things, there is no conflict. You know, air, mobility, and you know, that whole, the breathing thing, the mobility, respiration, kind of working hand-in-hand. That guy was fine, by the way. I also like to rock a sweet bandana, work on my Jeep, and I'm a professional, so I have these. That's a long story. Never worked there, but this is what happens when a company gives you cool blue shirts instead of arrays, and your employees come in, having decorated them with Best Buy accruedments that they picked up off the internet. So, why are we here? We are here because in today's society, we actually

sit on top of the largest data repositories that have ever been known. Literally access to everything. Um, And what we're going to do is we're going to take a little trip across the three dubs in a way that you may not have taken a trip across the World Wide Web. So we're going to look for some hidden things that people have taken from other places and placed on their sites. And we're going to take those things for ourselves. And mainly it's items that you normally find using technology that has been demonized simply because you're sharing things that you should not share. So today, we're going to pretend to be the bad guys. But we're going to be the bad guys utilizing Google as our agent. A couple

of terms. I don't know if anybody remembers back in the BBS days. If there's anybody in the room do BBS, rocking it. Awesome, awesome. I used to co-sys op a BBS called the Digitalized Fortress, which was pretty cool. And we had people on our boards that were like leeches. You put something up, people take and take and take and that kind of thing. Today they're more like ninja looters. They try to sneak in, take stuff, and just kind of rob a sight blind but never put anything up in certain areas. Today we're going to be ninja looters, just like in MMORPG. So, how did we get here? We got here because people wanted to share things that may or may

not have belonged to them. And the first tool that kind of let us know that, outside of private groups and the like, was Napster. Napster was created back in the late 90s and the individual that developed it was like, "Okay, I want to share music with my friends, my friends want to share music with me." And then it ended up being a tool that was utilized to share music for the world. And obviously it was immediately demonized because the music they were sharing was deemed to be copyrighted material. And it was. The funny part is, in the second iteration of Napster, It actually became an actual professional business and it became Rhapsody, but Rhapsody was not doing as well and actually had to change the

name back to Napster because it had better branding. So, later company projects did follow. The sharing example had pretty much become the standard business model and the interface that they developed pretty much still rules today. Like if you look at the other interfaces that have come along and there will be other examples of those, This interface was created by individuals who were not actually very, very good programmers. They weren't solid in programming, but they developed the interface for usage and thought, okay, this is the essentials of what we need. Fun fact, I don't know if anybody ever played with Napster back in the past. Of course, they can't see your hands, so you can actually

put them up. But Funny thing was, you could also directly interact with the IP address on the distant end, not just for the music, and you could surf other people's machines through that Napster interface, which was actually kind of funny. You're like, oh, it says I can actually look across the machine, so why wouldn't I leave the area that I'm in and go look at other areas of the machine? No one would ever actually do that. That's always theoretical, of course, but the fact was, it gave you that option. Nutella Protocol followed Napster, which was actually kind of developed almost in parallel around that same time frame. And this was actually developed by individuals who were programmers. They were programmers from anybody? Anybody? No? AOL. So they

were programmers from AOL and they said, you know what, this Napster thing is kind of cool. We can do a better job of this because this is what we do. And they did get in a lot of trouble for that. creating this protocol which became pretty much a de facto standard prior to what we recently demonized which is coming up. So as they developed the Nutella protocol and different interfaces and services became available for it, it grew to a point where there were 1.81 million computers that were utilizing that application and that protocol. And that's a lot. Like that is a massive amount. That took what NAPSA was sharing before it got all shut down and just boom, exponential growth. So if you looked

at the actual amount of traffic that was going on, it was a ton of traffic. The market share was, as my slide indicates, more than 40%. It was a very, very popular application. Lots of, lots of, and that's verified by the fact that yes, there was literally a different, does anybody remember any of these Anybody? Yeah. It was like, oh, come on. Bear Share, LimeWire, those were just like, they were pretty much giving stuff away. So their interface, not as simple because they're professional programmers. Here are the things that we would want. The average person, not so much. So it was a little more complex. It gave you a lot of different options as to

what you could actually do once you were looking for targets. They're running to this day, by the way. Oh, there's my, anybody that's watched the South Park, remember Varys? But no, those were really amazing applications for their time. Like, you could really get pretty much anything that anybody else had, and the same situation was true. If you weren't protecting your machine, there were ways of you to actually, you know, traverse the machine. Not saying that I ever did that, but saying it was a choice. So... In the respect of those technologies being placed out there, we had a flexible container solution and feline separation that was pretty dramatic. It was something that the world had never seen before. I am going

a little fast, but there is a reason, because I want to get to actually... No, I do not. Sorry. I want to say something at the end, but I'll wait. But yes, in terms of how the technologies were viewed, they were immediately demonized Fresh took to it, they lit it up, nobody was happy. But wait, there's more, because now we have an even faster solution, you know, in a peer-to-multi-peer fashion, and if you've ever seen or utilized any of the current technology, you know that with a standard connection, decent type, you can pretty much pull 4.7 gig down in virtually no time at all. It is amazing. And this was also embraced by standard companies where, hey, we have our distro. We don't want to

pay for that bandwidth. We're going to put one copy out. Everyone starts to replicate it. And it really offloaded that on them and saved them money. Unfortunately, because technology is demonized, there's some places where you might work. You might pull down, I don't know, a standard version of Solaris and have the security team come and say, hey, you're using criminal technology. I'm like, what is criminal technology? Because I've not heard of that. Like anything, it's a tool. Say again? I'm sorry, I thought you had a problem. Yeah, so criminal technology, because you're using it in a fashion that they don't understand that this is actually something standard, although people do use it for sharing information they should not share. Interfaces are very simple. Does anybody recognize

any of these? Like transmission? Transmission, awesome. U-Torrent, pretty awesome. Straight BitTorrent, of course. Never liked Zura because it was ad-ridden and took up a lot of your bandwidth with junk. It was like, no, I don't want to buy your boner pills, thank you. But it was... Say again? Yes. Yeah, no, the first version was pretty clean. But then it was like, well, we can also make some extra money on this. Let's put some ads in. And the spam that just came through there was just ridiculous. Never liked that. So once BitTorrent hit the scene, it was seriously nothing like that previous cat walking out of the bag. That cat didn't walk out of the bag. He made his own way. So as I said,

media, they totally embraced that BitTorrent thing. They demonized the hell out of BitTorrent. They made it sound like it was the coming of the digital antichrist. But the funny part was they demonized it, but they made money off of demonizing it. Because people flocked to these sites. I had to put Attack of the Show in there if anybody's a fan. Awesome show. Sorry to see it have gone. But they made money by talking about the technology. They made money by talking about people who use the technology in a bad way. They never talked about any of the positive things that the torrent brought to the table. Because remember, does good news sell? No. Bad news runs a marathon. Yes, sir.

That is true, but even those that put it in the correct context, what they did was they would have disclaimers to the technology indicating that we are just reporting on it. I'm talking about dedicated computer-related news sites. They would say, we're just reporting, we do not condone this technology. I'm like, so what you're saying is you demonize the technology. Yes. Right, because they didn't want to seem like they were promoting it, but the more people talk about something, what do readers do? They're like, "Well, you know, I wasn't interested in that, but now that you talk about it, let me go see what this torrent thing is all about." So, the developers were very concerned, you know, they took to heart what was said about them, and

no, no, they just thought that haters are gonna hate, we're gonna keep doing what we're doing, and to this day, the torrent is still around, it is still surviving. So, good for the torrent. So, we talked about the past, and remember, we're bad guys, we're being bad guys today, you know? We're faced with a choice, as bad guys, of knowing that there's a gigantic treasure trove below our fingertips and we have to ask ourselves if we're going to take advantage of it with this lovely Shakespearean power quote. To loot or not to loot, that is the question. And the answer is always to loot. The answer is always to loot. You're a bad guy, you don't play by the

rules. So you're going to take stuff. The funniest part is we're going to be able to do this by using very, very simple commands that are actually available to you despite the fact that the manufacturers told you to use some other things. Because we're going to go from the, anybody ever read Google Hacker Penetration Testers? Volume 1, 2, 3. 3 is good, it's consolidation of the both of them. But yeah, that is just like, boom. Mind-blowing uses of Google. We're going to do that today. We're actually going to make Google our agent to go and do the things that we want. So, that makes this proverb true, is a little bit of knowledge is a

dangerous thing, because you're going to go over four commands, I say three because the three main ones are the ones we're going to cover, but the fourth one is choice two, and it will show you the World Wide Web in a way that you have not seen it, or may have not seen it. So here are our weapons of choice. As I said, we're going to use the file type command. Using the file type command with whatever the extension is, meaning a PDF or any of the standard formats, you can also utilize it for audio and video, but in this case, These are going to be of your concern and whatever the variable is, you

will find so many things. So many things that people may not be aware that are out there. That file type command can allow you to find internal documents for corporations, documents that are marked confidential to a corporation, internal phone books, things that would be great if you're a social engineer. I'm not saying that I am, but you know, pretty sure that I'm not the only person in the room. The Google index is what we're going to be looking at, where we're going to look at what Google believes the site to be made of, not the site itself. And you don't actually touch the target until you go to grab it. And you may or may not be using a VPN, so they're not coming back to you. And

you may be in Cognito mode, so that it does not log itself into your thing. And that's pretty much the gist of it. Utilize the inURL command. We can use it to look at literally any URL. We're going to do straight HTML, but you can also do FTP sites and the like. and the in title command as I said we're going to utilize the index and whatever we're looking for as far as the index goes by doing that Google's telling us this is what this when we search the web we're looking at it from the front end the three dubs boom that's what it looks like when we search it this way we're looking at

the index it's this is what the site is made of and that is where things get interesting so our tools because we're going to need them and they probably exist in your office or on your system. And if you're a parent, you do not want your kids to use this tool. So we have a choice of tools, tools or browser. We have to pick one. Choose wisely. We're gonna choose Firefox. Seriously. That's a party foul. Because despite what people may say, the Microsoft browser is still closer to being a browser than literally any of these others. The reason why we're gonna use Firefox is because And I actually had this, I'm a Nova Hacker guy, sorry, forgot to introduce myself as that. I

am a Nova Hacker guy for the North Virginia Hackers Association. And my first talk there was, what's up with that? And it talked about how we in our lifetime have ruined browsers. We have made browsers shitty. And shittier and shittier we keep making them because there's no such thing as a browser anymore. It's a mini operating system on which your user is the administrator. And I presented my talk in that I have a browser. I'm running a web server out of my browser. I have a mail server out of my browser. I'm the damn admin. Because I've been on engagements where we've actually found a guy running an entire business, it was porn, out of

his browser. And I'm like, oh. And I made the comment, because you used to be able to put POW into Firefox, which was plain old web server, and it was supposed to be restricted to your network. But there were a few hacks to get around it. so yeah it's like running an entire business out of their business and we're learning um there's a giant repository of oh my goodness there's uh yeah there's your guy it's your man right here like this is the only machine so yes we're gonna use firefox because as the administrator on our firefox you know application we can do literally anything there so we've chosen wisely I kind of already did

this slide because there's a lot of stuff in that browser that makes it perfect for looting, unlike a lot of the other ones. Which, amazingly, the hypocritical thing is that Google Chrome browser doesn't allow you to steal from Google. It has lots of things that prevent you from stealing from YouTube and anything that is Google-associated. That just doesn't make any sense. But there's other things that you can take from, and they're cool with that. The extensions that allow it. This is kind of an eye chart, but Mozilla had a response to why their browsers configured the way it is and stated we were never intended for the corporate environment anyway. Because if you work in corporate environments and you're utilizing Firefox or any of the Mozilla products, how

often do they update? Is it regular? No, it's friggin' haphazard sometimes. They'll update two, three times in a week and our people don't keep up with that. Because there's standard patching schedules for a reason. Mozilla's not that corporate browser. It's more for your home use. But as I said, if you have kids or people that you may or may not trust, this might not be the browser you want them to use. You want them to use something that's centrally managed like Edge or IE or something where you can actually control what they can add on. And these just don't allow that. They allow the person to make that choice. And sometimes they're going to make

a choice to loot. I know I would. So we're going to pick our add-ons. If you don't want to buy a VPN, there are several that are actually free. It's like 100 and some of them. And you can go out and pick them. And you want to pick a VPN if you do not have one. Actually, just a quick survey. Does anybody in the room not have a VPN? Okay. That works. You want something that's going to obfuscate you away, even for your standard provider, because what do they have a reason to know what you're doing? It doesn't make any sense. So you're going to pick an actual VPN that resides in the browser and

then you'll start to loot. As you loot, you'll receive the web like this. In this case, at the time of this, I think this is from November of last year, I utilized the file type command to look at all the PDFs that Google is aware of and there were like 2.3 billion of them. That number has almost doubled, I believe, by this time frame. And you can look for samples of different elements. And I encourage you to do this. If you have a laptop right now, please feel free. When you start using certain terms, then you get specific things. Like when you're looking for internal use only. Hmm, I wonder what comes up. You may find out. When you start looking for taxes, forms, literally the things

that people are supposed to be protecting. Court records. I'm like, wow, yes sir. Cameras, yes, but you don't want to use the file type command for that. There are other commands that you can use. But yes, yes. Oh yeah, yeah. Yeah, well, actually what's really funny is with the secondary commands you will learn, let me buy this so we can get to that. So this is utilizing the inURL and inTitle command in order to look for comic books, which people have a ton of them. But getting back to your cameras, so you can find cameras online. But sometimes you're not really looking for the, if you're not looking for an actual online camera, you can just go look at people's Google Drive, which

is kind of funny, because people share it out. And they have a lot of material in there that is copyrighted material. uh, a lot of sites that may or may not know that they have copyrighted material on them, or on them, and we're not really the bad guys, so we can actually use this technique to find out, is my site, does my site have this stuff on it, too? Uh, so it actually is a, there is a benefit to actually running the commands against yourself, your actual, uh, web presence. But yeah, this site, I'm sure he totally paid for all these Superman comics and all the Batman, uh, Justice League and everything else that's in

the parent directories, um, He totally didn't, but it was hilarious to see this. And that is just from running the command. Yeah, I'm pretty sure he did not, which is hilarious to me. And you already heard this talk, too, so... But yeah. So continue to use the end URL and end title command to look at the index of what Google believes the site to be made of. This is my man in Iran. This guy is amazing. He has an extremely nice catalog. It is well documented. And we have the phrase for people who pick locks, it's nothing's ever locked because you can pick it. In this case, nothing's ever missed. If you've ever seen anything

on TV... There are people that have it. The part that really kills me is under the threat of death in this country, this guy is selling US TV. Movies, they call them a series or a I guess it's a series there. When they have different shows that have reoccurring episodes. And it just killed me. I'm like, uh... And they go the distance. They actually have translative sets, .srts, that they'll subtitle it. You can look for that. And you're going, okay, they don't understand languages. Someone's taking the time to step into the role of the actor and translate for them. Or they even have audio, where they have the audio that matches what it is. It's hilarious. All from your browser. You have not loaded a tool

onto the network. You have not picked up a client that we've saw before. You just have your browser. I'm sure, again, these movies were completely illegally obtained because other sites actually had the torrenty things on them. Like, if you've ever looked at torrents, just for research purposes, you would see that... Completely for research, I'm a researcher. I'm the good guy. But you can see that I actually, at the bottom here, started saying, I said, oh, these are actually available, can you actually pull them down? And attempted to pull some down, and then obviously canceled them, because, yeah, they were totally coming to my machine. I did not need that headache. This is another site looking at their

film index. I don't know if you can see, that's actually kind of small, isn't it? Batman vs. Superman. Yeah, any movie that comes out, people have the old school cam rips, like the dude sitting there with his phone in his pocket now, not just like a camcorder, which you can totally see. He's got his phone in his pocket, he's just watching a movie from the middle, not moving. I'm like, that dude is totally recording the movie. Well, here's where it ends up. Like, any movie that comes out in a theater is usually up on someone else's site or on a torrent site. And it's hilarious. So they pull them. Oh yeah.

It just killed me. I'm like, okay, so from just my browser, I'm able to find all this crazy loot? Yes. Oh, yes. They'll have... Oh, actually, what's really funny is the ones marked Farsi are actually in Farsi. That's where they've dubbed the audio over it. Even before the studio has dubbed the audio over it. Here's where I have a bunch of examples, but I really encourage you guys to actually go out and enjoy it. Enjoy looking at it and doing the research. There's one in particular that's on here, and if you go to this site for starving writers, I found these two sites. Starving Riders is an interesting site because it was people that were selling scripts and the like and they're encouraging you to, "Hey, use

me," that kind of thing. But what I found is they'd also embedded this site in them. So there was a site in the site. They had taken it and on this site was, was it Black Sails and a whole bunch of series off of stars that don't show up on the site when you're looking at it standard view. But when you look at what it's made of, you're like, oh, what are these movie files? Are these series files? And it's not the standard available view to you. But if you look at it from what Google believes the site to be composed of, the index, it's like, oh, here's all the movies, here's music, they had

comics up there, they're reviewing comics, but they had the actual comic there. I was like, oh, that's kind of weird. And then audiobooks, basically all the stuff you'll never rent, buy, or lease, available to you. And then the Swiss Bay was... They're not exactly a torrent, but they're not exactly legal. So that was kind of interesting to me. I was a little bit shocked by the fact that a lot of the sites, I can honestly say that about 70% of the sites that I looked at, even sites that were just completely normal, had stolen content on them. One I'm going to bring up, and it was disturbing, and I actually brought this up at our

group meeting, and everybody had the same response of, oh, you cheap bastard, get another site. Looking at people's personal sites, there was a guy who was a programmer, and he was promoting his uber-Christian thing, and so I said, okay, what's your site made of? I start to look at the site, and I'm like, oh, well, you have a series of stolen books right here. All about programming. Like, hmm, that's weird. And then I start pruning the URL, changing it was a numerical scheme, so I'm like, oh, look, what if I put a one here, or a two here? And I started putting that in, and I found his family photos. I was like, oh, lots

of pictures. Started going through it. It's him, his wife, kid, and he talked about being a Christian programmer. And not that there's anything wrong with being Christian, but it was just like, okay, he was really promoting that. Keep pruning, pruning, pruning, adding more numbers, and then I come to pictures that were not in line with what He was promoting. And I stood there like, did I mess up the URL? Let me go back. No, no I did not. Let me go forward. Apparently his wife was a swinger and he was videoing it and taking pictures. And I'm like, okay, here's the problem. You're a programmer. You have kids, or kid. If you're growing up and

you're making them into programmers, you know, they're interested in what daddy does, they're just gonna look at that site. You really don't want them to look at that site. And some people in my group's response was, get another website, you cheap bastard. It's $12. Like, why would you put this on here? We all had the same reaction. Oh, that is a whole new level of no. So, it's kind of weird. But, you know, people do what they do. And how dare you ask that question? Of course there is. But, you know, it's literally what powers the internet. Wasn't looking for that because that's literally everywhere. You don't need these techniques to find that. But the problem is, no matter how deep your logs are, no matter

how extensive they are, you would never see this as unusual traffic because the person is accessing through a standard browser the World Wide Web. They just happen to be saying, "Google, if you've indexed this site, what's this site all about? Show me the components." And so it's a great way to surf, it's a great way to look at things in order to see what the site's actually made of and see what else the site may have that is not visible to you at a first glance or through a standard view. There's your loop. That's it, right? But no, because even without the Google dorking, you could still loot a massive legal repository that no one seems to block. If

you block this, that's awesome. Most people do not. It is a very acceptable means of mass distribution. Can anyone tell me what it is? Anyone? No? So I have another disclaimer in my disclaimer. And that is because we're going to talk about YouTube. Or this version of YouTube. Because YouTube is ridiculously large, amazingly vast, and has just a crap ton of things that in some cases violate even their standard use practice thing. Even after somebody's identified that there's a problem, because they're now owned by Google. They don't make it between you and them. They're between you and the person that's perpetrating the crime. You have to do arbitration between the two of them versus in the old days when it

was YouTube and you could go to YouTube and say, hey, fix this. You have content that should not belong on there. They're like, okay, cool. You go talk to dude. We're going to let dude know you're talking to him. That's not cool. That's not what you signed up for. So, massive repository, a massive portion of the traffic on the internet is involved by YouTube, was caused by YouTube, because people, they want to see videos on every mobile device they have, on their laptop, desktop, and pretty much that makes this kind of horrible. At the time of this, there were 5 billion videos there. You know, being watched every single day. Most people watch them for just a few seconds up to, you know, X number of minutes. But

the fact is, they generate a lot of traffic, got a lot of content, good times. So, with that in mind, and the fact that we know that we've been talking about stolen material on people's private sites, here is a search for audiobooks on YouTube. I came across a channel that has 23,000 books, audiobooks on it. There is no way that person paid for all of those. That channel is still up. So if there's an audiobook you're looking for, just arbitrarily put in YouTube, you know, audiobooks, and it will really help you out. A lot of old books, a lot of personally published things, but amazingly a lot of things that came directly off the torrent.

And I can say that pretty much unequivocally because the extensions on them you can find on torrent sites. Like, it's the book, but then it's like, you know, YTS, or, and I'm like, oh, that's a known individual that distributes. Yiffy, also a favorite. If you punch Yiffy into YouTube, you will find movies sometimes still in the theater. If there's a movie that comes out used between two and three weeks, it shows up on YouTube. And it's hilarious. Usually it's a camera rip, but eventually it may become the real thing. Law Abiding Citizen was a good example. Notice the date this was published. It was August 10, 2016. It had been up for a year when I looked at it. And I was like, well,

this can't be the whole movie. This is the whole movie. And if you've ever seen this movie, it starts out really disturbing. No PSA, no warning, nothing. YouTube wants to pretend to be television, but not so much because you don't abide by the same things. They subsequently took it down and it immediately came right back up. So then it was up for a month. Then it was up for three. I think now it's been up for six months. And the funniest part about this was, when I started looking for it, I'm going, oh, but YouTube also sells movies that you can watch. Oh, you could either pay to watch this one, or you could just watch

New Action Movie 2016, Law Abiding Citizen. Oh, yes they do. Yes, sir. No, you are correct. They will put a picture up and they'll put a disclaimer saying, YouTube is taking the video down. There is that. There is that. It is the length of time of the movie and it'll be, yes, an ad to say go to our site to get the full movie. But that was not the case with this. And that was weird to me because it kept continually being put back up. People will actually, and they do change the titles, it'll be like, you know, new exciting movie 2017, you know, or action film US. And then you have to kind of

go to it to find out what it is, but it's usually the entire film, and it doesn't come down. They also utilize some other techniques, such as they will do embedded images in an image, or embedded images in a video, where they have a pattern that's wavy, and then the movie's this big, but if you have it on a 60-inch TV, it's viewable. Um... They will do a static image and then have the box in it because if, again, violating your policies at work, and if you're interested in checking if people are doing that, people can put in terminology here to look for porn on YouTube and it is there. It'll be embedded in an

image or embedded in a video where it's the smaller thing in the video. It fools their algorithm. It allows it to be up. somehow uh and it shouldn't because you can put in all sorts of of terms that are not supposed to be on youtube and find them on youtube so like i said like sometimes they do obfuscate the name but other times it's the real thing uh babylon ad horrible movie but it's there um that frozen is just a clip dogma that's not a clip uh infamy that's not a clip those are actual movies

Getting back to the torrent, if you put in your torrenty commands, like S01E01 for Series 1, Episode 1, amazingly, shows show up on YouTube. Farscape, awesome show. But they do show up, and it's hilarious. Like, you're able to actually look for an entire series. One example I have is I may or may not have been looking for it. Anybody remember a show called Space 1999? the great show martin landau it's horrible now but you know it was great when you're a kid um i went looking for it i'm bouncing around i thought i had found it utilizing the other techniques and i attempted to get them they were avis like wasn't available to me i was like oh that's kind of interesting i i won't say

where they were actually hosted i will after i finish talking uh but i i was shocked that i could not get it like i saw things that i actually couldn't get i went oh these guys were actually doing it right where they protected their proprietary information from being taken But it's okay, because YouTube totally gave them up. Like, both series were available on YouTube. I was like, that's awesome. Every episode is right there. That was just kind of crazy. But as I said, YouTube, they're trying to be more like a television station, because they have YouTube Red, Kids, all of those stuff. But They're not. They're just not. You don't have any of the standard things that you

have in TV where someone's reviewing the content of the material and ensuring that that content is appropriate for the age group that it's for or for general audiences and the like. So they don't have this. And without that, that allows some really odd things to show up on YouTube. Yes, sir? Oh, no, I have not. And that does draw attention to it, as with anything, and then people actually go to those sites. And in this case, this is... Anybody ever heard of a Serbian film? Oh, I wish I was in the place that you are right now because it's horrible. It is a horrible movie. It was up in multiple languages. The thing that frightened me the most is this one's been up for three years. This one's up

for a year. The best description of it was given by someone where they did it as a cartoon. It was, you know, kid sees pornography, kid becomes jaded by this, kid kills best friend. I'm leaving out some parts. Kid kills best friend because of trauma. That's the gist of it. I can say that one good thing is unless you're logged in, when you attempt to get material off of YouTube, you encounter that, right? The content warning that prevents you from getting it. If it's content controlled, it pops up literally every time. The bad part is, there are sites that will allow you to get it even if you're not logged in. This is one example of SaveFromNet. You're like, oh, well, I'll just feed that URL to SaveFromNet,

and then, yeah, I want that at 720p. You've never logged into the site. Yeah, there are a couple. I picked these guys because they actually had an add-on, but then the add-on's kind of malicious, so you don't want to put it on something you care about. But it'll do the same thing, where it pulls the content down for you, or it'll pull just the audio or the video without the audio. Why would they do that? Oh, wait, dubbing. It's like they just have it set up where it's like, we're not saying that we condone what you're doing, but we understand, and check out our click revenue. So I wanted to see if it could actually

work. It actually does work. Like, okay. This is the cleanest scene I can show you from that movie, and that's sad, because It's a horrible, horrible movie. But, you know, what are you going to do? Research sometimes sucks. Face of Death, anybody? Remember? Yes, sir. Seriously, it is. Yes. Thank you, P.W. Crack. Oh, the beginning of Law Abiding Citizen is completely like PG in comparison to the things that happen in that movie. You're watching like, this doesn't even make any sense from a continuity perspective. Face of Death. Remember Face of Death? Every... episode, every version of Face of the Death is on YouTube. Including, my favorite, the 30th Anniversary Edition! Which in four years it's been up! It's like, uh, this has

content that I'm pretty sure violates your, you know, whole content thing, but yet you're allowing this to be up, lots of clicks, all that stuff. I was like, okay, this doesn't make any sense. Blu-ray 30th Anniversary Edition, yay, Face of the Death. Again, you don't have to actually log in. You can grab it. Kids, same thing. The movie's disturbing because the actors all look very young, even though they were all of age. But, uh, content warning. Bad side is, it comes up. You don't have to do anything. It, like, plays. Several versions of it, actually. I was like, uh, but there's, like, graphic, you know, sexual things going on. That violates your thing, but yet it's still here, YouTube? Really? And as you can see, this was uploaded in

2012. Yes, that's another movie that... Really? Oh. Yes, you can find Ken Park, Short Bus. Essentially, seven of the ten banned films are on YouTube right now. And those films are banned for a reason. And we just named three of them. Yeah, there are some films there where you're going, oh, also, awesome episodes of Black Dynamite, completely up there, completely cheesy. So you're going to need some add-ons. You can pull add-ons to just pull your MP3s. Again, we're the masters of our domain in our browser. So we can pull that add-on and say, yes, I would love to have just the audio out of YouTube.

Let's say you've gone to a site where you're like, oh, I like that, but I want to change the format of it. In the browser, you can put an add-on that actually do a media converter and convert the media from one format to another. If you don't like it in MP4, it can be AVI. Again, you've never loaded anything outside of the browser. It's just all in the browser. Does that sound like a browser? Sounds like an operating system to me, but you know. Again, somebody say? Yes, yes, yes. It's like an Emacs plugin. Like, this is not a browser, people, and it hasn't been in years. Because, Lord, I gave that WhatsApp with that

talk like six, seven years ago. Like, yeah, browsers are horrible. I wouldn't want my users to get their hands on them. But the fact is, that's your tool. That's the thing that you actually would use. And by using that, you've now been able to collect even more loot. So you've got your hands on even more loot because that's the friendly, acceptable, nice tool. thing that comes in your environment. And if you're not careful, your users can, your user population can use different add-ons and plug-ins to pull things onto your environment that you would never want. You know? Like, plain and simple. I don't have the example in here, but there is one where even Game

of Thrones, I'm sure there's nobody in here who watches Game of Thrones, they have the Game of Thrones episodes up, but they have them as, like, three episodes at a time. They'll have a movie title like The Fall of Miran, of Meereen. It's like, no, that's Game of Thrones. Because there's dragons, right there. Like, oh, they're totally killing off these individuals. It's Game of Thrones. They've put together episodes to make it a movie. So therefore, it's not the TV show. It's a movie, entitling what may or may not be happening. Uh-huh. So, like I've kind of alluded to, you can have problems in your home, on your network, if you're at work, legal team, in your HR department. These individuals can all get involved. How will they

get involved? Well, if you pull movies or content to your environment that is either acceptable or even unacceptable, you're probably going to end up talking to these guys. Probably a given, because you will have violated something that you may or may not know of, and it's totally legal, because you shouldn't have been using the environment to store things on the network that you shouldn't. And because you did that, that team will probably have to get involved too. And everybody loves to sit and talk to lawyers, I know. I do enjoy my time with them. And there's that. I just want to state that the resources that I did use were obviously the Google Hack for Pen Testing. Big

up to Johnny Long. He's awesome. And I've talked to him. There is one other thing I've left out that if you also want to have fun and not be yourself, there are ways of utilizing those same tactics to look at .onion sites. that people tend to be really careless with when they're not in the .onion environment. If you go looking for .onion things, you may find that people have placed their treasure trove of what they do there on a service web. So they think it's pretty safe, you know, and I'm talking about their usernames, passwords, and the like, and all of the .onion links that they traverse, three commands. really allow you to look at

stuff. But in this case, this is the file type command. You're looking for file type for .onion elements, and you're able to find text files that they have, where they have put their items in, and I would never go online as someone else, but I can confirm through someone else that it actually does work that people have their usernames and passwords for their access to hitting up Tor, and they may be careless with it. which is completely crazy to me because a ton of links that are all active. And in some cases, even their storage that's below, like, totally crazy to me. This is literally the combination of three talks that I gave. I gave a talk called YouTube, YouTube, at two YouTube, and the

WhatsApp with that, and a talk on current and technology like such as that. And how I got here again is, as some of you in the room may be, through Misspent Youth. So, That is pretty much what I want to say and I am... Oh, I'm actually a minute over. A minute under. Are there any questions? Yes, sir. I didn't say I downloaded them. I said I attempted to and then killed it. I'm responsible. Yes. The funny part is, absolutely none. I found no malware. The items I was looking at, I did play with... I did look at some people that had what should have been questionable content. But the funny thing that I found, because I'm

trying to cover the spectrum, I found people who had taken courses in the United States and gone back to their native countries and put them up on their university. Looking for .ISOs for CEH, looking for all sorts of classes. I'm like, that is, that's course material. So somebody physically took this and then they took the material and went back home and said, well, I will share this with the university. I'm like, okay, that's completely off the charts wrong. The other stuff is bad, but that is for, and I usually start this talk with, if you work in media or multimedia, I am so sorry for you. There is literally no way to stop your product

from being had because the product is the advertisement. You know, you want people to see it, and you make it available, and then someone gets it, or you don't pay an intern enough when he's transporting the master from one place to another, he stops at a place and gets a grand, and then he moves on with it because it takes virtually no time to copy that item. But no, I have not come across malware. Not on the surface sites that I've gone to. You sound as if I was playing with video, audio, correct. If I were looking for some other things, certain applications... or that you may have had to pay for. Yeah, that, totally,

of course, trialware. Yeah, that's what I meant. You will probably come across some things that are quite infected with other things, but the thing that baffled me the most was that, yeah, with nothing, even if you got rid of today, the technologies, torrents, or any of the file shares, Google got you. You can just go in with the Google techniques. And they've tried to tell you that you want to utilize modified versions of them, like they'll tell you to use the file type command, but you don't use the arguments in order to find things, and it'll find stuff. You want to use the old school file type, colon, variable, variable, what you're looking for. Because

that is very specific and it points to where it is. And by utilizing index, you never touch the target until you're ready to pull something off the target. You're just looking at the index. You can go to the Wayback Machine and also see things that people took down. I mean, it's kind of cool. Yes, sir?

Great question. No, they do not work the same as they do in the Google search engine. And I'll go as far as to say that I've even used like Yandex. I've gone to the Chinese browsers. I'm like these, these Google commands work best in their own environment. They do not work as well in other search engines, which is kind of odd, but it's just kind of the way it is. The Google browser, the Google search engine is just optimized for those commands. It does not work as well in others. Not that I'm aware of. Not that I'm aware of. Because when I did do this talk at Nova, I actually had other browsers up. And yeah, just the

search commands do not work as well. I'm not saying there's not other things you can do. And I didn't bring up the fact of, or I did. I talked about people's Google Drives. People leave their Google Drives completely unprotected. One thing I left out is that when those sites do go up, that tell you where something is, like for YouTube, one thing that makes it very difficult, obviously, for YouTube to find out if those sites are available is the fact that people, when they come down, they do seem to come right back up, and people have this amazing, obfuscating method of keeping people in the loop to know when, hey, the site was taken down, it's now this name.

And it's called Reddit. There's a subreddit, you may or may not want to write this down, there's a subreddit that actually tells you what's available on what type of media. There's one called, you know, Movies on Everything, it's a subreddit. There's others where it's Full Movies on YouTube. And every time a site gets taken down, the guy goes in there, sometimes they don't even change their name on the channel, they just change the channel. Change the site name, it's still me, you know, DarkBaron52, and here's the same material. And YouTube does nothing. and I don't understand it. But yes, if you look at the subreddit for movies on YouTube, full movies on YouTube, you will find if a site's gone down and the material's back up,

you can look for movies on anything, also a subreddit, and it'll cover Vimeo and all sorts of other technologies that are not YouTube, where people have placed stuff like the Super Bowl or other material that you just normally should not see. It's just frightening. Because again, if you work in multimedia, you want people to pay for your content. If someone sees your content or gets it, whether they record it with a webcam or whatever, they have it and they're able to just put it up in literally any format, including the darling that is YouTube. If there are no more questions, thank you and go play! It's helpful to take what I learned from my regular day-to-day consulting clients and

try to distill it and make it easier for people coming up to learn from their mistakes and go out and make new ones. And this has been inspired by essentially a lot of my work now is taking small shops that are maybe one or two funding rounds past a good idea, a garage or a dorm room to They're making money. I'm thinking of a shop, one of my shops is they're doing something really intimate with sales force that makes it better. And then my eyes glaze over because they're talking about sales. But their problem is, from my point of view, as a security strategist, as a, I used to be a pen tester and now I don't call myself one

anymore because pen testing is boring. You'll strike me dead for saying that, but a lot of my challenges now are how do you take that small shop, and that small shop up until now has been nailing small businesses. We have two accounting firms of 30 people in the Atlanta suburbs. We've got A&Z Window Company with sales all across the Northeast, and all of a sudden the sales rep calls up, Bank of America and says, "Holy shit, I got a demo next week." And it goes from you were selling to shops of 20, 30, 50 employees. And you go and you just, you are going to get to pitch your something as a service to Bank of America. What's the difference? Anyone

run into this problem yet? Sales rep, what are they, what's the next thing they're doing after they're going to pitch Bank of America? What's their next great concern?

Their concern is, do I get a 35-foot boat or a 40-foot boat, right? They're like, holy shit, I get 20% commission on that sale, and it's all going to go smooth until there's, oh, we just have one question about this. Can you fill out this little security questionnaire about how you are going to secure our valuable data? And at some point, you hear the music skip. What are you trying to do, turn the lights off? No.

I've been a professor long enough, I'm happy to do that. So this is, you're tasked with that job. How do we make this security questionnaire? Because think of every shop I've worked at before I do this work. Who filled out that vendor questionnaire? Wrong! The sales rep! Right? The guy who wants to make the sale. Yeah, sure, we do that thing. What's DLP? Okay, that is the voice of reason. Yeah, I just said yes for all this. No! I didn't know what one of these words meant, but we can do all this in a week. Actually, my favorite one doing one of these, I'm helping a client pitch a large state agency, and I hear a sales rep say, we can pass FedRAMP in two weeks. And

I'm like, you don't have policies. Like, you don't have any. No. And I just kick him out of the chair. And unfortunately, the problem was that there was a woman who was in charge of like, something in sales, like after sales support. So she's probably wondering, why am I trying to play footsie with her? I'm like, no, I'm trying to kick him. Can you kick him? Stop the sales rep from speaking. Instead, she's like, why are you doing that? And I actually just kind of nudged her chair into him, like, stop. And I had to apologize after, like, look, I had to make him stop saying things. And if it was that, I was going

to fake a seizure. Like, that was the next move I had. That's the sort of commitment I have for clients. Like, I will stop you from doing something you can't walk back. So, who am I? Why am I justified to be here ranting about this? I am, I think that's my title. Risk and Advisory Services for Leviathan Security Group out of Seattle. I, of course, you know, they don't let me go to Seattle that often because my Philadelphia accent and my sense of time doesn't really work well there. I'm a lawyer as well, just not your lawyer. My day job is a security consultant, but not your security consultant. That can be fixed. I have business cards and I sign retainers. I do the stuff I'm

going to rant about. How to rapidly mature your shop, how to spend as little money as possible and get as much bang for your security buck. And I used to grade the stuff I'm going to talk about. My gig before this was being an assessor. Not an auditor, because an auditor implies a third party can benefit from it. No, I was contracted by Blue Cross Blue Shield to go and torture their subcontractors and go, so you say you do this thing. Maybe you lose the contract. Maybe we give it to the other shop that does what you do because they don't suck as badly. So, who are you and why are you here? You work for an adequately staffed and funded security program

and want to know how the other half lives. Right? I'm assuming not, you know? I'm assuming if you are working for those, like, oh no, we're fully staffed. All our recs are closed. I have a fairly unlimited budget. And then you wake up and go, oh, you're right. I have nine open recs for a team of 12. I'm currently 18 to-dos behind. So... Yeah, no one is adequately funded, no one has the staff they need. You just got promoted into security management. Hey, remember when you were like senior nerd and then your boss left and you're like, and the other senior nerd was like, "One, two, three, not it." "Hey, congratulations, you're a manager." "What?" "No, I'm pen testing." "Too late." Or maybe you

got suckered into it. It's going to be a raid. You are going to get the appreciation that you've always wanted. from vendors who want to sell you things. "Hi, this is Bill from Rapid7. Do you know what I do? Please stop calling." So the old managers walked away and you got promoted. That's actually one of my clients. Their security manager literally said, "I can't do this." Not like, "Jump to another shop." Just like, "I think he's delivering pizza or an Uber." Just like, saw the writing on the walls like, "I've signed too many. I've let too much shit slide." I gotta go. R. Anyone ever been this guy or gal? You're the first security hire? So who's the CISO?

Says in this org sheet, I'm the CISO and the firewall manager. IR response. Huh. Oh boy. And you look at the policy and it's like, the incident response team shall be headed by the CISO. And the incident response team shall be the CISO. and anyone else with a CISO can strong arm into "Help!" Right? So you're the first security hire at that shop. So you have to rapidly mature your IT operations and your other operations really, really quickly to be able to maintain existing client relationships and get new ones. This is sales driven, which may scare some of you. If you think about it though the right way, What's the advantage of a sales driven model for security? You've got allies. I'm

thinking of a traditional company where you have sales people. These are people who say nice things in order to convince other people to do what they want. So if you can get sales reps, people whose job it is to manipulate, convince in a favorable manner, and you get a sales rep who recognizes that the one obstacle Between him and that 35-foot Chris Craft is a thing called an IPS, IP urine, I don't know. Do we need one of these things? I will sit outside the CFO's office and get that blinky box thing you want because I want to sell Bank of America. All of a sudden, you don't have to do it. You can go and make those people your allies. Or you can go and

find compliance with somebody else. Because it's just you, you have no power. If you are their only security hire, you have a portfolio of nothing. But if you can go and make some allies, you might be able to win some of these. So, why do we care about security? Because the law makes us. These are just letters and numbers and bullshit. HIPAA requires that we. GDPR requires that we. 23 NYC RR is the new New York cybersecurity law. MGL 93H is the Massachusetts one. I could go into these, but that's often a driver, right? The law requires that we do something about security because these breaches are getting crazy. Yeah, that drives a lot because your customers require it. What's a scarier

phone call at 4.54 on a Friday? Your biggest customer complaining at you, a regulator. Odds are it's a customer. Or of course, why do we care about security? Because your organization cares about security. I have heard that line so many times and my answer is usually, yeah. I still love the Equifax CEO when he says, the guy whose job it was to make sure those systems were patched didn't do it and oh well. I'm like, you know what, you probably have more quality control over the press release. If someone used the wrong Pantone color, there was someone else to QC that thing. You used the wrong font. Whoa, you've got to stop and pull that back. Yet, did you QC? Because I'm

imagining the person whose job was to make sure that Apache Struts was updated wasn't like, oh, I'm just a complete moron. It's like, yeah, here's my to-do list. This 9,000 line spreadsheet. Apache Struts is 2,300 deep in there. I was going to get to that, but I was still trying to clear February. This is open bullshit. The few shops that actually care about security, maybe there are some. Maybe you even work for one. Maybe you still believe it. But really, there's a legal fight over it. Are your customers required? These are the two that might actually show up and make you do a thing. This is a great question. I use this to stump consultants who try to work for our shop when I interview them. What's the

appropriate expenditure for security? You had to answer that in one bullet point. What would you say? Whatever it takes, up to and including the budget we already have. I've often liked the 10% of your IT spent. Where'd you pull that one from? I could show you, but it would be rude. Usually the answer is, what do our customers demand? What are our peers doing? But really, we know what the actual question is. What's the least we can get away with? What's the least I can spend and still have an appropriate, and I should put that in air quotes, an appropriate level of security? What's the least we can get away with? Hopefully by the end of this we can figure out what we can

do in the least amount that still is good enough, that still meets your actual requirements. Use this phrase, actual requirements, not what the law requires or your customers ask for. Because there's a difference between Bank of America requests that you do this thing versus Bank of America will pull the business if you don't. One is a request, the other is a hard demand. Filter out the things you actually have to do and concentrate on those. Get wins where you can outside of that. The core hard kernel of your security program should be what can I do that actually is a hard requirement or I lose business.

Let's start defining terms because I'm speaking of security in this amorphous, you know, it's a gray goo. So, I'll have two securities please. In case you're unfamiliar, controls are individual measures to detect, prevent, or mitigate security risks. Pen testing is a control. A firewall is a control. A request that your employees show up to work sober is a control. Some of these are enforceable, but, so you've got, you know, physical. You have an eight foot high fence. with concertina wire around your secure facility. Procedural. How often do you do a vulnerability scan with Qualys? Once every, I feel like it, right? Some kind of procedure that you task with somebody to say, you got to go do the thing. Access rights review.

Technical. Do you actually have a blinky box to do the thing? And many of you are familiar, but just so we're talking about things, compliance. These are the It's written on policy that you do these things. If you don't, we may fire you. Or we may dock you pay. Or we may make you sing "I'm a Little Teapot" in the break room. Some kind of punishment to make you do the things you're supposed to do. Don't give your passwords to everybody. Don't put your passwords on post-it notes. Don't have a shared login for everyone in the HR department. Those are controls. Frameworks are big collections of controls that we all agree to. Such as ISO, NIST 853, COBIT. I

mean, NIST is already kind of for masochists, unless you actually have to. Usually when it's like, what should we pick? I'm like, ISO is the easiest to fake because you've got scope. You can scope down ISO to almost nothing. Like PCI, where you know, PCI certify

this box but nothing in it and therefore my SACD is compliant. You can play games with scope, you can play games with these things, but you've got basically your collection of controls, your stuff you promised to do that an outsider will assume if you say we have an ISO certified or ISO aligned policy kit, it means that you're probably doing a bunch of these things. So those are the two terms I'll use again. So many controls, so little time, including my proofreading, my doc. So implementing controls requires money, effort, and burden. Using this as three different things. Money, I've got to go spend money to go buy the thing. I've got to buy the blinky box. I've got to buy the antivirus subscription. I've got

to buy the something as a service. I've got to go pay Splunk some money or more money. Effort. I actually have to have someone who either works for me or I've hired at a high rate go do the thing, turn the crank, to design, configure, maintain, and use that control to do the thing. I have pen tests. Well, that's money. And then effort to go and remediate those findings, track them down and make sure they happen. And then there's burden. Policy that says we have a five-minute lockout on all PC screens. Does that cost money? You can set that with GPO in a couple minutes. Is it effort? Minimal. Will it result in complaints? Yes. Will

you get yelled at when you have things like, when you get those really stringent things like, we're going to air gap. Next thing you know, people on the Bank of America team have two PCs. One is for Bank of America business only. The rest is for other. Because you put up some burdensome control to satisfy a need. I'm going to use that burden because that's often what gets you the most kickback. Because I spend money, that gets spent, and the only person who's really worried is accounting, but you spent it. It no longer rears its ugly face. Effort, the only person who's really complaining about that is the man or woman you said, go validate

that. That's what you're doing today. Go check the firewall rules. Make sure they're compliant with this standard. Once a month. Yes, it's that time of the month. Go do it. Burden. Burden is the one that keeps coming back and the regular people, your end users, your customers are going to go, I have to do this. Why do I have to change my password? Because Bank of America requires it. So how do you select the controls, right? Because ISO or COBIT or NIST is going to have a set of controls that you should plunk in. HIPAA and GLBA and those may also suggest controls. Do you have to take all of them? No. No. You pick the ones you've got to do. Do the

things that are mandatory and here's, this is the biggest way to save money and do this on a budget is figure out what controls you actually have to The problem is that this control set, this actual real hard controls you've got to do, is specific to your organization in a period of time. That same organization two years later may have a different control set. Because you signed contracts with new clients who made you do things. Because you changed business. Because regulations changed. You got into this business. You exited this business. So you are figuring this out as you go. If you are... new to this, or your company is new to this, you're figuring this out. And

here's the fun trick: Basic rule of consulting is: I don't have to know everything. I just have to be half an hour ahead of my clients. You, as head of security, or you as a new security manager, do not know how to do everything. You don't have to know everything. You just have to know how to be ahead of your customers and your end users before they ask. You will come across as a genius. There is nothing wrong with pulling up the Wikipedia page about the thing during a con call. However, one point, if you've got like, if you're a fetishist about keyboards, Cherry reds or cherry browns are really really noisy. So when you hear the "So Alex, can we do that thing?" Yes, yes we can!

So I have been asked to get a quieter keyboard for con calls and I have winged this in such a way that I've made my clients happy. It's okay to wing it as long as you validate that you did it right later when you're not under crunch time. So how do you select the controls? How extensive should they be? We can do the old-fashioned way: cost-benefit-risk-base. Jet engine plus peanut butter equals risk, I think is what my old boss used to say, and I never listened to him because he's Canadian. Thanks James. The idea is this: the probability of an event times the impact of that event without controls, if the cost of that is greater than the cost of the control plus the probability of the

event still happening and the impact of the event with that control, you put the control in place. If they're equal to wash, if the cost of the control plus the probability of the, plus the potential impact of the control with it is higher than the control, than the probability without the control, you don't do it. You accept the risk. That's great, right? Like, remember when you took Calc 1 and Calc 2 and you learned how to optimize? And you're like, "Wow, I can solve problems," until you realize a problem with this. Implement the control. If the benefit is great, the cost of control is less than the unmitigated probability, it's really hard to quantify. What's the probability that

you're going to get breached this year? 100%. What's the probability? How much does that probability go down if you buy something really new and shiny from Palo Alto this year? It depends. If you are a Palo Alto sales rep, you'll say, our research shows that That'll, oh, nice. Actually, better yet, knowing if they're proper enterprise sales rep, 73.2. That added, that added little, oh. So the only thing you can actually guarantee, you have a good idea to guess, is the cost, right? Like, this would be great if you could quantify all this rest. So you can make some guesses, but... At the end of the day you're like, "I don't know." I think I feel better knowing that there's

a 4U $100,000 box from Palo Alto that does next-gen threat hunting, threat-centric something. I don't know, it's awesome. You hear that thing when it fires up, it's noisier than the freaking Sun E420 that's underneath of it. I don't know what that does, but it's still there. So, it's a great idea. It only really works when you absolutely have a, I know that the risk is dramatically reduced and the impact is dramatically reduced and the cost ain't that high. The rest is your winging it. So, as nice as this model is, it doesn't actually work. The second model is, I don't want that to happen. How many times do you get a regular line manager, a non-technical manager, waving around one

of those magazines, oftentimes it's the award-winning airline magazine. This issue is about cyber security. You're like, oh God, what the hell is this guy going to be on about? We need a flame wall. Huh? What? Because invariably they're calling you while waiting for their luggage. We have a flame wall, right? You're like, Firewall? No, no, Flan-- We have one of those. Yeah, yeah, sure. Because that breach thing, that sounds horrible. Yeah, yeah, it is. That's why I'm here. Why am I taking your call? You don't want that thing to happen. And I want you to guarantee that thing won't happen. You're like, sure. Yeah. I don't want that thing to happen again. Ring. Hi. Who authorized

to put the entire customer list on Pastebin? My favorite is, who authorized to put all of our prospectuses on Scribd? You're in trouble, because you're right. They all are unscribed. What are they doing there? So I don't want that to happen again. I want you to guarantee that that horrible thing that made us all scramble won't happen again, right? So you're lurching. God knows we do airport security like that. Oh, people's shoes may explode, so you have to take off your shoes. And I'm waiting for some entrepreneurial ISIS or Al-Qaeda member to go like, suppository bomb. Just like, yep, yep. Gonna make you all do some extra special time in line. You want to prevent that horrible thing from happening. And

I call this the Senior Management Protection Program. I told the security guy to do that thing. Now you're in this, well, did you give me the budget to do that thing? Why don't you take out your existing budget? Like, fuck that. So this is the, I told you to fix that, you didn't, it's on you. So, more ways of controlling selection controls. Some controls are soon because they're low cost and high estimated impact. We use encryption, you know, I often have this one, full disk encryption of laptops that may hold sensitive data. It's a no-brainer now, right? Because pretty much every OS you can get, unless you're that poor bastard who's still running some kind of like Plan 9 running on brain fuck,

If you've got some relatively modern OS you're supporting, full disk encryption, it's an easy thing to do. Just click the box, done. One of the many check boxes I've got fixed. Expensive, low impact, high expense, you're like, I'm not doing that. Kind of knee-jerk, of course we do that thing. Pick those controls. What we call the laugh test. If I heard that another shop wasn't doing this thing, I'd freak. I'd laugh at them. The problem is that... I no longer laugh because many of my clients don't do the primitive things. Why is Telnet still open? Oh, it's a useful way to administer Unix boxes. Dude, I like the 90s too, but they're over now. SSH has been stable. I don't know about that. I no

longer laugh. I cry. So I should replace this with, does it make me drink more? But, right, so you've got some controls because you do them. You do them, why? You've already got a budget for it. Why are we replacing the firewalls? Because we have money in the budget and we used to have firewalls, so it's easy to say, I am replacing the old ones with the new ones. Is that low cost, high estimate impact? No, but it's easy. The regulation requires it. This is a great way to cut through some bullshit, except I know way too many beardy sysadmins who will, my favorite is, uh, a friend of mine who literally stood up in front of a manager and

said, "I think you're violating HIPAA because you are sending protected health information in clear text and that violates HIPAA." Now of course I'm going to ask two questions. One, was HIPAA relevant? He was doing a pen test and he was sending a pen test report to the client or was asked to send a pen test report to the client. How many pen test reports have PHI in them? Unless you've really screwed something up Should be none. Right? You may allege, like, here is data and you exit out. But he was convinced that the pen test report included PHI. The second one was, he said, HIPAA requires encryption. Anyone want to argue that you have to encrypt

PHI under HIPAA? Good. You don't. Stressable. So maybe it doesn't. Maybe the regulation doesn't require it, but the number of times people want to get into a regulatory fight over this is rare. I do it because... a lawyer. But that's kind of neither here nor there. Let your customers tell you what they want you to do. You've got your base controls because you've already had them or you want to just not be laughed at by your compatriots. But then your customers will tell you what they want and that's a half decent way of figuring out what is considered normal for your industry. How can you tell? Vendor requirements and contracts and vendor assessments. The number of contracts I've said that have, you've got these controls. The ones that scare

me are the ones that have a blank thing. You will provide the following contracts. Blank. Don't let the sales rep write that. Don't, because they'll add things because they want to make the sale happen. Now you just agreed to do all those things. A week ago, you didn't know what they were. We got one of those DLP things, right? Don't. Would you like to try to install one today? So here is a body of knowledge that you can steal from. If you are new to the industry, you can go get them. Make some friends in your industry. There is a meetup for information security people in health care or financial service or whatnot. Get to know these people on a social basis. If they like you, and this

is one of those reciprocal things, what you do is you take the ones you're getting hit with File off the serial numbers so you have a body of knowledge about what everyone's expecting. It's a really quick way. It doesn't violate an NDA if you're just taking the information out of it, right? These are the controls. I'm not naming the client. I'm naming, here are the ones I see that all the ones ask for. You want the common ones. You want the ones that all your customers are demanding. That body of knowledge is valuable. Share. Don't be a dick. Share that information, right? Because it also, in a way, from the perspective of the vendor, the vendor would like to not have to fight you tooth and nail to put

in some controls. They would like you to have a pretty good idea of what controls they're expecting. So use that as your first body of knowledge. So talk about that. So you've got this big pile of controls, right? ISO plus you read the last eight vendor assessments and you went through and asked legal to dump all the appendices from the contracts and now you've got this whole selection. You've got 125 different controls to select from. How do you pare that down? Right? Because now like holy shit, I have 122 controls. My own kind of seat-of-the-pants risk assessment says we might have 20 in place, partially in places. How do I go from that big body to a smaller one? How do I cut it out? How do I

cut out ones that I don't think I need until you make me? And so I'm going to give you a couple guidelines and a dirty trick. So you have three requirements that you think you need. First one, only gets checked after an incident. It's checked. Did you have that in place? And the only time we're going to ask is post-breach. Number one. Number two, we ask that you, prior to the contract, do it, but you can self-certify. Like, say, PCI SACD. I say I do these things. Requirement three gets checked by people who can cost you money. Which requirement do you break? Because this and this, like, the chances that you get breached, it's less than 100%. And then again, even if it's like,

hey, You should have had this in place. Sometimes you're then arguing, would that have stopped it? The number of times they've had arguments about, DLP would have prevented that. DLP would have prevented that, except they walked out with a hard drive. I've not seen DLP stop physical theft. And I really want some sales rep from Symantec to go, ours would. I'm like, okay, awesome. "Will you sign that in a contract? I would love to see that happen." So, you actually go and say, "This is the requirement that I care about. Which of the ones that the vendor is going to check on, I want to make the vendor assessor happy." Not saying you don't do the other two, but the other two

are slightly lower importance. So, easily, cheaply verified things get checked even if they're lower value. I'm going to tell a story, and I love telling this story because it's one of those I have a rule that says many times I will see something that seems just asinine and stupid, but I'll ask why. Because sometimes you find out there's a really good reason for it. The story I'm going to tell is about a rock band called Van Halen. Van Halen had a rule that said if you had brown M&Ms in a bowl in the dressing room, they could walk from the show and you would have to pass. The reason why was they were one of the first, they kind of built the idea of a giant arena show.

It had massive loads of power, weight on the theatrics, and Essentially, people would just sign the contract riders. And these are like an engineering degree, engineering diagrams in these contract riders saying, "We're going to show up and we need all the power, all the mounts, and if the last thing you had here is a folk show, no. We need like real infrastructure. And we want to make sure that you read the goddamn contract. You read these requirements and said, 'Yes, we checked.'" We have the power, we can support the weight, we can move things in or out. However, to validate that you've got power every 18 inches 60 feet off the stage is hard. You can, however, force them to identify something that you can check. So there

is a bias towards things that you can easily show to say, we are doing that thing. Here is proof. If there's something you can validate and you aren't doing and you say you're doing it, it means you lied about something that was easy to check. That makes me believe you less when you say the hard to check thing we're doing too. So, what are the easily? So, like SOC 2. I hate them because I've gotten people through them and I have, well, because they always send like second year accountants who don't know what any of the words mean. And I've had, I've actually during a SOC 2, someone said, you don't have a policy on incident response. I'm like, have you read my document called Incident

Response Policy? And like, well, that doesn't discuss IR. You know what IR is, right? Well, IR is different than incident response. So like, find and replace, there's a whole IR section that was copied and pasted from the incident response. Like, see? And the fact that, you know, what security tool do you use the most? Microsoft Word, find and replace. Got a client, SOC 2. SOC 2 is easy to like vomit up and go, there, we do those things. Does it make you more secure? No, because it's a bundle of lies that at a point in time was not obvious to an accountant who's had two years experience. That's easy to check, right? I ask for, you know, if I'm an outside vendor and I

say, I want your last three years of pen tests. How many of you who are doing those pen tests will go, I'm not showing that auditor today. And why am I asking? If I'm asking for the last three years, what do I want to see? That's one, but thank you. Are they the same report? Because as a pen tester, the saddest moment of my time was realizing, reviewing the last pen test report, I could save the 12 hours I normally book to write a pen test report, change date. It was... bone for bone, unfixed. So I actually wrote in the bottom, why am I doing this? QC didn't catch it. And during the readout, I would like

to skip to section 4.5. Why am I here? I've identified a new finding that says, you are not remediating the problems. And this bothers me. Alex, you're doing pen testing. Why are you lecturing them? Because why? Why are we here? Now I'm in strategy where I get to go and say, the more valuable findings you didn't fix it last time. The pen test is useless. Why spend grunt time for someone to pound your systems if you're not going to do anything about it? And I get when you have the informational ones because there's nothing like, I can determine the version of the OS. Wow, that would be a big guess. It doesn't have any of the Microsoft services, but

has SSH. I'm going to guess it may be a Unix box. Just say it. That's okay. That doesn't bother me. But I can get RCE, and you didn't fix that problem. Three years running, that shows that trend. That's the easily verified thing. I want to show if I'm worried about a vendor, I want to see that you fixed the problems. So I'm going to say some crazier things. We all say this, right? Compliance, bang, equals security. Usually that's from people who are so well-funded that they can actually have a threat hunting team. I get called about that. Would you like to be on our threat hunting team? I don't know what that really is. That sounds like it should have a trademark by SecureWorks on it. That's

great, but really, I want to... Most of my shop's like, could you patch the shit from last year? I'm the bone hunting team. Did you patch it? Are there systems that have been out of compliance for 90 days? And then some are like, we've had systems out of patch for 900 days. I'm like, how are you allowed to do that? Like, oh, they support our core infrastructure. Stop. Please stop talking when you're just making this worse. So, yes, compliance, bang it, we'll have security. But one's easier to show. From a vendor assessment point of view, from a your client has... four hours to figure out whether or not to jump in front of the freight train that is the business deal. If you're a shop big enough, think

of it from the vendor assessor point of view. Your job is to write up a spreadsheet that asks a bunch of questions and all you do is read lies all day. Your job is to throw up a hand and say, "I really think this is a bad idea." and you have limited political capital, you can't stop that many deals. So what you want to do is show enough that means that you are not the biggest asshole we've met that week. That's all you have to do to make the sale. Someone else has openly lied that week and like, I'm going to go put a hurting on them. You, you're just a garden variety schmuck. I will let you pass. I might put a requirement or

two in, but you're going to make the sale. So, but proving it, oftentimes proving compliance as far as improving security. Because every time I see, I get a client who's like, they're demanding a right to audit, like a right to pen test. I'm like, I've never seen one exercise. I've been doing this for way too many years. But they'll ask to see all your paper, because that's a lot easier. So compliance isn't security, but from a vendor perspective, it's the same. It is possible to over-control. It is possible to over-spend on a control, because... That Palo Alto sales guy was really nice and we went to a steak dinner. I think the bill was like $800. It was worth it. And you're like, "Oh, sure!" And they,

you know, like you wake up the next morning and there's a thank you note from the Palo Alto sales rep like, "What did I sign?" That's really, that's beautiful and blinky box. You're like, "That just blew half my security budget. I overspent. I over controlled." Problem is, is that Checking a box really hard doesn't check the surrounding boxes. Checkbox security is bad. Yeah, but first you check all the boxes, right? And then when you're done with that, you can now go back to the castle. I think of the checkbox security as you go walk around the castle and go, there's a wall here, the stones are fine. And you're looking for gates open. Gates missing. Gates on fire. And when you jog around and come back, you're like, now

we can talk about higher end stuff. And my check boxes, all the check boxes are filled. I can now really think about moat design or tower configuration, stuff like that. But odds are, you're scuttling around checking boxes because that's all you have time to do. You've got open recs for five people to help you with that. But guess what? Checking the boxes is probably the best bang for your buck until you can do the other stuff. I'm thinking of every big breach falls back to one of those, you did something, you failed to do something that is relatively programmatically easy to do. Target, you got alerts that you didn't do anything about. Equifax, you didn't patch a known vuln for 90 days. for internet

facing systems. And clearly there are other controls that failed too, that that was enough to have it all happen. But while yes, compliance is in security, check all your boxes first. It's a part of this nutritious balanced breakfast. So good controls are the ones that people are actually asking for. This is customer based, but you're looking for them to go and say, this is what we really care about. The other stuff is stuff that you have to be independently convinced. You have no allies for that other control that you want but ain't no one asking for. And maybe you can justify it. Maybe you think, no, this blinky box or this policy or this thing

will pay off. But you're the only person in the room advocating for that. It's really easy to say, 60% of our customers require this control. It means that I can go to 6% of the sales reps and say, Back me up on this. Make that sale easier. And at that point, that sales rep will now pay attention to you because you just said something to them. You said, I want to help you buy a boat. All of a sudden, you went from the weird twitchy guy in the break room to the guy who can help me get a boat. All of a sudden, you are now relevant to me. You will stop doing the other

sales rep-y things and now pay attention to you for a minute. Don't freak him out. Explain in small words. You've got three bullet points and two of them are oxygen is necessary for life, I like money. You've got a third bullet point where you can explain why you need that control. So, proving your assertions. If you say you're doing a thing, how do you prove it? People ask for more than a self-certification. Many of my clients run into this problem. They're like, hey, we can promise everything because we self-certify. We have somehow found that a SACD... a line that says we have an ISO 27001-2 aligned security program, but we have not yet certified it. We

have an internal pen test team, so we don't use an external vendor. We do our own vulnerability scanning. We have not used a QSA because we're under the size, something like that. So sometimes you'll get a vendor who will demand, prove it. Show me that you do the thing. So, this makes you have a fallback position when you know that the vendor is going to ask for more, search the things you should be doing, assuming you are, and you can prove that you're doing. This is where I talk about policy and standards and artifacts of compliance and most people's eyes glaze over, except that this can be a magical thing where you can shut up a vendor assessor tracks is

you vomit documentation on them, they go away. They no longer bother because they figure anyone who's willing to drop a bunch of documentation that shows they're doing, that they're actually looking, they're going and checking the box to make sure the castle hasn't fallen over, odds are, just going back to looking at the world view of the vendor assessor, their job is to make sure that they don't get fired. What can get them fired? A breach from a vendor that they said was okay. Or, you blew all your political capital trying to stop a deal from happening. Those are the two extremes of their lives. So, if you can give them enough warm fuzzy feelings that they're not worried about you, you get a pass from them. You don't

have to be perfect, you just have to be good enough. And showing documentation is a good way to do that. So, artifact. Being able to drop a risk register of all the issues. These are all the things that bug me from a security perspective. Big, ugly Excel spreadsheet. Unless you're so big that you're using Archer or RSAM, and then if you're that big, I feel for you. I've tried to fight with Archer and RSAM and Risk Fabric and think that really they're all basically an Excel spreadsheet with an uglier front. All the stuff you're worried about that you have to fix goes in one ugly spreadsheet or a database or something that says... I am worried about those things. I will check them

off when I can. So at any given time I can say, "Here's what's still on my plate that I have to worry about." Because it shows that even if I have a horrible pen test, all the things are broken, but I have a remediation plan, and you can show better that the last time you had a horrible pen test and you remediated the things, it means that your promises that say, I will fix that before your company's data flows through me. All of a sudden, a little bit of process here buys you a lot of credibility for your security program. This is cheap. This is some effort on your behalf. It's not a lot of

burden on your end users. You can suffer through this. There may be times where you're sucking this into... If you have internal audit or accounting or regulatory groups, you may want to merge the risk registers to say that way you can compare and contrast their risks, the stuff they're afraid of. But you should be able to vomit out in one document, here's all the stuff that I'm worried about. I haven't yet remediated, but I'm planning to. These are the ones that we decided we're not remediating. We're accepting the risk. It builds good faith. It shows that you're doing the things you say you're doing. Because there's nothing like dropping a really nice policy kit that's perfectly compliant with ISO 27001 and 2,

and then when you ask, so, do you do any of this? There's that awkward silence, because you're not. When I did assessments, it would be something like, so, When was the last time you actually tested your IR plan? Like, oh, we've never done that. But your policy says you test it annually. Like, oh, it does, doesn't it? So if you show, had an incident, this is what we did. Even if the incident is Bob at the loading dock got a piece of malware on the combination malware collector and FedEx box. Did you do the thing? This proves that you do the thing. What did you do? ran malware bytes. Shows you do the thing. Shows the things that you can show easily you're doing, so

therefore the hard things you're probably doing too. Track exceptions, right? There's nothing like when you get the answer that says, are all your systems running supported operating systems? And people check yes. And invariably there's a kiosk in the reception area that's running XP. Like, But you said you were. Is this the XP extended? They're like, no. You shouldn't leave the proof that you're not doing the thing in front of the auditor. It's okay to have that XP box because you say, I do have an XP box that runs the incredibly important task of telling you what office any individual works in. We do have systems that don't meet our policy, but we accepted the risk.

XP box in a kiosk in St. Louis and isn't plugged into anything else. But it shows that I'm not lying to you. Because I pointed that out, that means that I'm less worried about a cluster of Win 2000 boxes that haven't been patched since the George Bush administration running something core. Because you called out the thing, said there are places where I'm not compliant. I think it's okay. You're showing that you're working what you say you're working. The cheap control. Control substitution. So, a lot of times you have a technical control that's really, really hard. I'm trying to think, there was one, I tried to get a client through this, and they wanted to show that you could not

access the jump box from an outside country. And we said, we can whitelist and blacklist IPs, and like, what if they use a proxy? Like, What, you want us to like decap something and figure it out? I'm like, fine. You know what? The only people coming into the jump box are the remote support team, half of which are in Canada. So technically, half of our staff is violating your requested policy of only U.S.-based people can log into our boxes. So I couldn't put in a technical control. I instead just put a policy control in that says, You must click "I hate curling and socialized medicine" to log in. Donald Trump is better looking than Justin Trudeau. Yes. And therefore, by

policy, I bar the Canadians from logging in. And it's a stupid control, it's a stupid request. But it was a vendor, it was a client that had a lot of money and were willing to pay what our client wanted for this service. So I'm like, we'll put in a compliance control that's a hundredth the cost. It's a slight burden, but I can't come up with a technical control to rewrite your application to make this happen. So, also, I can replace controls by SLAs, right? I have, say you get someone who's got very, very rigorous physical security controls. Quickest way to do that, I'm not building out a data center for your nitpicky performance of physical security.

I'm going to shove it in AWS. Those controls are an SLA that say, Amazon has these physical controls in space, done. I don't have to build a secure Kira Kolo. I don't have to do all that. Amazon's doing that thing. What, you're not going to trust Amazon to have decent physical security?

I haven't seen any Amazon racks at 401 North Broad in Philly, so I'm assuming they at least go to decent colos. I like playing the homeless vagrant or sysadmin on a two-day rollout at 401 North Broad. And we figured out that it depends on what boxes the person's sleeping on. I've done that. I've actually had to nudge someone, like, dude, you can't sleep in front of my rack. And they muttered something, and I'm like, let's see, they smell bad, they're ill-kept, be a sysadmin.

Okay, but nonetheless, I can replace it. You requested a control that I can't meet. Maybe I can handle this with an SLA somewhere. Because I'm doing this bit for bit. I'm not doing this for all of my services. I'm doing this for you, one vendor, or one client, because it's cheaper. So stove piping. This is more advanced. I'm doing stuff just for one control. They have a stupid, overburdened thing because they want to do something perverse with your service. You've got essentially like a box knockoff and all of a sudden a large health insurer wants to use your thing. You know, like, our tool is not built for this. We don't have all the compliance controls in. I would like to put them in, but that would piss off

all my other customers. Maybe I can cheat. and just create a second segregated system for them. I use the same software, I use the same colo, I mean not the colo, same configs and all that, but there's a different landing page that you guys go to that I can put all the controls you want in. So, say they have this list of unreasonable demands, and you're like, wait a minute, this is just for you. Sometimes you have two choices when you get a large client, that once something is otherwise like that's going to be expensive that's actually going to cost money and you have to now go from a technical decision to a business decision

I kind of went through this analysis with one of my clients they wanted essentially we had to rebuild their app and we like the cost to remediate all these requirements that no other clients asked for comes out to about sixty thousand dollars How much are we going to be making from that client? And the sales reps like a hundred. So $40,000 is a revenue just to put in, implement these controls. Odds are it may not be worth it. However, just say during the call when you're talking to the security people, we can do that, but for you, those additional controls, we're going to offer you a special deal. Instead of $100,000 to $140,000, we're going to charge you more to get those controls you actually asked for.

And sometimes you'll get a, we're fine with that, we'll pay you more money, or you may decide how many other people We'll also want these controls. We lay that off. But you turned a technical set of technical issues into a business decision. Do you as a company decide to shoulder these new costs? Do you spread the cost back to the customer? Be open and obvious about it. Like you've got all nitpicky and want all these controls. This is what it's going to cost. Odds are by this point, you've gotten deep in by the time you're doing the vendor assessment. All the other shops that were pitching have fallen out.

So you might be able to get away with that. We've negotiated that. Or you may actually have to decline the work. Like one was essentially a software testing app. A large hospital chain wanted to use doing live testing on live data. So you're all HIPAA compliant here. I'm like, no. There's literally like we crowdsource testing. So anyone can log in. So that means like 110,000 people can get PHI from the systems they're testing. No, we cannot make that HIPAA compliant because HIPAA requiring the, you limit data to the people who need to see it. A bunch of basically click farm level QA testers in Chandigarh aren't going to, that's not limited to the people who need to see it. We cannot become HIPAA compliant

for your needs. It's going to be too expensive to modify our service just for you. We can't take the work because we don't think we're going to get any more work from similarly situated customers. Now, of course, this isn't your decision. This is the decision you help the business people make. But you go and very honestly say, it's going to cost more to do this thing. So if you fail to convince a customer, make it a lesson. Because we didn't do that thing, it cost us this. Add the requirement to the next sprint or quarter if you think you can. It was one of those we could do it if we knew we had to, had time. That's

how you can kind of build a body of knowledge for the controls you need and get some support to build it. Any questions? As in...

Okay, I think I see what you're getting at. Usually the self-certification is something that's by an open body, like PCI allows you to, if you're below a certain size, number of transactions, you can describe your controls and without having an outsider test them, you say we do all these things. That's a self-certification. I have seen, you know, there are open certifications that don't require a third party to validate what you have done. Like, I can validate, I can say I meet HIPAA. for a client. I as their virtual CISO say they meet all the requirements of HIPAA and if you trust me that's great. That's a self-certification. A third party certification would be where a third party comes in and says we who have

no skin in the game other than we would like to not have a bad commercial name have reviewed this stuff and we think it meets those requirements. Like I trust is an example. PCI after a QSA, if you're a tier one provider, where you say a third party came in and looked at this and did this. I don't think you're going to have questions of illegality. I think you'll just have questions of, do I buy your self-certification, or do I make you, if I'm the prospective client, do I, if I don't trust you, do I go make you get a certification? Do I make you get a third party to say, I believe that their certification is

valid? Oh, okay.

So I create a certification test for my own software and so like ISC to get ahead. Until you make it a requirement, like I'm thinking SysP is required for some, if you are in DOD, you can make it a requirement. There would be a question of is it unfair? Is it some kind of unfair competition? I haven't seen anything that's... I haven't seen the case yet, because I'm thinking if you do... If it was specific to a vendor, say, for example, I require that you've got a vendor-specific certification to work on the device, then you get into where that is. There's sort of like a right to repair and the like. But I don't know where you would get a... potentially a legal claim against a certifying body

for creating a certification.

I don't know. My TLDR answer is I'm not sure. I felt the certification. Yes. There's a better rant about this by Duncan Minutz at GERCON. I think this is his talk two or three years ago about certifications. And it's a great rant about ISC and saying the SISP is a marketing exercise by this point. I don't know. I have somehow managed to stumble through my career with getting... only two certifications in my life, none of which are, like I have the CCSK because my boss made me get it. I have an old Apple laptop certification to do repairs under AppleCare. And somehow that's been okay. Like, I'm a lawyer, yeah, so there's that certification. Yes. Oh,

yeah. There are bar dues and CLEs and all that, but it's rare that my day job is being a security consultant. So the number of times the fact that I'm a lawyer comes into play isn't that often because I'm not giving legal advice. I'm giving technical and procedural advice and really marketing advice, like how to take your tiny little shop and go and be able to hunt people big clients. And a lot of times I do the, I play the role of, I walk the vendor assessor around the room a few times and basically convince them like, I will give you good enough answers so you can go back and not get beat up for not asking the right question. And I'm not going to lie to you,

but I will give very crafted answers that explain why our security, while it may look woefully bad, isn't. Because we don't have, you know, we're 16 people We don't have a separate group that does this, but this is why it's not. Certifications is an interesting. I don't, I think they have a value. I realize this is way off point, but I talk to people coming up, right? How did you get where you're at? I'm like, that's not the right question to ever ask me. Like, I took the wrong path to get where I'm at. You know, it's a meandering path. Clearly, to do security, I think you need to break into it. Certifications don't suck. A

person who's got a SISP is at least going to get listened to. A person without a SISP is like, who are you? Unless you've otherwise built a name. I mean, generally, at this point in my career, it's like, oh, you're the loudmouth guy with the ponytail who yells a lot. And that's more valuable than a SISP. Coming up, clearly you don't even get a phone call from a recruiter if you don't have something. So, it at least establishes that you could pass a test. You could sit still for long enough and answer questions about the appropriate height of a phone. Is that enough? No. Because I remember doing dot com 1.0 and meeting people who had all the Microsoft certifications and

couldn't explain what a domain controller did. I remember the one was couldn't understand how to troubleshoot a simple IP connectivity problem. This machine is looking for a DHCP server that doesn't exist. He's like, well, I have an MCSE. I'm like, I can troubleshoot. You've moved the PC from here to here. The old network had a DHCP server. The other one doesn't. You have to say MCSE again. I'm going to take your mouth. So I don't want to be the knee-jerk, certs are bullshit. For every story we have of someone who's certified and smart, there are three stories about the person who's got a lot of certs and has no common sense about how to do the thing. Whether it's, why is this not working? Why does this

not meet these requirements? Why does Qualys keep crashing? Why does Qualys keep showing all these vulnerabilities? Why can't I patch this? And some kind of context. I think search is to get in the door and then after that it's reputation. Okay, so this is a panel on CTF and lab environments as a teaching tool. My name is Alex Rubin. I'm Hannah Todd. I'm Shelby Ripsey. I'm Shannon Crawford. We are all either current or former students at Delaware Technical Community College, Cherry Campus. And we're going to be sharing some of our experience we've had with different CTF competitions, building lab environments in the classroom, and how it has benefited us and how we've seen it benefit other students in the degree. We have a few pre-screen questions

that we're going to answer, and then at the end we'll take questions from you guys. First question. All right. How do you guys usually prepare for? Well, we set up like CTFs in the classroom that we practice on. Alex usually makes them and then we test them out. Yeah, it kind of just gives us an idea of what we're going to see when we go into the competitions and if possible, like sometimes certain competitions will give us like the OSs they're using and we usually build those and we just test them out and see what they're like and get used to them. the least experienced. My first CTF was my first day of class this semester. So I just walked in. So there's no

preparation. You're doing this now, and I did it. There's not as big a learning curve as you would expect. Your Google Foo skills get pretty good pretty fast. That's what I experienced. Honestly, my first competition was in 2015 at the U.S. Cyber Challenge Camp. I had no idea what I was doing. That camp actually helped me for the next one, the next competition, that competition helped me for the next one. Once you start going, you start to see patterns, it starts to get easier, you start learning how these things work. So honestly, the best practice for a competition is the competition before it. You just got to jump in and do it. That's why, like, when we prepare to go to a competition, we look

up that exact competition and then a year before it and see what they did and see what it was all about to try and imagine, like, what the questions are going to be, what the challenge is, like, and how it was set up. Question. What reservations, if any, did you guys have going to your first competition? How did you overcome it? I feel like most people's hesitation is just like, I never did one before. I don't want to do it. You know, I don't know anything. But you're not going to learn if you don't try it. You know, it's like Alex said, like the best practice is just to do it, learn from it, learn your mistakes from it. I just, yeah, do better in the next one. The

first in anything is always the scariest. And so you're just like, I don't want to do this. Like, why am I here? It's good to go even like you're expecting people to be better than you because it's your first time going. Of course everybody's gonna like kind of know what they're doing but it's also a great place to like social engineer and like talk to people, network. There we go. The first time she just started absorbing the knowledge of all the other people. Yeah, and all those smart people that you're scared of, like, doing the competition, they had a first time too. You know, they were like a newbie like you would be the first

time doing it, so. Yeah, fortunately, in my personal experience, my first competition, they were like, okay, you don't know what you're doing. We're going to stick you on the trivia. Yeah. But it's been exposure. Like, at the time, I was like, oh, that sucks. But in hindsight, I actually got an opportunity to watch people who really know what they were doing I got to sit there, I got to watch, I got to absorb that knowledge. And the next competition I went to, I was able to go and use those things I learned from my team members. What are the most important techniques and concepts that you've learned and applied in all the competitions? I haven't been to any actual competitions. I

did the one CTF. I'm going to be doing CCDC in the spring semester. I would say like Just critical thinking regarding IT is one of the biggest things that I've gotten better at. Just from doing the one, I feel like everything that, any tool that I had to work with, anything I had to learn to do to answer the questions of the CTF, I've retained that knowledge just from that short experience. Versus like reading a textbook when you're trying to learn about something, it goes in one ear and right out the other, basically. The concept is reading, so it's like one eye. Definitely doing it yourself is better than just reading from a textbook. Just reading from a textbook, you're not doing anything, you're just like, "Oh, that's

how it's done." But then you sit and try to do it and you're like... You're actually implementing what you're learning. You get to use it. For instance, those of you who enjoy Linux, you'd be amazed what you can do with the cat and the grab commands. 2016 Net Wars at B-Sides DC, for example, there was a challenge where we had to find a 40 character flag and an unspecified man page. Okay, that's a ton of data, how am I supposed to get through all that? You have to apply these simple tools in a creative way to get the result that you want. So you can just cat all the man pages, dump them out on

the screen, graph them for any 40 character flag. Yeah, you start picking up patterns in the way that similarly they might be set up. It's using those simple tools that you've learned and you understand, but applying them to new concepts. Because each time you go to these things, there are new challenges that may use the same tools, but you might be using it in a completely different way. Yeah, and you learn that kind of stuff the more you do it. Like the CTF that we did in the classroom, like I had no idea what it was doing. I just didn't know which side. But by the time I did like net wars and stuff at B science, like that was like, I kind of like broke through

like a wall that I was like stuck at with like critical thinking. And it was very fun, the fact that I was able to actually do it now. And how have you as students benefited from building competition environments? Okay, so we actually built... to to CTF in the classroom so we have one based on mister about and we have one is something 2016 election so actually home home so this is the website you yeah or you see long in the is simple stuff first started and started getting a lot more How difficult was it for you to set up something to do that? To block all of the ways? Ubuntu, Linux, there may be 50 different ways in

the system. You have to figure out how to patch 49 of them. You don't want the students or the users that are doing your CTF to be jumping all over the place, getting off course. You want them to be doing what you intended. So you have to learn how to patch all those 49 vulnerabilities and how to create and know in depth how that one works. So that was a fantastic learning experience for me. We also have the other sites. So these are sites that we made in the class and we have a VM environment that runs this entire thing. So we have the two CTF environments running on the same networks. We can turn on the ones we need, turn off

the others, and it becomes an entirely new environment. But this is kind of becoming a cycle where in the capstone class, since associate sophomores, they kind of create these lab environments and then the students take the ethical hacking class and their second or third semester will actually complete this and compete it. Those are all the scripted questions that we had. If any of the audience has questions. So have you been jumping out of the lab experience with CTF and doing more of like a forensics or another type of challenge to CTF help you out? like specific tools possibly? - We have done a lot of things with Wireshark doing network forensics, things like that. We try to incorporate that

into the competitions actually. So you may, part of the competition may be now that you've completed this, you have done a network capture and here's the keycap you got. - What are the typical formats of the competitions and is it Jeopardy style, is it attack defense? Um, the ones I've been to, uh, some of them are Jeopardy and some are kind of like, like break into the system. And then I've done CCDC as well, which is like, you're, you're, you're not breaking into any systems. Yes. These people have been doing this for a long time. They know exactly how to get into your systems, no matter what you do. So the USCC summer camp for this year,

it was kind of a Jeopardy style. And I was kind of like the one that was stuck in the trivia. He did more of the kind of like the attacking part. Yeah. So there are two classes involved here, is that right? So it's a capstone course in which we're building the CTMs. Yes. And then there are some students, the next year, Yes. So, and specifically with me, I actually came back to moderate the CTF while the students were doing it so I was able to find out if I did anything wrong, how to improve, help them with the challenges. Okay, so then the whole class was the case? Was this just a tool? This was a tool to teach material. So... So it's

advanced security topics. So it's really getting into in-depth. It's a lot of operating system. So how to secure either Windows or Linux operating system from the ground up. And that really applies to billing the CTF because as I was saying, those 49 vulnerabilities that you don't want them getting in, you have to completely secure that system and open up one little door. Have any of you guys done cyber phishing in Israel? Honestly, this is something we've started within the last couple of years, so that hasn't become too relevant yet. I see what you're getting at. Yes, no, that's a very valid concern. Luckily the CTF environments are typically pretty small. We try to keep them contained within as few machines as possible

while doing as much as possible so that if we need to go back and do maintenance or patching, we're not doing so many systems. So back to your question, how are they formatted? Has anybody here done NetWars? Both of the competitions we set up have a similar style to NetWars. We really liked that format where we went and we tried to do the same kind of style. So, the seven to five levels. First level, you have a Linux machine and you're a standard user. The gateway to the second is you get root, you never have access. The third level is the, it was two websites. So for the election, it was both candidates had a website. And then four and five is

each of the internal networks behind the website.

I don't Well, we're kind of in that stage right now where we're like creating similar systems to last year's CCDC and we're kind of doing like practice run-throughs. Usually it's him and our teacher attacking like me like would be going next year so like they'll attack a system that I have and then I'll try to keep them out and then if And then at the end we'll go through and they'll tell us how to fix it or we'll research it, we'll do all that kind of stuff. All of the information that we gather, we keep on a shared drive. So that information stays on that and we usually put all the resources that we use on it so that way it's kept on there for the next

year. Knowledge repository. Plus we have like a cyber club and so it's not just second year students, first year students join someone out there that's in the club. So the first year students who don't have a lot of exposure to info-sec yet generally, they get exposed to more just being in club because we do things like, I don't know if anyone's familiar with like Club Hub? They have like XBWA, Extreme Vulnerable Web App that we have set up so you can practice like SQL injection attacks or OS command injection attacks, stuff like that. In addition to like the CTF environments and whatever other resources we have, we set up wireless access points to pack. as an activity so um we typically run that

ctf in the ethical hacking course and then throughout the rest of this master it's a pretty open environment where students can go back in and work on things that maybe they were stuck on the first time they read like we just went to cyberseed and they gave us an rsa pub key and was like okay find the private key now i'm like I had no idea how to do it. And they gave out the answers. We found them online. And this guy just had this script all made up and was like, OK, just put the pub key in here and it'll spin out the private key. So I put that into the shared drive in case we ever need it again. I'll say that as well. Once we complete

something, we kind of go back and see if, like, questions we didn't answer, see if someone has the answers to them. We learn from that. We study that. We run through it as well, like, doing it ourselves. Like I said, like, applying it rather than just reading how they do it. Yeah, it's not only where did I succeed, it's where did I get stuck, where did I make mistakes, how can I fix that for next time. Yeah. You ever think about, like, a website CTF?

So, entirely in the web browser? That's something I thought about, but there hasn't really been a need for it since most of it is still very much in-house and the students will be, and the students are coming in to do it. So, we run on our internal network. and the students aren't there to do it. So it's just something I thought about, but it hasn't really become an issue yet. - Did you do that? - All right, yeah, that's awesome. - After your presentation. - Yeah, awesome. - Any other questions? - Integrating any of your software into the CTM challenges, or is it mostly a service? - So, specifically for the election one, you start with the Linux machine.

The premise is, it's your friend's laptop, he's gone missing. Here's his Linux machine, see what happened. As you get farther in, the web servers are still Linux, and then you get into Windows Enterprise environments on the internal networks. So in one there's a domain controller, an email server, another there's a MS SQL server. So we are getting sort of both sides of it. - One of the favorite things about CTS is when I stumble across kind of obscure software that I wouldn't see.

Yeah, we try to spread it out a bit. Thank you guys for coming. All right. Thanks everyone for staying towards the end of the conference. I guess I'm the last speaker before you get to go home. So I'm going to talk about a lot of buzzwords. machine learning malware characterization big data deep learning and so this is work out of Amount of research that I've been doing at the University of Delaware I'm a professor there and also a spin-off that I found it called cyber 2020. So I'll just give my bio now now so I've been doing this for quite a long time. I like to say that I was Doing this kind of stuff before data was big neural networks were deep

so did grad school work on this application of machine learning specifically looking at code characterization for the whole time so I could say know a little bit about malware characterization and looking at malware code and just recently about two years now I started a company doing It's basically taking the research that I'm doing at the university, commercializing it. So, let me go ahead and set up the problem. So, you know, obviously we know malware is attacking companies. We hear about it every day almost. But what some of us may not know is that there's hundreds of thousands of malware variants being released into the wild every hour. So there's some estimates that there are over

a million new malware released every day. And so that obviously means that the bad actors have embraced automation. You don't have like a million programmers writing new malware. They use automation tools to generate all that new malware and release it. So the good actors, however, are still doing a lot of things manually. I favor more automation, although I definitely don't want to take the analyst out of the loop. I think it's sort of a semi-automated approach that we need to be looking at empowering security people with some of the automation tools that the bad actors are using to good effect. So that's the problem. Well, more about the problem. So we hear about all these hacks that are happening,

big hacks, like the Saudi Aramco ones that would have probably brought down any other company. They just happen to have a lot of resources so they could counter the attack, the massive attack that happened to them. But also, a lot of the more famous attacks that are happening. So we have these high profile breaches. Why are we still having a lot of breaches? Well, here are today's methods of protection. So software patching, You have a lot of secure communication going on. The instant messengers are now all sort of encrypted communication. You've got a lot of security products that are working to help defense and also some obviously employee training. However, are these effective? And the results of the last

slide say maybe not. They're still sort of not solving the whole problem. So we need some... more advanced solutions in conjunction with these solutions. So what I'm proposing, machine learning, and a lot of people also are proposing that, and malware characterization. So I think a lot of research still needs to be done in this area. There's a lot of people running towards this area. If you're a malware protection company and you're not in this area, then you're kind of old school. So there's a lot of companies, most of the companies that are doing malware detection are using machine learning these days. But we have a nice approach to the characterization of malware that I believe sets us apart from other malware protection and

detection companies. So I'm going to give you enough basics that you will understand the case study that I'm going to show you in a minute. So we're going to go over basics in machine learning, basics in malware characterization. So if you don't know anything about those two topics, you might want to pay close attention because it's a good primer on these two topics coming up. So let's start with machine learning, the basics. So basically what machine learning is trying to do is it's just algorithms that are trying to learn from data. So they analyze a lot of data, they learn from the data, and they make predictions from that data. And those predictions largely involve separating out good from the bad or different classes so

we can separate different malware families. And so the machine learning algorithms, what they do is they basically construct these lines in the sand that separate the data sets. So one possible thing you can use machine learning to do is separate malware from goodware. The problem that I'm going to show you in a minute is a little bit harder than that, which is determining what family of malware, a particular malware, comes from. So how does machine learning work? for this particular problem of deciding whether you have malware or goodware. You're looking at a bunch of files and the hope is that you have this data already labeled. So you've gone out to say virus total and you've labeled what files are malware, what

files are goodware, and you have a bunch of data that is labeled as such. You extract features from the data and then you feed those features into a machine learning algorithm and then the machine learning algorithm will output a model that is used to make predictions on unseen data. So that's step one, train the model with data. Step two then is deploying the model. So now I have new files that are unclassified. I don't know whether they're malware or goodware. So I extract a feature vector from each one of these files. feed that to a predictive model that I generated from a machine learning algorithm and then I can make a prediction based on that model. The model will tell me yes

this file is bad or yes this file is good. So that's sort of basics of machine learning. What we use is in my research at the university and at my startup is deep learning. We found that that really, even though it's kind of a buzzword, it really does do a good job. It's given us the best results of any machine learning algorithm that we're using right now. What do we do with the deep learning? So deep learning, by the way, is just the neural network repackaged, remarketed as deep. So it's just the larger neural network with more layers. But it's very, you know, neural networks go way back and a lot of the literature, a lot of the algorithms that are used in deep

learning go as far back as neural networks. So what do we do with deep learning? So we have a neural network here. The neural network takes as input the features that get extracted from each file, and then the neural network, given these features, has been trained. So this is the model that's been trained to make predictions based on the features. So we can make predictions like whether this file is malware or not, what family of malware it comes from, and other types of Things you may want to predict about the malware. Okay, so why now? Why is deep learning, why is neural networks now coming into fashion? Why is it actually working for a lot of people, including the research

we're doing? I believe strongly that it's based on these three ingredients. So now we have large amounts of data that we can train on. and these are readily available so you can go get a bunch of data out in the open source. We have a partnership with one of the companies that captures and creates the repositories of this data so we get large amounts of labelled malware from the company that we have a partnership with. Also, deep learning, there's now open source frameworks that you can just download and use and work in They really are industry scale, commercially hardened products. So Google has the open source deep learning platform. We used one from the University of Montreal called Theano.

And we've highly customized that neural network for malware training. And then finally, there's a large amount of compute power available on the cloud. So for example, recently we ran an experiment where we allocated 8,000 computers on Amazon, and we ran that for two weeks. So it's not like you could do that 10, 15 years ago even, where you could just say, I want 8,000 computers right now. And it's inexpensive. It only costs like $80,000, relatively expensive. relatively. So these are the ingredients that make this all possible now. So we can get some really good accuracy. You're starting to hear these success stories from deep learning. Things like beating the world champion in Go, or beating humans at

recognizing faces, or recognizing classes of animals. Not classes of animals. What's the... I think the one that Google has done is they're looking at dogs and they're able to classify what breed of dog better than humans can. So there's a lot of successes and that's really because of all the computational power, all these nice frameworks available and all the data available. So how do we use deep learning? I kind of went over it a little bit, but let me go into details in terms of malware. So we take a suspicious file. We extract characteristics. First we extract features, and then we characterize those features. And I'll go into this in a moment because this is important. So we do the

file characterization, feed that file characterization somehow into the deep learning. There's some magic involved here. and how to feed it in because some of these features are not our variable length. So you can have graphs, for example, that are for some files that are huge and some files that are small. So we feed these features into the neural network and we can get our predictions. And the deep learning algorithms, even though they may be slow to train, they work really quickly once they've been trained. So they can work at network speed or real time to make the predictions. Okay, so let me get into this portion right here, which is very important, the file characterization. So step one, we reverse engineer.

the malware we extract information from it so things these are some of the kinds of features that we might extract these are most of these i think are boolean so is the file encrypted does it you know does it send spam so we look to see what the api calls are in there does it read try to steal information does it try to encrypt your data so we can get all these kinds of characteristics or features from the files we're looking at. Once we get features, then we characterize those features. So we can look at extracting features from just the bits and bytes, so we don't actually look at the contents in the file. And we can also

reverse engineer the file and get the code. The state of the art involves taking these features, these different kinds of features, converting them into histograms, and then creating a feature vector out of these histograms. So the histogram is basically just a count of the different kinds of characteristics. So like, for example, the-- A bytes histogram would have 0 to 255 values for bytes. And then you would just count how many of those values exist in the particular file that you're trying to characterize. Or you might look at these are the counts of the different instructions from the reverse engineered code. And then we convert that histogram into a feature vector. So that's kind of the state of the art. But we do something

different, we look at GRASS. GRASS allows us to look at the shape of the code and that's really powerful and I'm going to show some results now on that. So I've given you the basics, now we're going to go into a case study of all these basics and how we use it for malware family classification. We want to decide what family of malware a particular malware comes from. This is important, as you might expect, to improve defenses. So if you know a particular defense of action worked on one malware, and you have another malware variant of the same family, the same defenses might work on that other variant, and things like that. So it's important to know what family

malware, a particular malware comes from. And there might be new malware that doesn't belong in a particular family that you've seen before and you might want to scrutinize that malware further. So that also is another reason why you want to do this kind of family classification. So this is what we built in the university that has kind of been spun off into the startup that I started. So what we do is we send files to this platform that's in the cloud and we do analysis. So we can do reverse engineering or we can do dynamic analysis. That's where we get our features. The features then get characterized so I can convert those into histograms and feature vectors. Or in the case of what we're doing

that's trying to improve on the state of the art is we're converting these features into graphs. Then we can feed those graphs into our deep neural network and all this is running on the cloud and high performance computing. So we use GPUs to speed things up, distribute the computation over a bunch of different machines, like 8,000 machines we can do with our distributed libraries. And then we can produce valuable insight because from here we have just data. We want to end up with this information that's useful to an analyst. So as I mentioned, current state of the art involves histograms. So we can compute things like entropy of the file. We can do reverse engineering of the code

and extract the histograms and then generate the feature vectors. So this is kind of the state of the art. And there's one particular company who produced a research paper a couple years ago called Invincia. quite well known for their machine learning. They were doing this kind of stuff in that paper and we're going to compare to what they did. So that's the current state of the art. What we're doing on top of that is graph based malware characterization. So we have the reverse engineer code. So this is actually a static graph. of a particular code. And then also we can run the code or run the malware or the file, the suspicious file. We can run it in a sandbox, in particular, Cuckoo, and we

can get dynamic behavioral graphs. So those are, for example, what are all the functions that were executed? And we can create a graph of those functions as they were executed. So let me move to right here. I'm going to show you a little-- demo the platform, see if this works. Okay, so this is the WannaCry malware. We ran this through Redari, and we extracted all the code, and then we generate these graphs, and this is a static graph of the functions that were executed, and this is the kind of graph that we feed into our deep neural network. So you can see it's quite, there's a lot of It's not just on that part of the screen. There's a

lot of code here that just goes on and on and on. So this is difficult for an analyst to look at manually. This is why it takes analysts a long time to look at the malware and decide what it's doing and where it's doing what. So that's the function called graph. And we also have a graph of the blocks. This is even more finer granularity. And this just goes on and on and on. So again, it would be difficult to manually go through this graph and understand what it's doing, but machine learning comes to the rescue here. That was static graph. We also have a dynamic graph. This is a graph of all the processes that Cuckoo recorded when this one crime

malware was executed. And again, you have a large graph It may be difficult for an analyst to inspect and try to understand, but again, machine learning comes to the rescue here. I was going to try to zoom in on one of these things. So we close Tor.exe. That's kind of suspicious. OK. And this is our UI that we provide to security analysts as part of my startup. OK. So going back to the graph. We're comparing the state of the art to how graphs work in terms of the machine learning. So the data. So we have a partnership with a company called Reversing Labs. They have billions of malware available in the repository. They also curate streams of malware.

So this particular, the results that I'm going to show you are on a financial curated stream. So it's all financial malware. And we started with 40 plus families, and then we selected down sampled and selected just 11 families that have more than 1,000 malware to run our machine learning on. It's not to say we can't run on the additional families here, but we would have to do something else to sort of balance the data. Otherwise, the overwhelming amount of malware in other families would cause machine learning to not really learn about those other smaller families. Okay, so here's a summary of the features that we're looking at. And we're going to show you some results for each of these.

So as I mentioned, there's some KDR here. There's byte entropy histogram. There's looking at a global-- We have statistics of the code. So we have the feature vectors there. And then we look at our different kinds of graphs. So we have function graphs, we have block graphs, and then we have operation graphs. So I showed you before the function graph and the block graph. I didn't show you the operation graph because that's just huge. It just blows up the system. So I didn't want to get on that. But here's some pretty pictures that I just came up with this week. So we try to cluster the different families based on the different features. So this is the My Instagram clustering

for the different families. This is assembly. This is global, just looking at feature vectors. One thing you'll see is that there's a lot more negative space here. The reason is that a lot of the malware will converge to the same data point. So even though these graphs each are constructed with 20,000 malware, however, in this graph, we have only 9,000 unique data points because the histograms that were generated for a lot of the malware go to the same point. Yes. Are you unpacked? So this is sort of the state of the art in terms of the features and we can look at the features. It gives a sense of whether there's some nice separation with the

families. So this kind of visual of how separated the features are representing the different families. So here are the different graphs, function blocks and operations. It's hard to discern anything really from these other than to say there's not a lot of negative space. All the space is filled up. And it does seem to do some separation of the families. So you can see clusters of families. And so what the deep neural network is going to do is really construct-- these lines to separate out these different families. That's kind of intuitively what the Deep Neural Network is doing. So as I mentioned, we ran our experiment on 8,000 nodes on AWS over two weeks. And what we did was we built a different

model, a bunch of different models for each of these different feature sets. So let me sort of point out some highlights here. So lower is better. So we're looking for a high accuracy or a low error rate. And then the x-axis is the time it took to build the model. We want high accuracy or low error rates, but we also don't want to spend forever building a model. If it takes us a year to build a model, it doesn't really matter how accurate it is because it's going to be out of date. So we need models that can be built rather quickly, hours or days, not weeks. So in this particular case, these characterizations here may be too large because they're just

taking too long. They took over. These are the ones that took weeks to generate models for Linux. And they're still sort of not low enough in terms of the accuracy. So they still need more time to learn. On the other hand, these other features here, these are graph-based features. They get very high accuracy, very low error rates. fairly quickly, sort of, in terms of like, we get it down to a K, basically. Okay, and here are the results after the training. So this is the state of the art of my entropy histogram. Global sort of looking at feature vectors, assembly of features in the file. And so we're looking at ensembles of deep neural networks,

so we can build one model, or we can do a, we can build two models, and we can do a voting scheme. for five models, and a majority wins in terms of what the five models vote on, in terms of what family a particular malware comes from. So state of the art, we get around 87% accuracy in terms of family, malware family prediction. Our graph-based alone gets us better than that. So we get like around 92%. Then in terms of predicting the right family of malware, if we add both graph-based features and the state of the art, we get even better results. So we get close to 94% accuracy in terms of malware family predictions. One best versus two best or five

best. The increasing number of using voting actually increases the resources you need. Right. Right. Sure. So you've got both to train the model and to make the prediction. So the bottom line here is that we get 50% improvement using graphs together with the state of the art. But the graph, as I mentioned before, gives us the shape of the code. So we can look at sort of the different functions and how they're connected to one another, how a particular function, the different blocks of code within the function, so how the particular function is shaped, and also all the operations. with the operation graph and how all these different operations are connected to each other. So this gives

us topological information and as you can see it's important to improve the performance. But we don't want to leave off the state of the art because we get even better results. While it seems like there's a 1.7% difference here, that 1.7 can be pretty dramatic when you're looking at thousands and thousands. That's it for my talk. I think I went pretty fast. That's okay. We'll all leave early. What I'd like to do is I have a gentleman here, my colleague. We're doing a survey of what the problems are that we're trying to solve in terms of the product that we're building. So if some of you don't mind to fill out our survey, it'll only take like two, three minutes. That would

be very much appreciated. So thanks very much. If you have any questions, I'm here to answer anything. I'm just curious, are you using any machine learning techniques to actually develop the training data? So you have the data to develop the data, right? Right. So we thought about-- and we're looking into synthetic generation of malware. So in particular, so if we look at this graph here, There's a lot of white space available. So what happens if a new malware appears that is right around here somewhere? We don't know what family of malware it is. It's sort of like equidistant away from other families. So if we can genetically generate more data to fill in more of the space and

get more training data, that'll be useful. The other place we use machine learning is to come up with all the different parameters that are needed to tune our machine learning model. So you have things like, what's the number of layers that works best? What are the activation functions in each of these neurons in the neural network? whether these layers are fully connected or not. Things like that. Those are called hyperparameters. And they can make a significant difference in terms of the performance of the final model to generate. That's why we ran the experiment on 8,000 machines. Because we were not only building one model for a particular feature set. We were building many models to try to find the best one for a particular feature

set. So what we do to tune the neural network is we use a meta optimization, otherwise known as genetic algorithms. So that's another machine learning algorithm to use within the platform that we built. Have you looked at taking-- so you looked at the-- No, the other directions are different. The other direction plus a couple of slides. That one. Not that one. We go slow. OK. Who would go relay the act and then the impact? Oh, right now. Yeah, that's a good point. The other question is-- so we know we haven't done that, but that would be interesting to see. It would also be interesting to see if we can learn From the packed malware. Absolutely. Absolutely. I mean, it might be the

only thing you have because you just don't know how to unpack it. So the partnership we have with the company Reversing Labs, they do all the unpacking for us. However, they're experts in unpacking. However, that's a great experiment. I've been thinking about that actually. Simply even just encrypting all the files or compressing all the files and seeing if we can train on compressed malware or encrypted malware. Yes. Why don't you pass out some iPads so they can start the survey. Sorry? A brand new family? Yeah, so train on some families and then test on unseen families, basically. That we haven't tried. That would be interesting. And I think it should work if... So we're not learning, we're learning

the very basic aspects of what it means to be malicious. So we're looking at things like, does the malware encrypt data? Or does it make a call out to some API that does an encryption? Or does it make a call to read the registry? So those are very basic fundamentals, so there's no reason why we shouldn't be able to detect a new malware family, but it's something we haven't tried, so that's another interesting experiment to try. There was another question? Similar question? So here's my contact info. If you can reach out to me, we're looking for a few people to pilot this deep learning cloud for malware for a bunch of different malware problems. We already have a couple of pilot customers.

We're looking for maybe one or two more pilots. So if you're interested in seeing how deep learning fits into your cloud, into your workflow, send me an email and I can send you these slides and also there's some other slides that we have in terms of how we can put deep learning in customized to your particular needs. So there are not any other questions. Thank you very much.