RedOps: Automating your Red Team Infrastructure

so to start out with if you go to these links you can actually interact with the presentation so if I pop a link up on the slide you'll be able to actually click on the link on the slide and it'll pop open in your browser and it's not malicious or anything like that it's just to be able to get like github links and stuff like that so that way you don't have to take pictures of it we'll see the thing is that if I go to a conference and do that then other conferences won't let me also if you guys want to talk or ask questions just go ahead in the middle of the talk just

raise your hand or just shout it out either way so as you can see this is automating your red team infrastructure or you'll see the reason why I said red ops later I love to automate and for red teaming Kali Linux is normally a pretty default intro distribution to go ahead and start testing things out so my name is alright seven for one or Alex Rodriguez either one so to start out with Who am I so my experience is forty I mean four years sorry I haven't not that old yet but I absolutely love what I do I love where I work I work at secure ideas I'm a developer there and I'm also depend

testing and operations two more IT infrastructure where a pen testing company we do lots of web apps and network pin testing if you want to ask them more about that just I'll give you a card afterwards so the community secure ideas also has their like open source or open the open source title of professionally evil so I help out with that I'm also part of the 49 Security Division which is at UNC Charlotte or University of North Carolina at Charlotte and that's where I learned a lot of my pretty much everything about security before I'd gotten to work at secure ideas and then another community that I'm part of in Charlotte's really awesome and has

awesome connections is cha or Charlotte hackers anonymous and if you go to went to those slides you can just click on all those links and be able to go to their home pages so before we start just a quick joke so has anyone heard of DevOps I'm sure there's been a lot more container stuff here at this conference so DevOps is something that's been kind of pushed into the community a lot it's got a really big push for it right now so after DevOps came out they came out with a bunch of other names like chat ops and dev sec ops sec ops so I made the joke of well this is red ops so take

the pill and go down farther into the rabbit hole in the oh alright um so now the actual presentation so we automate lots of things in pen testing we use different programming languages and scripting languages bash Python Ruby uses in Metasploit Lua for nmap scripts so just tons of automation in general but there's not really been very much automation for your infrastructure so I mean do you enjoy having the apt-get upgrade apt-get update every month for Kali and then possibly something breaking it's not very much fun whenever you have to wait two hours just for everything the update so this is a talk to kind of address that there are tools like vagrant and Packer that are used

for this type of automation to where you don't have to sit in front of your screen every time you want to do upgrade everything can be updated for you and then you can just pull down a fresh image and then whenever you get done with a pen test destroy that image so that way you don't have any residual client data on your machine and then you can just spin up a new one with your all of your custom-made customizations implemented in that cally box so that way you don't have to reconfigure everything normally you have to do like a snapshot I'm going to revert the snapshot to make sure that there's nothing residual stuff like that this

automates everything for that and makes everything a lot cleaner instead of having to do some hackery to get it done so there's also a tool called terraform and I'm not gonna really cover very much about it but it is a really awesome tool and I'm gonna show some links at the end of these slides um to go ahead and talk a little bit about what it's - what you can do with it and eventually if you get into red teaming long enough you hear about c2 or command and control centers and to have that you have to have infrastructure created like in a cloud or on a home server or something like that so terraform is to kind of help automate

that so managing essentially external infrastructure or even lab infrastructure so I know that for me I've had to deal with a lot of manual instructions for all infrastructure stuff so that's what all of these tools are for also I'm a big Pokemon fan so that's why Pikachu is on there so Packer is a Hoshi Corp says easy to use automates the creation of any type of machine images and so what Packer is really best for is essentially creating what's called a base box so you don't want to do heavy customizations that are going to you're not going to need possibly later on so if you want to install services or configure services in a certain type of way before you

actually use them that's what Packer would be for so if you wanted to install docker on Kali then you can be able to use containers whenever you get into an actual pen test because docker doesn't normally come in Kali by default so that's what packer is used for it you takes builders which can be a whole variety of things you can use VirtualBox ESXi AWS tons of them cloud services on Prem services for visionaries which is just shell scripts ansible scripts whatever you want to use for actual provisioning and then post-processing is essentially once all of that automation takes place it will push your image wherever you want it to go so they have as you see in underneath the vagrant one

I saw a vagrant cloud and just to show you what that is a little bit essentially whenever people build images with Packer they can push them up to vagrant cloud and so these are my Packer images or my vagrant box images and so this is the one that I'm gonna be showing you guys how to actually do today and then there's all a ton of images out there and there's made by canonical for Ubuntu search um maybe like a nautical by chef there's tons of images out there that are maintained by tons of people make sure that you always try and find well-known ones before going out there just because you don't want to have like an incident where

docker where people are just spinning up images all the time and there's Krypton liner and dr. docker images in data centers so definitely vet before you download the images but as long as they're from a reputable source then go ahead and you can use them for whatever and then these are just my github links to different config files which we're actually gonna get into later so kind of just to review as to what we're going to or overview as to what we're gonna do magic to be able to automate all the infrastructure no it's just using basic tools and then Kali you'll be able to create custom images like I said with docker you can

install that beforehand so that way you don't have to worry about the installation process of docker and then you can actually use terraform for the environments for like I was talking about for home labs you could do it if you want to test and exploit up against like five different anti viruses just spin up all five boxes by running a single command it'll interact with like an ESX I box spin all of those images up and then you can run your exploit against all five and then tear them back down so that way you don't have to manually spin them up and then put the AV on there and then do yada yada you can just have everything

pre-configured and just pull from those images and so the potential is limitless so normally I give this in kind of a class setting so these are normally the prerequisites that I show people but also this is what we're gonna be doing today so I've actually created a script um that creates a file called variables JSON and this is for all of your like sensitive information or dynamic information so what this does is it goes out and it web scrapes Callie's website to find the most stable version or the most up-to-date stable version so whenever it transitions from twenty eighteen point one or twenty eighteen point two it'll automatically detect that from their website because it'll

they'll update the link and then it'll just grab that updated link so you don't have to worry about figuring out the URL for the most current version of Cali and then it'll also go through and if you have I'm so working on configuring this part of it because not everyone has the same path to their credentials that I do so if anyone has any suggestions um so this will actually go through and grab your vagrant cloud credentials and also oh your username for vagrant cloud as well so that way you don't have to programmatically put it in and multiple people can use it because it'll just update with your username and [Music] unfortunately I'm not going to be

showing you my vagrant cloud credentials and using me or putting it up there because it's a video anyways and it's gonna be recorded um so I'll show you what that script looks like but you won't actually get to see the token in there the usual names in there but and then this part is for versioning so what you can do with vagrant boxes is have different versions and so I'm currently on my 0.09 version and so you've got eight seven and you can do this as many times as you want to up in vagrant cloud as long as its public and it's completely free so none of these tools that I've mentioned so far cost anything

to use it's just the time to learn them which is hopefully gonna be a little bit easier after this presentation so this is the JSON template that everything is involved around so Packard takes this JSON template and it converts all the different JSON data into what's actually going to happen to the machine real quick does anyone have any questions so far okay um so you can see I've got what type of builder I'm gonna be building it on VirtualBox taking an ISO so it's gonna be a debian box these are what are variables so this is the dynamic information that I was talking about so the ISO URL or the user ISO URL is actually defined down here as blank and

this is what's gonna get fed in from your variables JSON so all this dynamic information will get populated whenever you go to actually execute the build process so that way you can stick all these some figs up there and not toss your token in there by you can exclude all the important things and then this is the provisioning step for the provisioning process that I have so it all it does is essentially go in there and run a bunch of shell scripts to go ahead and install different things upgrade Kali to the most recent packages for everything make sure that it's got VirtualBox guest additions installed and then this is the post processor so this

is the vagrant box that gets created and so a vagrant box is essentially just an OVA or an OVF which is the hard drive or the export of a VM and it puts in some extra metadata into there so the vagrant box gets created and essentially just has your OVA export in it and that's what vagrant ingests and then vagrant cloud is whenever it gets pushed up to the app vagrant up calm which was all the different images of people that people make and stuff like that and then these are a bunch of different variables that get used in the actual JSON and the amazing thing about this is that you don't actually have to run this on your

machine every single time you want this to happen you could actually set this up on a server and run it headless and just have it on a cron job to repeat every week or something like that and it'll go ahead and do the whole build process without you interacting with it at all so this is really nice because then you don't have to worry about executing and making sure that you're staying up to date with the information and then these are just the scripts that get executed during the Packer build process there's quite a few of them there and actually took some from other people that were building Packer boxes as well and then this is an additional vagrant fire file

config and vagrant files are used for essentially like extra provisioning so as you can see I've got the memory and that declares it beforehand so that way whenever someone does vagrant up they automatically get at least two gigs of Ram of course at least one CPU let you turn the clipboard on in case you want to be able to copy things in and out and then this is actually used in another demo that I do and what I do is I spin up an entire situ like Network kind of and this cally box gets thrown onto that network so what I'm doing is I'm declaring that this the vagrant box that's getting spun up gets a specific

IP address on that network and joins that network and you can use this for a different variety of things so you could say I want to bridge the adapter every time that I start up this box so that way you can do like aircraft or whatever you want to do with that so vagrant files are then for more configuring after you've exported that base box so now we're gonna actually do now we're gonna actually do the demo okay so this is only going to be the weekly version of Cali because as I'll show you later the actual full build process for a stable version of Cali takes about two hours so I didn't think you guys wanted to sit through that

whole process um so this is just gonna be a short version and then while this is executing I'm actually gonna pull up the video for the stable version of Cali and I'll show you at the end that I took about two hours to execute so I'm just gonna execute that and as you can see it's downloaded the guest additions for VirtualBox and then it's also downloading the year of the ISO for the URL and you'll be able to see more of this in the actual video too I just want to start this process real quick does anyone have any questions yes

um some of them and the only reason why is because some of them like whenever you install VirtualBox guest additions the host needs to be restarted because it actually adds stuff into the kernel in this JSON template yes so depending on what your provisioning it'll need to go in a certain order but the way that it needs to go in now it is properly format or in proper order in the JSON

yep same one yep that's all of this stuff that I'm showing you today is actually in my github repo that I'm gonna show you at the end of this so you'll be able to do whatever you want with it pull it down submit pull requests I'm open to anything okay so well that's doing [Music]

okay so this is the actual full stable version of Kali installation and we've trimmed it down to about two minutes fortunately I have an amazing girlfriend that is really good at film so she helped me out a lot so this is like I was showing you before it automatically takes the URL that it get you gave it and then goes ahead and downloads it and then after it downloads it essentially if anyone knows what pixie booting is what it does is it creates essentially like a small network and then you feed it a file that will answer all the questions that Colleen only asks you so it's starting up an HTTP server on that pixie booted Network so that way it can

give it that what's called proceed file to go ahead and answer all your questions for it so then it goes ahead pops up a Kali installation and then it executes what's called a boot command and this is just more information to let the VM know where to go for the three C file or the file to answer all the questions for Kali you can run it on VMware you run it on anything so there's even you can run it on AWS and cloud providers as well so this is the actual installation so it just finished up I skipped the whole middle part so that way you guys didn't have to wait but it just finished up the

installation and now it's rebooting the VM so that way you can then further provision with the shell scripts that I showed earlier any questions

so it's just rebooting

yes but normally so well you can't do it anymore but if you're a student you can get like the ESXi hypervisor and normally I believe that one doesn't require that play paid plugin I'd have to check on that but yes I believe that the VMware plugin for vagrant is paid but they also might if you if you're a student be able to go ahead and give it to you for free or something if you email and ask I don't know about that but it's possible so now everything's provision it's restarting or it's restarted and is now provisioning with vagrant and so this is as you see it said connected to SSH right here so that

means that connected to SSH so what happened there is Packer finalized the installation process and then now SSH into the early installed VM and is now doing the running all the shell scripts that I was talking about so it's all inside that little small network and is it doing the install it's doing an app get upgrade right now update and upgrade so then after the installation is done it shuts down the VM and then goes ahead and exports they're completely updated and customized VM that you have and then it converts it to the vagrant box and this actually can be used locally so say you don't want to push it up the vagrant cloud you can actually just have Packer

push that vagrant box image anywhere you want because you can run shell script commands afterwards so say you're doing some type of automation for your company you can everything happened and then run more shell scripts on your system to then push it or move it to wherever you want so it's not just limited to uploading to vagrant cloud there's also lots of other providers that have made plug-ins for it so you go ahead and be pushed to all the other different repos as well or different platforms so it adds all that metadata and it's compressing all of it now and then now it's pushing it to vagrant cloud which depending on your connection can be really slow so that's

why I recommend now okay so I said forty two hours because 42 everyone in the first second knows where it is always the answer um so not really two forty two hours but yeah I figured I'd get a reaction of that but I mean depending on your internet connection it could because it's only about four gigs of a file so depends on your connection that's why like I said I recommend running it on a server that you just do headless after you have everything provisioning um so as you can see it took an hour and 41 minutes and I had I have a decent connection I don't have a gig of course that would be ridiculous if it took that

long but I think I have 300 down and 20 up so I mean it's a pretty average connection so it took two hours pretty much to run that entire thing so that's the video I didn't run the variable JSON for you guys actually do that right now um so

okay so that's the that's the script that I showed you guys in the repo this is to then take your information and plug in the dynamic variables that you need so so like user fit where my hands are failing right now sorry um so I'm gonna execute the script and not show you guys my token all right so what it does is it adds a key for Kali Linux so that way you can this script also does a verification that all of the information that you pull down from Callie's website the sha-1 some sha-256 whatever you choose it validates that that check like the signature of that image is good so that's what that's why it has to import

the key so then it says all right please keep the similar formatting the current version is 0.09 so just web scrapes from my profile on bigger cloud and ask for kept up with what actual version is the most recent version so then do something like 10 and then execute and it has the ISO URL and like I said I changed it to the weekly version for this but you can literally just go in there ones like I'll go to that in a second um so then I chose 256 for the ISO checksum type and then there's the actual 256 is a checksum and then the name that the Clapp over that the image the vagrant

file gets uploaded to and then the view the inversion and then right below that would be your vagrant cloud token but like I said I'm not gonna show you guys um so if you go into the script and say you want to do instead of the weekly bill you just come over here and just do

I think it's

well the colors are dark so I think does it know

here actually I'll just do it over on this one this is still the that week we build you guys see that or no this is actually my freebsd box at home it's on VNC but all of the VNC stuff is tunnel three ssh so VNC actually isn't exposed to anything just localhost there we go thank you for pointing that out it's pointless if you guys can't see it so set the Kali weekly and then from there you just type in current run the script again

oh wait I'm not showed you guys make you

okay I saw the long string of characters I'm like oh crap wrong keyword oh so now it's the most current version of Cali I'm actually gonna just copy that link and show you guys that it actually goes there so there's the kalyana so I'll go ahead and back it up one so that way you guys can see and actually let's say so he's literally just what going in there and changing Cali weekly to current and it completely changed the output of what will happen in Packer it'll build a completely different image well it'll build it from a different ISO so it will take longer but Cali stable is normally better for long-term effects but just

for this demo it's gonna be quick so I did the weekly version

see where this guy's at it's still updating any questions

I type it in so many times I know it by heart

do you want the link is that what you're asking for and if you actually go to the the bi t ly bitly slash red ops you can actually see the slides and that's where I'm gonna show you guys afterwards this is the actual presentation live so this is I mean that's currently where I'm at in a slideshow and then once this is done you can actually just click on down here and it'll take you to the slides and it has all my speaker notes and everything like that so everything that I talked about will be up here just whenever whenever you do this that on slide five there's a down um not very many people realize that but

there's a down right there so you miss a little bit but not too much and then six those are down so as long as you see that bottom arrow down here that means that there's a down to that next slide so don't miss the information this is actually the this slide right here is when I was gonna show you guys next really quick I want to show you is so this is a Packers website I have everything all the links for it in the slideshow so whenever you go to the bit lead or bitly slash red ops it'll be in there but this is it does a whole bunch of different providers and builders so

these are all of the different ones that you can use to actually build the image that you want to make and so like it's ridiculously long or none of them meet your criteria then you can make so this is completely open source by Hoshi Corp which these are all tools by Hoshi court they're really big into DevOps automation of infrastructure these are all the different provisioners so ansible is used there for provisioning stuff it's a Python variant that Red Hat makes I mean it's pretty much anything you can think of they probably have something for it and then these are the post processors so yeah so Metasploit OBO 3 is actually it has a Windows version so Metasploit

able to is normally what people are acquainted with but Metasploit about 3 there it's the new version it actually uses Packer to build the image on your machine instead of having a hosted location it'll build it on your machine for you and you can make customizations to it and they've got the windows right here

that's not this one and what they do actually during the boot up process I heard that the official terminology earlier this weekend it was called answer files and what you do is you pass in an XML formatted document and it answers all the information for you so it's like the precede file for Linux you can do the answer file for Windows and it does all the customizations for you that one I haven't document it as much but there is I heard about it this weekend so that's why I don't have anything in my slideshow these before detection grabs this is supposed to be an already made Windows image for Packer so it's called detection labs and I can

just add this into my slideshow afterwards but you can do and it's got a Windows 10 image windows 2016 r2 but they've got two windows images for you right there so you don't have to worry about figuring it out yourself because they've got the answer files as well right there Oh looks like they do have r2 so yeah it's it's apparently a really good repo I haven't messed with it myself but I was hearing it from one of my friends this weekend that it's really good because he's trying to build out pretty much a DevOps automated lab like I was talking about earlier and he said that these images are really good and then you can

just further customize it after you you run all this stuff so that's all of Packer stuff and then once you get the image built you're like okay so what do I do with it now so then what you can do is the tool that I was mentioning before called vagrant what you can do is just pull it straight down from you specify like which bigger an image you want and then you can you can then spin off the BM so all you have to do is type vagrant up and it what it does is it imports like it doesn't import like how in VirtualBox you import a VM it does all that for you and then further configures that based

off of your vagrant file so that's what I was talking about what hours where I was talking about the c2 Network it'll take that and then apply all those changes in VirtualBox for you so that way you don't have to manually go on there and adjust everything so since the Kali image is about 4 gigs it takes a little bit to important but this way the whole point of vagrant is that you can vagrant up in vagrant destroy as many times as you want and you have that same base image that you import so say you want further customizations like say it's you finished the actual Packer build on Sunday and now it's Thursday you can execute a shell script that says

all right in that vagrant up process run apt-get update upgrade to grab the most even more up-to-date packages but instead of taking 2 hours to do the stable build it'll take about 10 minutes because it'll be all the only thing that'll change from that time is the actual changes from that week so it'll cut down immensely on your management time of having to import everything so that's taken a little bit but any other questions see

okay so packer completely built out the VM and it just stopped the Packer Auto Kali VM for the weekly build and it's getting ready to then export the OVA that export process is going to take a while so I'm probably going to end the presentation before or before trying instead of trying to show you guys that but let's go back to I'll go ahead and pull up VirtualBox so that what you guys can see so it hasn't since it's still importing it's not going to show up but once it finishes importing which it normally goes a little bit faster after about 30 percent this is a completely updated Packer image or image that I built from Packer so it'll just be spun

up in VirtualBox also there is documentation for vagrant as well so VirtualBox isn't the only one

so this is a Firefox [Music] this is vagrants documentation so anything that I didn't cover you can go to their Doc's in their documentation and it's extremely detailed and then even if their Doc's don't have something you can go to their github repos which those will have ongoing current issues and fixes for different releases so since all of this stuff is open-source in public it's pretty much like you know exactly what's happening when and if there's a problem you can submit an issue there and go ahead and ask for help like hey something's broken do and then they'll help you along through the process they're really responsive as you can saw they've got almost seven

thousand three hundred issues closed so they've worked through a lot of issues but let's see you can also use vagrant with Windows as well so say you build a Windows box you can use win RM or win SSH - then essentially SSH into the windows box they've actually got a win RDP as well or a bigger an RDP something like that that you can get an RDP session as well so vagrant works with both Windows and Linux for boxes or VMs essentially and can do the further customizations for you based on what you need

so this is the different things that you can use for provisioning again it's pretty much the same set of tools that Packer had and then what you can also do so how I was talking about the c2 Network is you can have multiple VMs inside of the same vagrant file and then you can do a vagrant app and it will spin up all of those VMs for you so that way you can configure all of them just by typing in the code into the vagrant file and not have to worry about as you spin it up you have to go in and manually change and add each of the VMS to private network or whatever else you

need to do we actually use that for secure ideas for our our project called samurai WTF web testing framework and you can do a vagrant up and specify if you want a specific target to come up as well so you can then essentially do a pin test against that target and be able to learn different types of attack attacks by attacking another target instead of just locally on your machine but it's all inside of your VirtualBox and at a network or whatever provider you have for your VMs and then an amazing thing for vagrant as well is the shared folder feature that you normally do with the end instead of having to do that every time the vagrant

automatically adds a shared folder of vagrant inside of the VM so that way if you have any tools or anything that you need to pull every single time that you do a vagrant up you can stick everything in that slash vagrant folder and then if you need to reference anything in that slash vagrant folder you can just go slash vagrant and then the whatever you need the reference so say you need to install a license for a tool or something you can have the license there and have everything execute to license whatever tool you're using in the vagrant up process so my computer's not dying anymore let's see so the VM is

screen size is not liking me right now

goodness gracious all right it's not like you

well as you can see it's got the two years of memory looks like I've got the process of yeah it's updating right now should I remove that script but the documentation is awesome does anyone have any other questions

see if this

must have film the upload to vagrant yeah but all the versions that I've got up there so far have been built by Packer it works and to show you yeah right there so the actual vagrant box got created if you look right here red - retro box top box that was the box that we created from the Packer installation so the uploads a vagrant pod didn't work but you know I removed my token that's what happened um so yeah don't move your token from the script of course I did because of the presentation but if you don't do the tech V then it'll work but yeah so that's pretty much the presentation oh so that's my

Twitter that's my github like I said you go to that link the bitly dot or bitly slash adopts you can click on all these links and go to them so I always take any type of pull requests or comments you guys want to have on different repos feel free that's where all my images are for vagrant cloud it's pretty much just Kali right now so for my company I actually am doing a blog series on all of this to where I'm going to go ahead and get to the nitty-gritty of this so I've actually already done one article kind of just trying to get knowledge out there of vagrant and Packer there is so you can go to the blog and

read through this and here's kind of like the workflow of you creating the Packer image uploading it to whatever you need to and then pulling it down and you've got your calli BM on whatever operating system or provided you want to do so you can go there I've got some resources and then also let's see let me go back to the slides so those slides you can go to that or the bitly link that i was talking about

Oh can't actually good so that's the link

if you want to get up and then so this is the links that I was talking about earlier with terraform if anyone's seen the red team infrastructure wiki it's a github project that you can just google and essentially it talks about how to make your red team situ Network resilient and then defense like make sure that's secure and that it constantly stays like it's really a really good infrastructure for the backing of your c2 network or your red team infrastructure and then rastamouse goes through it actually does that with terraform files so you can go visit his blog and he's gone through and talked about how to do all the different things that the Red Team infrastructure or Red

Team wiki talks about so I really like being able to apply things that people talk about or show and then being able to actually apply it so this is a really good blog that I read through to actually learn more about Sarah form and then I actually met an individual his name was my colleges at Carolina Colin this past year and he's going and creating a framework to essentially help with masking c2 networks and he's trying to automate the whole process of that c2 network and his projects called hide and sneak in effect initially he was trying to program all the API calls into his actual project and if the reason why that's not maintainable is well it is

but it's difficult to maintain because if they change their API at all then you have to go through and change your entire framework around that and update all of your API calls to then make sure that everything works properly but abstracting it away with terraform that means that hashey Corp will then maintain all those API calls for you and the only thing you have to worry about that with them is if anything in their syntax changes which it doesn't seem to change very often especially for terraform stuff because there's tons of companies out there there's like I've heard of Blizzard in a ton of other different companies I can't think of anything right now to top my head but

they all use terraform to maintain their infrastructure so it's gonna be probably around for a long time before anything goes away with it so it's pretty good to go ahead and use in tools and it really helps with if you want to use different cloud providers you then don't have to learn all the API calls so he was initially doing it for I think AWS I saw in the black hat Arsenal video that he also had I think it was a er in there as well so he in theory didn't actually have to learn any of those API calls he just plastered it on top with terraform and abstracted all of that away so it

makes essentially a migration of infrastructure easier so that way it's repeated in the same fashion any other questions all right I think that's everything [Applause]