Topher Timzen - The Trials and Tribulations of Building Your Own CTF and Shooting Gallery.

um so I'm Topher wearing a Sauer shirt if anybody can tell me the country that the spant hails from I will buy you a beer across the street after the talk I red team at a large fortune 500 I'm a self-proclaimed vulnerability enthusiast who likes cause and constructive mischief previously to this I had lots of three letters of government fun over in a place far far away on the east coast where it was awful because living in Maryland boo because I'd rather be mountain biking and the Pacific Northwest is the best place for it so today I'm gonna talk about for the 20 minutes that I have firstly why you want to train offensively and we just heard

for about an hour about challenging your own biases and this is where I'm certainly biased because I do come from more of an offensive background and now in a position both from a product security standpoint and a more adversarial modeling standpoint of I do a lot of offensive nature but I also want defense people to know how that behavior and that nature entails the environment so they get a better idea of how to stop it then I'm going to talk about CTF and this notion that I've came up with called a shooting gallery which is a training mechanism that I've been utilizing in order to help train people offensively with and then I'm gonna go

into the actual tribulations of struggles that I ran in with and things that you can do to build and deploy challenges and then go into a little bit of infrastructure and hosting so first of all why train offensively and like I said this is where I'm a little bit biased because I do have that this offensive background in mindset but you know a lot of people will say oh well offense is better than defense or a defense is better than offense but I don't believe that's necessarily true in both sides and lots of angles have to work together and yeah you're gonna you know have your nation-states where they're only focused with offense and you do this cat-and-mouse type game but

internally in an organization I feel that the teams need to work in tandem together without going into this you know notion of purple teaming that we just heard is a terrible concept but you know security training from an offensive standpoint I think is important in order for defenders to know and understand what attackers do you'll get a lot of people on the defense side you know running edr tools or looking through logs and trying to early TTP's but without actually you know understanding what tools are being used and what kind of footprints they leave in the environment or why an attacker would want to use them from a you know methodology points of view into

why they would use you know this tool versus another you really get into their mindset and that helps you track down what they were actually going after how they operate so this you know helps answer a couple of questions like what do you look for in and seemingly endless cloud of blogs or if you're doing more of you know a threat model what parts of an app couldn't attacker hit or you know if you want to go back into the whole red teaming concept of you know constructive thought and being more analytical you need other people's perspectives not just offenders or defenders to answer that question of where can I attack and of course this

talk isn't about you know offensive teams and defense teams this is really about CTF so capture the flag I'm certain that a lot of people are now familiar with this since it's you know a concept that's being put into place across multiple security conferences and inside of organizations but it's you know an informal security competition it's it's the hacker Olympics where players are competing they're exploiting binaries or reverse engineering binaries or writing shellcode to you know essentially find a flag hidden in the service or on a on a website somewhere you grab that you cat it into a leaderboard and then you endpoints and the hacker with the most points was the best hacker in that particular case it

demo of course it demonstrates refrig proficiency or excellence in one particular area like if you look at the CTF that I'm currently running here most people are attacking the web side because web is you know the current hotness and that's where everything's really being attacked from an initial point of view but then there is other areas of exploitation or CTF such as binary cryptography programming that are often too overlooked in these competitions and this is where as an organizer you really get to choose which areas are being stressed over for a particular event so in are as we did a 4x4 so for shell code for binary for web and for reverse engineering there's others where it's you know entire

or entirely binary driven or if you look into you know how Def Con does where it's this attack defense model it's a lot of you know for fronted services and then behind those there's more vulnerabilities and it's more of a you know full-fledged systems view as opposed to you're just getting into web service with sequel injection or landing a buffer overflow so that kind of brings into the types so there's jeopardy which you've seen the show you kind of know what that entails that's what we're doing here at b-sides and then there's the attack and defense where teams are attacking each other services and some court of contained environment there is another concept beyond that which is called boot

to root which is exactly as the name suggests and this has really be become popular with vuln of and now hack the box which if you haven't heard of it or playing it it's a fantastic resource that just hosts you know a bunch of boots root exercises and you can go through the full you know initial access to a box all the way down to getting root on the machine and it's a great learning opportunity but you know these are intentionally vulnerable and if you want to take it into how I help train people that are on defensive to think offensively they like this whole cyber kill chain aspect and one thing that I really like attributing to this is you

have this you know enumeration model where you end map the host and then you do vulnerability discovery to see you know what's listening on it maybe it's vulnerable Tomcat or something and then you do your initial access your exploitation and then the big part of boots routes is there's a privilege escalation concept because you know you boot it you get onto it then you got to route it so there's you know typically a CV that's off the shelf that you can use and these are you know if you've ever played around with the OS CP the penetration testing with Kali Linux course the offensive security offers this is exactly what those are so I briefly mentioned the kill chain and you

know this is where you can really question my biases as somebody that does offensive but I like kind of relating this to the defense teams because they really you know it clicks when they're playing a boots route of okay you're at this phase you've done your enumeration you've done XYZ but then there's this fusion X kill chain that kind of took the miter kill chain and really brought it down and you know more of an adversarial view of you have this whole legacy cyber kill chain which is like The MITRE framework but then you do different things when you're both internal to the organization once you're on a box there's things you do past that but with boots arute and

where people are really getting a lot of you know steam with this concept is you're playing around with this internal kill chain quite a bit by solving these challenges which kind of comes into the shooting-gallery concept which is a form of CTF but it's not meant to be a competition it's meant to be a learning environment so you take the CTF idea and you build an internal isolated playground inside of your organization to build offensive security techniques and of course like I said hack the box is pretty much doing this but the unfortunate part with that is inside of your organization if you wanted to implement something like this there's going to likely be a huge barrier to

entry whether or not it be dealing with your corporate network and allowing you to SSH outside of it without doing weird proxying or maybe they're you know blocking particular ports you need to get onto these boxes and that's not really okay because you're not trying to teach people the intricacies of networking you're trying to teach people how to solve you know exploitation problems so I feel building something like this inside is massively useful that also opens up a lot of mentorship capabilities because you're able to do internal tracking monitoring of the environment and then the mentorship process becomes a lot easier because you can go into you know a conference room within your organization and just sit

down log into your infrastructure that you're the one managing that you have full control over so you know nothing's leaking outside and you can actually you know teach and get motivated and get excited about this stuff which is really awesome to see from you know the defenders because they and a lot of times are good at defense they've never done offense and things start to click of what they're finding when they do log correlation and whatnot and really the thing with this is it's reducing overhead needed to get interested in this and to have an environment set up such that you're able to do it relatively effectively in the open source implementation that I'm going to

be publishing it's just using things like KVM and libvirt which you know deployments pretty easy but not really because KVM and libvirt but you know you build a couple of scripts to like you know Buddha machine into the network throw it into your isolated subnet and then you're done and then you just need to import more VMS or more challenges and it's seamless I really like using vagrant which is you know you you get base box you spin it up you run one command and now you have a VM and that's really cool too because you get the redeploy ability aspect so if somebody is you know ruining a box because it's a multiplayer environment

you have the ability to really quickly revert or reload the VM and get it back to the blank state where it is exploitable and then the implementation I'm using which I'll publish is this whole notion of you use Open VPN and you privately connect into a subnet so this is the thing too right you don't want to have intentionally vulnerable machines in your network that's ridiculous we know not to do that so what you do is you can take a server put an open VPN server on top of it have everybody log into that and then it you know does firewall rules such that when you're connected into the Open VPN as a client you're now in a virtual environment that

you can't break out of and all of your VMs are hosted in that private environment so this makes it you know from an organizational standpoint and looking at with the security risk of having intentionally vulnerable security problems in the network that goes away there's a couple of things that you will need to be wary of though because you're gonna have this restart service in a in a tunnel in your private subnet you're not going to be able to as a player and this is kind of where I attribute the pwk and osep course as a huge inspiration because they allow you because it's a multiplayer environment to restart services with the click of a button you need to implement that such

that when three people are owning the same box and somebody does something dickish and their root and then they make it not solvable anymore you can be like well no restart and now it's solvable again and that solves the problem of this multi and multiplayer environment model you also if you want to track progress but not in a way that you you can sure it is competitive you need some sort of leaderboard I really like using C T FD for that which is also what the CTF here is using this year so I have source code for this it's going to be published shortly after besides I'm still waiting on open source approval but it'll be at this github

link and I'll tweet it out if you follow me on Twitter so as far as playing building and deploying challenges go participating in CTF and I've talked to a lot of people at this conference in the past and at other conferences where it's discouraging you see people at Def Con playing in the CTF room and it's a bunch of you know really smart people that have been doing this for years that are writing the tools that you use to solve CTF challenges and it's you know it's there is a barrier to entry such that people that don't necessarily have these skills are intimidated by it but really just do it there's a lot of really excellent babies first challenges

like if you've never exploited a buffer overflow before there's especially an AR CTF even you can go through and read Alif ones paper understand how it works and then land that exploit on our leaderboard and get a flag the whole notion of every CTF having this barrier to entry I think is per cuted in the in the community is oh yeah people that place ETF's are really good or they're the challenges are really hard it doesn't need to be that way and I feel b-sides is a really good environment to have more intermediate and amateur challenges that target beginners and not just people that are you know super lead at DEFCON CTF with that said it is a

good idea as a challenge organizer to have challenges that are going to be hard and unsolvable such as this year pwned 400 I'm going to talk about this in another slide with the notion of open sourcing CTF pwned 400 is not solvable in the way I intended it to be when I wrote it and as I was writing it I started thinking well it's a 400 level challenge people can solve the first three and then get stuck on the fourth and that's fine with the way that I'm doing it and where I'm going to publish a solution in source code you can go see and learn from that in a CTF that you participated in which I think has great

value there's also some resources here where you can read trail of bits actually does a fantastic job of getting people stoked on CTF and giving you very bare-bones here's how you get started and I highly recommend reading those and that's how I got started I mean quite frankly I'm not that great at exploitation but I really like writing and playing CTF so there's lots of resources out there for you and that's really where creating comes in a lot of people think oh the people writing CTFs must be good developers because they have all of these concepts of how the CTF is solvable and it can only be solvable in one way and that's not true

I'm a terrible developer I do security and red teaming I don't know how you know a lot of intricacies of a lot of programming languages work but the intention is your code needs to be intentionally vulnerable and even if it's not vulnerable in the way you and originally intended in your concept that's still okay because at the end of the day if somebody can land an exploitation primitive on your service or on your binary and they're able to actually solve it in a constructive way that's totally fine the way I started doing this was just by writing challenges and sending them to friends and saying oh can you solve this this is how I kind of think it goes and

then they'd come back and be like oh no you're an idiot it's solvable in a totally other way and it's like okay that's fine and then you learn and you keep progressing through actually being able to write these one thing that hasn't necessarily been done and this is something that when I was at Def Con I met up with the challenge organizers for besides San Francisco where I really model a lot of what I did this year on the b-sides PDX CTF is the content creators aren't open sourcing or showcasing the concepts and the the work of how their CTF worked there's write-ups aplenty and you can go you know Google whatever CTF imaginable and people are writing up how to solve

this the challenges more times than not the challenge organisers do not publish anything about how their infrastructure worked or about how their source code looked or about what the concepts behind before they even wrote the source code and what their intentions were and I think that's not good for moving the community forward in building CTF that do have this educational perspective and I'm looking to change that with the way we're doing it this year so I really want to see as far as creating CTFs go to open source challenges show concepts show solutions so you know and this is a problem that we ran into this you to where we didn't have time to make

sure that every challenge was solvable which is where we're in the problem with cone 400 but if you publish the solutions and you prove with your team that they're solvable challenges that really moves the event forward so I mean really as far as creating goes the first step is get involved people that are organizing these things aren't they're not gonna be mad at you for wanting to learn or to want to get involved it's open and it's super awesome and appreciated so if you're interested in creating come up and talk to me or any organizers that helped out this year or at other besides or others CTFs and mention that hey have this idea for a

challenge I'd like to implement it for your CTF and that would be totally awesome like I said I'm gonna open source the CTF as well and I'll tweet that out the CTF ends at 4:30 right but right before closing ceremonies and I'm gonna get push it pretty much right a ceremony start and you can start viewing the source of how the challenge has worked so as far as infrastructure in hosting goes this is really the most painful part of CTF and this could be a talk in like in itself the first CTF are really organized I did it in in VMs and I gave people you know dovey A's and I was I was talking to a guy that's like

oh yeah I've solved all I solved all of your challenges and I'm like oh great how did you get the flags oh you know I boot it into single user mode as a root and catted flag it's like oh so you know yeah if you want to cheat that's that's your that's you know you're not gonna learn the way that it was intended but also by giving people the opportunity to that to think Oh vn single user mode that kinda is defeating the intention of it so moving beyond that I ran a CTF where I was giving people I see a true to them I was running everything in docker but I didn't yet know how like kubernetes

worked so I gave people an SSH key they logged in and then they ran docker commands that I was allowing them to run and somebody broke out of it and it's like oh I probably shouldn't have done that either so now moving you know to where we ran this year and last year CTF it's all hosted in kubernetes in a secure way and if you caught the b-sides PDX 101 talk Yussef from Mozilla talked about how that infrastructure was ran which is also going to be open source and will be linked to in our repo with infrastructure and with creating CTF challenges as well there are organizers that are open to talk about it so one thing with legit bs running CTF

for the last several years at DEFCON veto now that they've stepped down after DEFCON twenty-fives has been blogging about how their team communicated and how they built and orchestrated CTFs as a team and it's been really valuable to see from the perspective of being a CTF organizer that didn't necessarily have the resources that we needed this year it was me and this this guy Eric who's sitting in the CTF room as well we wrote all of the challenges with somewhat of a constricted time frame and there were things that we could have done tremendously better and it really because of the help of Mozilla for our infrastructure if that wasn't the case we would have had an infrastructure that

was lackluster and that could have been easily pwned because kubernetes they're you know it's really easy to default queue brunet ease to being suck so we're fortunate for that with that said this hacker moon a hacker noon article is from besides SF like I said I kind of modeled this after there is because they were the first ones I really saw that open sourced all challenges all solutions and all infrastructure so I I took that as a huge inspiration and they did this whole walkthrough of how when they were going through and running kubernetes with their CTF somebody pwned the infrastructure live but was gracefully enough to tell them about it but like anything see TFS do kind of

need an attack model to be done to them what are we giving to the competitors if they were able to break out of the cluster what secrets are we leaking to them are there creds that AWS might still have in the cluster that now you can you know start mining Bitcoin so you really need to think about all of these intricate problems of the infrastructural standpoint that might get in your way don't let that be a discouragement like I said we're we're gonna open source the infra so you have at least a base reference to go off of which is the intention here so with that I encourage everybody to play CTF we are in the

second day and it is going to be ending at 4:30 but that's not that you can't come in and solve you know some of the 100 level challenges that are relatively amateur and if you're stuck on something I can direct you to a blog post that you can read and then solve the challenge and cut your first flag and you know have besides PDX ETF be the first time you ever scored a flag which that's happened so far and it's really awesome and empowering to see so yeah come come to the event room play help us organize next year's I would love to build you know a core organization team for the CTF to get it

even bigger with more capacity and more challenges to showcase so the kind of conclusion that and I want you to drive away from this is you can as an offensive security practitioner really instill knowledge and the defenders in order to get them were excited about catching you we're more excited about actually understanding the intricacies of how the tools they're catching work and why they're working or why they're being operated with CTF is also a really good way to challenge yourself and grow skills in this domain I encourage you to deploy shooting-gallery in your organization to have this you know self-contained resource within within in order to train and mentor individuals and really just go for go forth and hack

the blank deployment scripts are all gonna be on github I'm gonna tweet the link out CTF is going to be open source right as the conference ends when we give our prizes during closing so there are people I need to thank so CTF would not have been possible this year without the awesome team backing me so I had pwned P&W helped me with all the challenges and then Yusuf and Andrew helped me out from Mozilla on doing all of the kubernetes security I needed for the infrastructure so that's it if there's questions I'd love to take them I have like a couple of minutes left

you