CTFs for Fun and Profit: Playing Games to Build your Skills

this is the ground floor for day two of besides Las Vegas a few announcements before we get started you guys are old hat with this but we say it we say them every time anyway first of all streaming has started so if you could please remember to turn off your cell phones if you have questions at the end of the talk please use this microphone so that everyone on the Internet can hear you we'd like to thank our sponsors especially our inner circle sponsors critical stack and Bala mail valid mail and our stellar sponsors Amazon Robin Hood at secure code wire it's their support and all the sponsors that make this event possible so this talk is GPS

for fun and profit please welcome David Thomas shucks so I want to thank everyone that made it to the first talk of the day I know that can be a little bit of a challenge especially you know after maybe getting back into the Las Vegas party scene for the first time in a year so a little obligatory disclaimer the views and positions in this presentation are my own and don't necessarily reflect those of any of my employers past present or future the obligatory bio since this talks about CTF I've been playing CTF for at least 10 years I don't even want to try to count how many that I've actually played in but for the last several years I've

been on the staff of to what I consider significant CTF competitions one of them being at beside San Francisco and the other one actually being the pros vs. Jo's CTF here at besides Las Vegas which is literally in the room next door so I've seen both sides of the table and I wanted to share some of my views from that point in terms of employment I'm currently a security engineer at Google I work on our red team doing offensive security exercises and adversary simulation there I also occasionally blog and I sometimes post things on Twitter although not so much these days and of course the obligatory outline first I want to give everyone a quick

primer about CTF maybe you've never played maybe you've never had the opportunity before but then I want to get into really the core of this talk which is talking about the skills that you use and that you learn in playing capture the flag competitions it's really interesting to study one which of those skills actually apply to a career in security and to being a security practitioner versus which skills are more just about gaming and having a good time certainly there are both value in both of those but a lot of times people are looking at CTFs as a way to build their skillset for their job and so then I'll talk about how players can improve the

overlap and get more out of CTFs but I'll also talk about how organizers can improve CTFs and make them push more towards the educational side for the players participating in that CTF so what our CTFs right there's a bunch of different styles of CTF right so generally CTF capture-the-flag I imagine everyone has heard the term there's a few different ways that they can be run one of the most common styles probably because it's the easiest to set up is what is called a jeopardy style CTF and in a jeopardy style CTF as the name implies players get to pick and choose from a panel of problems that they want to attempt to solve next so it's not

linear usually the challenges may not even be related to each other they're each standalone so you can look at it and you can go mmm I think I'm gonna play Webb today and just go straight for Webb challenges and ignore everything else and for the most part those allow you to solve anything that you want in any order again just like the TV show Jeopardy right Alex I'll take Webb for 500 please on the other end of the spectrum you have what are called attack-defence CTF and in those cases you have actual networks that players involved in the CTF are either attacking or defending as the case may be or sometimes both and they're running

network services and often you have to find vulnerabilities in those network services and exploit those vulnerabilities and the network services and sometimes you even have to responsible for patching your own service in terms of preventing attacks on your system and then finally you have there there's some miscellaneous ones that don't really fit into the other categories right see CDC for anyone who had the opportunity to do that in college or may still be in colleges the collegiate cyber defense competition CCDC is essentially a defense only CTF they bring in professionals to attack the network and the college students are responsible for defending the network so it puts very much an emphasis on blue team skills on forensics on network

traffic analysis on firewall management on host management and all the other aspects that are come in the day-to-day operation of information security practice in a network there's also a few out there that are story style CTFs there's one pretty well-known called pony adventure 3 which is actually you play through like a video game client style thing and you actually go through a storyline with a little bit of a plot and everything there's also one brought by sans counter hack every year called holiday hack challenge that's online free-to-play and holiday hack challenge again has a storyline at a game client and everything like that so there's some different ways that you can approach these these things so it kind of is a

spectrum though when you think about it right like trying to categorize into precise categories is actually fairly difficult there's an entire spectrum of the challenges here from the realistic to the contrived so on one end you have real services a businesslike environment probably using real-world vulnerabilities CVE seen in the wild everything like that and at the other end of the spectrum you have really contrived CTF such as fictional architectures or services that serve no purpose except to have a vulnerability in it right like and I've built some of these challenges myself I've built challenges whose entire service is you send me shellcode and I'll run it for you right there's that's a pretty rare occurrence in the real world

I know some of us may feel like some vendor services are designed to just run shellcode but that's not usually the reason for their existence so in terms of contrived CTF sorry DEFCON CTF is probably the best known CTF in the world right it's top tier the players who are in it are absolutely wonderful but it's a contrived set of challenges it's a game it's for the purpose of finding out who's best at playing CTF not necessarily who's best at being an information security practitioner and if you ask the organisers they're totally upfront about it right it's not intended to be a test of practitioner skills a couple years ago they in fact invented their own

architecture for DEFCON CTF and handed players the ISA the the instruction set architecture manual 24 or 48 hours before the beginning of the game and this architecture was Middle Indian so the order in which the bytes were being considered was from middle first and it used nine bit bytes and a number of other weird architectural features and that's really cool to see who can wrap their head around a new architecture and wrap their head around porting their existing skills and their existing toolkit to this new architecture the teams were building just their own custom disassemblers their own custom decompilers their own custom exploit chains wrap chains everything had to be built from scratch for it but at the

same time having to deal with the middle Indian architecture is not something that security practitioners find themselves needing to do because as far as I know this is the first time anyone has ever built a Middle Indian architecture anywhere I do have to give mad props to the DEFCON CTF organizers though because I guess they had to build their and type their own virtualization system to run this entire thing and design a functional CPU architecture so I have to believe that years of work went into that I also give Def Con a lot of credit this visualization is actually a visualization that they use during DEFCON CTF these days it looks way more than actually watching a CTF if you've

ever stopped by any of the CTF areas at a conference or something like that you know it's just a bunch of people hunched over their laptops staring at a screen it is not what movies are made of but DEFCON CTF has managed to add a little visual interest for the audience Perez versus geo CTF as I mentioned that's a CTF located here I wanted to compare and contrast that to DEFCON CTF especially because they run in the same week there are only a couple of days apart it's defensed focused but not exclusively defense so our teams here the the Joe's teams start off as blue teams and for the first day their goal is to defend their networks and there's

four teams they each have their own network and they each have to defend it and then there's a red team of pros attacking that network for the players to defend however today the second day of the CTF we let the teams get a taste of blood themselves and they get to go purple and attack each other's networks some people think that pros vs. geo sounds like it's just going to be a bloodbath but it's not about and from a scoring perspective it's not about are you beating the the pros or you not it's about which of the teams is doing the best at defending their network and at attacking the other networks so it's not

about out swimming the shark it's about out swimming the other guy it uses entirely a real-world software and environment it's just maybe like a corporate environment where that software hasn't been patched in a while or is configured poorly I'll go ahead and tell you the root password for many of the machines is ABCD one two three four hopefully the teams know to change that pretty early on like rotating credentials is an important step if they didn't and any of you guys know anyone else out there you may want to tell them that they should consider changing their passwords and so this actually real services like I said FTP HTTP SSH real world services that are actually used a

Samba SMB on Windows Windows domains real worlds very real world I also want to touch base on war games war games are sets of challenges to be solved they're not a real-time CTF they are more of do it at your own pace kind of situation so they teach a lot of the same skills and can build a lot of the same skills but you don't have to dedicate an entire weekend to it right most CTF tend to run for like 48 hours they tend to be on a weekend and I mean some of us we do like sitting there staring at our screen for 48 hours straight but I understand that that's not everyone's cup of tea so war

games let you come in sit down for an hour two hours give it a try and then you know put it off till next weekend so now getting to the educational value of CTF sorry how can you best use CTFs to build your skillset and really grow your career path through get these games well can you learn from CTFs right like just as a general question is there a learning opportunity here well the short answer I give you is yes you can learn from CTFs the question is how much will you learn and what will you learn so in terms of what will you learn in a typical CTF the kinds of things that you're actually going to be exercising

are entirely your technical skills and most CTF focus on reverse engineering exploitation occasionally you'll need to use some forensic skills and very often you'll find yourself needing to write a script there have been many CTF challenges that are if the format like solve this a hundred times in one minute no player can do that just manually I've seen ones where you just needed to play a hundred games of tic-tac-toe in one minute and even though you may be really fast that's still a pretty big challenge CDF's are also good at helping you think outside the box are a challenge authors tend to be creative and they're trying to encourage creativity from the the players they very often want to have

challenges that are not just the same old thing and not something people have seen before so though things that encourage players to think outside the box that encourage players to think of new approaches to old problems or to learn new things that they hadn't seen before but it really depends on what you're playing whether or not you're going to get much out of that but as a practitioner what skills are you using in your day-to-day environment sure you're using a bunch of technical skills right some of us may do reverse engineering exploitation forensics but there's also things like threat modeling triaging I mean many of us are under-resourced and don't have enough people working on our team in

order to accomplish everything we'd like to do so figuring out what comes first second third is important in terms of a skill that you can learn they're an actual program and note that I differentiate programming here from the scripting that you might do in a CTF right if you're building something that's actually going to be dependent upon by your enterprise or by the environment that you work on you should probably put a little more effort into it then you would put into a solution for a CTF challenge although I've definitely seen code that looks to be at about the same level

there's other skills that are outside of the technical realm but are very important to a practitioner as well one of those being the attacker mindset right some people think of this as a technical skill but I really don't think alike an attacker to us thinking how they would come after a system but also thinking what their motivation is what their end goal is and there's different types of attackers with different end goals and different motivations and getting inside that hat that mindset is really important as a security practitioner and it's not something that you will typically see happening in a CTF right like in a CTF your goal is exploit X get the flag and get the

points attackers are not looking for flags attackers are looking for particular access to data particular access to systems and you have to understand their motivation as to why they're going after something and what it is that they're going after also being able to take multiple approaches to something right in a CTF there's usually one way to get the flag at least if the challenge author has done a good job there is one defined path to the flag in the real world a single service may have multiple vulnerabilities that you can attempt to exploit a single system may have multiple services running on a single data store that you're trying to get to you may have

multiple systems capable of accessing it so from the security practitioner point of view you have to look at it and think about whether or not the path that you are currently pursuing is the right path to pursue when you play CTF usually there's only one path that you're pursuing at a time another skill that's absolutely critical to the security practitioner is communication skill most security practitioners I'm sorry we have to write reports I know most of us don't love it I don't love it but it's an absolutely important part because it's really important to communicate particularly to non-technical individuals or non security aware individuals what the impact of a vulnerability may be what the impact of their business decision to

not use TLS may be what the impact of any number of other choices that are made could be as well as communicating to them the steps that they need to take to remediate issues right I've seen people file bugs and if you've ever participated in a bug bound and you'll see some of these where they come in and they just say there's a cross-site request forgery here well it doesn't tell you anything about the impact of that cross-site request forgery is like is it a cross-site request forgery that results in the user being logged out or is it a cross-site request forgery them it results an account takeover right those are dramatically different circumstances and being able to

communicate this is an absolutely critical skill and again it's not something that's traditionally exercised in the capture-the-flag space also teamwork a lot of CTFs are team based but at the same time what you'll see happening is it'll basically be just a division of labor the teams will sit down and one person will take this challenge and another person will take the second challenge and a third person takes a third challenge and I hate to break it to you that's teamwork only at the most rudimentary level of the the skill to true teamwork involves being able to work on a single problem together being able to bounce ideas off of each other being able to mentor

people who don't have as big a depth of skillset as you do in that particular area just divvying up the labor if not a great approach because what happens when your web guy runs out of web challenges to solve or in the real world what happens when you're tasked with performing an application review of a monolithic application and you're told you have three days to do it but it's okay you have four people you can't just cleanly divide the application in most cases you need collaboration you need cooperation in that space so there's some overlap between the skills as I discussed I largely think of the skills that are involved in CTF for the most part as a subset of security

practitioner skills there's very little that I would say is outside of the practitioner realm even things like steganography and obscure technology may occasionally be useful in the practitioner space but a lot of the things genuinely are useful to a to a practitioner but then there's a lot that's not covered by the CTF space right being a security professional is so much more than being a successful CTF player the flip side to this is not every security practitioner is going to make an amazing CTF player their skill set may more closely align with the areas that are outside of the overlap here but in terms of getting educational value from a CTF what our goal is to do

is to maximize this area of overlap right either as a player or a CTF organizer if you want to be able to use it to build your industry skill set then maximizing that a late area of overlap is going to be the best way to get the most bang for your buck so let's talk about communications I said that's a really critical skill and I genuinely believe that that is a critical skill especially if you're in consulting but even in an internal security team I think it's really important that you are effective at communication one way to do this is doing what are called write ups for CTF challenges some of the bigger teams do this quite a bit and what a

write up is is a description of what the problem was the approach you took any dead-ends that you may have encountered along the way and what your ultimate solution was and being able to explain these things in a term that someone else can understand actually build your communication skills a lot I find that when I have to explain something to someone else it forces me to stop and think about areas of the problem that I didn't really think about there may be things that just happen to work but I don't actually know why they worked right and in the heat of the CTF I don't and care why something worked I got the flag I'm moving on but when I go back

into a write-up I'm like why did I come across that as the base address that I was using for this right it happened to work but I don't really know why so going back and doing it also reinforces the technical skills that I used in there but being able to explain it to someone and break it down step by step really builds your communication skills some of the higher-level CTFs actually require the winning teams to be able to prepare write-ups for their challenges this is also done as a way to ensure that the work was original for those teams and that they're not just copying off of someone else or sharing solutions I also think it's important to include

how the vulnerability would translate into real world impact right if this is a service running in a real enterprise environment what would the impact of a vulnerability like this be because I think it's a useful learning lesson to people coming into the space to be able to see these write-ups understand vulnerabilities understandable 'nor ability classes and understand the impact of the vulnerability so you can say oh if this had been a real-world service this would allow full account takeover on this service other vulnerabilities allow remote code execution on the the machine which of course gives you access to any of the accounts in there right which is a different scope but for purposes of a

CTF both of them may Ernie the flag that you're looking for in terms of teamwork and leadership like I said it's not just about dividing up the effort I mean first off play as part of a team quite honestly CTF is a lot more fun if you play as part of a team sitting there even if you're not physically co-located with the other members of the team having someone else to communicate with having someone else that you know is just as frustrated as you are having someone else who's beating their head against the same challenges you know maybe it's a little bit of schadenfreude but I do love having that support system there in place and you can work on

challenges together it's really good when you have someone else that can be a sounding board maybe you can explain things even what's called rubber duckie debugging if anyone's familiar with that sometimes you start explaining something to some and halfway through the explanation you suddenly realize what you were doing wrong because you're forcing your brain to slow down and think about the problem step by step so even just being able to sit there and go okay so I was doing a B and O really helps but other times you can explain to someone and maybe they will just say you know have you tried looking at this and it'll be something you hadn't considered but that'll be the

breakthrough you need for the solution it's also a great opportunity to get experience working with other personalities or other backgrounds or things like that some people on a team may be really assertive and going after certain types of challenges and other people are not so much and are sitting towards the back and you get this sort of thing that you have to get accustomed to in the work environment working on a team with people of finding out what their strengths and weaknesses are and how to bring them into the fold and get them more motivated to work on the challenges you're working on it's also a good opportunity for mentorship many of us have built our careers through

mentorship in one form or another right a lot of us may have gotten our first security gig by knowing someone else in the security field a lot of us may have met someone at a con or something like that playing on a CTF team there's a really large CTF team called open to all and as its name implies it's open to anyone who wants to play on their team and I know at least two members of that team who have gotten their current security roles by other people that they knew through this CTF team so even if you're not getting value out of the CTF you may get value out of the CTF team and out of

getting to know the people who are there conversely if you're looking to build your mentorship skills if you're looking to move into a leadership position in the future or something like that this is a low-risk way to start getting involved in mentorship right you don't run the risk of HR problems you don't run the risk of conflicts within your work your your work team you have this opportunity to just get to know the things that are involved in helping someone else build their skill set in their career so what are some other ways that you can get more out of it right stepping outside of your comfort zone is important this may actually reduce scoring at the time that

you're playing it's very easy to fall into the trap of saying well I know Webb really really well so I'm only gonna focus on Webb challenges or a no exploitation really really well so I'm only gonna do exploitation and that's great if you want to win the CTF that's probably the best strategy for winning this ETF the flipside is if your strategy if your goal is to build your skillset and to grow yourself that's not a good way to do it continuing to do the same things that you have been doing and that you are ready excelling at is not a way to expand your skill set stepping outside of your comfort zone is how you

learn things as I mentioned earlier professionally I do offensive security adversary simulation so I love doing forensics challenges and CTFs it's not what I do at work it's a nice break from that sort of thing but it's also something that I'm learning new things in right like learning how to recover artifacts from disk images learning how to analyze pcaps learning how to do all of these kind of skills and then it turns out some of those skills actually translate it back into my job getting better at pcap analysis for example helps me when I'm attacking a system with a protocol that I don't understand because I can start doing more pcap analysis and more in-depth analysis of the network traffic

that the device or that the service is sending so stepping outside of your comfort zone even if it's not immediately obvious what you're going to get out of it can pay off in the long term also consider revisiting challenges maybe there's a challenge that you weren't able to solve maybe there's a challenge that you ran out of time to solve I know that I solved the most challenges in CTFs five minutes after the CTF ends but it's a good opportunity after the CTF has ended you can go back and take a look at those challenges without the pressure of the clock running down on your back maybe you'll have some time to sleep or get a cup of coffee or whatever

else you need to clear your mind and then you can come back to it and maybe by then some write-ups will have been published and so if you're there's a particular area you're stuck on you can see how other people have solved this and you can take a look at that and you can go back and say okay so this is how I miss this step like what can I do from there on you can either just use their write up as a a little bit of a hint to get you past that one point or you can look at their full exploit chain or their full solution and just understand the challenge I really like to just get

past the point that I was stuck on and try to solve it myself very much like a learning-by-doing kind of guy and I feel like that's the the best approach but maybe you learn better by reading these write-ups and understanding what it is that someone else did and how the approach they took likewise even challenges that I successfully solved I love reading write-ups for because sometimes you find a different approach and in the future that means that you know a better way to do something that you already did so I've had many occasions where you know maybe there was a stack cookie and I brute-force the stack cookie which is noisy and takes a while but it eventually gets to the

solution and then someone else points out oh yeah the stack cookies leaked in this particular position here knowing that and being able to know what to look for in the future and to go back to that is absolutely useful in terms of building your skill set because it improves what you're able to do there's a website CTF time CTF time has basically become the de-facto directory for CTF related information it started off as basically a shared Google Calendar for CTFs but now it has CTF rankings has C CTF ratings and most significantly it has lists of challenges and write-ups for it so it's become sort of a central repository for seeing the write-ups from in each of the

challenges right and they're things that you'll find on there you'll either find the entire write-up published on CTF time or very often you'll find a link to like a blog post or a github gist that explains how someone was able to solve the challenge that they're looking at if you're more the visual type be if you like to learn from videos and from audio there's some great YouTube channels as well live overflow I'm a huge fan of that channel he covers both CTF content but also covers real world xsplit exploitation things he's got tutorials on using Ghidorah the new reverse engineering tool from the NSA because we all like to install an SI software on

our machines as well as Gonville cold wind he's the captain of dragons sector just one of the top CTF teams in the world according to CTF time ranking and according to DEFCON CTF the results and he has also a YouTube channel where he goes through and explains how CTF challenges were approached by his team or by him in particular you'll find some other ones as well I think some of the members of the Carnegie Mellon CTF team Platt Parliament of boning aka PPP also do some YouTube videos of how they solve things so it really depends on your learning style whether write-up or a video is a better approach to it but they're both available out there and

they're both great ways to get an understanding of something you didn't understand while approaching the challenge so let's flip the coin a little bit maybe you instead of being a player are on the other side of the table and are designing and building CTFs like I said I've done both and you want to make a CTF who that has a goal of being educational for your players and not just a challenge for your players and the two are not mutually exclusive but they're definitely ways you can take steps to get more out of learning opportunities that you're presenting for your players so there's a couple of different approaches to educational CTFs there's Publix ETFs maybe running a conference or a

hackerspace or even just run as an online CTF there are many CTFs now that aren't even affiliated with any particular event they just run all their infrastructure in the cloud one of the things you'll have to realize is you're going to get a huge range of backgrounds and skill levels right like running the CTF at beside San Francisco we see everything from people who have black badges from DEFCON CTF all the way down to someone who just found out about like getting into the security industry and this is the first time that they've ever heard of a CTF like literally they happened upon it in the room and are gonna walk up and sit down and try and play and we

try to make sure that it's interesting to that entire spectrum of people which is really really really hard now no single challenge will be interesting to that entire spectrum I hate to break it to you that's just not gonna happen but having enough challenges at different skill levels can be interesting to them you also various interests in various learning objectives and your audience right some people really want it to be hardcore classic CTF problems exploitation they just want to get shells and cat flag submit the points and be done with it and then other people are coming there because they do want to build their skill set because maybe they're in IT Help Desk and they're trying to break into

security and this is their first opportunity to play around with some of these security things maybe they really want to learn about web exploitation maybe they really want to learn about binary exploitation maybe they want to learn about exploitation on weird architectures I've run challenges on arm and MIPS I've never run one on Middle Indian though I just can't bring myself to do that alternatively you might be running a private CTF and I've run some of those at my employer and other venues as well and these are more targeted to particular skill sets and very often to people with a similar background right so I've run CTFs for example that our security awareness CTFs where we

present a CTF to developers but they're not security professionals but we specifically present web security challenges for example because they are web developers and we want them to have a better understanding of these web vulnerability types whose names we throw at them when we say hey you have an XSS in your service we want them to understand the implications of that and I think a great way to help them understand that is to show it to them in the form of a CTF these tend to be more focused tend to have a more similar background so you can really narrow what you're doing in terms of building the challenges outright and this is again a

spectrum some of them will be you know more or less one way or the other for example there's a great CTF called so hopelessly broken right in the IOT village at DEFCON and some other conferences and but it's the IOT village so surprise surprise all of their challenges have to do with IOT devices and things like that so they have the sort of narrower focus but still it's a public CTF different backgrounds different skill sets so gamification right I've been trying to avoid this word because it's been a buzzword for a while but it turns out that CTFs are what education professionals call gamification of Education gamification is basically where you motivate students to learn by adding game aspects to the

learning process and it actually turns out that there have been many studies that have been conducted in psychology and education in other fields that have shown that gamification improves learning performance and skill progression it also increases retention and has shown that players are sorry player students feel like they have put less effort into learning the same skill as they would from just listening to a lecture reading a book something like that so it turns out that if you're having fun when you're learning something it's easier to learn it sounds super intuitive but apparently you can get research money to improve super intuitive thing so I'll come back to to the practitioner skills we try to teach in pros vs. joes

pros vs. joes as explicitly has education as one of our core goals that's why it's the pros versus jose model and we look for jose in that game we're looking for people who are students who are new to their career who are making a career transition that is what we mean by joe is we mean someone who doesn't have the ten years of experience that some of you may have in the security industry the environment is reasonably realistic I put quotes around it because it's hard to simulate a full enterprise environment but there's a Windows domain with domain attached servers and clients there's Linux servers it's varying versions you'll have Windows seven eight ten we finally

killed Windows XP from the game environment even though it was a lot of fun it was actually getting really hard to keep running Windows XP VMs on modern virtualization and it has real services in the environment we have mail server so we have DNS servers we have file servers web servers you name it anything that you might see in a real-world business environment sequel server my sequel is running in this environment that the players have to deal with and one of the key things is we have a score bot that checks these services and actually interacts with them so one way you can make a totally secure network is you can just turn the network off but if

you have the scoring based on whether or not these services are available and responding it makes that a lot harder for the players they can't just put deny all in the firewall rule and then grab a beer [Music] so how do we help the players to build these skills well we have the red cell pros as a sparring partner for the Blue Jays right so the initial start and pros vs. Joe's the players are trying to defend their networks they are hardening Network firewalls hardening hosts firewalls changing passwords disabling unnecessary services patching services all of the things that you do as a security practitioner or that you ask your IT department to do depending on

your environment but condensed into a very very short timeline and the red cell pros are there to apply pressure they're there to be the opposing force and to encourage the players to do all of these things but because we don't want to just throw Joe's in the deep end and give them no guidance every blue cell comes with two pros which are security industry professionals most of whom have also played our pros versus Joe's games before and they serve as mentors and leaders they go to the role of the blue pro is not to do it for the players but to help the players do it and help them understand what to do but why they are doing it right help them

understand the value of the firewall rules that they're implementing help them understand the value of the gpo policies that they're pushing to the Windows domain right not just blindly hey do this one magic PowerShell command and all your security problems will be solved but here's what it actually does but even as a pro you learn new things I've been a blue pro before I moved to the Gold cell which is what we call our game administrator team and I was a blue pro despite having a job that is very much on the offensive side and I learned a ton of new things even from my Joe's in the environment because some of them are expert Windows domain admins but

they don't know anything about Linux security or they come in and you know they can look at a Palo Alto firewall and it's magically configured but they don't know anything about host security so they're still learning new things but I'm learning things back from them as well so it's really great from an educational mission but one of the best things about it is at the end of each day of the thing we have what we call the hot wash which is where the red cell divulges what it is that they have done to the blue players the end of the first day red maybe holds back some of their information it is a game after all but at the end of the

second day red is fully open it's an open book how many of you as security practitioners get to compare notes with your opposing force with your adversary right you get to find out not only what it is you saw how they did that but you also get to find out what you didn't see on the network what you missed and what you could have learned from for the next time and finding out where it is that you have your blind spots I think is a wonderful way to figure out what it is that you still need to learn so let's say you're building a CTF and you want to build some educational challenges into it right one of the ways that I

really like to do this is building what I call progressive challenges which is a series of challenges that introduce new concepts or complexity as you go through and you build up skills rather than requiring a giant leap so very often you want to build these with challenges that have some kind of real-world applicability so based on real CVEs based on real forensic situations that you've seen if you are building CTFs and you do forensics though don't just grab a peak app from your live network and throw it into the CTF that's probably not a great option but you can build a clone network you know use a few virtual machines and do the the same sort of

thing so based on some real vulnerabilities for example an Android application with sequel injection via Android intents so that another malicious application can start reading data out of the private data store from the first one or just using realistic environments right forensics of a real system build up a VM image use it for a little while include browsing to a few pages that contain the flag or something like that in there and then take the disk image of that and give that to the players or fully functional apps right rather than building an application that is just the vulnerability wrapped in a thin layer of an application you can actually take an application that is a

real-world application and introduce a vulnerability into it as a CTF challenge so that way these skills of hunting for the vulnerability and of figuring out where the vulnerability is which are much more applicable real-world practitioner skills get exercised by your players so like I said the progressive challenges I like to do approximately three steps introduce the concept add a little bit of a complexity to the second step and then force an edge case or something that really steps it up at the end so I expect everyone to be able to get the first challenge there's enough information there that you can definitely get it and definitely get introduced to the concept the second challenge should be achievable by most

players if they're willing to put in a little more effort the third challenge will be a stretch goal for players that have never seen that concept before but your more advanced players may actually still find themselves challenged but accomplish the the third one and so this gives you not only a challenges for your range of players it also boosts player confidence because players can burn out quickly if all you put our how hard challenges and so including some easier challenges in the form of the progressives helps people have a positive feedback loop and helps them have a positive outlook about their progression through the challenges and it also builds the skills not everyone may get to the third level but they will

still get something out of that experience so as an example I've built before an encrypted file system where the first one is an obvious encrypted file system like a Luke's volume with known partial password right we know that this guy was always putting his carnate like you know the fact that he has a Porsche and his passwords so it's very easy to brute force so then the second one is maybe a file system without a known password so players have to think about password lists and where do I get good password lists and if you download RockYou you'll probably be happy and then last one is like an encrypted file system with a harder password that

requires password mangling and then inside it there's deleted files that need recovery you know I'm really throwing some curveballs in there right normally you open encrypted file system you see the files there well this was an empty encrypted file system unless you started doing data recovery from the encrypted volume and knowing how to throw away noise when in doing an encrypted volume is incredibly important because you get a bunch of junk back when you decrypt empty sectors an alternative is a sequel injection I actually did a for stage one with obvious sequel injection that would just spit back the entire query it was trying to run in the error message if you had a syntax error so that players should be

able to just fix their syntax slightly by looking at that error message and then on the next step I just tell you yeah it's you know bad requests 500 internal server error or something like that don't give you the error message but still give you obvious feedback when you're sequels not working and then move on to splined sequel injection and introduce that as a concept and then finally a blind sequel injection that was hidden in like a base64 encoded cookie so they have to start thinking about how web applications work and the fact that there's more than just form inputs as inputs to a web application so this builds up the skill set going from

the obvious to what should be the less obvious so what motivates players send these things right the competitiveness the scoring what we talked about is the gamification works for a lot of players but other players are motivated by a progression or a storyline which in CTF like the holiday hack challenge or poni Adventure 3 they have that aspect to it they want to see what comes next rather than caring about their score compared to other players and finally some players are actually just motivated by the skill building in which case these progressive challenges can work really well for them because they can actually see the progress they're making as they go through the series of related

challenges it's also nice to have categories or tags for your challenges if you're doing jeopardy style because then players can see that they're checking off in a particular area and growing their skills [Music] any questions about approaches to CTFs as a player or as an organizer an experience with applications like boot to route type challenges or doing absolutely so the question was about these boot to route challenges which are virtual machines that are pre-configured with ulnar abilities in them and as the name implies you boot the VM and then your goal is to get all the way usually to a root shell or something like that on the on the machine I think that they've generally fall into the same

category is what I refer to as wargames they're not usually time-sensitive and one of the aspects that they lack in terms of player motivation is having that scoreboard to compare yourself against other players because a lot of people are motivated by that I think they can be useful the problem with some of the boot to route challenges is for whatever reason people shortchange themselves sometimes on them by cheating since you have access to the whole VM and it's locally hosted there's usually ways around them to like just get straight to the flag I don't understand why someone would do that with with a challenge they're doing just to solve to learn for themselves but I've actually

seen people do that and then like post a write-up on like oh here's how I got the flag by like instead of booting the VM I mounted the filesystem attached it to like my Cali box and then just went in and got the flag and I feel like they're kind of cheating themselves but I do think they're interesting particularly if you're trying to learn a particular skill to find relevant boot to route challenges and use those as a skill builder hi my University's cybersecurity Club would really like to start sone CTF team some challenges we face are that students get disinterested in security when they learn about it they get like really discouraged from competing a

particularly hard CTF students also have to teach other students which is a challenge in itself so do you have any advice about building a CTF team or like starting like a good culture for students so they're not discouraged by CTFs especially in that university context yes I think a good way in terms of getting people started especially when you have others who could potentially serve as mentors even if not full teachers is actually the wargame approach there's like over the wire is a wargame site that has a bunch of different war games on it and it has ones at different levels of difficulty so I think that early wins help keep people interested right if they get into

it and immediately are just beating their head against a wall and are unable to make progress it's really frustrating so finding easier challenges and giving them the opportunity to do some of those war game style challenges definitely is helpful since you're in a university environment being able to play in person I find is also motivating to a lot of people I don't know if you've tried that like playing some CTF and then you can also just look at some of the CTF s there some out there that are easier CTF in general than other ones so if you try that and organize it for better for worse I'm sure you already know this not a hundred percent of people are going to

turn out to be wanting to play CTF even as security practitioners and not a hundred percent of people who think security is awesome are going to stay in the industry burnout is a problem industry-wide and I think cyber security club is no different than that

early on you said there should be one path to the flag I was wondering why so I didn't if I said that that was not what I was trying to imply I was saying that CTF writers usually try to put one path to a flag into their challenge because they usually are building it around this one cool vulnerability in their head when they're writing a challenge right so I'll sit there and I'll be like all right I have this great idea like an off by one memory corruption that lets you overwrite some metadata on the heap or something like that and so I'll build it around that and so usually you end up with just this

one path to the flag and I actually said that's one of the liabilities of CTF ss an educational model because in the real world most applications have more than one path to the data that you're looking for even like as a red teamer why exploit a service if I can just steal creds and get it that way right like I take the easiest path to the to the outcome but it turns out that the reality as most CTF challenges have one intended solution which isn't to say CTF authors are not perfect I've written many with unintended solutions and there including accidentally distributing binaries with the flag still in the original binary so there's plenty of

ones with multiple paths but they're usually designed around one path [Music] so as a pleasent developer one of the biggest challenges I find is evaluating a puzzle you know you think something's gonna be super easy and turns out to be super hard for people do you have any advice on evaluating puzzle challenges yeah so something we try to do but as anyone who's developed a CTF knows the 90% of the work gets done in the last 24 hours but something that we try to do is we try to keep some of the other author challenge authors kind of in the dark about the challenge until close to the end and then we ask them either to play

the challenge or if we run out of time we walk them through the challenge and ask them conceptually how hard they think it will be another thing I'm experimenting with for B sites SF CTF next year so check out how it works we're actually going to be trying dynamic scoring where instead of setting a fixed number of points to each challenge we're gonna be awarding points based on how many teams solve a given challenge so we don't have to worry so much about making the difficulty correct the score will come out to be correct to the difficulty of the challenge as our hope essentially grading on a curve yeah we think we think actually so that way

the challenge that gets solved a hundred times and the challenge that gets solved once are automatically appropriately scored so we're interested it's an experiment we're interested to see how it works out and play in that CTF it'll be available online and you can give us feedback on whether or not that's a good scoring mechanism I've received the stop sign here so thanks everyone for coming and enjoy the rest of your piece sides

[Music] [Applause]