Security Learns to Sprint: DevSecOps

hello everybody welcome in thank you for coming in today we have a really cool talk by Tanya she's also known as she hacks purple she's an independent security consultant specializing in software and cloud security so please give her your all your attention and focus while she gives their top hi okay hi everyone thank you for coming I am Tanya Jenkins day we're gonna talk about how security needs to adjust ourselves in order to fit in with DevOps because otherwise they'll just sprint away from us and keep doing DevOps without us and we don't want that so a lot of what we're going to talk about is how security is everybody's job and how as security professionals it's our job

to enable every single other person to do their job securely and so I made the sticker at one of my offices so that everyone could be reminded all the time and have the little raccoon stare at them I mean help them okay so the mandatory okay so where we going to talk about DevOps security becoming a part of DevOps which I like to call dev sec UPS please feel free to fight with me about it later on Twitter and this is how some people see security in DevOps is how some security people see it but that's not what I see this is what I see I see us and dealing developers and ops folks to creating more secure software quicker

and ahead of all of our competitors this is what I see and so let's talk about that okay so the mandatory about me slide this is the we are qualified to give our own talk slide we do this so that you stay the whole time but you already waited in line so I assume you're gonna stay so I'm a giant nerd I do all the things and I just start giving training on the Internet yay so I do lots of stuff I love o ASP I will talk about them incessantly if you let me all right okay so let's do this so first of all definitions what is app sec besides my favorite topic it is every and any

activity that you do to make sure your software is more secure right so that could be I'm doing a code review specifically to make sure that there's no security vulnerabilities but it could also be knowing that there's this old function that now has a vulnerability in it and you want to not allow it anymore so you just rep through your source code find and replace it am I am I going like way out of the line of the camera I didn't think to ask about that anyway okay I'll try to stay still okay other problems poor application security is a big problem we have been the winner according to the Verizon data breach report 2016 2017 2018 and spoiler

alert 2019 we have been the number one cause of breaches in secure software so that's very expensive and very bad I'm sure you all know that but until I did this research I was like oh god we're actually the worst so we're doing the worst job in security okay so we have work to do okay so also application security is missing most people who have gone to school have told me they have had almost no security training or if they did have security training the training that they got was not enough so they took computer science but they got nothing on secure coding all they got was this is what identity is use Azure Active Directory and that's

it there's so much more to writing secure software than that so Security's also hope numbered see her she's all alone and sad by herself that's me on my app sect team I'm just like it's just me and then there's hundreds and hundreds of lovely developers who I feel overwhelmed by right so being outnumbered means we can't just work harder we have to work smarter these are apparently the statistics in our industry I've never worked somewhere where my odds were that good I have never worked somewhere where there were only 10 - ops people and only 100 developers for me for each one of me so I want to improve this but for now let's talk about that one person just doing a

more effective job ok so the last thing on this I'm going to talk a little bit about water feel sorry waterfall um the security model for waterfall totally stinks it's not working and that's why a lot of companies are all moving towards the DevOps model because it gets suffered at the door software that's more resilient software that's higher quality software there are clients like better our customers love and we need to get on board with this so that we can make sure that all of those things are revolve around a product that's also secure boom so what does step set cops so my friend Imran Mohammed who is this awesome dude who he's like Tanya it's

what we've always done it's just that SEC except for in a devops environment it's just that's like changing stuff around he's like that's what you do everywhere you work every place you work they do application development slightly differently and you just adjust to make sure that this stuff is just the same except it's DevOps it's like you make it sound really easy and mom okay so now let's talk about DevOps who here feels super brave and wants to tell me one of the main three goals of DevOps anyone feels super brave today you're at B sides yes anyone else want to feel brave Oh what did oh sorry he said more deployments like faster deployments continuous feedback yes also

faster time-to-market also increasing the efficiency of the entire system not just your part getting things to market faster than your competitors that's like the end of that sentence so these are these are the main goals and let's talk about them employed improve deployment frequency thank you sir so as I have to do a security it means you can fix my security bugs now I worked somewhere once and I kid you not their deployment frequency or like once you ask for something to it being deployed with 16 months they explained that my absolute emergency the sky is falling change could absolutely with lots of overtime be completed in four months um Laura fill your rates so I feel this one's

really obvious but let's talk about it so a lot of people talk about resiliency that's why a lot of people move to the cloud they don't want things to not fall down but who remembers this from school or maybe not from school but who remembers this the CIA triad if we look back at here resiliency means if it's always up it's always available and that means your customers can always go to your site and you don't have something embarrassing instead like a stack trace so that is where that relates to security and is in line with the same goals and then the last one faster time-to-market I bet love you were thinking like that has nothing to do with security guess

what if we don't get to market quickly we don't like create products that delight our customers we go out of business and then there is no security team I might sound ridiculous but like it is our job as security professionals to enable everyone else and I have worked at a lot of places where the security team was the number one risk to availability and so we don't win if the business doesn't win all right so this is so I love a wasp I talk about a lot all the time and all my talks and in my opinion DevOps is the best thing to happen to the security of software since I ASP I wasp just for anyone that

doesn't know is the open web application security project of a chapter here there's chapters in almost all the cities across America and all over the world and I love alas but I don't like it I love I love my community and so this is the highest praise I could give anything so now let's talk about DevOps in the three ways so I'm gonna explain what DevOps is according to me and also the DevOps handbook and the people that invented DevOps so what are the three ways of DevOps does anyone feel brave again so there's someone up there scratching his head and thinking of trying to get him to answer okay the three ways of DevOps the first

one is to increase the efficiency of the entire system so if I make my part way way way faster but then his parts still slow and we're all still waiting on his part I should have just been helping him the second way is making sure we get feedback as soon as we possibly can the big thing with waterfall and my so many projects failed is because we didn't get feedback until after a year of programming and then the customer is like that's not what I meant and then the last way of DevOps is continuous learning experimentation risk-taking which sounds scary but if you do in a controlled way it can be really awesome so let's look at this and we're going to

talk about each one of these basically the rest the talk is just going to be me presenting a billion different ideas in each one of these three ways of how we can add security to DevOps and then free resources at the end because that's how you end all my talks so emphasizing the efficiency of the entire system sometimes I call this left to right or if you look at my amazing like I know right I'm so good at animation but the idea is is that we want to get through as quickly as possible releasing our products so this means security cannot be the bottleneck this means we need to learn to sprint but let's get into this a bit more so first

I'm going to talk about Devon ops and then I'm gonna talk about security who here does dev who here who here codes for work awesome who here does ops and by that I mean anything to do with operations and making sure things keep working okay and then who here does security okay so not quite all the hands by the end all you know all your hands are gonna go out okay so where's this MIA for Devon ops it means we security folk need your help we need your help tuning our tools to make sure we don't waste your time we need your help sometimes implementing the tools we need you to give us feedback like if a tool is giving you

false positives I want to know immediately I don't want to hear through someone through someone else through someone else how you know I broke your pipeline tell me so I can fix it please I need your help to make unit tests so that I can make negative unit tests I made basically I need really big bugs to break your build big security bugs need to break your build so that's the biggest thing I need I need you to not just like go over that and publish anyway so this is a photo slide I'm gonna have seven of these throughout the talk so I'm just gonna show you awesome pictures of people working in tech while

I'm talking and then I'm going to show you a photo slide after with a summary so that you don't have to take notes because I'm not great at taking notes myself there's gonna be seven of these this is number one I'm not going to give you as much time on the other ones because I have to be quick with the cameras or watch the replay okay so what does this mean for security specifically most of us in this room we can't be the bottleneck that's the number one thing it means that we have to learn to do sprints if the developers are doing sprints we have to adjust ourselves for what dev and ops are doing right now

Sprint's are the way and if they want to release 20 times a day we have to figure out how to work with that we can't say to them like Oh we'd really like to do a three-week code freeze that's nice you can go code freeze by yourself because they're not gonna wait for you and so we need to focus on improving all the value streams and quitting hours other things that it means oh yeah making sure that oh yeah there were more clicks here okay but next other things it means are we need to break our security activities into smaller pieces so you could still review code but you can't review like the entire code and expect us to freeze

them wait for you I'm not kidding I've worked places where I was on the dev team and you know the security person is like well we're gonna do this three week code freeze in blah blah and then they and I was like oh yeah of course they left the room and then team look at me I might know if I forget that guy yeah we'll just keep coding that way it also means we need to tune our security tools we cannot be giving false positives we can not be breaking builds for the wrong reasons and also we want to provide templates whenever possible so that we don't have to reinvent the wheel so if you have done a whole bunch

of security testing on this one login screen then everyone should just be using like replicating that as a template and then you test the main one and then if there are bugs again then you push them out to all of the ones that have been built from the template this saves a lot of time and pain this is photo site number two but don't worry there's more for security in regards to making your entire system a lot faster so what else does this mean so you can create more than one pipeline what so you can have a pipeline that does all the things and then goes out to prod but you could if you want to you could have

it branch off somewhere and just do this weird security pipeline that goes to nowhere I call this the asynchronous pipeline and then you can run your gazillion line static application security testing thing you can run all of your slow tests but it still goes out to prod and then you can come in on Monday and see the things that took 27 hours to run and then you can look through the results yourself and not annoyed ever apps yes you can also give developers tools and pay for them there's lots of free security testing tools I'm a big fan of giving developers tools because dev and ops people are highly technical that's why they do their jobs because they like

that sort of thing teach them how to break surf teach them how to defend things I have gotten a lot of people of my time to do my job it's awesome because like I said there's one of me and like 4,000 of them and oh yeah and more if you are able to rate your own libraries create your own tools if you have to I think the who here was at the panel early the deficit cops panel so ask though is there from Netflix and they've created a lot of their own tools and then open search them which is ridiculously cool of them and so she has lots of awesome advice to give about that so if you

follow some of the Netflix stuff but yes and the last thing is just be creative thank you and do anything you can to enable Devon ops to get their jobs done more securely if you have an idea and you think it will help ask them or try it they will appreciate your effort next faster feedback so this is what I call pushing left right so I want feedback right away I want to know amazing I want to know immediately so this is this is the thing that I show bosses and Mike look how much money we'll save but will also save over time will save stress will save our employees from having to do heroics like staying overnight and

other things like that I don't know if these numbers are exactly correct everyone that has done this study gets lately different numbers but let's just say it's exponential the cost increase every single phase that you find a bug later and more painful - okay so what does this mean for Devon ops please tell us when we make a mistake please tell us so we can fix things and we will tell you as well and it sounds really silly but it's actually really really important to communicate I know it's the worst right speaking to each other at work yes I'm also a big fan of inviting Dever ops folks on security activities so for instance if we're gonna do

security exercises if there's someone that is a security champion someone that's interested invite them to like a security incident simulation or exercise or invite them to training if you can afford it you might be impressed you may be really really impressed okay so what does this mean for security it means pushing left means giving a feedback right away getting feedback as soon as we can it means putting tests in the pipeline instead of having all of our stuff out of it means automating as much as we can so sometimes the automation that you do should be out of the pipeline so that you're not wasting time like for instance you could scan all the code

repositories every week for static code or software composition analysis to make sure that you know there's not any known vulnerable dependencies that are really scary that are being used that's the thing you can do out of the pipeline and then save a lot of time for everyone right and you could run that constantly you could automate it have all things go to your email if there's something disconcerting everything you can automate that's the best make sure there's no false positives I'm gonna say that a few times because there's nothing like losing trust of the entire team I worked somewhere once and I found out that they had an auto rule and the dev team to send all of the reports from the

VA team directly to the trash yeah I'm not kidding it was like their full-time job to send these reports Helen was reading them and then I joined and figured that out by causing trouble yes and security to every single phase of the system development lifecycle even if you're doing DevOps you're still having requirements for your project a security person should be there ok oh yes provide feedback often and earlier ok so what else does this mean for security it means even more it means we have to listen it sounds obvious right but it means we have to listen I'm gonna steal a story from my friend Travis who works at Netflix he's really cool he

gives talks and hears talking about a repo man so they made this tool to try to implement least privilege and it would go and it would just take away different privileges from different instances and they kept breaking everything and then they became quite unpopular it's not a way to make friends by breaking people's code or apps or bringing things down ops people really don't like that so they took it away right back to the drawing board and then they made wreaths oh kid and repo kid would just watch and watch the instances and then it would say you know what you haven't used that permission in quite a while and then it would remove them one by one and they

ended up implement least privilege in this amazing way where they didn't break everything and people actually still like the security team and still ask for their help and still engaged with them and it's beautiful he's gonna tell the story better than I did because you know he did it but the point is is like is that they listened and then adjusted and then tried again don't give up so this is the photo slide and then we're gonna talk about negative unit tests and sorry we have to go fast not a lot of time if you work at a place where they do unit tests and I hope you do you can take the positive unit tests that your developers

make the ones that test that it does what it's supposed to do copy them and then make negative unit tests and by that I mean make sure that it fails gracefully when I inject awful payloads this is a really great way to get regressive security testing this is a time intensive activity but oh my gosh it's magical especially if you've had a pen test done you take those results and you make make unit tests out of it oh I like to get all the value I can when a consultant comes in I like to get as much value as I can because consultants cost a lot okay so what else does this mean for security it means inviting dev

and ops to participate in incidents threat modeling security sprints etc if you can invite them invite them what else does this mean for security it means using tools like a wasp defect dojo which takes metrics from all the different security tools and then shows you beautiful pictures and graphs that help you see what you need to work on the most right and then take that and make a Lunch and Learn out of it oh is everyone doing cross-site scripting maybe we should make lunch and learn and give education on this topic yes and this way wait there we go so this is photo slide number five I really have to get moving cuz I think

I have five minutes left and I still have two more slides so the third way in DevOps is continuous learning and I don't have a cute little ass keep rowing for it but if we don't keep learning you will get left behind who here's worked with someone where they're you know super senior and everything depends on them and if they miss one day of work to go to training everything will break yeah that's because you haven't been on training in like ten years and you're doing everything the old way that person needs to go in training most if you can't have someone miss one day of work that person's life sucks if you want to

keep your employees for a long time you should try not burning them out this is my opinion as someone who has been that person okay so for Devin ops this means please accept security training if we offer it to you you could train yourselves I'm gonna share a lot of free stuff joining your local OS chapter attend this awesome conference again next year things like that and share information when you fix a security issue with all the other people on your team and even maybe the other teams but for security Oh give training whenever you can make every single moment a teaching moment if you find a problem with when security team and they're always doing this other

or when dev team or ops team you probably want to show all the other team members too because that problems probably happening there as well I have learned this we often make the same mistakes share information again when you fix a security bug so if one team has a problem assume the rest might have it and provide metrics whenever you can to help the team's know what to concentrate on we can't say the sky is falling I've learned that doesn't go very well and instead we need to make sure that we are emphasizing the most important things and explaining clearly why so we can help motivate them instead of just scaring the bejesus out of them

yeah another thing is so if you want to get more advanced in dev SEC ops there's things like chaos engineering and teaming where you introduce problems into production and make sure that everyone can react to them successfully and gracefully don't start with this do all the other things first also oh um also I'm a huge fan of security exercises so I used to work at Elections Canada I'm Canadian I'm sure at some point I've said aboot during this presentation if you've noticed my accent so I ran the security for the election in 2015 where we voted in that guy with the great hair and Elections Canada before every major election six months in advance they run a fake election they

build an office they do everything they had people pretend to vote and they invite hackers and they throw security incidents and do all this stuff and then they do post mortems on everything improve everything and then during election day we were you know not in the paper nothing everything was great it's so awesome no new gray hair so invite your developers on exercises like this if you can hold them they're so worth it it is our job to enable Devon ops in every single possible way we can I have one minute left so I am gonna skip this last part and instead get right to the culture part security needs to become a part of and this means we need to do

things like celebrating wins we need to work closely together we need to speak to them just to be clear that's what I mean we can't blame them we can't point fingers anymore security is really good at telling people when they've done a bad job and as best as we can for scaling create security champions security champions are your best friends so with that I would like to call upon all of you to remember that is our job to enable teach and automate security and give security feedback its best we absolutely can that's our job that's our duty as security professionals I'm gonna give you a few resources I hope your phones are ready because we have around

30 seconds all these resources free okay yeah so first of all please join Olaf you will meet lots of cool friends I have met so many of my amazing friends through them I can't say enough good things about them if you are a woman or identify as a woman consider joining lowsec women of security and we're gonna hold a CTF for women tomorrow morning and we're gonna crash RSA from 3 till 5 tomorrow night message me and I'll give you a link if you want to join and just meet a whole crap ton of women you didn't know yet every Monday I pair people with professional mentors on Twitter if you have worked in security 2 or more years there's someone

who wishes they knew enough to be able to do your job and you could probably teach them if you are looking to get into security this is a great way by preparing with someone that is in your corner and lastly I produce a lot of free content and and it would be my pleasure to share with you and with that I would like to oh yeah security is now part of DevOps it's everyone's job I'm not kidding everyone and now we have dev SEC ops and you can totally fight with me the dev ops done properly includes security after the break thank you very much for your time and attention today thank you so much