Developing a Rugged DevOps Approach to Cloud Security

so the sponsors for making this available and happen and also if you have a chance please fill out some feedback information is located on your ticket and also on the website on the schedule page and today we have tin Pendergast pleased to give the well welcome the bus thank you thank everyone your justice thing so I was conscious the fact that I stand between you guys and the bar which thank you whoever organized me to be the fall guy for that one but there's so much material to cover here I kind of took a approach to give you guys kind of the seeds of thinking about the change in approaching the changing paradigm here and I'll talk

to some technical examples but I'm not going to go through a bunch of demos we dive into a bunch of code examples because it's something that actually is better suited for you to explore the journey on your own but let's explore today kind of what it means to take a rugged DevOps approach to securing a protecting the cloud which is really as everyone thinks about it shut your data center that someone else is hosting for you now or it's the Dixon are you host that's using a very different way of all the things that happened around this cloud movement I think the most exciting thing that's happening now is specifically around security sure it's cooled like elasticized workloads and

how you know something I can spin up and down resources on demand and just pay for what i use but the value it brings when we talk about the approach we take the security and the challenges we face as we move into this era are far more exhilarating and while people get excited about like hey I can run it as a few servers or thousand servers the idea that aim of all those servers they can take dynamic security postures on they can be reflexive and react the kinds of requests for coming into them and these changes can happen in the second time frames is a totally different paradigm what we even use to insecurity and it's

something that we should really think about start taking advantage of going to give you guys like the step once I don't know everyone's background what is cloud i'll sell this argument the people have had forever is a programmatic infrastructure it could be anything that falls under that but think of it in the sense that it's API driven infrastructure it's not what VMware is going to sell you guys it's not what amazon web services specifically tells you it is it really is a different way for us to interface with infrastructure that goes beyond that rack and stack approach we took and since i reference it what is DevOps I probably ask people to raise hands i'm sure everyone pretty

much knows it is really just smart people building things in a more intelligent way i mean it means automation it means scale it means just get away from kind of the typical behaviors we used to do so when you think about this acceleration towards deploying infrastructure and applications and services we call DevOps there's specific needs that have arisen that drive people to get into this state and those needs are specifically focused around flexible rapid iterations and testing and validation of what you're doing and then iterating forward on that meaning don't wait six months to put this new innovation out there for your users means wait a week after you see people start using what you've built and

then learn how they're using it learn the cool things you can do with it and then iterate forward on that the same time security has never gone away I mean they're always right here for a particular reason all of us here have jobs that focus some aspect on this we have to bring to a business or an entity some sense of control some sense of stability some sense of risk aversion because we all know that a major compromised tends to end a lot of small medium businesses and large businesses while they kind of slop them off suffer massive reputational damage over time if I surveyed the crowd and had everyone to shout out the name of a company that

like has a bad reputation for security I know probably five or six and most everyone's going to shout out that's a sad state in the industry but it's also telling to how we as humans emotionally connect to people not doing it mean they can to protect us the thing that's really interesting about this is we inadvertently jam these two worlds together and we did that because security is still a relevant function for a business but the business is also said we have to move faster and historically acceleration is not the best friend of security the dynamics and the mass change introduces a little bit of chaos and makes it harder to drive signal from noise right but there's a

new way for us to actually think about doing this where you can actually have kind of the best of both worlds and this is the thing that I kind of struggle with which is I have to step out of my traditional ivory tower paratrooper knight in shining armor save the day kind of role where you instant response and that's like the end of the security discussion with a lot of people that's where you convince them to be part of the solution and into this educated roll this embracing role of like hey you know the guy in the other side of the phone that's building this application and managing this infrastructure it's a really bright person he just hasn't had

15 years again his butt kicked in this industry like I have and if I could actually teach him or enable him in some way to not have to go through that same pain that I went through and learn from what I'm trying to tell him is the right way to do things without me being the guy with a big stick then we could actually come out way ahead and just a quick survey who here thinks you have enough people at your company doing security I'll see you outside buddy who here has open job Rex for security engineers yeah basically everybody so this is the pain point we see we can't just poof into existence enough people

to actually you know manage security we can't say how cool next year's graduating class is coming out fully ready to come protect the infrastructure we've been building for years it's one of those things where you have to take advantage of the people that you can kind of train up and what's really interesting is the people that really get this right now are those engineers and product teams that are building the infrastructure that you have been used to protecting in the data center that you don't have to do in the cloud so this paradigm is totally shifted the one thing that you see that's really different here is not only the ownership aspect of the infrastructure layer but

really how we've kind of taken these broader concepts where we built walls around the garden right and saying we actually have resources that can exist for seven minutes I didn't get up my data center had a thousand servers yesterday and two hundred a day like been robbed right I also didn't get up and walk in there checking to c a1 racket service and i have like 500 racks of servers it doesn't happen in the data center it's your expectation of things being static is how we built security around it we said quad can put zones for network and and traffic domains in place I can put firewalls around those I can actually put IDs IPS online of

everything I own I have ways to get what I want to get so I can try and do my job better the cloud basically took that and threw it out the window it says he'll by the way you don't own this and you're not allowed to come plug directly into it and trust us we're doing everything for you the reality situation is they do everything below a certain layer but you as the customer own everything that's really relevant the a play or the network's track that the network traffic stack what's kind of really changed is the way you interface with these cloud infrastructure services right so you think about it it is anyone here done

API programming like tickle expect that kind of stuff against like old firewalls pixie's you know IDs I guess it's terrible nothing's the same the unifying constant of cloud instructors programmatic is we have an API interface that now speaks universally to identity network storage server it's actually a really different approach now we can actually talk on one language plane to all these different layers and then cross-correlate across them so this is actually enabling a whole new era of security for us and it's one of those things that we were excited to talk about yeah little box that's all changed so why is it cool because the vectors have actually totally changed this didn't happen in the data center we call

this the side door to the cloud so when you think about it it's not like someone puts up a new server inside your firewall in the cloud and then someone can get total control of your data centers that server came online that just generally doesn't happen that common in the real world unless you're getting hardcore Oh date but what's happened is we've taken this stack where you had you know you could call it a three-tier stack whatever you want cause it's logical your web application hosts are kind of the same these are containerized functions that you can support around wherever you want and the way we defend applications and services at the a player secure coding lifecycle

you know verifying your Oh offs top 10 and 20 or you know being guarded against doing typical you know koalas kind of apps can this kind of stuff isn't that new and it's not really changing much what changed was we talked about where all my data records that really matter we actually dissolve the typical layer running giant Oracle in sequel clusters and push those in to manage database services in a lot of infrastructure we've also taken the traditional storage hosts which by the way as much as we all pay for them there's a big stack at units your windows boxes right we've taken those and dissolve those and we've stored these objects into like storage services

that can be archival they can be object store it can be Dropbox for all we care right someone else is giving us a presentation layer that we engage and push data to and when we need the data back we ask for it and that contract of getting that back and forth at an acceptable latency has said we don't have to manage these things the other thing it said implicitly that we all don't accept is that now I have to figure out how to protect it over in these services where they're publicly accessible so in the data center no one can just walk through the wall under your and your infrastructure and start ripping out servers we have multiple

instances where publicly accessible cloudy api's have led to the complete demise of companies in this industry HD Moore is paper on s3 bucket walking there global namespace you can just type in random names and eventually going to get hits back and you'll get hits back to have medical records legal records because there's so many permutations and there's so many micro wrappers of security policy around every instance of an object that you can create or an asset in the cloud that how do you know that the three billions object you created is consistent with the other two point nine nine billion right how do you know if someone internally didn't change that policy and make it accessible pop

the link on paceman so everyone's now pulling down your entire payroll record like this is actually the challenge you face in the cloud now because we're no longer in complete control but we have a new way to kind of think about this I will become very unpopular very quickly we haven't done a good job in the industry of product and I started on the security products out of the house and the wind defender and now i'm back on product we haven't done a good job keeping up with the change I mean chef and puppet and these instructors code components have been around 57 plus years right and really popular the past three or four where's the security

innovation been like right we still think that you can just take and I kid you not I won't name names I had vendors who are like take this big load balancer you have your dinner and just you know employ our virtual appliance in the cloud it'll do the exact same thing and you sit there and you like I went through this whole this whole iteration of redesigning my architecture the way I think of things and concepts and loosely coupling my application and destroying all these concepts 20 years of em structure and now you want me to undo that and then put something right in the middle with its a giant single point of failure this

is not the way that we engineer for cloud and fundamentally the failing has been the forward thinkers and thought leaders that have embraced programmatic infrastructure have not been those security companies they're still building products and solving challenges they know happen on the data center meanwhile the world's kind of moving into new paradigm and we're lagging because of it and part of that challenge also is the way that we've structured ourselves in an industry and community and if you think about it the DevOps community is on a great job with chef con papa con you know all these DevOps Centrex conferences people get up here and they open the kimono and they dump all their innovation out and you guys

can copy and get enough github and go home and start from their new baseline fundamentally in the security industry we've not been as open and we not been as welcoming to others and we haven't been able to share as you know illicitly is as that community has that's one of the things that we have to think about in the social layer of changing over time or else we're going to be stuck here so how do you freshen the stack right think about it you know there's a whole set of major services that comprise probably sixty to seventy percent of your infrastructure in the cloud that are completely API driven no IP address can't network scan them can't

nmap them nothing all you get is hey the API endpoints open but you know what there's thousands of functions and millions of assets behind those AP is that you now have to actually protect and think about how you guard and if you go top down I got this I'm an old Network nerds like I kind of went to the OSI mile the OSI model kind of in around here you think about the business process that's ATP work date none of us are going to worry about that by the way they have the original cloud service that if your company says we don't use cloud services ask them where your payroll and your social security and all

your benefits data is someone else's cloud application security still relatively the same we're getting new cloud intelligent WAAFs signal Sciences shout out those guys are awesome put out some new product this week we have cloud where agents we've seen the endpoint evolve to understand but the endpoint hasn't evolved to understand that we're going to have you know totally serverless infrastructure like AWS lambda and some of these other code past kind of layers and then you have all the rest of this identity network administrative relationships and roles regionality of your data that is only accessible through API and protected as such I don't know if Adrian's here if he is I want five bucks after this Adrian's totally nailed it

the first analyst I saw that really got it and it was like you can now push a button and destroy a data center how scary is that right if the wrong person can get credentials or can fish you Boop good night oh by the way your backups are in the cloud those are gone too that's how frightening it is and that's how much onus is on the people running infrastructure and not to scare people the people running infrastructure tend to be people with not a ton of security experience and not the battle scars that a lot of people here have from making mistakes and having to do instant response and correct it over time so how

do you kind of bridge the gap totally makes sense that we go to security as code right if these api driven services are completely code oriented and on the back end of whatever security instructor looks like is like a JSON blob then we can actually build automation around that construct knowing that if what we do to the instructor that JSON is what comes out the other end because the providers are using automation to build from that same JSON blob then we create a one-to-one construct and security we say what I say is what I do and what I say is what you do so now that means watching the ship in the dynamism your environment programmatically allows you

to intercede programmatically so it's not oh by the way someone launched a bunch of Bitcoin mining machines in a region in a pack at three in the morning it means if someone launches anything that I don't want to be launched terminated immediately right don't page someone have a human wake up and go login so you think figure out what's wrong we can articulate what security means in the cloud and we can actually enforce it programmatically which is something we've not been able to effectively do before totally basic example this is Amazon JSON structure so it looks horrible on the screen sorry you know fifty percent of the time we see people who are building enrolling new images and

talking about patch management the cloud and all these things struggle with it and the reason they struggle with it is how do you know what should be where at any point in time humans are horrible at going around and taking like am I image IDs and knowing which one should be in every region because they are different in every region let's thank you amazon they're different every region they change as you roll out new patches so if you guys have an you know like the ssl issues that came out recently you have called some patch and rear all gramma just globally enroll it back out people have gotten sophisticated enough to actually bake and build am eyes using

instructure automation and push them but then on the back end the security process is broken we have successfully seen people start to actually integrate these audits and checks into the sea ICD pipeline so while you're actually doing a build and push of a new image not only does it mark that as a new golden image so you know what's no longer in date so you have a complete inventory what has to be taken out of date they're taking out of infrastructure you also now can say globally is anything running during the life cycle this image that's not approved like you want to know I'm an attacker I get access to your cloud em structure I'm not going to launch our

golden image guys I'm not going to launch the thing that's got your you know cloud passage agent or has your splunk agent so I can get myself away I go pull some from the marketplace and launch it and you never know there's no way for you to actually track these things unless you're actually looking for it and you have a known good state that you can track against at the same time you start talking about like hey what are the right security role so that can apply to these assets and a server machine is one thing you can do this around an object you store and s3 you can do this around things you store on

Google's object store like these assets all have roles and rights commission's they're all equal class citizens now that's actually a really unique situation for us that has never happened before everything was subservient to a host so you can go say hey is someone using the right security group for it and only that I can query back when something happens pull the security group for all the relevant image IDs and push CL updates that security group programmatically so now what you're seeing is the trend where we not only know what's not right not in compliance but we can take remedial action and we can actually start taking immediate defense and immunization against our environments where we can go and say

this events happen somewhere we want to immunize the rest of our herd of instances amazon accounts cross-functional teams or people that are doing threat feeds on the internet consume those and push a remedial action to guard ourselves against continued attacks I don't know how many people here run an Amazon or Google or Azure and raise our hands so most people how many people are programmatically auditing your infrastructure at least on an hourly basis okay good set of hands that's more than we've actually seen a lot of times how many people can take automated remedi airy action based on that data awesome so this is something that more people should be aware of it's not instant response in the sense that a

person does it it's instant response in that you force things into conformity through programmatic action other interesting things that all this API data is relevant across correlate against other known good states and other pieces of your infrastructure so hundreds and thousands of API calls and asset structures you can return this is one where you can derive seven or ten different security verifications and compliance actions and it doesn't stop there the api's keep growing so there's always more for us to be doing this is where we get to the topic right the rugged ization of your process you know I'm not going to go into talking and telling you guys how you build code securely I'm not going to tell you guys

how to build tests and assets I'm not gonna tell you how to deploy what I'm gonna tell you is that each of these steps there's a place for security to be part of the DNA it's not supposed to be something that comes at the end of the deployment where someone has to come in and do an auditor or scan as you're catching things to the development cycle through the test cycle as it moves from dev test stage you know prod and in production before you promote it from an a B or a Bluegreen deployment stack you can actually in real time fail builds based on security verifications audits checks whatever you want to call them

and in fact you can actually create corrections to prevent bills from failing so if your build goes through and you push out last week's template for security groups in Amazon so you know you got rule updates since then not only will the systems that you build catch this to actually fix it so you don't fail the build and waste a bunch of time by opening tickets and having someone go update a template you can still do that remedial action on the back end but in line you can fix and save a build which is something that security has never really been well known for like we're not always that hey you guys save something and made it go

out the door faster this is the time where security actually accelerates things and you guys get a better seat at the table refresh the approach yeah this is where we have some fun about examples of things you know anyone here scanning like still on a weekly monthly quarterly basis don't lie it's okay it's dirty laundry that's fully acceptable for a lot of people the funniest thing is a lot of those scans in the cloud world of the programmatic world or at a date by the time the PDF is generated mailed to you because events have changed people are building and deploying on an hourly basis right it doesn't work in dynamic environments the new hot stuffs

continuous scanning real time for threats and a threat remember let's boil it down someone making an accidental mistake in the way that they structured your security posture leaving an open attack service is a threat against your organization it doesn't have to be the scary people across the water it doesn't have to be you know the government trying to infiltrate and Trojan your software it can be anything as simple as it could be because the threat is your organization's at risk and will lose reputation data and customers and revenue if you aren't putting your best foot forward at all times give you guys a quick example over time things happen and things don't happen in a monolithic

sense an identity change alone is not generally a security event that a lot of people are too scared of because it happens but if you correlate an identity change and an unapproved image launch that potentially malicious that should totally trigger quarantine action because a sequence of events is a pattern of attack much like a set of network packets as a pattern of attack but these things happen at the infrastructure level and their events that create change and you tation your environment they're not things that are simply a chain of network packets that come together that you have to reassemble on the back end it's a totally different way of perceiving what's happening and then it's gone

the attackers achieve a goal they've changed the policy on a particular file and they delete the instance and the identity that was there how do you know it ever existed this can happen we've seen this happen as fast as 3 to 4 minutes people are programmatic and they're fast and it's really scared that you'll never know what happened if you're not really paying attention new cloud monitoring tech you know lots of really cool things this will go up you know a security monkey from netflix prezzies Red Alert apply meta I say partner scout to work commercial products all disclaimer and there's a bunch of cool emerging things that are coming out that we're staying on top of

I think there's really interesting things that happen in the OSS and commercial side here definitely something you guys should be checking out if you don't always build it yourself build on top of some work with other really good people built single-purpose host-based technology not only is host-based really in danger right now if the programmatic lambda style containerization worked really well like we could get away from running operating systems at all I'm sure a bunch of people here would like that time back from patch management and all that other fun hardening work but we're already we're already doing a lot of it I'm searchers code Papa danceable chef these things that all of a sudden you

think as a configuration state is actually OS hardening it's you know making sure you're reducing ports operating and guaranteeing secure traffic it's dynamically pushing firewall changes to the host level if you need to have it based on everything as simple as error code counts coming out of file logs things like I don't know patching heartbleed another great excuse for you to actually push security down to these kinds of systems that can really help some good resources here once again this will come back up I'll give you guys about five seconds before I kill desk people can grab pictures look in these config management communities they're really starting to push security harder because security people are coming to the community

rather than them having to chase us and that's actually an exciting time manual audits like the cool one right everyone likes the image automated compliance audit who here has to do compliance yeah how would you like to have a lot of that stuff happened on the back end and you only get a report that people are fixing things that would be mean most everyone else we have seen auditors come in and actually be able to go through a setup data not only showing where you are today is as a proof point because structured data from a system is binary it's not a human telling them they have to double check it for more evidence it is the evidence but also the

proof that as we deviate ninety-five percent of the time we're in compliance than an hour again because the systems actually reflects them or gets people involved fast compliance over time is much better than compliance in a fire drill and that's what we've been trained to do historically the small course corrections make it a heck of a lot easier at the end of the year when you have to recertify reactive behavior which is where most of us have spent a lot of our time automated defense totally I mean I've been alluding to this the whole time I'm going to call rich mullins there's a lot of credit because he was the first not only analyst but one of the first tech people

i saw that openly published the idea that you can actually forcibly correct things that happen that shouldn't happen and forcibly quarantine events another great piece of you know software you guys can take and build on top of if you need to articulate what should happen and what shouldn't happen and guard against what shouldn't happen programmatically it's the difference of fixing things in seconds or spending months cleaning up afterwards it really is you know getting rug is a journey this is kind of the last one I'll leave you guys some time to talk security doesn't happen like as much as we would like to buy it in a box and deploy it have its all of our problems everyone

here is pragmatic enough to know that we build and we made by certain technologies and build against them what really happens here is we need to be focusing on forcing the industry and forcing the thinking to be not only give me the functionality I need but allow me to extend and build on top of it that means API enables security products are the thing that all of us should be pushing the industry towards not special you know specialized gooeys or things like that those things are terrible you know automate the security behaviors this goes back to the same thing the environments to dynamic I don't care how good all everyone here is I don't care how good i think i was in

my heyday how much redbull i've had in me i cannot react fast enough to the millions of permutations of events that happen in a daily basis on a common cloud environment we have to shift our thinking we have to lean back on what we know well which is technology and the defense's that we understand and understand how they applied these cloud environments and then this is one where I'll get a lot of oohs and ahhs I encourage all of you go find your dev ops team or you're off team and sit down for lunch and be like how can I help you with security how can I make it part of the process and if you can champion the

marriage of these two you know almost silos a lot of the problems that we face and security go away you're welcome to see to the table you get more honest answers it's not adversarial it's accelerating the business and what ends up happening is you create relationships and you create a perception of the info information security professional industry that spark conversations and what comes from here as conversations where we can all openly talk and share more assets like the assets that I mentioned after various parts this presentation and drive the community forward because as good as we all are individually in our in our businesses or a hell of a lot better together especially when we're fighting against

common cause so I think that's it I don't time thank you and Tim thank you today for sharing your valuable knowledge and on behalf of b-sides and Fitbit we'd like to give you a gift alright thank you and thank you for joining today there's three things announcements basically there's a drink tickets available in the bar in the main room and also a real food with appetizers and there's also a raffle for very high end beats headphones so take your chances thanks