Pwned Cloud Society

[Music] thank you and you're at the hose cloud Society talk today and just want to talk to you a little bit about how you can expand access inside of cloud environment on engagement and then also like some mitigation steps for that so I am right whom I currently work at Adobe's and I run the red team they're at Adobe's digital marketing business unit they mostly focus on analytics as product or website then I used to work in the department defense and I used to also work doing defense at Homeland Security so right so cloud right it's a nebulous term and hopefully by the end of this talk it won't be so nebulous anymore but he can't get quite diverse

quickly you know seems like you know there's so much cloud now you know there's one cloud provider or them there's another cloud provider and then there's another cloud provider there's more cloud providers and I think the only thing that's really guarantee is that there's always going to be another cloud provider coming out right so see for the day right so while each cloud provider has differences if you can learn the general security concerns with the setting up the infrastructure and running infrastructure in these cloud providers and then you just have to learn the little nuances that are different between the providers and I know everyone talks about cloud talk about brand-new everything's different all your old problems go away and I love

the perspective that a lot of things can be better right if applications are written around it but I mean a lot of things are like both like the way that we've been operating the data centers for 8.now so I just want to show you this is kind of the old way it's very simple probably not even accurate but you got a developer you write code he checks it into sound repo and then you have like some physical server like a chainsaw and then that's going to push this towed out to the production servers whether that's like a web server or a you know something to research policy initiatives there's not like that but oh yes I know

maybe you send your dildo there for this like a standard build order now for pushing code out for like a production data center you with help from my dad then you know you pretty much have that same flow now in the cloud but you're using services so you're not managing the boxes and services to expand out and down as needed to scale with your with your application your product or solution all those things so you can still have developers pushing code to github or some other code repo and that's you know using some type of code deploy service which then deployed out you you know lambdas services in AWS which are kind of like dynamic web

servers or computers to get spun up on people across Web Services and then you kind of have this storage in the cloud area of that s3 bucket for Amazon where like persistent data is going to be stores all right so is that you know family services don't have persist and Forge so and these are all basically just eight guys you're calling to interact with these various services and then you know you filled admin right so in the old model you have an admin he might though he might administrate like an 80 boss or an LDAP server and then all the rest of boxes and an enterprise more or less trust a 80 or LDAP server

and so he's still able to get to all the servers and admin these various now individual servers propagated under and in the new cloud environment still have someone who has admin access it might be the same developer right which is also developing the code but instead of you know logging into servers you just logging into this web GUI management console and from there he's able to manage each of the services oh so you know you feel if you combine that developer and admin maybe like a DevOps model there's no separation of duties and you know he's able to kind of go everywhere and touch everything right so obviously you know this edge is compromised and you know that they take

some theories to talk about you know this isn't just like figment of my imagination a scenario I mean this is actual intrusions that have happened and have bankrupted company so the first thing is there's a is a guy postings as Stack Overflow asking I got a fifty thousand dollar AWS bill what what should I do now right so someone got ahold of some of the secret he was using to manage to date Amazon services this is decided to spin up a bunch of additional servers and racked up huge bill for them so you know there is a he wasn't able to go through a resolution processes Amazon do it the bulk of the bill is apparel if

you're sloppy with your security what happened to you and if you're running a very large amount of servers leveraging a lot of those services not even notice [Music] okay and then I think code bases is the quintessential example what not to do in the cloud base they built their entire web application in Ian Amazon compacted up to s3 bucket I've had ami images in Amazon and an attacker was able to compromise API keys that they were using and they emailed them and said hey you know if you don't pay up we're going to just destroy you know your company they weren't really detailed on how they're going to do it but they did take up and

attackers went in and they deleted

and then they terminated all their money servers the resources and it essentially bankrupted the startup and one day right so I mean this is like a real scenario that could happen if you're not careful with year you know you just set it up or there's something else in in in a closet Vidor okay so we're just going to talk about initial access so in the cloud providers there are secrets or API keys and really you want to get your hands on these atiq and there's many ways you can do that I'm going to show you some pretty I think they're amusing example right but you know you could take the traditional path of you know finding a

vulnerability on the server and packing that server and finding the correct in the file so you know the first thing is people will actually mine through open source

you

oh right jacket you know really all you're going to do is like Xenophon you you bunk you bought and installed AWS tool and I'll put a copy of these slides on SlideShare and I'll tweet out move to them afterwards so that way you know you want to write it down a little bit right and you're just as pretty standard if you admin AWS services you'll probably Airy happy goals at all and I'll just kind of show you how to use them so s3 bucket this is where you store files in Amazon so you store like images just or static content maybe you backup files maybe you create your own like OS image then you upload

it to s3 and you could like watch it as so any time you know you want persistent long-term storage you might use an s3 bucket so so people actually host websites straight off the s3 bucket and if they do that you can actually just you know team the domain and you'll see in the response that comes back like hey this is s3 - website and then it's going to give you the region that it's in Amazon's actually located good and you're just going to want to make note of that region information there's only so many region about 12 inches right I mean you could in future steps just get pretty far region but you know this is going to tell you

what to do okay today let's get me so you know that's not even creepy happy I just think the domain is all tied to the next three so um so s3 bucket can be set to be world readable which just basically means like if we go to a website and you see that index of and just the field listing of all the files in there there we you can set up your s3 bucket to be exactly like that and so if that is the case you can actually just use this AWS s3 command and it will list back all the files that are available then you can try and download each file these definitions and if you browse to humming this world

you

all right so you got secrets you got your box set up now you want to leverage those actually be something useful instead of AWS so the first thing I do is I configure profiles for secrets usually when I'm doing engagement and of getting as much as a ginjury goes on and get more and more secrets three to view us as I'm more more successful so I just Trent set up different profiles with different names and that way can switch between an easily track of what what secrets I'm currently using so you just from that yet of your computer command to do that and then I usually just cat that there's a dot 80 of us older who

has a user's home directory I usually make sure it looks good at people Iraqi you know also some time to read the spaces around that equal sign so I can keep them - tricky really quick okay so this is kind of the Who am I put away for for AWS so you somehow got these secret you hack server you found them in it open source repositories but you don't really know these these are like do they go on through or what to do or which their business it again access so you can just use this SKS get color identification API call using command line tool and if you get a little bit information about whose equations are I

need to give you one want or I can you just help back up right another technique for finding out more information about who you are is by calling I am I am is user management capability and secret natural capabilities inside of AWS so you may or may not have access to it I mean realistically you should not probably ever get access to that just hoping this trivial but even if it doesn't work it will air and in the air it will tell you a little bit of information about the users you're currently running at so once you know that the correct or right once you do that STS identification call my first thing I want to do is know am I being

log right there zombie laws I need to be very cautions about every API man that i issue and you know lest I get caught right or deviation is over so it

you

just some logging off though we're not being reported found the secret because you're packing a server he's the local source repository but now we just want to make sure we don't think it's out all right but I got two weeks left on this engagement I just want to make sure that I'll be kicked out or detectives before it's over so that way I can you know provide back a holistic picture to the top somewhere closer very here in this environment so I think this is a cool technique you can actually if you have an API you have secret cooking that use for like API access you can actually get a session token' say hey get session

token' and this will give you a completely different set of tea tricks that are good for 12 hours so then you just set a cron job that gets to secret every 12 like but not right like every 10 hours as long as you know this secret this session secret seems to work then you'll be here to get more and more session features so there's like some little nuances that they get annoying with using this like a little bit more difficult to set up and there is some um you want restriction any data left you can't do that the main super hand but this session people it is I don't know really how to view them but eight of us

and mock sure they are to vote maybe there's a way to do like that now I'm not really aware of it so yeah go ahead the default is 12 I think you can go up to 15 right you're on your issuance of it and I just visit meatball you can actually there's another switch that suspect by the time that is good for and there's documentation on it so you could go larger I just didn't want to like if there was a way to enumerate session tokens which I'm not sure there I just didn't want to stand out from anybody else's session so good okay all right so these is pretty stealthy and and then you just kind of set it up in AWS file

similar to the way you set up a preview secret but my names are slightly different and then from then on you know use that session token' profile and then you can keep you interactive the services no or the stable future another way to persist is every user that's in the console can have two AWS keep its associated account so most users will just issue one API secret and then they'll continue to use that for local operations and if they need something that's more application specific they'll create another account so just physics of that API fault so you can actually just go through list out all the users if you have I am access and then pick a

user that you think is a good candidate then add another session another secret to that users and then by adding that secret to the user you know hopefully that will go unnoticed by the user not people the parent in the web GUI I mean if the user goes in I'll probably just think they issued it and easily you can also just rate up add another user alright cool here I'm adding the user and Ryan every actor okay oh yeah I see you has a get a user then you can add a key to the user then you can add a password to the user in this scenario then you can actually log to that pretty web GUI so

there's really no reason any cousin attackers can be honest man all right cool so if you can get back to let's see secret has access to s3 buckets and usually what I'll do is I'll dive into that three buckets and I'll try and find more secrets right you can find more secrets that have access to I am which is like user management or have access to to which is instant management I collect and set up these servers in the cloud then you can actually just spin up a brand new server inside their their VP C which is like their private network and then you can just SSH into your own server that's now running inside their their environment then you

can do all your plastic you know pen testing techniques from that server right so why it wise is better right because you know one you have full control of that server that's inside their private network now you can do it as a school information that you want to you can now access probably a lot more services very accessible just like if you were to get access to a server side of data center you can now start hanging up and probably find like on authenticated reddit databases or other like memcache servers things like that whole wide range of services which a CL group from the security groups from the internet are probably denying but once you're inside that trusted BCC and

inside that trust network you can now start your packing in more and more servers right and of course when you hack another server you definitely go and try and line through it to find more secrets right so you kind of go fold it right do traditional time testing you're doing AWS interactions and then you're just kind of using each other to each other in the loop and you know this will like appear like a new instance is in their list so I mean you probably want to list the instance first and make sure there's a bunch in the region everything if there's like you know you know there's 20-plus servers in the regen weekend up one more

I don't know like what difficulties that Facebook and some of some of the AWS account settings under certain sensors right and then I just want to talk about the metadata turbit I know this you know they talked about before but something that I often run into people and they are unaware of so there's actually an RFC it just basically says for these cloud providers there's a magic IP address 159 I can address and based off of RFC you should be able to be read as IP address only from an instance that's running inside the cloud provider and then get back information about the incident so this is crucial for a few reasons like one if

you ever get code execution on an instant inside of AWS you definitely want to queries and metadata service and enumerate as much information as you can out of it you can basically just do that by making curl request right here's a metadata service to some types of web vulnerabilities if they are occurring on an instance in and Amazon are needed to be have their criticality used up right so if you think you know do you kind of like an open redirect on a web application you get a web application to make a call out on another IP address on the internal network and then return that result back to you you can actually have it reach out and go to this 159 IP

address and then return back the results of the metadata service to yourself very infrequently but it is possible you could find secrets actually stored the committed a service which to turn you back secret so I shouldn't happen but there is like some Federation data that can go into instances and boost them up and climb back data can be retrieved in the metadata and it really deceive the metadata survey labelled from any instance you just curl the 159 Effie dress and then get abused oh right pri and then if you that works and you want to go back to the documentation and just pull list of all the URI and they can be memory for each one of those pull back

all destinations but um but like I said before metadata service is actually not an am something there's like an RFC and more cloud spiders like a sure aim digital ocean and Google compute has once you are actually trying to build a similar text service so you get code execution on any of those providers you should be able to query that same 159 FPGA and then also gain some information about the engine so the problem is Amazon's implementation of it's much more mature than other cloud providers from my testing and Google's has a little bit additional protection you have to us by like a particular host a nerd when you're pairing it so that might give you some the web redirect

issues like you would be able to do it through them but but yeah in a sure you can query these two API calls is the only two that I was able to find and it gives you back some really limited data not really soul but what is cool in Azure is there's a W agent folder and in Linux underbar live W agent if you go into this directory you can get a lot of information about the incident and possibly then decreased access additional resources inside of azure and you have to be route on the instance here we do this inside of Asher so that's a little bit better than the Enzo setup but it's definitely some key

gated amount of it

okay cool so we got access to someone AWS now we want to expand access right if we PSIM you know we obviously we probably just add a show user logins web GUI equal there but you know a lot of times what customers feel like ok great you got access to that or like where's like push the credit cards like where's the production data how do you know how would you go from that to see you actually like all critical data right so you can take snapshots with your servers inside of AWS and if they've done this previous and you're able to set up an EQ image you can actually just mount those a shot as another hard drive if you

mount them as a hard drive then you can start mining through them for like you know at stage private keys and finance page private keys and you're inside that BTC then you start education to the boxers you'd be surprised hold off these snapshots so there you got to be thinking every Aldous comes back you know physical file you know files on disk and I mount those and I read them and if so if I'm more secrets or and I use those taxation of offices access boxes totally shares one and I just want to talk about a pretty extreme technique I do not recommend this and once you're in a desperate scenario but is very attractive right so

I call this top part hard boot technique alright so let's just say there's a server in easy to you got root credentials to AWS but you still can't get inside the server right this is fun up you can some big strategy that you don't have access to right and those aren't like retrieval rest of it one thing you could do though obviously this is get an effect production is you can actually just get access to manage it is that you can say hey shut down the server when the server shuts down you can actually mount the pivot card drive from another ec2 instance then you can actually modify that other other production servers hard drives like address H key to the remote

to that servers are good instance and then unmount it and then spin back up the server right you know obviously this is going to make this assignment very angry if you're sitting down their production servers but I mean unless they're implementing some has a full disk encryption at the cut like inside the instance of a cloud provider which everything done yet which probably should need to stop this type thing and the right you're going to be able now activation as a production server and pretty much access a walk X right so or you know you could have pulled it off we have the image right but relatively you know want to heat the time that the

curve is off the lowest amount possible if this not because that is right so yeah it great

yeah can you create a snapshot so that you don't have to do a hard vote right oh I think my experience of that is you can't create snapshots a public server is shut down and then how hard they do a snapshot but a good technique is to go through em another technique I don't know if I suppose we talked about or not is actually a lot of people will create am i right and you can actually share am I to another Amazon account those an attacker you just sign up for another Amazon account you can actually go into their images and share their instance images with you and sometimes the image will have you know secret surprise or

key sort in it and that will allow you to you know access that type of data which would be another technique although a lot of times I see cloud providers doing is the image ami just has a bootstrap script the bootstrap script reaches out to like a salt or puppet server it is enough salt and puppets over kind of push all the secrets onto it at that point so you know sharing am I haven't works great for me but you know definitely exactly okay so I'm just going to talk about mitigations for a second and then I'm going to a question so right so you know the biggest thing for me is just single purpose secret right so if you

want a secret to access RDS a relational database

you

the secret using the Bluth account that's types that you want to create another town and then login to another town and then create secrets associated that account at a bare minimum reasons that you want to do this one route secrets generally have access to do everything inside your Amazon instance so those get compromised they can do anything another tidbit about roof secrets is there's a technique where you can actually use these secrets to buy stuff off amazon.com and fill it back to the root secret credit card number so I mean I haven't actually seen it like an attacker do that but you know they don't like just wasting your money on a tow us they could actually physically ship

themselves off with it and then the other thing is rotating secrets very frequently right these are not things that you know you want some static and what they're forever like I said it's not if it gets from Isaac when it's gonna get compromised when that happens how quick can I roll the keys and you know how along with RT good or and then another really big thing and there's a lot of solutions now to actually do this using you know puppet like a black box like solution allows you to do this and it's secret management right you don't want your secret city in plain text inside of your cozy goes like yet and that's the end so you need to come up

with some type of software that's going to have encrypted blob inside there and those gets encrypted on the production servers because you know developers will eventually leave in a day or you know will get access to code repo closed do and if that has access to production secrets and they know there's high production environments people online

okay that is a presentation so thanks for coming out if does anyone have questions ok I'll be up here if you guys have questions or concerns or want to throw stones I mean Roybal thanks for coming out appreciate it