CG - All Your Cloud Are Belong To Us - Hunting Compromise in Azure - Nate Warfield

hello everybody my name is nee Warfield and this is Saul your cloud are belong to us a little bit about myself I a work for Microsoft as part of the Microsoft Security Response Center my day job is making sure that you get Patch Tuesday every month if you run Windows I own all of the vulnerabilities that come through for hyper-v for Windows for all the server products the kernel which means that last year I got to handle he lovely vulnerability we call it MS 1710 which turned into one and cry which turned into not petia and I also got the own spectrum meltdown which yeah it's been a busy year before I worked for Microsoft

it's been about 18 years doing network engineering was they admitted gray hats since this is being recorded by the NSA I'll say that my first act was over dial-up I was like about 12 years old and I've spoken at SAS this year troopers you can find me on the Twitter there and I have some corrode that I'm gonna talk about that I posed to github just because I had a conflict of interest if I don't give it away for free so what happened well I think it's important and I'm not gonna run through these bullet points because I don't really think that it's necessary you can read them yourselves I'd like to start with a little anecdote back in the late

90s I was one of the first kids that I first kids up my neighborhood to get a cable modem and I discovered and talking to a friend that was going to college across the country that once I convinced them to do an IP config all and send me their IP network information off their windows box I could plug it into into network neighborhood reboot because it was Windows 95 and when after the thing came back up it would browse and I was able to browse all the sudden that's at their college and I could see all the file shares hit IP zat see dollar sign look at people's mp3s fast forward to when I came to Microsoft

in 2014 I was trying to spin up an extra DNS IP address on a system we already served DNS through to the Internet it took me six weeks of paperwork and approvals and security reviews for DNS on fifty-three UDP right so the the sort of thing I'm trying to get at is there was a lot of lessons that were learned the hard way that were fixed in you know 1518 years cloud networking comes along and what I've been seeing is it seems like these trends that we were these these improvements we were making are now going backwards right so we're seeing every single VM is being exposed to the Internet people are just spitting things up

they're leaving it there they're not patching it and that's what I'm gonna get into here a little bit is some of the configurations on these VMs that come out of these third party like ayahs galleries are just ridiculously insecure and which led us to 2017 most of these are attacks that I had to track down and as you're this stuff with the exception of I think the s3 bucket stuff which is not really my thing that's more Chris Vickery um actually you see Chris I'd love to talk to him never met the guy so how did I start finding this stuff in see here August of 2016 one of the security fees I was reading talked about people were going

into these Redis machines and they were putting this key called crackit in there but this key was was a root SSH key then they would come back to the Machine SSH and his root do all sorts of nasty things to Redis and I'm like I didn't ever heard of what Redis was so I looked it up and I'm like okay this is no sequel database system fast forward to about January February of 2017 people start doing these attacks against and a whole bunch of other databases the points that I say have here they're not really trying to shame anybody but those are quotes taken directly from the security pages of all these no sequel products right they tell you in their

security guidelines don't put this on the internet never expose this thing to the Internet set up a password so when you tell somebody not to do something what are they gonna do put it on the Internet so couch to dupe elastic Redis Cassandra I thought there was another one but I can't remember off the top of my head what they were doing it wasn't actually ransom wearing these things they would come in they would just drop the database they would put around somewhere note in with a Bitcoin address and be like hey if you want your database back just send us some Bitcoin to this address which was if you were foolish enough to do it you're not gonna

get your data back they didn't actually ransomware it wasn't encrypted it was just gone globally 100 plus and he was around a hundred and forty five thousand machines got hit with this between January and June of 2017 what I added doing was just looking at Wikipedia to figure out what's gonna be the next one that they go after and it was almost clockwork that they were going after these things Asher we had about 3,800 customers that had this happen to them so how did I find this stuff in Azure as you probably would imagine microsoft owns quite a few IP addresses there's two point actually that numbers a little wrong we actually today have about 2.6 million IPS that

are facing the internet at any given time which if you've ever reported scanned a network for scanning is very slow and the problem with a port scan is just because that ports open doesn't necessarily mean this database has actually been humble a compromise it or been dropped um the other thing that was fun is that every no sequel solution runs on a different pour so even after I was going to go and port scan two and a half million IVs tomorrow I'm gonna have to do it again what I'm have to do it again I'm at the view again so the only way to find these things is to actually see the database names right so once you

you can't just port scan it even if I did all I see is okay it's listening right I don't know what's there it might not even be that thing so I wish I told you I had a school fancy tool the Microsoft wrote ourselves but no I did all this with showed an which kind of tripped people out because they're like I thought sure it adds just that site where you go and you find people's webcams and like stupid things they plugged in the eater and it's like yeah but you could also use it for blue team stuff during want to cry we had to go and figure out how many SM VIPs we had

that we're facing the internet just to figure out what our risk was our team that had their own port scan solution so no no no no your showdown doesn't know what our network looks like we're gonna for scan on ourselves I'm like I can give you the data in eight minutes they're like no no no no no you don't know what you're talking about 14 hours later when they got done they sent me this list of stuff I'm like well can I see what you found so ID duped their list against my list out of 14,000 IP addresses showdown had nine more than they did so like less than 1% difference and I might dude you could have had this

yesterday so the other thing to you is I got all this metadata right I know what version of the operating system it's running I could see like the everything is searchable I could see the database names and I can dump it all out through JSON which was really useful at first and I would just say okay doing was going to Wikipedia say find all of these different products and I would just download everything from that was in my IP space and started looking around and I was trying to go I started to catch these attacks within like a couple of days of when they started that was less than efficient and I like to think that most of us are lazy so we

write code to automate the stupid stuff so I talked to John Matthew Lee the guy that built showdown and said dude you know what you should do you should just do this stuff like natively you decide a showdown and he's like okay write the code I'll put it in there so I did last year December we plugged this code in I've since added I think two different database solutions and double the number of database names so what do you do is you can go to code and you can search for tag compromise this will tell you any database that's been seen and looks to be compromised when I was stuffing updating these slides a couple days ago there's still

like 37 thousand of these things on the internet the one thing I have to point out showed an enterprise license is not free it's not cheap I mean it's not really expensive for what it gives you but the same code that runs behind shoten that does gives that gives you this information is up on github you can download a JSON file with a free account run my python script against it and it'll tell you what ip's the database solution and then whether or like what the database name was which was a couple slides back that big list of IPs and weird names I probably should have been more clear about that so this got me

thinking a hundred and fifty odd thousand people on the internet did all of these people make the same stupid mistake do they really go and install not and read the docs put it on the internet and not have a password right I'd like that seems doesn't seem that like feasible so when I started looking at was well okay and Azure I know that this isn't somebody spinning up a server in their data center right they're going and they're saying okay deploy something um actually will pause raise a hand how many of you actually do anything with Azure it's more than I expected about AWS all right that's expected so what I figured out was maybe there's

something that's a systemic flawed there's a problem in the way these images are uploaded into the Azure gallery that people are not even realizing this mistake when we when talked to customers we got responses from like why they are you looking at my network or my stuff - I didn't know I had that thing on the internet right so it was this this weird like imbalance of it wasn't somebody saying yeah I know I made a mistake you know I'm a bonehead so I started looking into the azure gallery and it has this thing called a network security group which for all intents and purposes is a firewall that's set up with your virtual machine

when you spin it up the fun thing about this is this firewall configured by the person that builds the image and puts it into the azure gallery right so whoever publishes that thing they're like I think this is a good idea to open up all these ports right and if you're not paying attention when you turn it on they're all going to be opened up as soon as you spin this thing up you can configure the staring deployment it's optional I've tried to get our team to sit to make people look at this before they turn it on and they're like yeah the automation and then they'd have to actually stop and they couldn't script

all this stuff and I'm like okay now the thing that is different now is cloud than it was with the traditional networking that I kind of grew up in is like with Microsoft we had these big back-end networks of you know management from back flans we could connect in and we could manage these things over a private you know non-routable life.you space now with the cloud the only way to get in is that's the state you're already for you over the Internet right most people don't use back-end VPNs into their cloud networks so I can accept that if you're gonna have to have at least maybe two ports open like you know your management forward and maybe

whatever the serving plane of the web server or whatever you're wanting to to do with this so I enumerated everything in the eyes or gallery dumped out all of the configs all the firewall configs of every single vm like 3,000 of them that are up there and it's funny of them is 96% exposed more than management so I'm like okay that's weird how many other forts are open 562 this number is probably a little updated or the number of you just unique in random ports that are opened up by these VM images the thing that's really scary is there's like 135 37 138 139 443 or 445 I've seen and nets NFS I've seen I haven't seen

telnet which kind of blew my mind I was expecting to see talent but I'm like you have frickin that BIOS open to the internet by default and it's all these big large commercial storage providers that are like hey spin up our storage thing and it's got every like file sharing protocol known to man exposed to the Internet as soon as it boots up ya know finger on the actually the worst offender who I've because this is being recorded won't name they open up 83 ports to the Internet as soon as you spin this VM up yeah 83 so I've actually reached out to a few people the one that was the best was bitten ami they I found

one of them images had port or one of the new sequel ports and I said hey guys your Doc's say that you only open up SSH like what the DTF and they in like the two hours I got a response from their director the next morning they fixed the image I'm like some of the most of the other people were like no no that's that's required for the image to work and I'm like no it's not and if it is maybe you should tell the users hey when you spin this thing up you're gonna need to turn that on but don't turn it on for them so for all of you AWS users are probably thinking ah massager it's

Microsoft they're idiots AWS doesn't work like that I've had guys from AWS tell me AWS doesn't work like that yeah it does so it's hard to find it's kind of buried but this is the one-click deploy for AWS is Azure or Amazon Marketplace images and I've redacted this to not shame this vendor but yeah they spin up all these by default as soon as you turn the image on the thing that was tricky is that a juror I'm sorry Amazon doesn't have an API that I can enumerate like I could with Azure Microsoft we'd love api's so I had to I can only get a few pieces of information out of there and I filed a

request with AWS say hey it really cool if I could look at the default firewall config I think they figured out what I'm trying to do and they're like now No so as or has about like I said 2,000 2,500 of these third-party vm's you can turn on AWS has 11,000 of these things now a lot of them are Amazon or AWS is stuff and what the thing that I find really scary and I'll touch on this a little bit later is you can build in a VM image in AWS and you could share it with people you want to run my VM image on your network seems legit so a lot of the other clouds

need to do the same thing right so I looked at digitalocean I looked at Google I looked at Alibaba cloud and when I was looking at the attack traffic and like the baldies machines they were getting pop it's no sequel you know I'd hit tag compromising it's like AWS as your digitalocean it's all the same the Big Five cloud providers so I'm thinking this is a systemic problem I kind of got bored of trying to dig through at the eyes after the first two big clouds so I didn't set anything up with digitalocean but if you guys use it and want to want to pay me later and we can look at their stuff I'm happy to help the other thing

I found what I looked at these images in Azure default passwords so third party now because this is being recorded these is very much the third-party stuff Microsoft we have our own images that we publish we update them every month when we release Patch Tuesday I actually haven't found any flaws there it would be really embarrassing if I did and I would absolutely tell you so some of these things have a default password configured in the metadata of the image that you turn on right and you don't even the API I spent a lot of time reversing it out of the PowerShell code and then I realized they have we have a github page with the entire thing that's

all perfectly documented I was like god dammit I could have just looked at that but you don't even need to be authenticated to hit this end point you could just hit it with a simple HTTP request it's like a 12 mega and numerate through all of it pull out all of these configs and look out I'm without ever even having to like have an account or any sort of access to a sure so I found super simple script was like if description contains user name and description contains password flag it his password so that's not exactly the password it was like two two letters difference so I mean hey they got the leet-speak thing right because that's

super secure and they always say oh yeah I'll make sure you change this after the instil the irony is the biggest offender of this is a company who sells secured and hardened images in the Azure gallery I'm like I get the out of here but it's fortunately it's just databases right nobody cares about databases and nobody ever puts databases online that's global statistics that's not just Klaus so yeah there's a there's a few databases out there I'm not allowed to try to break into customer networks as I'm not on the red team but I'm sure there's a few databases in our cloud that will probably have these creds that you could still use so switching a little bit of gears here I

really enjoyed hunting threats in Azure trying to protect customers you don't make the world a safer place and all that so any time in new you are see or any sort of a Voland came out I'd be like ok let me go see how if I can find how many customers are at risk for this so this one the it was an exome mail server RCE and I think it came out back in March and I'm like all right let me take a look and see how many customers are running this that we should say hey like the POC is out there you guys need to patch this at the time there was like 17,000 ip's that had a mail server of

some sort right 425 567 980 don't quote me eat all the mail server ports um so I was like ok at least I could do product exome I'd showed an one of the other metadata is is that if it can identify the product that's running you could search it with a product tag right he's got 100 something products that you could find just by searching that so I downloaded it all and I realized that inside the JSON file it has this part that's called a common platform enumeration field which is just a standard that's designed to say okay this is the version of whatever this thing is so I was like okay parse it out

grep for these things look for the CPE information this is actually from the file I pulled down as like I said back in March I was like ok this is pretty cool out of 17,000 found 1221 VMs that are running a version of exome and are actually versioning and running the you vulnerable version of exome so I was like ok about five minutes of my time I'm like cool that's pretty awesome right 2 million IPS seventeen thousand twelve hundred five minutes can it go better yes you can um so I paint John again I was like hey dude you know what we should do you should have showed and just automatically a tag vulnerabilities based on if I know the cpe number if I

know the CPE platform information of this thing and I know there's a vulnerability for that thing because the there's a database out there that I think might err or NIST puts out that actually Maps all this stuff out for you I was like dude you should just tag the stuff automatically he's like cool write the code I do don't even work for you cuz all basically I did a lot of the legwork and my day job got in the way and I said hey here's like you know tons and tons of information and so two days later he's like alright it's there so now this is another one of those interesting features only available with

the enterprise API and this is not something I have code that will give you you can search by a vuln for CVE across showdown and it'll tell you every IP address that's vulnerable to it now probably imagine what this could be used for the idea that when I talked to him I was like dude you can't make this available to just your average script kiddie users like this will be an absolute nightmare if some could just go and say like I'm gonna find everything that's vulnerable like that otoscope own script that came out back in march or january ish he was livid about that he's like ah god I just making me look bad so I was like let's

just make this enterprise only so if you have Enterprise Search the you could search by vuln sort it by your organization if you go and just hit an IP address like if you searched an IP and show down it'll tell you what vulnerabilities it has so you can see it there from like a single IP at a time what we didn't want is people just feel to download a swath of like every vulnerable box on the internet so this now I can say okay show that and tell me how many bones so many boxes are vulnerable to this thing in my organization this is a few months later so the number had dropped dropped pretty

drastically so that's the top 10 vulnerabilities that are out there on the Internet as of a couple months ago pretty depressing when you see ones that are like 2013 so the other thing that's interesting when you look at like the actual detailed data of an IP address you'll see this vulnerability and I'll say verified true or false when it says verify false it's implied to be vulnerable so things like some versions of Apache you know Apache 2 point 2 point 6 or whatever there are vulnerabilities out there but if some of it it's like okay if you have fast CGI or you have these specific modules enabled then you've got a vulnerability he's not testing that

heart he's just looking at version data and mapping it out so if he hasn't tested for it implied false means man might be vulnerable might not be vulnerable verified true means that it's confirmed to be vulnerable for example ms 1710 he actually tests for the lack of the patches in Windows SMB that's on the internet so it'll say yes this is confirmed vulnerable I've actually connected it's done the handshake realized it's an unpatched version but it's I think it's fairly useful another thing I started doing and I'm kind of jumping around here I know I started running a database a year and a half ago highs pull up import stats out of show down for all the major clouds by the

five largest countries and then globally and I just do it every day I dump it into a database that's on the internet no I'm kidding I just dump it at all my sequel box and I can just kind of watch trends right I like to see I like patterns it's one of my things so I'm like okay let's see what maybe something stupid like if I see a weird spike of some port in Azure I'm like what is this like new level of like retardation so I noticed this thing called MQTT which is the mosquito messaging protocol it's like a message brokering thing I've never really heard of it and then I started digging around

I was like oh that's very popular with IOT facebook Messenger apparently uses this thing I don't use Facebook but if you do that's probably why your phone's always going off at you as you're an AWS both offer these MQTT solutions but here's the spooky thing I found in the last year and exposure went up by fifteen hundred percent so this little screen shot I took the other day when I was putting updating these slides and I'm like that's weird I'm like that numbers he was way higher than it was in March and then I go and look back and I'm like wow it went up by almost a half a million machines in 12 months so NK

mqtt in and of itself maybe it's not a vulnerability maybe it is but here's some creepy stuff you can do this is where it gets fun so when I was in I went to Germany to speak at troopers this year and was putting this slide together I was like what are we kind of cool stuff I can find with mqtt anybody ever heard of a software called own tracks sweet I want to blow your minds so own tracks is this open source solution that you can run its MQTT base it has clients you can put on iPhones Androids laptops whatever you want and it feeds location data to a brokering system this brokering system is exposed

to the internet with no authentication by default this screenshot let me step out here I redacted this because these are all like I did this one night in Germany and it was live and I was like oh that's creepy I don't really want to have somebody get killed because of my like hackery stuff so the important one to look at it I think lenovo so that i said ok let me take this mosquito client and hook into this system boom let me show me the subscription data here's the subscribers lenovo so i look weird to you it's latitude and longitude it's also let's see battery percentage of the phone these codes i don't remember what they

stand for and then there's actually it tells you one of these fields that I might have snipped off till Z's last time at phone home so I'm like ok I wonder if that's really legit so I plugged it into Google in the morning and I was like ok interesting but you know not really it's just one data point so I went back from the conference that night plugged the same IP drive hit this thing again looked at it again pulled out more data huh they moved so that's some poor person I think that's Czechoslovakia that's phone is tracking them and sending location data to something that I can see on the internet without anybody knowing that I'm doing

it I found systems in the Midwest that this some they had a trucking company and they were tracking all their employees with and with this stuff and I wrote a little loop script to say hit it and I could watch like every five minutes or so I could see where their trucks were going in real time I'm like I shouldn't be able to do that so more this is this is a so much finding compromise but this is like what I'm finding on people putting online I'm just like you got to be kidding me so Johanna is really cool at finding stuff from the outside looking in right I can see all this stuff I can see

pwnage I can kind of use it as a blue team tool versus just a script to the Kitty tool the thing I can't see showdown is something coming out of my network right and this is really cool these are these are this is stuff that I just put together this week just for b-sides and I actually was lucky enough to go to drinks a couple of drinks with the guy that built this system right everybody's like I'll buy nice you can't find them with Showdown you can't do all these other things I showed and I'll show - Yonah Yonah they're not invisible anymore has anybody heard of clay noise so great noise is super early it's a

super cool technology I'm really excited about this I talked with the guy for hours the other night like he's an awesome dude he's built he's building me features in like hours that I've asked and I'm just a beta user I'm like dude because if I could do this and this and this and this it's got a network of sensors and a whole bunch of clouds they just sit there and they pick up things port scanning them they pick up things doing weird brute-forcing they pick up things doing weird like HTTP path stuff looking for JBoss forearms looking for PHP myadmin or arms looking for all i've got a whole list of them so what's interesting is Andrew his name's Andrew

he gave me access to this like six months ago I was I was apparently used at number two and I started writing Python code I was like dude it's amazing look at all this stuff I can do so I sent him a script and he says it's a John a showdown and John's like that's cool let's put that in showdown so now you could search tag scanner and show then will tell you every box that's been seen scanning the internet or proving the Internet via gray noise data I can't remember if this is an enterprise license thing or not if you haven't access go ahead and take a look but he's gonna be speaking at Def Con 26 his

name's Andrew Morris he'll be at the AI village on Friday it's a really cool technology and because it's in its infancy he's super flexible it's like you want features if there's something your organization is or your organization has wanted or needed just talk to him he's very reasonable I hang out on a slack channel with him and a bunch of guys from rapid7 and binary edge all day and we just talk about all the stupid things we see online so the little clip it's what I did with the script that I sent him was say okay I'm gonna look at for me I dumped all the IPS that he saw from my Microsoft ASN that have been probing the internet and

then I would take each IP that's okay now look this up against Greg Noyes look this up against show Dan and I would say if I see that this thing has been probing the internet for say RDP and it has RDP open via showed an that box is probably a bit hacked alright somebody got in with RDP and now they're having it do whatever they want so that's not a feature that showed n knows about yet I'm trying to work with them with Andrew and Don to figure out how we could do this type of a thing right now it's just code that I use if you actually have a use for this I'm happy to share it find

me or ping me on Twitter or whatever you'll need API keys for both services though um yeah so probably about I've picked up just in just coming out of major cloud RDP berms SMB worms SSH worms telnet worms which is weird because tell nots not usually open in my cloud the fun thing is some of these script 80 botnet writers are so arrogant that they'll stick the name of their botnet in the user agent string so these things are you doing HTTP probes looking for stuff it'll tell you what worm it is so drain always has this so you could say okay like look for this like find me things that are seen with this botnet

find me JBoss worms find me these different vulnerabilities and you just dump all the IPS out he keeps a list of like the top 50 worst networks in the world of like how dangerous they are from probing stuff and it's mostly Asia I hate to tell you AWS guys AWS was like number 47 a couple weeks ago all right so here's a fun one like I said I did the Emma 1710 Wang I can't tell you the story of how we got that in detail here you can find me later I'll be black hat and DEF CON all week too so I this is probably even talked about too much of the news but this redact this this

unnamed government agency had this really beautiful SMB v1 I exploded I what I mean really beautiful like I see all the Oh days that come in for Windows and this one was like you can see there's the difference between like a researcher or a PT and then you're like who who nation-state like if it could shine it would yeah you're like wow dude like I thought the only thing they don't do is comment their code yes with this unnamed government agency put it in their Metasploit clone home and then of course like any government agency did would do they lose this thing so yeah Oh unnamed huh hi NSA so the thing that was fun about this was

so I came into work that morning when shadow workers put the password out I don't know why they do this stuff on Friday but yeah we spent the whole day ripping - this thing apart with like the initially the internet was oh my god like Microsoft is going down let's look at this thing and we spent the entire day ripping through the exploits looking at the post exploit testing it out and we realized we fixed all this stuff a month ago or almost a month ago yeah months ago there we go 31 days so the at the end of that day we got one of our managers posted this thing online they're like yeah we're good

everything's fine we're like alright and everybody patched right I mean in any ways even if they didn't patch no sane person's gonna put SMB online right as you find out about 28 days later yeah they did so when this came out the first thing we did was like okay we're looking at the exploit modules I'm saying okay well let me see how much as your exposure I need to worry about right like in the morning we're like this could be a zero day that we don't know about this could be really bad and of course because like what is any script you to do when somebody's cool shiny nation say cool toolkit comes out they

just start spraying at the internet right so the thing you probably all know about this if you don't the quick synopsis most good agencies don't want to burn their precious 0 days so you don't send them unless you have to you to this end they had this tool called double pulsar which would very secretly tell you if this machine have been compromised by your sexy SMB zero Today Show and couldn't detect this right there's no visible signs of compromise at first there's no database names there's no like you know yoloz or any of that stuff so it used this SMB era code right you set up the session you descended this weird command that was unimplemented and if it had

been in fact it would give you a 0 x 51 response versus a 0 X 50 which everybody else would would do like SMR Samba and all these other guys will do the same thing so I had to manually scan all these things I think some guys from NCC group were the first ones that wrote a python script they could do this whole session set up and then test for this thing so I just dumped it through dump the IP ease out did a bash loop it said ok test everything in my cloud for this and I was like sweet there's only 50 of these things like the end of that we are think that

by the Monday Tuesday after the shadow brokers drop that I only saw fifty implants I'm like out of 14,000 IPs that must mean that everybody patched right so I mean I work for him so I see we're like dude you should accomplish hey hey like rubbing like we fix it on time everybody patched on time like we got nothing and worry about like I said 28 days later we found out that actually they had it patched I'm not gonna bullet point all of these things mostly I like to do this comparison right everybody was like I wanna cry I you know and I'm still I haven't yet met Marcus I'm looking for him though because I want to

shake his hand and also apologize for what my government's doing to him but the thing was interesting with wanna cry is it was very childish right the most important data point on this slide is the fact that when wanna cry hit Asher's as an exposure was 14,000 - ip's a month later six weeks later it went up by 13% so you think like ms70 10 which we probably should have made a bigger deal of like how telling people how important it was to patch it okay shadow brokers drop this o day okay if I'm on network I'm like I know my SMB stores out there I'm like okay I should probably patch this thing maybe maybe my hacker dude at

the base was gonna be like Oh bro look at this look at how easily I could pop your Windows Server no we went up the thing that was fun with want to cry and why my theory has always been that it was a weapons test to see how effective this thing would be right it only went after having a 1710 and its propagation was kind of it was just like a script kiddie wrapping some ransomware around an exploit they didn't understand first it's not Pecha which is a obviously would like cyber attack our cyber warfare attack right I don't not going to imply who would want to attack the Ukraine but right instead of this

wall cause everybody pretty much agrees it launched V I think it's Southeast Asia some SMB machine maybe they didn't even mean to hit it maybe like I wonder if that works they forgot to turn off the propagation when they tested it not petia they hacked into ME docks they backdoor their update system that there's like hey patch Friday and pushed out not Pecha and took out the Ukraine in fact it was so effective one of the Microsoft satellite offices got even infected with this thing like they had to take them offline because they're like if this comes back over PPN yeah punch of the water cry attack was more effective globally Ryan I think mayor's went down for a months I think

the stats are something like 50,000 workstations had to be reinstalled because of this but the thing that was fun when I started looking at this I'm like there was obviously going after Ukraine right just tax software that's only used by that country it was all very very targeted but it got out right how is it gonna get out well it doesn't just use a port scan it has all these other propagation methods right PS exec VB cats EMA 1710 if all else fails right this thing was designed that if you get into one unpatch box you can take out the whole network right I don't care you could have the most up-to-date Windows 2016 server with all your patches if

you're one Windows 2000 box gets hit it's gonna feed us exact it is gonna ransomware your shiny new so the VPN thing is what I I kind of want to harp on a little bit that spooked me I'm like okay so this got out of the Ukraine right we had to shut down one of our offices because of this were like okay we can't let this thing back in our network does anybody ever heard of Express route and Direct Connect Johanns okay mm-hmm so yeah you know what it is it's a VPN I'm gonna put this a little bit together right so bad security hygiene in the cloud cloud subscriptions that you got anybody maybe anybody who's

setting them up maybe your sales guys are setting them up they'll I got to do a demo I got to show off this thing they're not patching it you may not know it's there your Express route Direct Connect connection comes back to your corporate network how are you managing your apples I've talked done this talk a few times I had somebody tell me yeah it's really hard as cloud he's like a she's got so many IP so what we ended up having to do was just let your entire as your IP range into our firewall I'm like you did what now there's ways to do this I'm not the guy to tell you this because I don't

sell as your I just point out the flaws but there is ways to do it it's not easy it's it can be complex but just in flowing' the entire subnet all of our sudden that's into your firewall is probably not what you want to do because like mr. Jefferson over says we spend billions of dollars making as your secure we hyper-v with all of our red team just all the things we tried to do to make our cloud secure we built special clouds just for the government special clouds for Germany we have special cloud in China right we've done all this stuff to try to make it solid by the end of the day what you do with your subscription is

your problem right if you get yourself popped sorry so do all your policies that you have in your internal corporate networks are you pushing that out to your cloud stuff right you have to stop people need to stop thinking that I've got my fiefdom up like my data center and I've got this cloud thing like if you're setting this up the way everybody wants to seem to move to hybrid cloud and all these like multi cloud things like that's your network too and you shouldn't if you're not gonna let your sales guy turn on something in your DMZ probably shouldn't let them be turning on something in a cloud subscription if you're gonna do that give them their own subscription

put it off to the side don't let it into your network like keep them isolated right because there you're gonna get pop sometime or you could get pop sometime the thing about patching most people probably have a patching cadence right a lot of the stuff that I've seen it's just bad hygiene right these aren't like other than the SNB stuff this isn't like cool zero days a nation-state stuff like this is just like cloud scale permissions problems right this is dumb mistakes that we stopped making 15 years ago that are now coming back because clouds are the new shiny and people forget no it's basically just an extension of what we've always been doing um people

patching causes downtime right we hear this all the time we hear this in Microsoft my cloud provider does my patching for me right the people people really love the idea of running in the cloud because they're like that's cool man I don't need a network guy I don't need a firewall guy I don't need a server team anymore I can have my DevOps bro do all of it for me sure you can but what you just threw away it was all the expertise from your network guy your firewall guy your server guy and security guys right so you have this one dude who's not the on DevOps but they may not have all these skills right I talk with guys that

build Azure that are like pack and networking what's that it's software-defined networking I'm like oh my god so we do have ways to do this we can help you with this I'm sure AWS has the same thing I'm not a sales guy I'm gonna kind of breeze over this there's what we call path and sales platform and service software as a service what we would love what I would love is to see every go to everybody software-as-a-service we patch everything for you like we have the stuff when when past usually comes out everything underneath your says platform has already been patched for you don't even have to think about it doesn't reboot you never even noticed it just

fire and forget your good has is a little bit more complicated and not everything runs and says right it's something we kind of have to build the platform that supports it so if you can't run it as besides you can run it in pass does this shared responsibility bottle you have to set up how you want the patching to happen with pass right they have these things I think they call them failover domains pretty basically it's like load balancing networks like if you ever worked with like an f5 load balancer you've got all my web servers you're like okay we need to patch that one we take it out of service we patch a rebooted put it back in service allow

the rinse repeat that's kind of what um what path does right you say okay these are my resources these are my failure domains that way we know okay this is the first thing to patch we push it over to this other you know worker set reboot this now we can patch the other one it's minimal to almost no downtime but you do have to think about it or talk to your friendly as your salesperson I guess so that's the I'm done with the salesy stuff I was asked to put that in so I'm not like gloom and doom and the cloud is like coming down on top of us so supply chains photo marketplaces those I kind

of been crapping on how their Bad's firewall configs they have default passwords they're set up by somebody else that if you don't know it's some guy that built it for whoever cloud that they they uploaded it to one of the funny artifacts I found in Azure was somehow during the packaging process where you build the VM and you upload it to Azure you basically say okay here's my thing and we package it all up all of these vials get uploaded and put in as what they call artifacts which are these different little little things some of them are like set up scripts some of them were like firewall set up scripts or I've seen some weird ones that are

actual like bizarre shell scripts that are apparently supposed to run once the thing turns on like it's very hard to reverse engineer what this thing is doing the thing that's funny is I find things that are like it'll be like named Mac OS X slash desktop I'm like hey you just pulled every file out of that machine and uploaded it there these are supply chains and we have to think about it like that right being in part of Microsoft the ccleaner thing was really spooky and because that came out if we're like wow that's internal windows build networks like those aren't on the internet you could swell used to be able to search and not

even fighting them on the internet um until ccleaner came out and was like looking for them this happened a couple weeks ago or a month ago docker they found all these machines that have been back doored running in Dada docker hub mining cryptocurrency for whomever Manero drives me nuts because I can't track any of it they're really good resources right most of the people that are gonna go into a cloud network well you know that there's a super-powerful box underneath it you also know that it's probably a business or somebody you know it's it's not like going after an isp network like these are businesses like these are high-value targets and there might be some really cool stuff if

you can bounce through a you know Express rail connection back into their network there's also minimal validation these third-party images I went through the publishing process or I looked at the publishing process for Azure and it's like you set up a set of a business license you fill out some paperwork you wait a little while it's a little bit more difficult than buying a gun I guess but yeah once you're done you have the publishing license you can put whatever you want in the Azure gallery for anybody to use so I asked my boss I'm like so could I um build a VM image and put it in there and and do some stuff and I threw a few ideas of like what we

could set up a you know timer script to do a little bit of narrow mining or we could just start proving like the metadata instances and finding out the subscription IDs of where these things are being turned on or because the firewalls allow all outbound connections by default there's papa call back to myself every time somebody spins up a VM image I would almost bet money that there's somebody doing this already right we're not gonna see it it's probably some guy in Southeast Asia saying look do you guys ravines turned on because it just called home they do a virus scan on it before they put it up there they're also really old so the

only interesting piece of data I could really get from the from the AWS side was the age and I can get the age for me as your images to the average age and Azure of these things is 172 days AWS 717 day now that was in the spring that I looked at it might be worse now I'm sure there's a couple of vulnerabilities that have come out for most of those things in the last two years the thing that's fun is even if we did fine and even if some would say we say somebody puts a VM image up there it's malicious it's doing bad things it's being used by customers people are getting theirs the encrypted

currency stolen if we find it and we shut it down and they fix it everybody that deployed it still has to fix it themselves it's not retroactive right with PA's and says if there was something wrong with the underlying image that ran those pads and says machines as soon as we fixed it all those things would be fixed with these third-party eye as images when you spin it up you've copied and pasted whatever that thing is into your network it's not going away until you fix it or you know about it or you find yourself in trouble and there's no way around this it's just the nature of how these infrastructure-as-a-service works so 2018 when I first started doing this

talk in the spring i theorize that it would be the year of the crypto minor and now I'm keep seeing all this stuff all over the Internet I was like I should have made a bigger deal about it then I could be like I said it first so the one thing I the riveting I hate about but narrow is I didn't get in back in like September of 2016 when I was like a buck a coin now it's like five hundred something dollars a coin everybody's going to that right ransomware is really useless I mean its destructive but it's noisy it kills the host people catch it really fast right I'm gonna want to cry people new within

minutes they're like I'll crap something's going wrong kripp don't mind the ego just run it and if you're not super greedy and you don't take over the entire CPU you could probably run it for months before somebody even noticed um attackers figured this out and it started to change they're no sequel attack campaigns I forget when this was the trend found out they're like yeah this couch TV thing that they used to just be dropping the database out of they're not doing that anymore they're putting crypto mining software into it right so they're like okay now nobody's gonna send me bitcoins I'll just make some Manero on my own um that's three buckets let's think the LA Times a

couple months ago somebody managed to drop a coin hive in there and people going to the LA Times we're getting hit with coin I've like every day I've seen like ripped of this script of that kind of sick of the term honestly and anything that they could get into that's the thing that's spooky this is a very technology agnostic or a platform agnostic thing right I could do it on a Mac I could do it on a Linux box I can do it on a server I can do it on a Pho and I could probably do it on a smartwatch if I really was that like patient so yeah this Oracle server quarter of a

million dollars they made before anybody even noticed granted its Oracle so probably had like a thousand CPUs but still this is one that I found in Asia this is my last compromised thing when I was at I was in Germany I was turned on to this somebody said hey I have a thing for you but you can't tell anybody who I am because German law by the fact that I even looked at this it was technically breaking the law so this individual was not named or credited he's like yeah I look at these doc reports so I go and look I'm like oh that's an HTTP admin port there is no authentication by default this was like docker v1 HTTP

interface or the admin interface right I don't know oh good I tried to brighten this up this does actually show up fairly well so what they did and what I did up doing was just pulling out all the IPS again and looking for XM rig CCX Emmerich daemon what was interesting is the naming constructs yeah here's the funny one right kind smart right or kind of Swartz the thing that was interesting with like the NSA stuff is they had all these weird names like double pulse or eternal blue or all these goofy things these guys their docker mining their Manero mining campaign we're using very similar names it was like he's weird like it almost looked like a spammer

like you get emails from these weird people you're like but that isn't even a real name there was thousands of these things all over the internet they basically just we got in drop this X and read daemon found there was some ones I found ones in Alibaba cloud it must have been a bot because they were putting in like 50 of these things on the same host right there would be a little bit different in the name it was like 50x emmerich daemon processes I'm like dude you're greedy just leave it with one eye I don't really know how the mining thing works maybe it's more efficient to run 50 of them on one box the thing that was

interesting was the guy that told me about this said hey there proxying all this Manero mining traffic through these Manero proxies so apparently it looks bad if you have my coming from like 10,000 IP addresses for your one wallet so they hacked it as your customer India in Europe put this system this Manero proxy on to a couple of their VMs proxy all their mining traffic through these two VMs they had the code on github I think I think the guys like yeah he's like I wasn't sure if what they were doing if it was legit so I sent them an email he's like in two minutes later they just deleted the repo entirely he's like but here's the way

back link to look at it I'm sitting there I'm like yeah they basically just set this up mind Manero Manero if anybody knows how to find track this stuff I'd love to know as far as I know it's untraceable so I don't know how much they made and the fun part was we had to get our lawyers involved because hacked customers were being used to hide traffic from other like illegal activity right so we could just turn them off these were paying customers so we're like I'm so glad I'm not a lawyer I just basically threw it all over I said you guys deal with it it's gone now so yeah shoutout to troopers for

connecting with folks like that I hope to meet more folks like that here Def Con blackhat so final slide kind of wrapping this up some takeaway points if you're running in the cloud if you're gonna run I as especially if you're gonna run a third party is update that as soon as you turn it on like you don't know how old it is you didn't build it you have no idea what it's running like apt-get update or Windows Update or however you Debbie and other Red Hat does it these days just just do like you would with your normal network don't assume that because it's there Microsoft is making sure that it's updated we're not we let them put it up

there it's like Google Play Store for cloud VMs check the firewall settings before you turn this thing on right you the same rationale behind the updating it you didn't set it up you don't know what they did eighty-three firewall force to open to the internet no the other thing you can do for some roles pass it says just don't work right you might have some interesting middleware or something that you have to do that you depend on you can build your own image like if you're paying subscriber I know AWS does this I know Azure does this build your own image get it all tuned get it all set up the way you want you can publish it to

your own subscription and then only your guys only you can use it unless you share it in AWS but that way you can say okay you know if you're selling widgets and you to do this all the time that's the one you guys turn on if you have a specific role at least you know who built it right it's like you did with your fiefdom only now it's in the cloud I've been trying to work with the other team to get better information about this for just public facing stuff right the age of the image is it exposed anywhere to you as a user like it's in the API so when you go to turn it on it just says

here's the thing it's new and shiny no it's not it's like a year and a half old so working on that trying to get them to block the firewall policy stuff what I'd like them to do is give me version information at Damons and services too so I can say the next time there's like a shell shock I say okay show me every VM image that has this specific version of this thing and I can say okay now turn those guys off and don't let anybody to ploy them until they're fixed if that ever happens I'll put it on Twitter and brag about it and then last point if you use Azure you can get free

stuff for magic Security Center it's like basic basic security stuff you probably can do yourself if you hear it besides so that's what I got thank you very much for having me and I think I've got like nine minutes [Applause]