Microservices and FaaS for Offensive Security

welcome to breaking ground the next talk is by Ryan and it is micro services and functions as service for offensive security so Ryan okay afternoon thanks for having me here and thanks for coming out to my talk today I'm going to talk about micro services and functions as a service for offensive security a little bit about me my name is Ryan I worked as a penetration tester at Centurion in Singapore and my first discovery of functions of the service started in January 2015 where an AWS came out with lambda available for general preview to all customers and what really became interesting about lambda to me was that it allows you to run 1 million executions of your code

for free every every month and essentially the idea is you upload your code and that gets triggered and it does some function for you similar to the idea of a micro service yeah so the when lambda first came out it only supported nodejs and what kind of got more interesting is when I discovered lamb - by Erik Hammond and that allowed you to kind of get a shell so you could run some commands inject that into the lambda function give you the output and you could explore this temporary shell environment so this led to the idea of server lists and server list is this concept of where you don't need to worry about the server's managing them running them

keeping them running paying for them when they're not being used and the ability to kind of scale up based on demand for your code if we look at the stack functions of the service is on the far right so we see that you just put your code in there and then the interpreter such as like Python and nodejs and everything down from the stack is all managed by the service provider one very simple example which is always given by lambda is you have a photograph taken by some users they upload to an s3 bucket and that triggers a lambda function and that will in turn maybe add some terrible filter to your photograph and then upload that back to

the user so we look more security perspective Airbnb came out with stream alert which is a way to scale up your log parsing ingestion kind of rule that you that you would apply to any kind of blog so if you have a scalable server infrastructure we're generating lots of logs based on user demand you can now have a scalable logging infrastructure to capture all that information run some rules this kind of detect abuse and then trigger another function to send maybe an alert the pager duty or slack and so on so it's been quite a few related projects with lambda that people have discovered another one is looking kind of automating so you might have a ec2

firewall rules that you want to allow a developer test H into a server so then maybe they hit an API endpoint that triggers the lambda function to run which changes the security groups and then the developer can log in you could also have like monitoring CDN Network so you look at maybe cloud flares servers and you automatically update that list into your your web servers security group so that CloudFlare can always connect in and get their latest cache content of your web servers so recently AWS came out with their web application firewall and they tied together the use of lambda to be able to make sense of the web application firewall logs in another way that you could detect abuse

and then automatically you know blacklist an IP address or kick out a user so now we're going to look at a simple hello world so if you have a few examples here to run through first it's going to be Google cloud platform essentially they only support nodejs at the moment but if you use Python shell by this extra bacon on github you can just have a Python script wrap it in this nodejs module and then you can just use your python script that you have so very simple we just click Next Next Next through everything as usual and we get to the zip file upload and basically we just need to wrap everything into a one

folder and we sip it up and we have a very simple nodejs file which is just gonna call our worker py and I work about py script just three lines in Python to do URL Lib make a call out to Open DNS get the car an IP address and write it out so we can kind of get to see a little bit of where we're offering from when we upload to Google Cloud so we do that we upload it and we test the function and we get IP version 6 address so what's interesting about Google Cloud and functions is that you get a native IP version 6 address and this is kind of unique compared to

the other service providers if we look at IBM's Open whisk what's unique about this is that they allow you to upload a docker image so instead of just having a piece of code or some script that you upload where you don't really have much control over the environment it runs in you can now upload like a docker image and have that run so a lot more flexibility there and this also they also give you a IP version 4 address so we talked about docker there's also this site called play with docker and the idea behind this is to leverage on docker swarm and so you go to the site play with docker calm you've clicked the

CAPTCHA and you get four hours of like basically a free shell where you can just play around with docker and see the environment and see what it would be like so they have a play with docker calm and they also have the code up there if you want to run it locally and set it up and test it on your own environment and essentially what they're doing is they're running it on AWS instance and they have a Python setup so some of the pros of this is that it's anonymous there's no account registration you don't have to identify yourself in any way just need to take a box on a CAPTCHA but the downside is of

course you use time limited and then there's a CAPTCHA so it can be difficult to automate kind of getting this free shell for four hours every time ok let's talk a little bit about cost so this is site called serverless kal calm which gives you a very good overview of the cost of different platforms if we think back to the example I showed just simple three line Python script to run get your current IP address and I'll put it it runs in three hundred milliseconds so they do measurements in 100 milliseconds at a time so this 300 milliseconds you could run that 10 million times a month for a dollar and 80 cents so we see and

you also get a million executions for free so we can see it is very cost effective to just run your code and upload it there and only pay when it runs and the time it takes to run so if you look across all the different service providers you can see Amazon and as you're clearly leading the pack because they have the most support by region and if we give a quick overview we can see the benefits of Google is that you get IP version 6 IBM you get a docker image that you can play with and for AWS you have 14 regions this year of 23 some of the differences like is your they support PowerShell so if your we

want to run like yeah you can have like more flexibility in the scripts that you're running so I would say in summary if you look at the advantages of function to the service this low cost you get signup credit but most of the like AWS you sign up they give you $300 it's pretty hard to use all that $300 if you're only going to use the function of the service platform you get unspecified source IP address because your code is being injected into some random server within their infrastructure which they manage and you have a global data center so you can imagine you can get maybe an IP address out of China and you can do

certain simulated attacks with that so this led me to what I started with Project thunderstruck which is finding use cases for functions of the service and offensive security and my whole goal with this is to explore the different cloud service providers and try to get supercomputer resources without paying supercomputer prices and today besides I'm going to be talking about searching an IP version 6 so a little bit of the past work in this area tobias 5-bit gave a talk at ccc about 6 months ago in december and he was building upon the work by peter van dyke and essentially their mapping out the usage of IP version 4 by looking at dns reverse entries so a standard compliant

name server if it complies to RFC 8020 if you go to search for a IP version 6 reverse address it will reply with no error if there is nothing at that specific address but something more specific and it will reply with NX domain if there's nothing there and nothing below it as well so by using this difference in response when looking or the reverse addresses you can narrow the search the amount of space you need to search and inside of IP version 6 so put more simply on a command line if you're running a dig and you dig for this IP address on the left you'll get a no error so you know that there's

nothing at that specific address but there's something further down and if you do a 5 you see you get an X domain so you can actually try these out on these IP addresses to see the difference Tobias has work he used the supercomputer as you know University which was used for machine learning so he had this you know 1 Gigabit Ethernet connection 80 threads is crazy supercomputer and he ran his script to kind of enumerate all the DNS records in IP version 6 it's the first first attempt ok in one week he found 70 million records but he also received a lot of abuse complaints from different ISPs because he's basically enumerate every single possibility out of ISPs DNS

server and all their reverse zones so some ISPs will just assign a reverse reverse zone for every single IP version 6 address search they have in their space including maybe IP version 4 and so a numerating all of them generates a lot of DNS traffic to that server so in the second attempt he then created kind of some parsing to detect auto-generated zones and the idea was that if he finds one that's responds and then three others nearby then he considers that to be a dynamically generated zone and he ignores the rest of that zone moves on so we ran a script for three days he got one point six million records now it's important to note that there are not all

DNS servers are RFC 80 20 compliant so this is not a foolproof way to you know search the whole space there is some limitations in this and what Tobias then did is he came up with a way of seeding so where do you start from an IP version 6 so we looked at routing information from route views and ripe NCC and then he carves out all the advertised IP version 6 networks and used them as starting points to start looking for reverse DNS entries within those advertised networks so of doing the whole IP version 6 just look at what's advertised and then drill down from there so in third attempt I ran his script ran for 70 hours he were

at 80 threads and he managed to find 5.3 million records and then in his fourth attempt he tried to scale it even faster by running more threads and he ran 400 threads and it ran for 22 hours but he only found 2.2 million and the limit that he reached is that he found out that he was only using one IP address on this supercomputer and his server ran out of sockets so when I saw this I figured this must be a good case for you think functions at the service because I'm running code it's running on all these different servers all over the place so maybe I can kind of overcome this limitation that he faced so I took

a look at this code they released and basically it's doing W get curl to get the information using bgp dump to carve out all those IP version 6 networks and then some sort in parallel and then some Python scripts to enumerate the DMS so my plan so then at the end of his talk someone asked him where's this data set and he didn't release it so he just said it's stored in a distributed manner across lots of DNS service so my plan was to kind of come up with a way to get same data set so I could start to look at some more interesting DNS entries that are out there so gonna follow the

same steps download the routing information pars it get the IP version 6 addresses start and numerating and go on so this is kind of the architecture of the plan that I had I have a get dated up py script can go and download all the latest routing information positive running through BGP dump write it all out to a text file then I run start workers and start workers it's just gonna start triggering lambda workers and every time it finds a no error response from a DNS server in a certain zone it's just gonna spawn itself again and this is because in a doriath lambda you have a time limited window of which your script can run so the maximum you

can set is 5 minutes so if you're enumerated a DNS entry zone and it's running for more than five minutes it's going to die and then you've lost that state you don't know if there's anything further down in there so making a recursive call to start again with the same information that's a way to overcome that and if I find any PTR records I'm just going to store them into DynamoDB and I use the web interface to look at it so I ran my script I got all the data I ran the start workers and I triggered all the workers to start in about five minutes in 48 seconds which it seems really fast and but I only managed to

get 250,000 records so scratching my head figuring out what what happened what went wrong with this and what I realized is that I didn't scale up dynamic DB enough all right so I had all these like maybe 200,000 workers all going at one time all turn around dynamo DB and dynamo DB is just limiting it to a certain limit to say nope you can only write you know 100 items per per second I'm trying to write like 300,000 per second so I got a kind of scale it down and try and figure out a way to more scale the back end so if you look at the capacity calculator that dynamic DB has if you want to do like basic just some

rough numbers I wanted to do 1500 writes per second to the database it would cost me 800 US dollars a month so it's getting a little bit expensive so I did my first run and the improvements I came up with was that I had to control the amount of scale that I have in my functions I can't always recall a recursive function I have to kind of slow it down so I came up with the timer if you look in the AWS lambda they have a context function you can see how much time is remaining in that script that you're running so if I have like a simple check that if there's 30 seconds or less remaining then I take the

current state and trigger a new worker pass it off to that one so kind of slow it down a bit I also created a depth search limit and the idea is that instead of trying to search the whole IP version 6 base I just search a little bit down at one time and then I use all that as a starting point to search a little bit further more specific and more specific and then starting the worker slowly is to not kind of overload everything that's going on and a trick with the name ODB is that you can scale it up for when you need it and then you can scale it down for when you're not

when you don't need that time so you can only pay for when it's scaled up and then you can scale it back down and pay a few dollars and the web interface for dynamo TV which looks like this is quite clunky and it usually has that pause button at the top so whenever you search for something it searches and searches and then it asks you to click resume and it's not not very great so what did I find so I found a lot of kind of KBM like management DNS entries that were out there so you can see things like the if me i lo out of bound like Drac the dell remote access controller we find

them on a lot of interesting sites so things like NIST gov is in there and some like service providers they just have like ESX servers all with their ILO's with IP version 6 and advertising them you know through DNS records I also looked like did a search for dot mil and I found some someone interesting dot mil addresses and gov gov you can see like routing protocols HS are P and V are PS they see a lot of network infrastructure firewalls and maybe some interesting naming convention that can give you a clue to what's going on and what those IP version 6 addresses are being used for I also managed to find some interesting infrastructure out there so

if you look at like a clinics a big data center provider they actually have site site links between all their data centers and they both about a lot of interconnectivity and when you look at the DNS entries you can see not only where like physiographically where those data centers are and where those routers are for the IP version 6 but also the clients who are using those systems so that's kind quite interesting to kind of see that you have like CloudFlare and oracle box and twitter and vmware and netflix you can kind of see all their customers you can also see kind of loopback addresses for routing devices right so if you look at networking

devices usually have a loop 0 of L o 0 and that's like the management interface for that network device also managed to find the kind of d-wave quantum computer that nasa uses so they have DNS entries in therefore I guess they have like a firewall a monitor a QC I don't know what those things are but they look kind of interesting alright so in summary I've managed to replicate two biases work using functions as a service to do it for much cheaper without access to supercomputers I didn't receive any abuse emails because I guess nobody really can tie it back between me using Amazon service to do all this DNS recursive queries versus you know Tobias

for using an isp using his university and everyone kind of knows word of report then to some of the things i learned were to kind of avoid using a recursive function all the time so when i was practicing this when i was trying out what i was doing i basically ended up recursing too much and spawning up tons and tons of workers and so i kind of got to learn how to control that and build something in place and in terms of the backend when I first started I used elasticsearch but that didn't scale very well it didn't scale on the 2gig of RAM like digitalocean server that I used and using Amazon kind of hosted service for

elasticsearch just was too expensive so I think in order to kind of take this a bit further it's about getting more data for seedings so looking at maybe like trusted sex they have the hard cider which is very interesting kind of gets same idea like getting more routing information figuring out what's advertised and then using that that data may be looking for some more public datasets as well the good thing with dynamic DB is you can set up triggers so when there's a new write into dynamic TV you can trigger another lambda function so you could trigger another lambda function to maybe do a port scan or maybe look up in like census mr. lecoeur

or like show dan to try and figure out what's the publicly available information on that IP version 6 and the trick with AWS lambda is that they give you a free those free 1 million executions which you have or per region so if you spread your function out and your workload across different regions you can maximize getting enough free time

okay so if you find this interesting I'd highly recommend you check out some previous talks on the left we've gone 16 milliseconds by rich Jones it's also some talks that happened at blackhat last year and this year there's a talk tomorrow at blackhat and another talk at Def Con on micro services and then I'll be speaking again on Def Con on Saturday but I'm going to be talking about two different attacks so one is using just creating a distributed denial-of-service attack and then brute forcing SMS OTP so if you find this interesting some more things that might kind of spark your interest and kind of looking into this this area is AWS also has a kind of high

memory instance for lambda which is 1.5 gigs in memory and you can run and two hundred and sixty-six thousand seconds for free every month in this high memory environment and if you want to look more into like the china hosting there is a Alibaba cloud allien but you need a plus 86 mobile number and you need to go and register on the China website and then I think IBM's open whisk with the docker support is very interesting because you get much more control over the environment that's running yeah yeah and if you want to try it out on your own I recommend looking at this github repository on the right because it's used what's used in a play with docker

if you want to if you don't want to play for a service and you want to try it out yourself so I hope that it part of my talk that I was able to kind of get across the message that my interest in function of the service and try and generate some of your interest it kind of sparked more security projects in this space and scale all the things and that's the end of my talk and I'm going to be putting up my slides and some code examples as a github link below

when do you want to take questions you have ten minutes more yes any any questions found you can raise your hands again I bring the mic to you yeah if you are doing some of these things with AWS in terms of trying to scan other sites and so on aw it doesn't particularly like you doing these kind of things yes so they'll shut your account down and they'll do all kinds and stuff like that so you have to be a little bit careful yeah I mean you raise a really good point if you look at using ec2 instances and Amazon you always get the abuse emails because you don't sign up for like pen testing requests which is what

I always end up doing but if but I didn't get any of those similar abuse kind of reports using lambda so maybe there's an opportunity for Amazon to look deeper into monitoring lambda

questions anymore

fine Thank You Ryan okay thanks [Applause]