CG - Prisoner Number Six - Nimrod Stoler & Lavi Lazarovitz

hello hello everybody welcome to this besides us Vegas session titled prisoner number six really excited to be here my name is Nimrod this over there is La Vie and we'll introduce ourselves formally in a minute in this session we will talk about the prisoner in 1968 television series in which a former British intelligence agent is imprisoned on an island also known as the village with other former spies all because they know too much on this island also known as the village prisoners are only referred to by their numbers without prisoner being number six while escaping from this village is our number six soon discovers is an extremely difficult task nevertheless our industrious number six never ceases to try a quick question how

many of you heard of the prisoner or actually seen it wow this is fantastic so we're gonna have a lot of fun with the prisoner during the next 45 minutes and so in this session we play the role of prisoner number six a containerized attacker that knows too much and our village is going to be a docker container that imprisoned us during this session we attempt to escape from our village from our docker container with the goal of taking over the underlying host and even eventually running code on that host do we succeed well stick around to find out alright so let's jump into the agenda and I meanwhile might make sure that the mic is set up so alright so just before

I will cover the agenda how many know how the serious the prisoner ends all right so you have another reason to wait for the end of this session is if you already know don't spoil our spoiler all right sorry all right we'll start with a short introductory section in this section we'll we'll introduce visitor number six but those of you who are not familiar with it will also take the opportunity to introduce ourselves and one important thing that we're gonna do is try to understand what the hell prisoner number six has to do with containers and docker containers in particular so this going to be the introductory section will then move on to the section called welcome to the

village this is a container introductory section we'll try to define what are containers the conditions that we like and what is a privileged container in that context we'll see how our containers can be deployed and we also check if and where those privileged containers are used out there in the world so this is the welcome to the village we'll then move on to a chapter called knowing too much in this section of the of the session will focus on the risk the risk of allowing the prisoner to escape and honoring prisoners new prisoner number six or any other prisoner for that matter to escape the village may be determined as the escape of an attacker from a container to the

house may be so this is the knowing too much section we then move on to a section that my favorite called another number I'm a free man this this section of our session will be in this section rod will be introducing or demonstrating several privet container escapes those escapes are based on our Linux security research at fabric labs and are also based on two patent-pending x' we have on the concept of the preventive containers in this section the word will go in a step-by-step walkthrough inspired by the prisoner who yelled at the beginning of each episode and I think you'll see some of it I'm not a number I'm a free man all right now one more thing that I

should say about this in this section that demonstration will be based on a docker container playground called play with docker and you will see that in a moment it's a wonderful way to showcase how web-based live web is containerized environment looks like and at last we have the mitigation and closing section we'll be discussing the attack vectors and with a couple of mitigations and final thoughts so with that okay so did okay can you hear me now okay so this is the opening scene from the prisoner and this is our intelligence agent who is now entering his superiors offices at the british intelligence with the intent of submitting his resignation then heads back to his London home where he plans

to pack for his retirement trip abroad [Music] but insubordination has its price and our former agent now entering his London home is Dan sedated and kidnapped to the village okay all right so delivering a political and social statement concerning life in a technological society a society which may be regarded as a collective prison the prisoner is a war of attrition between the faceless forces behind the village a community reminiscent of Alcatraz and its most strong-willed prisoner our number six was ceaselessly struggles to assert his individuality while plotting to escape from his capturers those who are after the knowledge he possesses we we have been deeply influenced by the prisoner but we are not the only ones the prisoner has

been presented in several other media for example Homer played number five in an episode of The Simpsons where he was where he was taken to the village and was chased there by a big balloon which references which reference s sorry about that which references village as a rover would see more of the rover shortly I remain in the clash and eighties influential rock band and other music bands have all directly referenced the prisoner in their sounds and with Iron Maiden even incorporating an entire song with dialogues of the prisoner in there the number of the beast album the prisoner has influenced even movies and TV series such as The Matrix David Lynch's Twin Peaks the x-files and also

lost and also has been featured in the movie Shrek descried industry we believe the prisoner is still relevant today with the rise of tech giants and their control over our everyday life multinational organizations track our every move both in the cyber realm and the physical one brainwashing us with ads and feeding us with information they want us to digest we are all living in a technological prison alright so just before we moving on to continue that we use a story of

can you hear me well now let's move alright so who are we we didn't reduce ourselves properly so this is the world store a security researcher it cyber-ark labs and my name is La Vie lazarevich I'm a security research group manager at cyber-ark lab so we are both based in Tel Aviv Israel and so what is it that we do we both part of a group called charlie group which is focused on the security research of emerging technologies like for example things that we do is security research for containers for example when we do other hardware and software stuff as well our offensive security research is the ground of which we work to develop new

innovative ways to mitigate the attack vectors and back surfaces that we find but at the core we are part of the group to read things and look at their insights much like the research the container research that you're going to see in couple minutes so what this is us as now I think they're ready to get to know number six okay okay so who is our number six so in our story number six is a British intelligence agent brought to the village with other former spies because they all know too much in the village our number six is put to many tests in order to extract every information he has about why he resigned and the organization he was

working for in this session we are going to use number 6 as our attacker process and this attacker process is going is going to attempt to escape from its isolated container that it is contained in Oh as our prisoner number six is constantly trying to escape from the village that is imprisoning him so is our attacker process constantly making every effort to break free from the isolated container is placed just as the escape of prisoner number six from the village where he is kept would be detrimental to the organization behind this scheme the organization behind the village so would the escape of our attacker process from the confines of the container that is holding him be detrimental to the

organization behind the host one okay oh I have to use it I guess okay so just to recap we're using number six prisoner number six as an attacker process and our village is going to be a docker container which imprisons our attacker process just as president number six is constantly trying to escape from the village so is our attacker process making every effort to escape to now welcome to the village so this map that you see here I was actually exactly part of the field and it was given to any prisoner that was just accepted to the village and the village is the fictional setting where everything is happening here but who runs the village so apparently the

village is run by a democratically elected council with a popularly elected executive officer known as number two but everybody in the village knows that this process is rigged and the operators of the village are simply using it as a means of control their control over the inmates a weapons tools and alcohol are strictly forbidden in the village but there are no walls or physical barriers to prevent escape and apparently there are no visible guards a control room monitors continuously closed-circuit TV cameras located inside the village and observers spy on every move of the villagers in order to foil escape attempts with the aid of Rover a large white balloon-like device the chases would-be escapees and carries

them back to the village let's see now such an attempted escape with the rover [Music] International swimmer at the age of 17 Olympic bronze medalist she'll be out of range soon he was kept in training I must say well orange alert [Music]

[Music] [Applause]

[Music]

[Music] all right much like a container escape you'll see and you'll see in a moment I promise so just before we're getting into the more technical stuff in the live escape and so on just to get us on the same page I want to show you think about the definitions that we use during it during the process and the first one is of course container so definition that we use for containers is a set of software processes that are running within and abstracted an isolated environment this is the finishing we use and for that matter privileged containers are any containers allowing an attacker access to the host or other containers the data or execution flows

this is the definition of privileged containers that we use now it's important to mention here two things first one is even if we run a privileged container and if you know docker you know probably the - - privilege lag even if you run the container with the - - pivotal flag which assigns all capabilities to the container eventually making it a privileged container by deploying various security tools and defenses we can transform this privileged container into an non perfect container and we'll see that in a few in a few moments you understand what I mean second thing that I wanted to mention here is that we did not create or invent this concept or definition that we use here the concept

of privileged containers or the different definitions that you've seen here this concept was actually known since the inception of Linux container our modest contribution to it was the extension of the definition of privileged container so it's not only the - - privileged like that creates there are a couple of others other things and I'll cover it cover those elements in in a few moments and the other modest contribution is the realization of how an attack might look like based on those containers we'll see that actually camera will demonstrate that in a few moments so those are our containers of people's containers and now what are the things that make a container approve each one so here are a couple of things

that we covered in our research so first of all extra capabilities adding extra capabilities to a container will make it a perfect container and I have a couple of examples for you just before that capabilities are distinct permission broken out of route those permissions can be either disabled or enabled allowing the container to do something connected to the host in some way so we have here three examples of such capabilities for example cap sysadmin which allows the process or a container in our case to run administrative operations like mount named set namespace and so on we also have caps this module which is a capability that allows us or the container to load new kernel modules or uninstall existing

ones or cap CC boot is another example of capability that will transform the continuity of privat one which allows a container to modify the kernel so those are a couple of capabilities you should consider we're talking about which containers turning security controls to off might also transform a container to provision one so we have second for example second which in simple words and without insulting it as a Cisco filter so by turning it off we allow the container to run any Cisco's that is permitted to run and eventually again transforming it into a provision container we also have a farmer a farmer which is used as a mandatory access control which confines the container to

a set of limited resources and just to give you an example NMR more can block certain read or write memory read or write it can block certain Cisco's from running and so on farmer is a tool or security control that is part of the security in-depth that docker container relies on for them so again by turning it off we're creating a privilege container attaching devices to a container hard disks webcams using it and so on also create a privileged container and at last we have sensitive file and folders allowing a container access to sensitive files and folders might also might create a privileged container and we have here one very straightforward example which is the docker sock doctor

sock which actually allows direct interaction with the docker daemon insurance is route on the host so all of these are ways to create from its container not only the - - privilege plan all right so enough of definitions let's talk about how privilege containers looks like look like in the world so I'm starting here with a simple maybe a bit naive example of kubernetes system named memphis one of the containers in the system namespace and surprisingly or not containers running within the system namespace are privileged containers and what you see here is under they field cap off like effective capabilities we have the 3f which means that container has all capabilities and second zero which means second two often as I

mentioned before that means we have here a previous container and this is how permits container looks like in kubernetes we have another example here of a cube proxy under the kubernetes infrastructure queue proxy which responsible for inter communications between part in in kubernetes environment it has access to all devices on the host you see here the the SDA is here so actually any process or an attacker on that container have full access full lightness to the house now those are probably naive you might say that those very sensitive containers are running within the system namespace and I first but let me share with you another another example that I think resembles another type of beverage containers and

this example is of data or data talk is a monitoring service allows monitoring of cloud scale applications so it allows you to monitor servers application services databases and so on and when you deploy it I'm not sure you can see here that's maybe a bit small but to when you deploy it you actually mount the docker saw into that container obviously creating a privileged container because any process any attacker on that container will be able to talk to the doctor demon it runs is root on the host and we have a compromised host and I think the data dog is just an example of very sensitive services that are running as privileged in our peripheral tapes alright okay so

that we just defined what privileged containers are and we've also seen some examples of privileged containers out there in the world as we define them but here during our live escape we're going to use a different platform for our life escape demo and the platform is called docker playground websites now both websites are actually two websites one is playing with docker and the other ways play with kubernetes and this platform of those containers start out as privileged containers as we just defined but they are then by using various limit defenses such as a power law and SELinux those container environments are hardened and they are transformed from privileged ones into non privileged ones so what

our play with or can play with kubernetes so play with docker and play with two monitors are a wonderful initiative they they are they allow users to load and run Linux containers in a matter of seconds it gives the experience of having a free Alpine Linux virtual machine in browser where you can load and run docker containers and experience the docker platform firsthand without the hassle of first having to load and configure it on your machine so this is pretty much the play with docker and play with kubernetes websites so now twelve story a few months ago we noticed a vulnerability with the play with docker container and after we continued to research that we

discovered that we can load Linux kernel modules from the container to the underlying kernel and once we wrote an exploit for that we managed to even escape using this route all the way to the host and eventually run remotely run code on the host or generally speaking Linux kernel modules are a dynamically loaded code which runs in the context of the kernel so it is an extremely privileged code we notified the play madoka maintainer and the variability was fixed a few months ago so back to our story we managed to help our prisoner numbers six escaped once before from the village and returned safely to his home in London but now according to a story

number six is back to the village and we have to find another vulnerability another way to exploit the docker playground again because our initial vulnerability has been fixed so what is knowing too much mean in our story well we know that prisoner number six along with the other prisoners in the village all know too much this makes them a threat to the organization behind the village and therefore because of their specific knowledge they can never be released back to society now what does knowing too much mean in the context of containers so if our attacker process manages to escape from the container and take all over the underlying host and gain privileges on the underlying host

that means the number of things for example if our attacker has access to the host our attacker also has access to the docker daemon which is on the host and that means that our attacker has access to secrets and credentials injected or loaded in all containers running on that host for example by using the docker inspect our attacker can inspect the inner definitions of all of the containers running on the host and extract secrets and credentials if our attacker has privileged access to the host it can also run run code on each and every one of those containers for example by using docker x ik and docker attached and I won't important thing that our attacker

now achieved is our attacker can stop and even completely remove containers from the host any container our attacker may want to and our attacker can also create new container or replace containers that were deleted by using docker create and docker run well this is another interesting point by having access privileged access to the host our attacker will have access to the docker hub credentials if the host is a development machine now the docker hub credentials are used to send or to push images to the docker hub now with these credentials our attacker can infect all the docker images or other repositories all the organization perhaps even demand ransom afterwards and last but not least the network that's probably an inner

network that our house is connected to this may be for example a container orchestration Network and by leveraging lateral movement movement within this network our attacker can move from machine to machine and perhaps eventually take over the entire organizational network a natural clip in onyx short clip we're going to see number 6 again who decides to run for the villages annual elections

[Music]

okay so now we've gotten to the stage where we speak about our live escape and here we detail step by step what we intend to do during the live escape so we already discussed the point where we managed to exploit the Linux kernel module injection issue but this is now fixed and we want to show that I've escaped so we had to find another exploit after some research on the play with docker container or platform we discovered that we can exploit the system console of that platform now the system console is a device is a Linux device which receives all kernel messages and also allows for users to log in in single user mode this feature was added to the

kernel back in 1991 so it's quite an old feature maybe this is lights it tends to be forgotten so what what we're going to do is if we have a valid user in the system on the host then that that user may use that feature which enables users to log in and type in their username and password and after that the system console would open a shell on the host and allow that user to run code on the host so this is our target and but before that we have another step because we need to add a valid user in the host city C password file so we have two steps to perform here one would be to add a new user or

to change an existing user on the whole city C password file so we could load login s and the second step would be to exploit the system console or to inject keystrokes into the system console and we do that by using a small utility we found online which is called TTY echo and we use it during the live escape well here is what starts quite small but on top you can see that these are all the messages from the kernel this is the input to the system console on my machine and below you can see a user trying to log in so choose the one and he's typing in he's using any password and after that the system console opens

a shell on the house to him so this is what we're going to exploit during the live escape and now to the live escape

okay we'll try to use then I'll try to use this I try to shout so we have here we have two machines on the right hand side is an open session with the play with docker container we are locked inside a container here on the right side and on the left side is our attack machine which we will later on run a listening netcat on which will accept the incoming reverse shell connection from the host so we start with a brief reconnaissance phase where we try to chart the borders of the container that we are located in and see if it is privileged or not so we're going to start with you name and I'm going to do

and here you name shows us the version of the kernel for 4:01 54 generic and the underlying host runtime libraries which is running Ubuntu but we may have other runtime libraries inside the container we're going to check that by looking into ATC release okay okay it is release now this tells us that inside the container we are we are running with runtime libraries from alpine Linux so we're going to first use alpine annexes package manager to ad lib cap which is a library that will help us decode the capabilities that Lavi was talking about earlier and we're also going to load our TTY echo and other helpful apps from our tech machine and antar it good next

we're going to look into proc cell status here we can see the status of seccomp that away was referring to earlier second zero means the second is off this is very good for us as attackers and the capability is inside the container which is this long hex number and we can we can use catholic safety code to decode them let's go over them really quickly and see what we have inside the container so we have the following capability we have captain air and net admin which allows us to the container to administer the network captious module which we've seen before which allows us to load and unload kernel system modules P trace kept CSP trace allows us to debug other processes

catch this admin is the catch-all capability that we was talking about that allows the container to perform administrative operations and catch this boot is a capability that allows the container to completely replace the underlying kernel so former an attacker perspective it seems like we are pretty much privileged here in this container our next step would be to look into prague command line command line so this file contains the parameters that are transferred to the kernel when the kernel is first executed so we see the first parameter is boot image that that's the image the kernel is running from and the second parameter is the root device that the kernel and also the host is located on as you can

see it is designated as a UUID and we're going to use find the face somehow find the face to find out the underlying device that the host is located on so we see the device is dev FDA 1 let's see if we have access from inside the container to their SD a 1 so it seems like they're inside the container we have access to diversity N 1 so as I said earlier our first target is to add a new user or somehow change the hosts etc' password file now to do that we're going to need a readwrite access to the underlying host file system and if we have access to del Este one we can probably mount it

inside the container so we're going to attempt to mount it with mount diversity a1 inside the containers mmnt folder unfortunately it seems like we cannot do that because probably this route is blocked by our containers defenses ok but that's not the end of the story here because during our research of the play with docker platform we discovered that there is a way to mount the SDA one inside the container but we had to change the mount application so that it will work inside our container we have this mount application here inside the container and we loaded that during the time will order the TTY echo utility but unfortunately at this time due to legal constraints we cannot disclose exactly

how we do that but we do have two other methods that we are going to discuss right after the live escape which achieved just the same goal and that means readwrite access to the underlying hosts file system so right now we're going to use our own mount and it's a success so let's take a look at the hosts file system here the root filesystem of the host and compare it with the root filesystem inside the container so you can see that there is a marked difference between the two and also please note that the first file on the hospital system is a dash so it seems like we now have readwrite access to the house filesystem and we

can add our user into the host et Cie password file so we're going to do that we're going to add user name by the name of number 6 we will make the password because we don't need passwords here and we're going to use UID 0 and GID 0 and user is root working from slash root and running being bash and we just append this into the host EDC password good so this looks like our first target is achieved now let's go back and take a closer look at pro command line so we've seen boot image and root but here we have a very interesting argument and this argument actually tells the kernel that the system console is to be

connected to the tty s0 device to a serial tty 0 which is actually com1 and that means that if we can inject keystrokes into com1 or tty r0 serial tty 0 then those keystrokes will eventually end up at the system console let's see if we have access from inside the container to TT y a0 and again we have access to TT y 0 so the next thing we're going to do is we are going to inject keystrokes through TT y is 0 all the way to the system console by using our TTY echo utility we are going to inject to TTY 0 what do we inject here anybody has an idea so we saw earlier that the system

console is waiting for a user to type in their username and we just inserted a new user into the hole CTC password file so we just need to do that and by pressing enter here since we don't have a password the system console is well it's supposed to at least open us a new shell on the host and give us access to that shell so let's press enter here now on the other side on our attack machine it's time to run a listening netcat on port 1990 our attack machine is listening for an incoming reverse shell netcat that was supposed to be running on the host now again on the host we are now running full fledged in a shell so

we can do whatever we want we can run being busybox netcat and we want the netcat to do a reverse shell to our attack machine port 1990 and run being SH now if everything works ok I hope it does then when I press enter here we should be seeing a connection on the top left inside of the screen on our attack machine let's try this okay

because we just helped number six escape from the village and return safely back to his home but let's see what what we have here let's first take a look at the file system root so these are all we've seen them already and you remember the - at the beginning we can also look at our ID and if you recall we defined our user as your ID 0 and G ID 0 well if we can also see who are we inside the host so here you can see that we are logged in as user number 6 and we are connected through TTY R 0 the serial TTY that we meant to connect through so this all

seems very good for us as attackers but let me show you another thing you see we can also if we have access to the hosts as we said earlier we have complete control over the docker daemon so we can for example run docker PS and see all the docker containers running on the host and as you can see there are multiple containers we are only one of those there are multiple containers running on the host so those may be other people using play with docker or a workshop or trainings or whatever so there are well a lot of them we are only one of those ok so that was a live escape and now as

I promised earlier we need to discuss what can we do when mount is not allowed because mount is a very privileged command and it is probably the first command that defense teams would if if they get the chance so we had to find other solutions if we can't mount it so here we show a couple of other methods of paths that we have and the first one is using D D D D stands for data definition and it allows a process running in Linux of course to read and write blocks of information directly from a Linux device so if for example we know the pattern that we are looking for so if we want to change something in the

e.t.c password file then we know probably looking for we can look for a pattern such as root column X column 0 column 0 etcetera then we can read blocks from the device until we find that pattern and then we change that block and we write it back into the device and by doing that we gain read/write access into the device another interesting solution and this is my favorite it's called the bag FS very shortly the bios s is a linux utility which is an interactive filesystem debugger for ext3 2 and for filesystem and it also allows to read and write state and information directly from the device and also change it now we had we

had a lovely demo here but unfortunately our time is up and we have to finish our finish up so just couple last remarks because they are pushing us okay so we learned a lot I know it's quite quite difficult to absorb all this information but do a couple of things from this session first one I hope you learn no presume in number six I know I did second thing is privilege container our thing so if you are a red teamer in your next engagement try to look down this container trips to enterprise if you're a blue team ER it's something to look for you should have the kind of visibility to what energy running and

they should have control over who can deploy such primitives and I know that we promised to show you how this how the serious ends so I'll play but feel free to move on I know where I know where we should in any session and we'll be here for a few days

[Applause]