BSidesWLG 2017 - Matthew Daley - Pandas and Rabbits: Xen Meets American Fuzzy Lop

cool so assume everyone's already sweet so um I'm gonna be talking about pandas and rabbits America's in meets merican fuzzy lot so my name's Matthew daily or as my friends like to call me Matty petty you can ask them why you can see some of my previous work on these websites here but out-of-date I need to update them if anything in this talk interest to you and check it out because I've found lots more vulnerabilities and virtual machines and you can see a bit more detail through there and this talk is completely unprotected to pull me off stage at the OWASP Oakland conference because I was going over time may still may be in this room so I have to watch

my deck okay so the first thing is first thing my friend one of his friends said when I started this introduces talk is why the animals why pandas and rabbits so panda is in so the Xin is the mascot of the Panda is the mascot of Stan apparently has this name but no one seems to know what it actually is originally made so Xin is a hypervisor it's a fancy word for virtual machine manager which is a fancy way of signing saying computers running other computers and there's two different types of hypervisor type 1 and type 2 now just be annoying I'm going to start with type 2 so type 2 is one that runs on top of an

existing operating system so this is things like VirtualBox parallels you can see there in orange you'll be running VirtualBox no run is an F in itself on a normal operating system type one is like a Zen or hyper-v or VMware and that's where the hypervisor is like an operating system in itself it runs other oasis on top just how like another OS would run applications so Zen is one of these now just a bit of a head trouble putting some of these things in the right order so you'll have to be with me here as I sort of jump between topics a bit but if you I'll come back to them and it will make

sense so now I'm gonna talk about CPU protection rings privilege shrinks so CPUs have a way of dividing up privileged just like how you have root and you know of standard users in a UNIX environment these are called protection rings so there's four rings zero to three because we're computer scientists and the intent is that the most well the intent was that the most privileged which is ring zero would run an operating system and 1 & 2 would be yeah 1 & 2 would be the drivers and 3 which is the least privileged would run the apps themselves however in the real world we only use rings 0 3 0 for the OS kernel and 3 4 own for user apps by the

way this is what the until architecture reference manual looks like they used to seen these up for free this is my favorite page turner many many heavy Knights and beards been reading this one good old vol 3 a this is part 2 describes protection rings but like half of it is actually and not in use anymore so this just shows the complexity of the x86 in x86 64 architecture as its evolved over time so back to the type 1 hypervisor how would it integrate with these privileged rings so we need the OS the apps the guest OS and the hypervisor all need to be running at different privileges but the Oasis expect to be able to use all the brings from frieza's

zeros so where would the hypervisor go so this is where hardware virtualization comes in this is called well Intel's version of hardware virtualization extensions VTX and of course AMD is called so what this introduces is the idea of a VM mix root and a V mix non-root surprisingly so this means that we can have the unchanged operating systems to continue to use ring 3 + 0 + 1 - if they really want and the hypervisor can run and bring whatever in the VM X root now this is ok but what this requires Hardware extensions to work so they're pretty common nowadays but they haven't always been and in addition is it's sometimes it's not the

most efficient way to virtualize a system so are there is there a more efficient way of virtualizing a system an operating system so looking back at the type 1 hypervisor stack again here's our operating systems normally they don't know that they're running on a hypervisor right unchanged those Windows Linux FreeBSD whatever what would happen if we would tell them they were running on a hypervisor so this is called enlightenment and Zen speech this is a very nice pun so we tell the operating systems that we're switching out the hardware underneath them for in the hypervisor so they don't freak out when they see that they're running on something unusual so that's what this means is that with our enlightened

operating systems they're able to run and bring three the same ring as the applications are actually running and the hypervisor can use ring zero and none of this requires any Hardware extensions because we're still using the same old privileged ring setup we've just used we've just pushed the operating system from ring zero to rank three now this is called para virtualization and I got sick of writing this so you'll probably see me writing it as PV so para virtualization is fast it's lightweight but as you can see the downside is you need an enlightened operating system so that means things like linux freebsd open source things i'll call the windows now they actually windows did actually write a porter

version of windows XP this is when Zenith started big speed was a thing pull it a visual version of Windows XP to run on Zen they never released it there was just a research project with the collaboration with this input as then team so who actually uses Xion well you might have heard of these guys so these guys use it for ec2 and these are slightly modified is in kernel and these guys Rex press rec space cloud similar sort of thing and chances are if you have a unix linux unix VPS provider chances are they can be using either k vm or it's in people that slightly it's about half and half use between the two

I know mine uses Zin for example okay so now the REA but that was about the Panda and how about the record the bunny so this is the bunny you believe it or not this is an American fuzzy lob it's a breed of rabbit so I just want to quickly talk to you about what fuzzing or what American the fuzzy lock does which is fuzzing so let's say we have some system that accepts potentially untrusted input fuzzing is the process of trying to break that system by providing it with malformed input so that malformed input might be you know Alicia's in nature if we're trying to actually break it or it could just be the wild world of crazy PDFs and JPEGs

and all the way it files out there so the data can be mutated versions of things that would normally expect or it could be things can be data generated from scratch from a specification so you might make a JPEG image by taking you might fuzz a JPEG reader by taking a JPEG image you taste a little bit so flips and Burt's swats and bits around drop some stuff and send that to the to the JPEG loader alternatively you might make a JPEG from scratch like generatively so anyway what we hope is that the system under the under test takes that input and completely freaks out and in the world of software what we ideally want to see is a crash of some

sort this would indicate that the software has done something naughty wolf memory and memory corruption vulnerability and these are frequently well usually exploitable so this is the way that we can use a fuzzy not just for reliability testing but to actually find security bugs vulnerabilities in software so fuzzer can generate a huge amount of random inputs from a small sample of test cases represented by these god-awful drawings so each of these represents a different randomized test case now this is all well and good but as you know software isn't quite that simple so this randomness can sometimes hurt us so imagine a system that checks certain predicates on tests and puts before it actually goes ahead and does any

processing on it so here we have the thing processor Enterprise Edition whatever and basically this is going to check you know some two key attributes of the test inputs it gets the color and that's height so if they don't match it's going to ignore it and if they do match then it's going to continue processing now chances are the cheeks are going to be implemented fine but the actual processing is weird the vulnerabilities may likely may likely to be so we want to make Kraft hist inputs that will actually exercise the system under test efficiently so it will actually test all all parts of the code not just these initial checks so for example those first two checks our first

check is going to rule out you know these of these ones flight test cases and finally the second chick is gonna rule out even more test cases so we spend a lot of time generating and running all these test cases on our target software that without only without actually doing anything we only got one very few small my small amount of test cases will actually do anything useful so this is where we're AFL stiffs and AFL is what's called a guided fuzzer so instead of just purely randomly generating test cases and throwing them at the the system under test well it does that to start with but what it does is it looks at the code path execution

the execution code path taken in response to each test input and so what it can do is over time it can watch to see these Co paths and figure out which of the inputs are actually exercising new test cases and which ones aren't and then after it's built up a sort of internal graph of what the program looks like as a general idea of which parts of the intestine put interesting mm which parts aren't so what we'll end up with is actually generating test inputs after a small amount of time that actually makes sense to be tested on the system under test so to give you an example I was writing a buzzer for a job I was

doing and it were head to fuzz network traffic now we used a dumb fuzzy first so that just generates random near word packets and throws him at the minute the device under test and it was all good so I lift up running over the weekend to come back and saw that it had decided to fuzz IP version 5 for about two days and you can see that's gonna do absolutely nothing right so I want to be able to reuse guided fuzz that quickly sees try off your vision to nothing does something try ot vision t3 same thing happened for something different happened okay let's try poking for a bit more this is guided fuzzing once we do

as a test and find a test input that does some different code execution path we hold onto that and then we we then start mutating from that one and so we get levels and levels of mutation this is really efficient and it's it's a one of the more practical breakthroughs to come through in the world of fuzzing there isn't from like that pure academic side so FL will run all of these test cases on the target and C's see which ones fail so for each tip failing test case we can get the bug data and so from there we can do the sometimes the harder part which is actually to find out why this test input crash their program and

usually during that process you implicitly also can sort of figure out well how would I usually start exploiting this how can I do tune this test important to something that does crash as the program into something that actually makes it vulnerable and does something that I wanted to do so AFL generates this fast inputs into the target gets execution bitmap which is a fancy way of to say in the code path that the program took an doubtful a bunch of results a crashing test cases so this is all cool but how would we go integrating this with AF how old we integrate AFL words in so it's easy to see how you would integrate AFL with a

file types on a file type handler right so a PDF reader you would just put your test your test case is the file in that case your GPA or your PDF or whatever and so for network protocols it's similar people have done work to modify a a fail a little bit so that instead of outputting its test cases to file it outputs them to in it wherever a network socket and then it can see the result of the program the server after it reads that socket by the hypervisor so hypervisor is a pretty low level thing and it's a brand virtual machine so like it's not potentially and I have done it you could fuzz like the file format that

it uses to you know load vm's like OVA files for example and VirtualBox and whatever else Oracle VM and anyway apart from the the virtual machine file format itself what else is there that we could fuzz so let's look at the type 1 hypervisor stack again and remember these remember our guest OS is our enlightened so they know they're running on a hypervisor they know they're running and ring 3 which isn't normal so on native hardware that the operating system can just tell the hardware the operating system we just tell the hardware to do things right to interact with hardware devices to allocate memory to do whatever but will para virtualization and I wrote the

whole thing for some reason Zen is in the middle right and so operating systems can't just have direct access to the hardware because otherwise I'd be able to break out of their virtual machine confines and the whole point of design is that it's there's hypervisor to prevent or afford any systems from being able to do that so when an operating system needs a thing and these are mechanisms to be able to request that from Xion and so then can then actually do the interaction with hardware if it likes what it sees the the request is okay and we will do it does its thing this is it that it ends in pretends that I did it to the

operating system now what would hit so this is useful because we can then this provides the necessary isolation that guarantees that we need so if an operating system tries to do something dodgy or just wrong because the mistake right the scene you can say no and that no can vary all the way from an error code to killing the entire operating system but either way he's in keeps running the hardware is still happy and the other operating systems are still busy sleeping or doing whatever they want so you can't Theory break out of your VPS and 2zn by through these Hydra calls or can you so in a normal operating system environment sits on the left hand side well in both both

environments applications talk to the operating system they're running on using system calls so the system calls are things like read write open close socket connect those sort of things if you've done programming using the Odyssey for example so these are the calls there an application can't do themselves it needs to talk to the wider world somehow so it uses operating system to do that and it uses system calls equivalent in a hypervisor situation as high for calls because that sounds even better than system calls so high for calls allow an operating system to although one of those enlightened operating systems to communicate this in so this is the list of hyper calls that we have there's quite a large attack

surface you can see there so what we could perhaps do is take these hyper calls as the input that we can fast so once again the cute little shapes represent a fuzz inputs so when if our malicious operating systems can say that it needs to do something with a first hyper call and we give that to zine and see what happens so Zen might get confused or it might get so confused that it kills itself and the hardware is unhealthy or it might who knows what it's going to do but that's what we want we want to see the system under test do something unusual so we can see if we can then exploit that how can we get a FL to do

this for us so fuzzing the application is pretty easy so I keep jumping between native and visualized worlds you have to be able to me here but so fuzzing an F is easy in the native world right you ran your fuzzer on the same machine as the application itself if the app dies and the fuzzer can just ask the operating system to restart it and that's cool your fuzzing an operating system well of course you probably want to use a virtual machine I've like a hypervisor and run your fuzzer and a different operating system than the one you're actually fuzzing so that way if that your operating system your testing crashes the fuzz I can ask through the

operate its operating system and through the hypervisor to launch that operating system and start again but if you're fuzzing a hypervisor well when the hypervisor crashes everything crashes and when I say crash I mean everything crashes like the Machine actually you have to pull the power or physically restart it to in the bug road you have to actually physically restart it to get it going again now this isn't conducive to fuzzing because I don't want to sit there playing with it all day so what we could do is we could split the fuzzer on two separate physical machine just like good old days before virtual machines and put the target host that puts us in the

retargeting and a different physical host so AFL instead of launching a program that wants to test it launches a stub little temporary stub that stub talks to a persistent connector which talks to a stub operating system on the test environment which then does the haifa call so remember fuzzing runs in the loop tight loop to generate results and we want this to be as fast as possible so in the world of AFL for example if it if you get less than 100 you know test runs per second it considers that slow and if you get less than 20 it considers asleep so this first initial team got 12 seconds per execution and that's not a

typo that's seconds per so that 0.083 executions per second remember a FL thinks 20 per second just way too slow so yeah so no so what we do is we could modify FL so that instead of having that whole stub and persistent connector malarky it just talks straight to the stub or any systems like toast sockets say here's some test data run this hyper call and it does it and they can stop any concern back at the code path through their same socket 10 executions per second ok it's a little light that's coming through this tunnel what if we just what if we just play AFL in the damn machine that we're trying to test

ourselves and say propose just have a monitor that can they can handle restarting machine and collecting logs because when AFL tells the stub to do a hyper call which crashes then of course that entire machine is going to be gone including AFL with it so the monitor is able to then restart the whole box and also little checkpoints so that you know when it doesn't fall over we haven't lost a whole day's worth of work I'm so hundred execution for a second this is good we can speed it up a little bit more and it's a little hard to see the difference here there's no error between AFL and Xion and this one and it isn't

this one so what that represents is we instead of the stub being told what the hyper callers to call and then it calling it throws in AFL can sort of pre-pre loads in with the hyper call data and then tell just tell the stub to run it so it's works like this AFL tells what's up with modified fo and with modified seen in this case because to do this we tell AFL here's some test data for you next hyper call and it takes it and then we tell the stub or a system to actually execute the hyper calls it's kind of like a latent pending hyper call it does it this way as in gets its

knickers in a twist it's the stub operating system will be the one that gets killed and not the operating system running AFL which is important because the context is important here of the machine with a the operating system of AFL running in it tried to do the hide call itself that would die so by putting in a stub or operating system and still doing this weird split we get the benefits of speed but also the fact that there are operating system running AFL doesn't crash so I mentioned before that the monitor on the separate physical host needs to actually restart the target host wins in crashes so we have these handy-dandy things these Belkin power switching off

in Wi-Fi things and I'm lucky enough to have a power supply which hasn't burnt out after like a thousand power cycles so far it would be it would need restarting probably once every four minutes I think it depends on what hyper call your fuzzing and what hard you're doing it and so this is what my puzzling setup looks like and my din so you've got two machines here the one on the left is the actual the test machine when it's been fast in the right is the monitor and this is my networking stick ache a heap of noodle and this is an important box because winds in crashes it's only way of telling you is

via a serial console because it can't show anything on VGA because that's used by the one in the operating system from sample and it can't use Ethan it because that's controlled it's complicated but can't so it has to use serial so originally I just used those serial we over to USB converters biased against so many boxes that need a serial output conversion I brought this from one of those god-awful you know import sites and it's it seems to do the job so it's serial to Ethernet adapter there's some graphs here this is what the graph AFL looks like when it's running I don't expect you to give a crap about what these are we need to explain what they

are but as one of my friends likes to quote to me nobody knows what it means but it's provocative it gets people going so you can see the lines go down lines going straight cool so eventually hopefully Zen will crash and this is what this looks like so that's what is English looks like on the serial console and you can see there this is manual reset required because I wanted to stop in this case I don't want it to automatically restart but the bug purposes and the the observe enough for you will have noticed there's a giant cat sitting on the networking table um gee okay it is there to inform me if I get like email those signals or whatsapp

source snaps and it uses a color-coded is to tell me and purple means Deena's crashed so this is something I like coming home to and seeing gee okay it also holds the proud privilege of owning its own Twitter account and you know I even I don't have my own Twitter and the reason it has one is because white Skopje okay just as a side thing is because it used to look at geonets Twitter feed and tell me the intensities of earthquakes near earthquakes using the colors of it sighs unfortunately GeoNet changed the feed their twitter layup recently so i need to go and fix it but yeah one of my internet of things internet kind of things here's a Twitter

account and I don't so anyway we run the puzzle like this for bit and we get a bunch of test inputs test results a crash let's look at some of those tests crashes so the first one is coming to do a bad Numa node handling so numerous downs for non-uniform memory access and I'm going really fast now sorry so non-uniform memory access is ability for a pro computer to have more than one kind of memory and and then allow the applications or the operating systems running on it to actually specify which memory is wants to use for an allocation so in the world of ZN an operating system can say tells ian i need memory

and a certain Numa notified by into J IDs so then we'll go looking at internal structures because hold state for each of these new Monod and some information now the problem is is that there's only a limited amount of for new monoids and you can see where this is going zyne doesn't actually check to see if the Numa node you gave is actually in range so it'll fall way off the end and it'll come back to you with like some really weird situation so what will either happen is the operating system will crash or Z in itself will crash now this is interesting this is good we can make a little Python POC that will just

do the crashing no exploitation but it'll do the crash can send that to the same security team and you can see they put all their security visors on this page and I'm proud to be the reason that they started this page back three years ago and in this is the advisory yeah cool number two is boring just trust me number three is also boring number four is slightly more interesting so zine allows operating systems to share memory between each other so if the operating system wants to share memory between two another one it just asks it and Xin does the magic and we're heavy there's a problem where there's certain types so you specify what virtual machine you

want to see memory worth buying as your ID but the student into dry J's which is special they used for like bookkeeping purposes you know they're like the the weird numbers off the top of the end which scientist racism numbers they're there to say you know operating system or all operating systems all that sort of thing so if we tell it's in to share memory to one of these it tries to do it so da Mayo is one of those weird numbers for us talking about tries to do it and of course a crash it so that's also cool we rode a Python POC for that we can get the Advisory thank you okay number five

is boring number six is boring number seven is too hard so okay going real fast okay so we've seen how we can get a test case and we can get frozen we can find his cases and they Croatian that's cool but how do we actually exploit those now so let's take the first one that you saw there I've caught a bad node that's not because I'm some cool marketing person that's just because that's the first seven characters that I typed in my get repository so this is CV by our exes a two three one so let's just quickly go over the vulnerability so Xin has two heaps in heaped on heap so domains can request and you know request and release

memory into the domain heap and there's in heap is for all the Zen heap has full memory that can't be you know accounted for by a single VM and the Don heap is for ones that can be account memory that can be accounted to a single VM so here's that list of hyper calls again this is the one that was causing the crashes we're getting seen these crisis with memory op this is the format that we part the structure that we use to pass in the data the parameters to this hyper call and this is the attribute where we actually pass in the new monoid ID and these are the macros okay functions that we used to write there to

modify the numeron old ID into the waves in once if we look here we see there's a maximum of hex or Hix if which is 255 decimal so we can tell Xin to request an allocation from an 8-bit wide Numa node so that between 0 and 255 but of course there might not be that many nodes okay this is the start off that this is the start of the actual they either call Handler and Xin so this code does some initial validation of the hyper call parameters but nothing to do with no monoids it's bit more generic and so it calls a function and as that function and this function calls another function and that's the function and this calls

another function and this is that function now the start of this function is where the important part is so allocate pages we're in the right sort of place here so we can see here this is where it gets the node from the wave shoved into the memory flags one of the attributes of the hyper call that the operating system is Givens in and we can see that a chick's to see if node equals Newman or node which is one of those sort of it's a special signal value that's is we don't actually care what Numa note it goes on but that's not a limit check it's just an equality check now we do see there's a limit sort

of check here assert node a smaller than max num note now this sounds pretty like you know good right we're taking that the that the node that the user has just find the hyper call is less than the maximum number of nodes we know exists there's a problem here developers will probably already see it in the bug mode so there's two modes that you can bot build it's the in just like he can build Linux and lots of other things the bug mode release mode debug mode is what developers will use release mode is what actually gets released to the world or the public so in the bug mode a failing assertion so an assertion that tries to

check something that doesn't isn't true and get this case like you know the number of numa numa node given is bigger than the number of new monoids it would stop the program supports to stop scene with a fatal exception and crap would scroll up the serial console and cat starts going nuts in release mode assert there's absolutely nothing so assertions aren't meant to be used for actual error checking that IRA's might conceivably happen at runtime in the real world they only mean to be used for developer sanity checking right things that if if this condition is wrong well all hope is lost you just have to kill the program right if something has IFFT up so badly in a case

that isn't one of those things you know guest operating system can give a node which is out of range we need to be able to handle that properly so that's their assertion there isn't actually doing anything in release mode the bug mode it does but in release mode so that was interesting in the bug mode which I'm where I do normally do my fuzzing I noticed you know it did a nice simple assertion feller I was like oh that's boring but then I noticed you know in release mode well I keep going at the actual full memory crash and that's even more interesting okay so this is the continuing on down that functions and we've tried to do

that validation but we haven't on the Numa node so now let's see what happens so it uses that notice the node variable that we've given it and a bunch of locations so it uses it to see if there's any memory available in that new manner which is fired to see if there's enough memory on that Numa node and then remove the memory from the heap link list if there's a second so the important thing is and all of these places we could coerce in to crash right by giving it a new node of 253 or something like that and it's probably going to like you know look into a sum off this end of the array and it's going

to crash in some exciting non exciting way but exploitation is just a controlled crash right a crash in a memory corruption situation usually means that you if you have the skills you can actually at a time you can actually turn that crash into something that works for the attackers benefit so exploitation so looking done that function a bit further we notice that that actually adds it can actually not just take memory from one of these Numa nodes that also give it back so why would it do this so this is because then uses something called Buddy allocation which is a very friendly term so it stores free memory and linked lists and it has a linked list per order and when

I say order I just mean it's a size right so order 0 is 4 K 1 is 8 and it doubles each time you go up in the order and those of you might notice that 4k is like a page size so you can only allocate a page or one or more pages so you know if we are seen to allocate space for example 16 K it might look at the 16 K door to list see there's a memory there and we're all having it cool the second case is that it looks in that list and that's empty okay now it's not actually ran out of memory it's just run out of memory and that linked list

for exact chunks of size 16 K what it can do is let go up the order list until it finds a non-empty list take that one's memory split it in half put that one back to put the unused half into the order to list and and now we have a properly sized allocation and we're all happy so buddy allocation means you take no it's buddy but you ask your buddy it for a favor I guess and then splits memory in half and then you stance did so this is where you have to hold on to your butt's a bit this is per Numa node okay so there's one of these per one of these there's this there's

linked list for each order for each Numa node ID so there's zero one two three and then there's all of those ones you know sixty-three so if we specify a node ID that doesn't exist once we get past all the weird crap we eventually is in starts looking at memory that we actually control I won't go over why but it does so effectively we can actually create our own fake memory list order structure thing and gets into work on that instead of the ones that it's actually generated itself so here's our test set up for example we've got an empty zero or two zero less than an empty order one list and we ask it to allocate an order zero it sees

that it's empty it looks at number one so it does the whole it rewrites the list takes out the memory splits and a half puts it does the things right like I just said before it does what I said okay so it's done that in the process of doing this it's done a bunch of writes to memory to update all these lists and stuff like that so unfortunately for us these writes here for example aren't interesting to us because they're to memory we already control and these writes to memory aren't interesting to us because of reasons those reasons are because we can't get it to actually write to out of bound me out of range

memory we can great we can tell zena's writing to memory you don't control in this case but it's not writing to all memory just a very small part of it right yeah that sums it up so what we can do instead is we can do a little trick we ask it to do the same thing we we've set up a fake memory list we asked in to allocate an order zero allocation looks in the empty list Co starts looking in the second list but then behind its back we secretly put in we start putting what sort of guest like that I guess we put in a node and the orders here a node memory chunk in the

order 0 list and we can normally we can do this because normally this in lot does locking writes ok you can't concurrently sin can't concurrently modify a memory list with by two three is like two three trying to modify a list it won't happen cuz it's got locking but unfortunately for us the the weird nodes that were you control we can just bypass that locking because we don't have to take the lock that sin does we just do it anyway right like a bad programmer this is called a race condition except that's not really because of Zeena's we've broken Zin so much that it's not even really a race condition that's just not not being

blocked at all so anyway back to this so it's looking at order one when we've particularly put something to order zero it takes the one the order one thing splits it in half and now remember previously it put it on it's a it puts the split the unused split shank on to the tail of the list that was actually you know mean to be on so in this case is putting it on to the end of the new node that we added an order 0 so got some different kinds of rights here so the most important one is this one here now this one actually is the value that it writes as is the what it's writing is

the value of the order 0 linked lists heads next pointer I think and so we implicitly control the value that it's written because it's the address of that node that it split into half and we also control the address that has to write that value into because that's the address of the fake first memory chunk and the order 0 list so I've got a variable there and a value that settings are basically and all of this miss we can write for chosen bytes anywhere in memory and this is what's called a right what we're print right what we are primitive ok so all its exploitation we now we need to know what do we write and

we had rewrite it so I'm going to introduce the page tables pretty damn quickly so in a normal operating system you have over applications running on operating system running in hardware and an operating system uses page tables and also to see for rate the memory spaces that each application just can see so on the Left we have virtual memory which is memory how an application will actually see it and hardware memory which is hot the hell the memory actually looks in Hardware and you probably have more than 64 kilobytes these days but that's all they gave us I can fit in so what will happen is if a applications tries to read or write from virtual me

you know uses virtual memory spaces addresses and assembly and stuff like that so well it'll say I want to read it right to this part of memory the CPU chorus looks at the page tables and says is all those maps to this place and hardware memory so what this means is that although there's a whole bunch of applications all fragmented all throughout hardware memory each application can get a kind of you know aruba tree view of that memory and only its memory the operating system can make the applications virtual address space look however there wants it to look like but it can also control enforce isolation by ensuring the applications can't see other applications memory in

the world of Xen pates tables still exist but as you can see they don't get used to control applications running on OS the scene users into control operating systems now the operating systems can use those hyper calls to modify the page tables because they might still need to modify the page tables because they still need to be able to control their own applications memory isolation but the thing is an operating system can't want to fly these patients directly because that would violate the key invariant that only then can control what operating system can see what memory but what if we just do the patch ourself right then we're able to use that patch page table to look at

our physical memory ourselves now I'm gonna have to go real fast here but basically that means that we can create a little portal into virtual to hardware memory so we can read or write for two we can rewrite any amount of bytes anywhere in memory but stability and usefulness important so you know cating give null over my friends operating system sounds fun but it's not exactly you know useful so what if we use one of those page table over writes to actually get access to the page table itself because the page table is actually in memory just like everything else so we use the page table over write to actually get access readwrite access to

the page table and this is funny because to get readwrite access to a page table only takes flipping one bit of the second bit so in fact to break out of the switch or machine you'll see everything you see here only takes flipping one bit right the second bit of one page table entry so this is what the code looks like for initializing their portal so we allocate some memory for that portal we do some crap and then we create exploit page tables yep so we make some stuff and then we fill the page tables we getting a bunch of stuff here sorry but basically it comes down to there's this there's two threads one thread

which is constantly adding and removing that page from the order 0 list and another one which is creating the fake page table the fake memory list and trying to trigger the Vaughn ability that's what triggering it looks like yeah it's just dead right that's what it comes down to at the end of it calling the hyper call to do the naughty thing so we we have to is I'm gonna skip all that because though it's boring so what we have is two threads one which is constantly twiddling as I call it adding and removing pages and one which is setting up those things telling them to look at them and checking to see if things overridden eventually hopefully

it does it takes between five and thirty three minutes pinning on the load of the other there is a machine okay so we're happy so we've got access to the page tables now what's a little F code execution we have full page table exists so that interrupts so CPU Santa wielding themselves right there's external events they happen and they need to be able to respond to them so this might be user input for example will knit web packet handled neatly packet data so what happens when its CPU gets an external interrupt it looks at something called the IDT interrupt descriptor table and this is basically a list of addresses that it can go to to see what it should

do we've given that situation now we can met there IDT just like anything else we can map there IDT and Pech it so now when the CPU looks at a the IDT to see what we actually jump to to handle an event it sees this thing and it doesn't think anything off that right um so then it starts executing our code stage one and it's doing this and bring zero so we've broken out of ring three and guest operating system to ring zero this is how it works okay the stage zero does some stage one does some cleanup and then it returns the system call handler so remember system calls were between Epps there's os's and OSS is in so if we so

normally operate system calls are used for hyper calls from our sisters in and they're also used from applications to your operating systems this in just bounces it back to the operating system so what happens if we touch the system call handler okay so and it's gonna quickly just jump here and then speak a VM it's a domain and domain user normal VMs domain 0 is sounds cool it's the privileged initial VM and I just a nice typo there the main 0 is a domain that can control other domains in and add some other hardware so this is the domain that is actually the control Virtual Machine basically is that the main you launch domains from it's the one that your your

VPS provider will actually all their scripts and all the launching and closings so from so I went into this one so when we patch the system call handler not only do we see all every system call we see system calls from Don 0 and specifically we see system calls from user mode applications running in Don 0 so we catch it and we call the stage 2 so stage 2 does things so the CPU is M there's an interesting thing where we can't just overwrite stage 2 the system called Hana directly because another CPU might actually be running it so that really confuses the crap out of their CPU so what we have to do is modify it

in place modifier atomically which is a whole nother kettle of fish so in domain 0 some user space program will be trying to execute some heavy thing so in order to assist call and that gets redirected all the way to stage 2 right so this is stage 2 so the first thing we need to do is we need to check that the system called with hijacked actually belongs to Tom zero and we need to check that it's actually belonging to a user space application and Don zero so this is what these first two chicks are for then we patch the victim apps page tables and then we jump to there's page tables stage 3 another

stage gets injected into that Dom 0 app and then in stage 2 jumps to that stage now this is actually kind of funny to watch because if I say here's a program running in gdb which is about to do a system call we can see here it's got to do a system call does a system call and an instantly jumped to looks like kernel space right if you're a developer you know there's waking you give a user space probe executing in high memory and there is it although gdb can't see anything dear the application can write so it's actually executing stage three and it's just so if that even Linux can't see it right but eventually it starts getting to

stage four execute stage two is copied stage three in and here's a training stage through and in eventually runs the execution back to the program the in stage three goes to stage four because of reasons so then stage 4 is also injected there now stage for it we have code execution is root and domain zero this is great but I want to show right I want interactivity currently we have our stages that are hijacked into domains zero application how do they actually get back to the exploit running in my guest operating system from the main zero in the applications point of view nothing else exists right it's like this picture which took me forever fine

right Stage four wants to break out of domain 0 into heavens above right so what we do is we get Stage four to make what's called allocate some memory and put some rings into it when I say ring I just means circularbuffer right so what can happen is once the extremely exploit knows where these circular buffers are it can write a command that we give to this circular buffer the stage 4 can see it seen that command turban shell and indel be an input ring an output ring and it can write the result to another ring and then where our external exploit can see that and then print that out to us and these are the rings because

they're circular right the hidden tail pointers will just loop around so this is what this looks like yeah yeah so stage force forks so one side of the fork will just go back and continue running the program is if nothing had happened like you saw in gdb and then the other one will handle to etch the actual shell execution so this is what the we currently have set up so Stage four tells our patch syscall handle where to find this memory and then I exploit can query that so then they can join a link together so now we actually have communications and this is the code that does reading and writing so now we have remote shell as route in domain 0

this is actually good ok so enough talk but first I did promise what happens what happens when things go wrong so here's me running in an if sorry for those bad video me running a early version of this exploit and as you can see this is the main zeros PS tree and you can for some reason everything is now loading the facility incopy he's a Python so engine X and so forth now it's python-based and the program the machine creations brew soon after this ok so enough talk let's have the real demo so here's and okay so Co have you pronounce it is I might main 0 so here it is showing the VNS writing

there's a domain 0 vm a domain and a bed no I'm normal I'm privileged domain here we are in there unprivileged domain so we'll make the exploit office when they clean of course and then we'll make it cool and we run the thing now this is D message output so the kernel output so remember it has to execute it has to win a race condition and do a dodgy sir so Linux doesn't like this a guest operating system does not like this module and it starts screaming out about it and saying you'll see if you use a locked up and stuff that's because I'm not yielding back to Linux normal modules are supposed to your back ok as

you can see now it's been triggered it's copying its stealing memory from zen its patching all the things I told you about and now it's found this shell ring and it's cleaning up after itself and it's doing more cleaning up after itself and now if we switch to another terminal we now hit this is still in the unprivileged machine and privilege virtual machine now we have a root shell in the main 0 [Applause] and we can do everything that a normal person could do in route including shutting down the machine from within the shell which is hilarious now I might have mentioned cubes I'm cubes as a security system I believe someone talked about it yesterday I'm in

theory this exploit can break cubes and it used to usually have to go for an unprivileged cube to an a privilege cube unfortunately when I tried to run it at 1 a.m. last night and this is what happens I've changed my code so much since I last run it unfortunately you and certainly insert the exploit module and the machine dies so obviously there's some work to be done there and so this is the final setup the Beck my friend said what are the takeaways for the people so I don't know so far thing isn't just for file and network format if you customized fuzzer it can be used for lots of interesting things virtualization is not a panacea right

there's waste could be people like me who will break out a few VMs and all that sort of thing writing a VM ex-players like writing a operating system parasitical operating system that the guest lights the real operating systems and for things that may or may not be there and that's really waited to bargain that's really exciting and debugging interrupt handlers is really really not fun thanks to Tim Loras is in project for dealing with my crap these sites Wellington oversee and Tamura level working with you guys thanks [Applause]

you