Framework for Embedded Device Analysis

thank you sir I appreciate that we both work at cert and the vulnerability analysis team my name is Madison Oliver I've been interning at cert for about a year I'm in my second year at Carnegie Mellon in the Heinz College studying information security policy and management it's quite a mouthful I'm Kyle O'Meara I've been at sei now cert for about two-and-a-half years and before that I was with fireEye and started off my career at NSA for those who don't know what sei is right so we're a the research and development arm that focuses on cyber for the Department of Defense how we're masked and I sit in the directive is we're insert we're in the threat analysis directive and on the

vulnerability analysis team and then specifically attack modeling right so we have sub teams of overall arching teams but we're gonna you know go over our framework for embedded device owner ability analysis so the agenda for this presentation is pretty obvious as follows we'll give you guys a brief introduction and the motivation behind writing this paper I feel like I can't get too close to the other microphones well we'll discuss the framework that we created and share our results with you of actually applying this framework to an embedded device go through future work that can be done to better improve this research and then a brief conclusion that's fine so as I'm pretty sure all of you know

embedded devices are becoming significantly more ubiquitous and readily accessible to the average consumer embedded devices are more or less everywhere now and they're going to continue doing so as device popularity increases vulnerability research about these devices should hopefully and ideally will be increasing too so there isn't a clear process when you're researching embedded devices there's no like step by step here's what I should do here's what I should be sure to look at everyone kind of just does it as they see fit and like hopes for the best that they have caught everything so the framework that we're proposing is a concrete scientific process a nice step-by-step relatively easy to follow process I hope to thoroughly complete

research in vulnerability analysis on embedded devices a motivation behind creating this paper was giving other vulnerability researchers a way to create comparable and thorough results so they can actually do something with the results that they get and we wanted to ensure that every single component of the device was researched at least at the most basic level a lot of the time and research especially embedded device research it's very easy to get bogged down and only focus on one aspect of it and these devices are so complex that there is a lot more that could be missed so we hope that our framework would prevent that from happening the framework that we are proposing has these phases it starts with embedded

device list curation information gathering firmware analysis web application analysis mobile application analysis hardware analysis and then obviously vulnerability analysis so we're going to go through each of these steps in detail starting with embedded device list curation so this was something that we had to do obviously to come up with the device to test this framework on but ideally those that are vulnerability testing embedded devices will already have a device in mind that they want to test so they wouldn't have to go through this process but for our sake this is what we did we developed a list of about 300 embedded devices all different kinds to pick one to test our framework some of the key considerations

when you're picking a device to test is that you need to have access to the physical device you need to be able to buy it otherwise you can't do any of the hardware analysis if you don't actually have the device itself so ensure that that is something that you can do and you need to have access to the firmware because how could you test the firmware you didn't have it so you should also consider the number of firmware versions hopefully there's a couple hopefully they've been updating their firmware the availability of this you need to have access to these files whether that be publicly available on the vendors website or you know somebody that can

maybe get you the firmware files and if the firmware still being maintained hopefully they have a version that's still being updated but if it's no longer being updated that's something that you should note in your vulnerability research because then it means that those vulnerabilities will remain present the next step was information gathering the purpose of the step was collecting information about the past and present vulnerabilities about the device in question we also looked at similar devices from the same vendor and then exploit vulnerabilities and exploits around that vendor entirely that should have said two steps not three but it's actually two steps so aside from identifying everything with the device in question you should identify all of the vulnerabilities and

exploits past your present with devices similar to that from the vendor all of them that that vendor makes and then obviously identify and make note of all the exploits if there are any which hopefully there are make your research a little bit easier and keep note of all of that there's many internet resources as I'm sure hopefull all of you might be aware of exploit DB virustotal that have exploits for free for testing purposes the next thing we did was like a little bit of firmware analysis right so we use two tools to do this this isn't like an exhaustive list as these are the tools that we found to be useful right one is

an open source tool called bin walk right it's used for analyzing firmware it's available on github it's maintained I believe still and it allows the diverse engineer the firmware itself right in those four looks for key values and extracts that you can extract that content out of there the other one is a custom developed tool that we wrote because I was getting bogged down with running too many of my own little scripts so I said enough is enough and took some time took a step back and wrote in our own tool which is also available on github I will provide a link at the end but what this tool does it looks for these key indicators of

possible vulnerabilities in you know firmware file systems themselves and it also ties in V feed ins that excellent open source tool that we integrated into our tool which is like sort of a correlation database of a bunch of CVE data exploit DB data Metasploit data that are expanding as I go I'm I talk the developer all the time so we integrated in that because it allows for us to when we find indicators in the on the in the firm where we can then query this database and get kind of a feedback up like say which we'll show you later on is this binary associated with any sort of CV ease if so maybe you should

now check that binary on that firmware to make sure it's either at the most updated version or not that's covered one of the examples that you'll see you later on

so most embedded devices have some sort of web application interface that they use it's usually administrative in nature but when you can't get to a device which can happen often with embedded devices like medical devices and plans things like that the only way to access it is through this web interface so this is a very obviously something that should also be tested when you're testing for vulnerabilities and the embedded device there's tons of open source free tools that you can use to do these obviously many paid subscriptions that you can get we decided to use a Wasa's app and nikto to test ours they were freely available they worked well in our environment and basically just did what we needed them

to do zap is a proxy that intercept intercepts the website request and then passively in actively scans the website for vulnerabilities and nikto searches for potentially malicious files checks versions and looks for specific issues relating to the versions that are used so for mobile analysis as you know a lot of devices might have mobile application they use to interact with the device in this sense were first focused on their Android applications themselves because they're more easy free little freely available to download then said iOS applications not that you can't do that but we focused on one aspects here so we use another open-source tool that actually D compiles a tool right and an Android application itself is a sort of

a zip file container so this tool this open-source tool knows how to read the internal files and extract everything out of there and then again we use the tool that we developed to then run on that expanded directory that is now a bunch of files from the Android application so the then the next step is that since you have physical access to the device you should take what you can do here is five additional steps to do Hardware analysis right so at minimum you can identify all the markings on the device what's on the outside the case is there Hardware versions is there regulatory information is there Wi-Fi passwords right you know is what firmware there is

what MAC address things like that and then since you have a device now you can open it up right there's a little more advanced these next few steps but there there's a really pertinacious you can gather right you open it up you look at the printed circuit board inside what components are on it identify those components as best as you can and then you look for data sheets from those components and tried to identify pin outs right for the next step which is then they try to dump the actual firmware from the device right so if the and the back of the device it might say that this has installed with this firmware right now you can either verify

or not that that is actually installed in the device and then you extract the contents this as we mentioned above using bin wok and then we run our tool drama on it as well and then the last step is then now you can compare multiple versions of firmware right if you're the vendor has their firmware on their website you can gathers firmware and I can compare it to what's actually installed on the device and look for comparisons are they updating their versions we'll talk about what we did later on and that and then you kind of a it can be a little cyclical here right you can kind of go out and do more open-source research based on what you

found right and in this sense we're kind of just looking for the flash component right but obviously there's gonna be many more components inside the device right there might be a Wi-Fi chip some kind of Bluetooth to you know communication and other RF things but in this case for our framework this time around we're just looking for the flash component right and then obviously at the very least we're gonna do some vulnerability analysis we broke this down into two different steps starts with a manual review of the firmware update for patches so you should have this entire list of vulnerabilities and exploits for the vendor hopefully for the device itself in comparable devices from the same vendor so you're reviewing

these firmware updates to see if these vulnerabilities have been fixed in what version they were fixed so then you can go through and test if they were actually fixed and then you can test the actually the actual exploits that you found for these devices against the device in question two you should test all of them even exploits for similar devices that aren't the device in question that you're testing to see if there's any overlap if there's crossover but vulnerability exists in one device similar device from the same vendor does it exist in the other one you should obviously always take caution when downloading exploits and highly encourage to review the exploit code before actually testing it

disclosure really depends on the vendor that you're working with ideally and hopefully they will have a disclosure process already set out themselves usually just something on their website you can submit and say hey I found this and go through them if this doesn't exist or you can't get ahold of the vendor obviously you can follow the certain guy to coordinated vulnerability disclosure and submit it to cert and go through that process so to test our actual framework we did it on the d-link dcs-930l I camera ample amount of testing has been done on d-link routers which I'm pretty sure we're all aware of but very little testing has actually been done on any of their Wi-Fi enabled

or embedded devices so starting with the curation how we decided to pick this camera so we compiled this list of all these devices all the information about them prices firmware availability if it's still being maintained all that fun stuff so we decided to go with d-link because their firmware it's freely available for download on their website we knew that we would have no issue getting their firmware this camera was also relatively inexpensive so we knew we would have no issue getting the physical camera itself we tested five different firmware versions for this camera one point zero four six eight nine and ten so once we decided on the camera we did ample amounts of research on vulnerabilities

and exploits that exist for that d-link camera and other dealing cameras we had got sixteen different current vulnerabilities with their CVs vulnerability numbers Metasploit modules exploit did basic exploits all all that fun stuff there were not actually any that existed at this time for this camera in particular so all of the vulnerabilities we were able to find just in background research that currently existed and had been disclosed for four other d-link cameras some of these included remote code execution remote access unrestricted upload cross-site scripting information exposure denial service directory traversal and cross-site request forgery so for the firmware analysis part we focused on the newest available version at the time of writing this paper and presentation that was

available on D links website to focus on our analysis so we use been walk write to extract the content to look at the content the firmware itself ideally like the best case scenario is our file system underneath right Ben walks able to define that and extract that content from you sometimes they're not might be a file system you might just be a bunch of files right there's other times where it might just be a binary but in this case we hope that the binary the firmware itself hasn't contained file system in this sense one of the known embedded file systems is a squash FS there's a few different types of embedded system file system that you can

find on devices we list those in our paper but in this sense we knew there probably a squatch FS and there was and was able to extract that content so once that contest was extracted that's when we ran our tool that we developed so in in identify the indicators it identified two files of interest and it also identified a version of busybox that was out-of-date that contained vulnerabilities itself so wonder files of interests this in this case these files contain a lot of product information that would actually expand your vulnerability research and analysis as far as like pin information more information that was on the back of the camera then there should probably been right so this information was either

maybe left behind not meant to be there should have been cleared out before it was uploaded you know to the image itself who knows but it provided me and us more insight to then do further analysis and looking in the different areas and then you know since using the V feed integration with trauma we were able to say okay run we identify there's a Vox binary on the file system now run that against this integrated database and then tell us if there was any you know vulnerabilities for busybox well there is and then we were able to then go and look at the version that we have on this file system compared to what's the most updated version of busybox and

found that we have an out-of-date version on the most up-to-date software and it's out of date version can has known CBE's for it right as obviously i've already discussed you have to test the web application interface so we use two different tools as i said before a wasas app we use to spider the web page and perform active and passive scans against the web interface I found four different findings to that we're really of interest and two that are more related to the best practice they didn't have cross-site scripting protection enabled and the X content type header was mission missing this header just prevents content sniffing from a third party so it these are kind of hard to do

on a web administrative interface because I have to already have administrative access to then get to this point to do that so these two vulnerabilities weren't were more of a best practice we did find two vulnerabilities that had not been previously reported and that were novel vulnerabilities and we've submitted that to the vendor website which I will talk about later nikto confirmed all of our findings almost all of our findings with OS app which is why we use two different tools to make sure that they found the same things and that this was consistent it did find one additional vulnerability likely because nick doe asked for the administrative login information and because we have an avid because we made

it we figured we would give it to it and see if it found anything once it was authenticated so it did find one unique result because it was authenticated which was this cross domain XML file which is part of Adobe Flash it has a wildcard entry which means that any trusted website can view user information on this administrative portal so if you compromised the trusted site you can use that trusted site to that and get user information from this target site so we were able to download this specific mydlink Lite application that was recommended by the actual d-link themselves from the Google Play Store we decoded it using the apk tool and then ran our tool against it and in this case

we didn't find any indicators that were vulnerabilities in any sense so next phase is the hardware phase right so we this is broken down in those five phases so the first one was identified the markings right so there's numerous markings on the back of IOT devices specifically cameras in this case right you can see here there's a list of different information that was valuable right Hardware versions MAC addresses regulatory information you know SSID passwords serial numbers things like that we what I'm looking for in the case is a regulatory information right so this is information that if has radio frequency communications on the device they have to register the devices in set countries right you have the FCC

in the states you have the IC in Canada Europe has one and China has one right so they're registered with these devices they have these databases are freely available and when they register with these devices they sometimes upload documentation to these databases which then can provide you more information right it's usually a picture of the device verify that the device you have is the one that they registered sometimes if you're lucky they take pictures of the inside device right see the PCB board in this case they did it but there's these this is where that cyclical information happens where you can gather more information and go back and do more research on what you found

so once we popped it open the device and looked inside on the printed circuit board we identified you know a couple of CPUs if they had a couple different memory informations the skyworks is actually the Wi-Fi chip itself but we were specifically looking for the flash component which then we identified as one of the Mecca onyx components so then the next step is to dump that flash off that device what I mean dump that flash to actually extract that content that's on that chip specifically from that device it's a little more advanced process however it's sort of straightforward once you've got it done you know in our paper we didn't get explicitly into how the step-by-step it

does it but if anybody questions I'm happy to go into details about that but we use a bus pirate right a Linux VM running flash ROM right so a bus pirate allows it communicates with different electronic devices flash ROM is a open source software that allows and it understands how to read different flash memory components from devices themselves so using these two components we're able to extract the firmware from the device and because on the back of the device it says that this firmware on there is version 1.0 for that's what we label that former version and then it's like okay now we need to do that step we talked about earlier now we need to look

at that firmware itself and then extract the contents and analyze it and we do the same process that we did from the one that we found from the vendors website here and then we you know extract the content we found that it - also had a old version of busybox that was out of date so this device that you get day one that you plug in I'm most likely it's gonna say we want you to update the firmware but say say no right now you're already like way out of date with a you know busy box if anybody know what busybox is it's sort of like the utility tool component that runs all the commands inside the firmware itself

right so it hides six CVEs our tool identified six CVS that's using four specifically for this version of busybox however luckily at the time there's no exploits for this you know these CVEs right but that doesn't mean that the device isn't still vulnerable it just means that no one is publicly you know made an exploit available for said CBE in the last part we then compared you know the files from the version we extract from the device the earliest version of the website and then the contents of the device and the most recent version of the website we saw very little sharing which that could tell us that yes they are updating their firmware from version to version

however we were only did a comparison on hash values right so as we all know all you do is change one bit the hash values are different right so you know there's some more testing to do there they actually see how much is actually shared however the most common files across all the firmware that we found and the one we extracted from the device is that shared library object files right or shared object library files now why is this sort of interesting right is that if a vonabell exists for one library file it's going to affect all versions of firmware right so you can see the impact that has there as we just as we discussed before if the researcher

should manually review the firmware updates and I say manually because usually they just released them in release notes it's like a nice little PDF that says hey here's what we updated and why we updated it and unfortunately I didn't find a better way to do it other than just manually reading it so this showed that the two new vulnerabilities that we did find that had not previously been reported had never been reported to them and had never been patched because they weren't available in their release notes which furthered our idea that they were truly new and novel we attempted the known exploits against the device's 16 different vulnerabilities some with exploits all different model cameras not

the one we're testing in question so they'd not return any results and that's not to say that our camera in question doesn't suffer from the same vulnerabilities as those that we found in other cameras if these exploits could just not translate exactly to our camera anklet in question and as we discussed before we're following the disclosure process with the vendor because they do have a disclosure process that is very clearly laid out so we have contacted them and it's located very very easy to find on their website so some future work and improvements that we could do for this is applying this framework to other embedded devices we could apply this to other d-link cameras because the purpose

of this framework is to create results that are comparable so we could apply this to other cameras compare the results with this we could apply it to different vendor cameras to continue that process or we could just apply two totally different kinds of devices it doesn't have to be cameras we could further this by doing that we would also adjust our analysis environment because our environment wasn't set up adequately to completely set up these devices because d-link is a little odd and their requirements to set something up so in addition we'd also like you know this is a framework that's supposed to be evolving right so we know that there's RF analysis done what I

mean by that I mean we didn't do any Wi-Fi analysis Wi-Fi testing things like that obviously with a new crack attack right out there like that's something we could test as well it should probably most likely be vulnerable but other our Wi-Fi testing we can do bluetooth testing advanced hardware analysis I mean in that sense is like change firmware see if we can upload our own version of firmware to the device is it doing some one kind of check sums against the device if a device allows you to push your own firmware device without looking at checksums we have bigger bigger problems to talk about so it would expand analysis there and then specifically dive into like looking at

the explicitly Nerys on the device right a lot of vendors produce their own binaries used in things like that you know I'm actually looking to doing some reverse engineering on those binaries are those binaries themselves vulnerable they're different operate you know built on different operating systems and your typical you know Intel right so there's a little there's not many there's a lot of people out there doing arm but not many people didn't miss out there right there right now so there's some binary analysis to be completed there to add into our framework at a later point in time so brief conclusion about everything that we just talked about we developed this framework to create a

universal holistic macro level approach to embedded device vulnerability analysis we want other vulnerability researchers to be able to use this to ensure that their analysis is complete and thorough and at the most basic level they have at least touched every single part of this device we tested this on a class of embedded devices a Wi-Fi camera and found vulnerabilities that had not yet been published so this helped us further prove that our framework does work and we were able to find things that no one else had found yet we developed an open source tool trommel to help other researchers in this process we expect this framework to not only explain with our future work but with

the evolution of embedded device landscapes we this is a living breathing document we expect different parts to be added to this different devices have different needs and we expect to expand on this in the future that's it so anybody has any questions comments concerns unfortunately the paper was supposed to be published today however our tech editor got in a car accident last week so I you know that's understandable delay right there however the tools available right there right now to use I'm open to feedback we're open to feedback it's not that's also living and breathing too right so I'm already have comments to update things and improve upon it and know that it's analyst code right this isn't

development code you know I'm not a developer by trade I can develop but I write analyst code meaning that it does what it needs to do and works as well as it needs to work that's what I mean by that right so but yeah we'll hopefully have it the paper published within you know 10 days or so we're in the final stages of editing like I said given the circumstances I couldn't harpin him and there was no time to actually get it done but yeah we're that's how you can contact us any questions from the audience we have plenty of time ten minutes I think we left perfectly yep

yes oh yes when I extracted the flash binary from the device how am i doing it I'm actually connecting to the actual pin outs right so when looking at the components out inside the print circuit board ideally you want to try to find the data sheets for that actual chip and then you can figure out what the pin outs are and the bus pirate works so nice and freely with like the sort of pin outs right so I was able to connect to those and then pull it from there

great and other questions here I think there's another talking at at 10:30 correct that's fine [Applause]