← All talks

0-day Research Disassembled

BSides DC · 20191:48:57127 viewsPublished 2019-10Watch on YouTube ↗
Speakers
Tags
Mentioned in this talk
About this talk
0-day vulnerability research is a hot topic these days. Adversaries, governments, and researchers all have their secret stash of 0-days. Bug bounty programs have become evermore popular. With a tuned skill set, anyone can get started hunting bugs. Some do it for fun. Others do it for cash. We do it for a living. It’s our passion. How does one pick a target to research? How does one improve the success rate of finding a 0-day? What skills are required? How does one deal with setbacks? We will go over these and several other questions via selected case studies of 0-days we have found in various high profile products. We will discuss ~10 0-days that haven’t been disclosed (at the time of submission) and go over various scenarios showing how and why these were found. In order to be a successful researcher, there is a broad skill set and knowledge base required. Additionally, the mindset of a security researcher is a key driver of success. We outline these points alongside real life scenarios of our 0-day discoveries this year, demonstrating that with the proper methodology, luck, and determination, anyone can achieve similar results and help contribute to making the world more secure. Chris Lyne (Sr Research Engineer at Tenable) Chris is a member of Tenable's Zero Day Research team. He enjoys dissecting complex applications and lives for the hunt. Having written code in a multitude of programming languages, he has deep roots in software development, but his true passion is software security. Chris is an avid learner, and he is continuously evolving his skills, capabilities and methodologies. Chris believes any problem can be solved with knowledge, intelligent decisions, and sheer grit. Chris plays competitive tennis regularly in a league. Occasionally, he enjoys smacking a few golf balls around as well. When he’s not doing something active, Chris can probably be found reading a technical book or tasting craft beers.
Show transcript [en]

that also requires you to have an understanding of the the architecture underneath and the assembly language so x86 are armed and if you're having trouble just just look at the strings and they'll give you lots of clues so you know you also want to be able to analyze the application while it's running so you know debuggers tools like gdb on Linux or window bug you need to know how to use those inspecting logs I find is also very helpful you know the developers do take time to to put in helpful log statements so you can kind of trace the application while it's running and you know also just adding print statements if you do have

the source code that can really help you as well most of the applications we look at our network based so being able to capture that at that traffic and analyze it with Wireshark or TCP dump essential and on the web side burp suite is amazing you know you can capture and modify traffic in transit so we write proof of concept scripts for virtually every vulnerability that we find so being able to write those tools and package up you know complex logic you know like client-server communications that'll make you more successful and in addition to you know making yourself more efficient if you can publish a tool and then release it that'll make that next researcher more efficient if

they're looking at the same kind of stuff and fuzzing is probably one you've heard a lot about you know finding memory corruption bugs and stuff like that but just having an idea of how to feed an application random or malformed data it's gonna help you generate crashes there are some frameworks that can help you with that like a FL or peach and if you want something more custom definitely go to like Python or your favorite language so I'll hand it over to David all right thanks Chris I'm gonna go over mindsets and character qualities that are helpful for finding zero-days while we pick we're gonna go over four I'm always for it helpful in anything you do in life we

picked these four because they're especially effective for zero-day research and so the first mindset that's helpful is how you acquire your knowledge base and knowledge base in zero-day research is almost everything you have to know operating system internals programming languages encryption methods file formats and also mitigations and ways to bypass them and actually vulnerability classes you can trigger so it's an enormous landscape and one attribute that can help you expand your knowledge base is just having a curiosity so if you're someone who likes to figure out how things work or read other people's research and how you can apply it to yourself this is a great trait to have for a zero-day research now but just

having knowledge base is not how you you know alone is good enough you also need to actually find the books and this is where attention to detail comes in it may be thousands of lines of a symbol you're looking at and it might be just one instruction an unsigned comparison that's your integer overflow you need a fine so this is a level of attention to detail you sometimes have to have an or a spot these very subtle bugs another way of saying it is when it comes down to attention to detail security wise you have to have more attention to detail than the developers and the code reviewers had when before they push our code to production so it's very

important the next trait want to go over is just intuition and this comes with experience but it's what will allow you to focus on certain things and not so much others and I'll improve your efficiency and turning over zero-days and bugs and applications so an example is sometimes you know you might see a the inner workings of an application and how various components are interacting and it's your intuition I can say you know I feel from my experience and how these things work I feel like this section of code is vulnerable and you can just focus on that first and start attacking the application it's also what helps you improve your reverse engineering speed a lot of reverse

engineering is a is a hybrid of manual reversing and also just kind of intuition of saying oh I think I know what this function does this is this is huge and finally I'll end with the persistence again important anything you do in life but in zero day research it's especially important especially when we're in an era where security is more practiced and often times your target has already been looked at by other researchers so it kind of becomes a war of who is persisting more to uncover UNTAC code base I'll end this one with a story I'll go at the end here we'll go over some like bugs we found and when I'll go over

zoom and zoom was very difficult I spent probably three weeks on it finding nothing I tried sequel injection tried fussing the protocol all types of attacks and I had the decision to either persist or just give up and move on to another target and one thing that helped me was I just reframed it by saying do I know enough about the target that I can prove exactly why it's not vulnerable and once I asked myself this question it helped me with my persistence and I literally went through and I would go well it's not vulnerable simple injection here because the sanitization routines are called here here and here here and as I went through it really broke the surface

since the bug started to come so this is just a way that balanced persistence obviously you can't go too far with it but that's a great boundary to ask yourself if you're truly done is ask yourself do I must clean enough with the target I can prove exactly why it's not vulnerable so that's all so I'd like to talk about the research process so our first step is to select a target right we consider a few different criteria we'll look at well-known vendors like Verizon no new cutting edge technologies like our low resume and as Joe mentioned in his previous talk SCADA it's a hot topic right we also like to look at popular or ubiquitous types of systems

like Windows or Mac OS or something more niche like mikrotik but ideally the goal is to produce something meaningful at the end so if we can't get our hands on the target we can't research it so you know we will have to select a new target sometimes and a reason we might not be able to get a target is because it's too expensive maybe we don't have the budget for it or it's or it's a large physical device or maybe we just have to jump through a lot of Hoops to get our hands on it because we're dealing with salespeople and you know more stuff that we want to deal with so here's an example of a Tesla that's kind of our RJ

okhla on the team we really want a Tesla but unfortunately it's very expensive and where would we store it right so that one's kind of sad at the moment another example I was just trying to get my hands on a 30-day trial but instead of getting that download link in the email instead I got a sales type of thing so I had to jump through some hoops in order to get the target so that's another reason I had to end up getting a new one ideally we want a free download 30-day trial or something within budget that we can license so once we get our hands on the target then we can set it up right and

this is where the virtual environments come in so we can take snapshots of whatever we want we can set up docker containers virtual machines so for every different type of target like whether it be Windows or Linux or Mac OS I'll have a base research environment set up so that's that's a snapshot right so I have that research environment set up and then I'll install the target on top of it so I always have my tools underneath of the install so I've listed out some tools I use for Windows but I really love Visual Studio you know just have a good IDE to look at code and compile Ida Pro burp suite for the web and have a

debugger on hand and another thing might not be as obvious is just make sure you understand the system requirements for your target some of them require you know lots of memory maybe more than your machine has maybe they require lots of processing power or or more storage than your your environment has and keep in mind the dependencies like different libraries and stuff some some targets are a pain to install so just make sure you know what you're looking at there and I mentioned snapshots those are probably the most important part of the process I mentioned the base research environment that you install on top of take a snapshot there after you setup your target snapshot that so you have a fresh

snapshot fresh install and if you think you're going to do something that might clobber your target like you know mess up configuration files or whatever you can think of snapshot before you do anything like that all right so Chris just talked about setting up your target and getting it configured and now's the part where we want to start tearing it apart and looking for bones and all the fun stuff but before you start tearing the application apart it's really important to find your attack surface first and you can find your attack service in your target by asking yourself two questions one is one of the privileges of the running target and two what components of the target cross into lower

privileged boundaries this will apply to finding like the spectra meltdown bones or a sequel injection across the web just depends how you frame these questions for your target and it's important to understand your tax surface before you start tearing it apart because you might realize that 90% of your target is not applicable for a zero-day research and by just honing in on the attack surface that's applicable for an attacker you cut down your analysis time extreme so very important - first on your attack surface and I'll go over an example here this is a basically an example of what you'll come across in any target you investigate whether there's an IOT device or an application what we have here is a

operating system with various privileged boundaries within it and might be connected to an external data source in this case internet but it could be Bluetooth USB and if we take discord for example well ask yourself the first question what privileges does this cord run as runs this default user unlike Linux and Windows okay so now second question is what components extend into lower privilege boundaries so what are our lower privilege boundaries here well we got a sandbox container environment or just the open Internet so right there that's our that's our Lord privilege bothers me to focus on and so now we seem to find components that extend into that that we can reach from an attack

surface and what do these look like well these components are things we're all familiar with they're like freaking me files network sockets devices named pipes shared memory RPC basically inter-process communication technologies they're USB bluetooth it goes on and on and on so if we do that for discs or like for a client for example what what components that can we extract a discord that are interesting for zero-day research and we do that here we can see like on the internet it might be listening externally for you know text and video to be processed through this port I can process hyperlink clicks from the internet you know you can craft a discord protocol handler and get discord a

trigger with the malicious argument perhaps so that's something to look at and on the sandbox container environments I'd like on a low privileged you know boundary on the operating system this Court has like a local service listening on localhost and maybe from a sandbox container you could talk to that and get a sandbox escape to occur or be a discord it also might have exposed shared memory objects with no protections or again hyperlink click so this is an example we just we just tore apart there most interesting components so we just need to look at this now and what do we do now that we found an interesting component oh there's a few office here and Chris mentioned fuzzing

so yeah fuzzing is definitely one of them there's also you know go through disassembly or debugging and I'm gonna go over the pros and cons of each one of these and these are more of my personal pros and cons I found so starting with fuzzing as we know Chris mentions like the brute forcing component with arbitrary input and the pros of fuzzing is they're good for large and complex components I think of things like scripting engines or graphics drivers just very painful to reverse but fuzzing can really help supplement that and automate that process in what we call code covers getting good code coverage with your testing it can also it's also good at finding more obscure memory

corruption related vulnerabilities like UAF stubble free type confusion some of these are so obscure like you might need to create three objects of this free the middle one then sort the two and it's like a race condition that does some type of heat exploitation like human mind might not be very good at spotting this when you're disassembling or reversing so fuzzing is great for attacking this and finding this very obscure memory corruption vulnerabilities but fuzzing does have some real cons to it one is that it must be context relevant to get proper code coverage so going back to my scripting engine example if you want to fuzz a scripting engine you'd have to be know

all the keywords you have to syntactically put them in a reasonable order or a graphics driver you'd have to know the right iocked dolls to hit with the right parameters to get code coverage in the driver it can turn into a big reverse engineering job which kind of defeats the purpose a little bit of fuzzing once you you know actually understand it so that's a con there and it also is not really good for finding logic flaws so logic flaws is because with fuzzing you need instrumentation to tell you that something broke or went wrong and logic flaws are notorious for being subtle and not screaming at you that something just went wrong like think of

past reversals or things like that and so if you're looking for a subtle logic flaw type bucks a buzzer is it's it's not really the best tool suited for that so moving on the disassembly this is one that I kind of personally favor but it does have some cons too we'll go over well one of the pros is that xref ability right we talk about hunting for components you know if we want to mail a find where that component is handled when this big application of discord you know X roughing is so great you know extract on the receive API and you find right where it's receiving Network data and then right after that processes that

you're like instantly there provides precise understanding of functionality this is because there's no more guessing you're actually looking at the code and seeing the copass and interesting things you can hit it's also of course good for finding logic books because of this you're using your human brain to really churn through the logic branches and seeing what you can hit but the cons of course are like brain must emulate Hardware sometimes disassembly when they're using a lot of like indirect calls or push pops on the stack or it's off you skated you have to turn your brain into an Intel processor which I haven't done yet and it can be pretty hard so maybe not the best for those

types of situations and it may miss more complex and obscure memory corruption vulnerabilities of course like the example I just gave trying to spot some really obscure heap corruption Bowl and so you could hit through static analysis like that can be that could be pretty tough finally moving on to debugging pros here can identify actual crash types and locations I mean ideally you want to combine a debugger with fuzzing so it can you know tell you what just happened and what line just went wrong it works well against obfuscated packs code is because the unpacking or deification process will happen in the debugger environment and you'll actually have introspection into the raw code once it's a the obfuscated unpacked

provides seamless inner library call support you know this something disassemblers are not good at a lot of times these especially in Volyn ability research you'll see a lot of inner library calls happening with other dll's that the application has and you know when it calls into it you can seamlessly step in step back out of it I posted this assembly where you just see a symbol and you don't know what it does you have to open another Ida instance and look at that DLL so I provide some slick maneuvering there the cons though of course it's not good for xref and it's clunky to navigate and in general it's just not a viable standalone tool debugger is something

you want to supplement with not use you know replace okay so now Joe will talk about finding a bug and what you do alright so say you just turned your brain into an x86 central processing unit and found a bug now what you did all the work already right so to handle it now always double-check your findings start with that make sure that you can reproduce your results revert the snapshot use a default and install on the application check it again so you want to document everything the versions that you use post install configuration changes additional modules that you had to install you want to be really thorough before reporting to the vendor and the reason for that is that your

credibility is a major driver as a disclosure as a disclosure to the vendor which is going to determine how they respond so you see researchers get researchers get blown off all the time by vendors they don't care about security they don't want to fix it they just don't want to spend time on it or money so if you're it's very important to be accurate and correct because if you're wrong in your initial report it's gonna be extremely difficult to convince the vendor that you're finding human matters so just check it again before you send it after you verify your finding you need to clean it up whatever tools or scripts that you used to prove the vulnerability exists you

want to get them finished and as clean as possible minimum minimum subset to prove your own your Bolton when documenting your tools or steps to reproduce a flaw be specific for example cross-site scripting as an example document the payload put the exact URL any options or headers that are required to produce it any type of configuration that's needed for the vendor to reproduce it and so documentation could be is really helpful here screenshots are really helpful it as another example if you're reporting some type of memory corruption you know privilege escalation escalation you need to provide an additional executable in your proof of concept to the vendor be specific in how the tool is intended to be used so the tool the

tools that you provide they don't have to demonstrate the most severe impact but they need to demonstrate and understandable impact and they don't need to be perfect perfect either these the vendors are not expecting some you know production level application just to demonstrate a PLC but it needs to at least be reliable so you need to you need it to do exactly what your documentation states as perfectly as possible and there's always some subjectivity to that but yeah at least one what you send to the vendor to work at this point write everything up take your time go back through your disclosure chop off any unnecessary information be brief but be specific the person that's triaging the report at the

vendor is probably not going to be familiar with all the ins and outs of every single product that they have so it's best to be specific at first and broaden reporting later as disclosure goes on rather than overwhelming all at the beginning with a ton of details and so now we'll talk about actually disclosing the finding so there's a few different types of disclosure private disclosure would be selling your finding to someone that doesn't really care about protecting people and it's not gonna publish the details 0 diem public disclosure on the other hand is publishing your research before giving the vendor a chance to review it so just dropping the exploit on github and saying here it is there's a ton of pros

and cons there but we're not gonna go into that so in general we advocate for coordinated disclosure where you tell the vendor what's wrong give them a chance to fix the issue and tell all their customers and users and coordinate your publication with them we'd we wrote a I need to get 90 day disclosure policy starts on the day that you inform them about the vulnerability and then they have 90 days to fix the issue before we go public sometimes they make it most often but most often they do make it sometimes they don't we publish and there's no patch so if a vendor is unresponsive you know will inform cert at the 45-day mark and

then publish pretty quickly after that but you know we almost always work with the vendors throughout that three-month time line to provide clarification to help them out with other issues that come up just you know clarification and communication issues and as well as to monitor the fix as they're progressing on it and our policy also has some leeway in there for us to be flexible with vendors that are clearly trying in a responsible way to fix the issue but are just not going to meet that deadline so we can we can provide extensions if needed and then as I already mentioned you know bug bounties have pretty well defined procedures as well so if you're

reaching out directly to a company find a security contract contact before you disclose a lot of the times you can't find one and you can first maybe reach out to sales information support or whatever okay but if you end up disclosing your vulnerability to an on security contact they're probably just gonna there's a 90% chance they're just gonna ignore it so once you sent the disclosure the book of the works done in most cases you want to follow up during that process just keep tabs on the patching progress and it's gonna let you know when it's safe to publish your findings so and you know and also what you can expect as far as bounty payouts

and stuff like that and it also is gonna ensure that someone else on the other side is actually paying attention if you're pinging them every couple of weeks or so and they're not just ignoring the problem and then ten days before the disclosure is up they're like oh no what do we do so all right so now we'll cover some of our own research and we'll talk a little bit about some of our findings and just general overview of our strategy yeah so we'll first talk about Citrix SD when formerly NetScaler so SD win is Software Defined Networking you know the goal is to replace traditional infrastructure like routers and stuff like that with software if you check out the topology

diagram at the top top note is SD LAN Center and beneath that you'll see three nodes those are all appliances so there are two different types of nodes in this diagram the SD wins Center is more like the head end of it you know it's the brains of the operation it's going to talk to the appliances and all the various components in the network however the su and center and the appliance both have web interfaces so that's what I looked at setting it up was pretty simple you just download a virtual appliance run it know configure required the only problem is the source code wasn't readily available so like when you you'll see that when you login you're

presented with a limited shell not the expected bash shell or that I hope for so what I ended up having to do was route the VM I didn't find a vulnerability in it but I had to log in and change some configuration so I did that using a Linux live CD you pop the ISO into the CD drive reboot and you've got a Linux desktop on the left you'll see that I could access the primary OS and that's the file system underneath the product and then when I checked out the Etsy password file you can see that the admin user has a custom shell it's a bin slash CLI shell hopefully you can read that and the admin user is a part

of the www and then if you look at the bottom screenshot you'll see www.supersportgames.com

was modify the Etsy password to change the show right so then next time I logged in I got that bash shell I was looking for and I can use sudo to escalate to root so that gave me access to the source code on the filesystem so my methodology is pretty simple when I have the source code of a web app I'm here to tell you you don't need fancy tools or anything like that all I do is I look for vulnerable patterns in the code and so that's you know programming language specific right so check out a wasp top-10 like for instance I was looking at PHP and perl so they would have guidance you know for

for what those type for what vulnerabilities look like in those languages and I use regular expressions next I'll look for sources of untrusted input so that could be command line arguments HTTP parameters or headers file i/o there's all sorts of input next I'll look for unauthenticated entry points into the application so how can I work my way in without requiring a login our session and most importantly make sure you really read that code and understand exactly how the application works that's what's going to make you successful so like I said they're two separate applications the SD wins Center is a cakephp application and the appliance believe it or not is a Perl CGI so totally separate so that means

you have to understand the specifics of the framework on Ernie right so cakephp versus perl cgi they're totally different different languages different logic the routing is handled differently the authentications handled differently and that untrusted input is going to look different in the code so like you know if a a get parameter named ID comes in they look totally different so when I was looking at the cakephp application I was looking for something specific and that's the off component if you take a look at the the red box you'll see there's a collector controller class that allows authentication or excuse me allows unauthenticated access to all of those various actions within the controller so I have to say thank you to

collector controller because what I found was that there were vulnerabilities so I found five vulnerabilities right they were all I could only get to them with authentication except when I found collector controller I realized that it would forward requests without authentication so I could reach those vulnerabilities without the off of it that you otherwise would need so on the purl side it's a little different he's not so framework II like in the red you'll see there's an explicit check for authentication right however I was able to bypass this because they're they're not checking untrusted input and validating it so if you take a look you'll see the SSL client verify header is checked to see

what the value is and if it equals success then we won't check for all so that ultimately further down in the script it led to a CBE so in addition to knowing you know framework specifics make sure you know how to identify the unsafe code patterns in those languages and so these are some examples that actually worked you'll see the exec for command injection and F open for you know directory traversal and then there's a sequel injection as well something else I'd like to hit on is when you're running your read regular expressions and checking for patterns and vulnerable code there's going to be a lot of noise so you really need to to work to tighten

up your irregular expressions and your patterns to give you exactly what you're looking for and then once you find those those syncs check to see how the input is flowing into those function calls so here are some actual hits for the bones that I found you'll see in the orange those are all command injections as well as the pink directory traversal with the F write and teal and a sequel injection in and green so just using regular expressions I was able to find all of these ultimately it led to five remote code execution Xin SD win Center that's the cakephp up and I was able to to reach all of those vulnerabilities with collector control or not requiring

authentication on the the appliance side I it was a little more fun I was able to chain born abilities together so I exploited a sequel injection in order to bypass some authentication and then that got me access to exploit a command injection down the road so it got me got me root there if you want to check out the exploit I drop link there so if you're interested in any more details like proof of concept strips they're all up on our research advisories and then we have some blogs as well outlining the the process in more detail all right I'm gonna go over exploit a fun and zoom I assume most of you are familiar with zoom conferencing

this is a critical remote exploit people of code execution as well and I'll go over a bit of the details here so first thing this CV would allow could allow attackers to hijack remote desktop feature during meeting which means you can force control and also send keystrokes as well you can allow attackers to a spoof chat messages to appear from other users in the zoom chat can allow attacker to boot other attendees from the meeting as well as kill the entire conference even without being meeting hosts and best of all these were all exploitable even by a completely remote non meeting attendee so over when you could actually inject keystrokes to another zoom meeting that's occurring in the wild somewhere

so going over how this worked this is a zoom to Paula GI kind of reverse-engineered how it worked zoom the way to clients dogs they'll make use of two channels TCP channel and UDP channel VTP will be for the audiovisual streaming and tcp will be for more like the meeting relevant status updates and this is what I call the trusted channel so this is what I'll say I'm sharing desktop right now or Who am i sharing it with or am i muted that's all tcp over the trusted channel to the zoom servers now the problem with what I found in there in their code is right here I took apart the part in the client app where it's receiving network

data from this from the server or the client and they're using a Bey a generic class socket IO which is abstracting all network receiving data so what they do is they just call this generic function here that received either TCP or UDP data doesn't really make a distinction which one and they create a message object like the one outlined here and they post it to a message pump and zoom so that I can get processed by the application and the problem is they're making no distinction if this was a UDP packet or a TCP packet of that accepted so here are like the four message poems we'll make use out of the one that we're

attacking here is actually the events message pump which is the top one and one that goes to the message pump the packet that it receives gets goes through a switch case to trigger various functions inside the zoom client whether it's like share desktop like I said or mute or kik the attendee out and these are some of the the cases that you can hit and what I realize is that well if there's no distinction between TCP or UDP packets what if I make my UDP packet look like a PSP packet with the correct data in it so here's a TCP packet the zoom would accept and I reverse engineer and some of the little components that

are important like in red here we have the actual message ID the function ID it'll hit Nats which case green is showing that it's telling the share desktop and the orange and blue are saying which clients to share the desktop with and in UDP the offsets are processed a little bit differently when I checked out how it worked and zoom but if I you know Pat it differently and supply that same function ID and this byte and then as well as you know sharing desktop instruction as well it'll go through that message pump and be interpreted just like it was a trusted TCP packet and in overall this is the what will look like on the top

right here we have like a malicious zoom client again it could be not even a meeting at Cindy's so also just someone that sent a UDP packet to the server or the zoom client and it gets processed over here by the zoom climb in the right goes through the message pumps if posted the events message pump then dispatch to the function table that we can avail that we can hit through UDP packets and like I say this could be hey share your desktop with me hey except these keystrokes run them and you can basically do all sorts of interesting things with that so if you want to read more curse you check out the try that we've the release advisor

you put out as well as my blog that goes over this in much more detail CET wells and medium so and now Joe will talk about Logitech Harmony hub cool all right yeah one more one more example here I hope these are helpful like it just kind of shows I think sometimes when you watch other people's work it's helpful to see how they approach some problem like if I looked at a zoo might be like nah there's no way there's a hole in there and like this one logic the logitech harmony I was just looking on Amazon just like what's an interesting device oh that one looks cool let me by and just see if I can understand it and find

something so so this device this device is basically a smart home hub it just aggregates everything connects to everything air conditioning your refrigerator your stereo your TV home security systems door locks and so the first step in attacking a black box for me is just to map out the network footprint of the device and it's a really easy way no effort required just to see what's available from the network because it's such a great attack stack starting point and then for this device it looks really promising there's like at the top you can't see it it's an nmap scan so there's three ports open and it looks really good weird port numbers a bunch of services open and then as it

turns out Logitech implements functionality over a couple of these network services so it has XMPP for user interaction and then it has a WebSocket service which is used for server communication and interaction with user devices iOS and Android apps so all of the functionalities ends up being implemented in Lua code in the firmware of the device so firmware next step is just to to acquire and analyze the device firmware to understand what's going on in there and eventually I found out how the device updates its firmware this is always a challenge for these IOT devices sometimes you can download it online in this case I couldn't download it I had to see where the device is

actually querying and it uses SSL so you can't just sniff the traffic but the my harmony application this application that you can update this device with on your computer doesn't pin certs so you can man in the middle of that application to figure out where it's requesting the update the firmware from so you can get a copy of the firmware step one and one of the nice things about this device and it's not always the case is that the firmware was really easy to understand it's easy to extract headers well known and file system understanding the bowl so a side-effect of using obscure file system file system or an obscure firmware type is that it

makes reversing it like another ten or 20 times harder but anyway so once I had the firmware for this it breaks down into a squash file system a kernel boot image and then a Linux update binary and I only needed two and in this like for this discussion I'm only going to explain the file system because it has all the application code which is where the vulnerabilities were so the hub uses a patched version of the lua 5/1 interpreter and all the code stored as compiled lua which looks like that on the left so you could try and disassemble it and understand reverse engineer lua intermediary language but it's much easier to just decompile it and there's

some decompilers out there I had to jump through a few little hoops like the normal Lua interpreter interpreter didn't work because the one that they were using in this device was had some patches from the open wrt community and so but once I got that all going it was able to decompile really nicely so in the end we published for vulnerabilities for this a couple of them were XMPP vulnerabilities that affected authentication but they're really two to the two that were actually interesting and that I'll talk about can be used together to by anyone on your network to remote root the harmony device so the first one is command injection and first let me describe the network flow at the

device so the harmony hub communicates with user applications on Android or iOS over the local network and it communicates with Logitech servers over the way in the user applications can also come in and control through the land to through Logitech servers and the hub communicates with a bunch of different Logitech servers but the command execution exists in the time synchronization code so we'll look at that one alright so time for a source code review that's probably kind of hard to see but so line 132 there it the device makes a request to a time server and then it takes the data from that request in line 137 and then it uses it in line 142 and executes everything on

145 so the patch diff here shows the changes on the left was the Berlin version on the right was was Logitech's patch after we reported this vulnerability and in the patch version all they did was just add a string match to parse out integers as a way of validating a timestamp but before that there was no validation it just takes the data from the get request puts it into a command string and executes sit all right so a second vulnerability is application command execution so you can so Logitech makes requests to the device to have it do stuff and so the hub accepts these commands from Logitech servers over the LAN on this web socket

service the handle post function in the device validates the origin of these requests but it only does that by checking the header an origin header in the request so it checks for this my harmony comm string and the result is that the origin can just be forward you can just add a header with my harmony comm string in there and it'll it doesn't matter where it comes from it'll just run it so you can basically send it any application commands and the result is that you can tell the device to contact our own controlled server from a time update and then trigger that false that first vulnerability to run to do command execution so you have

unauthenticated remote root of any logitech harmony hub that's on your network so you can use that to control just different household devices or security door locks and whatever kind of fun one all right so that's all the constant we have for you open up for questions or comments yeah yeah sure I think we all have varied experiences for that yeah logitech was really responsive with that you know very helpful friendly super nice like you know tons of thank you for finding this like they realized that we don't really have a dog in the fight like we're kind of doing this external research so they're real helpful no no they don't they didn't help me at all yeah mine for zoom I

think they recognize the critical miss and they they tested themselves and ended up getting a patch out so that was that was reasonable too it's like within a 90-day time frame so that was my my experience with them Citrix has a very good product security team in terms of the communication yeah they were they were great you know they're very responsive when I would send information or request stuff and however something we do run into every now and then as a vendor you know they're they're trying to be proactive and patch as early as possible but citrus in this case actually patched early and didn't let me know so we ended up releasing our advisories before the 90-day deadline

not that happens quite often oh yeah that's good good question there we try to take a balance between prevalence and kind of how maybe vulnerable the target might seem so for example like if you want to find like a Microsoft Bowl and that's really prevalent and can get really good impact if you fix get it resolved and fixed and report it but also at the same time too there's also low-hanging fruit out there that's a security threat to us right so it's also finding things that just seem less secure like I've looked at a few routers that don't have the best security that's another reason to that we choose our target so it really to

answer your question it's a balance between those two concepts so I don't know if you guys want to chime if you have your reasons you know I would just add that you know for me it's more fun when I figure out what I want to do instead of someone telling me so like we do have this list of like targets that I don't know whenever I look at that thing I'm like I don't want to do any of that and then I just go on Amazon and like you know find something interesting so like that's it there's a motivation factor there to pick something that you're interested in you know that you really could see yourself looking at for

like hey you know at least a couple weeks maybe a month or something just whatever can keep you motivated and pushing through it because you always hit those roadblocks and it's good to help you push past them I'd like to add in real quick so like you said we do have a dream list where it's like you know people in the company say hey I want you to look at these targets but some of them are you know really reaching for the stars so like we we really do try to look at targets that are meaningful like I said like IOT SCADA something you know if you find security falls it's going to make a

difference right so yeah exactly exactly we don't we won't look at Joe schmo's FTP server yeah

it depends yeah usually it's policy they uh it's called CNA so we're at CNA so if the if the yeah so we apply rel but in other cases they might be one and so we leave it to then this to fill the block yeah yes taking a look at some of them I can't talk too much about it at the moment I don't think there's anything that's currently in disclosure right now but yeah and a lot of them are hit and miss to like sometimes you look at and just give up after a couple weeks like yeah now this is not it's not worth the time to to like take a chip off and dump for

him we're on it that way so there's a lot of pick and choose in there how much can we talk about Jimmy would probably tell you a story like there are there are definitely vendors that are just you just scratch your head like ya know what's going on over here like they involved can't involve legal and yeah I mean like rare though it is rare yeah but it does does occasionally happen yeah we've definitely had run-ins with unhappy vendors but no law enforcement yet

[Music] I would say we all have a broad skill set but we definitely have our own wheelhouses too like I I come from a development background so Web Apps are there my wheelhouse but I definitely you know I work hard to get good at reversing like they do you know yeah we have a great assortment of skills I think and like he said there's a lot of overlap of course but like we found that some people like really focus on hardware and like I'm not I don't pride myself in hardware and so they do more of that and then others like are more like you know reversing or web app so it's it's a overlap but also definitely

specialty when you get down to the details yeah I got I mean I have one like on that I found and what it was weird the I won't mention the company but I found one of the internal the devs already found it but they yeah they found it but didn't talk about it and so by the time I reported they said oh yeah no we had to see me for this whole time but it was like not listed or anything so that's that's happened to us before I'm sure you have some experiences doing we all do yeah or like related vulnerabilities to like you know we've reported some for that the product has actually you know the vendor has a

security team and they take it and run it and find like two other things and we're just sitting there like how do we miss all those there's a great white paper you can read on this that talks about the overlap of zero days and a likelihood like I think it's like what about zero days thousand nights I think it was put out by Rand I think and it can it goes she goes over the likelihood of like when you're finding us here today how lucky it is to collide with another and the study was actually the shelf life for whatnot is actually quite quite long so yeah and actually to touch back on that our team was kind of born

out of that mantra you know like if we find bugs someone else will - right

yes oh and there's there's a couple other ones that we haven't really published yet that that exact thing has happened like firmware decryption keys for example where it's literally like you realize that they use it across their entire product lines so it's definitely things happen like that for sure

yeah I'd say for me personally I tend to stick more with reversing and not so much automation and tools I think it goes back like one of the themes that happen to happen we talk about our case studies I think we'd all agree they were all logic blogs for the most part and I think if you tooling for some of those might be I mean of course like scanning for code but I guess I'm more talking about buzzing in that case but I I personally like reversing and taking it apart and actually spotting it when I can but I'm thinking you know others like tools or all right to be honest I don't use any tools really when I'm

looking at if I have the source code and the no tooling I just have patterns that I'm searching for you know it's just being familiar with that language and you know what a vulnerability might look like we fuzzers definitely have their place and automated tooling you know when when you're looking at something very complex but for the most part I'm I like doing a manual

yeah yeah that's that's always why I try to mixture I don't move on till I find something yeah that's that's painful I think I think it's happened actually yeah actually did happen to me recently somewhat and yeah it happened some you know that's why sometimes the code that you just the reality is sometimes some of these programs are so big and wide you had time to look at one particular part and you just gotta say you know this is the part I exhausted and I moved on maybe someone else we've got another part and found it so part of is just just the nature nature of it and it happens but it's nothing to nothing to

beat yourself up over I had that specifically happen I was looking at the Verizon FiOS router and I found some bugs in the web interface because like you know that's my wheelhouse right but then a couple months later another research firm found like remote code execution in the UPnP service so you know he looked at a different component found a better bug than I did but you know it happens for sure go ahead I can answer that so yeah for the beginning of this team we were allowed to collect bug bounties and it was kind of nice we could we could submit to zdi and just take that money too and I think somehow our CEO caught wind of it and

shut that down but individual bounties we can still collect them we just can't submit them now to zdi so yeah well we try not to let it affect target selection I think it's like general fixing little disclosing bugs like you know where to fix them you know going through a disclosure process like this so really making making devices and more safe and as well as you know updating our own scanners so that we can find more things and also just awareness and building more a bridge between other researchers our stuff in tenable so

we don't but the the guys that do work on necess plugins they will occasionally get reports from customers and stuff saying hey your scan broke our stuff because there might be a bug here you know so they find bugs like that sometimes yeah that's that's pretty rare I don't think that's really happened too often maybe once or twice and the other thing is like you know we we also Red Team kind of like our own product stuff that we use I don't know if that's right but like slack for instance and zoom we use that in our company and so that was one of the reasons why David picked it up it was on our list of stuff to use

like our company uses it let's see how safe it is so any more questions sure

yeah I'll tell you the truth I do have sort of my own static analysis tool but it's basically just a big like so for each language I'll have patterns that I like to look for you know so I have a PHP pattern list I have a Perl pattern list and so yeah I do compile my own static analysis yeah pattern sets but it's it's not much beyond that you know for the most part I find I find bugs you know really through reading the code and just making sense of it like like for that header one you know that's that's going to be kind of tough to define with a pattern you know so so kind of look around the the

authentication mechanisms and trace the input into you know the the patterns you're looking for and that's it's already that answer your question

so I'll start with I'm taught a sink right I'll start with a vulnerable function say I find a system call right so I'll go to that system call and then in my pattern I'll have a check to see if it's processing a variable and if it's browsing pressing that variable then it that's of interest to me I'm gonna look at it check the code above it to see how that variable gets processed and if it comes from someone trusted source of input like a get parameter or whatever then you know I'm gonna hammer that hard and really look into it so I start with that that pattern and then read the code from there excellent so

thanks thanks guys for coming we're open thank you and we're we're open for more questions catch us in the hallway whatever you see it's in the con and we'll be happy to talk more thanks