Clean Forensics: Analyzing Network Traffic of Vacuum Bots

rights our next speaker is Quran duvetti Quran is a manager or technical lead and the security team at Google and his talk is titled clean forensics analyzing Network traffic of vacuum Bots please give me a warm welcome [Applause] thank you love the energy here it's still like 3 30 p.m and I was like I'm the last talk so I need to make it interesting um thanks everybody especially uh the organizers uh Matt uh Punk quarter everybody thanks for having me here and lovely audience all right so I'm gonna talk uh today about vacuum Bots before I actually you know dive into the technical content show of hands how many of you have a vacuum board or want to

purchase one or if curious about them oh quite a few hands all right how many of you think they are secure no hands all right I like the crowd so a little bit introvert me uh I'm Karan you can call me k um I'm a security manager at Google uh I'm an infosec alumni from Carnegie Mellon I spent two wonderful years there um for fun I write blogs at all thingspawn.com I mostly write about security engineering interviews try to help people land jobs I also spend time reviewing Journal papers so some of the research stuff and it's very exciting to see what's happening in the field and for fun I love to travel I'm here I

just flew in last night and I loved going to the beach so with that let's look at some of the things we'll discuss today right so we want to get started with why vacuum Bots like why are we even talking about that right we talk about iot a lot which is understandable but why specifically vacuum watch with that understanding we'll go into some of the functionality right how do they work what features do they have and you know some of the missing things that are not so popular then we'll dive into the interesting parts right we'll talk about how every one of you in this room can conduct Network forensics on your own right it's

not rocket science it's very very interesting and then the most interesting part I would say um or maybe not uh mostly secured in privacy issues that I discovered in some of the Bots and then for fun I'll also show you the reported issues so I actually talked to one of the vendors no names will be taken in this room I know this is being recorded and will remain on the internet for a while so but I'll show you the communication and I'll show you the challenges of when you report issues what really happens right and then I'll give you time for Q a if I see the audience engaged I'll end early I promise all right so with that and obviously uh

the typical disclaimer this is my own research Google has nothing to do with this right so you can blame me all right so let's talk about why research vacuum bonds and I think if I just leave this slide and just let it be it speaks volumes but I'll still speak um so if you see this trend this data is taken from grandviewresource.com um some credits to them they have shown that over a period of time since 2016 at least on the slide we see an uptick in the number of bots being sold right year over year it is projected to grow by 23 percent every year by 2027 the numbers is expected to be 60.9 million units these are Bots

running everywhere it's like an army of parts some of the major players uh let's look at the market right let's see who was actually capturing this market so I've listed on some names on the left so you'll see uh these manufacturers but it's not really Consolidated right no single manufacturer is dominating the market but also not it's not also fragmented right so it's kind of like there are territories and then people have actually got their territories right and they marked their own territories um it's kind of interesting and we'll see why so with that with that intro let's look at some of the functionality right what can you do when you go and buy one of

these so the most common and obvious thing is cleaning right but it's not just that you can have Auto so you can just tell about hey go clean and it's supposed to figure out like the area it's supposed to clean how fast and how much time and all that on its own you can tell it to clean its edges so it can say in this room go around the edges right just the edges you can also say hey there's a spot that's dirty clean that spot or you can just say hey I'm going to manually control you and you can control the button through a path you want you can also set up schedules and this

is interesting right you can say hey every Sunday do not bother me 7am start cleaning right and yeah it will bother you because it makes a lot of noise at least some of them do um some of the fancy Bots have a health monitoring feature so you can in their app you can actually see how much you know life span you have left for the brushes right how long you have to replace them stuff like that here's a here's an example right so I've taken this in an undisclosed location here's a bot that's trying to map an area that I was trying to clean you'll see the bot is the dot generally and someone has a zzz probably sleeping

and it's actually mapped this area on its own right so the blue lines the solid blue lines that you see it has gone around and figured it out the white lines right sort of gone through it has followed that path right it's a combination of from what it seems like it's a combination of uh breadth first and then depth first algorithm right that's what I saw from the lines but anyway some of the Bots also give you um features for multi-floor mapping so let's say you have a house that has multiple stories you can take the bot at any story and it will map each floor and sometimes the auto detect Which floor they're on pretty fascinating right

um only few of them at least in my research gave you the ability to view a live stream so technically the bot is looking around that makes sense all right so with that understanding of functionality let's look at when you when you buy one of these parts right off the shelf how do you set it up what happens so in this example I've given some of the you know basic components you have a phone right you install the app generally every manufacturer has an app you can just click and install um there's a bot and you have a modem connecting Ur home to the internet so the the border that you see on the left the black one is your home network

and anything outside is external so step one you install the app you create an account right you enter your basic credentials your name email stuff like that and you say hey create an account for me that's what happens in The Next Step you turn on the bot right and the app generally tries to search for that bot right for that manufacturer it will say hey are you around is anybody around that I know of and it's kind of interesting how they have made it work so it's not as simple as hey bot connect to my home network directly that's not what I saw what happened the app will do a search and the bot will start up a Wi-Fi

network of its own and it's going to broadcast it out so that's another Network apart from your home network and it's going to say hey connect to me I'm here what happens there is your phone or that app connects to this wi-fi network and it exchanges your credentials off the home network so it's a way to securely exchange credentials that's really what's happening in these three arrows and after that once it's can talk outside from your home it can say look I know this person this account I'm in this house or this this person owns the bot right let's associate myself with this account right so that's the call it makes finally and now you have a functioning part in

your house house make sense I see no it's cool all right so that's very basic of you know how the Bots work how you can set it up but let's talk about how you can set up your own home network right and this is something you guys can do or all of you can do you go back so let's say you want to look at the traffic right you want to examine what is the bot really sending out of your house do we know like do we know this yet I didn't right so I to to show the point I actually took a picture in my apartment and this is how it looked like don't be scared of

the wires I'll simplify them in the next diagram but the point here is it is very very inexpensive for anybody to run the setup so case in point I was able to do it in around 150 ish right and this includes my modem right so can be anything I had a basic switch in the middle that you see there um it just needs to have Port mirroring which is actually a very common feature these days and then I also have an internal router and I'll talk about why and then a bunch of cables you can buy them all for 10 bucks not that expensive and at the bottom I had you know a Mac and a phone to test this out so the

phone had the app and the Mac was used to look at the traffic makes sense all right so here's the icon view of that diagram right and this makes it a little bit simpler I hope to understand what's Happening Here so we have the bot right and we have the cable modem on the right so let's start from the right go to the left so it's easier to understand so from the right the cable modem talks outside your home network that's your point for any communication inside the home to outside simple enough the app or the your mobile phone is connected to that Wi-Fi network it's not connected to the cable right it's through the Wi-Fi network of the cable

modem and it's talking out so you don't see anything this site from the phone right it's just talking now the switch is connected to both the internal router and the external modem right it's kind of in the middle and that's for a reason because it has a mirror port and what I've done is I've connected my machine or the Mac to the mirror port okay the bot is connected to the Wi-Fi SSID of the internal router which is then connected to the switch so now if you think about this setup for a second I know that's a lot of words but if you think about communication of the bot right let's say the bot says oh

associate account or I'm here that packet is going to go through the router to the switch to the modem out but it will also go through the mirror port when the response come back it will go straight back but it will also go down because it's mirroring all the traffic and that's how you can set up and listen to all the things that are happening through the bottle right that's the reason of using the internal router and the switch simple enough right any questions here no all right cool now let's get into a little bit more fun part right what did we see or what did I see for some of the examples that I ran

this through so let me explain like what I tested first so with this setup right I tested several functionalities we just talked about so what happens when you establish you know when you set up the connection when you set up the bot that's case one what happens when you say oh start cleaning what happens when you say move the bond what happens when you say start capturing video I don't know but that's the setup will enable us to check that data and then I conducted analysis of okay I see you know I see this traffic going on what's the protocol can I look at it can I examine it and why and why and why right

so here's an example I have not named particular bots of vendors just to make this talk a little bit more generic right um but I've seen a lot of patterns and I'm going to stress on the patterns here okay so the most common thing I saw is a lot of bots are configured to talk to a cloud service that makes sense right they need to talk externally why not a cloud service and the first thing to do is hey I want to talk to this domain I need to know the IP the only way to for them to know that is DNS so we clearly see a DNS response coming back and I know the Wireshark screenshot

is really really big and I tried to blank things out you know but essentially this exchange is saying hey the first packet out is DNS tell me you know a particular domain that I belong to so cloudservice.manufacture.com right it can be hosted anywhere and I want to talk to that IP right the next thing I saw was once you get that IP the bot established a TCP connection so now things are getting real right so I didn't put Wireshark captures on purpose here because there was a lot of data and I try to abstract out some of the stuff that was interesting and I know people are shaking heads does anybody recognize what protocol that

looks like on the right looks familiar I'm hearing mumbling but a little bit louder so I can hear and it's fine to be wrong it's totally okay yes close enough and what protocols use XML something over the network or something like an example whoever is behind thank you eggs yeah correct xmpv exactly right so they are actually using xmpp in this case is xmpp secure it's plain text so no right but we'll get to that right but it's very clear when I when I say on the app turn right what really happens is you see that IDE and you see a particular atom feed it's like a published subscribe model right so it's like chat Bots

a lot of these Bots are chat Bots they're just talking as a chat app right and you say Hey you see the action spin right I don't know what TD is maybe it's a category of command no that's that's how they have implemented it but it says move right and then I get back a response that says result and a type but it doesn't say anything it's probably saying success right so this is what I saw on the network when I hit one tab on the app and it's very fascinating you can map every feature and you see what's happening on the network all right similarly I said start cleaning or stop cleaning same stuff

just for a little bit more information so if you look at it it says clean and there's also extra information like speed there's also a clean report coming back right so all of this implementation can be reversed engineered just by looking at the traffic simple enough right there's also keep alive so after you're done cleaning and you're happily you know moving on to your work the bot is still talking out maybe it the Cloud Server might say hey bot are you alive and it should say yeah and this happens in your house without you knowing it's not malicious but it happens right yeah good question right how often is the key for life is a question it it depends on

the implementation the bot or the particular vendor and as I said I'm trying to abstract it out it can be anywhere from a few minutes to an hour it depends generally when I saw Bots cleaning it was more often just to see like you know things are on track it's not stuck it can communicate but like when it stopped and like charging maybe less right but this was clearly seen on the network so and there's a clear giveaway because there says exempt right so you can see the protocols all pretty clearly here authentication we talked about security right this this when I saw this um I stopped looking at the traffic for at least 10 minutes

there was clearly a plain text password being used in XMP before Authentication didn't expect that but you know it's it's it's also sometimes not surprising because the functionality you know maybe people just want to implement it and get it off the ground and you know get in the market I don't know how the development life cycle was but anyway we saw it there I removed it from from this slide because you know the essence is pretty clear here and then uh it comes back from the server as an acknowledgment the protocol actually used here which was the interesting part was sasm and everybody here can look it up and read about it it offers different types of

authentication so you can do a password you can do sorry you can do a plain text password you can also have a better version of the same thing but in this case it was just plain password so in another set of bots so there were some of them involved but in another set of bots there was a better version of of this authentication something excuse me something called as secure mqtt mqtt like for those who don't know is also a machine to machine protocol just like you know it has a published subscribe model so the same stuff on the ground but there's a secure version of the same protocol I saw the port being used as 8883

hmm okay their status and this was really unexpected and I'll I'll talk about this why so you have you have sometimes all of us look at phones right so we're looking at our phones like this literally like this and you can open an app and you'll be like okay let me open that we don't think about it but when I opened this particular app I actually saw traffic and I was like wait I didn't do anything what's happening turns out the bot is trying to get the time it's trying to get the time which is correct for itself and the weight was doing that was by getting the time from the phone and the way it was clear to me is I had

to reverse engineer it a little bit but if you see the bolded line time T equals something right some big number that's an Epoch time right so you can convert it into the current time which I did here right and you can see when I was testing this um and there's also a TZ it's a time zone I was in California at the time so TZ is minus seven UTC makes perfect sense but is this good is there any issue with this you think so why location yeah it does reveal some of the locations thank you exactly and we'll talk about that in one of the issues that I actually ended up reporting correct so we saw this unexpected right I didn't

do any action on my phone but I saw this so with that a little bit of the shock I'm seeing in this room right now let's talk a little bit about the security and privacy issues that I actually you know collected and then I'll show you the reporting part as well so can these Bots be hijacked or controlled that was my obvious question when I did the research I'm seeing a lot of traffic on the network I'm seeing a plain text password can somebody just come in and say it's not your Bot anymore it's not it's mine and it can be somebody walking into my apartment right so there are two parts to this one

is remote can somebody do it remote they don't have to be a local attacker on my home network right this is hard in my experience when I thought about it I looked at the traffic it's a little bit hard for you to get into the home network you'll have to compromise it but an easier way is just get control of the app the app does not have any two Factor it just has a username and password so you can set your password as clean me and really that's the only control you have or security you have quote unquote uh to get control of any device through control through that app that was fascinating to me I'm like huh

now I need to worry about the security of apps on my phone so you see how the thread model is moving from the iot device to the phones and that's why it becomes complicated and we talked about the local one right so you can replay the commands we saw play index commands right as long as there is no nonce value that's saying okay this is a new command and it expires after like 30 seconds you can replay those commands back over the network and the bot will do what you say what about floor plans it's also moving around just like you it's just another human because it can also look at live video what about the privacy of floor plans so

this is interesting right sometimes I've seen and this varies so I'm not going to generalize my statements here but it really depends on where the floor plans are stored so they can be stored on your phone maybe okay but they can also be stored in the cloud associated with your account again on the network I did not explicitly find traffic that looked like a floor plan it's very hard it's very challenging and I'm being very transparent about this because I'm not saying oh this method is like complete in any way but it gives you a lot of information so I didn't find it if you do first anything else please talk to me or best to talk to the vendors

course location somebody correctly pointed out uh the location issue so yes it's not precise by the way because we still have a time zone uh I would say an area of the time zone but let's say I taking this phone I have I have an app I have Bots I travel to I don't know Zurich Switzerland for example right and I have a bot in California the moment the time zone changes from this and I open the app my bot is gonna think it's somewhere else it's it's in UTC UTC time zone of Zurich right and it's going to mess up the schedule of the bot so instead of starting at 6am or 7 am on

a Sunday it will be off by seven hours and I tested this accidentally I didn't even know about it and it suddenly started one day and I was just testing my time zones I just changed the time and so sometimes you discover things by accident as well but it can be proven if you look at the traffic so it's all visible at the end uh this was interesting because you can mess up schedules and that can have different human effects people might be pissed off video recordings right um this was interesting I wasn't very scared about it after I saw the data but like for people who spend a lot of time in what shark

if you see the left screenshot the screenshot on the left you see almost equal sized packets being exchanged over UDP you know sounds like something that's not reliable like a video stream or an audio stream right so this was a clear giveaway okay I clicked start video and I see this boom boom on and it's just really quick because it has a lot of data right when I examined the packet I said hey give me the details I luckily did not see it being a sum protocol that I recognized I couldn't decode it per se but I did find some metadata right and that's the part I've highlighted for everybody here so if if it's too small I'll read it right it

says message type start streaming so you still get a sense of when the user started looking at the video it will also give you a session ID I don't know what you can use it for that for for what reason but that's still data and give me more data I'm happy right so there is some metadata being leaked that is you're not completely aware except you know the stream is just fine at least from what I saw all right so we saw we saw the functionality we saw how to set it up for yourself we saw some of the issues now this is this is the most uh I don't know fear slash interesting uh

conversation so I'm going to show you a bunch of emails and that's an exchange with a vendor the vendor names or anything else that gives away the name is blanked out so don't worry but it's interesting nonetheless okay so that's me and the part that's highlighted is the date and time so I want everybody here to watch the date and time okay don't worry about reading it I just put it as a proof that I'm not bluffing everybody here this actually happened right I can show you on my phone later without blanking out anyway but um essentially If You observe the date and time it is November 18 2019. so I've been doing this research for about good two years

now and that was the first time I reported so I said hey by the way hello XYZ support here's what I found and I had to write a long one because if you write a small one they don't think you're serious so I completely write down my research and said hey this is what I found so I told them here's the background right I told them issues one by one so I said hey by the way I see insecure protocols I see plain text passwords I see potential man the Millet attack on the local network right and that's all the text here I also see course location tracking right and for this one I was very

specific because I showed them what I captured because they won't believe me otherwise right anyway um it was like explaining everything that they knew but they already knew hopefully but anyway and then I finally you know was also transparent about the privacy of floor plans again I remember I didn't test this completely or thoroughly and I actually wrote that but I wanted them to investigate and like look at it and say is this okay are we doing good or can we change something who in this room can guess what happened next [Applause] who remembers the timeline what date or what month did I send this it was four year four slides ago November 2019. okay cool everybody's

everybody's awake awesome I have to do these pop quizzes all right uh guesses for what happened next random guesses they ignored it reported you to the police somebody reported me to the fullest I don't confidently say on with the recording on that I'm clean so I'm I'm still here I'm still here giving a talk about the same thing so either the you know I don't want to comment any further any other guesses before I hit next

later to the day A year later today awesome [Music] they ignored it so whoever said ignored it was right um I actually closed the ticket so they went one step further if you see the date if you see the date it's not that far so I I would have to give give it up to them for for the promptness but if you see this part

I didn't click any of those links by the way what do you think happens next this time it's probably on me right well maybe on them I don't know public disclosure good good thinking you're vacuum is seriously stopped working I would question my if so the audience said uh one person said my vacuum's not working I would be very concerned at that point and I would re-examine my methods again if I miss something because I definitely miss something but really good call all right so what happened was you know I obviously followed up and I said hey if you look at the time again it's a little bit on that side because email threads right

um if you see the time that kept keeps me awake so 12 21. I said request to reopen the ticket number X because I've not received an official reply to blah blah blah right and I also somebody said responsible disclosure extremely good call because that's I reminded them of that there is let's think about this for a second a bit more seriously we just talked about the market we just talked about it's a multi-billion dollar industry there are millions of units out there we should take this stuff seriously I don't mean to talk anything negative about this vendor or anything else it's all for fun right but in general right I would expect prompt response dialogue

all of that what happened next close again that's a good call fix the issue and gave you some bug Bounty money at least at least the latter part I don't know about the first part right maybe the issue is fixed uh we'll see we'll see in Fallen flights but I didn't receive any conversation yet so after a while after a while and I say why because see the date I got an acknowledgment a good one because I was following up I again this is all for humor right I don't mean to demean any vendor here right just just so people know but the timeline is pretty critical right so at after close to a month right

you get a knack and I and I have some explanation this is good this is actually good I I wanted something like this but look at the part that's red it says at the end it says yes we implemented using xmpp which some of you folks kind of recognized through the slides says which is an older model I don't know what's old and what's new because I just bought the bot and I wanted it to work the way it's supposed to work all our new models now use more secure technology to ensure user information security what do you mean by what do you mean by use more secure technology to ensure I I don't know I I had no idea right but

that got me worried because what about the old models what about people who bought the old model are and don't know about these right I hope people can see the next slide but what happened next no guesses tired of guessing huh sorry what was that close the ticket um not completely I'm trying to recall exactly yes they may have closed it but I have to check my email thread again um I wasn't satisfied with this to be honest I love the response I like the came back something substantial came back right not just close the ticket but I personally wasn't happy and I gave them time so I really wanted to be responsible and adhere to the

responsible disclosure time frame so I said look 90 day 120 day whatever you want I'm cool with that I mean today it's two years almost so three months later I followed up but then I got a response something like this right and look at the time February now we are in February we started off in November so somebody said Thank you right somebody said they thanked you that's also correct because they did right and they passed on the team back passed on my notes around hey I'm not happy with this we need to CV a sign we need a cwe something that shows this is acknowledged right not just that email I'm passing this note blah blah blah I

was fine I said okay I'm cool no worries right what happened next

I bought a new one I was buying Bots sure [Music] sorry nothing nothing close enough all right so nothing really happened which is right actually uh so you see me following up on March 30th so just think about this I'm not the first person who bought this bot right I maybe have conducting This research that's another story but I'm working with them right now I'm not like against anybody I'm working with them I'm reporting responsibly trying to understand and help out but we are at March 30th and I had to follow up a little bit more strictly right and say look is there a response do we have something assigned is somebody working on this if

so I'm fine I'll back off I I have no issues right but I want something to be done and thank you sure find final takes on what happened next because that's the final guess I won't ask you guys to guess any further they tried to run me over oh my vacuum wow I'm really scared of that guy now

so I again I I didn't do any public disclosure I didn't I also have blanked out their names right um I've done this on my own no employers involved I was very honest and transparent and I reported it for good reason and like with good faith right and even here I'm not talking bad just just to be very clear I'm not saying in a negative manner I'm just saying this happens with vendors and sometimes it's some of the priorities right it's one person reporting something like it it really matters how you are responding and prioritizing so I'm not saying it's all of their fault but like for some security issues we should be more cognizant and that's really the message

right nothing really happened so I had actually pulled back and I said hey it's been like a lot of time in seven six seven months I actually started writing more emails um seven months later I did get response that says yes right and the the part that I want everybody to focus is the red part right so there's obviously a date that's July right and there's obviously escalations happening so look I'm not I'm okay with the process but I'm not okay with no work being done so it got escalated I would I would put it in a positive way the the future Bots with the unknown vendor here they were better right A lot of them did not have those

issues I was happy at the end I mean it was an old one I understand but it's an interesting case study you can pick one up and test this out yourself right and I didn't follow up after this it was way too long it was kind of old several years old I don't know how many people use it but the objective is today's awareness and not just raise awareness you don't have to attend a talk to know about this you can do it yourself so in conclusion I promise to end early so I'll stick to that promise um they might use vacuum boards like other iot devices this again this is nothing to do with specific vacuum watch

that can this can apply to your fridge you know cameras anything right by anybody they may use insecure protocols they may use insecure configurations it's also very challenging for the end user to get guarantees of privacy and security I understand their pain points I get it that's why I'm not complaining too much about it I'm just saying hey we can do a better job the DIY method I described really that's one of the main takeaways of this talk is you can replicate this do-it-yourself talk to people raise concerns I think one way to really get the industry moving is more people getting involved and saying hey we should fix this now right and finally again not finger

pointing but it's in general challenging to get a response and also a response and a fix it's hard to fix these devices they're everywhere like how do you tell how do you follow up with a customer and say hey like for that bot that you have you're not going to give you a new one it's hard to do logistically right it happens with cars there are recalls but like for Bots I've not seen them yet with that thank you everybody for listening thanks for keeping it clean and you can reach me at my website for yeah I have my LinkedIn Twitter everything thanks all [Applause] right any questions I do want to allow time for questions yes

were they firm or available over the air so the question was were any of the Bots firmware updated over there yes uh there are some and actually that helps a lot what I didn't do is retest the same thing so it's definitely possible to your point and that's why I was not very negative or like trying to criticize anybody here some of them may have been fixed maybe I didn't hear back a lot of now vendors are following this method where they say oh look update available click update and you'll be fine um I just don't know whether each update has a security patch or they are just functionality improvements or bug fixes that should be another

topic over the year updates that was a question I that's an interesting thank you I think I'll think about that I haven't done it yet but we could do that sober were they all plain text uh that's a good question yes a lot of them uh the question is I'll repeat for the recording did any of the Bots use SSL um I cannot say that specifically SSL but they were secure protocols in place for a lot of them so there there were times when I actually got a bot and I saw nothing on the network and it's very sad as a security researcher but that's a good thing right after that sadness passes away you're like good job

so yes yeah so I I couldn't see that some of the slides curly enough for those packets UDP packets the slides which sorry I'll take you back sorry about that was there a lot of UDP that's the one yeah those are all UDP packets yes those are not even in fact it's kind of small that's why I was talking about it yes those are all UDP packets yeah I would be concerned if there were TCP in using video that's a lot of bandwidth and tissues anyway any final questions going once going twice sold [Applause]

thank you for that talk it was super interesting I was