Alexa and Cortana in Windowsland: Hacking an Innovative Partnership

[Applause] Cheers

okay let's get started my name is amica Schumann this is you Val Ron and we're here to talk about Alexa and Cortana in Windows length so it's a great opportunity to show how we take a partnership between two companies and then use it to hack into computers Wow and we'll use this opportunity to show you some more cool tricks on how to take over locked Windows machine using just your voice at system so as I said my name is Amica Schumann eval Ron is with me the homework was done with Professor LED m from the Technion and what we'll do today would give a quick introduction into Cortana give you some context about the research work that

we've been doing for the past two years talk a little bit about the previous results from this research and then concentrate on the new result which is mostly about the partnership between voice assistant and then some more nice cool tricks with Cortana even Cortana on Android because this is not confined to just Windows machines I will have to talk a little bit about the whole responsible disclosure and the engagement with Microsoft around publishing vulnerabilities and hopefully we have some useful conclusions for everyone who thinking of building new systems so without further ado let's talk about voice assistance those are Cortana and Alexa and Siri and Google assistant and we call them voice assistant but the reality is that these

are intent resolution systems their whole purpose is to take human intent expressed in natural language or some type of human interaction and translate it into computer actions like browsing the web or launching program or getting some data for you now the thing with this assistant is they emphasize the hands-free operation okay so you can do that without going and interacting directly with your computer and that implies that they have to operate over what we call the locked screen so just to give you some context about the work we've done here we started two years ago and we wanted to evaluate the effect of that hands-free that above locked screen operation of Cortana were able to

disclose 17 vulnerabilities and report them to Microsoft but we only got two CVEs yes itself why is that okay now this is troubling actually this is troubling because it turns out that software vendors only register cv if they have if they are forced to deliver a customer patch if they can fix the vulnerability in their own servers in the cloud they never register cv they never report the vulnerability and it just fades away which means that as a customer as an organization you can't even ask the question was I affected by this vulnerability I think this has to be changed in the future and we got some nice bug bounty Thank You Microsoft so here is Cortana we all know that we have

a Cortana client on our machine it's a fat client I'll talk about it but most of the heavy lifting is done in the Cortana service in the cloud actually the client takes the digital audio sends it to the Cortana service were sent to a speech-to-text engine text goes back to the client just for display purposes and the client says okay I get the text now resolve that for me and it goes back to the cloud goes to a text to intent engine and it's being transformed into a command with parameters now based on the specific command this Cortana service looks for the proper plugin the plot the proper skill to execute that specific command execution is usually through some

third-party web services search engine other applications and the response is then packed into a JSON structure which is called the card data which is then translated by the Cortana Service to HTML and JavaScript that goes up until the client for merely rendering a quick example I'll be asking who is George Washington for example voice goes the speech text text is going back just for display purposes and then resolve that for me text goes to the text to intent engine and the command is search parameter is George Washington now there is a plugin there's a skill within Cortana service that goes to the Bing search engine okay it takes the query George Washington goes to the web finds

the answer packs it as a JavaScript and HTML page and then it goes for rendering if we take a deeper look into how this looks wouldn't ii as you can see most of the processing is done on the cloud there's no decision made by the Cortana client on your computer there are two phases of processing in the cloud one is audio processing the other is semantic processing that's the more important there's a lot of machine learning involved in in this process of Cortana it's being used for improving the speech recognition which I don't really care about more importantly it is being used to extend the intent resolution capabilities and we'll see how that affects security okay this is how the

JavaScript looks like this is a specific use for launching a browser that goes to a specific song this is how it looks we talked about the plugins the skills most of them are written by Microsoft but this framework or third parties to add their own skills it's basically an azure boat that you write you deploy it in the cloud you connect it to a Cortana channel and it works there is of course a validation process with Microsoft it's very easy to bypass or you can actually put anything you want into a skill interaction of the skill with the actual machine is limited to card data okay it's very limited not to say that we couldn't hack our way

through it so our journey into this starts with the announcement in 2016 that Cortana will now work on your lockscreen and because it sounds kind of dangerous right Microsoft says well for sensitive tasks or those that launch applications Cortana will prompt you to unlock your device so they thought about security and because it's now dead secure we can have it run by default on lock screen okay ah mmm that's interesting we have a Fed client on your machine that can basically do anything through a very powerful JavaScript API it has no brains it makes no decision but itself and it's available on lock screen once you say hey Cortana what could possibly go wrong with that so very soon after we

started the research we found the first of all nur ability we called the divorce of Esau and and these are all attacks that are classified as evil made they require some temporary physical access of an attacker to a locked windows machine first month allows the attacker to launch over locked screen and Explorer browser into an unsafe server which means that you can quickly translate this into a man-in-the-middle attack and then download the exploits and compromise the browser in the entire machine it's a bit complicated okay so we have denied the second one which is Open Sesame that allows you to actually open the equivalent of a command line on locked screen with just a few words and

keystrokes I'll show similar examples soon scale of death I talked about third party skills we showed how someone can deploy a skill to the Cortana service the third party skill and use that skill over locked screen to take over a machine and there were a couple of others found McAfee as well we talked about being able to execute arbitrary code over lock screen so here's another example of a vulnerability that we found out over time and interesting in enough the same day that they announced Cortana over locked screen they announced a cool skill called the reminder skill so you can actually go to your machine and say hey Cortana set up a reminder of course

do it over lock screen and insert a new reminder into your library for some reason for some reason when you create a reminder you can add a photo to it not only text so here here's how it goes I invoke Cortana and I ask to set up a reminder so far so good you see the ad the photo button out there now when I try this functionality I can pick up from a library which is a different name to say it's my D so yeah this family is not that happy anymore so I can go over there photos which is cool and nice at the same time I'm inserting a drive with USB stick into computer I don't but it's

convenient because I can now take all the photos and download into the drive and you're probably asking yourself why just photos you know it just clearly no reason why I should go for just photos I just look for any file on the computer yeah they have their passwords there Shane ah but it's not only data access it turns out that this winder this is an actual command line okay so I can execute stuff from the USB stick I just plugged into the machine I can go and download load file in it and execute it and you know by now it's game over okay it's a length demo okay it could happen in 20 seconds okay so again complete takeover over

lock machine by using a proton scale and the fact that I can open this Explorer window over lock screen Microsoft were very fast to fix that in the cloud they had a patch in no less than two months and what they did they remove and photo button over locked screen so if you do this when computer is not locked you get that button if you're doing it on a locked screen you don't get that button functionality of having a command line executed from comment from Cortana is still there okay remember so that was cool but here is something cooler that can interact with each other so we can stop imagining because this already happened and we

found a very cool vulnerability that we call Alex and personally I really like this slide but we will have to move to the next one

it's Lee it just did [Music]

show me one way okay cool so lesson one and what is this partnership all about well it's a collaboration between and Microsoft that allows a Cortana users to open Alexa on Windows 10 Alexa to open Cortana on their echo devices they simply say hey Cortana open Alexa and then the Alexa skill is running on a Windows 10 and Alexa users can say Alexa open Cortana so the idea behind this partnership is that now users can get the best of both worlds it means that Cortana users can now have access to more than 50 thousand skills of Alexa and Alexa users can now use some of Cortana skills that interact with the out Lucas other office products

but in terms of security they get the worst of both worlds because if we know already that Cortana is not perfect then also Alexa is vulnerable to for example little shells for my chat mods I've shown last year that they are able to develop a malicious skill with unlimited recording time session which can play like an eavesdropper in your home so if we have to assistant very unique very powerful that are working together what could possibly go wrong and apparently a lot the first problem that we found is when you are not signed in into Alexa then it will require you to login into your Amazon account which is very illogical but the way that they

implemented this login mechanism is by opening a browser and Internet Explorer browser on your lock screen which the meaning of such a thing is that the attacker can now navigate to his own website maybe download is exploit and also it can login into the Facebook account or other social accounts using the cash for themselves so let's see the demo here we are asking katana to open Alexa and we get this window which asked us to sign in to Amazon but if we do not log in whether clicking on this link we see that this is simply a browser on your lock screen a customized Internet Explorer and well in the Amazon website we can search for items and our goal now

is to escape from the Amazon website you see that we do not have a URL bar and above so we need adult weeks here we get into the facebook login and we can see that if the browser has saved the cached credentials of the user then we will be able to hack it just a second here we see that in this button if we click then it's already saved the credentials from the previous session and here we are on the Facebook of the victim device so now I want to ask you something until now we saw how you can steal data how you can open a browser but what a kid what heckles love the most

what money okay so how can we abuse this partnership in order to steal money from the lock device well known without Bitcoin so here we see Alex on the lock screen and there is a very unique skill of Alexa which is called donation skill and it is really really simple you simply say hey Cortana open Alexa donate money to a name of a charity and you can donate up to five thousand dollars in one donation and what is really amazing is that by default this ability to buy stuff and donate something is turned on but the setting of having a voice code to accept this donation is turned off so yes and although everyone can open his

own fake charity it requires some some effort but it is possible so we cannot just donate to a specific charity can donate to your own so I want you to imagine that you are very responsible and you left your device locked in your locked room before you left or eating something but still someone an attacker can simply cross the hallway shall take odana open Alexa donate money to my charity and it will work here is a quick timeline this integration was released on a 15 August a couple of weeks later we already reported this to Microsoft and they were really quick with the six actually I have to say that in all vulnerabilities that they were very fast

and they simply removed Alexa from the lock screen and let's think for second why this happened then I thought of two main reasons the first one is that Alexa is mainly designed to increase Amazon online sales so this is why the ability to buy stuff and also donate stuff is turned on by default and the second reason is that Alexa which is usually running on smart speakers is not aware of the term locked screen they simply do not have they do not have locked mode and it doesn't have a screen so when adding this together we get this vulnerability and now we are done with Alexa and we are moving on to another interesting integration between Cortana

and Spotify and maybe you think that in the previous SEC integration with Alexa the ability to open a lecture on your lock screen was simply a mistake they didn't mean to do it but here we can see in the integration between Cortana and Spotify that on the official website of Spotify that will be proud of the fact that it works above the lock screen as well as below so this is a design flow okay and you can already think what will happen if you are not already signed in into Spotify right we'll get this screen shot asking you to link to Spotify okay then link to Spotify and we'll get exam again the same window that is called connecting to

a service and from here we can again click on other links for example the login to Facebook with Facebook and from there to other websites and to download your own malicious exploits so and we found many vulnerabilities in Cortana some of them disclose private information like Skype contacts your calendar events and many more things and we won't have time to see all of them but we wanted to share another vulnerability which is a very funny also so mm here we are on the lockscreen and we ask Cortana what is the phone number of Microsoft customer service okay maybe we want some help for Microsoft and then we will get this phone number okay of Microsoft which is

actually converted into a link and when placing on this link we get the people application running on your lock screen here again there is a button named at the photo and you already know where it goes from here it was doing the World Cup this is because of the football pixels of all the you and we can see all the contacts all the people and here again we search for a Open File dialog and we found it and for me again we can run executables and that can be everything we can we seem to take over on your log device and what we can learn from here is that you need to solve the root cause of your of the vulnerability

it's not enough to cut the the path to this dialog it's not simply to remove this link or an other button you need to solve the root cause which is the opening of a file dialog on your lock screen so we talked a lot about Cortana on Windows 10 but we didn't stop there we also wanted to check what will happen on other log devices for example and though it forms Mac devices also and even Windows Phone and we found in all of the categories we found vulnerabilities and we want to share something on Contin on Android and if we think about it then having a voice assistant on your mobile device lockscreen is even more logical than on your locked

PC because we are instantly using our phone and it is not very convenient to say to insert a pin code every time so if you can simply talk to Cortana on your lock device it is very convenient this is how it looked like okay we have Cortana this is taken from a commercial advertisement of Cortana here we have a button that we can simply swipe and get a menu of what of the calendar of the weather and something else so they have an ability that we found here and you again Microsoft thought about the security issue of this of this device when you want to ask Cortana something it will ask you to open the device first

but if you return to Cortana and ask you to do it again then it will accept it let's see how it looks like now we lock the phone and you can see that there is a pin code and well opening Cortana while pressing on this button and we are simply type we can also type to Cortana you don't need also you do not need just to say something and we typed something and here is written please unlock your device to continue but if we open Cortana again

if we open Cortana again then we will see that it it stinks what to do with what we've written and then if we click on the search button we again have a browser on your log device on your Android device and from ill we're also already signing into a user account and from here we can access your emails your contact list your calendar and simply all your information on your sensitive phone

we access the email we can see all the old conversation everything from your look to a device and as I said before we found also stuff on Windows Phone and a Mac and other stuff but we won't have time for this and I mean I will continue from here I want to hack in maybe need to say twice or three times and then it works let's talk a little bit about the whole process of finding the vulnerabilities and and then working with Microsoft to fix them and that kind of will lead us to the conclusion at the end we're responsible researchers so we do responsible disclosure and there were a couple of cases or it was very clear there is no

other way to fix the problem other than send the customer patch and register a CV and and I have to say Microsoft was very efficient in doing that you know I remember reporting vulnerabilities to companies Microsoft others 10 years ago 15 years ago that would think sometime between 6 months to 18 months for a company to fix vulnerabilities no they were very fast in fixing even those that required patches but then for most of them they said hey we have this very easy fix in our cloud like okay there's a choose a photo button in some skill let's change that skill to not have a photo button there is still a functionality for open that command-line over locked screen in

decline but presumably it cannot be accessed anymore oh we have this connected to a service thing that sometimes pop up when you try to execute a skill but did not register first you know and it kind of gives you trouble so we go one skill after another and kill that functionality it still exists in the client okay but but hopefully we cut all connections to it probably the worst we had it's the first one voice of eso there were certain phrases that you would say to Cortana that would invoke and explore over locked screen going to unsafe servers would say something like at the beginning hey Cortana go to bbc.com and for some reason while maybe

see that tom was still not a safe server at that time it would just do that so Microsoft went and fixed their cloud engine to wire these phrases into safe searches but do you remember machine learning cortana actually taught itself around this fix because we constantly found out more and more phrases that would actually do the same and it's not that they existed before it's that people were teaching cortana that go to BBC is just the same as go to bbc.com but it's not to a safe search or something like can you tell me something about BBC yeah i can tell you about me see I can take you to their side and it's not safe okay

so we kept finding this thing and reporting these things and they would keep making all those small fixes instead of changing the state of mind and and really fixing it in the client which required of course if he is sending a page of the client and so on at some point your vial became so good at that that Microsoft said you know what that's enough that's enough we don't want to admit defeat okay so Cortana would still work on locked screen but the reality is that whatever you would ask Cortana to do it will say I'd be happy to do that for you please unlock your device first so it would actually write on your lock screen

you can ask Cortana this and that but you go and ask Cortana dances I'll be happy to do that for you but you need to unlock your PC first and that was like a really nasty change especially for a research but also for users because it was not announced it was not published it just happened overnight because it was fixed in the cloud and at that point say okay so security problems are gone right and if you unlock your device first and here I can really say the hand is quicker than the I really need nimble fingers for that we're invoking Cortana over lock screen we're asking for a reminders skill and again we're being

persistent because we're doing that twice while we're clicking on the password field and see who turns back it's the same reminder skill with the same and a photo button okay so here's my conclusion from that you say you know what's the moral here and and many of you would say well the moral here is that you have convenience and you have security and you have to balance them that's not the case that's not the case Microsoft did not disregard security Microsoft did not choose convenience over security they they had security in mind okay they they invested in it okay they did code reviews they why there SDLC so what happened and I think this is the whole

purpose of this research teach us a little bit little something about how to create security system you need to ask the right questions at design-time okay because people were taking the code piece by piece and say well is this code secure reality is the code is secure problem is with the interfaces problem is with taking one system like Cortana that has some security assumptions taking a different system Alexa would have very different security assumptions okay we tied them together and we maintain the same security assumptions now you ask the right questions per the new concept that you are going to invent the right question here would be for example can I create a voice malware now that's a very stupid

question can I create can I create a voice ransomware can I come to a computer and say hey Cortana please encrypt all my files it's a stupid question right but when you start asking that question which is what led to this home research when you start asking question you understand that in the context of having voice assistant this is the right question to ask and we start asking this question you start uncovering the issues and the problems and and you can start finding where you got it wrong now the next is of course you have to solve the root cause okay if you have the Cortana agent that can make no decisions and allows you to link to

insecure URLs at some point someone is going to tell that engine go to an unsecured server if you have a client that has the functionality of displaying a browser arbitrary browser with arbitrary partner pages on your lock screen at some point someone is going to tell that client do just that okay you need to realize what the root cause is and then you need to solve it in the right place did that find files window that you've seen in a number of the vulnerabilities it will strike again there is no doubt about it it will strike again because again there is a client there that can make no decisions that is able to display that

functionality now what should have been done that client should have made the simple decision I'm saying I'm not displaying that Explorer window on locked screen very simple decision that would eliminate all these vulnerabilities and the one that will find that there is no doubt about it okay but needs to be patched and solved in the right place so that's our small contribution to building secure systems and to 45 minutes of fun thank you

question well few cases we try to ask them why don't you do that like this one [Music] it's a big machine it's a big machine they have a very efficient front-end that communicates with you they don't have the full picture always they try to escalate a few of the issues back as they can so that's that's a bit tough I have to say telling them how to fix it was was was not working anyone it yeah [Music]

[Applause] yes and no no because if you look carefully at what Cortana says about identification it says you can only apply that for the hey Cortana thing ok which is not always necessary for invoking Cortana and they say we will try to do it as best as we can reality is that in this case they prefer convenience to security it's very easy we've done a different project with the Google assistant where you need to do OK Google in order to open your phone and we and and you do need to be identified with your own voice there that was a fun project we did it again in the Technion with graduate students and it was not

very difficult to have enough combinations of people saying OK Google that that would open that for you even if it's not your own voice it's not an extremely strong measure especially with with you know short phrases anyone else thank you [Applause]