Privacy Vulnerabilities and Mitigation Techniques in Machine Learning

opinions next up we have a story of Tara's our well crisis and vulnerabilities and mitigation that needs machine learning okay thank you so yeah I think should I check another applied research mathematician with CTC if you're in Johnstown that happy here what I do to chat with you today about the realities try to field within security and privacy within that field it's sort of doing there's a lot of energy level American security privacy and everything sort of changing it by day this is certainly one of an example of that later this is a very active research field within community right now very young but very active and that's smaller drilling so I'm gonna kind of talk about like these topics

with you they convey so we worked at their party let's see to see is done within the greater environment well machine learning that's a relativities where the new and younger field as well so it's exciting to be a part of one that's within the field I see the gun itself okay that's all with you about today it's just sort of think about what the kind of level set talk about machine learning thought about the current state machine learning or we've got there then once we have an idea of that part about some of the vulnerabilities in relation to privacy and security that's discipline in machine learning and then prevail give you dramatic to one example

the membership difference in time and one possible mitigations that meeting against a technical time call that differentially prior sarcastic gradient descent so if you take a couple minutes later to level set here the last decade for machine learning has been very energetic very exciting there's a lot of turbulent energy that feel that now good part that comes from 3d External Relations I mentioned to the first one of those forces is tremendous amount of advancements of that procured in infrastructure and distributed infrastructure to use that a fatal place at places like Amazon singles occurred and all of us seniors have been done just really loyal to say it's to the point now where anyone that has a

machine learning problem or analysis problem in general can go online and buy some infrastructure relatively cheap cost they can all those who think they can make a model back it's highly accurate and it can make it a string that the predictions that they could then go into your own application normally we're a lot of time to do space for these over here I go so they want to make some sort of machine my mom that they can deploy it to your iPhone and that gets into the second sort of unity factor that is behind this launch an expensive machine right and that's that's data so when they have them all off with application on your mobile device they reflect this

very unique data at the China time discover causation correlation to behaviors

and served with that with that data it's more scales my first method able to make the models you be more last number of years and in particular one of the one field that is deep neural networks in particular those models are extremely accurate and been very successful so I'll be about 10 years ago or so and beating up for that for companies like Cisco or Google it was really exciting about the source code was very unique and proprietary and after the secret sauce existed that's started changing in a large way about 10 years nearly said we're no longer considered the code to be idiomatic ate it was being stolen that was being reverse engineer and they

really had exactly the business well specialized machine learning and the story there in terms of free and open source they concurred as well because it's the point however since anyone can fire off these really advanced infrastructure and do a relatively cheaply that's no longer the differentiating characteristic for business models it's all about the data more data more unique data that's what it's all about because right now it's often said you might hear that data is the new source care that sort of lesson that quietness differentiating factors the primary information is all about the data that's what we really want to keep sensitive or trying to keep it from being exposed

so that's sort of where we're at in terms of the success of machine but it's all like I said it's all about the accuracy walking mafia so how do we build these models was the model of way up here on the Left I want to collect some data for dinner that's this way it's a little bit hard to see but it's not consists of ten rows so there's ten rows there and each row corresponds to an object so there's a Ferelden planet that it's a picture of that and I'm 10 columns such as ten examples of that class what I'm going to do is on the train the machine learning model the enrollment to identify that object class

based upon evacuation so what I can do is I can take all that data this is just a small small sample of how much data I had and I pass that through when I train off this machine learning mom once that's done and I do it to an accuracy that I feel is good enough for my application I can employ that model so that I can use it or those can use it and how they use it is they want to pass in new new data so that the classifier here it doesn't play so I asked him in mere image that the model has never seen before and if I built the model correctly it

should be able to tell me what object class that Richards service gets that passive force should identify on the right hand side that most likely this object is efficient that's the most or the machine learning pipeline that's all fit a nutshell within that within just this pipeline itself is a number of all abilities the first one is that since this model is out there it's meant to be used and Samantha have data coming a classification of what the hell is on the airplane so really as someone who's using this model I think a second answer to that as long as it's up of mine so format it may be the right height I can throw anything in there so I might put

the image of the horse in there but I might also take that image and just tweak it slightly and do it such a way that I can't tell when you look at it can't see that you've any difference there but to the model you want to classify a completely different and possibly in any way so as I move back another type of all nobility there a this model stealing so these models are out there if I can access them if I can query them via any API I can see what it's doing I know what it's trying to do and so there's nothing to stop me from anything another model that does the same thing

to the even higher Anderson says call model ceiling as I move north to left if I have access to an adversary has access to either data that you're using or software or the structure again tweak it in such a way that the model will behave normally when when you're looking at it Wow there are certain circumstances it'll react completely different in unexpected ways they call it model boy thing or and starting our back to the model we're calling on that that the right now is that with autonomous vehicles that are using these neural networks they've been trained on all stop signs at all times and you can poison that with just a few data little bit of ice

so that when it sees the stop sign with the yellow you know that by that as a yield sign whatever else whatever other kind of Simon wanted to recognize animal style so that's an obvious politic as I move all the way to the left this thing I'm going to talk about today the thing I'm most concerned about is that that data that I use to train this model them this is the sustainable items that are very sensitive and very poor to my business unit there are tax sent that can leave some information that'll tell you about that training set that I used to spend a lot of money collecting and curating potentially it will give you information

that these are two types of all bottle version and membership difference what I'm going to talk about today is membership inference and the question behind membership inference is that certain wants to know be able to determine whether or not the specific piece of data was used to train that so it's fairly simple question I have a piece of data was it used to train that mop if you have questions at any point by all means of the time so this is a sort of effective representation of it so on the left there I have a cursor the models in the middle that that's what's deployed you can access that on the right hand side you have inserted wants

to get to be my data that says they can't really the service see what it is but they want to be able to determine what time information about so things I may be years they can obtain some data that they believe might be in my mom okay so so here are some samples of birds that I'm using and since the model sounds where they can put those data points through tomorrow and get the outputs pop and the answer what they want to do is just be able to look at that Apple from be able to say whether or not that data for today maybe I can determine just by looking at the output which one of these days in

sure so one way to obtain both accuracy and generalizability in terms of machine learning trend know there was a number of techniques that have come out of the last of years about how to do that so that your model doesn't uh over training on two different data points and one of those things you mentioned was dropout there's all that health either accuracy and in data augmentation they help I democracy but there's still part of the model and it's an open question as to how they impact these attacks that's what I think people that as music so to urban over the question this is a very open research group and that impact of model design is one of is that

researched but nonetheless you give me that off I can still tie

it's a way that I'm going to attack this hall whether this mention infants attack works it's sort of the basis of it is the shadow of come on steel so Stuart your mom and I'm going to create these other models called shadows and they're basically going to behave exactly the way that the target column should be relatively the same accuracy the same objective they were going to classify these objects and I'm gonna create them using some data that I might have that I believe that you use private day to help they're on the table I have my tea about what that is about what you're using and I want to test out the hypothesis well I can't see your

your sense of the data perhaps but I can see your target home I can do that father ceiling based upon my access to that and how much data you have and actually you want to achieve you can create as many of these shadow balls as you want with their sound almost I have data that I cut off to the side once I have that auto bills I can pass data through to create that's a to attain an average of these probabilities okay so I've never stayed over here who was in the training center he shot a model that I feel that I don't fit I also have some additional data on the side that I didn't use with each other

test set this is also a kooky new machine Miami generals know about a test dataset to date an idea probably among gentlemen so I have my date over here I can pass it through my shadow malls and when I get on the output is the probabilities probability vector of what it thinks that facet over each one of them is that I passed through I know when I was in the training good set and so I can add an additional label over here get in and out in representing it was in the training set and how being in love so what I have now over here because I have new things that quickly you did set for

the features of these probabilities and I want to be able to predict that they will in vs. up so this is a common problem in machine learning tell a binary class prediction and they contradict it around and I will create the machine learning model to be able to predict that for me it's a sort of light really cool because you're taking machine mind and focusing and turning it on itself and what I'll do is take the Pugliese feed into that build a tech model that will tell me whether or not I'm Jay Larson so once I have those attack that attack model forget it this is not attack actually works so I have access to your model it's a part of

all here and I have data that I want to know whether or not those in your training center the I can ask each piece of data into your model and observe the output I think that out and I know what the true label is and I can pass that through my attacks model over here and that attack model if I built it through a high action will tell me whether or not that specific bit of what we said do it by the other side that's not that long it's relatively simple it's your weapon understand the question comes to what extent how I measure success if I run this attack cut my measure success so what I'm going to do

is I can create a graph on the bottom x-axis there it's going to be the accuracy of that target long that's the one of the times so I don't necessarily care if it's state-of-the-art accuracy just have to be relatively recent like she said something 70 80 to 90 percent whatever it is you know it was just pick up the baseline what I really care about is the y axis y axis is a measure of how private fit that model is to this attack it's just as this is a yeah it's a binary classification right for any piece of data it's either Anton Zeck right Sarah over enough data that's been I guess you would be good either saying he did this

an attack enough on our journey right half the time so up at the top is perfect class that's where the adversative can't tell on average whether or not a piece of data the opposite extreme of that is whether an adversary can't predict a five percent whether or not any Jews than that and said that's all the way had not gone no practice in this way probably does about this model and have certain yes correct me at the time okay so ideally I want to be more over the top you can wear that - so everyone I want to on experimental I'm creating just a we're staying the convolutional moment passing the data some complications on our component on

the output we're going to tank class label so you're passing some day that train off the turn your mama I can take different data and create these shadow knowledge from it and perform the attack that resulted for the better than this when I do that I get this data point right there so it's about the model itself was about seven five percent after that's too bad but the attack right at home and it's about seventy percent I also but it would be up here Nordberg comprising so this is telling me that tomorrow is leaking information about my dataset and I want to create five ways they're kind of pushing this stuff this day for to a perfect privacy take sure

that my biggest now one some ideas behind how that happens is that third point that I talked about earlier the third force behind what makes machine that I said successful right now that comes down to the water for solution learning it's all it's very fascinating to sample so that's also a call usually just a refrigerator - TV so this algorithm it looks like your data and trying to create this model which is just a bunch of weights really and you want to optimize those weights so that your your predictions are the most accurate that they do and since it's really just a mathematical function and what we do is you can take derivatives of that

function so you can take the gradient here with respect to you or your whiz and based upon that gradient I'll take your model parameters along the steepest descent and then you never ever look over your train Hanson gets it actually that you want so what I believe out there is that during this process of stochastic gradient descent there are data points that have a larger gradient that other data points and because the good meeting is larger you're going to follow that way that's what's going to get stored within your model essentially the storing information within your model you can sort of work your way back to information about those data points with the largest information the largest

rating and so that eventually driver version of this model what it does is for each data point in that it'll calculate the gradient and then it will clip it to a certain level a certain threshold in there okay so you say I don't agree needs to be this large they're looking at the same time I'm going to do that you have aluminum's big a little bit more wrangle just to kind of it that's how to remove that lot of certainty about and that's the gradient that you're going to not all fit your your and on such with this is a visual what that looks like so the top row there is a visual representation of

stochastic gradient descent those great to be any signs and you're just kind of clipping them down and adding random directionality to it so that converging okay when you do that when you use this technique using that same model the same sort of training process when you use you can take snapshots of all the training process and perform this attack at that snapshot if you do that problem up here that you know for headaches of this training you get a lot of that is pretty secured against this attack perfect right up here unfortunately the accuracy of that part about two percent so it's security usually you get a trade-off in there are seven accuracy gears up to 55 percent of decisions out

there at the same time as its go I hope you get a dinner and yet again it's extracting more information about that data out and so the nobody to attack it goes up as well ends up being enough that's 60 percent so using this technique I did lose a little bit of accuracy welcome back 75 at the same time they gain privacy because now it's a little bit harder to attack I do it like to have something that's both accurate and right any questions about so as a whirlwind but at the end of the day the point is that as machine learning and security practitioners I have to be aware that machine learning models since they've

been trained up with accuracy is the key great measure of success there's some compiler that exposes your data and these models are leaking that information ways that we don't fully understand that instead of whether you're building these files or entertain a certain actually but you also depending on the years case we keep in mind that they don't need that information and there's this trade-off there between utility matters we've also seen how she no independent sort of be used against itself in sort of interesting ways and this is just one example of it we also talked about membership in first attack how to sort of mitigate against that we're gonna grow an introduction into it any

questions or comments

got a few bags but if not so many times that me out all these projects will