Strategies for Secure Development with GraphQL

all right let's get started I hope you guys had a good lunch break sleepy this this talk is definitely going to help this is an amazing talk on strategies for secure graphql development and uh I'm excited to introduce you to Mr glass yeah welcome thank you yes this is a strategies for secure development with graphql uh it's a terrible title not on but that's what we're going to talk about uh my name is y Silberman I work at Aon Cyber Solutions um they are very very very great to pay my way here and also they're sponsoring tomorrow at the happy hour so please check that out I also am staff here besides Las Vegas um I work

on the website it's an immense privilege to be able to work here as staff so that's me our goal today is to make a developer experience where security is actively nurtured and and supported I I see a lot of projects where we talk about security being important and we say oh the team you need to do security but nobody actually helps them do anything secure so a lot of what I'm talk about today is trying to help make something so that developers have the tools they need a note um most of my talk is framed around graph q but this should be applicable to like any engineering any development so don't worry if you're not

a graphql developer this is really more about applying standard practices standard uh techniques for security security development to graphql um also a little note unfortun I made some really awesome hilarious memes for today but there was a snafu and I can't use them so you're just going to have to assume that there were awesome graphics on each slide okay so I decided uh to frame this around three questions I actually I spoke at besides a decade ago uh and it's it's been an experience since then and I was trying to think what I would tell to me a decade ago as a long young engineer about security and I decided there were three things that were hard

for me to get answers to and I would tell tell them and that is how do I figure out what I need to secure how do I build something like sure secureity Foundation everybody says you have to start with security but always tells you how um and what should I la which is something that's it's a little very specific but it's a question that I never got an answer to ever and so I want to talk about it okay so our first question how do I figure out what need securing anybody have an idea no that's okay there's an obvious answer it's threat modeling threat modeling is uh has been around for a while at this point it's a very well

understood process you you the idea is to model your application build a model of what you are building so you have a representation of it you know you can look at it and then try to apply different threats and different threat factors to it so uh stride is a threat modeling framework uh came out of Microsoft and you model out your application and you try to apply specific vectors of spoofing tampering repudiation information disclosure the all service and exhalation of privilege monic strive pretty easy remember and these are the things you're going to look at when you're threat modeling on your model you're going to look for areas where this could happen so this is

a high Lev model of what pretty much every web application looks like on the back end um I mean there are bigger ones people do custom stuff but most of them have a a um public endpoint like a laugh or AWS endpoint and that goes on to an actual graph ql framework so now we're talking about inside the code the W will call and call the your server software which calls your framework and your framework is going to run and the framework is then going to call your model logic that's how all of the web uh Frameworks are set up that's how everything is set up so on a high level I actually worry about

um this and and here I worry more about like stuff that would affect the framework system so stuff that would affect off which is part of the framework for me and part of frame most people the spoofing um anything that could do denial service would probably end up in the framework I mean you could have bones in your code but I worry about framework level issues for denial service and same thing with escalation of privilege I'm worried about since I use the framework for privilege access I'm I need to make sure that my framework has what I need so I need to worry about it on that level and then in a lower level in

graphql you can graphql it's complex but it's not too crazy there are few parts to it the first basic part is a query and a query is you post a payload to the server and ask for what you want um you can graph Co you can request multiple objects at the same time through the graph so you can say I want the user and I want all the users posts and it'll get all four you in one go which is really awesome but also opens you up some attacks so if you have a cyclical reference you could have a very long cyclical reference attack to get service um there's other attacks that are out there and

subscriptions are actually very similar to queries uh the only difference between a subscription and a query is that a subscription is done over a steady connection like a websocket and then the server keeps posting new payloads as things change so there's usually an event there's a specific subscription that you have subscribe to and when that that gets triggered the server will automatically send it to the client so that the client get updates in real time so that's it it's it's almost exactly the same as queries but it's a separate thing um and then there's also mutations mutations are awesome back when I first started everybody was moving to rest and you couldn't use RPC RPC is evil RPC is a remote procedure

call and so what happened in the industry is like we have to use rest rest is where rest rest apis have Paradigm where you're posting a document and pulling the document back right that's what rest is all about but let's say I need to uh fire off an email when when somebody puts this thing well now I'm just having a side effect whenever somebody changes it there's a side effect in my code trigging something else it doesn't fit the Paradigm of I'm posting a document it's all side effects so I have hated rest for those things it it doesn't apply and RPC is server do this so so if I say server send an email

it doesn't and it gives me an okay or a not um so mutations can do all that stuff they return basically the same thing as a query payload inside so you run your mutation and it can return uh data out of your mutation or from just your whole database same thing with the graph all this supports following the graph to the uh child objects and everything um and I worry on this level about tampering repudiation and expolation prage because this is where their code is actually this is where they're actually hitting your code and this is where you have to worry about actually having lock down access and actually having locked down who can touch what and things like that so

mutations are where you're going to have your code doing things that aren't standard and you're going to need to worry about that stuff so my big thing that I try to tell people is to treat security like a feature request developers we get feature requests all the time and we tend to get like oh and this needs to be secure we never get a feature request of add MFA as a feature but that's a huge feature so approach security improvements of features a you can probably sell it you can get planning on it and the it lets you approach it in the full process like we do everything else because developers all have processes for approaching

features and I also really like to test them like a feature I skip a f no um I also really like to test them like a feature so don't just test that the user uh my favorite example that I almost never see in a codebase is a unit test that says posting the wrong password not allowed in right you have a lot of tests that say I put my password in it works almost nobody has has not putting the password in getting blocked but don't just check that that got blocked check that you logg whatever you wanted to log check that you responded properly so that you're actually saying back to the user hey this was blocked

for this reason um and all that stuff make sure you actually check all of this it really helps it helps make security easier when you do all of this it it it really helps when the problem happens having all this supports having somebody be able to figure out what's wrong so adding in this test very important to have the logs and everything else just make sure all parts of your security feature are happening um I really like functional tests unit tests are ult um and they're not bad they're they're pretty good the thing about functional tests if we look back this is from the earlier slide so we have the framework and the urm and these are

controlling a huge part of what we're worried about right we're worried about how the framework is handling off and how the orm is handling checking actual database access to the items Etc uh and uh validation on on inputs as well and so if you're doing a traditional unit test what you do is you mock out everything outside of the the function that you're trying to test and you just test your specific unit of thing that you're trying to test and so if you're writing a unit test you mock out the uh you mock out the framework and you mock out the urm in a real unit test you only have your logic so you can't tell all

this and at the same time you a functional test also will get you a whole breadth of coverage a functional test instead of un unit test is just one little feature a functional test is literally hit the full page have the full process of the pro of the website run and get a payload back and check that the payload looks like what you wanted uh so similar what I did here right I'm not going and checking the um add user to group function I'm calling a query from the outside and I'm getting the full response back from the server um that's a functional test um and they're awesome and they actually allow you to test the other stuff so don't

just do unit tests okay how do we build a secure Foundation what time is it sorry I have to keep an eye on the clock okay so start with the framework um Reinventing the wheel is bad especially right now there are tons of great Frameworks graph is all new code from the past five six years so you're going to the Frameworks are all well made and don't reinvent the wheel graph wheel has a very complex uh has a very complex query structure that it needs to parse and parsing things is where a lot of mistakes happen so don't write your own parer use one that's already pre-made when you're picking a framework look for an active community and good docks I

like to check the Repository to see what how they've responded to security issues before because you can immediately see if oh they actually respond took care of it or they said I don't know if that's a problem oh maybe we'll fix it in two years you probably don't want that framework that like is it's saying you know it's not sure if it's going to fix the the security thing right away you want the framework that's like oh security problem here's a hot fix um so check that they've done that in the past in case it happens again and another big thing that um I've seen a lot of people fall into is they'll pick a framework that's very

specific to how they want things it's very opinionated so if you want all of your code to be formatted a certain way that's cool and you should do that and we'll get to that but if your Frameworks want you to format your code a certain way you can't change it if you want to change it you can't change it if something comes up that makes you change it right so if your framework is is kind of blocking you from doing other things that weren't your opinion at first it will come back back to bite you because it's going to block you from doing other things you might want so don't look for a framework that like perfectly matches

what you want look for a framework that's more of a tool for you to use to build what you want because what you're going to do is you're going to take that framework and you're going to extend every class in it if you can so there's a standard model class that you're probably going to have in framework you're going to extend that and add what you want uh big ones are non-sequential IDs timestamps and uh I actually really like to disable super user functionality in Frameworks um so that there's no super user like they need a permission to actually do the thing instead of having the ability to just do whatever anything all the time so I like to

disable superer entirely I like to build in off checks into the the note resolvers which is part of the graphel side in the objects um this is where you really should enforce your opinions so whatever you care about being in the object like nonsequential ideas is a big one for me um whatever you care about enforce it at this level this this is where you're kind of making your framework but you're not making your own framework you're using something else but you're making into your own okay that will allow you to build all Security One into it then you're going to have to go beyond that because it's not going to cover everything and you're going to

need to find a lot of simple stuff it's probably going to seem simplistic that developers are going to need to do in the course of their business and write the functions for them so that they don't need to figure out how do I check those users active right they just call the isactive function I don't need to I don't need to um there's a whole bunch that we have in my on my team for things like permissions right I don't need to understand how the whole permission structure to work I can call the permission helper that sets up the permissions so for security and specifically for authorization stuff you can centralize it in one place you can

well vet that thing and then they can call the function and know we work for them and do the job so this is part of again building an environment where they can do what they need to do is finding these simple actions I mean the the most common one um is a decorator uh the most common one is a decorator you put over your function decorators are little comments you put over a function um they're supported in a lot of languages and they usually like add something as a wrapper to the function so they wrap it in some logic and so I have decorators and I've seen decorators in Frameworks a lot of Frameworks have it

um or you can basically put a decorator that says this is loading this object at the start and it will automatically load the object automatically check permissions for the user and just when the when the developer goes into in the context of starting their function they already have a vetted object that they know the user has access to and they don't need to worry about how to get it securely they have it already so that's the thing trying to enclose that logic developers don't need to worry about how do I do this securely um it seems stupid to put something like is active in this but I really wanted to point out that doing the simple stuff helps because

they they might see the active thing and not be sure also you're going to get people who do it slightly differently in two places if you don't have this you're always going to have slight differences everywhere and if you ever need to expand the active thing so say you know you get a call and we're like hey we need to make sure that users who are over 2 years old can't sign in either right so you can add the check that the user hasn't signed in two years two years active if you have it here so really try to enclose your stuff in helper functions um it really helps uh checking if you user in tonal is another

great one you know uh often you have functions that you don't want clients to use but only your people to use only your employees to use and so just like as active is a very simple one that is it somebody from the company is a very simple easy check um and important and again that can be very complex logic depending upon your company but if you enclose it in this the developer doesn't need to worry about knowing that complex logic they can just call your function and I also really like making test helpers because you need to help the developers make their tests right and you need to help them make their tests secure so a huge part of my work

in starting my this my mid lest project was making helpers so that you could easily make a query and get it back so those function so that functional test that I showed you was one line to make a query instead of 100 to build the query and insert the stuff and get the user and say okay now pass it into the framework this way I enclose that all in the function where you can just say give it a query out a query string and a user and it takes care of everything else and gives you the response and that allows developers to develop tests quickly it make them more likely they're going to make tests and it it really helps keeps

things secure because it encourages testing of this layer of stuff and we want to encourage enourage testing of this stuff so if we've done our job well at this point we have secure Behavior by default in a lot of our classes as well as sorry I'm just checking your time as well as sensitive logic that's enclosed and centralized centralization I should talk a bit more about um when you have code in multiple places it's impossible to keep up up to date it's a possible to keep all them up to date in right the same way and for off it's huge important that you don't end up with what's called like a shotgun parser so where all your checks are

happening all over the place you want all your checks to happen in one place so you can understand what's going on you can understand what the checks are and and you can clearly develop it so centralization is very important for for things that SEC processes and particular authorization is a huge one um and build Tex with broad coverage I I can't stress that enough that'll help you in multiple ways but in in ter great really helps to have broad coverage I have test that I have test that check every if a user can load an object if a user can load a list of objects all that and I have a helper that does it for me so I just have one

line that says get object as User make sure it can use it right if I didn't have that helper developers would probably either have a 100 different ways they're calling it to check that and they'd probably spend a lot more time or not even make the thing to check the full thing but because I made a helper and because I I made these help I let them do the Bro coverage they have a full all-encompassing test so the helpers really encourage the developers to do the right thing um we're on to the last bit of the talk which is what should I log so anybody have an answer everything is the first answer I always

get you shouldn't log [Music] everything mhm resp or what to exactly what you need to audit is the big one right if you're logging everything there's a huge amount of stuff but also it you you're going to get a lot of private data in your logs too right if you're logging every request you're just going to get a lot of data that you don't want in your logs you don't want to have to treat your logs that way so pay attention to what you are logging and log specific things that help you thinking about repudiation is a lot of what I do so the audit stuff think about okay somebody's going to come and say I didn't do this action you're going

to need say no that user click this button at 10:53 on Thursday um when somebody asked what on Earth happened to this you know you're going to need to figure that out if it's something important um and I stress Don't limit yourself to logs um logs are great but there are other ways to capture information that will help you tell the story later so already I mentioned time stamps on each object um time stamps let you see when was created when was updated they're built into most Frameworks or they're super easy to add and that lets you know when the object was edits you get the call what happened to this well Bob edited on this date

right because I have the edited by and the updated by and the updated app so I can easily go and tell you Bob edited that one last that's why it's different right and the most important the very important thing that you should not admit in logging is authentication you should be logging what's coming into your authentication you can see a TX coming so that's like the bare minimum to log if you don't have this you either have a very specific use case um so there the fun thing with repudiation is sometimes repudiation is also a security feature so repudiation is the ability to say I didn't do this right so if somebody wants to claim they didn't do

it you need to prove they did but let's say I had secure messaging or past bin right ppin doesn't want to be able to prove who posted they want it to be a public dump right so that's actually a feature of ppin that you're able to repudiate that you post to it I think P has that um so unless you have a specific reason not to have the logs um these specific logs these are like the bare minimum that you should had and again these aren't the TOs aren't really logs but they let you tell the story of what happened to the object um my favorite story about the refugi um is from Defcon um few years before I started

going to Defcon there was this s called Cisco gate speaker came and spoke about Cisco was going to speak about Cisco routers and vul that they hadn't patched yet and Cisco got an injunction against him and all sorts of stuff happened they actually ripped the talk slides out of the printed Black Hat books you B of the slides and they like ripped the whole section out um and one of the things that happened was that the feds said I need a copy of every I need a log of every person who downloaded the slides from the website um and they were forced to turn that over uh and as Dar tangent talked about this in a talk a few years later

and as he put it and that day we turned off the locks cuz again they don't for deathcon they don't want to have the thing to turn over to the feds of who downloaded the files right they don't want that information so again thinking about it it's a feature um think about what you're logging uh yeah and I actually I really love that story because it's just yeah we Defcon stopped logging they stopped a lot of their logging because that way they didn't have anything to turn over more intensive than that but still in the same vein is object versioning so you can often find a plugin for this or build it yourself really easy

basically every time you save you save a copy of the old record in a in in a table somewhere in its own row and that way you can go back and see each individual version of the object so you can look in that table and be like okay the edited version from this date was like this and then it was edited here this way and here this way every individual version it's really easy to set up in most Frameworks there's usually a plugin for it um and again telling that story seeing exactly what happened to the object as you go through explicit logging of sensitive events um I can't stress this enough this is again

if the authentication intends but also if you have uh sensitive parts of your application every application is custom logic in it right so you have a sensitive action that a user does I really I consider anything that changes an off access permission so them inviting you to a group giving you access to that group that changes the the access permissions of the group so I want to log that he made that change at that time that's a sensitive event there might be other sensitive events to like real word payments or things like that but actually like having a easy to use logger which is not hard to set up so you can just say log a message here this

happened um it there's no substitute for it and again you can very specifically say on the important things this happened here right your company has something important they want to know about expl L log when it happened and what happened um again you want to avoid including private information don't put the user's name WR I use uh uu IDs so I just put The UU IDs in it and make sure if you're like changing like the owner of an object one mistake I made early on was to just say oh Bob made whoever the owner of the object and I have no record who the pre owner was so make sure that you have your log say fully Bob changed

this from Steve having access to Tom having access right um don't you want to have as much information available on that log to see exactly what happened point in time DB backup systems are fun just kind of like the object versioning where you can go and look at every version of your object in the database Point time backup systems let you basically say roll it back to Thursday at 300 p.m. exactly or 30:2 and 30 seconds um the way it works is is that they have like a base a base um database copy like a like a snapshot and then they record all the addition all the changes to the database are recorded as individual events rather than put

into the database normally and so when the record as individual events it can play back up to a certain point so you can it can literally say okay start from this it's start from this backup roll forward to 3:45 p.m. and you see exactly how your database was um I put this in here because it's remarkably easy to do um and I say that because it's built into RDS so if you're using RDS and most people I meet are you can just click a button at a point in time database backs now they don't go so far back unless you tweak them to go very far back um they go about a week or two I think by

default um but that can be hugely helpful as well uh I want out because it's it's a button in RDS to turn that on press log in graphql so going back to rest typical rest logs a way we way most people log them is to log the um endpoint the method but not the act any actual like post payload because again the post payload is likely to contain user information that you don't want right you don't want to have private information you're logs so we don't usually log the post payload uh unless you have a really secure application we need to have everything right um and instead we just log the uh the method the

endpoint uh usually there's a time stamp but I left it off here right and 200 these are the response codes that say you know this is a good response this is something was actually saved and this is an error response right and with rest this works great you can clearly see what H what this user did they went and looked the post and then you and you went and tried to get a group that you didn't have access to right so this work really well again for going back and telling our story with PR you can do that but there's a problem with graphql graphql uses the same endpoint for every request and graphql by by standard does

not give error response codes um the reason for this is it's um the RPC style of it um if you post something that fails in a normal way like not having authentication that's a 404 your request worked but the inner thing that you asked for didn't and so graph responds with a success payload that inside said here are your errors and it can give you multiple errors multiple problems and it can partially give you your request too um so it's the error responses are very different um and so you can't really tell what's going on from just that so if you have your standard API logs on a graphql server you probably aren't knowing what's going on it's just you

know you can't see anything um looking at a graphql request so I I lied a bit before when I said that mutations are IPC everything in graph is an RPC the way a graph the way graph queries work you declare a query that gets compiled in real time basically and then you declare an option name an operation name until it's run that query so again this is is why we have the 200 responses um what this allows us to do because it's so nicely structured you can probably log just this and get a good idea of what happened now you don't have the variable so like you would have an ID usually in rest logs and you won't have it if you

just log the query portion but you can still sort of see what they did and since probably you control the graph parage unless you if you have a public graph ser you can't control what quer is coming into you if you have like public consumers but if it's only your employees consuming the graph B API you can make sure that they don't have any sensitive information inline that they properly put in the variables and then you can log the query part and be pretty safe and not worrying about all the other stuff and that is almost functionally equivalent um it's not quite function with P three standard rest logs because like again a rest log you'd see the ID

or probably a search you might see it in the in the end point that you're hitting like as a get parameter but you you're you're not logging that here um and it's not that you can't log user input but it's that unless you have a real um need to have that level of logging unless you have a real application that's really sensitive and they want to know everything rolling back and we don't care it really is not as worthwhile to log the actual payloads you don't you want to avoid having user content in your logs um don't just log put your logs in this three bucket and walk away so one thing I see a lot is people

who they they they trap I mean I worked at one company and actually was the first thing I did for them they're like we want to know every click on the website okay so I set up a whole thing every click got posted and saved into an S3 somewhere and nobody ever looked at it again right if you're just dumping everything into a place without any tools to look at it you know it's pointless so if you just toss your logs in a free bucket that's not great you need a way for you to actually search it right so you can go and do investigations and have it ready because if something happens you're not going to

want to go and figure out how to search at the start of Investigation you want to know how to do it already right so using a loger that outputs a structure Json is uh pretty important these days it allows the the the seam and other tools that can parse logs can parse the individual things as individual um they can parse the the payo as an object and see the individual parts of the object instead of a line that's harder for them to parse out what is what so if you have a scene that's automatically acting you can see the time stamp is a is its own it's an own Fu on the object instead of just being

on the line right right it can it can interact with it better and you can search it better because you can search specific Fields right the Json structured LW Union J unless you have different fields in your loger instead of everything of just one Big Blob of line um it's really just a matter of outputting that log line as a Json uh object um there are Frameworks that do it for and they do really complex things um loggers will are loggers are really good for helping you add in extra information for happened there so if you have a good logger and you call it it's going to it's going to AOG what you said to log

so you know log user click this right it's going to log that but it's also going to log the time it's also going to log the user ID it's also going to log um bunch of extra data about that and a good logger will automatically add that in for you and have facility for you to add in whatever you want so good log would have facility for you to say I want every log structured log every structured log output in the log to have a field that says whether the user is 10 years old right I could write that for you take me 10 minutes um they you can get a lot of utility out of

it and again have standardization um so struct log very important provide a service like a like a an elk stack or something to search um or documented method to search there are a million ways to search ns3 about um and so there are million weights load it search it efficiently but have a method ready to go make sure if you are logging information that's user content or you're logging user actions that might be sensitive log your stuff securely right again don't put in my but that's open to the public right make sure it's been put in a place that's properly made for storing that sort of information um and finally run drills it's similar to database backups if you

you know there's old saying if you don't test your database backups they don't exist and they won't exist when you need them um if you don't test it you can look at your logs you're going to have trouble looking at them when you need them and again at the start of an incident is not when you want to try to figure this out so have a documented method or something where you can go ahead and do the drills oh there's an invasion um so that's most of my talk um we covered three basic ideas planning your security features like features build a code base where you're really helping the developers doe secure behavior and pay attention to what

you're logging please thank you again a huge thanks to my company Aon for sponsoring and sending me here all our other sponsors everybody else all volunteers are are you going to interrupt me I need to take questions give you a minute any questions yes I had a question on expanding on the drills what's like a good like it's always hard when you're like building a feature oh sorry uh I wanted to expand upon drills it's always really hard to try to like budget time like what's a good way of like selling these drills and Chaos testing and play to in my application we've been able to do it simply by um following up on support requests to the nth degree so

you get a support request you want to know what happen it's like you know I'm going to figure out exactly what happened and that way I know I can figure out when I need to know exactly what happened in other cases so that's the easiest way to do it um selling it otherwise it might be hard selling security features security things is hard that's one of the reasons you want to build it as a feature because security improvements are usually seen as a a cost they're just a stupid cost there are some cost we have to do this it doesn't give us anything if you think of it as a feature I'm building out this new thing for our

users so our users are more secure you can get more buying so that's a huge part of it hey Damon I finished your answer I just want you take note everything here is still sealed in the original including the silverware this is an unused bow thank you this is my outrageous speaker reest we have one uh and we had a little trouble getting kosher ice cream so we got a few of these that we're going to use instead I'm about to attempt to create a kosher Sunday got it I'll take more questions

yes so like most uh most things it comes down to do things well from an sdlc perspective right I'm sorry hello okay sorry uh so like most of most things with related to security it comes down to have a good sdlc process right that's a huge part of it um I didn't want to focus on that cuz I wanted to focus on stuff I haven't heard before but yes and you can see a lot of that is the process I'm talking about yeah well I I want to plug two different things one is immutability um which you know you don't have to log things if they exist Through Time right yeah immutable records and that's sort of like the the the the the

the the the version backups right which also lead to dry right don't repeat yourself so if you're going to have data in a database you don't need to log it if it's immutable right any more questions but oh it's important to not logs are different than your database so you still might want to put things in your logs that are also in your database there are good cases for that logs are a different thing n sure any other questions well thank you thank you all for [Applause] coming