David Lindner: So You Are Comparing a WAF and RASP?

all right well thanks everyone for joining um i am going to talk today about rasp and waff uh hopefully those mean a little something to you if not i will walk through each product type and kind of dig into them in depth hopefully so i am david linder i am the the cso at contrast security uh i actually lead our internal research teams as well so kind of a broad background been in in afsc specifically for the better part of 20 years mostly in consulting and so on and so forth a lot of data engulfing uh huge into iot mobile stuff um and go hawkeyes i although right now i think i'd rather be up in calgary than

down in the us so you know not so breaking news software applications are vulnerable uh [Music] big light bulb right uh of course they are uh they've been vulnerable since the the dawn of time uh and frankly over the years you know not a whole lot has changed other than us creating more and more software and creating it faster you know you look at the the average application today um and this is based on data uh that we have you know about 21 of the average application is actual custom what we would consider custom code or first party code um which i guess is being called now uh where your developers or someone you know in your organization is actually

creating and writing that code and in that code we're seeing you know close to 27 serious vulnerabilities on average uh and then you look at the the rest of it right i mean all of the rest of that code is something you're using so you know seven and a half percent is coming from uh somewhere else a third party uh maybe it's open source wherever that may be right and you know for the most part um we don't see as many vulnerabilities in that custom code today um but that being said it's still a major problem and you know that could be a whole other talk um but we are in kind of a software

security crisis um but the reality is is we have been right you know it started off by saying you know uh software is vulnerable right it is look at over the years the number of things that we have tried to do uh to make things better and and we've done a lot of amazing things you know all the way back to you know formal methods methods we had vulnerability disclosure uh dynamic scanning os top 10 in 2001 can you believe it's been almost 20 years you know next year will be hopefully another rev of the oas top ten um so that'll be kind of the 20-year bang for that um you know and ever since then you know

static analysis different compliance metrics where they're you know they're baking in the top 10 and application security into you know pci and missed 853 and you name it then we had laughs and and you name it we we've gone through the gambit of change um in appsec specifically but it really hasn't changed uh the fact that people are still attacking our applications um you know we we have the luxury of having a bunch of data of the types of attacks that we've seen and uh in our yearly report uh from the last year or so we noted that there are just over 13 000 attacks per application each month um which may sound like a lot it may not

you know obviously some had a whole lot more than others and you know that kind of weighs into it as well um but you know only you know two percent of those attacks would would have actually been successful or reach a a you know a vulnerable type sink and we'll kind of get into that as well and also one thing that we noted is you know vulnerabilities per language um we're actually a little bit different you know the the ones that were targeted you know via java.net applications specifically uh we saw a little bit disparity uh between those two um and then you know you break it all down and you know the likelihood that

your api gets attacked versus the likelihood of having an actual vulnerability you know we can tie all these things together but what is it what does it really mean and it comes back to this this software security crisis if you will uh you know this is something that i've been talking about for quite some time um you know we only have so many specialized security staff that can help with uh all of the code that's being developed i mean we're creating code so much faster and there's more of us right there's you know 30 million plus professional developers in the world creating billions of lines of code there's not enough people not a security people there's not enough tools or

budget to keep up with that right so we have to have like a broader approach because the the reality is is that the economics don't match up i don't know if anyone here knows of the bsim uh so the build security and maturity model uh every year they go out and they they work with organizations to see how mature their software security group is if you will and the most recent one uh the bcm11 that was released it might have been a couple months ago um they basically said the median of number of software security group people uh to every developer is about one in 159 and that's only getting worse the bsim 10 was about 1 in 73

you know so that right there tells us that we've got to we've got to have a bigger broader approach so one thing that you know we tried to do early on was hey we know firewalls right you know way back when in the early 2000s like that was the thing you know make sure your firewalls are good make sure they're not allowing things in or out that they shouldn't make sure they're patched all this stuff we're like hey we know firewalls why don't we create a firewall for web applications uh so you know quickly we're like hey we can probably do this and quickly we deemed that the laugh a web application firewall right so you know we tried to create a

firewall to solve a problem that wasn't really a firewall problem so in 2002 mod security was born i'm sure most of you who have worked with mod security have probably done it for a while you know it's been around for you know again almost 20 years and it's definitely come a long way in that 20 years from when it was first born but we quickly noted that there's definitely some issues right um it is a layer seven thing so it's it's used to stop scanning potential attacks against the application layer hence the web application firewall or waft name right um it does require a lot of tuning so you know there might be bypasses or

maybe a framework comes out with some new vulnerability uh that was discovered some cve or whatever you've got to make a patch to your laugh it just there's a lot going on there um you know i've seen things from folks where it's like hey uh does the rasp cover these things you know because we've had to patch all this stuff for our laugh i'm like yes out of the box right and then alerts so for those of us who have worked with laughs for a long time like you know if we were face to face i'd be like hey what do you do with these my guess is not much right it's there for kind of peace of mind it's there

maybe in some cases you're using it for some you know threat intelligence things like that but for the most part there's too much going on it's blocking too much and the only time you really look at it is when you need to tune it because it's either blocking legitimate traffic or it needs to be updated uh or tweaked to block traffic that you know is malicious so how does a waff work um so laugh is i don't wanna stick but but the reality is is it it really is it looks at the incoming traffic at the layer seven it looks at the application http traffic and it says hey does this look malicious and the way it does that

is it says hey here's a bunch of other expressions or maybe it's got you know specific keywords and things like that that it may be keying on to look for in that http request headers body cookies uh you know method type whatever it might be and so in my example here if i submit contrast or one equals one dash dash that looks like i'm trying to perform sql injection on the back end so if i submit that right and the laugh sees that more than likely it's going to block it saying hey this looks like sql injection someone's trying to be nefarious or malicious um no reason to let that go but what if i

told you that the back end wasn't actually doing sql or there wasn't a sequel call um the laugh doesn't care or know what's going on beyond it right uh so similar here what if i change the input to semicolon cat etsy password which is a you know pretty well known uh proof of concept for command injection to get something to run on the local system in a you know command context so laugh would also look at this and say hey this looks malicious or nefarious i'm going to block it or i'm going to you know whatever my rule might be i'm going to take care of it uh and not not really spend any time

uh you know investigating anything like that and these alerts probably don't go anywhere they probably go in some bucket and no one really ever looks at them because you know we're comfortable with it so one analogy that i use uh is you know going to a zoo right so let's say there's a zoo and there's this this new fence that they put up outside the the monkey cage and it's a very smart fence and that smart fence only allows bananas right so uh when the guests come and they visit the monkeys they can buy a bulk of bananas and throw the bananas uh through the fence and it lets them through anything else it'll block them in and

say no sorry you can't throw those apples oranges whatever else but what if i want to allow some of that stuff right how how would i and how does it determine when uh the difference between you know a banana an apple and orange whatever it might be um so again it gets back to you know and every laugh is a little bit different it does all sorts of scoring and things like that you know we could spend a week talking about the different scoring mechanisms and methodologies that the wafts use but for most parts it's using regular expressions so it's got a bunch of these regular expressions based on input types you know whether it's a header parameter things like that

uh and it looks for you know maliciousness and things that should just never come in in the input context that it might be coming in right like so you see like the on click events right any of you who have been a part of bug bounties or uh bug hunting or anything like that you know when you're looking for xss a lot of times using those on events will prove out your proof of concept right but you know you look at the the on events here there's only what three of them but in reality if you go through all the browsers there's a almost 400 that need to be blocked so it's one known area of concern in a

laugh is you know out of the box they may not be blocking all the things that it should but a lot of that gets back to hey if it's blocking things that it shouldn't now i'm blocking real traffic so now i gotta tune it because i i know i want more of the bananas i want my monkeys to be happy uh you know they they like other fruits and stuff like that so i'm gonna tune it so now it's gonna allow bananas apples oranges uh you know whatever else the monkeys wanna eat um and so i have to tune in and this gets back to that really understanding in depth what's going on in the system and anyone who's

ever you know added a mod security rule or tried to tune one knows that they're they're ugly uh and and it's not the most straightforward thing you look at this you know request here and it's and it's talking about different mail it's talking about white spaces different character cases uh system commands it's not very straightforward i mean you basically have to have someone full-time that's doing the tweaking that's tuning these things um and that can really understand the the quote-unquote language that is needed to apply to you know mod security or whatever waff you may be using now some of the laughs like the aws left and things like that they try to make it easier for you so

you don't have to really get in depth like that but i've seen a lot of things show that mod security is actually probably the best one with the olaf's rule set applied to it um but then again you have to have someone manage it so but we run into problems still right um laughs have been known to have lots of bypasses over the years um you know back to our analogy where we wanted to allow the bananas and apples and oranges but what if i send a you know a rotten apple over it still looks like an apple to to my my specialized fence um and so it gets over and then my monkey gets sick

right so you know they're just ways around that laugh um it is it is a good control layer i'm not going to say that it's very good at bot blocking it's actually very good at cross-site scripting detection things like that and it definitely plays a major role in an overall architecture um you know but some of the common downfalls we've already kind of talked about you know there's alert fatigue uh which is a huge problem in uh cyber security today uh is alert fatigue i mean you know i've been on teams where we're using systems and it's like oh my gosh we can't even tweak this to the point where it stops the alerting so we just

shut it off well then it's not real useful right uh patching fatigue like i said anytime a cve comes out you've got to usually apply some patch so it can detect the signature that that attack uh you know that attackers are using to to exploit that vulnerability uh and it stops real traffic um some interesting bypasses this is actually interesting if you go to like twitter or something you can search for the hashtag laugh bypass and things like that and find lots of these um there's all sorts of really really smart people out there and they're always going to be looking for bypasses some of the interesting ones here are the some of the command injection and

sql injection ones uh sql injection one is is really interesting with the slash star slash it's actually seen as a space um and you know for for certain interpreters and things like that it bypasses that that laugh because it's looking for spaces because to them without a space you can't inject sql while you can with some certain back-end sql interpreters so um again you know laps have their place but they also have some some known issues so you know incomes rasp uh so runtime application self-protection this is uh i would say it's still in its in its infancy uh you know it's pr it's been around and it was coined probably around five years ago

um and it's it's a little bit different and then a laugh and how is it different well it's the same it's it's at the layer seven right it's used to stop real attacks um but instead of being a perimeter based thing now we're inside the application it's living in the actual application itself in the vm the jvm whatever the the application is running in um shouldn't really require a whole lot of tuning which hopefully results in less experts needed and also hopefully uh results in very very few alerts uh with low false positive rates so we don't actually shut down actual uh valid application traffic some things that are different um is it has more vision right it can see

into the application because it's living in the app it's instrumenting the code and know the context of the data and the attack something that a laugh will never have and can't because it can't see into the application um you know so it has more intelligence it has observability into what's really going on uh you know we can monitor security like we do with say new relic i'm sure everyone here has used something like a new relic uh to to monitor the performance of their applications it's very similar right it's living in the app and it's monitoring what's going on uh and it's really anywhere anywhere that application can run this can be run as well right so you can

enable security into it so i mentioned so instrumentation has traditionally been used for debugging uh in the developer world if you will uh but over the past few years with your your agent-based observability tools uh new relic is one that comes to mind as well they started using instrumentation uh for observing observing what's going on in your application and this is really important for rasp because it can make much more intelligent decisions about what it detects as potential attacks so as we go back and we look at some of the things that what i showed you before the the different types of attack against the olaf so if a rasp sees this sort of tactic

contrast or one equals one dash dash and it says okay this looks like an attack it doesn't just stop there so what it does is it says hey i'm going to follow this through the application while i instrument through the path and see where it ends up if it ends up in a sql query and it doesn't have the prevention mechanisms like you know primarizing it and things like that i'm going to block it because guess what now i know that hey i've got tainted data coming in from some request and it's ending up in a location where it could potentially run against the sql database so rash say hey hold on we're going to

block that and now when when the the sock team or your absolute team or whoever is managing this gets an alert they're very confident that either one they know that they have a vulnerability there or you know two they need to you know get on that uh and get it fixed um and and retrospect you know it's it's one of those things where you know it's probably something we could have done years and years ago but uh just never really got there and now like early on the olas days of a friend of mine created um oh gosh i forgot what he called it it was a it was a tool kit that you get

like a java filter uh that you could uh put into your application and it would detect attacks like that uh but it was never really thought about using instrumentation uh so now we look at the similar uh attack that we talked about before in the left the the semicolon cat etsy password and since the rasp knows like hey this is ending up in a sql sync it sees this says all right this looks nefarious looks malicious someone's obviously trying to do something they're not supposed to but it's never gonna succeed it's not ending in a place where it's going to ever succeed so i guess i'll let it go for now but i'll probably mark it as is like a

probe or something like we're seeing something uh but no action is really needed right so the real difference here is when i see a an attack i know if i need to take action or not versus in a laugh i don't really know so what we can do is we can we can break it down so i want to talk through one uh kind of in detail rasp example and we'll look at sql injection so the first thing it does is it it has to know about sources the sources is something that we are a rasp has to label uh as tainted data coming in from somewhere uh you know in this case you look at get

parameter that is an http request get parameter is just a parameter that someone is sending in like a username or or you know favorite color or whatever it is but since that's coming from the end user coming from a tainted location or say this is a source so this is a source of taint um so in this in this example here user is the taint so we say get parameter is our source and user and the value is the team and on the other end we have to know about the sync so where does that request end up where does that taint end up um so we know what the source is now we need to know what the sync is so

if the the sync is execute query now i know that my chaining data is ending up in a sync that in most cases is vulnerable right so now i've got the full picture of what's going on so how do we how do we detect this in a way where we know we can still be very consistent and very very low risk for false positives because you know we could just say like a weft like hey this looks like one i'm gonna block it but we we you know a rasp wants to take it one step further than that so it looks at the the whole story you know because the rasp is in the code

it's instrumenting the application it sees exactly what query is being committed or being interpreted by that sequel um to to be able to to make decisions on so in this case it's select star from users or user id equals star and password is password1234 bang right um so that's what it wants to run so it's grabbing the user that's coming from the taint from the the end user and it's just putting it directly in this dynamic query so how does a rasp look at that and say all right okay now i've determined like in this case you see for the username i i entered test at example.com or one equals one very common uh hackerish type thing to see

if the application reacts differently so what what the rasp can do is it can break that all down uh it can it can uh para or not primaries tokenize the actual sql query and know exactly what's going on and and determine if there's been boundaries crossed right so for a user id it's expecting a literal that's it right but in this case i'm getting a literal i'm getting an operation and then i'm getting some sort of expression uh and then all of a sudden i have a comment block completely different from the original intent of that sql query and the rasp can detect that something's changed and a boundary is being crossed right so and this is all done at the

sync so it's like okay we're going to do this execute query and the rasp is going to step in before that query is executed and really dig into what's going on and what should be so it's basically kind of a sandbox if you will um but it's going to look at that full query and be like oh wait a sec this shouldn't succeed uh someone's trying to be malicious here uh and we need to uh we need to block it right so the rasp gets to be really really intelligent here versus you know the waff uh only sees that input says ah looks malicious i'm going to block it right so so then we we send alerts to

somewhere or we block that action and you know the rasp would then send an alert you know however your sock might work maybe it's your asset team whoever would get those alerts it can go to a sim uh you know wherever you send your your alerts to and those alerts aren't going to be nearly what you would get from from a laugh um you know you know alerts that are blocked and things like that and you know you may not even want to see blocked attacks you just don't know you know a lot of folks utilize a rasp to put into you know more antiquated applications that and i'm sure some of you are understanding that there's a lot of old

old applications out there that no one wants to touch anymore and the person who wrote it 20 years ago is no longer with the company and people are scared to touch it because it's going to break something so they throw a rasp in there to make sure that there's protection for some of the more egregious vulnerability types so to really look at this from a different perspective and what what a rasp really is it's like a monitor like a tire pressure monitor for your you know you get that instant feedback and you know like you know your tire monitor says you're at zero you're probably at zero right like it's the same sort of thing

and and now you know you need to take action and fill your tire up same sort of thing with the rasp you get that block you get that exploited depending on what what mode you have it set in and you know you should take some sort of action or need to forensically examine something [Music] and this is what it looks like right so way different than a laugh um you've got your your high level uh volume types command injection chronic scripting crest forgery so on and so forth and you're basically just turning them on or off right there's there's no real tweaking um you're not you know creating these esoteric rules and needs someone who's like full-time

uh tweaking these rules and applying patches and doing all these things all the time um so you know it's it's just a different mindset of similar technologies but they they have different insight and observability uh points into your application now perhaps sounds great right well there's also downfalls of rest as many of you can probably uh think it's very i mean very language and framework specific since i'm instrumenting or the rasp is instrumenting some application it's got to know that this is java then it's running spring xyz version that it's using time leaf that's using this that and the other because it has to be able to follow that input all the way through the

application from source to sync right so it's very language and framework specific so if organizations are tweaking frameworks if they're you know bringing in a framework and rewriting it you know whatever they might be doing there may need to be some customizations done by the vendor the rasp vendor to apply to your frameworks unfortunately bypasses so yes there's bypasses there's never going to be a product uh that's perfect um bypasses most of the time result in some sort of code change it is a nature of rasp um it is a little more difficult to install you know because it's it's very language specific so it's got to be installed usually by different teams right so this

is going to be probably your devops team or your cloud engine team or whoever is managing your your pipelines they're going to be the ones that need to you know make sure the agent is installed when it needs to be installed and things like that and the other thing that's been really hard for folks is we're so used to laugh um in a rasp you oftentimes don't see anything right because uh it's it's so specific and false positive you know are very very low it's not like a laugh where you're seeing all sorts of traffic all of the time um as i mentioned early on these application security is starting to get into standards not starting is really in

standards and rasp is already in they're in fisma in the 653 uh they're they're they're in pci now um so rasp is definitely a thing and and i don't see it going away anytime soon so real quick i wanted to compare the two um because i think these are some important things uh my team here did some comparisons between laugh and rasp um you know and we looked at kind of the differences between the two um and we noticed that you know you know since you know we have customers and rats and whatever else and they're they're frankly a lot of times they're they're comparing them as they're one in the same uh so we got some some info from from

one customer and said hey this is what we did we sent these payloads to this laugh and sent the payload to the rest and this is what was blocked this is all that was shared and as you can see the results are all over the place right there's so much missing context here um the testing methods you know they're just inconsistent because you know one of the questions that that i've asked folks like when they're comparing the two is like hey what is the back end so are you running a a back end that is actually vulnerable to those attacks a lot of times it's like no it's just a dummy uh you know web service or

something right well then for sure you're going to get different results from a laugh and a rasp because the rasp is only going to really notify you when it's a problem versus the the laugh or it's just going to block it and so the reality is the results a lot of times don't really mean a lot um and so we dug in a little bit deeper to this as well and for those of you that are doing some of these evaluations um we went out and we looked at all the popular payload lists you know uh exercise radar awesome laugh payloads all the things is really the big one um all these different things that have

all these built-in payloads to see like you know what what are people testing against these waff and rasp products you know what what can we learn from it and quickly we saw that you know most of the payloads twenty percent weren't even valid or there were duplicates in xss uh same sort of thing with sql injection 27 not valid and what i mean by that is like they would never even result in like there there was one in xss that was something that was used in netscape netscape hasn't been around for 10 plus years right no one uses netscape maybe it may be some guy in some deep dungeon and in the uh dod or something but you know

it's just it's one of those things that these haven't really been curated so we're using inconsistent testing methods and then we're using tests that aren't even valid so now it's just all broken right i mean command you look at patch reversal 82 82 percent on average of the path traversal um payloads were not valid or duplicate of you know what of all these different payload lists so it's just one of those things where um [Music] i think they're different and why do i think they're different because it's all about the data it's all about the visibility it's all about observability into what's really going on so you look at what i call the data from the past

right you're running your sas tools das tools you have some bug tracker you have your waff and then you have some running application but you don't really know what's going on in your running application right unless you're doing some extensive logging and uh you know correlations and things outside of it into some sim or whatever else you don't you don't have a lot of insight there and there's no real way to even tie laugh logs to sassed and dast and there's no actionable items and it's just kind of broken um but with a rasp involved now we can start to tie this whole thing together and really understand um what is going on in our

you know entire application space um but i still think that that waff is still important to be there because it's still good at some other things um what one analogy that that i love is looking at you know av and where it was and where it's come right and now xdr right so you you start to see the evolution and i see this with laugh and harass right it's evolving um but i don't think you can get rid of um a laugh quite yet right so i think you need to do both and and this is why um so hackers are taking advantage of lag um and you know unfortunately uh equifax was one of the companies that

had this issue right so there was a cve disclosed it was apache struts issue uh there was no updates equifax breach occurred you know a couple months later they didn't detect it for even a couple more months and then it was disaster right so that being said a laugh can help in this case right because you know it could have been patched or whatever to look for the specific um you know attack string and things like that that were going on but that still requires some sort of update somebody's got to apply somebody's got to tweak things um which still needs to be done because if you stop at the perimeter that helps with the application performance

but harass if running would have prevented this sort of attack because it works completely differently right so i look at it this way if i'm driving a car um if i had to choose between a seat belt and an airbag what would i choose and i would say i still want both right because one is like you know proactively uh preventing things and it's annoying right like you hit the brake a little too hard and that that seat belt gets stuck and it's a little annoying but guess what it is helpful right if you have to stop really really quick but then you know if you hit someone or someone hits you that you've got that safety net that

airbag that's probably gonna break your nose but guess what it's probably gonna prevent some more serious things from happening so in tandem they work really really well together and that's really how i view waff and rasp um so this is the the kind of the chart that i get to the strengths and the weaknesses right so laugh has weaknesses alerts fatigue false positives false negatives patching tuning required and no real actionable results there's not a lot i can do with um those alerts that i'm getting from a laugh right other than maybe block an ip because you can tell that someone's just running some scripts or exercise payloads or whatever at our application on the other hand

the the rasp strengths kind of match up they're accurate there's often no results very little tuning required right uh and the results are typically actionable so in tandem i think they work well together but on the flip side you see the rast weaknesses performance is definitely impact i saw someone asking about that um the performance impacts of rasp you know i've seen everything from nothing to you know depending on the application i don't want to lie i've seen 30 percent and that was because this customer or this person was sending uh huge binary payloads uh and obviously the rasp is trying to process all this right so it is a potential issue whereas a laugh

there's really no performance things that you need to worry too much about because it's outside of the app right so if most of the things are getting blocked outside of the app i still would have my ref rasp running and hopefully you know maybe i can even shut off some rules if i'm comfortable with where my wife is at right uh rasp is language and framework specific no way to get around that um you know there are there are a lot of semi rasp tools in the market but the reality is true rasp is actually running in the vm it has to know the language it has to know the frameworks period whereas the waf language agnostic so you

can put that in front of things say you've got some some old php or or something that's running that there's no real rasp for you can still have your your wife running there too one of the weaknesses i didn't talk about four for rasp is it's not environment aware right uh so environment aware to me means that it doesn't know what else is going on around it um you know so that sometimes does result in some issues uh whereas the waff might have a few more um pieces of understanding what's going on in the environment and in the environment around them um so i think you need to use them both um you know i think someday

they'll kind of merge into one where some of the rules like cross-site scripting and things like that are probably better at a perimeter and easier to detect a perimeter um that would move those out but i'm hoping someday they'll be you know kind of a single solution and that's it um so if there are any questions let me see uh yeah you can definitely use the charts um i can even probably try and link to the uh yearly report that's freely downloadable um i think i kind of went over the performance impact um ras definitely has performance issues uh no doubt about it um and there's you know there's no real way around it um you know because there's there's so

much going on when it's instrumenting your application now it definitely doesn't have nearly the performance impacts that you know i asked or uh the full-blown interactive application security testing model has because it doesn't have to do a lot of the instrumenting of the propagators you know through the the way the data flows but there definitely are performance impacts um scalability not usually i mean so uh internally we use our own rasp in our systems um we're a heavy devops shop uh you know we spin up uh new systems frequently when load changes i mean it's it's it's really easy i mean if you have everything done as like infrastructure as code and things like that i mean it is so slick

um and you know it's it's just it's really really easy to to spin up and spin down and you know continually add and remove and scale cool anything else awesome well thanks everyone for for joining me today and feel free to uh reach out and if you have any other questions have a great one