BSides Glasgow 2018 - Paul Johnston - Replicator: Helping Developers to Replicate Pen Test Findings

then everyone thank you for coming to see me talk to you about replicator now mmm this is a burp extension I've developed to solve an issue that's that I've hid when I was working as a pen tester I now work at port surrogate and as well as helping out Burt users one of the things I do is look after the map store and now the story of this particular extension is um when I was working as a pen tester I often found there was difficulty getting developers to be able to understand some of the more complex issues so you worked really hard you'd be three steps into this multi-stage process you bypassing JavaScript valid as you bypass some

server-side filter and you get this really great vulnerability but promise you you'd write this all up and the recipient they would struggle to know what to do with it you get follow-up questions by email they'd want to have a conference call to discuss it it was a challenge and what I found from working with developers is what they want more than anything is to be able to reproduce an issue on their workstation you can give them a strap test you can give them a cord of you could tell them the exact line in the file that has got a book you could even give them the link to stack overflow with three different highly recommended solutions and still what

they want to do is replicate it on their workstation from maze with some of these difficult security vulnerabilities these issues can be hard to replicate now well I'm always doing an earlier draft of this talk I made a minor Freudian slip on that second bullet coin and I wrote reproduction can be challenging and what I was trying to get at is there can be technical things like for instance there can be all sorts of technical problems that come okay

but that can occur why are you trying to do this and a good example would be if this JavaScript validation on the site so you go to the forum you put in your payload in a text field and the JavaScript on the site kicks it out now for a pen tester that's easy to get around but you have to use some kind of security tool right Bert proxy or one of the other lesser tools now one of the ways that my previous employer tried to deal with this was to keep a full copy of the request in the report so you report real Oh we'd have the title have a description of the vulnerability to

say hey you want to reproduce it here's the request so the idea is the developer they'd copy that they'd paste it into some kind of raw HTTP tool and they'd say this issue we produce the problem however is that there are normally session tokens in that request so what you're testing you can paste that in and the issue reproduces now by the time you've sent this report to your internal QA let's come to the account manager and it's been passed to the security manager who's gone to a project manager and eventually it'll lands on some developers desk is going to fix this by this time the token has expired so they fire this will request it in

their app and they don't get any meaningful response so given all these issues there is a need to do something better and what we thought is what if we can make replication as easy as saying tea Earl Grey hot and I'm going to show you the solution that we came up with

apologies if you can't read this at the battle is it that there's a bit of a trade-off here between the font size and actually being able to fit everything on the screen now just put your shoes yourself in the shoes of this developer so they've had the Panthers commissioned they've received the report and it's got these findings on it now because the pen test company that did this particular job is super advanced than they're using replicated files as well as receiving the PDF report you also receive a replicator fire so you fire it your trusty copy of burp suite and you can in fact do this in the communtiy Edition and we load the file that we've received

from the pen test company now we've just got a little security warning here because it's gonna update our configuration but this is cool oh yeah I very much trust this pen test company so I'm gonna say yes here now what we've got we've got a list of all the issues that have been found in this pen test so this there's three in this particular pen set now and the very first thing that I'm gonna do is I'm gonna see if I can reproduce the issues that have been sent and that's all it gives us a single button to do this this is a slight moment of truth about whether my demo labs actually work or not so we're gonna

hit test all and all of them has come back as vulnerable which at this stage is exactly what we want so the pen test company have identified we're vulnerable and now we can reproduce that locally now in this example we have the pen test firm work on the staging version of the site and but I'm not actually going to be deploying fixes there and initially I've got a private dev copy of the site and so what we can do we can redirect the target now when we say yes to this do we want to change all the targets and the request macros it's going to do some magic under the hood this magic and hood was significantly

more complex than I thought it was going to be initially but hopefully it's going to work now just as another test we're just gonna check that all these issues still replicate because there's always the the concern that you've got a slightly different configure your dad versus your staging so the next step just test them all and thankfully they're all still horrible okay now we can start working away at each issue so the first one SQL injection the pen test report this is going to have a description of why SQL injections are problem is what the business impact is and what strategies you can use to fix it and they hope that these reports are going to be lovingly

handwritten and not in fact copy and pasted from the burp output originally so having read and understood this I'm gonna go to my source code and I'm just I'm just running a local VM here that's got the vulnerable app on there

now form a thermos file a little bit and so there's two two lines here they both decide week one of those vulnerable to SQL injection and one of them aids bizarrely in Python the two lines look so similar it's ridiculous but this one this secure version that's currently commented out I'm going to make that active and I'm going to comment out the vulnerable version okay so we think we've done effects the next thing we want to do is prove that this fix actually works now we can go back to the replicator tool here and we can try testing it again right and it's resolved which is good now there is a little caveat in bracket so it says resolved

brackets sensitive now this has been a necessary inclusion based on some of my experiences dealing with developers because of course what we want is for them to address the root cause of the issue so if there's SQL injection we want them to move to having a parameterised query unfortunately time and time again I've run into fixes where they've literally blocked the attack vector that we've reported so there's always the worry that some different attack vector is going to you know mean the vulnerability is still present now one thing we can do to help deal with this you can do an active scan at that

oh that's fine okay there's some really minor issues there that we don't greatly care about but the crucial thing is this SQL injection issue seems to be fixed so that's great we can move on so the next one on the list was cross-site scripting now similarly we can we can look through the code and we can find in this function here the developer is following very poor practice in that they are taking a variable from a form parameter and they are dumping it straight in HTML without any escaping which is the classic cross-site scripting attack now if I uncomment this line here it means we are going to escape less than characters so I can quit out here I can

rerun labs I can go back to replicator and I can try testing this issue okay so it appears to be resolved and that's if we look at lower down on the screen you see this bit here where previously there was a script tag echoed the less than sign has now been escaped so this isn't this is no longer gonna fire up JavaScript now just like before it might be a very good idea to do an active scan to see if the issue genuinely is resolved right now this is more concerning because something has popped up on the target tab and this time it's red it's a high-risk issue and it appears that we still have cross-site

scripting and in fact if we look at this more closely we're better looking at the response to see this what the active scanner has done there is its founder sir native payload that's got through our very simple escaping so the escaping was only removing a less than sign in this example the injection was into a value tag so it's possible to have a across that scripts before without using the less than character I don't know if you've used this particular trick before this is one of my favorites for going into an input tag and that you can use the on focus handler to put some JavaScript when the when that object gets focus and you can

also give it the auto focus now so as soon as the page loads up you don't need any user interaction your JavaScript payload is going to execute and soon from there you can continue working through these things to fix all the issues and then that would be a very good point to send your application back to the professional pen test company and have them do a retest and hopefully you're gonna get the best benefit out of your retest because I I can think of a number of times when I've been asked to work do a retest for a client and when it comes look at it nothing has changed in the behavior of the area that was

vulnerable and even if this can't prove things are fix it at least should stop that

okay so the the key points that made this good for a developer everything was embedded into a single file so it can be passed from the pan tester to the developer it can be passed around between development teams it'd be chosen as share points replication can be done in one click so this is this is quite in line with agile and test-driven development it's the sort of behavior that you want it's adaptable to the environment so the fact that the Pens have worked on bla bla bla staging comm doesn't prevent the developer then working on dad calm or or whatever whatever other environment but the big thing to remember is that this is not a silver bullet in itself and

that a manual retest is the way to prove that the issue is properly resolved ok so that was the flow for developers um but I think a lot of people in the audience here at Penn testers and you'll be asking what Hamilton we make this replicated fan we need some kind of like workshop to help us craft this and I'm going to show you exactly how we go about doing that now ok so first thing I'm going to do is just to clear out what's in replicates and now you may not know this if you hold down control when you click this tick and extend the tab it does a quick reload I only learned

that quite recently I wish I've known that ages ago and and what we can do is we can put replicate into tests of you so it's going to we get a few more buttons and options now now

so I've been asked to do a pan test of this my demo staging comm and in fact it turns out we've been given a a test account to use as well now I'm going to focus on the kind of burp heavy workflow of doing the pan test and we're not interested in my demo dev at this point just that's my demo staging and and I'm just gonna okay so I can start by spidering the house just to find all the all the different pages within the site and we can do an active scan

okay now some of these I'm just I'm just gonna delete because either they're low-risk issues or because of the environment we're in we don't actually care that the password is is playing so we found this issue cross-site scripting reflective so this is something that we want to communicate to the developer and we can right-click it and we get a context menu option here which notes to send it to the replicator now we've been replicated because this was found by active scanner is able to automatically pick up most of the details so we've got a title for the issue we've also got the path and parameter which are normally key bits of information that developers want to see and here on the left there's

a probe request so this is a request that will trigger the vulnerability and the other thing we've got is detection logic in this example the intersection logic is a graph expression so with this regular expression matches the response we're going to treat it as vulnerable there is also a collaborator detection master that you might want to play with if you if you look at this later on so if we can try this now and if we test it it shows that the the website is vulnerable

I've just realized that I'm that I need to undo to demonstrate sort of it I need to undo the face okay now you this scam didn't find the SQL injection vulnerability now I don't like to rely fully on automated testing so let's do a bit of manual testing as well now the the go-to starting point for any manual testing is a single quote so if I've got a single quote this field instantly I get an SQL error message so this is ringing all sorts of alarm bells about this has got a floor and I'm thinking why has an active scanner found this now one of the first things I can do to investigate this is to dig the request

out of proxy history you can see in the history we can see we've submitted the single quote if we just hover for a second we get handy inline URL decode and we can see the response with the suspicious-looking string now if we send this to repeater what we would expect is that we can send this off again same payload and we would expect to get these same result but we don't this is forbidden this is a bit strange so if I go back to here I will refresh the page if I put in my payload I put in my payload again and get an error message so what is going on and we can dig a couple of these requests out of

the proxy history and send these to the comparer so this is going to show us the exact differences and what we can see is that this token has changed between requests we've got this dynamic anti cross-site request forgery token so what happens is we refresh the page and we get a valid token we submit one payload and that gets through to the app and triggers the error but by that point that token is burning it's been used and we can only use it for any further probe now it turns out that Bert's got a useful feature to help us deal with this and what we can do I just remove these because these are these were set up so

the first thing we can do is define a macro so we have this request which would get the search form and the response contains this token so if I create a macro like this I can call it fetch CSRF token now the next thing I want to do is I want to configure this post request so that every time the post request runs it's going to fetch a fresh token now the first thing I need to do is capture the URL of that request and then I can create a session handling rule and I'm gonna I'm gonna call this update CSRF token in the rule action I want to run a macro I'll select that

macro that I've just defined and crucially I'm going to leave this option here enabled which says update the current request with parameters from the macro response now we also want to define a scope so I'm going to paste the URL they I just copied and because we're going to use this with replicated down the line I'm going to enable the extender scope okay so now if we go back to repeater what's gonna happen when press goes under the hood it's going to issue the macro is going to fetch that fresh fetch that fresh token it's going to update the request we can see here and then we are going to get a different response and you can see you can see

each time we do it tokens updating but it's not just good for repeater this we can now send this back to active scanner we just give this a minute oh yeah try that one more time right and we've now found the SQL injection floor now this time if we send this to work replicator now just as before it picks up the vulnerability description it picks up the path the parameter it gives us a probe request it gives us detection logic but crucially as well it's identified this session rule which applies to that request so now when we save this replicated file embedded in that file is that such a rule and that macro and then when the developer

receives this when they load it they get that configuration preloaded into burp which which helps them with this sort of you know just clicking go experience now you cannot rely on active scanner to find everything so something else that I tend to do during a test is to use the content discovery function now I know loads of people use deer Buster and Durbin things not everyone seems to know that there is a similar functionality built right into Burke which lets you leverage a whole load of other stuff like their set and handling rules so we can kick that off for inning and actually already after a little while we've already found something new on the

sitemap this admin page oh I wasn't expecting there we send this [Music]

it's need to turn on the cookie jar for a pizza there we go so if we request this admin page even though we were just logged on as the test use them with no special privileges we have gotten authorize access to the admin panel now again we can send this to replicator and but because this isn't an active scanner generated issue we have to do a bit more work so for instance we need to write in the issue description ourselves so now we've got the probe request but we also need to manually define the taxon logic now seems to me a pretty fair thing to search on would be the admin panel so if I I select that string I can then

instruct it to use that selection and now any term that probe requests gives you the admin panel it will trigger a vulnerability and we can test that now

now all these requests have got session IDs in them remember I said earlier one of the problems is while you're testing that session ID is valid but down the line the session ID may no longer be valid now we've got some tools here which can help us and deal with this so the first thing I'm going to do is I'm going to scrub cookies from all these requests so now I've removed the session ID so there's no chance that we're corrupting ourselves with sort of having a session ID that won't be valid in the future now that alone won't solve everything because it burps internal cookie jar will affect things so I need to also empty that cookie jar so now

we've simulated the situation that all our cookies have expired and those vulnerabilities are unable to replicate now what we need to do to fix this is define the login macro so I can go back here if I look down history this is the original post request where we logged in is use men's hat so define this macro and call it log in and as test now back in replicator we can select a loggia macro we now try it they all replicate as vulnerable and now we can have a lot of confidence that if we send this off to a developer that they will be able to replicate them just the same and there's one other feature

that's come in handy in some some situations and so you might have some particular bird config setup that you need to access it could be an thailand platform or authentication it could be hostname resolution it could be all sorts of things now we've got the capability here to import some config from the currently running burp instance so I'm actually going to from the project options within connections I'm going to import the hostname resolution and it so this Maps the my demo stage intern IP address so if the developer tried to run this and they didn't have this confit it wouldn't work so we can save that so now everything it all get packaged up and we can say that as a

file now obviously there is a fair amount of confidential stuff in this file and but you've already got some secure communication mechanism to get your report from your company see your client so you can just leverage that exact same communication mechanism to send the replicates file as well and then they've got the ability to reproduce all the issues

so that that's the that's the workflow for a tester the key points I thought first of all is that there is a list of all the vulnerabilities that every vulnerability has got a probe it's got a crafted request which you can prove used to prove presence or absence of of the vulnerability and it's got detection logic so it can look at the response and it can say okay this is vulnerable or this has been resolved and then to support that it's got a login macro it's got all the session handling rules and it's got the relevant sections of your bit configuration now is it so so you can serve up a wonderful post battle banquet now sometimes even with the best

starship technology things can go wrong and one of the scenarios that we envisaged happening was that you would you have a developer one and they say it just doesn't replicate on my machine and you've got the pen tester and then say well it works for me you think how how would we resolve this impasse so we've included a trace feature so if you're having difficulty replicating an issue you can turn on the trace feature so now once this is turned on every HTTP request which goes through Birds Network stack and the response that comes back gets recorded in an in-memory log so you can then repeat all the tests and then when you're done you can stop tracing

and you can save everything to a trace file and you can then send that back so the person who gave you the replicator file and hopefully between you you can debug it it will still take some degree of skills to debug that

yeah so that was all about the extension um I just wanted to add to this about some things I learned about developing extensions because this this is the first like major extension I've written that's ended up being in the bat store I've written one-shot extensions before that were to deal with particular technologies but they never they'd never been released publicly they were always kind of written to a standards like well it works for me now but if it breaks stuff so things I've learned that work really well and you really do want to use an IDE to do development and um I've certainly found that IntelliJ is very pleasant to use so compared to Eclipse

and NetBeans it just does a bit more code suggestion immediate static analysis or smade refactoring and just all those little bits they actually add ups making it look there's a lot more flow as you're developing Gradle works really well as a build system and and just like maven it lets you reference external artifacts from your build files and this is actually a really handy way to integrate the burp API quite a lot of the extensions have exported all the interface files and kept them in their source code and this this works fine there's no problem with it just put as your source code up and that's a referencing the artifacts is a really clean way to do it and if you're

building forms while there isn't IntelliJ form builder we found that the NetBeans form builder is is preferable and actually they work together fine it's absolutely no problem to be adding a form in NetBeans editing the code and in salad if you change one they ought to reload and it's is it's actually really hard to mess things up and now something I'd not done much of at all when I was kind of like a pen tester who did a bit of coding was using a debugger now if you've got your captain breathing down your neck and you to get your Wharton those cells going at what dr9 fast a debugger can greatly help you with that if you used to

debugging by putting print statements in your code and a debugger can really make things a lot easier for you and I'm going to show you how you can use IntelliJ to debug a burpee extension this isn't specific to IntelliJ but it is well I'll show you how to do at this Java has this facility to do remote debugging so if we go into our debug configuration we can create a remote debug session now IntelliJ here it gives us the command line we need to use to start Java so when you start you ever with that command line it will open up a listening port and it will allow a remote debugger to connect to the

process so if I quit out of this instance a bit now I can run Java from the command line I can paste in those command line flags for debugging and I can start bits wait so I'm just going to like let it start up a little bit it slows down a little bit when the debug is running but not not tons and when it started up a bit I can go back here and attach the debugger it's a running process now if you're not used to debug your before one of the main entry points is a break point so we can satellite in the code and in fact here I'm going to just set a line which

is run when you press the scrub cookies button so

okay so we've got this breakpoint set so when I click scrub cookies it's going to bounce us back into IntelliJ so it's frozen the application and at this point we can analyze all the currently running data structures so down the side there we've got the stack trace I've got all the local variables on this side now if he did do debugging by printing what I found happened all the time is he print something out when you saw the value of that wow I wish had printed out this as well but the DES Buca really helps you there because you can just explore all the objects in your current context and you can just expand as deep as you want

and there's also capabilities to run like inline javis going to up your game and become more of a more like a professional developer I would say that Linux user debugger is really up there with good things to do now something else a few people asked me about is highlighting tabs now you know you know we need to send a repeater you know how the repeater tab flashes up for a little bit people have asked well hey I've got an extension I want to do the same but there isn't an API to do it well replicator can do this because although there isn't a burp API you can still access all the Swing components now I'm going to let you into a secret

since Bert won seven to eight that is the exact color you want if you want your highlights being exact color match the internal tools that's the color and if you want if you want to get the five details right 3,000 milliseconds is how long they highlight for now it turns out actually making them highlight is is a mild pain in the ass because what you have to do you've got your tap and you have to walk up the swing hierarchy so you find like the container of all the tab then you the one with your caption the problem is when your components created it is an orphan it doesn't have a parent so what you have to do is register a hierarchy

change listener which is then triggered once your component gets connected to the swing tree you can then walk up the hierarchy find the correct tab highlight it and when when activated you can set at the timer now you may not want to do all this hard work so I have created a gist to do this in fact if you're into doing extensions there's quite a few gist's on the github there and sometimes when people ask for things if I'm feeling in a good mood I'll write them adjust to do it and and there's now a reasonable collection of them now another little pet peeve that I see extensions getting wrong is whenever any sense and creates a pop-up dialog it

should be given a parent now swing will let you provide null as a parent now I have this idea that all the extensions that do this the developers only have one monitor because we've all got dual monitors at ports wicket and when you've got dirt running on your right-hand monitor it really grinds when an extension pops something up on the other monitor now the way around this is to give all these pop-ups the parent the parent should be the burp frame now again there's one there isn't a burp API to practice you can use the standard swing api's to find a bit frame and again there is a gist to do that for you now this problem may be familiar to a

lot of you because there are so many wonderful burp extensions that you want to add them all and that navigating your tabs becomes like a skill a skill all all of its own and now what a few extensions are starting to do people have realized is this this area of the street this is getting pretty closer but there's still a good bit of space up here now well again there's no official API to access the menu bar you can still directly use the swing API and some extensions notably active scan plus plus and backslash powered scanner which recently gained graphical configuration screens but you access them from menus rather than from tabs to reduce the clutter issue and

another thing a few people have wondered is how an extension like replicator is able to access the session handling rules and macros and other parts of the config now that's down to these two calls that are on the I burp extended callbacks interface so save config as Jason Takes and the running Bert project configuration serializes as Jason and returns it as a string it's your extension so you can read any aspects of the configuration and then load config from Jason works the other way around so you give it JSON as a string and it takes that and it updates the running configuration and it can be a little bit icky to do it because because actually

they there was a lot of configuration options so this is there's quite a lot of like nested data but very powerful the one limitation note is there is no change listener so if you remember earlier where we needed to create that login macro to make our replicator file reproducible now when we switch back to the replicator tab the drop down had the name of the macro there immediately well the only way the extension manages to do that is every time the tab activate see we does save configures Jason just on the off-chance you changed something now there was one final thing which was quite subtle and you might not have noticed but I'm just going to turn off this break

point well no yeah while I do this now um when we click text the token update says f3b click it again c19 now I thought it was it was a really helpful feature of the you worry that that screen updates based on all the session handling rules because it gives you confidence that you has session handling rules are working however again there isn't there isn't a direct API to do this but you can cobble together some other pieces of the API to get the effect so if you register an eye HTTP listener you get a callback you get process HTTP message and that gets the message just as it's ready to be sent on to the destination web service so a a

tool with Imbert says he wants to send a request then any session handling rules platform authentication a and other things get the opportunity to modify the request before it goes on just as it's ready to go process HTTP message gets the final request so we've got the original request which is in the text box in the URI we've got this final request it's going to the web server the last piece of the puzzle is how do we connect the two messages there isn't unfortunately on process HTTP message you do not get like a request ID this is an often requested feature but currently it's not there so instead what the extension does is just before you send

it it adds a header it's like X replicator ID and it puts in a random value and then within the process HTTP message if it sees this header it first of all it strips that out so it doesn't go to the target web server but it uses that to update the UI and this gives you the effect where you can see immediately the effect that all your session handling rules are having so there you go few tricks you've got all the tools now to run your sdlc at warp speed you may have some questions about this which are welcome and I'd just like to leave you with these final thoughts

[Applause]

Jordan yeah Evan the past few months so at first I first had the idea in the final months when I was working at pence s and I thought are they thought this will be great you know I'll have the opportunity to do this like in a in an even more conducive environment and in in in the first couple of weeks so I said that hey let's Ruth this would be a really cool feature and um it's quite often happens when we discuss these things that stuff will come back and say well this would be good if it can do this and this and this you would have to do this to be good and so I thought about this and I

realized it didn't need to be a core feature and it could be done as an extension and then yeah it's sort of as a backbone of projects it came together

any questions would be good yeah right so yeah so this current version doesn't support unattended running so you couldn't you couldn't put this like in a Jenkins pipeline a few people have requested that um although personally I think for doing my odd job security you're better using scanners I know there's some merit in checking that you've not broken the same things you've broken before but I think there's a lot more merit in in using scanners and setting up I mean whether you want to use Bert with the carbonator plugin or you know or whatever tooling but having having ongoing scanning as part of your pipeline

oh so it's soweth so if this could be integrated in the pipeline this would automatically give you the take the failing test that you need as the so that so you were doing like full TBD there all right well that's that's a that's a good suggestion thank you would that do the trick for you then right hello sure um I mean would it just be enough if if you could do it like on the command line and you just got some like XML output on the command line would that be enough to script it yeah I don't know do you do you segregate your pipelines do you have like you sort of a fast in a long pipeline us and thin

yeah I mean I've always thought one of the issues with doing agile security is as scanner does want to run quite a lot of payloads against your arm and especially as you sometimes worried about kind of spooky action at a distance you know that you might want to do a security scan of the entire product just in case a change here had a far-reaching change yeah sure yeah unfortunately this thing will always would always require burp to start it because it's using all the logic in the session handling rules I mean there is some I don't know if you've used the there's a feature to copy as a kill command so if you've got something that

it's relatively simple to replicate that you know that doesn't require a login session so if you could copy that as a kill command that would be a one-liner for you okay that's all the time for question I think it's lunchtime now so enjoy can we find me later on if you're on a memo [Applause]