Automated Dorking for Fun and Pr^wSalary - Filip Reesalu

alright I'm going to talk a little bit about a side project I've been working on developing around Dorking mostly you probably know what it is I'll I'll do a quick introduction I probably won't use my entire 30 or 40 minutes that I have to be for you to drink beer my name is Philip Breslow i'm currently a security researcher at recorder future i used to be a software engineer in linguistics and then I was a much-hyped data scientist role for a while so start off by introducing what dorkiness looking at a few tools that exist today and the issues that I see with those then I'll show door key which is the thing i built

and then i'll discuss some of the future sort of projects or ways this could develop Thai food try and cool the dorking or search engine hacking is nothing new it's been around for ages it sort of took off in 2002 when Johnny long started collecting a little database of the works as they're called these clever search terms and we can see why it's called why it's called a dork from the original page Google dork an inept or foolish person has real by Google and this is what it looked like way back when this has sort of morphed into this shiny looking thing exploit DB I was the google hacking database and it's it gets new search terms every few

days basically and it's related to especially when a new vulnerability comes up people want to find servers that are vulnerable this is an example of work so searching for ham dita XL back door in the title of the page brings up web shells of a specific kind and if we click through to these we have access to that server so since this has been going on since 2002 why why bother doing the I think there's a talk on this every few years it's still as relevant if not more relevant we got the internet of things you know toaster d dos webcams someone can sneak in and look at your parents what can I make it everything is online

you get ICS SCADA systems online I'll show an example this you and then again nothing has really changed it's still the same issues people are publicly posting sites without having login credentials or the use the default week credentials so it's still it still matters so I was looking at trying to find a good example of this old Dorking thing a google search technique aided new york damn hacker in iran you could think oh that's that's pretty cool or pretty bad but then again when we look at when this was published it was a few months ago so that's that's a bit of a fail so when most people think dorking they think google searching or like

normal text search i just wanted to bring this up just to highlight some of the issues with these things google has great advanced search operators most dorks that you find especially on the google hacking database as implied by the name is our game for Google Bing filters a bunch of dorky results you can't find SQL files like searching for the file type extension and as limited advanced operators on the flip side Bing has an api google has removed their search api they only provide one that's called custom search engine and you can sort of trick that into searching the whole web that you get very limited results i'll talk a little bit more about what search engines i support here

later some of the existing tools they all seem to be focused on single-use there's a lot of pretty you ice and most projects are dead hopefully this won't be one of those projects you know hopefully not and what I wanted for this is something i did at work which i could later publish open source and free for everyone this because i wanted machine-readable results that i could set up alerts on for example i want to be alerted whenever a certain domain is has popped up in one of these dorks that means that something is going on for example for a specific company notable here search diggity they have a very nice website with a lot of good

information but the last updates i can see where from 2013 so I built this little thing called dorky this it's hard to read here but it's basically a little management interface for keeping track of which dorks you want to run let's split up into two different components we have a runner that collects these dorks from a mongrel database and runs them and close the results waits a little bit and then it does it all over again and then we have the UIL component that I showed a picture of just now and in between those two is the mongrel database that just stores the information the dorks the results I built this using Python I like Python I

do it a lot for work I used because it's it's free it's he set up and if you want to do some rapid development you don't have to specify a schema or anything you just shove it in there and Burkes org which is a great little web application thing for Python it's what powers flask if you haven't used it you should really try it included search engines I i included google unfortunately since they don't have an API you will need a thousand proxies but it works Bing using their API Google custom search if you have something specific you want to do as well as shodhan and if you haven't used rodin it's it's basically like a

searchable n map that runs across the web and they collect screenshots and things from open and rdp port to think it's very cool you should check it out if you haven't so the UI is very simple I didn't I didn't want to do something very cool you put in put a query and put a description you put it in a category you give it a source you found it somewhere and then you pick what search and you wanted to run it from and then it shows up here at this room a little bit hard to see here and as soon as you add it and you have this sort of runner component going it will pick up that

career next time it runs and it will execute it and collect the results and looking at the results this is just in the UI you just get to see which which IP in which pork and sort of the header that came back if you look in the database it's the whole cached response from Sheldon and for for example for a google search it's the URL the sort of title of the page and then the little description that comes in your google search result so for configuration it talks through a database that you need to set up you need you select which search engines that are active you can input your API keys for certain sites ok

Google you need to Roxy set up and then you need some filtering because you don't get you need to filter out because a lot of people repost these dorks on for example blogspot or whatever and then you'll get noise in your data as I said some of the issues here our results are not perfect sometimes there's a lot of noise there's right limiting you have to sort of pace yourself and if you use the api's then there's API costs involved because if you run all the dorks from the google hacking database then you'll run out of money pretty quickly if you're using baying for example and one of the biggest issues and the big reason why i think a lot of

these projects fail is because you need when the api's are updated or when the websites are changed you need to update your approach and I could tell that when we'll turn off their search api you could tell a ton of project just collapsed and then never updated again

so some some of the future things I'd like to add more search engines github virustotal hybrid analysis or some ideas some better logging and error handling I have been a bit lazy but i'm working on improving it then adding a pipe to elasticsearch i think would be cool if someone wants to set up their own works the way i'm using this i'm running this in our production system right now but I'm piping the results into or or sort of platform and in there we have alerting capabilities and stuff like that so I did not add the output to elasticsearch myself I figured I'll show you a little bit of a live demo I'm not on the on the Wi-Fi so i won't give you

the actual running the dorking but i'll show you how the UI looks if you want to if you want to check it out it's I github.com / recorder future / dorky it's MIT license and if you have any questions free salut at gmail com I did not sign up for this peer list thing I apologize yes I think I have it running

another real so this is dorky running on my machine I have it's not on get up yet but I have a script that imports the entire google hacking database into your own local database so its grapes that website it puts them in as not enabled to begin with because you don't want to lose all your money and in here we can sort of if we wanted to enable this guy hit it it's enabled and it will start running next time it's run the query I showed from showdown is in here you run some query sq webcam it gives you a bunch of webcam results and you can look at the sort of ugly results in here but it's a ton of

webcams available

and I didn't intend to this to be a long talk I wanted to keep it very brief because if you would like to try this out please go on github start running it there's no reason for me to stand here and drone on for 30 minutes about something quite simple that was it if anyone has any questions or comments please yes I they are I mean that's for for me it's been working out so far it's I have maybe maybe a gigabyte or something in the in the production one and it's working fine yes you mentioned machine readability towards the beginning but it's not a command line tool is the idea that you have that

people would know sir yeah so what would I use this for is that I have a separate script that I run the pulse data from this manga database as soon as new and then I sort of pushed that into wherever I want it

no I'm currently pushing it into so what I'm pushing it into a recorded future which you guys won't have the option to do unless you start working at recorded future you can talk to me afterwards if you want to work at record a future anybody else well if not then thank you quickest talk at bsides what work