One OSINT Tool to Rule Them All

I'm seeing this and sorry about the date it's you know what happens you're in con time so welcome to one ocean tool to rule them all I'm happy to see some friendly faces out here and for those of you that I haven't had the pleasure of being introduced to yet I am Emily st. Pierre I am a security analyst over I rapid7 I've been working with them for almost a year now I've been part of this community since Def Con 21 so about five years and I'm also a director at large at our local hackerspace here in Las Vegas it's called sin shop over there I host our quarterly crypto parties some technical and privacy workshops we have

a software defined radio meet up tonight at 6:30 if you're in town and you don't know what to do tonight please stop by it's gonna be pretty cool I also go podcast it's called gray noise it comes out once a week on Fridays and one of my co-hosts is here today if you want to reach out to me i'm on twitter at live fox if you want to send me fox pictures or adorable kittens or if you have questions please feel free to ping me all right I'd like to start first off with the reason I'm here today there's a little bit of a story behind that a little under a year ago as I was

starting my penetration testing job as doing one of my first engagements as an external network penetration test which means we're looking at finding vulnerabilities on the outside whatever's networked facing or internet facing and I came across this login page and it required two things valid email address and a password so I thought this is pretty straightforward I'm going to do a password guessing attack right so if I can find a valid list of emails for that one company I can try to see if there's any typical passwords you'll find out there and hopefully I'll get a hit and I can I'll get internal access so being pretty fresh I didn't know that many tools at the time that gathered publicly

available information so I used the harvester it's a command-line tool it comes with Kali Linux by default and it goes in searches for email addresses and sub domain information via search engines so I entered company coms a domain and I got about a dozen or so email addresses back I didn't get any hits unfortunately during the engagement but it went well wrapped it up and thought that was that the second week on the job I was introduced to this one book by author Peter Kim it's called the hacker playbook - and in it the author talks about this tool called discover now this cover has a little bit more features and just for fun I decided to

try it out with my previous engagements domain and the results were wonderful I got about two to three times more emails back and the most the majority of them were all valid and so that got me thinking right what if I had been able to use that tool right off the bat you know would I have gotten into would I have gotten internal access and what other tools are out there you know if this tool is better than the last one well then maybe there's other tools are even better out and what about tools that collect different types of data and as I was wondering that this is where the O's and tool comparison table came into place

right would it be great to know which tool gives me the best user name lists wouldn't be great to know which tool is the best at analyzing metadata and basically you kind of get the gist of it you know we want to see what tools are out there and which tools are better than others maybe depending on our needs and so I got together two things methodology and then a list of tools and I got working on my project so I was talking about publicly available information a little bit earlier and that is ascent so ascent is all about finding locating that publicly available data and analyzing it and transforming it into something that's valuable to you

as an attacker as a competitor and that's a penetration testing execution standards definition but really oh scent is available is whatever is valuable to you you don't have to be an attacker you don't have to be a competitor to find OSINT valuable so let me show you some examples of this as I mentioned earlier you know valuable types of OSINT would be user names if you're trying to do any kind of logging in attacks emails not just for logging in but also for phishing campaigns or maybe you're trying to find a list of emails and do some fishing technology and use would be what kind of environment are we talking about right if you're looking to attack

an environment that uses maybe Citrix or Mac or Windows you know maybe you're going to tailor your attacks differently location data if you're doing any kind of physical security for example perhaps you'd like to know more about some smaller branches that might have a little bit less security or that are easier to infiltrate and corporate data would be any kind of data regarding maybe the company's registration records or or state records for licenses and such all of these are just examples there's a lot more different types open-source intelligence data that's out there but when you start putting them together and piecing them together that's where they become really really valuable to you and from there you can see maybe you know if

you want to attack the network or maybe use it for some other purpose so now that I've given you some examples let's get back to my project so the methodology is talking about the first step towards that was collecting all of those tools so I compiled a list of tools that were related to organizational data since that was what was most useful for me and when I'm talking about organizational data I'm just talking about companies any kind of tools are targeted towards companies aka organizations so of course I took all the tools that come in by default in Kali Linux then I went through the different types of pen testing books as I mentioned earlier the hacker playbook

was great for that word-of-mouth people like you in the audience that gave me suggestions or colleagues and then finally different tools are listed in lists that are available online I found the osun's framework by Justin Orting to be really wonderful at that he does a great job of keeping all of these resources together in one place all right so now that I compiled the list the second step in my methodology was to compare all of these tools against three different benchmarks so the first one is data variety what I mean by data variety is how many types of data can I find with a tool right this is to only find email addresses or does it go find maybe corporate data or

legal data right in my first example in my story we had the harvester that I kind of two different types of data that came back but then discovery had a lot more so that's what I was looking for second benchmark is data quality it's great to have a huge list of emails or user names that comes back but it is worth noting if it's all invalid email addresses or but it's not valid data so what I mean by data quality is is it accurate right and also will I get well will I get a lot of that data back right is it incomplete or am I gonna get a large amount of data that I can work with back

and then finally lastest benchmark is relevancy and what I mean by relevancy is is a tool up-to-date you know there's some tools out there they haven't been updated in a while and that's what I was looking for which one will I'll be using maybe some of the better API or maybe some new functions that we don't see in older tools and which tools used may be broken api's or even insecure libraries in some cases so these three benchmarks with the list gave me just a data I'm gonna show you in just a few minutes I I know there are some data limitations to my approach it's not perfect there's a you know it's this is not an

exhaustive list meaning there are some new tools popping out all the time and so just keep that in mind some tools contains other tools in the case of discovery discovery for example contains two harvester there's a lot of tools recon and gee maybe we'll have some models modules that are the same as others and this is what I meant by there might be a little bit of overlap in there some tools are hybrids I found that a lot of tools do more than just ascent they tend to do a lot of scanning and finally data accuracy could be biased I had a sample of 42 organizations I got them from different area different industries so anything

from healthcare to education to not-for-profits to regular retailers I kind of mix and match a lot of different industries to try to get that data but of course if depending on who your target is you might have different data all right so now that all of this is out of the way I'd like to get to the results so compiling everything I built myself really neat data sheet it is super useful for engagements and here it is the Othon tool comparison table now this is a small fraction and not to worry I am totally sharing this with all of you today and actually I even invite you to collaborate it if you for example have

have any ideas or if you have did some ascent tools I would love to talk to you and connect with you so that it can as you to this table a brief overview the table the reason why I use that format was because it was easier to see but also depending on which column you're looking at I've made the values so that you can get them in ascending or descending order or you can find for example easily in this column right over here all of the tools for example that come by default in Caly I did two columns for modules one is just a brief overview so if you just want to look at all of the tools that grab DNS data you

can do that if you want to just easily see all the tools I grabbed email data you can do that and then a short description relevancy would be under the last updated column because now you'll you'll be able to see well is this tool accurate is it still being upkept and finally the repo just because if you do find a tool that really grabs your attention then it's gonna be easy for you to go ahead and download it not shown in this screenshot but that you can go over to this links to grab it our other columns that you'll find where I'm waiting on a system of one to five how basically the result of my methodology

and my analysis you'll have a rating for an email for DNS for ease of installation and if you have ideas for anything else you'd like to see there maybe you're hearing me talk right now you're like mmm there's maybe some part of Oh sent that I've skipped over please collaborate with me reach out to me I'd love to make that happen and make that table as useful as possible so doing OSINT as I was mentioning earlier it's not just if you're a competitor or an attacker right you can use this if you're doing any kind of blue teaming if you want to see what's out there what do attackers have access to if you're maybe an investor and you

know you're looking at this company you've never heard of it you'd like to learn a little bit more about what it is or maybe what kind of technology it uses maybe you're a lawyer or there's a it just because you're not in offensive security my point is you'll find osun's valuable you'll find a way to make oesn't valuable to you all right does everyone have this links everyone good ok awesome so you're all here and listening to me and this table is awesome and magnificent but I know what you all came here for it says you're probably asking yourself so Emily well which tool does roll them all after all this is a name of my talk well I didn't find one tool

that really did everything but I did find a few tools that really were shining in this apartment and I apologize I see that I forgot the URL for recon and G sorry in this slide but you can find the URL for that Ashley on the table so best email lists definitely recon in G and that's because the tool uses some Salesforce data and has a couple of modules linked in Salesforce and it grabs them together and it parses them and makes a really nice list most user friendly spider foot it's easy to install it's gooey it's a local server and it does a ton of things like different kinds of data grabbing such a vulnerability analysis

it looks like DNS domain information there's a lot I couldn't list them all but it's really easy to use so if you're trying to dip your toes into Oh scent I would definitely suggest a spider foot easiest metadata analysis it's another okay it's another GUI tool called Foca Foca is a window space tool it's awesome it goes and grabs files over search engines and I don't know this is still work alright okay awesome and then it will download them automatically for you and do an analysis and then present all that data to you user names passwords it's it's it found technology overall great tool even though the free application is end-of-life it's still very relevant and

I think now they have a subscription model instead um again these are my picks you might find that there are certain tools as you discover as you use them that work better for you but I so if you do I invite you once again to collaborate with me and here is the link to Osen comparison if you're looking at some of the resources I mentioned in the talk today you'll find them here again if you want to reach out here's the length of the table here's an email please let's collaborate if you're new if you're curious about tools I'd love to hear what you what you have in mind what you're thinking or maybe you want

to create a tool and you'd like to see maybe if there's some gaps in the table that you'd like to implement in your own tool and so that's it for my talk here but as you know so this is my first conference talk and to do that I my

the liveliness of this microphone but you are there any questions before I wrap things up no if you want to talk outside I will be right outside probably having a beer celebrating the fact that I'm officially on vacation now so if you'd like to come check please do the outside thank you everyone [Applause]