← All talks

Pwning all the Internet of things for fun and profit

BSides Toronto · 201423:19396 viewsPublished 2014-12Watch on YouTube ↗
Speakers
Tags
CategoryTechnical
StyleTalk
Mentioned in this talk
About this talk
A network security talk that is grounded in reality! Yes, attack is way cooler than defence, but at least 50% of people have to do defence, so here's a talk for all those in the corporate sponsored seats. At Etsy, we've traditionally eschewed common knowledge on what works and what doesn't and hand sewed our own path to security. This talk, briefly, shows some of the more interesting things we've done, despite being a website that sells knitting patterns. Topics include: * IDS that isn't a complete waste of €100,000. * the one firewall per child project. * rootkits, making them work for you. * what to do about the "Zero Day Problem".
Show transcript [en]

[Music] uh I'm Ben Hughes I'm clearly from Etsy um I'm here to talk about poning all the internet of things for Fun and Profit not really I just thought if I did an Internet of Things slide first everyone would be like God no so when I did a defensive security talk everyone would be really for that because everyone's bard of Internet of Things talks um got to say it's good being in Canada where you do measure ments properly because I live in the US um I'm not so pleased about the temperature um I obviously work at Etsy we're a Online Marketplace that deals in vintage and handmade Goods um I hear the holiday seasons are coming up we missed

your Thanksgiving um but there's other ones coming please shop there uh we're a retirement home for Burnout Security Professionals so we have a surprisingly large um actually like scarily good security team that's better than most like Boutique pentest firms because they're like yeah I can't be bother being on the road anymore um no one really cares about the slide so I'm here to talk about an intro which I'm doing um laptops workstations why they're the bane of everyone's life servers you might have some or you might attack them data which I hear some people have and then conclusions otherwise it would just amble off into nothing wow these don't update in time so the key thing of this talk is we no

longer have this really neatly find border between um like uh a secure place and an insecure Place uh many people now I believe if you have a company up to one per person have laptops um those are a good thing to attack and the line between inside and out no longer exists um the threat model for securing the front door which we used to enjoy during the '90s is no longer relevant because there is no longer a front door people keep talking about Moes as if anyone was alive in the medieval times um as my amazing clip art shows uh this is cloud thing I don't know if you have it here because all I see is snow and cold

um and like nowadays you don't go most people don't attack servers or like most real people don't attack servers you attack endpoints like uh laptops are more and becoming more common of phones as the malware on Android or Android as we call it talk earlier discussed um so um securing your laptop is a thing if if you don't think you need to secure your laptop then um it probably has company data in it um if you're an engineer you probably have some source code on it uh unless you're one of those weird places that doesn't let source code leave sight uh it's great for lateral movement if your laptop doesn't have access to things then you probably don't need a

laptop to work at your job um I'm sure you can think of more reasons why attacking laptops is easier than attacking a filed web server or whatever not the web servers are secure but whatever different talk um so if any of you use Chrome uh I hear it's the coming thing uh oper just isn't taking you off anymore then um creme does a real good job at stopping you from getting malware there's recently been some malvertising the Kyle and Stan malvertising campaign recently of um you buy ad space with someone like Google or Facebook or Bing they serve adverts to you and like so you go and search for text Wrangler and the first link is not Tex Wrangler it's

a paid for ad advertisement to something that then downloads crappy malware to your machine and then kindly installs whatever you ask for on on top um and like this is just to install adware to steal your uh browser and your search but if you're willing to pay enough money like kyl and Stan whoever they are did you can do more malicious AP blah blah blah um the nice thing about this is Google's motto is don't be evil unless it's for financial profit in which case it's absolutely fine to take money to uh do this kind of thing because that's fine that's fine despite having an entire browser Department designed against this um so one of the

cool things on it uh Etsy has dig a really big hole in the ground uh for DNS syn Hing and DNS syn Hing is just where you take your uh like shitty domain that you don't care about like download.co which is super legit um and then you just in your corporate DNS you just dump all queries for that to Local Host big shout out to IPv6 um or if you want to be real clever you redirect it to an internal site where you can track and um monitor this so we have a landing page to go you tried to download something crap please come and talk to security or help desk and they'll try and make you

click on the right thing rather than Google tricking you for money into downloading something malicious that's only happened once this week um really important thing this will break with DNS SEC so you can all have to really think long and hard about that because I know DNS SEC is a big big thing used by no one Speaking of used by no one I recently did a similar talk in Europe where this slide was not as funny because they have both those things there um but North America is like we don't want those we've got loads of IP addresses we we made them up to begin with um real Global of you um so whether IPv6

whether you Lo it or completely indifferent to it you can still have lots of fun with it as many would do uh and if you don't think you're using it then you're probably wrong if you own a modern telephone maybe even a Blackberry but who knows no one's ever used one um if you own any Apple kit or like run iTunes then that will just talk IPv6 to whatever it can uh Netflix is probably the largest IPv6 consumer publisher I whatever Netflix do they're the thing you tunnel to America to to get the decent video um I've heard rumors um so there's been there's been a bunch of Talks by other people and the basic

attack is uh you can't throw up a DHCP Network on ipv4 to steal everyone's traffic but you can certainly throw up an IPv6 one uh and because of things like slack the auto configuration thing you just go hey local network you can go and talk IP ipv4 over there but how about you go and talk IPv6 over here and most modern Stacks will uh pick V6 before V4 and if you kindly do DHCP V6 and go hey here are some domain uh DNS servers you be like cool I talk to those and then you just return every host you want to intercept suddenly everything goes over V6 to that hop and then you can just translate it back to V4 and now

you're man in the middle in everyone's traffic um so IPv6 is pretty useful for that [Music] um I'm sure there's other uses other than other than Netflix um you can in most kit like Cisco or jiper you can disable this with a router advertisement guard which same as you would with DHCP of going like my DHCP server here if anything else says they're a DHCP server they're full of don't listen to them um I would suggest turning that on and if you are an attack team I would suggest enjoying this um another thing we've been doing is um handing out ids's to everyone uh what are you saying a big rack mount thing on everyone's desk cabled into their laptop

no um from the little example there PF because everyone uses Maxs uh again not as funny in Europe uh where they have lovos who even knows what they are um so just using like PF or IP tables if you're one of the eight people on a Linux desktop um you can just do simple pass logging for a basic very basic IDs uh where you can go I'm talking to bad hosts on bad ports such as like bits horen or 31337 because I'm showing my age or whatever other Port you want yes it's not quite a full IDs in the snort sense but the benefit of this is it isn't full of remote code execution so

you've got to weigh the two up um then you take these logs off your end points uh and then you throw them in like elk or whatever you log to if you can afford Splunk you do that if not you just have money fights um this is not despite what it says there this is not a genuine ETS dat Center none of that's handmade um except maybe that delicious sweater do you say sweater or jumper cool good to I'll go and edit that slide um so if you're here you probably have some servers or access to servers or have some Cloud wherever the cloud lives uh managing servers apparently quite a big topic um I hear some people have

done that before um does anyone know why all new vulnerabilities have to have a core name has anyone figured that out was was there an announcement RSA I happily missed um if someone can answer that that would be really nice because it's getting really boring um uh I'm sure if any of you have worked in operations you'd be like yeah this is really cool look at that amazing up time 2,000 days and that's why um you're owned basically um so this is stolen from the the rude presentation from black hat Europe got too close to the mic this shows the conal vulnerabilities going up and up over time the the red to D was uh presentation of like you can

attack applications which have aslr or right no execute or you can just go and attack the konel and see as there quite a few kernel vs across all operating systems if you can just own the kernel then like that's it you don't have to worry about aslr as much because you've got right access to the kernel um which you know it's it's it's a good uh good talk I suggest you go read it but that leads into everyone worrying about the uh zero day tax because that's cool and that's how you get funding and that's how you go work for ven blah blah blah um but like if you have a 2,000 day up time on your machine why you wored

about zero day you have like you have like 2,000 more days to worry about first um and like why there's however I know half dozen dozen teams who can go out and WR mad OD day there's like a few more who can write shitty 2000 day um actually past the point it probably gets harder to write 2,000 day cuz you're like where am I going to get that copy of Linux does anyone have a CD drive um so like how can you how can you uh reset this dichotomy of worrying about panicking over oday because threat post tells you to uh and still having like up time on your machines rather than just rebooting them every day which

operations people some developers probably some Finance people not so keen on so uh here's some of the stuff we've been looking at some of this will be familiar to you um and some of this we haven't yet done so that's cool uh SE Linux is a way of frustrating your sis abins in the way that they can do nothing on their machine uh anyone who's used SE Linux will have used the command set and force zero so that they can actually run what they're trying to run um it's a kind of sandboxing and policy model you say Apache should never talk to these files it should never run the system CES and it's much more fine grain

than if you are uid zero you can do whatever you want if you're not then you can't do as much um we've got pretty far with that model but SE Linux is a frustrating way of um extending that model to be more fine grained um there's an argument that again like once you attack the kernel you just go to the bit it says turn off SE Linux cool I'm done uh because it doesn't do quite as much hardening in its uh policy model um the a cool trick to do is enable SE Linux put in all the policies um but not actually make it enforce just make it log so you get the Apache is for some reason running Binh

and talking over Port 88 eight which it shouldn't do or I'm guessing shouldn't do may I don't know how you run your web service um maybe maybe that's a thing in Canada you just use binus AG for your web traffic um and then you you put that in a log file then you throw it in Elk or whatever and then you go cool that's bad let's do something about that um that's a more workable model than having SE Linux go nope you can't do a thing because a developer uploading new version and now everything's broken there's uh gr security or G security if you like Tigger um and that's uh a set of hardening patches against the Linux kernel uh it's

written by a crazed underpaid guy who um just releases new patches every time there's a new Linux kernel um they do lots of actual hardening um like every time a new vulnerability comes out in the Linux kernel he really goes on Twitter going yep still fine um and that's actually a way of making it more secure conversely though you have to recompile Linux patches which your operation people may tell you not that they don't want to do for some reason um another another fun thing you can do is K spice which is now owned by Oracle which is the utter batchet idea of applying kernel patches without rebooting the Machine by just splicing the changes into memory and changing the

pointers that go to it um the interesting thing here is you have to have something so important you can reboot it but something you can definitely definitely change the kernel memory of um like this thing can definitely not we can't reboot this that would be critical but like cool let's just stick something in here and see what happens um totally totally fine uh a friend of mine pointed out the only thing this could be good for is if you have lots of developers running Linux desktops and they never bother updating them um I would say like who's had a Linux desktop that hasn't crashed uh this leads me on nicely to my next run of um don't know if you can

read this but this this is a red hat great MySQL announcement with a list of unspecified vulnerabilities Oracle don't know their own bugs that are reported to them why are you trusting them with your kernel memory so annoying all the bug updates just go yeah an unspecified uh vulnerability here that could be used to remote code or dos something you're like any any clue as to what like severity no um another option um is you can uh you can run open BSD which only had two holes in a heck of a long time uh but I don't think anyone is going to go full Calgary on their entire Cloud infrastructure if you can indeed run

open BSD on cloud does anyone know that no one's tried um I present to you a fifth option you can just reboot things but uh or if you use AWS you can wait for them to reboot them at random times and not tell you when that's going to happen um but no one reboots databases because uptime um caches um so the thing is there will always be unpatched machines because who's going to you're not going to reboot that database server you're not even going to log into it you're too scared everyone is um breaches will occur um I the US has had some exciting breaches recently luckily PCI protected them so they're fine um uh and knowing that you've been

breached is more important than living under the false pretense that you're never going to be breached um it's certainly why I drink at night um it's so having awareness is much more useful than than going yeah I applied an update I'm fine you can even if you want to worry about OD days Now's the Time to do it so one of the cool things we've got is we run elk which is elastic search log Cabana and we have can anyone see that I this is not the best projector in America we have better projectors um but they're all com branded so they're rubbish um this is the output of commands running um as rout on a machine and this is just using

um linux's audit D framework which is uh built into the Cel it does more than just command logging this it logs into any system core you wish uh it outputs via the most horrible worst design system I've used um and you hook into um many different kinds of events I'm just hooking into exv uh and then outputting them and then mangling them so that they're in a useful format and then throwing them into elk um I will open source that once I finish the bug apparently python isn't enough to keep up uh so I'm going to have to do what everyone else does and rewrite everything in goang because I'm cool like that um if

you don't want to use that then Milla have open source theirs which is written in C because they're older than me um threat stack do a really nice one which is all cloud-based and you pay the as a service and that does more than just igv uh the problems with it is it outputs multiple lines from a kernel and then you have to read multiple lines and then tie them together with IDs because kernel developers have never used logs in their life um which is just ridiculous so the threats like people have actually Rewritten it and output it as Json because that's what you have to do in 2014 uh here's a bunch of links I didn't

expect you to remember them I'll probably share these links out and then you can go look at them whenever you wish um but the being able to see commands running on every single machine in your entire infrastructure at once just super powerful like five years ago having that kind of visibility would just be like how could I have that I would have to tail bash history and pipe that into a thing over CIS log and now you're like yeah or you can get it from the kernel that seems the better plan um and and we use that on both laptops and servers and absolutely mindblowing do that data so if you're here only one of them is

Big um if you're here I'm going to assume you have some data um I love that one of them is in a kilt um um and so this is this is an amazing screen grab from The Amazing film sneakers which you should all watch um and this is related to backups how you say don't worry I'll get to that um this is them bypassing the electronic key lock so they secured a door with an electronic keypad um and the thing here is like so you have some databases you probably have some ACLS on them you're probably only able to access them from certain IP ranges or hosts all that good stuff and then you run myql dump or

whatever database you use and then you just store them locally on the same machine or somewhere else so if I was an attacker I'd just go after the backups because there's no acl's on the backups they're just sitting on a file system um or they're sitting on another host with all the other backups and every backup of everything else like why doesn't everyone attack backups it's just backups are great um so I would say if you are doing backups of your data if you're not doing backups of your database go and do that now uh we're here till 7 um if you are doing backups of your database please encrypt them um and don't do it with a symmetric key

that's stored in the same host because that's just the slower way of doing unencrypted backups um um and like put them on a machine that has at least as much ACLS as your um database uh like all our backup servers have go toofa to even get onto them which is more than all of our database servers because apparently getting web apps to do 2fa is kind of hard um some your DBA May argue that uh restoring from an encrypt backup is a pain uh I'm sure your liability people people will go yeah but having our databases on the Internet is more of a pain uh this is where you can game day it uh if you

think you're going to be CPU bound on encryption then you should buy some new hardware because AES has been in CPUs for a bunch of years now and it's no longer slow so do that hi um don't worry I'm not drunk this is about canaries um how awesome is that bird look um so what we done is we've put a bunch of obviously fake data in various places and then uh using our proper IDs the aren FI running on laptops uh looking for that on the wire of course this doesn't work if you if an attacker is like encrypting things out of there but it does work if they're not so it's worth doing um so like uh we have some TLS Ru uh TLS

rules IDs rules something end in in in s rules uh for spotting when Lup goes in the clear for when certain database uh strings go over the network which should never happen in regular usage and that's a really nice way of going hm something has gone over my network that really really shouldn't we should probably go and investigate that um it's really easy to do with just putting false records in false users in false passwords in of things things you don't access if you have a large data set then put them at the start and then put them in uh chunks all the way through so that like if they try and do the top or the bottom of the

database to just try and get some of it you'll see it um another cool thing we have is our load balances if they ever see um real code or in our case PHP code go over be returned from a web server then to drop that node immediately because you never unless your GitHub want to return code out your load balance so you're like yeah that's probably bad um so that's another cute way of things you can do with canaries of dropping connections as soon as they start sending things you shouldn't so to conclude with some ducks um mon desktops still blindly trust the uh network uh they will trust name of servers they will trust um DHCP

servers they will trust anyone sending them router advertisements um because then home networking becomes easier um you can exploit this or you can use this to your advantage depending on who pays your bills I suggest you do one or the other um servers like don't have to run blind anymore um the person who was talking first about having multi-user systems and not knowing who was owning what from them well now we're actually at a stage where if you throw the right tools at it you can see every command executed in every single machine in your Fleet in real time near enough real time um you don't have to run blind anymore um and be careful with data uh and it will be

careful with you um make sure you know when your data leaves the building without having to spend stupid amounts of money on dop products because if you saw the talks at black hat they're all terrible cool any questions awesome no questions you got free [Applause] [Music] time