Defense in Depth Against DDoS: Building Resilient Networks

all right well hi guys um i'm daniel gordon hi i'm a contractor to the department of defense cyber crime center dc3 welcome to my presentation on defense and depth against ddos against distributed denial of service attacks and how defend defense and depth can diminish the dollars destroyed um don't say that too many times fast i was advised not to put a quote on my title slide but i did it anyway i wanted to get out of the way why ddos matters it is not as sexy as conventional cyber attacks but it costs money that's why it matters all right let's get started um first off i'm going to quickly whip through conventional denial service tax

i'm going to talk about some unconventional denial of service attacks i'm going to talk about defensive solutions and how to evade them i'm going to talk about defense in depth some and then if your defense in depth doesn't work i'm going to talk about building resilience in your network and failing gracefully i will not be talking about cat-based denial of service attacks i just thought this comic was awesome so i'm including it anyway all right so conventional denial of service attacks classically uh tcp udp icmp these are layer 3 and 4 protocols tcp is a stateful protocol it requires a connection so the way that these are typically conducted is a session is started but then nothing

happens so it just sits there open consuming with victim resources udp and icmp are connection or stateless protocols which means that they are more prone to amplification attacks where somebody bounces traffic off of a third party and they're spoofing somebody so that results in traffic being directed back to a victim these are a couple examples of icmp and udp uh attacks respectively um memcached has been uh patched for the most part so unless somebody sets up a vulnerable server you're unlikely to see that but these are both ways to bounce traffic off one party and that results in a flood back to a victim dns and other exposed resources can be targeted by conventional live service

tax yeah there's no real way to prevent that except for some stuff i'll go into later um botnets are another common method the famous example being the miraibotnet which caused somewhere in the neighborhood of 110 million dollars worth of impact to various uh organizations um and had the brian krebs website down for a little bit um ddos for hire is another classic example where particularly in the gaming community people will pay somebody to knock off the connection of somebody they don't like temporarily and finally application layer text so layer 7 these are people attempting to use website resources by you know entering content into the website until it's not responsive buffer overflows are a way to

consume additional resources all right so that's conventional denial of service tax let's hop into unconventional i wanted to talk about the china great canon this is something that was tested live in 2015 against resources belonging to an organization called greatfire.org they the china great canon takes web requests from people outside of china to websites inside of china and inject javascript into their session that javascript causes a call to a victim website which generates a huge flood of traffic it took down cloud amazon cloudfront briefly and github briefly even though they have massive resources uh massive bandwidth um this vulnerability still exists uh isps are still injecting javascript into people's sessions so uh and no matter the size of your organization

you're still vulnerable to this so it's worth your time to think about being resilient to large denial service attacks bgb hijacks are another unconventional denial service attack these are more commonly unintentional they are than they are intentional though they still do happen intentionally uh often due to specific countries i won't name but like unintentionally you know a nigerian isp i think last year misconfigured some of the routers and ended up flooding some things or basically taking down youtube for a minute because you know this is a mistake that does happen um and finally uh not yeah not finally um authentication brute force attacks can cause a denial of service condition um so be that password spring or

credential stuffing um if they're done improperly and you're locking out accounts when people try to brute force things in the wrong way this is a problem that you should pay attention to i will talk about some mitigations later ddoses can be used to distract from or enable a conventional intrusion it's not something that most organizations have to think about but if you're a ukrainian power supplier then maybe it's something you're aware of and uh need to be prepared for um and finally email ddos services um i am sure that some people here have been a victim of a reply all storm um don't do it don't hit reply alt if there are a thousand people on an

email liam neeson will find you all right so i've talked about unconventional deny of service tax let's talk about defensive solutions for most typically people will contract with you know a cdn an isp a cloud provider somebody with a ton of bandwidth and just kind of hand off this problem to them they third-party ddos mitigations mostly work against layer three and four those are tcp icmp udp attacks um they have some limitations so i think that their uh terms and conditions may prevent you from testing your ddos mitigations which um i don't like that uh they may struggle with dynamic ip ranges or uh having large ip ranges and i don't know it's handing it off to

a third party may not be the right solution to everybody wafs web application firewalls are intended to cover that layer seven um type of attack um they do take some configuration some setup but they will hopefully handle it uh there are i've seen situations where they have missed layer seven attacks but they do a pretty good job next-gen firewalls are can handle some things but they have they do packet introspection and packet correlation which can help kind of identify patterns that would identify make up a ddos attack but they have performance limitations because of what they're doing they will fail if they get too much activity it's actually documented in public now that some next gen firewalls

have a specific infrastructure whitelisted i will talk about why that's a problem more later [Music] cdns content delivery networks are a way to host things external to your network that way they're not subject to a potential ddos against your network um if i i think of this in a way as kind of sharing risk if if you're hit by a big enough ddos against those resources or somebody else who's hosting stuff alongside yours gets hit that may still impact your content um they do have a lot of bandwidth so hopefully you're fine organizations can move their services to the cloud and traditional firewall blocks still have utility and depending on how you have them implemented can be

used effectively in this situation all of these protections have their utility i encourage folks to use them in combination to provide depth defense in depth that's that's part of the idea is by by themselves any one of these is going to be seriously limited but together they can be used effectively all right so let's talk about evading some of these solutions so i mentioned particularly with next-gen firewalls if you spoof a customer or partner or user of an organization um they can't easily block you without also blocking their customers so that is going to provide significant challenges to some of these defensive solutions bad guys can also target unprotected intermediate infrastructure it doesn't seem like it would be a big

deal to dias a certificate authority unless you really need to update a certificate and then it's a big deal bad guys can use proxy lists which cycles their their ip address through a list of hosts they can also use carpet bombing which spreads their activity across an ip range which uh the idea behind that is to keep the level of traffic on any individual ip under the threshold of something that would trigger a ddos mitigation all right so now [Music] now i've talked about defensive solutions and evading them i'm going to talk about defense in depth a bit so i encourage organizations to make a ddos plan um and also test them to see if they

work to test your plan um when testing your mitigations you should not only see if they're working but also how quickly they start working if your website's down for 20 minutes before a ddos mitigation kicks in for some organizations that's unacceptable they're going to lose lots of money and that's not good it may also be a problem if somebody's using something that like a proxy list that moves between a series of ip addresses if they respond too slowly they're not going to be able to catch it in time to do anything about it all right website input validation this should be fundamental sql injections still work sometimes so clearly not everyone has figured this out but please do it

blocker limit the source of inbound protocols such as icmp which is sorry i should have explained these earlier icmps used for ping stuff dns is used for resolving domains ssdp is simple service discovery protocols used by iot devices to discover what else lots of things on a network like printers and fax machines not effects missions to discover what other devices are locally on network these are commonly abused for this so block or limit them limit the source to specific ip addresses that are trusted if you can but also put in rate limits there's no reason you need lots of icmp traffic entering your network blocker common block common reflection sources so there are websites out there

like ddsman.net that track uh sketchy infrastructure that's being used for reflection attacks there's no reason you have to wait until you get hit before going out and blocking uh sketchy things in china that are likely to pro be used as a reflection attack um and finally uh not finally uh transition to a multi-threaded ids solution every defensive technology must fail open or fail closed if it gets too much traffic let me repeat that every defensive technology if it gets too much sent at it must fail open or closed if you're running a single threaded ids and it gets long sessions or too much activity it will either close causing a do not service attack or it will let

things through the biggest example of a single threaded one that i can think of is source fire i encourage folks to move to cerakata if they haven't already done so hopefully everyone has done that but i'm sure that there are still people out there running source finders all right more defense in depth um i mentioned [Music] did i skip over something no i did i didn't sorry um i mentioned email denial service attacks earlier um mitigation is to implement dmarc dmarc is a framework for auditing sending rights email sending rights from your net your domain it makes sure that if somebody sends an email from your domain they're allowed to do that it uses spf and dkim and some dns

stuff and it requires some work but i encourage folks to do it alternatively move yourself to the cloud implement captcha or password off delays make it harder to brute force things externally patch your iot devices for the love of god or manage them properly um iot devices i i mentioned them right botnet earlier iot devices are a frequent flyer in uh in terms of being used for ddos attacks they they can cause a ddos based on external traffic or traffic leaving your network to ddos other people so it's not just to be a good citizen you should do it to protect your own network as well fingerprint authentication sources using whatever mechanism you have available

that way you can block people effectively even if they're hopping around trying to use different uh different ip addresses you may still be able to track them using other means all right so now i've talked about defense in depth and hop into building resilience um first i wanted to explain this picture this was a flood in north carolina where lots of houses got destroyed but there's this one beautiful house that is in probably perfect condition i don't know maybe they got messy but i encourage everyone to maintain a network map and asset inventory you will not know that your devices are protected unless you actually know where they are

people do spin up things in the cloud set up servers on their own uh if you i encourage you to try to have a way to track that kind of activity implement network network segmentation so if part of your network is down because of a ddos attack there's no reason your entire network needs to be down segment off some resources from your web facing stuff diversify service providers or cloud providers if you can put in a load balancer if you can if if it makes sense um these things can add overhead costs they add effort to your uh to maintain for your organization but for for some these will absolutely be solutions that provide kind of reliability even if

amazon or google aren't available which does happen not often but it does happen figure out which which services are critical by talking to stakeholders i know some of you probably don't like talking to people but it's important to actually know what things in your business or organization need to be protected and what things don't matter at all and then focus your resources on the ones that actually need protecting all right so if you really want to be resilient though you should host your infrastructure on a botnet just kidding don't do that it's illegal [Music] all right um so i've talked about building resilience uh now i'm going to talk into failing gracefully implement cloud service ceilings i don't

recommend finding out about a ddos against your cloud resources when the bill shows up it would be much better if you were able to shut that off you know while it's happening invest in multiple ways to talk to your customers if you're running voip over the same infrastructure that is hosting web services and it gets ddosed people may call you about your website being down not get an answer and then maybe they assume you just got went out of business or something i don't know [Music] yes have a way to communicate with customers with partners etc even if your primary infrastructure's under attack um instead of locking out accounts uh that are being brute forced maybe you could limit

uh what they can access to say only internal resources it's a way to uh prevent that kind of external problem that i mentioned earlier configure actionable alerts for defensive systems even if they fail into log only mode so even if you're getting a ddos attack that is just absolutely creaming a next-gen firewall if it still logs c2 activity command and control activity from a bad guy you want to know about that you want to be able to act on it even if your next gen firewall is unhappy prioritize which events you want to hold on to even if your log files have just blown up so if you get an alert for me cats on your

network that's probably something you want to pay attention to even if uh this ddos attack is happening at the same time and finally configure a sim or orchestration platform so your analysts can silence things that they don't care about so that they can still do their jobs so that sorry let me explain this picture right quick this was flight 1549 that somebody it had engine failure they landed it successfully in the hudson nobody died um that is the best example of failing gracefully i can possibly imagine uh that is amazing all right so this actually that was the end of my presentation um i talked about defense in depth against ddos attacks and how it can save

you money um i went over ddos attacks talked about defensive solutions some building resilience and failing gracefully i hope i tried to provide some practical examples that hopefully you guys can take back to your organization on monday and uh you know implement and use for actually protecting things i hope that provided value and does anyone have any questions i see a hand in the back

um i don't know uh i i can ask though um oh thank you uh sorry the question was uh i does dc3 bring back a forensic challenge um the answer is i don't know i'll ask um yeah sorry okay comment is that it's missed i uh i'll see what i can do [Music] any more questions any comments okay thank you guys i really appreciate

it