Oh! 365 - Avoid an "Oh ****" moment in Office 365

I really love that you guys just heard from this guy about like how to create an oh moment so I'm hopefully gonna flip the perspective around a little bit before we get started just a little bit about myself I'm based out of DC I work for a cool company called expell we do awesome stuff one of the things I've been working on recently use like helping organizations defend against attacks in their cloud infrastructure and against their cloud applications huge banner besides besides do you see go where first why I'm here and besides Portland has been awesome so far I think you guys are doing a really great thing with this event and let's dive into it

so this is what you guys are in for today we're just gonna talk a little bit about the problems faced I probably don't have to explain it to most of you so we're not gonna spend a whole lot of time there we'll talk about how you can set yourself up for a success when you think about protecting yourself defending against attacks on office 365 we'll talk about how you can consume data and identify unusual things evil you know office 365 we'll talk about how you take an alert something that might be suspicious and actually run that down and determine what the heck it is if it's good or if it's bad and then we'll kind of round things out with some

trends that we're seeing in terms of what you can do that actually get better and prevent these types of attacks so should be no surprise you guys so like cloud is a thing so quick show hands like who uses office 365 right now yeah that's that's kind of what I thought everybody right these days you're either cloud born or you're rapidly moving there a lot of great reasons for this it's a it's a great service convenience liability generally better user experience as we know in the security community however generally when we move on to adopt something new Security's often like an afterthought right so when we start to leverage these cloud applications software the service

like office 365 we have to kind of wrap our heads around this idea of security as a shared responsibility so Microsoft is actually a big proponent of this themselves if you think about it you know as you offload more functions to your SATs provider like Microsoft you you don't have to worry about certain aspects of like security for example you don't have to think about worrying about physical security for the Microsoft data center right that's their problem you don't have to worry about processes running on exchange boxes like that's Microsoft's problem and even if you wanted to like Microsoft doesn't give you that control or visibility right and in some cases it seems like it's tempting for us to

overestimate the responsibilities of that cloud provider right there doing all this stuff do we really have to worry about anything at all but you know when reality strikes when we actually move here often we kind of left like feeling like this guy on the right Robinson realized we actually we still do have some responsibilities we're not super clear on what they are we don't really understand them and also we have to deal with all kinds of new threats new security controls and that kind of creates some uncertainty and a little bit of it makes it difficult for security teams to understand what they actually need to do what they actually need to worry about with office 365 and

that kind of leads us to the motivation for this talk so we've been on this journey to expell I've been on it as well trying to figure out how we can help organizations detect and response to attacks in office 365 and I think we've learned some insights along the way that I can share with you all hopefully some tidbits and tips and tricks that you can take back think about maybe implement your own environment hopefully avoid that oh moment so right off the bat like what do you need to do to set yourself up for success I'm sure many of you are familiar with office 365 audit log it's basically this magic button that you

need to press right off the bat and that will start to record everything that happens in office 365 why it's not on by default I don't know but super important that you need to enable this right like the last thing you want to worry about like is somebody's owned and all of a sudden you realize you don't have the evidence to go back and figure out like what's going on so definitely recommend you do this right off the bat it's a simple button press or you can use some powerful magic to make it happen it doesn't really end there as luck would have it there are some more configuration things you probably want to think about

although the audit log does record most of the things that go on go on and office 365 that you care about it doesn't include like all sorts of specific operations that occur with when users interact with their mailbox so every time they log into their mailbox when they create objects delete objects move things around create inbox rules things like that and we'll talk in a few slides like why those events are important but just another thing you need to be aware of it's kind of a hidden thing that's not enabled by default and you definitely want to check it out and run some PowerShell to make it happen some weird stuff to note like that was confusing to me when I first

looked at this some things that are maybe not intuitive like how long those audit logs live actually depends on like how much you pay Microsoft so if you're using the very popular III subscription you get it for 90 days if you pay a whole bucket load of money for e5 which not not many people do I mean you get a year just something to be aware of also not intuitive all but like not all events actually show up and a lot of log in real time so if like if you're trying to run something down that's actually happening right now keep in mind you may have to wait like up to 30 minutes for some types of

events to show up and in some cases like for like it's your Active Directory it's literally 24 hours so you have to sit there and wait so it's kind of confusing but something to be aware of so let's say you do check all these magic boxes you've turned it online how do you actually access that data right a couple options you can use these or interface the awesome Microsoft user interface everybody loves you can run a search that way in the security compliance Center there's a powershell commandlets it easy as well if you're actually developing a product and you want to consume this data like continuously microsoft recommends that you use their management API for that

so once you've got all those buttons pressed and enabled and you've waited like a painful 24 hours for the data to actually start flowing you'll be able to query events but that's really no good used to youth like you don't know how to understand interpret those events right like how do you actually interpret the data that this thing records look at luckily for us Microsoft actually has put together a really good knowledge base article on this like just bookmark it it's invaluable it like pretty much describes every single field that exists in the art log it has a pretty good description of like what that field means there's some high-value fields here that you're you're very frequently

gonna using things like the user ID which is the user that did the thing there's a client info string which is like the user agent so like what application generated this request IP address obviously where the request came from and operation which is like the thing that the user did cool so we have like a we have auditing enabled we have a basic understanding of what those audit events look like so how do you start to leverage that for detection response we talked about a few weeds you can like query things ad hoc and look for stuff during the course of an investigation that's fine and dandy but that doesn't make continuous monitoring right so how do you actually monitor

this stuff in real time the easiest path forward there is to pick the data up to your sim of choice it doesn't have to be like any of these Legos but I've had good luck with them like a lot of these vendors it's just a couple of button presses and you'll be good to go one thing I'll call out like if you're not familiar with the juror log analytics you don't have a sim and you're going to which are already you definitely want to check it out it's literally a few clicks and a lot of this data just shows up automatically which is pretty nifty and they're rapidly developing into so they got a pretty awesome great language

all right cool so we've got data flowing now the challenge becomes how do you transform there's like hundreds of thousands of events that you're consuming on a daily basis into things that are actually actionable like that are interesting to you so let's say we're interested in for example an attacker activity users doing things that they shouldn't there's a lot of ways to slice and dice this data there's a whole bunch of different types of events in the audit logs but we're going to talk about three kind of high-level categories today one being authentication to being like post authentication so they've logged in what kind of activities they're doing are they doing afterwards and then we'll

around out with some examples of like say they pop an administrative user what kind of doors is out unlock for them so starting with authentication like right off the bat one really easy thing you can do is just like look at the source of where people are logging in from right this is not exactly a novel technique we've been doing this for a while with other sources of data and it definitely gets flack for being like over simplistic like you know filtering based on country it's easy to get around it right but at the end of the day this is a very low hanging fruit and you'll be surprised sometimes like what falls out of this so if you look at you know

where you're doing business as an organization the hotspots become very apparent the big red dots the stuff that sticks out that's really weird is like instances where like only one user has ever wanted from that location and it's like not a common travel destination right like who vacations off the coast of Nigeria joking aside somebody to do that once that was yeah that was weird cool so ok this is cool well it's a good start not exactly now what can we like take this up what else could we do is let syndication locks what are the things that's pretty cool about authentication in office 365 it's just like the sheer volume of data that you

get especially when you like enable mailbox auditing you actually get pretty granular information like where your users are on a pretty like a daily on an hourly basis so we can use that to we're an advantage to kind of track the location of our physical location of our users and try to identify maybe a compromised account right the general idea here is like you should only be able to travel so fast between physical locations unless you're shopping yourself like to a rocket like this guy you're not gonna travel faster than 650 miles an hour so when you look at user authentication you can identify things like this user normally like just logged in from their

New York office and then an hour later log in from Nigeria like they would have had to travel 4,000 miles an hour to get there so some common false positives I'll call out here that you'll run into if you try to do this VPN proxy services that's gonna throw you off if somebody hops on their VPN it's gonna look like they traveled all the way around the world in a few seconds and obviously they didn't and mobile networks can be kind of a pain as well so if you're connected to your corporate Wi-Fi on your phone you walk across the street to get Starbucks you're gonna bounce it around between Verizon your Wi-Fi Starbucks Wi-Fi it's gonna look like

you're traveling like all sorts of different directions pretty quick there's a couple ways you can pretty easily filter that stuff out you can identify VPN providers and like proxies there's services out there like IP hub that makes this easy you can filter out the mobile traffic based on the ISP that makes it pretty obvious and just user agents as well right

I love the Segway so speaking of user-agents I didn't pay user agents one really thing what this is another really interesting property about authentication right like is you get you can get a sense for what client application initiated the request now we all know like you can't necessarily trust user agent people can for July yeah right but all that aside there's some pretty interesting stuff you can look for right one you can just try to profile the number of devices that a user has right you think about the average user you're gonna have like your phone your laptop that's pretty much it once you start to see like three four devices that gets a bit unusual

especially when you start to see like flip-flopping between different operating systems like if somebody normally connects from their iPhone and then all of a sudden we see Android and iPhone back and forth like that's pretty weird obviously like very minor changes like version numbers changes in the browser that they're using that's less suspicious that's no big deal and the other thing you can look for with the user agents specifically with office 365 is like you can see if they connected to through OWA or through like an application installed on their on their device so like to talk through the table on the right this is an example of like a distribution of a compromised user

accounts user agent strings and we're just acting based on the frequency of the user agent string and we've normalized out the the version numbers so this just replaced with an uppercase D things that stood out about this case we can see already like the colored dots correspond with the operating system that's associated that user agent string so already this guy as like a Windows laptop he's got an iPhone and then he's also connecting via OSX so three devices like not unheard of but it's kind of unusual and then we also see like he connected via OWA which is pretty weird because we can tell from the user agent strings that he's got Outlook installed on his like his Windows laptop he's

using a mail app on his iPhone so like why what purpose would this guy have of like logging in for through it'll be way

there are exceptions to these rules yes some things that can help with this there's actually some pretty good user agent databases out there you can bounce usage of strings off and you'll get like hey here's the vendor here's the the browser it's associated with which definitely helps with enriching this data one of the property that's pretty interesting for authentication is just looking for like the use of anonymizer services in general there there are legitimate use cases for a non riser service we love VPN services in the security community obviously if you for example have an authorized VPN service that you install on like all of your laptops you could look for anything falling outside of that people using

other services one of the thing we came across here I'll mention like is like kind of evolving TTP's so like we've had a couple clients where we continuously kick these people out of compromised accounts and when they come back they always do something a little bit different right this client in particular had decided like hey we're gonna use conditional access policies we're gonna block everything outside of the US great so we would start to see these messages coming across like attempted off indications and they'd be blocked due to that conditional access policy so it's kind of interesting about that is if you actually read the nuances of Microsoft's documentation on conditional access policies that's evaluated after a successful

authentication so when you see this message show up it means somebody is where they shouldn't be and had the right password so that's kind of interesting to look for by itself even more interesting is when you correlate that with like a success from a proxy or VPN service what we were seeing was they would attempt to authenticate from a blacklisted country and then when it didn't work they'd hop on a VPN pop up in the US and success from there so queerly aims two things together

all right so we talked a little bit about what you can do with authentication events let's talk a little bit about like okay they've they've got compromised credentials they logged in they're gonna do a series of activities after that and what can we look for that might a flag is that something suspicious is going on we know attackers generally want to persist an environment which they they have compromised credentials they want to fly under the radar and they have some sort of goal in mind so we can take that and it's kind of already how about like how office 365 works to identify some behaviors that they may attempt to you one thing that we see all the time is

this idea of evil inbox rules so I'm sure we're all familiar with like the concept of an inbox rule we use it to keep our inboxes clean we will send all that spam to the trash we'll send those reports to reports folder very useful it turns out attackers use this all the time as well to send stuff they don't want you to see to places where you can't see it so if legitimate users using box rules and bad guys use the inbox rules like what's the difference how can you tell all right couple of things you can look for look at the action so users typically don't create inbox rules that delete messages like we like to keep our

data somewhere just in case we made a mistake right that's unusual you can look for actions where they're moving stuff to an unusual place we'll get into an example in a second and you can look for unusual keywords things like infected malware virus no I know what you guys are thinking right like you're telling me if this guy has compromised credentials and he created an inbox rule with the word malware in it why would why would you do that so let's talk about it so we've got three examples here we'll go left to right and these are all real examples that I sanitized from recent recent incidents so the first one on the Left somebody

created an inbox rule with the name one and it's deleting all messages that have any of these keywords so things like delivery failure document email spam virus hack it was kind of cool when you look at then you analyze these inbox rules as you can sort of understand what the attacker was trying to do you can get at their intent by looking at what these inbox rules do in this case this guy was trying to mask the fact that he had compromised an email account and was like spamming out email to a thousand other places and he's trying to essentially send all the bounce backs like to the void so the compromised user doesn't realize

that they've been popped right now obviously that's not pretty subtle one glance of that rule and you can tell it's pretty unusual but as I mentioned like as you start to kick these people out out of your networks they'll come back and they'll they'll evolve their their techniques to try to bypass any de Texas that you put in place so the second rule in the middle there is an example of that same customer they created a rule with the name period great but interesting instead of using subject keywords now they're looking at the from address so if you created some sort of detection rule that looked for keywords in the subject this would have ate your detection so they're

essentially trying to do the same thing right they're trying to delete any messages that are bounced back so they're doing that by looking for keywords like postmaster mail Microsoft in the in the from address so the last one is kind of an additional example of them evolving this technique this is probably one of the more subtle ones that I've seen right off the bat the rule name this is like the default rule name if you create an inbox rule so if you were just looking at that this wouldn't stick out to you at all they're using the from address again but what's kind of interesting here is they're no longer deleting the message so they only

caught on to the fact that like people are using that to detect badness and now instead of leaving it they're just moving it marking it as right and moving it to the RSS subscriptions folder which like I don't think anybody uses RSS anymore right you're never gonna look at that folder so that was pretty cool this one was actually a little bit different and they weren't doing spam they were trying to do wire transfer fraud so you can see they're masking replies that form addresses that had to do with invoicing payment why everything like things like that

so here's another interesting behavior that's almost never a good idea even if it is you were a user doing this on purpose you know office 365 you can configure your mailbox you automatically send every email to a different dress and when that address is not owned by your organization that's a terrible idea for a thousand reasons that I don't need to explain to you guys so it at best it's a policy violation and at worst this is a method for persistence as well the nice thing about this from a tax perspective is like this is a passive way to chill in somebody's inbox you don't have to log in and pull down their email every time you want to see it you

can just siphon it off to an email address that you create that looks very believable right so what we've seen we actually had an instance like where this was the CFO that was targeted and they went out and they stood up a gmail account that it was first named out last name and it looks exactly like him so if you just skimming by it but it told me it was not him so moving away from like mailbox activities one other thing you can look for us like obviously there are other services and applications in office 465 like SharePoint and onedrive depending on you know your attackers objectives they may actually start to poke around and use those services to

try to collect a airing there's some definitely interesting things you can look there look for there so when you request like a file on SharePoint or you view a file on onedrive that does generate events in the auto log and you can use that to detect a weirdness so one thing about the bat is just look for a bursty behavior like no users gonna manually go look at 50 SharePoint sites in five seconds right so when you see stuff like that that's that's kind of a red flag the other thing that's kind of interesting here is these concept of public links a very convenient feature I'm sure we've all used it regardless of the service in

office 365 if you've got a resource in SharePoint or a file in onedrive and you want to collaborate with someone you can press the easy button and it'll generate like a pseudo-random link that you can share with anybody in the world and they have full permissions to that file I'm super convenient right unfortunately sometimes this can lead to hairy situations when users share things maybe they shouldn't share so this is kind of interesting as well one thing that we've come across is the users accidentally sharing a parent folder that contains way more files than they thought it shared they thought it contained whoops so something to watch out for and just ask yourself hey hey

what are my users sharing publicly and you know you can actually view this view OTO long as well something worth reviewing every month or whatever just to see what's out there the scary thing is like this stuff never goes away like they don't expire right and once somebody sends that leak out you have absolutely no control over it so especially with stuff like this kind of makes you a little uneasy alright so the last category i'll talk to you here is like what if what if they do pop in admin obviously that kind of sucks and unlocks a lot of doors one of those doors is this idea of delegating permissions which is essentially impersonation I don't know if you guys

have seen master disguise it's a terrible movie but it does I apologize it's a cult classic I know but that's essentially what this is right executive assistants use it all the time to manage inboxes and calendars for you know executives bad guys use it all the time to you you know gain access to other people's mailboxes right so a couple things you can look out for there like if somebody's been delegated privileges to another users mailbox and they're not an executive assistant or somebody that you'd expect to have that kind of privilege that's super weird if an admin assigns themself permissions to a bunch of other users mailboxes that's kind of weird the other interesting door that it

unlocks you know obviously because you're using office 365 surprise you're using Azure is this idea of like persisting via app registration in Azure Active Directory the interesting thing here from an attacker perspective is like a lot of times these applications are not subject to the same security controls that you're using your multi-factor authentication your conditional access policies chances are your apps aren't subject to all that because you want those automation accounts to work right so definitely a stealthy way to persist if you're an attacker all right so like we've run through few examples of attacker behaviors detection use cases let's look over now to the investigator process this is kind of harder than you might

think if you're you know traditional security analyst security team you've had a lot of practice in enterprise forensics like how to run down a piece of malware on a Windows box like we're very practiced but when running down stuff like this stuff targeting your users in office 3c5 like there's a lot of things that are just different and aren't natural to you a couple of things changes like we're very used to the host right something goes wrong in my enterprise like what what's going on in the host how did I get here right in office 365 sometimes there's just simply no host to pivot to right so we have to change how we think gathering sources of evidence and the

sources like just the evidence itself is different than what we're used to and as a result like the investigative question like is it bad there's a little bit different from traditional instant response so like how can we overcome this insecurity when one approach that we've taken and have had success with that expell is this idea of a playbook so like I've been on both sides of this my history is like manage security services we have tons of play books I've seen getting bad play books I've written good and bad play books a couple couple key things like keep it simple keep it consistent and keep it clear I've had good luck with this format so I tend to

start off with just like general context about like what this use case is this is super important when like the problem we're trying to solve or the thing we're trying to run down is not natural to us so we want to include things like what is this attacker behavior that we're trying to detect it's important because it Orient's the analysts around the problem gets them in the right mindset and then discussing how we're trying to detect it talk about your detection methodology this again gets them thinking about well you know why this all it exists and helps set them on the right track to answer is it legit or is it evil also it's important to have like a guidelines

for validation steps one thing I'll call it here is I I've seen in some cases people go away to overboard like in defining process for like validating a thing and it gets to the point where like analysts aren't even making decisions anymore they're just following like and clicking buttons that's super bad like that kind of takes it in the wrong direction we want analysts to use their brains right so in the in the better play books I've seen it's generally like guidelines nots not explicit instructions right that being said examples are good and remediation steps are important as well again some of these things are not what we're used to like you're not going to pay the box

if somebody is 365 like crayons are popped so but how would you respond so we want to take the guesswork out of that especially when analysts aren't used to interacting with these services so you may for example document steps like this okay once you validate you have user credential controls that are compromised like figure out what user it was revoke all their session tokens and like reset their password for example and obviously these steps will be different based on your organization but like just giving those examples and especially when this like these are new muscles that you're building as a team are super important finally and this is kind of often overlooked like this concept of resilience which is like how

do you I should get better as an organization like yeah we have to put out fires we have to fix this using the short term but like how do we prevent this class of thing from happening and like what do we need to do security strategy wise to prevent this type of attack a lot of times this is multi-factor authentication you know whatever the greater security control is but it's important to track this stuff on an instant by instant basis so you can sort of sell that business case so this is example of little playbook looks like the in its full form again like providing context about the attack we're looking for how we're detecting it

and what you need to do about it as an analyst cool so I mean we've been doing this for a while it expell and there are a couple like themes that keep popping up one for sure is multi-factor authentication like everybody has been beating on that dead horse and it's way harder than it sounds but whatever you can do start with your admins start with your high-value who users but this seems to be the big sticking point for people the other thing that seems to be causing a little bit of confusion within security teams I've seen is like there are a lot of native security controls in office 365 and honestly they're continuously evolving and being developed like it

seems like every month or so Microsoft comes out with a cool new feature it's hard to keep track of all that but it's definitely worth reviewing like there's all kinds of cool capabilities like conditional access policies I talked about Microsoft has like DLP Mauer anti-phishing policies all sorts of cool stuff that you can leverage without having to like buy a whole new product in some cases yes there are different subscriptions so it depends if you want to go that route versus you know roll your own or buy a product but it seems like a lot of people just aren't aware of all the different knobs and levers that they have available to an office I

think that's like office 365 advanced threat protection you actually have like anti-phishing where Dell well I think they call them threat intelligence events or whatever it will though identify a phishing message and like tell you if it was delivered to users inbox pretty cool I think it's a relatively new thing and then just generally reviewing policy configuration so a couple of the things that we talked about like the ability for users to create public links to file so you can just turn that off and things like mailbox forwarding again you could turn that stuff off so especially as as you think about like how you're going to get attacked and maybe how you have been

attacked go back and think about those policies like what can you do to actually prevent that class of attack from from occurring again cool so just bring it all together again like office six office 365 is great we all of it brings a lot of cost savings cool new features we don't want the security of it to be an afterthought we don't necessarily want to overestimate the responsibilities of Microsoft or any other cloud provider right the investigations in the cloud in general are sort of different from what we're used to so we have and we have to make sure that we start to build those muscles as a team and in general just plan ahead for

account account compromise don't wait for that incident to happen before you start thinking about this stuff so cool with that any questions yes

that is a good call yeah actually Microsoft has a lot of pretty legit reporting now on is your security recommendations all kinds of good stuff there's almost too many features to keep track of but it's legitimately just a button that you click count I'm sure

yeah it's basically ignorant sis blitz right yeah I mean the parallel I would draw is like what if they shipped Windows without like the Windows Event log turned on though be catastrophic right like yeah I don't think you can press a button but you can use PowerShell to shut it off yeah it's pretty easy I don't think there's like a big like stop button exactly yeah the thing I've seen is like most people go with III and then they sort of have this philosophical like do I pay a whole like truckload of money to Microsoft to get some of this stuff or do I pipe the data off to assume and do some of the stuff

myself so we've seen people go both routes but there's a lot of options yeah that's an understatement yeah yes sir

those sharing files that's a good point so by default they allow you to create public links and you can share stuff to the world right there are pretty granular controls one of the things like the easiest things you can do like obviously you don't want to hamper people's like work workflows right you can just force them to actually share it with specific people in your organization versus a public link which is a lot safer right because then only the person that you actually granted the access to has access to it and you can actually do that like per user for a group it doesn't have to be an org like thing so like you could you could allow

certain groups or people to create public links if they'd you really needed to but locking that down seems like a no-brainer and I think it's almost just as convenient to like force them to share it with a specific person or group versus publicly right is that alright thanks guys thank so much

you