Redefining Threat Modeling: Security team goes on Vacation

all right i think we're ready for the next talk uh so please welcome jivan singh uh jivan is a security engineering manager at twilio uh where he's working on embedding security into all aspects of the software development process uh jivan's gonna be talking to us about redefining threat modeling [Music] [Applause] i got a little fan section here well thank you all for attending my talk um i definitely want to talk about how we can redefine what threat modeling is so that security folks don't have to be everywhere all the time and uh let's make let's actually make security everyone's responsibility um so that we can actually take longer vacations all right um i always like to start off with a

quote you miss 100 of the vulnerabilities you do not threat model the great famous michael scott said that he's a fantastic security engineer um we should definitely follow his advice okay what's on the agenda for today we're going to cover threat modeling basics i want to make sure that we're on the same page of what is actually threat modeling we're going to talk a little bit about the self-serve threat modeling program what does it actually mean we're going to dive into how we actually run it at segment um then we're gonna do a program retrospective there are great things about it they're also challenges we want to make sure that we talk all about it

as part of this talk okay um who am i i'm a guy that did way too much software development in my career um i've always been really really interested in security we had a great vp of security at one company that told me to join his team once i did it it literally changed my life it's the best career decision that i made and i try to do that for others so i'm part of oaas vancouver i try connecting with other oasp folks because security is transformative i was going to talk about the fun fact of being real good at uh knowing my family but another fun fact i'm really good at estimations i think

my boss and my children believe that for example i believe that there are between two and three thousand people in this crowd i think there's about 2314 right now so yeah my boss loves my estimation abilities okay so let's talk about threat modeling uh again i don't really want to talk too much about the depth of what threat modeling is this is not what the talk is about but i want to make sure that we're all on the same page what is threat modeling the main goal of threat modeling is to determine the assets in your system and the risks to those assets threat modeling should be simple it should be transparent and everyone should have a voice during threat models

everyone's viewpoint is different everyone's career experience is different and when we all work together we can make much more robust threat model in general and why is it important well we always talk about shifting left and how important it is to shift left well threat modeling is actually done in the design phase it's possible that you can actually fix vulnerabilities before they act you write a single line of code and it's really really cheap to fix the design rather than if you discover something once it's been launched and you have to re-architect it so it's super super cheap from that aspect it's a great way to discover assets as again threat modeling is you have to

discover your assets and then you want to talk about the risks to those assets so when you architect a solution and you know what your assets are if you're an attacker you want to start thinking about how you can actually attack that asset or if you have a defensive mindset you want to protect that asset so when you discover those assets as part of your threat modeling exercise it just makes it a lot more resilient um we know as security folks that we can't fix all of the things it's not possible and it's not good business wise so threat modeling really helps prioritize your remediation efforts when you discover the risks to your system you need to figure out which ones you're

actually going to fix so you want to prioritize those and for me personally i love threat modeling documentations especially of old when i join a new company or a new team i want to see how they threat model how do they think about risk and when you have a bunch of different documents on it you can be on the same page with your team um or you may discover that they need to mature so you want to as you discover how they do threat modeling you can figure out how you can actually grow that particular team another great thing about documents is all threat models go stale so uh today it might be a low priority

vulnerability a year two years from now it might be a high priority availability so it's always good to look at your documents and my favorite is i make a ton of assumptions as part of the threat modeling process and it's always good to have someone validate it so if you are able to share your threat modeling documentation with security researchers it's a great way to get a quick check on if if your assumptions were valid or not and if they were not valid then you can actually look at additional controls so that's what threat modeling is let's talk about redefining it we all know threat modeling is really hard to scale is there a way to make it easier and i

should also note this is just one way of looking at self-service threat modeling there are a bunch of different ways that you can actually do it so this one's worked out really well before i get into what self-serve threat modeling is let's talk about potential problems so threat model is really hard to scale it is really hard to scale you in most companies you only have very few folks that actually know how to threat model and as that company grows it's not like you're going to get a lot of more folks that have that ability to threat model and if the company grows really quickly the issue can get worse and development might be blocked by security

so if development starts getting blocked by security or if there's going to be a larger queue of threat models that need to be done there's only a few ways that you can get out of that situation so either security kills themselves by threat modeling everything which is not realistic you want to burn out your security engineering team the development team will have to wait for their threat models which is not great for a culture you don't want development team waiting weeks months for getting their systems threat model or security will have to pick and choose which things that they want to threat model which is usually the situation that really happens the security team will

have to pick and choose the items that they that of highest priority so and that's not great because you're not going to have eyes everywhere where you might need it so um and i actually believe that security engineers should focus on the most important problems in general threat modeling might be it but it might not be it um so if you are you have so much operational load where you have to threat model all of the things you're not going to be actually focusing on things that might be way more important for the business might be implementing security tooling looking at things that maybe actually threat modeling is but you can't get to all the threat models

so you we really want to make sure that we are able to allow the engineering team security engineer team to have the autonomy to figure out what they really need to focus on okay so in my utopian world developers are going to do the vast majority of threat models that means that they're going to identify the assets they're going to discover risks to those assets they're going to prioritize those risks and figure out what they're going to remediate it will be the responsibility from end to end to do all of that the security team should really focus on training developers to become better threat modelers they're going to be gaps as part of the training process and you'll discover

that when you see the threat models you have to understand what those gaps are and build it into your training program new employees will get all of the training but you should also be building training for those that are good at it to make them even better we want everyone to be better and better and better ideally they can become engineers can be as good as intermediate security engineers or even senior security engineers self-serve threat modeling is a great way to scale with any size organization if your engineering team knows how to threat model doesn't matter if you have 100 engineers or 10 000 you're going to be able to threat model all of the things

and last my favorite goal is actually to make a bunch of mini security engineers throughout the organization when you have a bunch of mini security engineers they're going to be finding bad things all over the place and as an engineer if you find something that's wrong you have incentive to actually fix it yourself you're empowered you want to fix the problems that you've discovered so it just that cycle is so much better that way the centralized security model is dead we cannot have engineers come to us and ask us about security problems we really have to push all of our knowledge to the engineers themselves they have to understand security they have to own the

risk themselves okay so does that solve our problems threat modeling is hard scale well if engineers are good at threat modeling it doesn't matter we are going to be able to scale for any size organization will engineers be blocked by security well nope since they are able to threaten model themselves they don't have to wait for us they can go ahead and do what they need to do and finally security engineers will be able to focus on the most important tasks and it might it might exactly be threat modeling but they are able to have the autonomy to sort of figure out what's the biggest risk to the systems okay does that mean it's vacation time

well unfortunately just not yet uh self-serving doesn't mean that security is no longer involved as part of the threat modeling process i say that they should be doing the most threat modeling but me personally i still want to be a part of the conversations where there's authentication or authorization if there's phi or if there are services that are directly onto the internet i want to be part of those conversations i want to make sure that we're making the right decisions there um self-serve modeling also means that we cannot blame developers if they miss something so if they miss something there's something fundamentally wrong with a program we and if it's a similar sort of problem maybe we're really bad

at broken access controls we want to build that into the training so that they get better at discovering those things and unfortunately it also means that we're not out of a job as we all know if we have one of our responsibilities taken away there are going to be two like a security hydra two of them will pop up in its place okay how's this program actually set up well first you actually need to get buy-in and buy-in shouldn't be that difficult it is challenging but it shouldn't be that difficult you need to get the engineering and the executive teams to buy into the program and mostly because it is a big commitment we're shifting a

lot of responsibilities from the security team to the actual engineering team but you want to hit on the key benefits you're going to have faster development time and better security so faster development time because they're no longer being blocked by the security team for threat models but we're also for every single feature we're also looking at threat modeling which we're doing security by design and by the time it gets to production we should have fewer vulnerabilities in general and there is minimal engineering time required for training and i'll talk about that when i hit the other sides but there's absolutely minimal amount of engineering time for training required okay four phases training phase observation review and security optional

my opinion it's really training and security optional um training phase observation and review phase are just different sorts of training phases and security optional is when it's actually security optional and i'm gonna break it down for all of them so self-serve threat modeling means engineers have to know how to threat model it kind of makes sense um but we have to make sure that we enhance engineering teams motivation to learn they want you should want them to want to be there and in order to do that for me um i would need to make sure that the material is very fun and engaging if it's not fun and engaging i'm gonna it's gonna glance over for me

so you wanna make sure that the sessions are interactive they're small they're intimate so that you could actually talk to the individuals they're not afraid to speak up so that they can ask questions ask a lot of questions for the group themselves keep them on their toes and you have to realize security is a marathon and not a sprint you don't have to give all of the training at one point they're employees you can give deliver training all all the time they're they're with you most of the time and it makes it the training an easier way to learn when when you can spread it out and also teach engineers things that are relevant to your business

so it should be done in the same vein threat modeling training should be done the same vein as how you threaten all in your organization so if you do if you use stride to discover vulnerabilities teach them how to use stride if you want to do risk assessments teach the engineering team how to do risk assessments the goal of the training is to train the engineers to retain the most amount of information with those principles in mind we actually built out the training to have uh three distinct trainings over a six six week period so week zero we had one week three and week six and we broke it out because anytime i went to

black hat or an oauth training i didn't really retain too much information i get eight hours 16 hours of training and i i i want to say i think i'm smart but i probably retain about 10 of what i got trained so these are your employees break it out make it digestible make sure that they can actually learn the the first training that we did uh we i want to introduce the concept of threat modeling so we talked about stride with a lot about the theory we did very little actual hands-on because i really wanted them to understand the theory the second one we introduced assets and data classifications which are important to our business

so we got to get them once we introduced those topics it didn't take too much long too much time we got them to do a lot of hands-on threat modeling at that point so get their hands a little bit dirty a lot more than the first session and the third one ideally you're threat mulling with a single team you're teaching a single team find something that they own find a feature that they own do a deep dive into it and teach them how to do threat modeling from end to end as part of that session exactly how you do it in your business the good thing is segment has allowed us to open source the training all the

training is open source please use them don't start from scratch i'll provide links at the very end so a few takeaways from the training phase it takes years for security folks to get good at threat modeling i know it took me a long time so don't expect the engineers to get it right away be there support them build them up and you need to bring the training to them um and the way that i did that is we started off with physical concepts that they understood personal safety i talked about a house how do you secure a house what are the type of threats there and once you move from once they understand the different type of

vulnerabilities as part of that you move it into the software world you show them okay it's the same thing but it's just different terminology it's the same thing at all and then lots and lots of hands-on practice i need hands-on practice when i'm learning things so making sure that they had it and part of it we actually broke out into many groups where people were able to discuss some of the vulnerabilities that they discovered as part of the training exercises um failing fast uh training you can spend so much time on building up your training um don't do that fail fast come up with the training deliver to engineering team and iterate on it i struggle for about a month with the

training my boss smart guy he pushed me to just put it out there and iterate on it and it took me two weeks to get it to 98 percent and training is hard over zoom i've always done workshops in real life i can look people in the eyes i can see if they're actually understanding things training was really really hard to resume i got people to keep their videos on and i kept everything engaging so i asked random people questions so they were always kept on their toes and they were kept on uh they were definitely kept on their toes so much so that i remember one engineering manager was so afraid to go to the washroom because he might get

called on so definitely keep that fear in them for the training so that they're engaged all right and then we have a lot of real good testimonials if you make the training really engaging they want to be there they want to learn more and they're going to deliver your threat models even better if they really want to learn the material nothing can replace good training okay so observation phase i sort of mentioned that it is training version 2. you want the engineer themselves to put together the entire documentation they have to lead the sessions themselves you're going to security is going to be there they're going to be observing what's going on you want to make sure that they

can again the things that they need to do is identify the assets figure out the risks prioritize the risks and figure out which ones they're going to remediate you'll have an engineering dri and you make sure that they know what their role is as part of this process the goal is to get engineering to do this by themselves and to find all the critical and high vulnerabilities on their own you don't want to be telling them what they are as part of the threat modeling process so some takeaways be a coach be a mentor don't tell them what the threats are as part of the threat modeling sessions you actually want to just let them co try to

give the right questions to spark that idea in their eyes so that they discover it and then they can do it on their own next time i actually helped a lot of the engineers with the documentation so a few days before the threat modeling session themselves i jump on a 101 i talk through and say okay these are your responsibilities you have to lead your leader in this threat model you have to make sure that the people are talking everyone's everyone's bringing their ideas and then it gives you an opportunity to actually sell threat modeling thermal is a team support so you make sure that the dri that's responsible for the throttle engineering dri they're

bringing the right people into the threat model and they need to make sure that uh that again again that no one's quiet as a part of it and ideally you can close the loop with the engineer afterwards so if they did a great job let them know that they did a great job if they needed a little bit more coaching give them a little bit more coaching please note some people are really really natural at it literally the first self-serve threat modeling session that i did was me and a principal security engineer we were doing it with two senior engineers and we provided zero value as a part of that session i felt actually i felt great that the

program worked but i felt bad that i wasn't able to really provide anything to them they felt like they already learned everything so super super proud of those individuals again close the loop uh this particular engineer did a fantastic session he was without prompting from us he was calling out everyone that was really really quiet in the session i had a conflict i had to leave halfway and i felt comfortable leaving halfway because i knew he was doing such a great job he's still one of our top threat modelers review phase this is where things are a little bit more challenging security is no longer in the room we're going to actually be looking at the

artifacts and making sure that everything looks uh well so again their engineers are going to keep iterating the four things they have to discover assets they have to figure out the risks they have to prioritize the risks and figure out which ones are going to mediate they're going to do that all without you and you're just going to look at the artifacts and you know that they're doing a good job when they have all of the critical and high vulnerabilities done by themselves in reality we've run into this situation a lot of times when it should have been the observation phase some folks like me i don't follow instructions very well some folks they were

they thought they saw the threat modeling section in the design document so they filled it up when we got to the threat modeling session it was fantastic um there were 90 percent of the risks were already covered we just had to push them to find the last 10 percent um hopefully when we give the talk next year we'll give you a lot more actual metrics of how this all turned out on the review phase security optional uh everything should be threat model like literally everything should be threat model all your developers will know how to threat model so there should be no part of your system that is not getting threat modeled security should only be involved in the

most sensitive things i already talked about being part of the authentication authorization phi anything on the internet security is literally optional we should be spending times on the most important problems if it start modeling sure but if it's not threat modeling let's spend our time there in this phase we should start seeing a drop in the number of vulnerabilities that are being discovered as part of third party tests so bug bounty pentest we should see a significant drop at that point and hopefully we have to actually pay our researchers a lot more to find vulnerabilities as well so that's another signal that you can use and for now you can kick up your feet for reals

okay program learnings and there were a lot of things that we learned about this particular program so this isn't rainbows and cookies uh this was a really really challenging program there are a lot of challenges as a part of it we'll start with it is a multi-year program i'm going to emphasize the year part you need it can't rely on a single dri security to actually run the program especially in this hot market it's uh you don't rely on one person but you also don't want to burn out that one person running this sort of program as well so you need to make sure that you have a group of folks that are actually running

this program they're all on the same page and it's a large and key investment from the secure team and actually it's a large and key investment from the engineering team as well so you need to make sure that there are a group of folks that can actually run it themselves training is hard training is really hard and it's hard for a lot of different reasons first reason is you have to train a large number of people so as part of the program i personally trained 120 engineers it was the course of two quarters and i burnt myself out with trading um definitely don't do that make sure that you have other trainers as a part of it

uh i took a quarter off from trading's because i just didn't want to do delivering the same training in and out day and day kind of can get boring as well so you might want to make sure you have a stable of trainers as a part of that um have you ever delivered someone else's presentation i've done that in the past that's really hard too it's not your content you don't own it you're not really super familiar with it as well so you want to make sure that the trainers that you have you give them the autonomy so that they actually own the training themselves let them modify it make it their own you're going

to be covering the same sort of topics but it's going to be their own and they can deliver that training much much better themselves you still have your regular threat modeling challenges are devs threat modeling all the things um maybe maybe not so you've got to make sure that you continue to focus on uplifting your threat modeling program we have some challenges on the segment side where we weren't sure if developers were threat modeling all the things and the way that we were storing our threat models was all over the place fortunately twilio is a little bit more systematic we have a single or folder where we store things but there are some learnings that from the segment that we

can do things a little bit better i talked about training folks at segment twilio is a lot larger than segment it is considerably larger it's 20x size of segment so scaling this program to large enterprises is super super hard we have maybe now 200 engineers on the segment side we probably have 2500 3 000 engineers on twilio so you really have to have a strong investment to and you have to have a strategy on how you're going to train everyone so it is not an easy program at the enterprise levels i'm hoping that someone on the team can talk about it more next year we'll be definitely rolling this out at the twilio scale at some point

lastly it's really hard to maintain a good ratio of people that are trained i had wrote this note before the market got really hard and there's a lot of attrition it's doubly triple even harder now a lot of the trained people leave the organization and you have a lot of new folks coming in so the ratio is really really tough to maintain a good ratio is really hard to maintain so um i talked about the challenges but there are a ton of wins as well we are threat modeling so much more it's not it's night and day with how much we're threat modeling now the organization is less afraid to threat model it's achievable they understand they

understand the process engineers understand the process they're not afraid to reach out so we they're doing it a lot more often we embedded the threat modeling template as part of their design document so they are always aware they know what the steps are they have to take in order to properly do their design document um engineers are really good at threat modeling it is crazy how good they are at threat modeling there are times where we've been doing threat modeling sessions and they know their system so well they know everything where all the bodies are buried into the system so when you tune them to find discover security vulnerabilities they're going to find them security folks like me we come in we

learn about your system it's an hour two hours maximum to learn about your system and we're supposed to find all the vulnerabilities uh which is not realistic um there will be situations where it would take me maybe eight sixteen hours to find vulnerabilities that engineers are able to find just because they already know their system so it's way way more proficient to actually train the engineers they don't have to find all the big things they have to find big things but they don't have to find all of the things so again i always harp on making sure that they find the critical and the high vulnerabilities we're going to be part of all the threat

modeling sessions where it's really really high risks risky areas of the system there we're going to find all other things but all the other systems that we don't find are high risk they'll be able to find the critical and high that's and that's good enough for us um our security culture improved leaps and bounds and it's been crazy how well our security culture improved one of the stories i have there is that at the time a lot of our security engineers were on the pacific time zone and one day segment is a golang and node shop and one day node dropped a high vulnerability patch so one of the engineering managers on the east coast he discovered that

vulnerability he's like okay this is applicable to my team he patched his team he notified the two other managers that were east coast and they were a form they had put plans into place on how when they're going to mitigate it this was all before anyone on the pacific coast came even online so there were a lot of different anecdotal situations like that where it just vastly improved our security culture in general and i'm going to harp on it again failing fast was critical to the rapid improvements both with the training itself and the process as well so with respect to the process we didn't really know when should we promote someone from the observation phase to the review

phase fail fast figure out what's applicable for you and your company all right resources so um i sort of mentioned that we have the pdfs of the training available um it was it's very close to what we used internally for our trainings we removed some of the vulnerability information i didn't want to share those things with the open source community so we have a blog post on it email me at sstm selfservice.modding at segment.com or dm me i am very open to sharing our experiences there's so much we can do as part of a talk but i can get down to the nitty-gritty i already talked to at least a couple dozen companies about how

we have implemented it i don't mind spending the time i feel that uh like we we are all a community and i want to emphasize that we're all a community we need to work together to make our software safer so if you need to reach out to me i have no problems there will be times where i need to reach out to you to better understand your systems as well we don't need to hide things from each other all right thank you [Applause]

do we have any questions yes

a great question okay the question is how do you overcome the challenge of finding bad things in your own system because you're sort of like a parent to this code it depends on people's personalities as well i remember working at a company a few companies ago where i spent five years building this one system and then i knew for the business to move forward i have to put all the effort to actually kill this system and so we can move on to the next generation so some parents are bad like me there are a lot of good parents where they want to protect their child i i don't think i came across that sort of

problem um as part of the process we did have folks that were just harder to for them to understand vulnerabilities as part of their system we tried coaching them so again as part of the threat modeling the actual sessions where security as is in there your job is to coach a mentor i don't want to tell you as a good parent i don't want to tell my kids if they're i'll hint at them when they're behaving misbehaving but i want them to discover when they're misbehaving so similarly i want to make sure that the the engineers themselves are finding the vulnerabilities themselves and if they aren't i'm going to drop hints if they aren't i'm going to point

a little bit closer and if they're not then i finally will actually tell them what they are but these are mentorship sessions we want to make sure that we're continuously telling them and getting them better as part of the process yeah go ahead okay

yeah uh that that really depends on sorry the question was how long does it take uh engineer with no experience to actually be good proficient at threat modeling that really changes uh with engineered engineer um most senior staff principal level ones they didn't require much training like they did the training and then shortly afterwards maybe a threat model maybe two they're already really really proficient at it it was the junior and the intramural folks that really needed a lot more help which is okay they're junior and intermediate folks they they that's why they're at that level but we're there to coach them and guide them and then the ultimate situation would be if you train

the engineers so well they can deliver that training they can deliver that mentorship so that circle as part of their team is closed very quickly perfect i i know i had a question here first

good the question is around how do you how do you deal with the pushback from engineering management management so we talked a little bit about the amount of threat modeling training there's three training courses it was a total of five hours first one was hour and a half second one hour and a half last one was two hours five hours of training is not bad at all and then the threat modeling session they have to do them anyways they're supposed to be every feature should be threat model so we're adding maybe about another half an hour to the engineer's time as part of that threat modeling session so the buy-in itself was very easy to implement

and at the at the end of the day we we sold them on the fact that your velocity is going to go up considerably so first we're going to be threat modeling all of the things so we're going to be having a lot theoretically we should have a lot fewer vulnerabilities if you have fewer vulnerabilities it's going to be fewer hours put into doing security work and it'll be those hours you can use for development and future work so that's how we sold it to them told them long term you're going to be saving a lot more time in general perfect yes this is yeah we can definitely tie it to eric's budget talk close the loop

so the question was what's the ratio of security engineers to engineers and especially as an organization matures i've seen it different at so many different companies it really depends on the companies themselves but again the program's goal is to actually build mini security engineers everywhere and at companies that are smaller smaller startups you can do that considerably faster at larger enterprise you have to be very focused on the people that you want to actually develop so maybe you want to develop your architects first because they have a lot more sway in how the technology is going to be built so you spend the vast majority of time there or maybe your principal engineers in general so at a larger organization

you want to be very focused on the type of folks that you want to actually train perfect yes

yeah so yes great question by mistake we discovered that the best way to oh sorry good question okay how do we document vulnerabilities discovered as part of the threat modeling process by a happy mistake we found out the best way to find to create vulnerabilities threat modeling vulnerabilities was actually create vulnerability tickets so that we actually filed real text after every threat modeling session and it was great because then we can keep track of it if it's through documentation we lose track that either the threat modeling ticket stays open um and it was just a challenge that way so we just actually filed real tickets they'll have to follow slas if they end in slas it's

it's kind of weird because it's not technically a vulnerability because it's a design flaw at that point but they'll come up with slas and as part of the sla extension process we can sort of figure out hey is this actually going to be do we are we actually continually working on this feature was the priorities changed so we have those discussions as a part of that whole process perfect yeah

love it the question is how do you ensure the quality of threat models stay well over time during the security optional phase uh one of the things that we want to do is make sure that we continuously take a sample and we're actually reviewing the artifacts after the fact so that you you actually can make sure that the quality of threat models stay well over time and in addition to it you should continuously build more threat modeling training so you have your 100 level you should have your 2 and 300 level ones so that you continuously training folks that have been threat modeling so that they remember and do other things as a part of it

perfect all right oh sorry yeah

perfect the question is how do we is threat modeling the only thing that we should do or are other solutions like using product management to really build out security user stories as part of their process um i think um when looking at kovid we want to use a swiss cheese model where we want to wear masks we want social distance we want to do all these other things we want to vaccinate and same with security so if we can get our product managers to do the security user stories we threat model we do bug bounties we have security tooling all that combined will make sure that our products are really really safe so i wouldn't say that there are different i

would say that we will work on those on parallel to make sure that we have a much more robust life cycle

great question so the question is security should still be involved with threat modeling especially the more sensitive areas how do we ensure that engineers are looping us in it depends on the sort of situation context i know that security is involved with a lot of conversations just in general with the engineering team um and actually the engineering teams most engineers want to do the right thing um and they will know most engineers will actually know the processes so they'll know that and you can also have small little security questions to really help you tune that out as well so there are a bunch of different strategies for that so if you're a smaller company where you

just have uh documentation engineers will reach out to you they'll let you know but at larger companies you have a ticketing system make sure that they're answering really easy questions that will let you know if you need to be involved or not i should also note that's a great question in general not all teams are at the same place so some teams are going to be at the training phase other teams are going to be at the review phase someone might be at the observation phase not all teams are alike and you want to make sure that you cater it for your teams yeah

yeah that's a great question companies big and small when do you want to actually push push on that investment that's a really good question i guess i would say that when some of the foundations are already built um so that's a hard one yeah i would lean on the foundations and it has to be the next evolutionary step for the organization because you have to have the right buy-in from management engineering management and executive leadership so you you have to make sure that it is at the right priority level it's a really roundabout answer but uh yeah you'll know yeah i'm sorry i can't hear you

ah perfect so the question is how do you keep the engineers updated on security vulnerabilities what we i we don't want great question i want to make sure that we distinguish between security vulnerabilities uh as well as your systems so secure availabilities are always going to come third party vulnerabilities you're gonna have pen tests and all that sort of stuff that's always gonna come you're gonna have a workflow for that you shouldn't have to worry about it what we're really looking at is betting security during the design phase let's architect these solutions to reduce certain type certain classes of vulnerabilities just in general so it doesn't matter if log for j is out there that is going to be a completely

separate type of work than actually looking at your system and figuring out how you can architect it in a way to reduce certain type of vulnerabilities in general so separate things but both of them are extremely important

yes yes great okay um the question is how did we actually roll out our program we did it by team um so on the segment side um we focused on individual teams i wanted the conversations small and intimate i wanted people to actually learn my goal is to actually teach them so that they can learn about security so we rolled it out with teams with the last training to be on a feature of their own but after a certain amount of time you've already trained all the teams and there's new people coming into the company so what we would do in that scenario is we would put all these folks that are from different teams together

we train them up and then we would do a threat model of one of the systems that we have to threatmauld anyways so that they get trained up on that so a lot of the statuses on which phase they're on was based on teams as well so i sort of mentioned earlier that some folks were in the observation phase and some were in the review phase so it's we made it mostly team based from that respect as well

that will be a talk for another day eric built up a way to incentivize folks to really do security work so um it was our security leader point board i think it was leaderboard that's right so security leaderboard it was fantastic it really you saw how folks wanted security points so finding vulnerabilities writing threat models fixing vulnerabilities would give them points and there was actually competition for being on top it was a great program yeah yes

perfect the question is is there a mechanism to share vulnerabilities across the entire platform so i sort of mentioned that self-serve threat modeling won't solve all of your problems so those are some of the problems that we continue to have but if you do build out maybe with sharing you might you might want to make sure that there's certain attributes that you want to label there is x accessible abilities or broken access controls you can still label those attributes to see the type of vulnerabilities that either you're missed by your threat modeling program or things that we discovered as part of the threat modeling program to sort of know where their gaps are as part of the program

and add more training there but no we personally didn't but that's a fantastic idea in general yeah

perfect i think we're good um again feel free to hit me up sstm segment.com hit me up on twitter i'm more than willing to chat more about this there's a lot of great reasons for doing a program like this there's a lot of challenges i want you all to know the investment beforehand but i don't mind uh walking you through how we did it in any particular details that you want all right thank you all [Applause]