Securing your Open Source Project

[Music] my name is Jose Pala Fox and I used to live in Portland I'm really excited to be back here for this event so thanks so much for having me um I moved to Portland in 2007 uh when I got a really generous package from Reed College uh so finished up at Reed and uh helped start a software company here called Puppet Labs uh if anyone's familiar with with puppet uh we did really well for a couple years and then you know didn't didn't end out so great but you know it was a really fun ride and um you know Portland was my Portland was my first real home you know where I felt really

comfortable so I'm just super happy to be back here so thank thanks so much for having me um I work at GitHub and my job is Loosely to work with all of the largest software companies in the world and talk to them about open source security um help them adopt uh better security practices and help them build more secure applic that we all all use so what I'm going to talk with you about today is really um kind of the state of GitHub and and maybe why you should care about it as a security professional um and and I'll take you through some of the things that we've been doing at GitHub over the last 5 years or so uh to

really improve the quality of um security of Open Source packages on GitHub we've introduced a ton of tools and to share kind of what those things do how you can use them for free on GitHub if you you interact with GitHub um and then I've kind of got a Persona based approach to give you some actionable things that you can take away today kind of no matter who you are in the room um there's stuff that you can do to help us secure the open source software on GitHub so uh that's sort of the the structure structure of what we'll do today um so so what is GitHub who has a GitHub account everybody all

right fantastic tech room we nailed it got it all right so you are one of the 100 million plus developers that are on the platform but it's also the home to most open-source software um and that's probably how people are first familiar with with GitHub I talk to students all the time and they'll say GitHub that's the thing that I download you know our scripts from and I really like that and like a yeah helped you pass your stats class um so you know it's it's great it's a place where people come and they share they share code um and and why you care I think about it is because open- Source really changed the way that

Enterprise applications are built and you know anything that we interact with today whether it's you know the wristwatch that we have or the phone or the car we're driving in or the elevator we're riding in most of the software that runs all of those things is based on open source software which hopefully is a truism at this point this used to say open source is changing but I I've changed it to past tense because it just seemed too too obvious that um we use a ton of Open Source now and and all of that lives on on GitHub um a couple years ago we started having this problem on GitHub and and it alerted us you know

to this responsibility that we have to help secure open source software people were leaking their API credentials all over the platform and this happens you know 300 times a day people will push a live GitHub token to github.com malicious actor Will scoop that up from our events API and then they'll take over that account so not only were we dealing with um a lot of account takeovers um you know it was starting to put some of the open source software on GitHub at risk because if a maintainer lost their own identity on GitHub you know somebody else could uh create a supply chain attack out of that you know identity of the maintainer that owns

like a popular package uh so impersonating somebody on GitHub is a really bad thing and this was happening you know 300 times a day so we had a a serious problem so we built some tooling I'll talk about to address this but this was like the first big like Whoa We got to do something um you know this is more of a longitudinal thing that we looked at over time but what we saw over the years and from running static analysis tools on GitHub uh the number of security vulnerabilities that we found on the platform basically stayed consistent with the number of lines of code that were being written so even though people were becoming more aware

of security vulnerabilities and you know better application uh coding practices we were still introducing loads and loads of vulnerabilities and that's what you see with this blue and red line basically staying in the same same curve here this is a complicated Double Y axis and one of the lines doesn't have a uh an axis so you know bear bear with me um I promise it's not false news um the purple line there is really you know people becoming more aware of it you see this sharp spike in uh 2018 2017 time frame where people you know started commenting a lot about how their PRS were resolving security vulnerabilities but still uh the number of vulnerabilities stayed about the same

ratio with with lines of code written so um you know the situation wasn't getting better so so we said okay we have this Secrets problem we have a problem with introducing uh you know code vulnerabilities um and this is happening all over the platform and when you look at uh analyst data you know um they'll tell you kind of the same story in these big numbers and what jumped to me about this data is really this last bullet 81% of devs still choose to ship vulnerable code to meet deadlines and as security people I hear all the time talking with you um that you just feel like your devs aren't listening or they just don't care does anybody feel like

that no sometimes few okay no one wants to admit it's all right I've I've heard people you know you get a few beers into people and they'll really Let It Go but it's okay you know sometimes uh your devs aren't listening to you right and and it's not that they're not listening to the advice you're giving them but they're inundated with so many things I mean a lot of stuff falls on a developer's plate all all day and as you know working at GitHub um we're we're advocates for developers we we really uh try to understand what's going on in the developers um day-to-day and how to help them and and we went out and talked to

developers about why they're ignoring security vulnerabilities like what's going on that's causing them to just sort of like bypass you know any noisy alert you stick in their face um and they told us two really big things one is that there's two to many false positives um you know this industry went through this huge ground swell of VC investment and now there is an alert for every single thing that could possibly happen in code and developers are inundated with with thousands of these um you know if you work at a knock maybe you're like Ah that's my job I see lots of colors and I I like responding to that but developers don't and um they're

you know they train themselves to be efficient and they see too much noise it it you know it's something they go and investigate and it turns out to not be a problem to them and then they just stop paying attention to it after that um the second thing they told us is that context switching is more expensive than people realize and I think as technologists we understand this but you know the frustration that developers go through when they're trying to remediate security vulnerabilities is sometimes really arduous like they don't have the right permissions to view the dashboard to understand what the alert is so they could spend a week trying to understand if a vulnerability that's being reported

to them through you know maybe some internal tool that pings them on slack is really something that they have to care about and by the time they get to it it turns out to be a false positive or something that they don't care about and that whole cycle just discourages them from ever paying any attention to security alert so we've got to fix that right there's a user experience problem um that's going going on here so we uh we took kind of a multi-pronged approach here um there's an aspect of this where we need to introduce tooling and so we'll talk about some of the tools that we introduced um we also had to go and

educate people and so part of uh me being here today and my wonderful marketing team uh giving me funds to come out here uh you know is helping do some of that so I'll help kind of share uh the goodness that we're working on so so that you're aware of it and and three we're building a security community so um I'll share some of the resources around how you can join that security community and what assets are available to you there um there's some really Nifty stuff we have uh on the platform I want to spend a little bit of time talking about three of the features that were on that um page and this is

all free stuff on github.com so if you're an open source maintainer or you use GitHub uh you you this isn't like a sales pitch for you this is uh tools that are freely available um you know they're just buried in settings pages and so I want to come and kind of share where to find some of this um but the three big Investments we've made are in static analysis uh the first static analysis investment we've made is code scanning so this is uh an acquisition we made of a tool called seml uh seml is a static analysis security testing tool um it identifies you know insecure coding patterns in applications and it's built directly into GitHub so on every poll

request we can run a static analysis check to say hey does this new code contribution introduce any vulnerabilities and if it does then we can take some action on it the second piece of our static analysis story and you know static analysis obviously because we're the SC we have all the codes sitting there at rest so we can look at it um so the second piece is uh secret scanning and and this was going to that first problem we talked about of having you know 300 plus tokens being leaked on the platform a day um we teamed up with AWS and sort of just a very informal way to say hey we're sure you're having this problem too how do we

try to solve this let's run uh a regular expression you know tool like let's identify patterns and code uh as they're coming into GitHub and if we find them if we find API keys that match our our Rex pattern for them uh we'll we'll we'll tell you about it right and we'll start um notifying the users that hey their their key may be compromised and you know it's not just in the uh the the push that you're making to GitHub a lot of times this happens in the get commit hist so users are just unaware that the get commit history comes with the commit and so they'll think oh I tested you know this local database connection and

everything worked great and then I deleted the key and then I published it um and now you you have a secret online so um we we worked with AWS to solve that so I'll talk about both of those um either I'll say code ql for code scanning or um secret scanning you know but these are kind of the big bucket features um the last piece that I'll talk about a lot is this product called dependabot again also free um dependabot is a tool that looks at all of the dependencies that are in an application and it compares the versions of those dependencies and transitive dependencies with known vulnerabilities uh for those packages and if we find that you're

using a package that has a known vulnerability in it then we'll go ahead and alert you that hey you know you should upgrade this version or there's a patched version over here or um you know this has a cve that you should pay attention to so this kind of comprises the the big rocks in the story the rest of the stuff on that side was U more configuration settings so we'll talk about both of them but just kind of wanted to lay a baseline down of what what the tools are and what we're what we're talking about here as important as the tools are the way that the tools are built is also really important and so um I mentioned

we talked to developers and we talked to a lot of developers about this security problem uh to kind of understand what we needed to do and so our approach to introducing all of these static analysis tools um was was really this kind of three three pillars here one is um developers want collaboration they need ways of asking questions without having to change screens and going to other places and so using some of the um you know the collaboration features built into GitHub um now if you turn on the security features it's very easy to escalate a poll request that's being blocked by a static analysis tool to a security team to say hey I don't

understand what this alert is do I need to pay attention can you help me with this can you help me remediate it right so give them ways of of working in the community and you know contacting senior devs or or security leads on projects um the second big piece is they want it all in the platform right they don't want to go somewhere else so whether they're you know working in the web UI or the IDE you know as much as possible they want alerts delivered to them locally without having to navigate away to some other other location um and last they don't want to think about it they just kind of want it to be automated it should just

be a thing in the background that developers don't have to care about and so um that was sort of our approach when we built all of these tools and I've had this graph this is one of our oldest you know views or oldest slides I've had when when having these talks with with customers um you know but but it tells the story just so clearly when you um when you automate and inject a security process into the developer's workflow they respond to it and that that's roughly what this graph is saying when you turn on dependabot and it automatically raises a poll request for you to update a version it's really easy for you to go yeah I do want to update

that version I don't want to be exposed to that vulnerab anymore that's simple for people and they get it and they respond to it and you just see that engagement happen as soon as we start putting these tools in front of the developers in this way oops the second thing not the second thing sorry we we see this um with all of our tools uh so I talked about those three big tools and they've all basically experienced the same response rate from developers um when we went back and looked at static analysis scans of code um we saw that alerts were getting remediated you know maybe 15% of the time within the poll request right so if a tool blocked you

or told you like hey you know this lint is off you should fix it U you know 80 plus perc of the time developers were just ignoring it when we started adding PR blocking uh scans in you developers respond to that right they they will remediate them 70 plus per of time you know in seven days which is really wild right um as security people this lowers the amount of alerts that we have to look at that a developer is claiming are false positives or that are you know being pushed into code and bypassing our security processes for us to go back and remediate later um we've also blocked a ton of these secrets and I'll talk about

how we've done this in a little bit but um we we've gotten the the scans for API credentials down um in many cases to less than 1% false positive rates and so we've started blocking commits before they're written into the project so that keys don't get distributed to other copies of the the project when it's downloaded and so we've we've blocked roughly 20,000 of them um which is really exciting so um all of these tools you know are implemented in the same kind of way where we create uh you know automated poll requests wherever we can we we sort of run automated you know in in PR and provide as much context to developers you know natively in the

platform and when we do that we get really good results out of it we started taking these tools that we were building and applying them to the open- source software and I think this is probably the first of what will be many QR codes so if you feel like taking pictures of the slides or you want um relevant blog posts that I'll link you to here feel free to pull those out and zap a picture um or or or follow the QR code but they'll be links to everything this one here goes to um the uh the security lab landing page that you can take a look at but we we you know through these Acquisitions we hired all

of these Security Professionals and so we we put them in an organization called the security lab um and what they do is they use our static analysis tools to go out and look at open source projects and evaluate whether or not we find novel vulnerabilities in them um you know we do this you know 200 plus times a year but what's really exciting is that when you add the tools aspect that we talked about plus a little bit of helping hand from somebody that understands what's going on we can get that remediation rate up to 95% so you know 95% of the PRS that we make on projects to alert them about an open source vulnerability

get fixed which is really really exciting so um this has allowed us to build uh the largest or maybe second largest database of uh open source vulnerabilities in the world uh so all of the vulnerabilities that we find we make freely available um on github.com advisories and this is just like nvd or other vulnerability databases you can pull it down freely um it's also what our dependabot tool uses to check whether or not any dependencies in your project have a vulnerability associated with it but that's been built up and curated through all of these research efforts um put on by the security lab okay I know I've been talking a lot give me a second here I get some water

any questions [Music] yeah we report the CVSs number associated with it um so that that will come out um where we're headed with this longer term is trying to tell you whether or not uh that vulnerability is reachable um we can talk maybe more about that offline but that's sort of where the industry is going is like hey it's not only that I want to know that it's a critical but I want to know if it's a critical to you know and so we're working to solve that problem I can I can give you a little update there okay um so that was a lot about the like GitHub why you should care and and what we're doing to try to better

secure the open source software that all of our daily lives depend on um but there's a lot that you can do as individuals to kind of help us on this journey and so the rest of this presentation is really kind of a Persona based approach to how do I use some of these tools um to to to improve Security on on GitHub or even in your in your own company um and the first thing I've got for you is please please please turn on two Factor authentication this is the single best thing that we can do uh to uh prevent people from taking over our accounts and to stop supply chain attacks who has tofa enabled on their

GitHub account okay security room all hands let me tell you look uh if you glump GitHub in with like social platforms right you think about social platforms broadly and I'm you know this is some Twitter math that I did a while ago but uh what I he tell is that most social platforms have an adoption of around 2% toofa right that's really spooky um and and this is you know we're we're a bit better we maybe five seven 8X that right I don't know where we are now so don't take that as a hard number but um we're we're drastically improving and we um we want to do even better uh so by the end of this year anybody that

contributes to github.com will have to turn on two-factor authentication it's no no longer going to be an option we're just going to force people onto this path um so that's a really exciting Milestone that we're moving to um this has taken three years for us um we've identified cohorts of you know top maintainers and open source groups that we needed to migrate on the TFA I ran a program um with a with a handful of other people and the op ssf which is uh the open source uh open source software open source software security Foundation more s's than I I care to remember um to give out tofa tokens so I went around emailing you know hey Twitter bootstrap

would you like you know would you like MFA credit like tokens um and we mailed them UB Keys uh to to turn on toofa on the platform so um this is something we're taking really really really seriously um so seriously that we created the GitHub mobile app um you know in a large part to be a a onetime based you know password Tool uh so if you don't have uh a top provider uh you can use GitHub mobile as one um I downloaded it for that purpose because they uh made us as a part of working there but um I actually really like the app so I think a small plug Mobile's very cool um you're not going to code on

your phone but uh you can review a PR uh you can plus one things and it makes using comments way easier it's really really good um the coolest thing we've done here is gone password list so now if you want to use the bio key on your laptop or you have a UB key bio or some other two-factor authentication device with fingerprint or um face scanner or something like that you can also use that to protect your account and I think over time what you'll see is is um more just in time uh two- Factor tests uh during you know deployments or changes on GitHub so you'll see us get more assertive with how we're requiring Toof

on the platform over the next couple of months but our first really big milestone here is just requiring on the platform uh in order to contribute to it another thing that you can do as an individual on GitHub is you can enable a feature called push protection and what this does is it opts you into a security feature that will scan any commit that you're making anywhere on github.com and if we find an API credential uh we will fail your commit so it won't hit the project um it'll come to GitHub we'll evaluate it and we'll say nope it looks like your Amazon token's in here uh we're not going to let this go through

and we'll just block you from uh making the commit you'll get a little output in your IDE or in the UI that says hey you know on line 43 it looks like you left your token go back and fix that and then repos it um there is a way to bypass this is a gentle block you can uh say nope I'm definitely sure I'd like to get owned let's do this um and and you can do it uh but this is a really great like just passive check on yourself um like I said and I I'll show you how we did this in in a few minutes um but you know this is not noisy um and I worked on this

program to collect all the Rex patterns for all of these API tokens we've got a less than 1% false positive rate here you're not going to get a bunch of noise out of this tool um we're really cautious about not wanting to block developers from being um productive on the platform while still being secure so this is a really really cool feature that you can turn on for yourself again all this stuff is is free okay uh let's say you're going to go another layer deeper on GitHub and you actually make a project you have a JavaScript library that you distribute or some open source community that you're active in um you can do things as

a maintainer that are even more important than what you can do well not more important but are as important as what you can do as an individual um the big one being enabling the code security and Analysis tools and this is uh in a UI page you go to setting code security analysis I say code security analysis all the time it's my favorite page on GitHub uh this has all the settings for turning on code scanning secret scanning and dependent bot for you um there's a number of other features on there I sort of cropped this image so it it told my story better but um you know it's a nice UI page that you can go down and enable

all the security features for your open source project um this will give you uh the same kind of Enterprise grade tools that we use with our customers uh completely for free so um please go use these they're they're awesome this will opt you into dependabot to code scanning into into secret scanning another big thing that you can do as a maintainer is to turn on uh private vulnerability disclosure reporting um and this is another setting on that code security and Analysis page at the at the repo level as a maintainer um and what this does is it creates a security policy for your open source project for you if someone wants to alert you about a security vulnerability

that might be present in your project they can you know they can responsibly disclose that to you through GitHub um there's also kind of a UI experience to create um a branch and and work on a patch privately with the person disclosing the vulnerability to you so you know instead of having people reach out to you over Twitter um or trying to email you to an email that they're not really certain is going to you or you know going through a contact us form and saying hey I think I found of security vulnerability on this tool um there there's a way for you to alert open source projects or to get alerted um directly in GitHub now so uh really cool

feature but you have to turn it on on your project so um this is another one that that you can go ahead and enable and this is how we contact you know many of the open source projects that are on GitHub today um when we contact you or you get contacted by somebody um if you agree that it's a a vulnerability you can submit it and we're a cve enumerator so we'll actually issue you a cve uh for that vulnerability which is really cool and that's made GitHub um the number one issuer of CVS for open source software in the world um behind merer I think who the government pays to put on the whole

program but um really exciting stuff um it's turned into a a really fantastic feature so um if you work on any open source projects uh you know it's an easy one to flip on okay let's say you work at a service provider you make a product that touches developers in some way or you know maybe at work you just have services that developers use that are internally developed um the big thing that we can do with you here is add you into this secret scanning program this is a free program so anybody that makes any kind of developer tool that spits out an API key uh we we can add into this program and you can um you know you follow the

instructions on the QR code if you want to reach out to us to join this program um I've personally added like I don't know tons of companies into this it's really exciting but you know what we'll do together is we'll work on a regular expression for your API credential and then if we see that API credential on GitHub we'll send you a payload that says hey at this URL we think that you have a user credential Exposed on the internet please contact them and help them remediate it if you get an email from a service that you're using telling you that they think that they found an API credential associated with your account on GitHub this is very likely

the service that's getting used to notify them um we'll send them a payload and say Hey you know here here's where we think the match is go look it up and and contact your user so anybody can join this program um it's really really cool um what does this look like in practice um this is just a fun demo that hopefully I'll get some laughs out of but if not you know no pressure to laugh but hopefully it's funny um I I tried to prepare a demo for this for a different um you know uh a different talk and I leaked uh an AWS key intentionally into my public repository on GitHub um so this is what that whole process looks

like um AWS sent me an email basically saying hey uh we found this key in this URL um has anybody gotten this email before someone's lying I'm just kidding um all right so then I got this email this is really cool um this is telling me that I'm turning up compute in Japan and I'm not and that happen 20 more times in a couple of minutes uh so if you would like a really quick way to burn down 100 bucks uh leak and AWS credential on GitHub and 4 minutes later you'll have spent your $100 um you'll spend the next four hours deleting all of these instances and regions and then working with AWS to

remediate it and when you get through that uh they'll tell you that that you're good to go but um that's what it looks like if you push a key online and it happens tremendously fast um when I did this demo and and when we were introducing this feature um you know the big thing was people grabbing you know compute credentials so that they could mine Bitcoin with them right so they get into your Amazon on environment and then they turn up the biggest instances in the most expensive regions and just uh party party doing math problems um but we added this thing push protection and I talked to you about this as individuals so now if you leak uh one of

these patterns this is what will happen in your UI or in the IDE uh you'll get a notification basically saying hey it looks like you're about to make a major whoopsie please stop and and don't do that uh so this is this is push protection it's it's very cool but this is powered um by regular expressions and regular expressions are are really messy um anybody who's worked with them will tell you a really funny regular expression joke I'm sure um but yeah they're they're they're not fun um so so we have a couple best practices and and this applies to any internal developer facing services that you may you know maintain at work or that your teams are

maintaining or using um there's a couple of good good rules to follow here so one is to create a prefix um and what this lets us do is really confidently understand that this is a token this is where a token begins right and and if we have a consistent prefix we can tell what service this tokens from we can tell all kinds of information about it so a prefix is just very helpful we also use underscore anytime that we are creating space in in the line or in the string um if you use dashes when you try to highlight it it will only highlight the segment in between the dashes if you use underscore it'll highlight the whole thing which is

a very silly UI thing but it matters a lot because it prevents people from pasting around the wrong thing uh so underscores are also another Pro tip here um obviously you want a lot of entropy in the string you can dial that up or down however you feel like but the last big kicker is adding a check sum into the the credential somewhere and what that really lets us do is dial in that false positive rate if there's a check sum in the credential I can validate that it's a real key without having to do a database lookup which makes it super easy for me to just say yep that's a real key that's not a real

key and if I change two or three characters in that AWS Kio into leak it'll just let it go because it knows that it's not a real token so when I said the tool is super duper accurate the check sum is like the magic that's the magic that happens there so if you're building any Services internally or you build a service that developers interact with um embed a checkm in your token and let us know about it and we'll prevent people from pushing it onto.com okay um security researcher this one's going to go a little like extra nerdy so I hope folks are ready for that um um this is a niche slice of the world but

it's super it's super interesting to see what's possible here um if you're looking at getting into doing security research or you want to make money um from bug bounties or you just want to practice um identifying the vulnerabilities that you're here learning about today or you know through through other education um this is a great tool that you can use it's called multi- repo variant analysis and it's uh very poorly named or like very long winded named product but um very cool very descriptive um so I'll leave this up for a second uh just so you can you can grab the QR code if you're interested um but this is a really neat way of leveraging that code scanning

capability um in kind of a unique way to hunt around for for bugs and to understand how you do that um we've got to know a little bit under the hood about how our Cod scanning tool runs um what it does is it looks at the source it checks it out and it builds it uh or compiles it and during that process we generate a database representation of all of the data flows and control flows in the application and then we have a query language code ql AS name implies uh that evaluates whether or not there's insecure coding patterns present in the code all of the queries that we distribute are are open source and again

this tool is all open source uh so we do the evaluation and then we Plum all of the results back into your repository so if you turn on code ql uh this will Happ using GitHub actions in the background for you um and you'll start getting results into a security feed what we also do is we save that database uh because why would we throw that away we just spent all that compute building it so let's keep it um and over time what we've done is we've sort of put most of GitHub through this process and so there's databases present on GitHub for millions and millions and millions of projects and so if we invert the model

rather than asking hey of these open-source queries that we know find known security vulnerabilities what if I write my own query because I want to know know if this pattern is present somewhere else I want to look for a variant of a vulnerability I want to look for something that's similar but but not quite the same um I can write a query and then I can query all of those stored databases on GitHub and get a result these 10 databases have this type of problem and you could do that for performance issues you could do that for any kind of like data about an application um you could write a query and code ql and then you have millions

of databases that you can you can run this against um so as a security researcher if you think you find something new um and you want to understand if it's a common problem in other open source packages that maybe your company depends on or you know you're just interested in submitting these bug bounties you can write a query to find it dial that in a little bit on the project you're working on um and then very quickly you can go and query all of GitHub to see if other projects have the same vulnerability so this is a really cool way to um you know develop kind of your security research skills and also just participate in the

community if you find something you know tell tell people about it right and and definitely check it out if you're interested in becoming a security researcher or um you know auditing things using Code ql uh you're welcome to join us on slack um this is all run by the security lab group that I talked about earlier but they have a a whole community of people that participate um with us in you know responsible disclosure and and writing queries for for new vulnerability types um responding to zero days together this is a very active Community um if you're interested in writing code ql queries this is this is a great spot to get involved in you're also obviously

welcome to participate in our bug Bounty um we do pay so if you find something wrong with GitHub please tell us um and we'll uh we'll adequately reward you hopefully okay uh last little bit here uh so what can I do at work and this isn't a sales pitch but the the headline is like everything I talked about you can buy for your Enterprise if you want but there's a a few specific things that you can do at work that are that are totally free um and that I would encourage you to check out um one is to use dependabot if you're a GitHub user um this is included in your standard GitHub subscription so you can turn on

dependabot it's a no cost security feature that you can you can turn on and start evaluating your dependencies um but also the the tool itself dependabot core the thing that evaluates the um dependencies of an application and then Compares them with our database is open source so you can use it with any uh Version Control System you want you can extend that application you can you can run it internally so um that's on a very permissive license that that you can go grab so um use dependabot or something like it you know but but check your dependencies um and more important than checking your dependencies um talk with your developers about why they're taking dependencies on certain libraries

um sometimes it's easy to go really fast and to not think about um you know the quality of the security posture of a transitive dependency that I'm somehow inheriting when I want to add like cool color scheme to my my website um and so you know I I would develop a talk track that you have for for developers um about what you're looking for in a good dependency and and educate them about what they should be looking for and if you're not sure how to have that conversation or um you know you you don't know what's important I would encourage you to check out um this project called the open ssf security scorecard and what this is is a group of

you know security folks got together and said hey here is metadata that we want to know about a project how many contributors does it have how many companies are those people from how frequently are they contributing do they run static analysis tools do they run an SCA tool do they run any you know this tool the next tool the next tool right and and we want to know whether or not these open source communities are following security best practices before we take a dependency and this tool will just quickly evaluate a given repo for all of those points um so we we work to help them uh get the data from our API and to kind of correctly call it and

search for it um and they store all of that in a big query database hosted by Google so um if you go to this project link you can either run the tool on a Target repo or you can just check the database and and see um score output and also the individual values for any project that a developer wants to take on so you could imagine having a workflow where you know a developer tries to modify a um you know a package manifest inside the repository that Tri some workflow where you run you know the op ssf security scorecard on the Target that they're trying to to bring in and then if it you know passes whatever your

bar is for including a dependency um you can go ahead and let them continue on and get out of their way or you can go have a conversation with them about it um but this is a really cool tool for for being able to talk with your developers about um security vulnerability sorry I see a few phones out so I'll leave this up for a second another thing you can do at work uh is educate suppliers about this Secrets problem um I have spent tons of hours going to contact us forms and trying to you know dear you know company whatever you make a SAS tool that interacts with developers please make an identifiable token and nobody um you

know not nobody but often times no one responds to that um and I have to try really really hard to get people's attention and finally when I get someone's attention sometimes they'll say I'm not hearing that from my customers and so I would really invite you to tell your fers during you know whatever Security review you're running them through that you want them to have a downstream identifiable API token and this isn't just for our service I'll consume that um information if if it's given to me um but there's open source tools that do Secret scanning um as well and and they all run on rejects right so we're all doing the same kind of

approach with with regular expressions and this is still the best practice for figuring out whether or not an API token has been leaked so if you're doing a Security review with a vendor that you're onboarding um tell them this is a requirement please that you just want to be able to know if their token has been leaked in code easy right easy I really appreciate your help I I this is drop uh not dropboxes this is digital oceans token and I took their logo off because I never know the approach I'm coming into this slide so I want to make sure I say it positively but um we worked with digital ocean to upgrade their token type so you can

actually see what this you know before and after for both us and for them look like right and these are both products of um you know just using random numbers to generate entropy right we just didn't think about Downstream token identification as a thing that mattered to us so um when we change these now we can we can identify them anytime they're they're leaked anywhere so um tell your vendors to follow the the lead that digital ocean set and uh hopefully GitHub set and and do this I keep repeating myself on that point because it's like I've worked so long on this I really appreciate your help um okay so uh last slide I think I did okay on time

here we're 10 minutes early nice um so so what did we learn about today um we learned about the open source supply chain and how it comprises most of the software that your teams are writing today um and most of that me much of it comes from GitHub uh so so this is important um we talked about the developer experience on GitHub and how security tools need to take a developer first approach in um you know being presented to developers otherwise they just ignore them right and this is uh this is important if we want results from those tools right if we keep buying tools that generate noise and and don't um produce remediation action um we

don't get any further in the problem and that's been sort of the the underlying you know note through this whole presentation so um you know when you're when you're evaluating security tools or when you're thinking about how you want to make security changes to developers pick and choose how you're going to do that and when you're going to be assertive about it and when you're going to try to fit into their process think about it a little bit um with TFA we're asserting it right we're just tops down saying if you contribute to GitHub you got to have tofa but in other cases you know we're not opting you in to push prote ction you have to go and do that

we're not opting you in you know to code scanning on your code if you don't want that to be there because that's just a source of noise that you're not asking for if you don't go turn it on so um you know meet meet your developers sometimes um you know and the last piece um we talked about was sort of this um community of researchers that were building around the tooling on GitHub and really hope um you know if nothing else take this as an invitation to join that community and work with us on making open source software more secure um so that is my presentation for [Applause] you um okay so this QR code goes to a

landing page and if you want you can dump your email address in there and I'll follow up with you after the presentation if you have a question we don't get to or um whatever you can you can reach me here um but we have 10 minutes I think so if you have questions I would love to hear them I got a question in the back yeah so um the first question was how do you get developers to respond and I I think primarily what you're you're referencing hopefully I'm getting it correctly is um you know when we're presenting a static analysis alert and we're saying hey uh you've created a SQL injection here go fix it how do you get

developers to actually do that uh it it comes down to a bunch of stuff um when you get the alert um it's not presented to you in the way that the standard kind of static analysis tools output you know when I run analysis as like a binary tool I put the binary in and then I get a big list of all the vulnerabilities at the other end of it um and every time I do that I generate a lot of duplicates right I constantly am getting the same list of stuff that I'm ignoring because I'm really focused on a couple of criticals there um so what we did is we just diff that list so the only alert

that a developer is going to get when they're stopped by code ql is when they are introducing a net new vulnerability and so they know it's not only a problem um that's happening right there but it's the thing that they wrote right so it's their responsibility to fix it and they didn't have to sort through a bunch of other people's dirty laundry to to understand that it's only presented to them if their code is introducing the vulnerability so that's one big piece of it the second big piece is um they're still in context with the flow that they're in right they're um they're actively writing that code they're trying to make this poll request when

you think about how security alerts are generally delivered to developers um you know the developer makes a poll request they push it it goes through without any problem somewhere later in the process somebody runs a security check and then if it's a critical it gets escalated to a security manager who then puts it on a spreadsheet who then sends it to the PM who gets scheduled on a Sprint who goes back to some developer somewhere adjacent on that team you know 12 to 50 days later and that takes everybody out of the out of the flow right they don't know the context that that alert's being generated in anymore um when I find out about a problem um let's say it's a SQL

injection and I'm in the middle of writing you know this form uh I can just add you know parameterization right there it's really simple for me to see like oh yeah I just took raw input from the user I need to just stop doing that um that's easy uh but if I get that later then I've got to understand okay what's this block doing where is this present what am I doing so um giving it just in time is probably the the second big uh thing that we did to to get the response rates up the third thing is the quality of the data that backends the alert um the queries as I mentioned are

all open source and so um when a rule is triggered you get to see the textt which is also open source about how to fix that thing so it's a it's a community project to curate and understand when I'm telling a developer you're committing you know a SQL injection vulnerability um I what what are the resources I want to give them I want to give them resources you know in the language in the framework that they're using at that time right and I want to give them links to the cwes and I want to give them links to what happens if you don't do this and all of that gets delivered with the alert uh so that's

another really power F thing that this tool does is is there's good curation of the data that's delivered to a developer when they're trying to triage it so they don't have to go Google around aimlessly and so that's kind of the trifecta that gets those response rates way way up um the second question you asked is whether or not you could do this on private repos and you can um it is a paid feature and I'm not trying to sell you anything so all stuff there yeah um you know if you're a maintainer of the project you can see all the alerts that are associated with the project so if you're um you know working on an open

source project you'll see um the poll requests that are having issues you'll see all of the alerts that we found in the project historically you'll see all the secrets and how those were committed if you're in uh you know if you're in an Enterprise context on private repositories there is a dashboard that provides you many of those metrics and um you know in the case of Secrets what I hear a lot from from customers that use the feature is that they like it because it will tell them where groups are using uh unapproved software you know um someone will swear up and down we don't use you know G CP or whatever the the not approved cloud is right and

and um you know there'll be some group that's constantly committing tokens associated with that so you know that there's some problem over there that you can go stick your finger on um another really cool workflow that I've seen Enterprises adopt um particularly when they have education platforms so if you use something like uh secure code Warrior or some other kind of like education training system for developers you have to send everyone through you know sock 2 or something like that um what they they'll do is uh say okay if I keep seeing alerts maybe it's three alerts in 90 days or something like that from a particular user on the platform uh I'm going to assign them a training

in whatever our LMS tool is and then if that training isn't completed in 30 days or whatever then I'll write a little workflow to reject their I am privileges to get to GitHub so now I can Target the training in the language that the developer using and that they keep contributing this vulnerability in um I can even Target it to the specific vulnerability that they're making right and then I can require that they take it uh if they want to continue working so that's a that's a somewhat aggressive workflow like I wouldn't Advocate doing that without talking to anybody but um you know it's possible and all the apis are there to do that kind of stuff so if

you're having a challenge where you've got a team that's just not you know hearing the gospel um you can definitely you know dial up how how assertive you want to be I haven't I haven't added it uh yet because it's still in data but it's stacked updates for dependabot so dependabot is going to look at the build manifest and then evaluate that against the dependency graph um but what gets tricky is when you know One update causes another update and that causes another update and that causes another update you're not sure which tree of updates to take or every change is going to re-trigger the run right so now there's a feature called stacked updates

that will tell you hey this particular set of upgrades of all of these libraries can be done in Tandem and they've passed whatever test you have as a part of the pr so um they'll be presented with a single poll request to to approve and then can approve that and I can give you the link for that we wrap up here if you want private vulnerability disclosure so uh this is a way of notifying the open source project privately and then starting a branch where we can fix a patch with them that we'll check for it yeah in other places yeah of disclosure data sometimes it's finding where the root cause of the problem is and we just

worked on a cve that was like this where it was very Broad and so we had to go okay it's in this Library owned by this other company right so so that sometimes you just have to Traverse who's responsible for the actual vulnerability and then go talk to them before you can notify everybody else because it gives all of those projects time to update so you know if we make a an update on a single project then a PR comes out for dependabot on all of the subsequent projects that are depending on it they can all update so we just kind of I guess do that manually I'd have to talk to maybe how the security team does that

you know but we we can search GitHub you know what I mean and if you want to uh search GitHub in a really sophisticated way that's not um a security related uh definitely check out cs. github.com it's codes search. github.com another really cool beta feature that does search of all of GitHub so if you want to know like where is this dependency across all of GitHub you can you can get that information out of Cs so um we'll use a combination of search tools to figure out who we got to go notify but this is the the feature product that we use um to to notify people directly often times this data is just in a security. MD file

right and they say email me if you have a problem um so this is kind of what we're we're trying to onboard people onto um I think worst case maybe we could I don't know I don't I don't even know I don't know what we do I'll talk to you later anything else I got two more minutes one more question any more fun all right well okay here we go so it's a great question no it's a fabulous question I didn't make those decisions and if you were to ask me I would be very like I'd make bad decisions you know what I mean um we you know we have really awesome product managers is what it comes down to and

you know this goes back to GitHub being this platform that everybody's on and giving us lots of data that we can analyze you know it's easy for us to do an AB test and understand something we have access to maintainers and open source communities to ask them how they'll interact with different parts of the platform so it's part of just the uh you know advantage of being GitHub is you just have this this miraculous view into how developers do their their work day-to-day and so we can we can understand those types of things but the product managers worked tirelessly at this like there's a whole group of people that have to think about any button or widget that goes on to

github.com and if you see a new feature on GitHub be be sure that you know hundreds of people plus one that thing before it went out the door because it's um it's not a low bar to ship a feature on GitHub and and they work really really hard at you know over focusing on on developers problems um in order to solve them so I I think we have a great PM team and I'm I'm super privileged to be able to work with them you know github's a a really exting place to be because any problem that is associated with source code is kind of in our realm of stuff we can solve and so the the hardest problem of

being you know an employee there is figuring out what to focus on um there's a million decisions that we could make that could probably positively impact someone um but figuring out what the highest Priority One becomes really really difficult for us because um our our scope is just really big um but there's lots more good stuff to come you know this is just the security features we've launched in the last you know 3 4 years um but there's a gajillion other features um we've launched six new product lines so if you want Cloud hosted idees those are available on GitHub if you want security tools you can get those on GitHub if you want AI

code completion assistance you can get those on GitHub um so there's tons of new stuff that's gone in um it's very hard to keep track of all of it but follow the change log and you'll you'll see more of those decisions and uh yeah know know there's a whole you know miniature Army of people working in the background to to make sure that those are are really high quality changes to the platform thank you you so much for uh being an awesome audience if you asked a question today and you'd like a t-shirt I have some coupons for you that I can give you if you come up to the front and they'll let you uh buy a

t-shirt on the GitHub shop uh so come grab one of those and thanks for being a fantastic audience thanks so much for having me here and I hope you guys have a a great [Music] day