BSidesMCR 2019: Do I Need To Change The OSS In My Product? Making Informed Decisions. - Alex Burrage

yeah so this is the last talk on this trip today so they've told me that I can talk for as long as I want so you guys are like a 3-hour talk so no we talked about OSS and managing OSS in products that have already been released so I'm a security incident response engineer for blackberry I've also been a software developer and they systems architect so I kind of seen all of the different parts of managing OSS putting into products deciding what goes in what I do at the moment is where my team is responsible for maintaining the security of blackberry products so right before us it's already out there in the market we want to keep those to go and keep

those up to date what that practically means is that I spend my time looking at OSS libraries and the vulnerabilities analysis like these so these are the ones I've looked at in the last month or so and that's because I ran out of space on the slide but yeah so it means like a huge amount of visibility into OSS libraries which ones which ones are bad what makes them good me bad and so hopefully what I'm gonna do today is help you guys to make products that are more secure so I have to do less work keeping them secure so you can put good code into your products and then you have to do less work keeping them up

today so there's not really any technical details in this talk so hopefully everyone can get something out there if you use RSS if you develop - s hopefully you can get some idea of what your users are looking for it can hopefully reduce your risk reduce the cost of your company like I think everyone can get something out of this cool so where are we today these are some stats I pulled off the internet as always all starts on the internet have a pulse but hopefully this gives you some idea so this was the first one was a black duck scan this is all the things that they scanned there were an average of 250 seven separate RSS components

which to me is huge number to manage and the use of OSS is going up not going down this is well be living now and I want a big disclaimer upfront I think oh it says it's a good thing I'm not here to bash it you know the world wouldn't work without it I think it's great it allows us to make better code hopefully we can use it better than we are doing in the moment but so I don't want to give anything on the impression that Isis is bad unless it does that policy so and just to make sure we're all on the same page what I mean when I talk about open source is the source code is made

possible either of the cable yeah I'm not going to talk about like if it has to be free well what licensing model they have to use like I'm just going to ignore all that so we're gonna say that anything where the source code is made public catch and what that means is that the security risk to all our companies or products that we're making is different that it would be in the olden days using the traditional model or floats or steamboats and what I mean today but if we convince you guys off is that our decision-making hasn't caught we're still lagging behind the use of us off that's why developers are doing things but we're not keeping up with so

open source so what public I said the source code is public actually in pretty much every case there bug tracking systems on topic as well so anyone blocked an issue against a library you can go and look at that it's usually on github some companies were on their own bug tracking things but a lot of them use github you can go and see what issues have been logged and when issues haven't been fixed you can go and look at the old bucks you can see what fixes they did develop you can see what tests were on or get keeping the walls or reduction tests there were all of that information is public on the internet anyone can go

read it so what does that mean well it means that your headache is public if you're using this code all the information out that is out there in the world so anyone trying to attack your product have so much more information than they would have done before probably is a great example of that if you were if you want to see in fact amusing non-secure RIFs you understand up the server that uses an unpatched version or open SSL and see how long they take somebody's at it and you'll be counting in minutes not hours because people are just scanning internet because all the information about properly this out there this isn't open SSL vault I'm not here

to bust open SSL that's just how we work so it's all bugs as I say we have a huge amount more information about the OSS library we can look at the history we can look at their previous fixes we have access to their development life cycles we can see what developers are working on and see what their future versions they have in development you know we can read their code we can decide which functions we want to use so we have all of this extra information as I say all of this is just out there publicly and what this means is that we can make informed decisions and also we are doing I'm saying we can do and we should be

making more important decisions with the information we have so the kind of the point that my talk today is that is to answer is to help us make a decision and to get to the point what we're just talking about the same thing I'm going to make a bunch of assumptions so I'm going to assume we know what OSS is in our in our products that maybe not completely realistic if anyone does know every single bit of software that they're using only surprise but for the sake of argument we'll just or say we know what we're using and what version we're using which is again not realistic and it's non-trivial to find out out of

the same value we know what you're using we know what version it is so like we're still supported don't use on supporting loggers and that we have today for if we're not today we have a good reason for not being up today we know that the version we're using has vulnerabilities that don't affect us or you know we're planning to do an update to whatever we are happy with the version that we use so once you've done all of those things once you're using software which is still supported and it's up to date or as of today it's practical then what else can we do well how about putting the only thing you can do to reduce

to your company or your project is to replace your access so take the excess likely that you're using find a different one which gives you less risk and then replace it if anyone knows anything else you can do other than keeping your libraries up to date and replacing them with relaxed risky ones he's come and tell me because as far as I know this is the only thing you can do hmm so if we're going to talk about the placing access you might be seeing that thinking well that isn't realistic nobody's actually going to do that but that's not true or at least all the situations that I've seen that's not true people make this decision all the

time they might not you know have a meeting sit down what the documents say we're not replacing our library for this reason but that doesn't mean they're not making this decision and usually they're going oh it's too much or are the the security benefit I'm going to get from that it's not enough ooh how do they know that how to project managers know that how are they making this decision well they're going on gut instincts they're saying you know I we did this before and it wasn't worth it so I'm not gonna doing it but as I've said with no access we have a huge amount of data we should be using methods that we should

be making informed decisions so before I go all need to kind of the bulk of the talk there are a bunch of tools out there that help none of them give you the whole picture and so kind of what I'm talking about today is pulling whole bunch of information together but these will help if you want to talk to me about any of these any of these tools in particular like come grab me afterwards I'm happy to talk about them but for the sake of completeness here are some things that help but none of them give you the whole picture so what are the factors that we want to take into account so say we want to get gather

data what data do we want to gather so if you're looking at a library a good way or assessing insecurity risk is what honor abilities has it had will see visas out so we'll see these as it had in the past how many does it have today which it hasn't passed yet how severe were they what issues where they all this information is out there it is public you can there are tools that will grab this all together for you so this one plots the number of other parenthesis per year and it's a twenty BFF school so you know you can see that all the last few years the number of quality of these has gone down

maybe that you know gives you some confidence about the quality of the library on average the cbss school was glad to know about a poly again maybe that's okay so you can use this information to say well this libraries you know is not as risky as some other ones so maybe that's not too bad you can also look at how quickly the issues are fixed so as I said we have at least read the bunch wrapped industry so we can see when an issue was raised when a security researcher when a pen sets became a long moment but here is my POC for this vulnerability and then you can see how long it was before the fix was in

production and you know if a library consistently takes six months to do that any zero days that let me guess that you get because you were using that library they're going to be around for six months maybe that's not very few duration so we can take all this information into account and we can get an idea of the help of the library so we have one the manatees the thing that comes along with ability is a patch so we can see the commit history for a library obviously they're not all going to be security commits but we can pick out the ones that are relevant for the information that we need and this gives us a lot of information about the health

of the project so you know all the security fixes that again checked in are they the kind of simple things that we would have expected to have been done already like are they just putting in null pointer checks before dereference it pointers or are they you know just checking for the integer overflows they doing that kind of basic level that we would assume a mature library would have found all those bugs like they're all on the other sides you know if the patches that are going in and capturing weird obscure corner cases or race conditions or you know those hard to find then we go okay this is the kind of thing that's getting back it probably

called the low-hanging fruit this is a mature stable library that we can trust because we have the patches we can see all the source tree we can see which directories which features a getting patched and those might be the features that we're not using so like before the security stuff all the patches that are going in and you know the feature that we don't use the whole pirate tree that we don't use for that we can say that doesn't affect us that's good we can see what regression testing is done while gatekeeping there is the last point I put on this slide is time is about backporting I think this is really important if but library develops

patches and they're not able to back port it to their older versions that gives me a lot that makes me very nervous about that library and they you know is their coach or coach and so fast that they can't back for things because the code is so different that's generally not a very mature library there are some good reasons sometimes but generally if the library's changing that past that makes me as a user nervous like the state of our library in general if things are being backported you know they have the developer resources they're putting the tiny audacity fixing things that makes me a lot happier so I visited a couple of slides about code reviews art Kohn we've

done like when people make check-ins is there a gatekeeping system do they have to be plus formed what regression tests are there you know do you have to check in a unit test when you check these some new code does that do you have to have a certain level of code coverage before it's accepted you know all these kind of gatekeeping code use give us an idea of the quality of the librarian of the majority of that library you know how many developers have to look at a piece of code before it goes into production and is it your cup because you're using this like if that's just one person that's you know gonna make me feel not

very happy if there's a proper regression test you know if it has to be peer reviewed all that kind of stuff it's going to give us a lot more confidence so all of these things are going into contributing to our impression of the health of the library so as I said before we have we have all the check-in history so we can see which users or which developers are checking in code we get an idea of the size of the team so you know is this team is this OSS library that I just found and get help is it just one guy you know SanDisk average making a product or is there something that has a proper team

of developers working on it and it's gonna be you know well supported going forwards more eyes on home generally means less issues but you know we're not just gonna count the number of developers we want to look at which developers are making significant regular contributions just because you have ten different people who have made the check in doesn't mean little what you want the same amount feature is a big one there so sometimes with some smaller effects you might have a large development team but in reality each developer is just working on in their own sandbox who's working on their own feature and so in reality nobody else is looking at that code is there the person

that does that bit and if there's a vulnerability pair the person's get a fix it so you have a potential bottle may bear on you getting fixed because the essentially warned about it but you can develop that things for you so you know just because they have a lot of steam doesn't mean actually they're all working across across the codebase and they're all up in each other's code documentation everyone hex documentation they're working for a team documentation but what a lot of really documentation right and what I'm going to say is code OSS libraries that are well documented generally haven't been well plans have enough developer resources on very mature maybe you're gonna make this feel

worried about the security risks they're going to introduce so you know I found this was an article I read some of you might be familiar with it but basically you know it tells you how to integrate it on each of these ten different environments and there are probably detailed step-by-step instructions if somebody's gone to the effort of doing that you know they're properly looking after their they have popular developer time to do that kind of thing it's not exciting it's not you know it's not sexy so why they're going to develop but if they're doing that I'm feeling a lot more happy about the code that they're writing for me I'm not saying that's the only way to

do with documentation I'm not here to talk about what councilors good documentation like it's a whole separate topic but in general I think we've all seen good and bad examples for brick so you know the amount of effort those points of the documentation gives me a sense of the amount of effort people are putting into making their livelihood and that's going to make me happier about the security risk that I'm taking on release history so we can see how frequent releases have been in the past and we care about security you know we can look at do they do security point fixes or do they just wait for the next major release in six months or a year

and then hold in a whole bunch of sugar it's and fixes when they get there and therefore even though they might have converted the pigs I am in production until the incremental version comes out that's a problem for me as a user because I'm saying they're using known vulnerable code until they do their next major release future roadmap is also another good one if I know that this library has like two or three future versions planned ahead the development team is doing some planning they have plans to support the library for at least two more versions why but that might be six months a year the library is less likely to go out of support so

that's going to make me feel happier so as I say all this information is out there I can use this to build up on my picture of how good the libraries so this using my code this is probably the hardest thing to quantify and what we're trying to do here is answered two different questions how are we to be affected by one abilities and how hard is it going to be to update our code so I'll talk about the first one so how likely are we to be affected by fun abilities so we've said before we've come to the vulnerabilities in have in the past that might be an indication of where it's gonna happen abilities in the

future so if we're not using the features or the API calls that are the ones that had one abilities well then I don't care if I'm using a library and it has a one ability in a feature that I don't use that doesn't affect me I'm not bothered however if I have a big library you know I only use one part of it but it's the part that's always getting pinged by by the security people that's gonna make me worried now exactly how you go about doing that is not something on your answer today you can just count features you could look at how many distinct API calls you made you can talk about your code coverage statistics this

is a really good code coverage tool but like in general getting some idea of how much of OS is likely use and in particular which parts of it you use is a really good idea so when we talk about the second thing so well I said at the beginning was what's the question we're trying to answer should we replace our OSS now we've talked about risk the other thing we want to talk about is how much effort it's gonna be to do that replacement work and so that's the other part of this equation you know if if we only make one or two API calls replacing that is likely not going to - that the

every file that our code base calls this library because I don't know we're a crypto librarians colleagues OpenSSL the whole time that's gonna be a massive headache so again exactly how you convert code use into man-hours and like their effort is a topic that I'm not gonna go into but my point here is that all this information you shouldn't happen like you know it's not public because how you use this code is not public but you can you can put out up your own code bases so you can get an idea how hard it's going to be to make this up so those are my seven impactors that I think go into judging the security impact of analysis library

these are the things that we have users are looking for if you know of any others please come tell me afterwards I'm trying to make this a comprehensive list I've missed any I would love to know I think these are the things that go into judging the security threat that our libraries providing to us so I said from the beginning what we're trying to do in the area to make a decision and if that decision is going to be to a place for access well then we need to look at the alternatives but hopefully what we've done here is we have a methodology we have a set of factors to look into so we can run the same analysis on those

envelopes so you know I have a few libraries in transit so I just want to use I run the same analysis of the same factors and then I can make a proper multipliable in - but I can defend the last point as I've said is probably the hardest one is how much effort it's going to be but there are methodologies out there we can we can we can come up with an estimate and so then what you've got is a basic business decision this if I put in this much effort I'm going to get this much this much risk reduction everybody's appetite is going to be different some people will tolerate no risk at all you

know some people are so overworked that they have no infant affair and therefore they'll tolerate huge amount of risk unless it's super trivial - to make the change so that's the decision that has to be made on a company by company basis possibly on a project-by-project basis I don't know who but my point here is that if you have data if you do multiply both of those things then you can make a defensible decision you can make it good data-driven decision so we did this example and we did this exercise we haven't come in super in detail I'm not going to say which two levers it was because I haven't spoken as those delivery so they

can't defend themselves but we were using my px with the alternative which provided the same functionality it was letting boy and so we looked into the details and you can see that the average number of vulnerabilities doesn't actually go down that much like if we were any matching what none ability per year but that's not a big deal but the key point was development team size so the Allah be on the left was basically just one guy no library baby on the right it was a property or property development team and what that meant was when we looked at how long it took patches to be introduced the time the time it took for patches for

vulnerabilities to get past the library hex was like six months so you know researchers will come along for their POC on this company's tracking system and then it would sit there and then eventually things were going to production on average about six months later locally y-you know things were patched within weeks and then they would release a security point base and then the users will be up to date so this is the kind of information where if you're just going on a gut instinct if you're going well it might be too hard I don't know how much risk and how much risk reduction I'm going to get making that petition is going to be hot once

you have all the data it makes it a lot included to make that decision so where do we go from here so I've talked about these seven measures you can rate each one of these on a scale I didn't do that from the previous slide but you know you could read each one out of ten how much how much threat you think it's introducing then you can weight those measures and you can calculate a risk for placement school so I'm thinking of here is something that looks a lot like CDSs you know you can look at how many CDs though how many have how good the patches were how many developers they will plug those all into some

algorithm it will spit you out a number between 0 and 10 that's how much risk there is you can look at how much code you're using that gives you your risk replacement school and as I've said a couple times now all this information is out there publicly on the internet there is no reason you couldn't develop a tool to scrape the internet pull all this information together and do all this for you automatically when it comes to looking at your usage of the courage you would then need to plug that all into your like code testing your regression testing software obviously that wouldn't be public but inside your organization you could do that and and so yeah so

doing this is then requires no manners at all you can have this just running in the background you can set your level you know if my risk reduction goes above this much for this amount of developer effort take me send me an email and they'll make that change and then you can automate all of this and the hard work goes away and you get security benefit for free we're a long way from that if anyone wants to develop this please do so you'll make my life so much easier I'm not gonna steal the IP from you or anything like that but this is where I think we as an industry need to go to make everyone's life easier and that's

the end if anyone has any questions you can ask now and I'm outside ping me an email part the reason I'm giving this talk is to get feedback so so selfish reason I want people to you know provide me with the things I've missed the the factors that you take into account you're looking to assess so yeah get in touch with your angles and things like this pressures down so you measured what with github that's where I get an saying like this code develop home so these files are all this current about but I totally ship it did it at home this yeah and I've had that been talked about for do you know how many tools that help you visualize

that let's say like this this these files on this log directory this code this to be very personal you know I don't have any tools nope I don't it feels like it wouldn't be very hard to develop one you know you could just look at the gear check-ins and you know you can draw you can draw like a directory Qi just want to do that I mean is taller than baseball how many developers is contributing to reach directory but oh yeah I don't know but he's not to do that today that's just and yeah some of the metrics I do see is quite typical because all scoot off the meditation for me as a developer would

be code of adèle straighted chalk where is to complete newbie they expect to be sort of step through the guidelines and those kind of things this is quite a subjective measure that and I also think people try and gain the system I love github Austria whole bunch of accounts I don't have how much people the project may serve the heavy protect from game that's a very good point I would say if a developers going to that much effort to game the system it's probably easier for them to just make good code like nobody makes byte code and then choice them place videos give us true Google thing is something far now because if you're in competition

with one of those others use some support packages from the OSS a competition group some songs that you give away the free version right and then this is all ramped up around outside make money yep so there will be an incentive to appear high obviously something like that metric will be very good in an answer pack scope my nexus result yep it's a frog and it would score all the things that never follow you get embed them yeah yeah I mean that's kind of what I haven't minded yeah there are ways of getting the system it didn't take you long to get all the bottom and then Amazon's if the score isn't it I would say I

think it's hard to do that for all the metrics I think ultimately it's going to be hard to completely fake the library's reading is and and so that's why I had like you know I mentioned weighting the different letters so you might as you said like for your use you might not care how good the documentation is you can just adjust that wasting down to zero the number of vulnerabilities that like this having and passing out how to get them how critical they were that's something conveyed but it's like something poking it right now so that is gonna be a lot harder to manipulate the metric on that one so yes you're right it's not it's not perfect but I think

there is some benefit at least I'm getting users pretends this last is a she's paying between keep replacing something contribution yes open SSL anything that is expired the community large was that if uses SSL depolymerization waiting yeah who's always doing so however blackberry now other companies I worked for in the past tend to be reluctant to contribute code that they've both as a company job if it's likely because there's an IP issue there that's not always the case and in certain I felt you know big companies have but those were things that I mean but there's massive source license use my own source code you've got mister changing right so that that's one point is that companies are reluctant to make

any changes to talk to poetic about the parrisha listened to doing that because they know they have to contribute back so they will either use the library has published although we use a different largely as part now yeah people should contribute OSS everything like completely agree in real-world situations where I fit him in my personal experience the decision they're making are we not going to spend our developer effort on improving just likely I want and quit pegs column I want to reduce my threat phone today the quicker way of doing that sometimes it's just a place of love it yeah this decision make it won't be for everyone and some people will want to be back to libraries but having

misinformation out there I don't think it hurt anymore if they could see this information yeah do you share your assessments back to the projects and so we don't cut me this is still quite like an immature project it's not really up and running in any serious way we should do like the vision I have for this that sounds a bit grandiose but like where I see this going this doesn't all this information should be made public as well so yeah country maps back to the other source libraries then as I said part the reason I'm giving this is so that both if developers can see the kinds of things that we looking for you know they might

think but nobody actually cares about them if a story about the right name but what I'm trying to say here is that if I'm trying jobs for security risk that's what other perhaps as I've taken - ok so yeah like the whole point of this is to give back to the community and to help people to make more mature or shorter again Thomas a cubist and obviously the information that you need out is hi penny Lightman CDA's do you have any tools as part of our test which looks at inside of a balloon to do with that and you do have you know like I said code review set to a developer how do you know that the hum injected a

malicious cordage good then I was taken aback I'm doing nothing until days such a day arises but you know that pop the wide open I see coming off me so suspected but yeah I don't know how many tools that do that I'm just rolling something compacted could get up it's all to do that if you can catch people checking in militias homes you know that mm that would be a massive win if you get something to do that I don't know of any automated ways of checking for that so if that's the thing you're worried about then one of the things I mentioned these period protease so game you know two sets of humanized and look a piece

of code as I wished before you could wipe the different metrics right so if that's the thing that you're worried about then you wait that tightly and you say you know I'm getting massively more trust Pro where two sets of the two individual developers have looked at kotodama coded you because that reduces that risk I know everywhere you completely eradicate that will catch every instance but that's the major piece

[Applause]