DevOps Methods for Patching an Open Source Operating System

we are ready to kick this thing off um our first talk today and we also want to thank all this the people watching on the live stream so hello live stream our first talk today is devops method for patching open source operating systems so let's take it away all right thanks um can you hear me okay through the through this mic does that work hello how about now better okay all right so uh so good morning i can't okay so we've said thank you like a million times but honestly can't thank you enough for actually having a venue like this is so cool to be in a room with other people a lot of other people so

thanks a lot yeah and i know also um i did want to mention the streaming folks right so this is being streamed we really appreciate the folks that are working on that um so i'm going to go ahead and get started with my talk i want to talk a little bit about um open source operating system patches this is something that i kind of fell into uh and i think it's if you're like me like open source is open source but how this whole stuff works like it was really kind of i had to tease it all out so that's why i wanted to present this to you let me give you a little bit of

background i've been in in security for for a number of years um and uh you know i went through a number of different positions uh throughout different parts of the field ended up as cso one of the fun companies downtown uh and an opportunity presented itself for me to join uh the security team at canonical now canonical is a company that produces ubuntu they had an opening to add security into the robotic operating operating system roz so raz is a complete open source ecosystem that you use for robotics right and they wanted to improve the security in there and so on and so on so it's a really neat opportunity to work both uh creating and building the ros

security ecosystem while working with the very mature canonical open source patching ecosystem so that's kind of what i'm going to share with you is and that's that's the ben the twist that you'll see behind it um we were putting together extended security maintenance for raw's an open source product so how you actually maintain the product after the open source community has moved on to the newer releases and we'll talk more about kind of how that whole um ecosystem works right um so you'll notice um like we prepared this i was working with with uh folks on this presentation because i thought it was really cool uh and then covet happened so i never got to give the presentation and then i

changed companies and things you know whatever i still want to get this out there so you notice none of my slides are really branded at all they're very bland and so on just because i didn't know what to do with them right so there you go um so now i'm with plus consulting so hopefully i'll see you all around town um and yeah so let's let's move on what i plan to cover today we'll talk a little bit about how vulnerabilities get triaged uh in the open source ubuntu world uh we'll talk about open source and how that happens what upstream means and what downstream means and how code actually flows throughout this ecosystem to end

up in my laptop right versions and distributions become really important when we're talking about this and also this concept of back porting a patch and back porting software so if you haven't heard that that'll be a new thing we'll cover right so please stop me as we go along this is the first time i've given this presentation end to end so it i i may not cover everything in enough detail so if you get confused just let me know all right so we're going to start with where the security team starts out at in canonical which is to patch ubuntu right ubuntu is just a collection of a lot of open source software so where do we begin

uh and where we begin is at the cves right so we mine the cdes to take a look at what's changed there's probably about 150 usually less than 200 changes per week right so they go in and they get you get eyeballs on them to see if they apply um the interesting thing if you didn't realize it is that cves are all on github i i thought that was pretty cool right so if you want to do this you can go right ahead you can go in and get your cves from 2021 you can watch any changes that anybody's updated all the details are there and they're in json format so you can parse them out you can have your own

copy on your own laptop all right so we'll take a look at those we'll pull down any of the changes and evaluate them to see if there's anything first of all that applies to the ubuntu open source ecosystem all right uh and then if it does apply then let's take a look at it and see how important it is right so you do an actual risk assessment has anybody here ever done decision making based off of cvss scores for a cve have you been successful yeah yeah it's kind of like it's a place to start but you know cvss scores have their challenges because everybody's different and so on so we do the same thing with

with ubuntu right so you'll get a high rank cvss score on the cve you got to assess it and you realize oh well when ubuntu creates the the code and they compile it they use a compiler flag that compiles that vulnerability right out so we're going to re-rank that as as low not high because you're not vulnerable if you're using you know distributed software from ubuntu right so all that kind of all fits together and that's part of the daily routine on the security team full in cves look and see what's changed use that to decide what the workload is going to be what we're going to work on today what's high risk what needs to go out to our

customers all those results end up at this really cool website that's the ubuntu cve tracker right and this is our cve reports so um we're going to if you notice here there's some some of the most current work is listed here then some of the recent changes that are updates that have been things that have been triaged um i want to spend a little bit of time given that you know i got into this in robotics we're going to look at a package called opencv right and so it's just a package we'll talk about what it is in just a second uh but if you notice there's 33 cves for opencv okay so 33 results 33 cves and it goes from

2019 down and there's the priority ranking so that's the manual triage it comes back and says this is how important this cve is to ubuntu users and it's got all these columns about all kinds of different statuses right things that are needed things that are not vulnerable things have been deferred and all that like that gets confusing rather quickly right what do i need to do uh and that's what i want to dig into is where this chart comes from so that you can make this this all manageable for you so this looks really confusing but to understand what it is you really have to understand a little bit about how ubuntu comes into being and that's what

i want to talk about to understand um that we're going to look into um oops sorry we're going to look into what is going on with how ubuntu is created right so again we're looking at the opencv package and i'm going to take one of these bowls and walk it all the way through from the open source patch that's in the public github repo and bring it all the way back into an ubuntu distribution all right so we're looking at opencv um have you ever seen pictures like this right they're really cool you know the little like the the rectangles that follow around eyeballs and you know recognize things in your pictures that's what opencv does

open computer vision library so this is a library of software it's not really an application on its own it's an open source library that gets included into other things so it is included in a number of different products if you're going to put out a product that has to do with some sort of image recognition you're going to pull down a copy of this open source library and embed it in your product right so obviously in robotics we had a copy of it embedded and that's why i care about opencv all right um so here's all the all the cdes that are related to this side note the reason why there's so many cves related to the opencv project is

because they are very active and very good participants in securing their software and tracking them by cves it's not a recognition of the quality of software sometimes you think a lot of cves means that it's bad software not the case these guys are really good at tracking cves against their software all right so again we're going to dig into this one cv in particular um cv 2019 15 9 39 let's take a look at that

where's my handle

there we go

so this is a cve entry from the mvd right pretty straightforward there's an issue in a divide by zero all right um and this is the cve entry that's originally in you know github that you can pull down through json and whatnot all right the nice thing is here this one actually has references if you can see them down here all right it references the issue on github that first identified this vulnerability uh and then it even has a code fix so let's take a look at that

all right so this is the issue and again this is standard github issue and this is the way the opencv ecosystem works you have a bug you file an issue the issue gets triaged and then it gets worked on and fixed right this one if i'm not mistaken i believe this was the result of a pen test vendor it might have even been x-force or something something along those lines where they were doing a pen test they recognized this divide by zero error and they filed this report they filed this issue and then they walked away right they're not the code maintainers they just identified a vulnerability nice thing is in this particular one they gave us a lot of nice information

about it okay so here's a system that's that's vulnerable you know the version that's vulnerable here's some debug information and then the one other thing that you'll see is invaluable is that they also included the vulnerability the exploit all right now this isn't an exploit that gives you root shell or anything like that it's just an exploit to say here's the problem here's a file that you can use to run against this piece of code to reproduce my problem right and as you'll see throughout the whole process that becomes really handy right all right so any questions about that okay so now we've got a vulnerability that was identified by someone uh and another thing that goes along with that

is the actual fix that goes that was used to close this issue

so the fix itself is also in github github remembers everything forever right so um you'll see this is uh the fix it's been merged uh into production this is actually a little bit old it's from 2019. so it's merged into production uh and also if you see it was merged into uh this tag right version 3.4 right and in august and the other thing is so we actually get to see the code that was

changed looks like the internet's not running too fast all right so here we see the code that was changed all right so it's in this file right hog.cpp all right and it's pretty straightforward right okay let's check and assert to make sure that this variable is not zero check to see if this is empty check to see if this is empty all right really straightforward stuff just checking for zeros all right so easy fix everything's good right so where are we now we've got a vulnerability that was identified through a cve we've got a an issue that was logged against the code and it's been patched so therefore we're done right so unfortunately no this is only just

the beginning all right to answer the question on whether or not you're patched you've got to understand a little bit more about how ubuntu gets created all right so let's talk about that for a second all right an ubuntu release is a collection of software that works very well together right so i take all these different packages of all different versions that are all being you know churned and created all the time but i take specific releases of them and put them all together i test them very heavily and then make sure that that everything's nice and stable and that becomes a release all right ubuntu issues a release every six months uh they issue a stable release once

every two years so the interim releases our interim for testing to get to that eventually stable release once every two years right this release cycle right comes from their upstream so uh the initial load of software comes from their upstream which is debian all right so ubuntu is derived from debian and it gets added into the ubuntu ci platform which is launchpad so launchpad does all the ci work for you it builds the source code runs the automated tests uh bundles the binaries up into packages and gets everything ready for the distribution you test the packages out make sure that they again they all work together and then that becomes a release all right once we have a release and we push out

uh uh particularly a long term lts stable release we don't want to change it right so we don't want to bring in any new code unless we absolutely have to so version updates feature creep all that kind of stuff we don't add that to the old software because we want to keep it stable so if you're running ubuntu 1804 you're going to stay on 1804 you're not going to get the updates to open cv if they come out you're going to stay with the version that was released all right so in order to patch ubuntu we've got a patch the version that was released with that version of ubuntu right that's a little bit confusing so let's take a closer

look

so are you familiar with upstream versus downstream have you dealt with that in the open source world right so upstream produces the source code downstream consumes the code from upstream right we're all talking about source so this is all open source right opencv as we saw on github maintains the code base all right so if you're going to program an opencv you're going to program through github you're going to commit to that github repository how does that make its way to ubuntu ubuntu it's the downstream and it consumes the software from opencv ubuntu has other downstreams there are a number of versions of ubuntu that that are sub versions if you will that are derived from ubuntu proper

and ubuntu itself is derived from an upstream as i mentioned ubuntu is derived from debian all right so opencv actually pushes its software to debian and then debian uh the debian it's actually the unstable branch gets copied into ubuntu as we start a release so ubuntu is a derivative of debian right this i don't know if you've ever seen this tree this is this is wikipedia so it's got to be right right there's a really cool timeline here that talks about it's a timeline plus a list of the derivatives of different variants of ubuntu and it's amazing if you look at the whole tree of who comes from what right this is all the open source upstream downstream tree

for linux all right if you notice we're talking about ubuntu and we talk a lot about debian and debian packages if you look at the genealogy of red hat it is not the same right red hat doesn't have an upstream it is its own upstream it is you know the root source there so it has a different packaging system and a different way of taking in the updates so we're going to talk about how ubuntu happens a lot of you probably use cali linux do you know who cali's upstream is debian right yeah so we'll find cali on this on this chart um and it's upstream is debian and so cali will get updates from debian

all right so now opencv is a very active project right so they're constantly updating code and and taking in updates you know there's a lot that's going on in the computer vision world um a lot of it is very academic so if you're doing a thesis and you find a new way to recognize a new thing in images you're going to probably commit that code into the opencv open source ecosystem and from time to time open opencv has got a stamp code at a certain version uh and tag it and call it a version all right um and that's standard software engineering practices right version three two three four four five yeah so on right so it's constantly

churning different versions of opencv software every once in a while debian pulls down the updated code from opencv so they'll pull down that just simply pull from the get repo right through github and bring that into the debian source tree right this actually goes into debian unstable which is also known as sid so i feel unstable from time to time whatever but um so this gets pulled in from time to time by the debian project all right and gets pulled into unstable and unstable gets promoted into testing and then becomes a debian release all right ubuntu follows a very standard cadence as i mentioned every six months they start a new release cycle and the release cycle

begins by pulling that whole bundle of software that exists in debian right in debian unstable so use that to populate then we add in some more stuff that makes ubuntu ubuntu you know some different desktop different features and and whatnot and then compile that all together and then start the testing for an ubuntu release that's all that software working together right what that means is that the version of opencv that gets pulled into ubuntu mirrors what was in debian at the time and it changes it's not every release of opencv uh and if you want the latest features of opencv you're not going to get them every time that you install ubuntu 1804 you're going to get version

3.2.0 of opencv if you want something more more recent then you have to do something different and you have to install it in a different way but also understand if you're gonna you know upgrade your ubuntu 1804 to version four of opencv then how are all your dependencies going to work how well is that going to work together with the whole ecosystem that's when you find yourself in dependency hell if you've ever been there right so this all works together as long as you can stay with the features and the features are what you're looking for quite frankly if you want to use the latest version then update your operating system yeah go to the latest

stable release and things usually work pretty well and usually opencv keeps an eye on what's going on with the debian and the ubuntu universe to keep things working fairly well that way right so that's how we get a specific version of software into an ubuntu distribution that's how it gets picked any questions on that does that make sense or am i getting blank stares we're good okay all right so another interesting facet of this is that you've heard about building from source you know you can download the tarball and you know make it and and make install it right um or or do whatever from source right if you install from source you are going

to get the latest version right um and so if you want to keep track and you're doing in active development that's probably what you want to do right but if you want the stable release that you get through using the apt toolkits and the standard updates and and everything that's built into ubuntu proper you're going to get this version if you're on running version 2004 you're going to get version 4.2.0 of opencv okay this has an interesting takeaway all right so we'll take a side note here uh based on what we've talked about so far um is that we're talking about in this talk about how you update ubuntu packages all right how you get those automatic

updates when you have automatic updates enabled and software's coming down you're automatically installing the updates everything's good everything's working together but that may not work because you need the latest releases of the software right so if you're not using the ubuntu you know standard distributions and updates if you're building from source then you're not going to get the updates that we're talking about we're not going to get the patches that i'm talking about right if you want an update from source then you have to download the updated source and rebuild it right so if you're installing from source recognize that if you want to update you're installing from source or you're building from source you've got to do it yourself all

right so if there's a security patch or anything like that you've got to do that hopefully you already knew that you already knew that if you're going to build from source that you know you've got to maintain it yourself and that's a manual type of thing however this gets really challenging in an open source ecosystem if your application providers if the people that bundle in this software don't pay close attention to that right so we mentioned opencv is a bundle of libraries that are all pulled together um into into you know a just a standard set of utility libraries um maybe there's an application that pulls down that code and then bundles that code and

releases it right so that application then um is gonna have to get updated from time to time and usually they do right so there's an update from opencv the application pulls in the update and then bundles it up and ships it out the door and you've got an updated application that's good sometimes your your vendors will pull down the open source and they'll vendor the open source create their own custom features so they'll modify the source of opencv right because they own the source it'll add some features in add some checks right into there they won't commit them back to the upstream to the original open source opencv tree right and if they don't get committed

that means they can't take in the updates because it'll wipe out any of the customizations they've changed so a lot of this gets really icky really quickly if you're not paying close attention to what's going on right so if you're in the open source world hopefully you got that if you're not in the open source world hopefully you don't have to deal with it okay so all right so let's go back to this uh cve tracker website right so let's see where we're at right now so we figured out um that these rows represent individual vulnerabilities we knew that right uh then these columns represent the distributions but also implied in that is which version of

opencv we're talking about okay so we saw we looked at 16 18 and 2004 these all have different versions of opencv all right if you do an apt upgrade right you're going to upgrade that version of opencv and that's what all this means is that this one's not vulnerable and this needs an update and this one's not vulnerable all right triage the triage process that happened by the security team said hey we recognize we actually do need to update the source code here right so that's how to kind of understand what the cve tracker is and how what it means to you all right so just to summarize again where where we've come from ubuntu is an entire ecosystem

of software and a specific snapshot in time the software is continually evolving but an ubuntu release takes it out of snapshot in time make sure it all works together and you get a stable release and you want a stable release you don't want any updates that are going to break your stability your system right right so to maximize that stability they're only going to accept tiny updates small updates there's a stable release update process that's documented the only time we're going to update is if we got to fix the security flaw or we have a significant uh you know bug that impacts uh uh you know software doesn't doesn't make it work all right so

normally we're not going to see feature updates should see that and also we want to update the software that was distributed at the time that release was made right we're not going to update the software as it currently exists we've got to go back in time to do that that means that the software version that you're running um rarely changes as long as you stick with the operating system you're running ubuntu 1804 it's going to be supported for five years um you stick with 1804 and you're good it's not going to be updated you're not going to get new features you're going to be stable all right so if we want to fix this vulnerability now

we have to patch multiple versions of opencv and that's where we're at we got the source code source code worked for version 3.4 and that's not what we're running all right so again when we looked at the commit and github to the to the open source tree um it was to open cv got published in opencv version 3.4 all right um this is the date again we're working on 2019 and let me ask you an interesting side question right anybody here know what a hog detector is yeah so like if you're working this do you have to figure out what that is right you have to understand the software how much do you have to figure

out the software in order to figure out how to bring this patch in because it's not now you know i'm working for canonical working for ubuntu not working for opencv so i don't have all the context of what all this software does so how much do i have to actually figure it out uh in order to do that it's actually pretty cool a hog detector is part of that smart logic that somebody did with the phd and a couple other phds refined on how they figure out gradients and images and all this so really cool stuff and you can see it's actually being used a lot all right so that's why we want to fix this

vulnerability or want to take a look at it right

all right so we took a look at the file the source code that we got and we saw that for ubuntu 1804 i am running version 3.2 all right this patch went into version 3.4 so this patch was does not exist in version 3.2 all right so when i compare it i can see uh we'll take a look in a second at the source code i can see that the version that i have in ubuntu 1804 does not have this fix all right um and uh yeah so that's you gotta have some c plus plus food to figure that out if we look at a version at ubuntu 1604 right that's version 2.4.9 based on at least what the triage said

that's not vulnerable and 2004 is running version 4.2 which should carry the fix all the way forward so that shouldn't be vulnerable either all right let's look at what this vulnerability actually is right so um if you're into c plus c code right you'll get this but it's pretty straightforward right so we see over here this is version 3.4 which is the patch version it has this assertion up here which is not in version 3.2 it has an assertion over here it's not in version 3.2 and some assertions over here that are not in 3.2 we've got the source code we saw that way a long time ago all i have to do is bring that

source code back into version 3.2 and then issue that update into my ubuntu update channels and i'm good to go right now patched pretty straightforward stuff right that process of taking something from version 3.4 and bringing it in backwards into 3.2 is what's known as back porting and so if you haven't heard that term before you're going to be backboarding a patch all right normally your upstream only keeps the most current versions updated maybe a few versions recent but usually not as long as you know the five to ten year operating system life size the life side so that's why you got to back port the patches all right if we take a look at version uh

2.4.9 you know this is 3 2 over here on the left side and 2 4 9 is on the right the code's not there the code that causes the problem which is these i believe it was these two functions right here those functions don't exist in the older code so the older code's not vulnerable we've done our triage we know that we don't have to make any updates to the 1604 ubuntu 1604 or version 249 of opencv right now if i look at version 42 just for the sake of completeness i can see all my search ends are copied over into version 4.2 right so the insertions here are there and here they're there all right

so we're good we only have to patch that one version all right so that's going back to that ubuntu cve tracker you saw that 1604 is not vulnerable 2004 and beyond is not vulnerable only 1804 is vulnerable that's the only patch that we have to backboard okay so again looking at it a little bit graphically we see the same thing here we're going to take some code changes that were here in 3.4.0 and bring them back to version 3.2 that's in bionic right ubuntu bionic 1804.

all right so in order to do this again you know i stress that again minimal changes minimal impact to the end user we want to use the original code that was published and being open source that's all available to you so if you want to start working on patching stuff you can do that this is the opencv package in ubuntu that is on launchpad so launchpad is the ci cd platform for ubuntu right so the package sits here all right if you notice these are all the different libraries that get published with that package there's a lot of them here and then these are all the different versions that are distributed with different releases of ubuntu so this is a whole

pack of genealogy here and you can see going all the way back to trusty xenial bionic focal hair suit so these are all the different versions that were released with ubuntu so if i want to actually get the original source code for the original opencv that was pulled from debian that was pulled from opencv that was used to create the original 1804 version of ubuntu here it is right here right we're going to take a look at the current version here all right so the current version for 1804 um if you do you know an ap install or whatever and you get the source code or you get the binaries it's from this source package right here's the change

log on it so this is kind of neat um eduardo was busy eduardo typically maintains this opencv from the ubuntu side all right and you can see he actually updated all these cves is pushing out one update so normally what happens is the mediums you get to them when you get to them but the highs you get to all of them and then when you have a high risk vulnerability for a package like opencv then you pull in as many of the mediums as you can you got the patient open let's pull in all these updates so there's probably a high risk update in here somewhere along the way that caused eduardo to start working on opencv

pulled in took a look at all the cvs that he could patch at a time and put them all into this incremental release right so that was released in september 2018 our vaults from 2019 which is why it's not included in here all right but you can also pull down this specific tarball if you want and open it up and take a look at the source and modify the source all right and so that's what you do when you're patching it um you go back and you you back port that patch so we have the three floor which is not in the ubuntu ecosystem we bring that those updates into the 3-2 version add them in and that's how i patch in my

source code all right so again going back to the source code it's pretty straightforward i know i have to add this so i just forklift that over into the 3-2 code add this over add this over use a little bit of smarts to make sure yeah it does do what the vulnerability you know it's fixing the vulnerability take a look at it you know it looks good all right make minimal changes don't change white space don't add extra stuff just put it over there um and and i should be good right so i go ahead and compile it and it fails to compile all right so my back port failed why did it fail because in version 3 3

they added an empty method to the size object all right so now i've got a problem right so my back port my fix is relying on something that was added in version 3 3 and used in version 3 4 so if i'm fixing version 3 2 i can't use the same fix right i've got to write some code so it's not as simple as just lifting the code you actually have to write it and and and change it so this is where back parting becomes kind of challenging right this is pretty simple because we're just backboarding something that's not too different right it's only probably about you know six to eight months worth of development time difference but imagine

if you're backboarding something that's four years old six years old um it ends up getting to be kind of challenging right so um i talked with um seth seth is one of the guys that works on this team he's been doing this for quite a while and we're talking about back porting and he dropped me a note and he said um he gave me this bolded list of problems that he runs into from time to time when he's back porting patches all right so again imagine that we've got a patch over here from a new version and we bring that back to an old version because there's a vulnerability in the new version that's carried back to the old

sometimes all you've got to do is clean up white space move it over and you're good to go all right sometimes the code is moved right so i move from one source file to another source file so you got to track down where the code is and you got to start reverse engineering what this piece of software does right sometimes all the variables got renamed so i forklift my code but it's working on variables that don't exist in the older version and you got to tease all that out all right sometimes as we saw here the patch depends on macros or functions or methods that don't exist in the older code so you have to rewrite things you

can't just forklift that method back over because then it's not going to be compatible right and that goes um to another point which on my next slide but it has to remain the binaries have to remain compatible so if you do any reverse engineering you probably know that you know you enter into a function with a structure it's a specific structure right if your patch is going to change that structure of how you call a function it's not going to be compatible everybody that depends on that header has to recompile that that piece of code gets to be a a yeah a nightmare so you usually don't make anything that's not compatible in that way right sometimes the code was

completely rewritten and then you have to figure out whether the ball exists in the old code or not and then it gets into some more reverse engineering the code and so on and also when you're looking at a vol how many times have we looked at it and we say oh yeah they patched the bowl but there's more to it than that and you have to patch and patch again so that whole idea of the vole being the tip of the iceberg becomes one of the problems as well all right so let's assume that we got over all that we patched all our code now we've got to test it before we push it out

right so you begin by testing it locally and you're testing it against the original unpatched you take the vol the exploit right so we got an exploit with this piece of code so you take the exploit you run the exploit against the code that the the original code first of all to see that yeah it fails and now i know how it fails and that's what i got to fix and then you run the exploit against the patch code to make sure it's not exploitable if you don't have an exploit then you probably have to create one in order to really test to make sure that you've patched that vulnerability right and then you run all the other

tests unit tests and so on the opencv library is huge um their tests take hours to run and there's thousands upon thousands of tests unit tests that happen in there right and then you have to test all the different versions right so if i put the patch out for ubuntu 1804 plus 2004 plus 1604 i've got to test all those different versions um and then well if it's python right then you got to test against python 2 and python 3 because we're talking about older systems that support python 2. right and then you have to test it against different architectures right so you want to test arm i386 you know and so on and so on so you got to do all that

testing and then for opencv opencv often ties into hardware so you really want to do some kind of hardware testing so can you emulate the hardware testing and so on and so on and then you want to kind of test patch delivery so you want to put it into a mocked up patch delivery system and then make sure it gets delivered and gets updated and so on and so on and so on testing's hard right back porting to older versions gets really challenging really quickly right so when you're back porting to older versions you really have to step back in time to the way the world was in 2018 and then test the patch in 2018. that

means all the stuff that you use to test right anything that's in your ci cd pipeline for testing has to be the 2018 version because imagine if you're testing your 2018 code against signatures from 2021 against a new vulnerability or new code pattern uh problematic code pattern and you find it in your old code or you can update your old code right so really messy stuff that's why it's hard to maintain older operating systems and older software right um it just gets really challenging back porting patches is hard hats off to the guys at canonical that do this day in and day out i don't know how they can keep in their head all these different software packages all

the thousands of packages that make up ubuntu rip out the high risk vulnerabilities and rip them apart and patch them for all those just different distributions they do extended support as well so they're patching all the way back to ubuntu that's 10 years old right and have to recreate the environment as it was 10 years ago and patch that so it gets exponentially more complex and this is why we put software into extended support this is why sometimes you have to pay for extended support because there's just a lot of work to make sure this works so hats off again to the security team at canonical that makes all this happen um uh this is this is their launch pad

group right um uh the manager over there sean murphy uh sean took over from if any you know joe mcmanus he used to be here at the sci he used to manage this team um and also uh you've got a lot of help from these guys and putting this presentation together steve bailey manages the uh kind of the day-to-day work on how this all happens all right so um some really really cool stuff that happens there all right so one last thing i want to leave you with we've talked about ubuntu and how this works and that's hopefully informational for you to understand how this works as long as you do your automatic updates you're good this is the stuff that

you're getting they're prioritizing i feel comfortable they're prioritizing things right all right but if your software looks like this and you're not getting your software through a standard apt update it might be time to ask how they do their updates how do they get their code from upstream how do they consume it have they forked the code are they maintaining their own branches if so are they back porting the patches this is the kind of stuff that we're working on doing for the robotic operating system to maintain that after the upstream stopped maintaining it this is something that that is you have to be very aware of in the open source ecosystem so i'm going to leave it there

and see do we have any any questions anybody uh have any thoughts yet

that causes um numbers being involved being different um

yeah so that's a great question so how do we get the version numbers right and why does that matter is because when you do a an apt update and then an apt upgrade and look and see what software you're going to get down it's all based on the version number right so the version number has to be bumped up for you to get the new software installed right so if you're running version 4.2 and you have an update 3.2.1 that's not going to be installed because it's an older version right but if you're running 3.2.1 and you see at 3.12.2 you're going to install that right so so they there is a standard and i think

it's documented in their public wiki about how they actually number these and if you'll see this um these are number 3.2.0.dfsg and so on that versioning and the way that they number the releases make sure that you are getting the updated code if you have the older code installed so if you have version 3.2.0 we will update it 3.2.0.1 right so there so the way the version numbers happen make sure that you get the right code

so if you do the apt info i think the apt info is going to give you the whole forget exactly the command because i'm stuck in windows now with my new job but um the the the within one of the apt tool sets it'll give you the full description of what is installed and what is available and actually that comes from that comes from information that is distributed in this package like it's actually just a text file that comes with it so they should be built into the overlay that debian puts on top of the code when they package it right and then the red hat is going to have a different overlay but yeah yeah

absolutely so versioning is i'll just like there's a lot of nuances to versioning and versioning is challenging but it's also algorithmic so very standard you can make sure that you get the right one there's also the issue of this is an open source world shouldn't they commit the patch back to upstream so if i patch version 3.2 shouldn't i give that back to the opencv community and that is an option for some packages although the community has to be still supporting 3. 2. you know the community's not supporting 3.2 they're not going to take updates to version 3.2 anymore so any other questions all right so uh thanks a lot for uh thanks a lot for

attending thanks a lot for being here i really appreciate it uh oh we got one more

i'm sorry can you say again

yeah so if you uh under ubuntu advantage you're gonna get the support you're gonna pull in the updates um the lts versions supported for five years under ubuntu advantage you get the 10 year support so that you can run your operating system for 10 years and again in the robotics world that was why that was so important to us because your hardware is going to last for 10 years for sure so you want to make sure your software will run that long as well all right so if you've got any more questions stop by and see me i'll probably be at the plus booth and if you stop by we have this uh wearable wire

shark self-decoding packet capture so stop by and grab a t-shirt i love to chat more about this so that's all i got back to you thank you [Applause]