Hardik Parekh - Navigating DevOps Security Journey at Scale

[Music]

this devops era there is increasing pressure to speed up time to market to meet this demand organizations are relying on increasing number of tax tracks deployment models and open source software without clear security requirements it would be an understatement to say that modern software is growing in complexity as a result 75 percent of the vulnerabilities are application-related yet there is no prescriptive and measurable way to analyze and improve software's assurance posture for organizations to address these challenges we recently released oversam2.org my name is hardik parekh and i'm here to share my knowledge about how to navigate devops security journey at scale with os software assumes maturity model 2.0 a little bit about myself i started my

career as a software engineer i got an opportunity to lead various security teams and programs at medium and large corporations like dell emc intuit amazon and splunk i'm also on advisory boards for a few security and data privacy startups and a non-profit organization comptia which issues id security certifications such as security plus and surplus throughout my career i contributed to industry through various safe core publications sans dropped 25 programming errors cvss version 3.0 bsim since version one and oversam and recently missed ssdf for those of you who might not be familiar with bsim bsim is another software security maturity model which is descriptive in nature it was developed by sagittal by interviewing six independent software

vendors from safe code from founding member companies and emc was one of them it's more of a descriptive model of what activities other security organizations are performing at each maturity level i started my career as a software engineer and ever since then i've always focused on how do we optimize our software security assurance posture using automation i also started contributing to was sam back in 2016 and have been co-author and one of the core members of os sam project since then now before we start the talk i would like to get the legal stuff out of way i'm not speaking on behalf of my current or previous employers nor am i here as a representative of my

current or previous employers my opinions are solely my own and they do not reflect those of my current or previous employers now let's look at a quick agenda i assume many of you might not be familiar with the was sam hence i'll spend some time on introducing wasan covering what is sam why do we need a sam what are some of the core principles on which sam is built upon and a project history after that i will go over the changes we introduced in sam 2.0 or 1.5 and last we will discuss how to apply sam in your organization so you can help sam to navigate devops security journey at scale now if you are playing one of these 15

plus roles you will be able to take some valuable information from this talk and apply to your organization starting from tomorrow let's look at what is sam osam is one of the flagship os projects now flagship status is given to projects with strategic importance to both os and appsec in general os sam is a framework for software assurance that provides effective and measurable way for all types of organizations to analyze and improve their software security posture that is tailored to the specific risk facing that organization sam is full of useful resources that will help with evaluating organizations current security practices providing recommendations or suggestions for growing and maturing those practices providing a way to demonstrate concrete

improvements over a period of time and defining and measuring security activities throughout the life cycle one of the big benefits of sam is that it is vendor agnostic sam can be done in-house or you can have one of several apps consulting firms help you with the assessment and creating plans and roadmaps why do we need model like sam in quests to increase speed today's organization is growing in complexity without much clearing security requirements as a result almost 75 percent of vulnerabilities are application related if you look at uh later security breaches like equifax or capital one they are more application related to standardized security activities in such a complex software environment we need a model like was sam

i like this quote from george box who has been called one of the great statistical minds of country's surgery the most that can be expected from any model is that it can supply a useful approximation to reality all models are wrong and some models are useful i'll repeat the most that can be expected from any model is that it can supply a useful approximation to reality all models are wrong and some models are useful and sam wants to be that particular model which is useful enough for all different types of organization and versatile as well the point is not that you know you you can find an exact model that is going to exactly describe

the reality in your organization because there are too many variables but you can have a model that is close enough to be useful and that is what sam is aspiring to become sam was defined with flexibility and versatility in mind such that it can be utilized by small startups mid-size organizations and large organizations using any style of development additionally this model can be applied organization wide for a single line of business or even for an individual project sam is both measurable and actionable it defines maturity level across business functions and provides clear pathways for improving maturity levels thus providing a step-by-step navigation plan to achieve higher levels of maturity now let's look at project history

version 1.0 of sam was originally created through the open sam project led by praveer chandra an independent software security consultant after a number of years in 2015 a small group got together at oas and worked together to bridge some life into this uh sam project as an oas project first version after that version 1.1 of osam expanded and restructured its predecessor into four complementary resources the code document how to guide quick start guide and the tool box which is a spreadsheet that provides simple automation for data collection matrix and graphs back in 2017 we released version 1.5 of sam which incorporates a refinement of the scoring model to provide more granularity to the scoring in an assessment

we recently launched sam 2.0 in february 2020 where we have chained the measurement model one more time to add qualitative measurement to represent how well an organization is performing a particular security practice in addition we made few structural changes as a result sam 2.0 is not backward compatible i repeat same 2.0 is not backward compatible sam is built on few core principles first an organization behavior changes slowly over time changes need to be smaller and iterative to really take hold and make a difference second there is no single recipe that works for all organizations sam is built with this in mind and supports an organization building a program that is tailored to the risk that particular organization is

facing the culture of that organization as well as the maturity level of that organization third the guidance related to security activities must be prescriptive too often security initiatives fail due to poor details lack of communication or invalid assumptions overall the success of the program will be based on being simple well-defined and measurable now let's look at maturity levels and assessment scores sam is defined in three levels of maturity ad-hoc provisioning increased efficiency and comprehensive mastery at scale mainly through automation unlike other maturity models and sam predecessors there are full assessment scores for each security activity which makes fine grain improvements visible no implementation implementation across few or some projects implementation across at least half projects

and implementation across many or most projects the key point over here is in order to make this more practical and pragmatic we got rid of all and that is why we chose many or more more most projects because in real life there's always going to be one or two projects where uh organization might be planning to retire they're in the process of rewriting uh things like that so there is no point in spending your limited resources in securing those projects i would rather have you focus on mainly the projects which are being rewritten which are being developed newly which are going to be in production for a longer period of time and that is the reason that we have

consciously made a decision to not call out all projects to reach the highest level not everyone also needs to make level three in all different areas similar to six sigma or cmmi the goal is really not to max out on each and every practice honestly speaking that probably won't even be good use of your limited resources what the target maturity should be for your organization is largely up to you depending on business drivers and the risks your organization is facing in version 1.5 we have modified the scoring model to provide multiple choice answers to allow for more accurate assessment previously in sam and most models the questions were yes or no which is great from academic perspective

but in real world the answer many times lies in between the two for example if you try to answer the following question for education and guidance at level two are those involved in development process given role-specific security training and guidance you know you have trained some of the developers and would like to train some project managers and qa testers but given the question do you answer yes or no the dilemma is if you answer no you get no credit and it looks like you aren't working on it but you are if you answer yes you now get full credit and may have issues down the road asking for training budget for other roles because the dashboard says you did it

already with the new model you cannot answer something like some or at least half and take the credit appropriate carry but also have the ability to show improvements in your score over the period of time when you train safe project managers and qa testers now based on the user feedback we have made changes to the measurement model one more time in sam 1.2 sorry sam 2.0 we are assessing activity security activities along two axis one is the coverage by means of questions and two by means of quality by means of mandatory criteria now this is high level snapshot of sam 1.5 so you understand what changes we have made in 2.0 sam is defined in three different levels at the highest

level sam 1.5 defined four critical business functions each business function has three security practices and each eq practice has three maturity levels now motivation behind the new versions are we want to align their five motivations basically first is we want to align the recent development methodology such as agile and devops to make it development methodology agnostic now version 1.5 and its predecessors look more suitable for water waterfall methodology even though it was not meant to be we realized that it is because it's missing a key aspect of guidance in terms of how to securely build and deploy software especially since uh csd has become an integral part of the agile and devops methodology the second motivation is to improve the

measurement even though sam 1.5 address the feedback about granularity levels it still didn't address the question how well an activity is being performed at each level thus needing some qualitative measurement the third motivation is to avoid orphaned or unrelated activities there are quite few security activities in sam 1.5 where defined such that it lacked consistent theme across different levels of maturity within a security practice which also resulted in few orphaned or unrelated activities such as code signing at level two the fourth motivation is to arrange maturity levels in order of increasing difficulty other shortcoming of 1.5 and the other pers its sam's predecessors was sometimes security activities at high level were easier to implement compared to

security activities at lower levels which really didn't make sense and we had to fix that the fifth and the most important motivation was to improve the production process of sam itself sam production process was slow and waterfall resulting in major work every time we need to release new version so as a result of that we had to come up with a new version and hence sam 2.0 came to light let's look at sam 2.0 framework this is sam 2.2 framework at a very high level and the areas highlighted here are the changes from version 1.5 which will cover in a few minutes sam version 2.0 is defined in three levels at the highest level sam

defines five critical business functions each business function is category of activities related to the nuts and bolts of software development methodologies for each business function sam defines three security practices each security practice is an area of secure related activities that build assurance for the respective business function for each security practice sam defines three maturity levels as objectives each level within security practice is characterized by a successively more sophisticated objective and more stringent success matrix than the previous levels overall as you increase level of maturity you should expect higher cost of implementation if you look at the framework you can see that governance is more focused on the program itself looking at more strategic elements strategy and metrics policy compliance

education guidance etc we have renamed construction business function into design and introduced new business function implementation our design implementation verification and operations cover the core of software development lifecycle design is focused on threat assessment and security requirements during earlier phases implementation is focused on secure build secure deployment and defect management aspects verification is more focused on testing or verifying aspects and operations is focused on incident detection and management and environment management where the app lives on let's take a closer look at security practices each security practice is divided into two streams stream a and stream b the purpose of these streams is to align and link activities within the practice over different maturity levels each

stream has an objective to be reached and this objective can be reached in increasing levels of maturity this way we measure that there are no orphan activities that seem only relevant on a single material level for instance quote signing as we discussed in sam 1.5 let's take a closer look at one of the security practices requirements driven testing under verification business function the goal of requirements driven testing practice is to ensure that the implemented security controls operate as expected and satisfy project's status security requirements the former verifies that the application security controls satisfies security requirements and validates their correct functioning these requirements are typically functional in nature negative testing addresses the quality of the implementation of security

controls and aims to detect unexpected design flaws and implementation bugs through misuse and abuse testing these streams align and link the activities in practice or different maturity levels it does so by incrementally building a side of security test and regression cases and executing them regularly in its most advanced forms the practice promotes security stress testing such as denial of testing and strives to continuously improve application security by consistently automating security unit test cases and creating security automated security regression test cases for all the bugs identified and fixed now let's look at secure build this practice focuses on creating a consistent repeatable secure build process and according an accounting for the security of application dependencies the secure build practice emphasis the

importance of building software in a standardized repeatable manner and of doing so using secure components including third-party software dependencies the first stream focuses on removing any subjectivity from the build process by striving for full automation an automated build pipeline can include additional automated security checks such as sas and dash to gain further assurance and flag security regressions early in the life cycle by failing the bill for example now we are not recommending to start filling the bill on each and every security issue that would be too much of disruption for your cicd pipeline but once you build more assurance in your sas and dash tools and their lack of false positives then this is the highest form at which

you could start filling the bill based on some specific criteria such as fail the bill if sas tools identified any critical issues things like that the second stream acknowledges the prevalence of software dependency in modern applications it aims to identify them and track their security status in order to contain the impact of their insecurity on an otherwise secure application in the most advanced form it applies similar security checks to software dependencies as to the application itself now let's look at secure deployment this practice focuses on automatically securing uh deployments to the production environment and all required secrets one of the final stages in delivering secure software is ensuring security and integrity of develop applications are

not compromised during their development and deployment to this end the practice first stream focuses on removing manual error by automating the deployment process as much as possible and making it success contingent upon the outcomes of integrated security verification checks it also fosters separation of duties by making adequately trained non-developers responsible for deployment the second stream goes beyond the mechanics of deployment and focuses on protecting the privacy and integrity of sensitive data such as passwords tokens and other secrets required for applications to operate in a production environment in simplex form suitable production secrets are moved from repositories and configuration files into adequately managed digital walls in more more advanced forms secrets are dynamically generated and at the deployment time and

routine processes detect and mitigate the presence of unprotected secrets in the environment now my goal is to arm you with knowledge so that you can apply or sam starting tomorrow so let's look at how to apply this model in your organization now this is a typical approach rolling out sam 2.0 which includes implementing this phases in continuous fashion in cyclic manner but before we do that we need to spend some time preparing this phase is the most critical phase of for the success of osam application in your organization i have witnessed several organizations who skipped this phase which resulted in derailing their efforts completely it consists of following four activities define the scope identify if you would like to roll out

osm across the whole organization across a particular business unit or an individual application or project level once you define the scope you need to identify key stakeholders and get their buying you need to have different stakeholders buying as per your scope once you get stroke stakeholders buying you should start spreading the word and evangelize some activities one of the other most important activities here is also communication to measure current level of maturity you need to start with assessments by conducting interviews with key stakeholders to evaluate current security practices we recommend in-person interviews versus emails or slack this way you can explain the key intent behind any activity and clarify any potential doubts your stakeholders might have now there

are three ways in which you could perform an assessment a lightweight assessment a detailed assessment and a hybrid assessment the lightweight assessment is simply interviewing key stakeholders and recording their responses during details assessment you ask for evidence for performance and quality of each activity being performed in hybrid assessment you ask for evidence on a need to know basis for some activities and not all i personally use hybrid assessments depending on your situation you could use one of these three different types of assessments now if you involve third party abstract consulting firm to come and do the assessment most probably they will do more detail assessments but if you are doing this assessment yourself in-house you would probably choose more of an

hybrid assessment model there are many organizations which go through the full detail assessment for the first iteration and for the subsequent iteration they choose either lightweight or hybrid assessment models so this way you could apply different assessment models in different situations now once you record your responses you can assign maturity levels using sam worksheet that we have provided let's look at how to calculate maturity scores finally we came up with a new scoring model which still primarily based on the coverage however we also added quality criteria for each question to add another dimension to the score our guidance is to score zero if a quality criteria are not met going back to one of the sam core

principles of simplicity we decided to add quality criteria for each question this way time to complete an assessment did not simply significantly increase with sam 2.2 model overall maturity score for the security practice is calculated by taking average of maturity level 1 between stream a and stream b and adding that to the mature level 2 and metro level 3. now once you finish an assessment you need to define the target as per your as per your organization's business drivers and risk profile as we discussed earlier it is very important to spend some time to understand your organization's business drivers and risk profiles during this exercise now if you look at healthcare organization they are facing different

their different risk profile and their business drivers are very different versus financial versus government versus startup once you define the target the most important step is to estimate the cost of implementation many semen initiatives fail when folks forget to estimate and plan increased cost of implementation resulting in lack of resources dedicated to security implementations that they plan for that's why i highly recommend to work with your program managers and senior management to plan an estimate for the increased cost of implementation or sam improvements the cost of implementation would now become direct input into defining a plan during this step you need to determine change schedule as per the upcoming releases and develop or update your roadmap plan over the

next four or five phases we recommend implementing sam change or minimum of three phase and maximum of five phases each phase can span between three to four months up to 12 months in order to focus on the highest impact you should start with high impact security improvements such as training and awareness as well as threat assessment now once the plan is defined you need to start implementation implement activities using sam2.o guide in addition one could leverage other was projects now sam aspires to be an umbrella project for all was projects that means each os project can map back to one or more of the sam business functions and security practices here are some example was projects and

how they map back to various sam business functions now there are some projects was projects that can map to multiple sam business functions such as wash top 10 and os mobile top temp after the implementation we need to create and update scorecards on regular interval by capturing scores from before and after an iteration of assurance program build out and communicate progress to the senior management i have used this scorecards to demonstrate and communicate security improvements to the highest level of management in the companies that i work for this health management visualize the progress in overall security improvements which results in to even more support rolling out next phase of security improvements going forward now these are some of the

available resources we have to get you started using sam 2.0 all these resources are linked to the os sam website i would like to also point out couple of newly added resources so sam always provide that excel spreadsheet toolkit so we call it sam toolkit which was basically in excel spreadsheet we now have the same spreadsheet or same questions and quality criteria available in google docs version as well we also have concord usa donated their online sam 2.0 calculator to us so concord usa sam 2.0 online calculator is another resource for helping you do the perform the assessment online there is also sam 2.0 dashboard created by satish ashwin who also donated this his contribution

to sam 2. project and if you have any further questions or feedback you can always use osm assessment feature request now one more thing before we conclude this talk we recently introduced sam benchmark initiative which helps answer the question how do i compare to other organizations now science benchmark initiative is inspired by bsim as i mentioned earlier bsim is a model which is descriptive in nature which tells you what other organizations are doing at each maturity level versus was sam is more of a prescriptive model which tells you what you should be doing at each maturity level one thing we like about bism is bsim score can help you identify where you stand compared to the other organizations

sam benchmark initiative is aspiring to do the exact same thing the goal of this project is to collect the most comprehensive data set related to the organization's maturity levels of application or software security programs and we have make this data collection available in two two ways mainly one is a anonymous data collection and second one is a verified data collection so in anonymized data collection one could choose to submit the real world data without revealing their name their contact information or even their company name in the verified one as the name suggests we verify that and we uh depending on your preference we can make it available to the rest of the world or we can just

put you as part of the data collection initiative now as someone said proof of the pudding is in eating it so if you haven't yet i really invite you to start using sam 2.0 starting tomorrow hopefully i'll come back to besides philly next year in person to discuss the lesson learn lessons learned i really wanted to thank b-side philly organizations organizers volunteers and sponsors for putting together a great besides philly conference i'm really looking forward to visiting in person and thanks to all of you for listening to this talk if you have any questions feel free to contact me at this email address thank you

you