Crowdsourced Security

buta states have fun man all right thank you very much first of all I would like to thank the besides company for inviting me here and we'd like to thank you guys for your interest in my talk so the topic for today will be about crowdsource security and which is a trending topic in the past few years and how we can be like an efficient and cost-effective approach in order for you guys to augment or complement your internal security processes all right so let me introduce myself first so my name is Ian AppleCare I grew up in Morocco I'm based in France now and a security analyst at hacker one if you guys you guys know hacker one you know one here

that's perfect that yeah you know it always yeah so hacker wants like a platform it's like a vulnerability coordination platform and a bug bounty where we connect businesses with the hackers so yeah yeah I'm also a white hacker so basically my role at hack one is to do triage which means we validate security reports we received from from hackers and white hat hackers and security experts and we manage program Oni programs as well in my spare time I'm also a white hacker white hat hacker I used the hacker 1 platform to hack on different companies it's it's so much fun I mean you get to put into practice your own skills and it's very challenging and you make some extra cash

on the on this side so currently I'm a ranking in 12 on the hacker 1 leaderboard over 100,000 hackers which is good I hacked on Facebook I hacked on Google Microsoft Twitter etc alright so let's get into it so the agenda for today we'll be talking first about regulations compliance and helping iteration testing is helping with that we're going to talk about cross or security which is the main topic for today and what is that teach me some history we're going to talk about how the whole backbone I think started then we're gonna dive into how crowdsource security can be efficient and cost effective solution then we're gonna talk about some of the back money drawbacks and yeah then we're

gonna talk about how big money can actually be fits into your software development lifecycle and finally an interesting part where we're gonna talk how you guys can launch a successful and healthy by Pawnee program all right so regulations compliance and penetration testing so yeah the businesses with online prisons today and businesses that deal with all kind of user data or user personal information they have to be completely compliant with different regulations in place like give you the example I'd I choose the these two examples the PCI DSS the payment card industry data security standard which applies to companies that actually deal with in payment information so the PCI DSS is actually mandatory in like for a merchant for

online merchants and we also have the GDP are you guys all know the GDP are I guess you got the that wave of spam back in April or May so yeah GDP are is an European regulation we would find up in case of a security breach do you would find the company up to 20 million euros or up to four percent of their annual worldwide turnover so what companies yeah so if companies have to become to be compliant with these regulations and one of the solution the refer to rule is security audits and penetration testing is one of those traditional traditional approaches that they actually do or conduct so but the thing here is that

penetration testing is has been like it's not been really an efficient or cost-effective solution and it has so many limitations that so many companies now have been trying other alternatives one of those alternatives is called source security which we're going to be talking about today all right so yeah source security what is there so basically before we talk about what's our security you have to define what is crowdsourcing it's pretty basic crowdsourcing is when a company has a task has a problem or a project to accomplish and what they do is they seek the help of a large community people from all over the world asking for help it's like the freelancing platforms like app work if you know it it's like you

have a project you want accomplish you ask for help and you pay them in exchange for your help so crowdsource security actually is in line with this definition we talk about cross or security or what hacker one actually if they define it as hacker powered security so yeah when we talk about crowdsource acuity it's a general term but here today we're more interested in what we call the backbone II programs which is a form of crowdsource security a pepperoni program is a sort win company actually ask the help of hackers or white hat hackers or security experts they ask them to find vulnerabilities in their products their web services and in exchange for each valid and unique vulnerability that they

find they actually gave a reward that we call a bounty so that's why it's called the bounty program so when we talk about the bounty program we have different programs that will have two sorts of programs actually we have private program it's when you only invite like a selected few hackers like only a lot like about 10 or 20 hackers that they are like and did it and we or we have a public program we just announce your program publicly hey guys if you want to make money they come hack our program our products or services so this is a private and public program and below here we I have a quote or a testimonial

that I really like and it summarizes the whole purpose of background programs it says hackers have become part of an it have become an essential part of our security ecosystem this quote is from General Motors different so this is really important because it says like instead of like fighting white hat hackers and considering considering them like malicious adversaries you actually incorporate them as a component in your security ecosystem you actually work with hackers in order to improve your security so yeah that's why the I really love this coat is it summarizes the whole concept of background programs all right teach me some history pray some history about the background programs how it all started it was back in 1995 when it

scape you guys all know Netscape the browser they launched a very premature program and the with asking users to actually find bugs on Netscape 2 point 2.0 and they had like a limited budget like 15k box and they yeah in any way they like operated this program until they really is that the next version of miss cape then the idea just didn't make its way to other companies so they just ditched it and it's not until 2002 when I defense they launched an initiative where the action asked hackers to report security vulnerabilities they find on software's and applications and what would I defense would do the play and intermediary role and they would report it back to the vendor and pay the

researcher so that yeah it's not until 2004 Mozilla they started their own program they are paying they were paying minimum $500 for every security vulnerability that you find in their browser which is Firefox all right then we yeah it's not until 2010 that Google started their very massive and huge program the like they promised to pay up to 20 K 20 K dollars for a critical which like inspired the other companies like Facebook to start their own program as well nowadays we have Microsoft we have all these companies that we run in their program on hacker one we have Salesforce snapchat slack Twitter Airbnb Starbucks Shopify instacart a lot of companies and what is amazing actually

is that even the freaking US military has started their own program they have distorted the hack the Pentagon like half the US force and hack the US Army this this was huge actually that in this year actually in Las Vegas during DEFCON the US Air Force actually throw their first life hacking event where they gather top hackers from the world to hack on their surfaces web services the public facing ones which was really amazing and the very turning point in the industry all right so let's talk about efficiency and cost effectiveness so yeah it's it's not really easy to measure the efficiency and cost effectiveness or crowdsource security crowdsource programs but I try to like to elaborate this basic model

where I that stresses out and in fact the adoption of cloud security actually results in better efficiency which is measured by the number of valid and unique vulnerabilities identified as part of the background program it also results in a better cost effectiveness which is measured by how the how better the results you achieve at a lower cost compared with alternative solutions like penetration testing but this whole positive like relation actually depends on the level of crowdsourcing like if you're running a private or a public program all right so I gather some data online about some different like companies that are running programs like Facebook Google the they blog about their stats about this that's about

their program and Salesforce over Twitter they have these stats publicly accessible on hacker 1 for example so as you can see here you can see that Google for example they paid they've been running their program for seven years and so far they paid over 12 million dollars for for for about for about like 300 hackers which is amazing and they like the batter the background program actually helped them identify more than one thousand two hundred and thirty bucks which is crazy like apparently he's like I don't expect a penetration Google would pay like way more than 12 million dollars in penetration testing like engagements and I would not even expect it to uncover this this minute bucks to be honest and

we have this is just Google escape to Twitter for example for Yahoo you have for example they paid like two million dollars and they've been running their program for five years and that helped them the program the batpony program actually help them find like four point five K bugs that's amazing that's really huge and they only paid like two million dollars which is not really a big thing if you consider how large yahooeys and how much money they make so let's break it down is this an that for example yeah I'm gonna compute the average the average number of bucks they find on a yearly basis and the average annual bonus they pay on Google they paid like

12k in total which means like if you break it down that means like they pay 1.5 million a year for for about 100 in these six bucks that is a good deal to be honest and Yahoo they pay two million in total lack and they find the program helps them find about eight hundred eight hundred bucks which is crazy actually I don't expect any penetration I'm not against penetration testing at all I think penetration testing is really important to be honest and but my pony program is really helping augment the the augment the internal processes you have in place so yeah the only pay like three three hundred thousands like dollars a year for for this backbone II

prefer running this by Pony program which is not really that all right so yeah I've talked to like about 25 people employees from various inaudible organizations and I've asked them about how much they pay for penetration testing on Iona on a yearly basis and as you can see it does not really does not really come at a cheap price for example like for a medium organization organization they pay from change from 20 K to a 40 K and like some of them actually pay more than 80 K a year for penetration tastings which is really not easy and really cheap given that the penetration testing can actually sometimes can yell like zero results it's not like an obligation to

find a bug but you just like do an assessment which might not yield any results at all anyways here I did some basic comparison between penetration testing and running a background program and like when I used different criterias for example time a penetration testing the average of our penetration testing engagement is actually two weeks but for a backbone II program it's actually a continuous assessment like you're running in a program continuously and you have you have researchers you have the security experts and the white hackers poking around your assets like continuously constantly so you have them cut like cover your ass all the time while I penetration testing only a like lasts like for two weeks all right let's

talk about this scope all right for this scope penetration testing actually you actually professionals they they they make sure to cover the whole scope like contractual contractually but there is the time constraint which means they might not actually cover the whole scope in depth but for a back on the program these researchers do not give you the assurance that they will cover the whole scope but one thing is that these researchers have they don't have this time constraints which means they might like like come back to hacking on your assets all the time which means that they might actually cover it cover the scope in in greater depth actually compared to penetration testing all right for the code for the cost for

opinion for a pen test actually you pay for the hours you pay for the effort like the the professionals they've been working their asses off like for two weeks you have to pay them for the hours yeah not even like it's not even necessary that they find something they found something or not that you pay them because they work their asses off it doesn't matter but for the backbone a program which is amazing about it is that you actually pay like food results you pay for the bugs if you find a bug I'm gonna peel for that but if you don't find anything it's not my problem which this is yes the pepperoni program actually is a results-driven which is

good and with that that's why it's really cost-effective like you get it you get a better results at a lower price for the personnel yeah like yeah for a ping test actually you the incan whole engagement is probably on average assigned like two professionals to do the penetration testing but about but for the Badminton program crowdsourcing your security means you actually seeking the help of a large pool of researchers like so many hackers hacking your own your applications and products you like you have all different kind of skill sets hacking on you which is amazing and for the competitiveness the for the pen test there's a lack of company that competitiveness like you have to you'll

have you and your coworker doing a pen test there is no competition between you who finds the bug or not but in the background you programs even won't everyone wants a reward everyone wants that cash so there is like a huge competition between security researchers who's gonna who's gonna find the first bug which is really amazing it's gonna help you like improve your security anyways for the methodology professionals they have this really standard checklist the OWASP top 10 error note probably very all not even updated but for the researchers actually for the back many programs people that are hacking on your program they are up to date to the latest research the latest hacking techniques and they

always like sharpened their methodology is their recon methodology is to find the the craziest bugs on your programs which is really amazing and they're really up to date I'm talking actually when I'm talking about this I'm talking out of my experience as well because they do this actually i hack on myself on different programs so I always make sure to sharpen my recon in sons and find like invent some creative ways of finding different bugs that nobody would find actually all right yeah so I've talked to the 25 employees and asked them about what they liked about bug Monte's and they said that we actually are very satisfied of the large pool of researchers they are satisfied the owner

of the fact that bat bounty programs are result-oriented and they are satisfied of the ongoing testing like the constant security assessment they are getting the constant feedback they are receiving but is it really a perfect approach I might have been like painting it in gold but actually there are some drawbacks about the crowdsource security that I'm gonna be talking about now one of the most annoying things about running a crowdsourced program is that you get a lot of noise you get a lot of invalid submissions especially if you're running a public program what I mean by that is that sometimes you receive some issues like hey you are you have actually import 80 open on your server

we have port 44 3 open what the hell that is not even a security issue or you receive like you have an SSL cookie that is that does not have any HTTP only or secure flock so yeah you have to be ready to receive a lot of noise from the community because like there are some script kiddies out there there are some beginners there's you're just starting to learn a few things here and there anyways and you're gonna receive a lot of poor quality submissions and by that I mean you get a report it doesn't have any details it does not have any description even if it has a description nope you see no proof of concept so you

have to go back and forth and discuss with the researcher and tell you extract all the needed information out of him and the third one is non respect of the program scope ok so you actually run it you launch your program you tell hackers I want you to hack only on the core application but some people would actually go out of scope they would go and enumerate all of your subdomains they would hack on them those subdomains actually point into third party services so they end up hacking out your partners which really sucks I know so this is one of the third annoying points here and then we have automated scanning some people they don't know how to hax when you just like

by an at UNIX license and you just running against your server so you receive a lot of requests from the automated scanners which really sucks again because if you want it if you want to run an automated scanner you have you would have done it yourself you would have not needed back money program so I know it sucks in anyways there is also a responsible disclosure and I think it will have has talked about it like some people may not be very like they may not accept your decision they may not accept your security assessment of the report so they get pissed off and they take the finding and the blog about it publicly probably to brag to brag about it I

don't know what their motivation is but that could happen but not all time actually anyways so yeah you have also the non respect of the company's decision so we get a report you fix it then you reward it's a low real report you give it the minimum reward you give it 100 bucks but then this researcher is not really did not accept that your decision I want 200 bucks sir so yeah they might not respect your decision and yeah it happens a lot of time I do a lot of triage and sometimes we have to deal with this kind of behavior but I have just like it's all it boils down to the communication you

have to to try to convince the researcher and give some solid arguments while you've paid them much money or why you reiterated the report as low not medium so it's all about communications and that we have a lot of professionalism yeah sometimes you might actually get insulted yeah I actually have this screenshot here and anyway talk about it yeah this is this is from uber program and so over what they did is that this this hacker his admitted report which is not really a security issue it was like [ __ ] and it was actually it was [ __ ] and it was out of scope he did not respect this scope then they closed it as informative

and he was so pissed off what he did is that he called them the worst like the worst really the worst insult there's like bloody mother blah blah we're gonna skip that part you taxi driver he was an uber employee so he called them a taxi driver and this is really popular joke joke in the back Pony industry I actually have a sticker here that says taxi driver so it was really popular this yeah anyway so you have some people that really like professionalism and they would go beyond and insult you and do whatever it takes just the PCO of as well and also you're gonna get a lot of spam especially if you're running if you you're asking

researchers to do their testing on your production environment what I mean by spam is that some people would actually just come and [ __ ] on your public forums they would just like send contact forms as well just do for tasting so you should expect a lot of spam anyways how does background it fit into your software development lifecycle I think you're all familiar with this software development lifecycle here which is like a process that consists of different stages that a developers actually follow in order to produce high-quality software and the thing about the is DLC is that you have you have the testing you have the testing right before right at the end before the deployment yeah so what

happens is that this actually puts so much pressure on the release date which means that yeah like you want to deploy you want to deploy the application but you still need to do some security testing which may take may take a while and like like as I said it's puts pressure on the release date which might result in conflicts between departments but I mean in the era of like agile development right now we're talking about agile it's like it's it's not really good and that's why we have started we have started talking about the security development lifecycle which means that you actually do all the testing throughout the whole the whole process but if you're doing this the the

security development lifecycle you are already in the right direction but the thing about this is there is security development life cycle is that it also presents some drawbacks and limitations that we're going to talk about next one of them is the increasing other attack surfaces you might have done your risk assessment your threat modeling but there are always some as it's some some some things you don't know even know that you own so young this'll only hackers with their reconnaissance with their reconnaissance would uncover it and there are always new vulnerabilities new hacking techniques you don't even know about so on a daily basis there is an increasing of attack surfaces and you have a shortage in terms of cyber

security skills like you have your own developers application engineers but they lack basic security knowledge they don't know how they know how to build and develop but there are careless about the security and yeah and you also have inefficient testing methods like you're doing your testing yeah but you're only using a kinetics or using perps with scanner which is now which is an inefficient middle-middle of testing because like those scanners that actually cannot uncover some kind of vulnerabilities like the logical vulnerabilities that actually requires the human human-like mind like human intervention so these are inefficient so yeah so the question was how do backbone II actually can help you throughout your hdl-c the the answer is that by phone he can

help you through the different stages here I'm gonna explain how for example yeah in when you're talking about your risk assessments the bank money actually can help you identify areas of highest risk and exposure which what I mean by that is that you actually get reports from hackers from different hackers and then you for example you notice that you're one of your functionalities you have built in your application is the most vulnerable one so now we have more you give it more attention and it helps you helps you actually recognize the area that is mostly vulnerable or what kind of vulnerabilities you mostly receive from these hackers and then you work on those also backbone II actually really

helps you reveal non secure coding practices and some of the risks that are associated with some design architectures can explain how so for example your building your application you're building the password or is it a reset functionality and the logic you built into your application is that when a user enters a valid email associated with an account you display this message hey we have sent you the reset the password reset link to your email when the user the second scenario is that when the user enters an invalid email that does not exist in your database you just say hey this user is non-existent so this is this is the user friendly because I know my account does not exist

on your application but security wise it's a really bad practice because I can enumerate all the all the valid accounts I can have like mail list and brute force the password reset functionality define all the valid accounts on your database so this is like a design decision which is user friendly yeah I can either think in user experience but security in a security standpoint is a bad practice this is one of the things that you're going to receive in your backbone your program is gonna like help you reveal all those non secure coding practices and there was design bad design decisions also the third one isn't uncover unknown security bugs and missing best practices to look out for

during the development as I said you're gonna receive a lot of vulnerabilities so you're gonna know all the kind of box that you should actually pay attention for when you're developing so yeah this actually can may help you actually plan your training programs for your developers so now you have all kinds of bugs that are are you are vulnerable you are honorable tools so this is gonna help you a lot and heavy continuous dissing and vulnerability assessment just like I said before you have hackers poking around your assets all the time you get constant continuous security assessment as opposed to penetration testing which lasts one or twice a year all right so the interesting part is how

you launched a successful or a healthy back money program yeah what are the minimum requirements you have to pay attention to or you have to meet alright so the first one is really important this one is very important it's about preparation and logistics you have the first component is budget you have to secure the budget because you're gonna run a bug bounty program which means that you're gonna be obliged to pay bounties to pay monetary bounties and also you have to pay fees for a metaphor for maintaining and managing your program on some platform like hacker one or zero culture so we have to secure the funds you have to secure the budget you have

to already pitch in the idea to your report of Directors and get them to give you the green light then you have you need a managing team but what I mean by a managing team is that you need to have people that are gonna help you manage your program and one of the first one of the important things is that you need to elect a leader someone who's gonna be only in your program has gonna be managing the program and making sure it's healthy and successful and you need other engineers to help you with the program like you're gonna do triaging they're gonna pay out bounties they're gonna communicate with hackers they're gonna have to escalate vulnerabilities

internally so you need to have a team supporting you and show some commitment this is important this one is you need a vulnerability management process you need this management authority management process because you're gonna have to escalate vulnerabilities internally you need to communicate those bugs to the right owners like to find the the responsible developer or engineer so you might actually already have this vulnerability management process in place but just know that when you launch it back on a program you are adding in you stream to your vulnerability management process yeah adding so on new bugs that are gonna be escalated internally alright so the second phase is that you need to find a by Ponty platform it's a cloud it's a

cloud-based platform where you're gonna launch your program and maintain your program we have like you can actually run a new manager program in-house or have a self hosted program like Facebook does like Google does but it's not really easy trust me like uber Yahoo Spotify Sony they all managed their program won't hacker one for a reason it was like when you have to pay these hackers you have to pay them their bounty and they are from like geographically dispersed everywhere you how can you deal with all the tax forms how can you like yeah this is just one one of the points like one of the benefits of running your program on a or your program on a on a bounty on a

platform vendor so also you have to keep track of the status of the report you also need access to different stats see this is what these platforms are actually offering and it's really hard to manage your program like in-house so my recommendation is to find a platform that's gonna help you manage the program we have different vendors we have hacker one the company I work for they're doing great I recommend them but I'm not gonna be biased here in a way so you have a crowd as well you have to see that you have zero copter zero copter biz is based in Europe in Netherland which is which is great and those three are actually in United States but we have

different customers from different areas of the world all right then the important thing is you need to define the security policing as my my managers manager Adam says here this is a good quote he says the seek the front door for hackers here - any bug bounty program is the security page I'm gonna explain what that that means it means that when a hacker wants to hack on your program when they want to participate in your program the first thing they actually see is your security page with which lays out all those rules of engagement things to do things not to do vulnerabilities we are interested in is we don't want to hear about like their

SSL cookie thing anyways yeah and so yeah throughout the next slides we're gonna explore the different things you have you need to include in your security policy the first one is you need a clear and well-defined scope what I mean by that is that you tell hackers hey I only want you guys to focus on this or this you want them to focus on your core application you you add it to the scope like app dot example.com and you want them to hack on your audio your subdomains you add a wild-card but be careful when you add a wild-card and you just want them to hack on your core applications that's your fault so you

have to be very clear and you have to maintain a well-defined scope and this scope can have like web applications mobile applications hardware open source code so yeah this is an example of Twitter's actually Twitter's scope so Twitter they say to hacker hey we want you guys to hack on vine Darko all the subdomains are included in this scope anything you'll find there it's gonna be eligible we're gonna pay you for that they also had their Android mobile application below and they have periscope TV which is an acquisition and mo pop comm anyways this is how a good scope looks like and if you're a hacker you have to pay attention for this scope as well when we when they say for

example we only want you to hear how come the core applications don't go on hack on their subdomains because they might end up hacking on their partners or third-party services they are using so be careful guys anyways so also we need to add a qualifying and not qualifying phoner abilities like hey we want you to look for cross-site scripting sequel injection remote code execution this is the kind of box we are interested in and would qualify for a reward but we also have this list this list were not interested in we don't want you guys to look for it we all want you to hear about it so you have to be clear about the things you are actually interested

in and you want to receive as part of your Pokemon a program then this is the most important part you should be transparent about your bounty table and your bounty tables should be like preferably be severity based which means that hey guys if you find a low severity bug we're gonna pay you $100 if you if you find if you find like a critical one we're gonna pay you 1,000 and 400 bucks but it depends if you if you just start if you just you're back on a program you want to start with the median bounty like the lowest they can get is 100 and the highest is 1 1000 bucks but then when you're actually your program is getting

better and in and the your scope is hardeneth and it's really not easy to find bugs you would want to be able to be competitive in order to drive in hacker engagement to your program you would go for example competitive bounty it's in the middle for the lower you pay you pay 250 bucks for the critical you would pay 9,000 bucks if you are if you have a good budget you have like a really good secure application you would want to go for top bounty these would attracts really good hackers then because if if I heard that hey I'm gonna be paying you like 500 500 bucks for a low bug I would definitely go hack on

them yeah yeah that would be a good return on investment so this is important you have to be transparent about the bounty table and you hit shooting included in your security page then yeah we talked about the bounty table and we I recommended you to be that that you maintain a severity based table but how would you how would you rate the severity you can either use the what we call the CB SS I think you guys are familiar with services if you are doing a penetration testing I don't know if you're using it but it's very important CBS is it allows you to actually like rate or like rate the severity of the pocket they have

different you have different parameters here that you can you can use anyways I don't have much time so I'm gonna rush a little bit and also you can use the vulnerability of rating taxonomy it was it was built by book crowd which is really fits the bug bounty industry and it has like for a critical its p1 high P do the it's it's based on the impact of the of the bug like if you like for an exorcist could be medium as it could be critical it depends on the impact what are you can achieve by them by that and also you have this is important define this security you have to define your service level agreements which means how

much time you're going to need to process the the the the submission successes it to process the report for example how much the time to first response and how much time jina cannon need to aknowledge that you receive the report we recommend that you actually do it within two business days time to triage this one is important okay you receive the report how much time you need to actually triage it and confirm that it's valid we recommend to business day and then time to bounty if you want to drive hacker engagement you want a hacker hacker to hack all your programs you gotta improve your time to bounty okay your save the report you a knowledge is valid how much

time do you still need to give me my bounty so you have we actually we recommend that once you triage you pay the bounty will you pay like half of the bounty then you pay the remaining when you resolve the report time to resolution it time you need to actually fix the bug it depends on how critical the bug is for a low bug it would take you like a month is okay for a critical one it should be within hours or within one day is this a critical you don't want to take so much time anyways that finally I'm gonna talk about the legal safe harbor this is a new thing we've been talking about in the industry shout

out to meet Ella Zuri she's a researcher that's been working on this thing to introduce it to the bounty industry so what I mean by the safe harbor is that I'm gonna give you the example or I don't know if you have heard about this so DGI the drone maker DJI deal watch the problem there the program publicly and they say they promised hackers that we would pay you up to 30k for a critical than this guy is hacker he found a nasty bug he was he had access to their clutter their store a cloud storage cloud and he he found some keys in their github like they were public there and he got access to their cloud storage

swamp and when he reported it they actually threatened him with legal charges so that's why this safe harbor is really important you have to assure your hackers that nothing is gonna happen to you as long as you have you acts on a good fate and you're gonna responsibly report the vulnerability to us alright so yeah the next phase is actually launching your program but before we launch it starts small then you work your way up what I mean by that is you start with a private program you don't just go and launch publicly you're gonna get hit by a wave of a report so you start with a program you only start with one or two

assets your core application or and the second one it might be your second core application wherever I don't know anyways and you invite a few hackers like teen hackers or maximum twenty hackers then yeah and then you do your program scale and which is the next phase like you've been running for a while now and the the next thing you should do is expand your scope you have two assets now you can add more assets to the scope now and you increase your bounty payouts like if you want to still drive drive the hacker engagements like make hackers thing into B is still interested in your program you need to increase your bounty table you started

with meet the media no court for the competitive pls like I showed you in the bounty table then you invite more hackers and finally then we have the program skilling note that you have a CD volume of reports you are your developers and meeting the SLA is we have sufficient budget and you have an effective vulnerability management process and you have happy hackers you I think you actually know with like you're ready to go public take your program public and announced it publicly alright so briefly I what I want what I'm seeing here like the whole purpose of this talk is I just want to say that I'm not against penetration testing are all I think penetration

testing is really important but getting and receiving a continuous security assessment is very crucial so guys think about it thank you very much you can't win a meal

awesome thank you brilliant race to the end as well any questions right if you ask me arrow thing actually it's a good standard because PC hi they only require like the penetration testing once a year unless there is a significant change and like one one penetration testing a year is not really enough like you have the code that is being pushing all day or on a daily basis so no it's not really enough I think probably like doing a penetration testing along with having a backbone a program would be fine yeah the other questions guys anymore I hope you protect this crowdsourcing buck-20 program from fraud for example the internal insider program that is leaking the but to someone was doing the

crowdsourcing banty program this guy can you rephrase your question is how do you protect this crowd sourcing program from fraud for example some internally guy just leaked one of the undisclosed work to outside Marty and then this guy just use this information to do the bug bounty program internally you mean like you it means if somebody in the company leaks bugs they know to somebody outside to get bounties well actually I don't think that that's actually a company's problem because the employee is supposed to have signed an NDA like why would you leak it I don't think there is like anything we can do about that unless like fired his ass out of the company

para there is no solution to be honest yeah that's a problem of the organization the employees don't have to do not trust the organization so all right more having a question here yeah yeah this is a similar question the guy at the back could you like could you imagine a scenario where you get penetration testers in for two weeks and you pay the money but then they keep the bugs they find themselves which they then submit to bug bounty programs afterwards so you're getting a double payment it's not a common scenario in the industry or that actually that actually could happen but the things that is that just like as I said to him the problem is with the with the

penetration testing company actually I mean they could file it as I have as part of the back on a program and get rewarded for it but I mean Italy it's bad it's ethically it's bad but still the the customer gets the vulnerability through the bug bounty program which is fine as long as it gets fixed so it's a question of the way attacks that could happen that's you know you could have not you know that is true actually that happens differently I confirm that yeah all right if you have any more questions rights talk to him afterwards you will be here thank you very much again