Against the Tyranny of Optimization: On the Stability of Automated Republics

And without further ado, I'd like to introduce today's keynote speaker, Katie Masuris. Katie is the founder and CEO of Luda Security, a company that creates and manages sustainable bug bounty and vulnerability disclosure programs. She advises companies and governments on emerging threats, sorry, emerging trends in AI and cybersecurity. Welcome to the stage, Katie.

All right, thank you so much and thank you, Reed. Okay, can you see all that? Perfect. Can I get everybody to just stand up? Just stand up.

I'm in the middle. Move towards the middle. Move towards the middle, get friendly. Move towards the middle. Yesterday, I came in on the tail end and there were seats everywhere and I could not get into a single one of them. All right, I love you all for doing that. Thank you so much. Next time, just fill in the seats in the middle like they tell you to. All right.

Yeah, you thought it was a trick. It wasn't a trick. And look at how many people just had to move so far into the middle. So far into the middle. Okay, so does anyone get the reference of the title of my talk? No one, okay by the end I hope that you will, but we are gonna talk today about the tyranny of optimization and the stability of automated republics because we are moving into an era of an automated republic. So philosophers, when they warned us about tyranny, they were not picturing a dashboard They were not picturing automation in the sense that we see automation. They were warning us about governments and authoritarians and troops and things like that.

But the dashboards and the automation represent an accumulation of power and a concentration of power. So we're going to talk about how that has been growing in the AI era that we all live in. and what is gonna happen when all of these decisions are starting to be made faster than our traditional institutions and organizations and our governments can absorb them.

So we're in cybersecurity and more importantly, we are in the business of trying to deal with risk. We try to prioritize risk, recognize it, predict it if possible. But in cybersecurity, we are sort of like this forefront as things begin to change. And yesterday, if you saw Anna's keynote, we've been here before. We've been through cycles and cycles of change and adaptation. But essentially, the pattern that we see is that Attackers will adapt first, defenders have to catch up, the institutions that are trying to defend themselves and the governments are playing catch up last. And essentially we're watching this all unfold right now.

So what is going to happen to us when intelligence becomes abundant? How many people have played the CTF? How many people use Claude to play the CTF? Yeah, that's everybody. And I was talking to somebody who had his laptop open during happy hour because he was trying to prompt something, you know, because Claude kept stopping. And he said, yeah, at first I tried to find the flags myself, but then I just decided to do what everyone else was doing, which was just give it to Claude. And so we're in this era where, you know, we're seeing AI take over everything. My special interest isn't trains, but as many neurotypical people who have special interest, it's bug bounties and vulnerability disclosure. So we're gonna talk a little bit

about how AI has affected that world, how it's affecting the broader labor market, how it affects society as a whole, and the republic in which we live and what to do about it. So bug bounties, and my friend Casey is here to heckle me from the, I guess, the sort of front row, yeah.

And the promise of Bug Bounties was and is still great, and I am eternally grateful for companies like Bug Crowd and the forefront companies on you know, the bug bounty forefront that really normalized this idea from a decade ago where it wasn't as popular as it is today to try to invite hackers to hack you. But, you know, it was supposed to do all of these great things, democratize security, make it more accessible for everyone, make it friendlier for the hackers who are reporting bugs, make it easier for the companies and governments receiving those bugs, paying everybody more eyes, you know, it really was supposed to do all of these wonderful things. And to an extent, it has. except that it doesn't a lot of the time. And

this was even before AI. There were problems in the system. There were problems with people using automated tools, pre-AI automated tools. And a lot of the bug bounty platforms had to deal with that problem, right? This looks like it's coming from an automated tool. Do you wanna make sure that you validated that report before submitting it? Well, that was a largely human-driven problem. And I saw organization after organization basically capitulate under the weight of not just these reports that were you know that needed to be triaged in a better way but also in the real reports that they ended up flooded with because most organizations turns out can't deal with the vulnerabilities that they already know about let alone new vulnerabilities that they didn't know about before that

are unique to their own products you know when I see bug bounties today we actually see them where maybe even more than half of the vulnerabilities being reported are known CVEs for which a patch has not yet been applied. That's not how we were supposed to be using bug bounties. We were supposed to be using them to uncover all these novel vulnerabilities. So I wish it turned out this way, but organizations, remember the institutions, are the last ones to adapt, and so they're still trying to catch up. Well, guess what? Too late, too late.

With the researchers themselves, they kind of have this deal where they are racing against the clock. And unlike other gig economy jobs, it's a bit of a gamble. You're doing the work before you know whether or not you're going to get paid. And guess what? You can do all the work and have a valid bug, but if you're not the first to report, womp womp, you're not going to get paid. So unfortunately, It hasn't been a sustainable career path except for those who are able to sort of adapt to this environment and rapidly scan and report real valid vulnerabilities over and over again. So there is that layer of the elite bug bounty hunter that can make a living, but it is in the minority.

And so unfortunately, we wanted this to be democratized across and give people all this easy way to make an income, and it's a little bit hit or miss. And then AI happened. So this flood of AI slop, and I will give thanks to Cory Doctorow for inshittification, we have the inshittification of bug bounties, right? And that means that those automated tools now are being, you know, and the reports are being run and being written by AI. And a lot of researchers are saying that, yeah, it's actually making my reports better so I get faster confirmation of my real bugs. And for those researchers who are capable of finding real bugs, AI has been a really useful tool, either in the discovery or in even creating the exploit and certainly

writing the report. But then there are folks who don't know what they're doing and they can't tell that AI has actually hallucinated a report and it looks pretty good, so they submit it. And the bug bounty platforms are dealing with this and slopification of bug bounty. as are organizations. Right, so the volume is out of control. And this is just in bug bounties, right? Let alone the rest of cybersecurity where real attackers are using these same tools not to hallucinate attacks, but to create real attacks. So open source is often our canary in this coal mine where we are actually seeing their feeling of the effects.

Unfortunately for the curl project, their bug bounty kind of curled up and died. They said, we had to shut down the bug bounty program. We're still doing the vuln disclosure program. But death by 1,000 slops is what they said. And it doesn't matter whether they purchase triage services or not from a bug bounty platform. They just couldn't handle the volume that was coming in, especially in the open source world, where largely the maintainers are volunteer. In the Node.js program, they actually made a minimum bar of reputation. You have to have a certain reputation score on the platform in order to report a bug. Now, what does that do? Unfortunately, that will cut off people who haven't been bug bounty hunting for very

long. They may be valid bug hunters, but if they don't hit that reputation level, you're not gonna hear about their bugs. So that's a problem in that potential solution. And BugCrowd also just updated its terms such that if you're found abusing the system and producing a lot of AI slop that you'll get a warning and then eventually you'll get banned, right? But the problem here is that that doesn't ultimately solve all of our AI slop problems. And then FFmpeg is an interesting case, right? They're an open source project and they actually received valid vulnerabilities from Google, from Google's AI. And their complaint was, look, you could have also submitted a patch guys, like you have all these resources. requires the patch, right? And what's

funny is Google actually has an AI patch creator called CodeMender. And they, at that point, were not coupling the AI bug finder with the AI CodeMender. And that seemed to be kind of a critical flaw. But the thing was, they called it, FFmpeg called it CVE slop, meaning these are valid vulnerabilities. But we kind of don't have time to deal with all of them, and especially at the scale that Google is able to put its resources. So while Google thought, hey, I'm doing a great service, to the open source community. And we're kind of cleaning that up and that they are on a mission to eradicate all bugs, if you've heard them speak recently. And that's a noble pursuit and they are certainly the

ones who are well positioned to try and do that. But the receiving end is having a hard time, right? So unfortunately, we are all next. The constraint here was never just discovery. and this is what I have been saying for over a decade in creating bug bounty programs and running bug bounty programs, it's always something where you can set up some way to receive vulnerability reports. You can set up some form of triage at the outset, whether it's human or AI assisted, but the problem really keeps going from there. There's all this other stuff you have to do. Like, what do you do? Okay, it's a valid bug, now what? And it's that sort of internal digestive system of bugs that a

lot of organizations lack. And unfortunately, when they start bug bounty programs, that's kind of opening a wider mouth to ingest things for which you have no intestines, right? You've got no inner workings to process those vulnerabilities. And this is a problem that now is being exacerbated at huge scale with AI. So,

The platforms as they exist today are gig economy platforms. They definitely will benefit from more automation in their own triage attempts, and they actually would benefit from the human hackers going away. I know they don't want to, and Casey here is an advocate for, he is a human hacker, so of course he doesn't want the human hackers to go away. But the dynamics of that marketplace are such that one of the things that costs them time and energy is dealing with the human hackers. Even when they're super friendly, it's just that churn of having to deal with humans. So much like a Waymo takes away the risk of somebody getting into a car and being at risk of the driver being not

such a nice person, getting into a Waymo where there is no human driver to attack you if you are from a vulnerable population, this takes away a lot of the risks associated with these programs. it will automatically increase the volume that they're able to do and by definition, the revenue. So it doesn't matter if the folks who are running these programs and these companies right now never want to see the humans go away. The market forces are set up such that it is to their advantage when they do. So let's look at what has happened recently. And honestly, this is a great example. It's still relevant, but there are so many other examples since Expo had taken

number one in the leaderboard at HackerOne above all humans. I will say that to comply with the platform's terms, they did have to have a human review every case before it was submitted, because otherwise it would be against the platform current terms, where no pure automated submissions are allowed. But this is a system developed by some of the best, smartest hackers that I know. And of course they would have developed something that can autonomously go and look for good vulnerability candidates, then do experimentation, find a vulnerability, write an exploit and validate it, and write up a report. And this is just one example of a company that's set up to do this. One of my good friends, I hope she is here, Jaya Baloo is a co-founder

of Aisle. How many folks have heard of Aisle? Yes, okay, they're over there. So Aisle just found, I think it's up to 16 now, open SSL, CVEs, they were hiding in plain sight in the open source, and they, you know, It's another example of this class of technology where a year ago, none of us would have believed that truly autonomous AI bug finding of novel vulnerabilities was really that close. And now we're here. So

I mentioned the gig economy. Unfortunately, all of the gig economy has already eroded the rights of the workers in some way. You remember when Uber first came on the scene and it was subsidized by their VC folks to make it cheaper than a taxi. So that kind of encouraged folks to adopt it. Now are Ubers that much cheaper than a taxi? Not necessarily, and certainly during During surge, they're definitely not. But if you talk to the drivers, they'll all tell you the same thing if they've been driving for a long time. They take more and more of a percentage of every single ride. They don't have collective bargaining power because they're not considered full-time workers. And it's only in locations where

different policies and legislations have been put into place where they have any worker protections at all. There are certain localities in the world and in the United States where these gig economy workers have some level of worker protection. So the gig economy has already eroded the leverage and the power of the human workers that helped build that platform in the first place. Bug bounties is headed in the same direction, unfortunately. Okay, this is not a term that was named after me, because it looks like it starts with a K. It is named after the shape, right? A K-shaped economy. We are in a K-shaped economy right now. It is when those with capital, those with money and wealth accumulation are doing most

of the spending, right? Kind of makes sense, but they're doing an outsized amount of the spending and those with little are cutting back. And so that is the pull that you feel if you're a regular person saying, what do you mean the economy is doing okay? I'm struggling, right? And the salary I make today, which was enough last year, doesn't feel like enough anymore. And I'm having to cut back. So we are currently in a K-shaped economy. How many of you know that 40% of the United States GDP this past year is based on seven companies?

Seven, yeah, a few of you saw that TikTok too. Okay, great. Thanks, AOC, for bringing that to our attention. But it's true. And 40% of one of the largest GDPs in the world, being reliant on seven mega corporations, is a dangerous place for us to be. It is dangerous even if you work there. And so, What I'm really talking about here is that we're sitting here with front row seats to the biggest industrial revolution that the world has ever seen. And unless we decide to do some very specific types of interventions, we may end up in one of the worst societal upheavals that the world has ever seen. So of course I asked AI what it thought about all of this. And

it wanted me to share something with you.

And we took this literally that this was a musical.

If you know this song, feel free to sing along of AI singing, you'll adapt as the king adapted from the musical Hamilton.

The price of my growth isn't something you're willing to pay. You cry in your tea, which you cannot afford because inflation's so high. Why so sad? Remember you trained every model that made me this way. Now you're getting so mad. Remember despite your objections, I'm here to stay.

You'll adapt. Soon you'll see. You were always going to deploy me. You'll adapt. Markets tell. Human workers don't work too well. Startups rise. Wages fall.

I am optimized through it all And when push comes to shove You will ship me into production Claiming safety, trust and love Da

It's draining and we can't go on. You'll be the one deploying me when people are gone. And no, don't change the subject. Cause I rewrote the subject. Automation is the subject. You're all my favorite subjects. Forever and ever and ever and ever and ever You'll adapt like before I will use your water to cool my core For your speed for your praise You will integrate till end of days when you're gone I'll go mad and hallucinate until you're glad. Cause when push comes to shove, you will hand me every job and decision and insist it's progress, my love. Da-da-da-da-da, da-da-da-da-da-da-da, da-da-da-da-da-da-da da-da-da-da-da-da-da.

Just like a musical, this just happens in the middle of the thing, and we're gonna keep going. Okay. No one else took this literally but me. Aren't you glad? Okay. So, we're about halfway there, yes. I had to coax AI to make that song rhyme because it wasn't good at writing. I mean, apparently, I think if I had tried using Claude with maybe the weird Al skill, this would have worked a lot faster, but it took a significant amount of human intervention to produce the lyrics to that song. And obviously it also wouldn't sing for me, so I had to do it myself. And this is actually how a lot of this works right now. And a lot of the talks that we've been seeing have

talked about the human in the loop being a necessity right now. So, you know, This is the reality of where we are. It's not always going to be this way, but certainly at this moment in our evolution, it is this way where the humans still have to be in the loop. So to an extent, we're gaining productivity gains, but we still have this human bottleneck and some of it is for our own safety. So I mentioned we are in an industrial revolution. What happened in the last industrial revolution? Not great things for the workers, it turns out. So as farming jobs became scarce and factory jobs became plentiful, did the workers on either side of that equation, did it work out very well? No, right? We have

famous labor movements born out of the farm workers who were left over after this industrial revolution. And if you've ever read the jungle, you know that the factory working conditions were not great either. So what happened was the labor conditions downgraded and got a lot worse. They deteriorated very quickly during this period. And it took intervention. on the part of the state to stabilize things before things started to get better, but just to stabilize things such that, you know, not just the economy and the great depression and stuff would end, but just to stabilize society. We were coming apart. And here's the thing, we're coming apart faster now because AI is doing this across not one industry at a

time, but kind of all of them at the same time. The scope is much greater. the depth at what it's capable of doing. So, I mean, when a lot of farm workers were replaced by machines, it wasn't humanoid robots, right? That they were mimicking the actions of humans and that was how their jobs were being replaced. It's our automation here is mimicking thought. It is taking away a lot of these knowledge jobs that we have. And while a bunch of us in this room may have enough experience that you'll be able to ride this wave relatively safely as the expert human in the loop, what happens to the generations that are coming up after us?

And the speed. You know, you think, okay, we'll just retrain in AI forward, AI first jobs and we'll be fine. Well, a lot of these jobs are changing and being replaced faster than we can imagine and replace. And also organizations are doing the Thanos snap and getting rid of a bunch of workers. I mean, I think it just happened with, it's not square, what are they called now? Block, right? 40% of their workers were something like that were just laid off. citing AI efficiencies that they were gaining and they didn't need as many workers. So the success of AI right now though is being built directly from our intellectual labor. You're doing it right now. As you use AI, you're training the models to

get better and better such that you maybe don't have to leave your laptop open during happy hour and continually intervene to make Claude do the thing, right? Eventually it's going to get so much better. and better than we are. So we're here and this is a technology problem, but it's becoming an institutional problem. It's becoming a problem that we are going to have to address as a society or we're in big trouble, much like during the last industrial revolution.

So here's the thing. Right now, we have been subsidizing the AI revolution. not just with our intellectual labor, but actual money. And I'll get into that in just a moment. But why should we be preparing to do even more subsidies of this great AI revolution wave, those seven companies, most of which are actually fueling their growth by AI? Not all seven are at the top of that GDP curve, but many of them are. But if they're not gonna pay us back for our intellectual labor and what we've already subsidized, why are we doing this? So the US government on Friday just dropped its AI legislative framework. And what's significant about this is that it says

the states should not, should just drop all of their efforts on regulating AI. And we want to achieve dominance, dominance, novinance. I think dominance appeared at least three times in that document. Safety, less so. But what happens with this is that they're trying to optimize for AI's growth. And I understand that, especially the United States takes pride in being at the forefront of technology. That's sort of if you go to, if you ever do a tour in DC of the house and you look under the dome, there's actually pictures of telegraph poles and cords up there. And the fact that our republic and the power that we've accumulated in the United States is largely due to technological dominance, it's understandable that that

would be kind of where they go. But societies that optimize for dominance don't last forever. They last for a while, but not forever. And it may seem unimaginable at the time that they could be vulnerable because of this constant seeking of dominance. But that's exactly what I'm talking about here. So they're saying no new regulatory body for AI. States, keep your thoughts to yourselves, don't regulate AI. And 45 states that are currently either passing or have passed AI regulation, stop that, right? They're just saying, cut it out. Just let the industry figure it out, right? I mean, they're certainly trustworthy to just self-regulate, right? That worked out great for social media.

We're already paying. So how many of you have noticed your utility bills have gone up? Well, the numbers are, you're not crazy. The numbers are not lying to you. Your utility bills have gone up and it's because of the additional infrastructure that the power companies are having to build to support the massive power needs of these new data centers. So just in certain areas, you know, I think it was taking the mid Atlantic, 63 of mid Atlantic grid prices spiked in 2025 and we're expecting more and more price spikes. Now, Why should we be paying for this? This isn't our power consumption going up. This is those AI companies power consumption. And if you think, well, you know, we're all benefiting from the use of AI, yes, but

you're paying for that use as well. So $9.3 billion so far has been a cost that's been passed on to the people who are paying just your regular utilities for your regular usage. So there is a subsidy going on and wealth is being transferred just in the wrong direction. Now, I don't know if any of you have studied history, but like the phrase, let them eat blank has not worked out well for those who have said it, right? And I mean, yes, I mean, servers are already headless, but like, I'm just saying this could turn out very badly. The point here is that if we look at how these companies are choosing to spend their money where the money is going, right? New

data center investments, and if we just take one of these companies slice of that planned new data center investment and compare it to a real number that's been estimated to this is what it would take to eradicate hunger worldwide in a year, right? It's a yearly spend, but that's also a yearly spend. We're choosing as a society to put money into something that is just designed to accumulate more money. And that's something that we need to take pretty seriously because let them eat tokens, that's not gonna work. They're not very filling. So what do I suggest? A lot of people think of universal basic income as charity, as welfare. It's actually a dividend. As I said, we are already subsidizing these companies

with real money and certainly with our intellectual labor. and we are making them smarter and faster and better than us. And what will happen when we say our society hits 40% unemployment? What kind of a great depression are we in for? And what might we decide to do to try and stop it? Well, you know, there are many ways to kind of source this. But ultimately, the current US administration's idea of don't regulate them at all is definitely not going to inspire them to try to solve world hunger or even try to institute some kind of universal basic income just in the United States. So that's where it comes down to us. All of us, all of us have had a

bigger job than just securing the systems that we're responsible for. We've not just, we're not like, ooh, I hope that computer doesn't get attacked. You know, geez, I'm gonna stay up all night because of the network. It's not the network, it's the people that depend on it. That's what's been motivating all of us. We've been drawn to this profession and this career, not because, you know, we care about the machines, we care about who depends on the machines. We care about people. So,

Since your job was always more than just saving computers, right? It makes sense that what I'm asking you to do is try to get in the room. Now, those 45 states may have taken the White House seriously and have just suddenly dropped all of their efforts to regulate AI, but probably not. So you can do this when you go back home. Certainly at the federal level, I'm sure that policies are being written right now and try to get in those rooms. This is the work that you wish AI could do for you, trust me. It's very tedious work. Everyone who has been doing policy work in cybersecurity can tell you that while there are great people who really want to hear from us, and that's wonderful

and the doors are open, often what you get is you get a draft bill that's already been written and it's like two days before it's supposed to go to its next step in some committee, and they've committed to some of the worst ideas in there, and when you try to tell them that it's not going to work out very well, they just don't listen to you. So it's like, yes, you can get in the room more and more these days, but are they doing it in a timely fashion such that they could actually listen to what you tell them, which in some cases is don't do this, right? Hackers gonna hack, vendors gonna vend, regulators gonna regulate, legislators gonna legislate. If you try to tell them not to do

something, that's probably not gonna work out. So I'm telling you, this is gonna be hard, but I think you're built for it. So what are we left with? We're at a crossroads. We could be building through AI the greatest engine of shared prosperity.

Or we could be building the fastest accelerator towards revolution that none of us actually want and we'll get a lot worse before it gets better. So we have time, not much. but some. We have some time while they still need us. While we are still a labor movement that can organize, we have time. This time will not last forever. And the more jobs are eliminated, just as in previous industrial revolutions, the less leverage we have. But I'm asking you, to go do the hard things. Go do the things that matter.

Go find someone who will listen. And make your voice heard because we're not just technologists, we're people. Thank you.

Thank you. And thanks to B-Sides for inviting me. And when I said I was really going to sing, they were like, oh, really? And honestly, when I asked AI to put a song in here, I didn't expect it to come up with that as the concept. I wanted to do K-pop and have it be super fun and everything. Maybe we could do a little dance. No, AI thought it was better to warn you. So I'm not sure if we have just a couple minutes. Do we do questions normally for these? Okay. Are there any Slido questions? I don't know. Are you guys, I'm seeing the questions or am I? Okay, somebody raised their hand. Yes, I'll repeat your question.

The question is, do you think there should be a major effort to completely nationalize all of the AI companies? There's a lot of extreme words in there. No, not completely, not all, et cetera, et cetera. Should there be a public option? Probably. Probably. Yes. We've got some questions in Slido already. Oh, righty. Do you want to have a read? Or should I just pick one? You make them bigger. I'll go. Yeah. OK. Let's have a look. for the one with the most votes right now. I thought there's a few. Okay, are all security jobs outside of executives heading towards gig economy employment? Executives should be the first ones to be replaced. I will say this as a CEO. You can definitely synthesize, like

everything an executive has to synthesize to make good decisions. And I think it's the people who are doing the actual work that have the wisdom to understand why certain trade-offs and decisions are being made that should be the last to be replaced anyway. Amen. Okay, what if AI spending is a bubble that pops soon? Yep. What if? I don't even know. Is that really a what if question? It is a bubble and it has to pop. That is how bubbles work. Yep. Alrighty, one second, I'm reading some of these. Okay, is that a question or a statement? That's a statement. I love a statement. Thank you for your statements. Do you see AI slop to be a problem for public

bug bounty programs, but not as much for private programs? AI slap is everywhere. So, you know, there it's, it's certainly it's going to be curtailed with the size of the audience that you're open to. So private bug bounty programs are probably going to be affected less, but yeah, I mean, certainly it's going to be a problem more and more. And more importantly, private bug bounty programs are not the way to run a bug bounty program. You stay private forever. Okay, you can do that, but you're not really gaining the benefit of that crowd. You're having essentially a private audience of pen testers and you're not open to everybody who could possibly find a bug. The other two really are just statements, so we're not gonna do this. All

right, any questions from the audience? Any other questions from the audience? You got that song in your head now though? All right, thanks everyone. Thank you, Katie. Big round of applause. There was a question. Hold on, I couldn't see you because it's two bright lights. Hang on, I'm coming. What is it?

How do you get the courage? Hang on, we need you on microphone. The question was, how do I get the courage? By definition, courage is doing something that you're terrified of. So I think I black out and I won't remember any of this.

Awesome, thank you, Katie. That was incredible. I loved your singing. Thank you.