Measuring Protection Efficacy in Real Time for Fun and Learning

But we designed this to be a workshop. So what we have is we have a little bit of an intro into it and then we have an opportunity where people could deploy some example software that we have here for efficacy testing. And it'll be very interactive. It didn't come through on the B Sides info, but we also share with you a set of labs that we're releasing today that are absolutely free for everybody to use for learning as well. So we're kind of keeping that theme open for learning and we definitely keep efficacy open as well as testing so you guys can see firsthand what's going on. The title was Measuring Protection Efficacy in Real Time for Fun and Learning. And why Fun and Learning because it didn't come from efficacy originally? So I wrote a whole tool called Cyfest that measures accuracy. And in security speak accuracy is referred to as efficiency.

And the reason I wrote Cyfest reached from more common tools was because some tools work with some malware samples. And we'll sort of iterate with you today how that reaches out to malware authors like in the case of Black Hat and Omigay. Black Hat reached out to the SEC or threatened to report victims to the SEC. So measuring efficacy involves real-world insights over a long period from multiple choke points. We're testing more methodically on choke points because detection is efficient to target and successes on future iterations. This moves understanding how sensors work for actual vs expected outcomes over time. Manual testing is hard and automation makes life easier. So we're going to demo why manual efficacy measurement is like my ball of wax. Let's see some results from the Cyfest test, which we are running safely in a local sandbox here to check detection. So you can see here, after the run, we dropped Black Hat on the disk. They went quickly and actually were blocked by our current settings immediately for that sample set.