Speaker 1: Welcome to the Cyber Risk Management Podcast. Our mission is to help executives thrive as cyber risk managers. Your hosts are Kip Boyle, virtual chief information security officer at Cyber Risk Opportunities, and Jake Bernstein, partner at the law firm of K&L Gates. Visit them at cr-map.com and klgates.com.

Jake: So Kip, what are we going to talk about today on episode 203 of the Cyber Risk Management Podcast?

Kip: Well, first, Jake, you have to fawn over the fact that we have gone over 200 episodes, right?

Jake: I forgot to fawn over that. I mean, I've been fawning over it for so long as the lead up and now I'm like, "Yeah, 203. Okay, it's cool. Starts with a two now. All right."

Kip: Well, good. I just wanted to give you a chance just to get your last horizon.

Jake: No, I'm good. 203.

Kip: Okay. 203.

Jake: I'll get excited again as we approach probably 250 and then 300.

Kip: Cool. All right. Well, today we are going to learn about how cyber risk can actually be quantified, which those of you who listened to the podcast for any length of time know that I'm a bit of a curmudgeon on that, but I still think it's a good idea if we can do it. And I'm happy to report that we've got a guest today who's going to help explain what's possible, but we're also going to look at why all cyber data vendors are not equal and whether cyber insurance is worth the investment.

And this is going to make sense to you, because we're going to do this with our guest, his name is Scott Stransky and he leads the Cyber Risk Intelligence Center at Marsh, which is the new name for Marsh McLennan. That was the old name. Now, what's great about Scott is his position gives him a rare view. He can see both the data and the dollars behind the cyber risk, which is fantastic. Scott, welcome to the podcast.

Scott: That is a rare view.

Kip: It is. I'm so glad you're our guest. We'd love to hear a little bit more about you and your work, so please introduce yourself.

Scott: Yeah, thank you for having me, and congratulations on getting to 200 episodes.

Kip: Thanks.

Scott: I'm very excited to be here and chat with both of you. My background, I have an undergraduate degree in math and computer science, and then I got a master's degree in atmospheric science, which doesn't sound at all like cyber. So my first job out of school was not related to cyber, but it was related to modeling and quantifying risk. And I was quantifying the risk from hurricanes. And then I did other types of natural disasters, like thunderstorms and wildfire risk.

Jake: Tornadoes?

Scott: Yeah, tornadoes, all that type of stuff. But in 2013, we had the Target breach, as I'm sure everybody knows about. And that breach really got us thinking that a lot of the same mathematical tools and technologies that we use to quantify natural disasters could actually be applied to quantify cyber risk. So we started to take a look at that and we built a few generations of models. Then I came to Marsh about four and a half years ago, and we've built out what we call our Cyber Risk Intelligence Center. And fundamentally, my team's goal is to quantify cyber risk, to make business people understand the cost of cyber risk. And we do this using real data from cyber insurance claims. We use real data from lots of sources that I'm sure we'll get into in this discussion today.

Just who is Marsh? Some of you may not know who Marsh is. Marsh is a very large professional services firm. We have lots of different angles to what we do, but the main thing that I will be talking about is our insurance brokerage practice. Basically, we help companies get cyber insurance, and there's a lot of caveats and quirks and fun things related to that. You may not think insurance is fun now, but hopefully by the end of this, you will. And our job though is to help our clients, advise them to understand the risk and then help mitigate the risk, whether that's through cybersecurity techniques or purchasing cyber insurance, a combination of the both and beyond.

Kip: Scott, that's a pretty tall order. Did you love insurance?

Jake: I mean, that's a really tall order.

Kip: I mean, we could shoot for that, but just don't be disappointed, okay?

Scott: No, that's okay. It's okay.

Kip: I worked in insurance. So for seven years, I was the CSO for PEMCO Insurance, which is a private line or personal lines property and casualty insurance company based in Seattle. And when I went to work there in the beginning, I was like, "I can spell insurance. I know how to send them money, but I didn't know anything else." Now I know so much about insurance that you're like, "I've got to keep my mouth shut at parties, because nobody else wants to hear me talk about it." But anyway, it's fascinating. It is.

Jake: Well, and I mean the entire world of insurance, I mean it's about actuarial science. It's about trying to understand risk in ways that are not just abstract. And Kip, you and I have for years at this point gone back and forth. Actually, that's probably not fair. We've largely been on the side of more qualitative risk analysis in the context of cyber risk management from a kind of practitioner company level standpoint.

Kip: How to make it practical.

Jake: How to make it practical.

Kip: And affordable.

Jake: You can't do that. You can't really do that with an insurance model. So Scott, I'm really curious. Take us through some of this.

Scott: Yeah.

Jake: Yeah.

Scott: So it all starts with data. We can't build models unless we have good data. So it really starts with underlying data. And people think, "Oh, there's no data available for cyber. It's all very sensitive and undercover." It's really not. I will say the data is maybe not as good as the data we have on hurricanes. I mentioned my background is modeling natural disasters. If you think about hurricanes, you could go to the National Hurricane Center website, download hurricane tracks going back to 1851 that are very accurate for basically every hurricane that's ever existed. There's no such single database for cyber. You can't just go to the cyber data center and download cyber data. It doesn't exist.

So it's up to us to actually source the appropriate cyber data. And it starts with what we call firmographics. And firmographics are the basics of an organization. They're things like your company size, your industry, your revenue, your employee count, even your geographic distribution. People say cyber is not bound by geography, but there are definitely geographic impacts from cyber events. There's a lot of different regulations in different places, I'm sure you know. GDPR in Europe, privacy regulations in Illinois, also two different things, that if the breach happens at a certain place, it could be a different in terms of the consequence. So we do model the geography. So that's the firmographics.

Then we get to incident data, and incident data is really the big challenge when it comes to cyber risk modeling, but it's also where cyber insurance has a huge advantage. If you think of a CISO, they may know the incidents their own companies had. Maybe they talked to some of their peers, direct-to-industry peers or other folks they see at conferences. They may know some things about what has happened, but they don't have a broad view on cyber incidents across the world. On the other hand- Yeah, go ahead.

Jake: And just sorry to interrupt, but I just think it's worth pointing out too, that even a well-connected CISO who works for a big company may have direct firsthand knowledge of a couple, three, four incidents, and then secondhand knowledge from his own personal network of maybe 10, 20, even 30 incidents. That's not a lot of... That's nothing.

Scott: It's not. So that's where insurance comes into play. So there are ways to get incidents without insurance data. There's ways you can do web scraping of news articles and SEC filings and things. It's very biased though. You're not going to have an SEC filing if you're not a publicly traded company. You're not going to publish something on the internet unless you have a mandatory requirement to notify people. So that data is really biased, but again, insurance data is the key.

So for companies who purchase cyber insurance, the hypothesis, which I think is a reasonable one, is that if you suffer an incident, you're going to want to collect on that payment, because you've been paying into insurance, you're going to want to actually get the claim. So our assumption, which we think is a reasonable one, is that if you have cyber insurance and you have a claim, we know about it. So while the universe of companies that purchase cyber insurance is certainly smaller than "all the companies out there", for the companies that do purchase it, we have an unbiased data set. And that's really the key advantage of using cyber insurance data to start to think about quantifying cyber risk.

Kip: That's why I said when I introduced you that you're in a unique situation, because I don't have access to that data.

Scott: No, no, no. It's very sensitive data, as you can imagine, because these events have not made it into the public sphere. Our clients hold us in very deep confidence to keep their information private and only share it with those who are absolutely necessary to see it. But one of the things that we can do with that data is analytics and modeling, because it helps our clients in the end. Taking data from tens of thousands of clients, building models with that data then helps those tens of thousands of clients make better decisions moving forward. So they are willing to let us use their data in modeling as long as we don't reveal any of that raw data back out to the world.

The way that I like to think of it, the analogy I like to think of it is if you're making a model, think of it like a stew. When you cook a stew, maybe you add some salt to the stew. The salt makes the stew taste really good, but you can't actually extract the salt from the stew. So once the salt is stirred into the stew, the claims, so to speak, in this case, you can't get a single claim out. The salt is in there. It makes the stew taste good, but it's not like, "Oh, there's the claim from this company or there's the claim from that." You can't do that. So it actually protects our client's data at the same time as actually helping them.

So that's where we think about incident data, but then there's a lot of technographics as well. Now, technographics is the parallel to the firmographics, which I talked about before. And technographics are the cybersecurity measures that our clients have in place. And there are several ways to get these as well. The first way is with outside-in scans. There are several vendors out there, and we'll talk more about vendors in a bit who do outside-in scans. These are folks who are looking at things they could see on the public part of the internet about a company. They're usually not obtrusive scans. They're usually not going inside the network. You can find out some information about a company with those scans. But then there's inside scans, and you could do that with a piece of software or hardware you place into a company, or in our case, we use insurance applications. Our insurance application, which we call our Cyber Self-Assessment, it's a very lengthy questionnaire. I'm sure there are CISOs listening to this podcast who have filled out a cyber insurance application. We have.

Kip: Yep.

Scott: It's lengthy. We get it.

Kip: I've helped people do it many, many times.

Scott: But it does add a lot of value, and I'll explain why. There are several hundred questions in this, everything from passwords to training, to EDR, to antivirus, firewall, you name it, very broad. But what those can then be used for is correlating with claims. So if we have these answers for thousands of our clients, and then we know which of those clients end up suffering a claim over the next year, we can start to do correlation exercises to figure out which questions actually have the most correlation with risk. And again, this is a self-feedback process. It's a very positive process that over time we build up more and more information, more and more data, which we can then tell our clients, "Well, we've determined that these 10 or these 30 questions out of the hundreds out there are really the most important."

And this allows them not just the question, but what it's actually about. So is training more important or is EDR more important? If you have a limited cyber budget, you may need to make a decision like that. And using the data that we have, we can start to make better decisions.

Kip: And that because cyber is a dynamic risk, once you think you've got it dialed in, you don't.

Scott: That's definitely true. We update our models and our analysis every year, because there is plenty of new data, both from those self-assessments and then also the clicks.

Kip: And new attacks, right?

Scott: New attacks.

Jake: And I think there's a couple of things that stand out about this technique. And it's interesting, right, Kip? Because you and I, when we first started doing this podcast way back in the depths of time, we would see one-page insurance quality questionnaires with five questions on it and we would ask the insurance, usually it was the brokers who we might be doing a shared presentation with, we would ask them, "Why don't you ask more in depth questions?" "It doesn't matter."

Back then, but things have changed a lot. And the other thing I wanted to point out was just how this is empirical data. This is as close to a true science as we can get. And I think I want to differentiate that from the kind of theory crafting that a lot of cyber risk managers ourselves included are excited and inclined to do. And as much as it can be fun to engage in theory crafting, it doesn't have the same impact as empirical data of the sort that Scott is telling us about. And I think that for people who think, "Oh, what do insurance companies know about cyber risk?" Well, they know the actual data.

Kip: It's getting better, right?

Jake: Yes.

Kip: 10 years ago, 15 years ago, I would say they didn't know much at all.

Jake: Five years ago, I'm not sure they had this, they didn't have this level of information.

Kip: No, and every year it's going to get better.

Jake: Yeah.

Scott: And to your point, those questionnaires were very short a while ago and we didn't get very much data. And I think that was because cyber insurance was just getting really into its heating.

Jake: It was. Yes.

Scott: And basically every company was like, "Here's a cyber insurance policy, here's a cyber insurance policy."

Jake: And then you started paying out.

Scott: And then there were some payouts, especially as ransomware took off in 2020, 2021, and then people realized they needed to be a little bit more diligent about how they wrote cyber insurance policies. But that's great for me. As somebody who cares about data, now we get a lot more data on these policies, which allows us to do these analyses that we're talking about here.

Kip: And the more data, the better for you, right?

Scott: Of course. Yeah, so then what do we do with all of this data? Well, we put them into models. And what the model does is it takes all this raw data from firmographics, incident data, technographics, all of these different places, and it puts it together into a cohesive view of risk. And we can model single companies and we can also model systemic issues. So why would we do that? Well, for single companies, if we have a single company client, they want to know what their risk is, their risk from things like a privacy breach attack, from a business interruption, from a ransomware attack, on and on and on. But then there's also systemic risk, that this is what the insurance carriers care about.

Now, we as a broker, we're passing risks to many different insurance carriers and these carriers are trying to understand, if we have a portfolio of 10,000 risks, what is the chance that something happens that impacts all of these risks or a good portion of them at once? So because we know things like which cloud providers are used or which DNS providers or which payment processors or which CDN, all of these different hosts, all these different aspects, it allows them to understand if that provider goes down, how much of my portfolio will go down with it and how much might that cost me? And that's something that the insurance carriers need to know. So when we bring them a new risk, maybe it actually helps them diversify their portfolio, which is something we can tell them.

Kip: Oh, I get really excited about the systemic piece, because I like to compare the evolution of cyber insurance to fire insurance. Because back when fire insurance first started, one of the systemic risks was that in the Boston metropolitan area, for example, there were many different types of building sprinkler systems and they were not interoperable. And so, people maintaining those sprinkler systems had to be trained how to maintain each one. They obviously weren't. And so, a lot of maintenance issues arose that prevented the systems from working correctly. And so, that caused the insurance companies to pay out a lot of money. So they got together and said, "We're not doing this anymore. We're going to form a consortium and we're going to standardize these sprinkler systems so that we don't have that problem."

Now, that seems to me like what you're talking about is a systemic risk that only the insurance companies can really see and that they are incentivized to do something about it. Is that right?

Scott: Yeah. So systemic risk is definitely what keeps people up at night in the insurance world. I would say an individual CISO may not care so much about it, but the insurers definitely care about it. And if you can convince an insurer that you're helping diversify their book, that gives you an advantage, because they're more likely to incorporate you into their portfolio of risk. Whereas if you're just piling onto the cloud that they already have a lot of risk on, maybe they're not going to be as excited about your new risk. But yeah, systemic risk is a big deal.

And I'll take you back actually to hurricane modeling, which is where I got my start. And hurricanes have always happened. People have started modeling hurricanes back in the 1980s. So it's a relatively new thing, even for hurricane modeling, which hurricanes have existed for all eternity effectively. But before the 1980s, they were writing insurance risk for hurricane. They were writing the insurance. They were just using rules of thumb. They would just say, "Okay, I think about 8% of my properties are going to be destroyed by hurricanes each year, so we're going to charge a little bit more than that to make a little profit and hope for the best."

And in most years, it worked out okay. In most years, they were able to pay out the claims that happened, but occasionally there was a very big storm and that very big storm maybe made some insurers go out of business. And it got to the point where in the late 1980s, they decided to build some mathematical models. And these mathematical models, as I said at the beginning, have very fundamentally similar mathematical techniques to the cyber models that we're talking about mainly here. They're what we call Monte Carlo simulation models. If you're not familiar with the Monte Carlo simulation, what we do is we simulate what could happen next year, hundreds of thousands of times.

And in the case of cyber, most of those simulations will not have any event. Most companies are not suffering a major insurance loss each year, so most years will have none. But some years will have one, two, maybe even three events that could be very, very costly to the business, especially if they don't have a proper cyber insurance policy and they may be on the hook for the whole loss themselves. But this is what happened back in the 1980s. They built these models for hurricanes. And at first, people didn't believe the losses. They thought they were too high and they didn't really rely upon them. But then Hurricane Andrew happened in 1992 and the losses from the model were spot on with the actual claims that came in from Hurricane Andrew.

Kip: I didn't even know that.

Scott: It was about 18 billion US dollars back then.

Kip: Well, I remember it just destroyed with a vengeance. I mean, it was just like a finger went across the state of Florida and just destroyed everything.

Scott: And I'll tell you, I would not be here talking to you about cyber modeling if the model estimate for Hurricane Andrew hadn't been so accurate. Because as I said before, that people kind of laughed at the models, they didn't trust them. But when they saw how spot on the Hurricane Andrew estimate was, that's when they really decided models are useful, we're going to use these things. 11 insurers went out of business during Hurricane Andrew because they did not use models. They were not prepared for such a large loss. But since then, yeah, their models have existed for lots of other natural perils and now for cyber. But fundamentally, these models are simulating next year lots and lots of times what could happen to your organization.

Kip: Fascinating. Fascinating.

Scott: Really fascinating.

Kip: You did some data, you were doing some data science before you became an attorney, right?

Jake: No.

Kip: No? Oh, okay. All right. Well, I remember you were in science, so...

Jake: I was a cell and molecular biologist, and so there was data and there was science involved, but-

Kip: But not like this?

Jake: Yeah. Not to the same degree. And the other thing, where I was going to go with this kind of comparison between hurricanes and cyber, which I think is so relevant, is I think insurance companies need to also be able to handle hurricane level cyber events like when major hub style companies go down. A good example being the CrowdStrike issue from last year, or was it two years ago, may have been two years ago. And the parallels between a hurricane and those types of core cloud service provider outages, or Kip, the NotPetya attack that basically was a cyber weapon with a huge amount of collateral damage, those are much more similar to hurricanes than maybe insurance against severe thunderstorms with tornadoes in the Midwest, where a tornado can be hugely destructive, but it's super random and in a relatively tiny physical area. Those are much more... And I'm making this up as I go, but it's making more sense.

Scott: No, no, you're spot on. You're spot on.

Kip: I'm agreeing with Jake because Scott agrees with Jake.

Jake: Those tornadoes are more akin to kind of the everyday opportunistic cyber attacks, but I feel like there does need to be recognition that there are hurricane level cyber attacks. So you already said I'm spot on, I guess that does make sense.

Scott: Yeah, so I don't think we've seen the major cat events. We call them cats, catastrophe events. We've seen some, I don't know, kitty cats, like small cats, but we haven't seen any major cats from an insurance perspective. Even some of the events that you had mentioned, they cause some insurance loss, but many insurance policies will have what we call a waiting period. If you're familiar with deductibles on car insurance, property insurance, a waiting period is effectively a deductible expressed in hours, and these waiting periods may be eight hours or 10 hours or 12 hours. So you need that much downtime before the insurance policy would end up paying. And most of these events have been less than the waiting period, which we could have debates about why that is, but in general, they've been less in the waiting period, so the insurance world has lucked out. If the events had been slightly longer, the insurance world would not have been as lucky and we would've seen a major cat instead of just a kitty cat.

Jake: Well, and I would imagine just based on timing that the NotPetya attack, which was widespread, was probably before widespread cyber insurance.

Scott: Cyber insurance has really taken off the past few years. Cyber insurance did exist back in 2017, but it wasn't-

Jake: Not to the same levels.

Kip: No. It wasn't as comprehensive.

Scott: So again, hopefully CISOs understand that cyber insurance does pay out and we'll talk more about that. So why didn't it pay out for the... Well, it's because you have these deductibles and waiting periods built into your contract. You need to understand what you're buying when you buy cyber insurance. And we as a broker will help you understand what you're buying. Of course, you can get a lower waiting period, it'll cost you a little more in premium. Or you get a higher waiting period, which will cost you a little less. There's different ways you can go about cyber insurance, but waiting period is a very key thing when you think about business interruption from cyber events.

Kip: Right. And a lot of companies didn't have cyber insurance at all and tried to file claims under property insurance and other things. And I think, was it Merck? Had a huge lawsuit that dragged on for years until they got some kind of a settlement, but-

Jake: That was about warfare. That was about the warfare issue.

Kip: Right, because there was an active war clause that the insurance carrier said, "We don't have to pay this, act of war." Anyway, but I think that's just part of the maturation of this insurance policy, right, Scott?

Scott: Yeah. I mean, the cyber insurance world is much more mature today than it was five, 10, 15 years ago.

Kip: Yeah, I think that was 2015 is when that happened. Okay. So vendors, there's a lot of vendors who say, "Hey, we can assess cyber risk." But Scott, you've said, "They're not all equal." Okay, well, if I'm trying to make a smart decision, how in the world do I tell the difference?

Scott: Yeah, so I spend a lot of time with vendors and I'm going to put this offer out here on your podcast. If you're a vendor who has some sort of cyber data and wants to put that cyber data to the test, talk to me. But you actually have to be willing to share data. And in many cases, we talk to vendors who are like, "I have 3,000 years of cyber experience across my team. You don't need to data test, you just have to trust me." And I'm like, "No, I don't trust you." So if you're willing to share your data, then we can talk.

But there's a few things that I want to mention here. First of all, we are what I would call data-greedy, but data-skeptical. So we love data. The more data that we have, the better, but I'm very skeptical about new data sources until we actually test them out. And we've done a lot of tests with a lot of vendors. Some are quite good. Some are not so good. The first red flag I always see is a data vendor who comes to me and says, "Their data is 90% accurate." That sounds good. From a marketing perspective, the marketing team probably signed off and 90% sounds great. To me, that's actually a big red flag. Why? Well, we can build a model right here in the next minute, less than a minute that's 95% accurate by doing almost nothing.

How do we do that? Well, I'll tell you, we have in our dataset, about 5% of our clients get breached each year. So if you build a model that always says, "No, I will not have a breach," you're going to be 95% accurate. If 5% they get breached, you're going to be wrong.

Kip: Sure.

Scott: But you're going to be 95% accurate. You're also completely useless. And hopefully that's going to-

Jake: That does make sense. I like it.

Scott: It's a silly example, but hopefully it kind of rings true. It's like 90% doesn't really help.

Kip: That's just math.

Scott: Yeah, it is. Yeah, so again, we can build this 95% accurate model that's totally useless. It actually reminds me of one of my favorite quotes by a famous statistician, George Box, who said that, "All models are wrong, but some are useful."

Jake: Some are useful. I love that. That's a great quote.

Scott: He said, "This model is wrong and it's not useful."

Jake: On that logic, I can build a 100% accurate model right now. Any given company will or will not be subject to a cyber attack.

Scott: There you go.

Jake: I'm done.

Scott: There you go. Yeah, so that's a huge red flag. But let's say a vendor is willing to share their data, they don't give me their market... I hate marketing pitches. I really want a data pitch. How do you get this data? Where is it from? What do you think it's going to do? That's what I want to hear. I don't want to see beautiful slides. I don't want-

Jake: Unless those beautiful slides are about data, then maybe it's okay?

Scott: Sure. Fine.

Kip: But we like black and white slides.

Scott: But what we do is we get data from these vendors on a set of companies that we have, a portfolio of companies, it's in the tens of thousands, and we ask the vendor to send us data going back about six years. Why six years? Well, we don't want today's data when the claims or the breaches actually happened two years ago or three years ago. We want to know what the vendor would've seen leading up to that breach. So we want to know if the claim happened in 2022, what they saw in the 12 months before that claim. We don't want today's scores.

So assuming the vendor is willing and able, many vendors don't save their historical data, which boggles my mind, but assuming they're willing and able to share this type of data with us, we then perform a correlation exercise to correlate whatever they send us, whatever scores or metrics they send us with whether a company has a breach or not. And without naming vendor names or shaming vendors, some of them are really good, others are not so good. Others were like, "This is garbage. This is snake oil. This is completely useless."

And the ones that are good, we basically have strict criteria on how we include them in our model. To be included in our model, you certainly have to be correlated in the right direction. We've seen vendors whose data has a strong correlation in a counterintuitive direction, meaning a higher score actually leads to more breaches, which doesn't make any sense, and we don't use that data. So it has to be correlated in the right direction. And secondly, it has to add statistical lift or predictive power on top of our existing model. Meaning if you have a really good dataset that's correlated the right way and it's predictive, whatnot, but it doesn't add on top of what we already have in the model, it's great, we love you, but we're not going to include you in the model. So you have to meet both of those criteria to be included in the model.

So all those vendors that I said to contact me, it's very tough to get into our model. I'm not saying it's easy. We only have about a handful of these outside vendors actually inside our model. And we talk to dozens of them. Dozens a month even. We talk to a lot of vendors, very few make it into the model. But-

Kip: Oh, boy, I have so many questions about this.

Scott: Yeah, go ahead. I'm happy to dig into it.

Kip: So if I'm a vendor, what is in it for me to show you my data and get into your model? Does that help me at all?

Scott: Well, I mean if your data is in our model, we're paying you for that data. So it gives you some revenue. If your data is not in the model, oh, well.

Kip: Yeah. Can I tell people my data is in the model?

Scott: If you meet that bar, we will license your data. It's simple as that.

Kip: And can I tell people that you're ingesting my data to sort of help my credibility?

Scott: I think every vendor that's become part of our model, we've publicized papers with them. So because they're public, I can name some of them. We've done one with BitSight, we've done one with SecurityScorecard, we've done one with a dark web vendor called Searchlight Cyber. Again, these are all public papers. You can Google them or click some links that we'll provide to access those papers.

Kip: Yeah, I'd love to put those in the show notes.

Scott: Yeah, sure. The idea being that if your data is good, we don't just want to put it in the model behind the scenes, we want to make sure people understand how we did this study, what type of data we use to do this study, and then show the results of the study, show the correlation itself.

Kip: So my next question is, Jake and I love the Verizon Data Breach Incident Report. Have you looked at their data? Do you have an opinion that you can express now about it?

Scott: So we have our own data. Honestly, we use our own data. We think our data is going to be less biased than non-insurance data, as I mentioned at the beginning. So we really do use our own data.

Jake: Yeah. And that would strike me as one where the value of the DBIR is different than the insurance data. By definition, the DBIR only has bad things. They don't have any information on all the companies who don't call for incident response services. Whereas the insurance companies, I mean, 95% don't have instances.

Kip: Jake and I have been using the DBIR, because it seems like it's the best thing available to us of all the other choices, so...

Scott: As far as public reports, I mean I can't complain about it. We're not going to say, "Don't release this." No, it has value.

Jake: I think it's good for trends. I think it's good to understand attacker trends, technological trends. And actually, they go out of their way to say repeatedly that you can't just look at this data and determine what your quantified risk is. They say that all the time. They've got all those percentages throughout there and they're like, "Yeah, this is what this data shows. It's for trends. It is not quantifiable data." And I think that's fair.

Kip: And I think that's fair too. And I'm glad that we surfaced that, because we do on this podcast spend a lot of time talking about the DBIR. I just wanted to make sure that it was clear, "Okay, well, how's that data different than the data Scott is talking about?" And so on and so forth.

Scott: Our data is very... It's different. Insurance loss data is going to be different than that.

Kip: Of course. Now, we're already over 30 minutes on this episode. And I always get people asking me, "Do we have enough data for an episode or do we have enough talking points?" Yes, we always do, because we always talk.

Jake: Nor are we quite done, Kip, because we're averaging closer to 40 and I have more questions. I want to know about this argument that cyber insurance creates "moral hazard", right? The companies may actually invest less in security if they know they're covered. And the reason I'm curious how this has changed, because Kip, I think it was a real problem for much of the last decade of cyber insurance being commonly available. As Scott said, at first-

Kip: It was easy and cheap.

Jake: Insurers were just like, "And you get one and you get a cyber policy and you get a cyber policy. Everybody gets a cyber policy."

Kip: That's right. I remember that.

Jake: And so, when we were telling people to do, when we were giving cyber risk management advice, it would be like, "Well, do you have cyber insurance? If not, why not? Go get it. It's a no-brainer." And I mean, Kip, this would frustrate us to... I mean, as useful as it would be to tell clients to get cyber insurance, it was also frustrating because we saw the insurers of the time not doing anything to try to actually manage the risk. They would just write the policy. And I think... I mean, not I don't think, it is clear that has changed. So I'm curious, Scott, what you think about that. What are the tools that insurance companies, insurers use to make sure that this doesn't happen?

Scott: No, it's a great question. We have a lot of our clients who are very skeptical about cyber insurance at first, but there's several myths that we try to bust. The first myth is that insurers kind of are your adversary. That's not the case. The bad actors are your adversary. The cyber attackers are your adversary, not the insurer. The insurer really wants you to do better. They want you to be mitigating your risk. They want you to have lower risk, because then they're not going to pay out as many claims. So the insurer will actually incentivize you to do good things. What do I mean by good things? Well, if you don't have MFA, get MFA. The insurer will give you incentives to do that. In some cases, they'll help you, or in some cases you won't even get insurance unless you have MFA. So there are things like that where the insurer works with you, not against you. The insurer really is on the same side as you.

Another common thing that we hear is, "Oh, if I buy cyber insurance, the bad actors are going to know this and they're more likely to attack me because they think, oh, I'm going to pay the ransom," or whatever it is. So we actually did a study on this, and it's going to be coming out very shortly, and we use data to say, "Okay, what is the probability you have an attack given you have cyber insurance versus given that you don't have cyber insurance? And how does it change after you purchase cyber insurance?"

And it turns out that there's no statistically significant difference between companies who purchase cyber insurance and those who don't in terms of frequency of events. So we can alleviate some of those fears that companies have that, "Oh, I'm purchasing cyber insurance, now I'm at more risk." You're not at more risk, for various reasons. The bad actors probably don't know who has cyber insurance fundamentally, but even if they did, they're not specifically targeting those that have cyber insurance, which is great. But having cyber insurance is part of the strategy. It's not the whole strategy either. And we see this, because you can have all the best cyber controls, have the best CISO, the best team, the best firewall, MFA, antivirus, training, EDR, all of these different things, you still have some risk. There's no zero risk cyber. There's no zero cyber risk company out there. It doesn't exist.

Kip: Especially because cyber is dynamic, right? You can have everything sorted out based on what you know you need to deal with today, but tomorrow that's going to change and then you've got something new. That's the whole idea of zero-day vulnerabilities and so forth. So, Scott, I think that the insurance companies are going to take a leading role in the cybersecurity profession. Today, I've got people saying, "Well, hey, Kip, should we do the Center for Internet Security top 18? Is that what we should do?" Or I might say, "Well, I think you guys should adopt the Essential Eight from the Australian Signals Directorate." So we've got these checklists. People always want to know what's the checklist. But I think that in the future, and I don't know how far this is, so I'm asking you what you think, I think insurance companies and their sense of what baseline controls should be is going to be way more reliable and way more effective than what we've been using to this point. What do you think about that?

Scott: I agree, because it's based on real data. Obviously, CISOs have a lot of expertise hands-on, but as we talked about at the beginning, they know about their own risk and maybe some of their close peers. They don't know the broad swath of all companies. And the insurance world has that data. So one of the things I'll also send for you to put in the show notes is a paper we studied where we took that questionnaire, we divide it into 12 categories. So I know you mentioned 18 categories and eight categories, we have 12 categories. So we took all the questionnaire data and put it into 12 categories and basically rank ordered each of these 12 categories, which are the highest correlated with risk all the way down to the lowest.

And within each category you can see that there's a range. Some categories have a wide dispersion. For example, as you'll see, if you click the link, the training category. Some things you could do related to training are really, really critical and will reduce your risk a lot. Other things are maybe important, but not nearly as important. And I'll say one of those which may not be intuitive to people, the frequency of mandatory training, it's actually not that important. What is much more important is that the training is good and effectual and trains for the right things. That's much more important than just, "Oh, we're going to train you every month or every quarter."

Kip: Right, right. Oh, that's fabulous.

Scott: So things that we can pull out of the data and help our clients, our CISO clients understand, "Okay, I only have a limited budget. What should I be investing in? Should I come up with-"

Kip: And it's the quintessential problem that CISOs have to solve, which is I have infinite risk coming at me and I have limited resources to deal with it. So how do I reconcile that problem? And I want to tell one quick story before we wrap up. I was helping a customer get their cyber insurance policy for the first time. We were going through this AIG questionnaire, which is enormous, probably the biggest one out there that I've ever seen. And they asked about how many service accounts have administrative privilege. And this particular customer I was working with had something like 60 and they got bounced. They couldn't get a policy with 60. It had to go down. Way, way, way down.

And so, I called up the underwriter and I said, "I have never seen this as an issue. Why is this so important to you?" And then they explained that every single ransomware claim that they had handled for the past three years included the case where a administrative account was hijacked and many of them were system accounts, because nobody was monitoring system accounts that had administrative privileges. And that's when the light bulb went on for me and I was like, "Holy crap, these insurance companies are going to figure this out and they're betting billions of dollars that they're going to figure it out." So what do you think about that, Scott?

Scott: No, you're absolutely right. The insurance world are making very large financial decisions based on cybersecurity tools, techniques, practices, and beyond. And that's why, again, as I said, hopefully we'll convince you that cyber insurance is actually very fascinating. It really is. We have very unique datasets in our industry and we've deployed those data sets to really help CISOs, risk managers, companies, C-suites make better data-driven decisions about what to do to improve their cybersecurity. That's really fundamentally what we do.

Kip: I don't think we've convinced anybody to love it, but I think everybody can agree that it's fascinating.

Jake: Well, and I want to end with maybe a little bit of pushback on this, only because Kip, what does fire not do?

Kip: Oh, let's see. What is it? Starts with high-

Jake: Fire doesn't innovate. And what I'm thinking, what I'm realizing is that, what are the classic things that we insure against routinely? Weather, floods, health problems, car accidents. And while some of those are under the control of individual humans like car accident, behavior, things like that, I think there is a core difference between cyber risk and many, if not most other forms of insurable risk. And what made me realize this was the concept in geopolitical and war strategy, which is, everybody always prepares for the last war. And they never see the innovations of the next war coming until it's right in front of them.

Everybody knew for a fact that Ukraine would fall in three days, that Russian tank columns would be inside of Kyiv and it would be over. And it didn't happen and it hasn't happened. Nobody saw that inexpensive drones with bombs on them would destroy literally tens of thousands of Russian armored vehicles. I don't want to say that it's a skepticism of insurance data, but it is the same basic problem, which is by definition when you are dealing with cyber attacks, you are fighting, you are preparing for yesterday's war and you can't predict tomorrow's war. And so, I think my point is that cyber insurance data is going to be a critical component of cyber risk management, but I also don't... It can't end there, right?

Scott: This is one tool in the tool shed.

Jake: Yeah, it's one tool. Yeah.

Scott: But I'll just respond a little bit. I know that we're already over time, but I'll just take one minute and respond to what you said.

Kip: Okay.

Scott: What I will say is that we're modeling outcomes more than things that happen. So we don't necessarily care if it's a hacktivist or a state actor, this or that. In the end, we're modeling things like, okay, I lost some number of records, here's how much it'll cost the business. Or I'm down for some amount of time, here's how much it costs the business. It doesn't necessarily matter how the cyber actor took you down. There could be a new innovation that takes you down a different way, but you're still down. That's what's going to cost the business money and that's what the CISO is trying to prevent. Some of the ransomware extortion payments, things like that. We're modeling much more on the outcome side than we are modeling the root cause. So it's not like there's a new hurricane invented. We're modeling the damage to the building. Here, yes, there could be a new virus invented, a new worm, a new this or that, but we think it's going to cause maybe records to be lost or stolen or a downtime or an extortion or something along those lines.

Jake: I mean, that's really interesting, because I think that is both the value and the limitations of cyber insurance data, is that if we reduce cyber attacks to the outcomes, then there aren't that many, right? There really aren't that many. Downtime-

Kip: And they don't change that much.

Jake: Those don't change that much, but at the same time, and I want to be careful here, because I think it's... I don't want to underplay the value of insurance data too far the other way, because I think there's a reason that you require MFA or you create an incentive to have it. I mean, I also am seeing more and more cyber insurers just say, "If you don't have MFA, you're not getting a policy." I mean, it'd be like insuring a smoker. Sorry, your rates, if you can get health insurance-

Kip: You're not going to get premium rates for being a smoker.

Jake: We just know that. That's just a fact. So it isn't the case that it's completely disconnected from the paths to the outcomes, but I don't know. It was just a thought, right? Which is-

Scott: No, no, no. It's a fair thought. The other thing that I would just add is that we update these models very frequently.

Jake: Yes, and that's critical.

Scott: Say an earthquake model, that may get updated every 10 years, because geophysics is not changing very often. There aren't very many earthquakes. But when they have, they're bad, but it's not changing. This model is updated, a major update every single year. But because we're ingesting data from some of these vendors, it's actively updating every day. So as we get data from some of these third-party vendors, the model literally is getting "updated every day".

Kip: Got it.

Scott: That's very different than a hurricane model or earthquake model.

Kip: Okay, fine. We're over time. Damn it. And my job is to keep us on time and I'm failing, so I've got to stop failing.

Jake: This is too interesting.

Kip: It is wonderful. It's a conversation I would love to... Scott, you should come back and we should have a part two episode.

Scott: Sure.

Kip: No doubt about it. I love this, but we do have to put a bow on it at this time, I'm sorry to say. But Scott, thank you so much for being on our show. Where can listeners go if they want to find out more about you and your work?

Scott: So we'll send some links and you can put those in the show notes. We have a webpage with lots of reports. We'll send you the study that I mentioned about controls and which are the most important and people can click that link and check it out and there's a contact us button on the page.

Kip: Okay. Can I put your LinkedIn profile in the show notes, the URL?

Scott: Yes, for sure. Yes.

Kip: Okay, that's great. Thanks so much. Everybody, that wraps up this episode of the Cyber Risk Management Podcast. Today we learned how cyber risk can actually be quantified, why not all cyber data vendors are equal, and whether cyber insurance is worth the investment. We did all that with our guest, Scott Stransky. Thank you so much, Scott. We'll see you all next time.

Jake: See you next time.

Speaker 1: Thanks for joining us today on the Cyber Risk Management Podcast. If you need to overcome a cybersecurity hurdle that's keeping you from growing your business profitably, then please visit us at cr-map.com. Thanks for tuning in. See you next time.