Speaker 1: Welcome to the Cyber Risk Management podcast. Our mission is to help executives thrive as cyber risk managers. Your host are Kip Boyle, virtual Chief Information Security Officer at Cyber Risk Opportunities. And Jake Bernstein, Partner at the law firm of K&L Gates. Visit them at cr-map.com and klgates.com.

Jake Bernstein: So Kip, what are we going to talk about today on episode 135 of the Cyber Risk Management podcast?

Kip Boyle: My god, Jake, I never thought you'd ask me. Geez, thank you.

Jake Bernstein: It's weird. I do it every time.

Kip Boyle: I know, but I don't know, it's the anticipation.

Jake Bernstein: Was it just 135? We're approaching 150. That's exciting.

Kip Boyle: I hope this isn't what it means to be 150 years old, waiting all the time for everything. Okay, look, let's get to it. We're going to talk today about something that will not shock our listeners. We're going to talk about measuring cyber risk. And we're going to do that with a guest. We have Ryan Leirvik here. Now he's the CEO of a company called Neuvik. And Leirvik and Neuvik rhymes, so I'm sure there's a good story in there somewhere.

Jake Bernstein: I was wondering that to be honest.

Kip Boyle: And so now how did I meet Ryan Leirvik? Well, I have a good friend named Josh Mason. Josh and I both served in the Air Force. We met at Wild West Hackin' Fest about a year or so ago. Anyway, so Josh works at Neuvik, he's a senior instructor there, and he said, Kip, I think Ryan would be a great guest for your podcast. And I just trusted him. So if this turns out to be a bomb, I know who I'm going to go see next.

Jake Bernstein: There you go.

Kip Boyle: But I don't think it's going to. Okay. Well, so Ryan, thank you for being here.

Jake Bernstein: I'm going to interrupt. I'm going to interrupt. I'm super excited about this, because to date, I know how we measure cyber risk and how we've done it. I'm really excited to get another perspective.

Kip Boyle: Like how real people do it?

Jake Bernstein: Other people, Kip. Other people do it. Maybe we can build on our process. But anyway, I totally interrupted and this is why scripts-

Kip Boyle: Intellectual property attorney is saying things like that.

Jake Bernstein: 135 episodes in, and everyone should know by now that the script is at best a-

Kip Boyle: It's a mere suggestion.

Jake Bernstein: Mere suggestion.

Kip Boyle: A formality.

Jake Bernstein: Yes. So go ahead. Go ahead. Let's introduce Ryan for real.

Kip Boyle: I already did. Okay, Ryan, thanks for being here. Tell us who are you and just tell us a little bit about your work in cybersecurity.

Ryan Leirvik: Kip, Jake, great to meet you. Thanks for having me on. If this doesn't go well, you can blame me. Of course not Josh. Has nothing to do with that.

Kip Boyle: Hey, man, I can always fix the recording. It'll never have happened.

Ryan Leirvik: Exactly. So if you're listening to this, it went well. Well no, really thanks for having me and welcome at home. A little bit about my background, 10 years at DoD doing cyber warfare, but prior to that I was a commercial guy, so I started my career at IBM. So as I went through the DoD process of going from identifying what assets are really critical, to protecting them, to then flipping to on the offensive side. When I left I wanted to get back into commercial and bring that capability and thinking to the commercial space. So to really bring that tactical feel, identifying the strategy, and really bring in the tactical feel to the commercial space. And that's where we're pointed now, and we've been back in commercial space for about 10 years.

Kip Boyle: Fantastic. And then go ahead and tell us the story if you would. Why does Leirvik and Neuvik rhyme? What's that all about?

Jake Bernstein: Maybe there isn't a story.

Ryan Leirvik: Pardon me?

Jake Bernstein: Maybe there isn't a story, but I bet there is.

Ryan Leirvik: There is a story. Yeah. Well, so here's the real story, and Jake, I should have known you would've asked that question. The reality is about nine years ago, I was at a SANS Hacking conference and I won a Garmin Nuvi. And so about that time, I was still at the DoD and I was being asked to do instant response work outside of the organization. So I had to come up with the name of an organization that worked on the outside but didn't conflict with my DoD work. So I combined the Garmin Nuvi with the last syllable, the last name, and that was the only unique name I could get the URL for. And that's how Neuvik was born.

Jake Bernstein: That's a good story.

Kip Boyle: Can't get the URL. What's next?

Ryan Leirvik: That's right.

Kip Boyle: Can't get this one, what's next? Okay.

Ryan Leirvik: Now we could go deeper and say the Garmin Nuvi provides directional accuracy, but we don't have to get that philosophically deep.

Kip Boyle: All right. So there's actually some headspace here. That's really cool actually. That's actually pretty smart.

Jake Bernstein: This episode is not sponsored by Garmin or Garmin Nuvi.

Ryan Leirvik: That's right.

Jake Bernstein: Or for that matter, it's also not sponsored by Neuvik, just FYI. That's not how we roll. We roll with guests.

Kip Boyle: That's right. And if we did do a sponsorship, we would tell you upfront.

Jake Bernstein: We would.

Kip Boyle: This is a sponsored episode, but that's not what's going on here. The reason why-

Jake Bernstein: This is taking us a long time to get into the substance, Kip, I don't know what our deal is today.

Kip Boyle: How is it that we're not on script? I don't know.

Jake Bernstein: I don't know.

Kip Boyle: Who could have possibly have trained me? So listen, so thank you, Ryan, for being here. We think we're going to learn a lot from you. Jake was right. We have our way of measuring risk. We work with certain customers, but you have a way that you like to use. In fact, you wrote a book about it, right?

Ryan Leirvik: I did.

Kip Boyle: Tell us about your book.

Ryan Leirvik: Great. It's called Understanding, Managing and Measure Cyber Risk. A really catchy title. I wanted to call it-

Jake Bernstein: Kind of like this podcast.

Ryan Leirvik: Exactly. I wanted to call it Risky Bitness and Apress were the publisher's like, yeah, no, we're going to name it something much more practical. I'm like-

Kip Boyle: See that's the problem of going to the traditional publisher, is they have a voice. What's up with that?

Ryan Leirvik: I don't know. But they did name it something very descriptive. And so there it is, Understand, Manage, and Measure Cyber Risk. But Kip to your question, the reality is leaving the DoD, going back into the private sector, I was realizing helping CISOs in organizations at big banks and retails organizations and such, looking at the cyber program, seeing some gaps that were common across all CISOs. Almost every single one that I talked to literally would have a problem. This was seven or eight years ago, saying, listen, I'm playing Whac-A-Mole here. I'm hitting all these things. I'm trying to communicate to the board. I'm trying to communicate to my peers. I'm trying to communicate to the members, and we know the story. Right?

And what was lacking was this one, two, three, how do I orient myself to the problem I'm trying to solve? And then once I'm oriented, what do I do? And then once I'm oriented to that about what to do, how do I even know if I'm doing well? So over time, just these common themes were picking up. And so I literally just sat down over the first year of COVID and wrote them down and published it. To be like, okay, let's just put a guidebook out that says, all right, do we know what problem we're solving for? Right? Number one, is it well-defined? Then two, if it's well-defined, how do we get under management? And then Kip and Jake, to your question, then how do we know we're doing well? And that's the measurement.

Kip Boyle: We're going to talk about, as we said in the beginning, how you measure cyber risk. And I know that you referenced that in your book, and I know you're going to talk from that. But rather than have you read a passage from your book, because if I was Terry Gross, I might ask you to do that, but I'm not going to.

Ryan Leirvik: I don't have the voice for it anyway.

Kip Boyle: No, I don't.

Ryan Leirvik: I don't have the voice for it.

Kip Boyle: Her voice is buttery smooth.

Ryan Leirvik: I know. It reminds me of

Kip Boyle: I'm talking about thin margarine.

Ryan Leirvik: Remember the movie Ted, comment about Morgan Freeman? Your voice is like a pillow, I just want to sleep on.

Jake Bernstein: That's a good voice. It's hard to have that voice.

Kip Boyle: Indeed.

Ryan Leirvik: It is.

Kip Boyle: Okay, Jake, so I believe you are going to pitch the first question to Ryan about the definition of cyber risk.

Jake Bernstein: So what is the definition of cyber risk and why is it important for an organization to standardize on the definition? Kip, did you give this to me? Because you know how much I love to define my terms.

Kip Boyle: Guilty as charged counsel.

Jake Bernstein: So I thought. There's a long running theme on the podcast, Ryan, as an attorney, I always find it important to define terms because if you don't, you don't have a common ground and the same sound, what we call a word, just may not have the same meaning to various people. So what is your definition of cyber risk?

Ryan Leirvik: Great, Jake, that's exactly the problem that we're solving for. What was surprising to me is that every time we walk into an organization, there was no clear definition of what risk is. Everybody says it, everybody uses it. But just like you have in contracts as a lawyer, you make your definitions upfront so it's very clear what we're talking about. So everything from that point on is oriented to the scope of the definition. I was finding this very similar. I think a lot of people are in the same boat of, in their organization, they don't have a definition. So what is the definition? Well, obviously it depends on who you go to, but there are a variety of definitions out there that you can choose from, right?

ISO has a great definition, not that dissimilar from NIST, which has a great definition. And others have other definitions. The ones that seem to stick really well are the ones that identify threats. Let's take the NIST 7621, IR 7621 definition. That's a really good definition for us in the US, right? Because we're mostly NIST focused with orientation versus ISO. And the definition there is it's a function of threats plus vulnerabilities, plus impact to the business, equals your risk. And the issue there is what's the impact? Why does it matter? And that's where the risk really seems to sit. And what's interesting-

Jake Bernstein: And how do you calculate likelihood, et cetera.

Ryan Leirvik: Precisely. Now, for the sake of this conversation, I'm happy to take likelihood and throw it out the window for now, because that is a conversation that I think in the security community we're all split on whether or not likelihood even matters or not, right?

Jake Bernstein: True. You could either say it's either incalculable or just assume 100%.

Ryan Leirvik: That's right. And Kip, in my experience what I've seen, people spend a lot of time trying to get to the perfect answer on what the risk looks like or what likelihood. And we forget that your investment in time for your risk equation should be less than the amount of time you spend mitigating it. Right?

Kip Boyle: I knew this was going to be a good episode, Ryan, because you're already starting to put your finger on what matters most. I just got to say that as a profession, the fact that we don't even have a common definition of risk speaks to some of the struggles of just doing our work. And when that spills out into conversations we're having with the C-suite or the board, we look silly. And especially when we get caught up in it and we spend more time trying to define terms or just get the perfect definition or the perfect algorithm or the perfect formula, and then all of a sudden we're not doing real work. We're not delivering business value to the organization. I got so fed up with this that I don't even really, I barely talk about risk in the work that I do, in the sense of trying to define all these terms.

I may be guilty of going the other extreme, I'm very practically focused on let's bias ourselves towards action. Let's make decisions and course correct later. That's where I stand on this topic.

Jake Bernstein: I think I'm safely in between. I think it's useful. I think it's important. I think clients want it, but I think getting it is difficult. So let's continue. Okay, you've got this definition from NISTIR 7621. Where do we go from there?

Ryan Leirvik: Great. But this gets to Kip's question, right? You can get so furious, not being able to move the needle forward on your risk program if you don't know where you're trying to point to or everybody's using different nomenclature. And the problem there really is, could we bring up the board and everybody else? The challenge there is they don't know the right questions to ask. If you're not in security, some of this is vaporware. So level setting, just on a definition, no definition has to be perfect. You just have to define it. So that when people throw things at you, at a CISO, we all know the situation. You're in the security practice, and all of a sudden people, you're asking about what's the risk of Log4j? What's the risk of the Cactus group for ransomware?

Well, those aren't risks. So Log4j was a vulnerability. It's a vulnerability. It takes a threat actor or a threat to use the vulnerability to have material impact on the business, leaving out likelihood.

Jake Bernstein: It's fair. It's fair. It's fair to do that. It really is.

Kip Boyle: Because you've got to simplify this stuff. We overcomplicate everything all the time. You got to keep it as simple as possible. I can't tell you how many times I've been in front of the members of the C-suite, the board, and it doesn't take much to cause the conversation to just spiral off in some crazy direction because you accidentally opened up a can of worms.

Ryan Leirvik: That's right. And they will take the newsworthy items and ask you immediately about them. So as you say, without a common framework, without a common way of defining things, the conversations can go off the rails quickly distracting us from the main point, which here's where our real risk is, which has material impact to the business, and here are mitigating efforts that we're trying to get to to get a handle on it. And if somebody walks in and says, hey, what's our risk of the Cactus group? And you're like, well, again, that's a threat. Unless we have a vulnerability as a user on one of our IT systems that's connected to our OT system, and this is like an OT case, then it's not going to have a material impact on the business. So it's low.

And the ability to just have that conversation. It's almost like you put up the shield of, hey, unless it cuts through this clutter, the simple mechanism, we're not going to address it because we have to focus on managing the business. Right?

Kip Boyle: And I want to say something about the fact that people come to us in a tizzy because they read a sensational story about Cactus group and it's like, whoa, whoa, whoa, whoa. Since when do we let the news media drive our risk program? That's not a good idea, right? Just as much as it's not a good idea to let vendors drive our risk program either, vendors sell useful things, we should buy what we need. But I don't think we should let either one of those sources be the driver of the program. And unfortunately I see a lot of people default to that because it's the easy button. There's a lot of complicated hard things in our world. We all want easy buttons. Those are easy buttons, but I just don't think they are, just because it's easy, it doesn't mean it's good or right or the best thing for you.

I just wanted to comment on something that you just said that I think is really important risk managers think about, which is, what are my risks? Not what do other people say are my risks and what I ought to do about. So this episode's about managing cyber risks. We've defined risk for our conversation, which I think is fine. Now, Ryan, let's talk about measurement because that's really important. And the question I want to ask you is, what if we measure wrong? Because we could talk about how to measure right all day, but I think it's interesting to look in the other end of the telescope and say, what kinds of awful things can happen if we measure the wrong things with respect to cyber risk?

Ryan Leirvik: Perfect. Yes. The measure question. What everybody's trying to identify. And to your question, the reality is if we're measuring, quote, the wrong thing, right? Our resources will be pulled in distracted places that aren't necessarily solving the risk. So great, that's easy to say, but the reality is like how do we identify what's really measurable? And this is really where the tussle is, because we love to measure things that we can measure, but we don't necessarily measure things that we should be measuring.

Jake Bernstein: Particularly when it's hard.

Ryan Leirvik: Exactly when it's hard. Because it's hard. And we got to tell people what we're trying to do and we're trying to get to it. This is where starting with the definition of where is the actual risk, we can start with this is where frameworks come in and they're great for at least guiding the program. All right, let's just for arguments, we're on the NIST train here, so we'll put ISO and all the others to this. Go ahead.

Jake Bernstein: You're on the right tracks because Kip and I are big NIST fans.

Ryan Leirvik: Good. All right, great. So let's pull up the NIST CSF, right? Why does that help any information security professional frame their program to manage the program? Because it starts with what's critical. Identify your most critical assets. Then protect them, then detect what issues might be around them. And hey, by the way, don't forget to be able to respond and recover. Now we have at least a framework to say, all right, if we're going to measure something important of where our risk is, let's start there. Instead of measuring what, DDoS attacks happen, which is not actionable at all other than, unless our load balances are failing, right? But we can identify, all right, well, do we know our most critical assets? That's a really good measure. How would we do that?

And again, we're back to simplicity. You start to identify the problem that we're solving for which, Kip, with your background, it's like you're in charge of all this massively critical information should any of it get out, that's a problem in your case for national security. Businesses have the same problem, should some information get out or assets be compromised, CIA, triangle on any one of your assets, data and applications or networks or users. If we understand what's critical, and we have a measure assigned to that, now we know what problem we're solving and we should be able to hang most of the actions and activities and mitigation measures off of something like this. But Jake, to your point, that's really hard because who do you know that does asset management well? Almost nobody. Who do you know has a good risk register? Less people.

Jake Bernstein: It's actually kind of striking. But on the other hand, how many people do you know that can tell you how many firewall rules they have or how many DDoS attempts they've fought off or how many phishing emails they've received? That's easy. People will gladly report that. But like, so what?

Ryan Leirvik: And that's it. What's the so what? Big deal. Because from the attacker standpoint, we're just going to get right past that. So if you're spending your time measuring this, and Kip, this gets your question, if you're spending time measuring things that are not actionable or give us an informative trend over time, well then we're wasting our time. We're wasting resources for the person tracking it. We're wasting wherever that's pointing you directionally from a management standpoint, and you're leaving this gaping back door behind you open that says, hey, you've got these assets out here that no one has access to, or no one has visibility of, that are on your network, and now you've got this magical backdoor in your network and you're not even looking at it because you're looking at your firewall rules and some other network that doesn't even apply.

Jake Bernstein: Let me ask this-

Kip Boyle: I just want to say, before you say something, Jake, quick comment. As a practicing chief information and security officer, this constantly vexed me that the stuff that I really wanted to measure was extremely hard, but all the easy stuff was at my fingertips and I didn't care about it. So it was really awful. Go ahead Jake.

Jake Bernstein: I was just going to ask if we can dig into a little bit about what's an actionable measurement in the cybersecurity frame of reference? And I'm just thinking about, I don't know why this came to mind, but my son worked at, his first job was at TacoTime, and he was often time in the drive-through. And one of the things that they do is they measure the amount of time it takes for a vehicle to go from ordering to exiting the drive-through. And this is a highly actionable piece of information. If it's five minutes, that's way too high. And I forget what their target was. I want to say it was 90 seconds. Something quick. And that's a really good example of the risk is that people will think your drive through is too slow. What do we measure?

We measure how long they're sitting in the drive-through and we can create actionable plans based on that. What is the cyber risk equivalent or what's a cyber risk equivalent that we can talk about here?

Ryan Leirvik: Great. This is the tussle. There's two things driving what we can measure. We have operational measures. So very similar, Jake, to what you just mentioned. How are we actually performing? But above that, we have strategic measures which says what's the goal of the organization? Why are we in business and what are we trying to do? So for TacoTime, at the strategic level, you can imagine a set of measures that says, we service our customers within a short period of time, or we service our customers within a reasonable period of time. Full stop. But that's a strategic measure. It's not necessarily quantifiable at that level. It's qualifiable, but not quantifiable. Then you have supporting operational measures that align to that strategy to say, how do we know we're performing?

So in the security world, that could very well be, let's just say, let's keep the focus on the first part of the NIST framework, percentage of assets identified as critical. That is a really easy measure. In fact, it's hardly even a metric. It's just straight math, right? It's the number of identified critical assets, as long as there's a definition for critical assets on top, over the amount of total assets. So that's the strategy. So even you could see this at a board level, or at least at the executive level say, we need to know what we have so we know what to protect and what to monitor for. So that would be a strategic measure. Underneath that would be supporting any of the things we can measure to get us towards that goal.

So for example, percentage of assets identified. Do we know what we own? This is really hard for people in cloud environments, which everybody's in now. How many business units do you have swiping a credit card and spinning up a cloud instance that all of a sudden opens this magical backdoor into your network that you don't necessarily have visibility of? Okay, that's something that needs to go in front of the executive, say, we're trying to get this measure of understanding what our total assets look like at a tactical level, and we've given authorization to all these business units to not slow down the business, but we're opening up a huge part of risk because we're pushing critical information in there and SOC doesn't have visibility of it. Oh, well, now that that'll force policy. That's where the actionable piece comes in for that tactical measure.

And we're using a very simple targeted example for that. But Jake, that could be one example where both of those measures together tell the start of the story of why we're even in business, and now we know we're focused on the problem. So that might be one example. We can go on from there. But that's just one example of a strategic measure that's supported by a tactical measure.

Kip Boyle: Are you convinced?

Jake Bernstein: What was that, Kip?

Kip Boyle: I said, are you convinced?

Jake Bernstein: No, it's super helpful. I'll tell you what I was looking, the audience can't tell, but I was looking pensive and the reason I was looking pensive is I was thinking, what's interesting about security is that relatively few companies are in business for security. Security is itself within most businesses a support or we like to call it's a cost, but I was going to say business enablement, Kip, I was not going to go down the cost.

Kip Boyle: On the best day of the week, it's a business enabler. On the worst day of the week, it's that damn line item in the budget I wish I could get rid of.

Jake Bernstein: It is. But I think in some ways though, that strategic vision for every security department is going to be pretty similar. And here's the big challenge, is let's just say, not a reasonable expectation might be that we don't get breached today. The problem though that I've seen in security is, a lot of the times it feels like you're trying to measure a negative or I would say prove a negative. And this is particularly hard when you go to the board and ask for resources. It's like, well, the TacoTime example is good. I can show a number and in this instance, if that number goes down, it's good. Problem is if zero is your starting point, I can't show you negatives. I can't have a negative event and I can't show that to you.

And any amount of increasing the number is terrible. So measuring becomes very challenging. How do you get around that or what do you focus on in order to take the pressure off of this idea that if you're not perfect, you're not good enough? Because we all know perfection is impossible in anything, but particularly in cybersecurity.

Ryan Leirvik: Great. Two things. One, demonstrate a security roadmap, if you will. And then two, progress over time. Now that sounds really basic and hand wavy, but here's the context behind it. One is identify what the strategic goal really is. What are we actually trying? Why are we in business? Zero incidents or protect our assets. So whatever that large piece is that we're all trying to satisfy for. Then we're managing it in this way. Let's just use in this framework, right? Identify what's important, protect it, detect any anomalies against it, respond and recover. Now each one of the measures we assign to that need to speak to each one of the categories. And if they speak to the categories, now we have a framing for real measures that we can measure over time. And this gets to the second piece, which is, we don't need to solve it right away.

I've seen a lot of, once an organization gets their head wrapped around this concept, even board members are fine with, okay, let's take the critical assets one, for example. Well, you're telling me we don't know everything. No, we don't. But now they're on the same page. Okay, what do you need to get there? Well, here are the five initiatives I'm planning on doing to get there. The plan is in year four or five or quarter three or four, whatever roadmap time out the board will or the executives or whoever, all organizations are organized differently. So wherever main decision is being made to say, look, we're going to try to get to 100% in X number of time. And then I've seen it work with interim measures in between to say, our goal is to get to 100%. We're nowhere near that now.

So we're going to try to do is get to 25% in the next three quarters or whatever, four quarter. And work it that way. So Jake, you're not trying to proverbially eat the elephant all at once. You're saying, this is what we think the elephant looks like and we're going to gradually eat it over time if you will.

Jake Bernstein: That actually makes a lot of sense. Effectively what you're doing is you're chunking, you're taking an enormous, almost unfathomably impossible goal of being, quote, being secure, whatever that even means. And breaking it down into pieces that can be measured and that you can show progress on. And I think, Kip, that's really, really similar to, it's what we do in a sense, right? I'm not sure I would've put it quite that way prior to this conversation, but that is one of the goals, is to show progress over time. I think that's hugely valuable and probably something that, it's one of those things where you need to make mole hills out of the mountain, to reverse the phrase. Because the mountain is this immovable object that kills most people who try to climb it. It's super daunting.

I think that's a really good way of thinking about it. So you've mentioned before, and I'm curious, if you're just starting, whether you are enterprise grade or small business or anything in between, and you've got this mountain in front of you, where are you going to go to start breaking down this problem? And I think you've mentioned framework. So how do you use them in this sense and them to tell if you're measuring the right stuff?

Ryan Leirvik: Perfect. This is where frameworks are really useful, right? Because I think we all know in the business now that no one framework fits any one organization well, it just doesn't, right? But it is a great guideline for where the categorical areas of focus need to be. So for example, the NIST CSF, right? Is it perfect? I'm not going to opine on that. I think it's great. And the reality is where the real strength is, is in the five functions, right?

Jake Bernstein: Soon to be six.

Ryan Leirvik: And soon to be six.

Jake Bernstein: Govern.

Ryan Leirvik: Govern, yeah. Is going to be the sixth one. Yeah, precisely. And this is what's important. They say, all right, so Jake to take the mountain and build little mole hills out of it so you can get them into things you can manage, aligning to the five or now or soon to be six, this October, categories help you get categorically aligned to know that you're actually focused on the right things. So starting with identified, it's like, do we even know our critical assets? Okay. Great. Assign measures to this to say, you can actually build it out to be your whole program, if you will. At least a starting point to say, well, what are the activities that I have in identifying critical assets? Here they are. Who owns them? When do they do? Or you can almost build out a program management sheet based on that.

But then the next piece is if we've got an understanding of where that category is, then we can move into the next one, which is, all right, are we protecting it? All right, so let's assume we have at least the concept of asset management defined and we're moving in that direction. We don't expect to have it 100% ever, but we're going to try to get to 98% in three years, right? Okay. Well, what privileged access management do we have? What true access management do we have on those critical assets? That tells us that the real thing we need to identify in that particular category, which is, all right, we know what we own, well, who has access to it and are we even managing that?
Now we're really starting to at least get focused on where the real problem lies and the frameworks can help do that just by putting it together. We spend a lot of time intaking information that may or may not be accurate to what we're trying to solve.

Jake Bernstein: We do. I think the way that I look at it, I think that an organization that has done nothing or doesn't know anything about cybersecurity can get, I don't know, this massive jump, I'll call it, out of 100, I'll say it's almost 50% just by having someone recite, identify, protect, detect, respond, recover. And just that. The reason I think this is true is that a lot of the times people think of cybersecurity and they think it's all about the tools, what Kip and I call blinky light security. And what it really is about is organization, and organization of thinking, organization of management and organization is really the difference between an army and a mob. And if you think about military science evolution over time-

Ryan Leirvik: I think about that all the time

Jake Bernstein: Okay, that's good. Maybe you do, I don't know. I don't usually think about this, but sometimes I do. And just the concept of how people went from completely disorganized tribal raiding parties in the Stone Age to the Roman legions, to all the way up to World War II style divisions and battalions and companies and et cetera, et cetera, all the way down. There's nothing magic or even all that technical about that. It's just organizational thought. And yet the effect of that organization is immense. I think that is what a lot of people still to this day miss about cyber risk management and measurement, which is, this is not about some incomprehensible technical computer type thing. It is about simple human organization and administration, and that why frameworks-

Kip Boyle: And human resources.

Jake Bernstein: And human resources. And that's why frameworks are so valuable. And why, if you're not aligning, if you're not using a framework, look, the Barbarian Hordes do occasionally win, but it's not going to be all that reproducible, and you better have a Genghis Kahn style legendary leader in front, leading your security group, otherwise you're probably not going to be successful.

Ryan Leirvik: That's right. And even if it-

Jake Bernstein: This really went off the rails, but I think it's good.

Ryan Leirvik: I think it's great because that's a perfect example of the reality is they're going to win at some point. And if you're not focused on response and recovery, which is where a framework will at least make sure-

Kip Boyle: Don't forget it.

Ryan Leirvik: That's right. That you're thinking about it, then you're not caught by surprise. Because how many organizations do you know that are like, we got all these great tools. Jake, you're bringing it up. So many tools. Okay, great. So you've got monitoring down really well, don't you? Okay, well, what are you actually monitoring? I don't know.

Jake Bernstein: Critical question.

Ryan Leirvik: What assets-

Jake Bernstein: I don't know. But we have a SIM and all kinds of four letter acronym things that collect a whole bunch of logs and analyze them-

Kip Boyle: Blinkiest lights you've ever seen.

Jake Bernstein: It is the blinkiest lights. You don't even know how much the blinky lights, there's so much blinky lights. It's the best blinky lights ever.

Ryan Leirvik: And the colors are great.

Kip Boyle: inaudible 11.

Ryan Leirvik: Yep, exactly. Yeah, we're shooting for 12 on a 10 scale.

Jake Bernstein: But at the end of the day you can say, so what? Unless it's the right organization, then it doesn't matter. The framework helps with that.

Ryan Leirvik: And that's really it. And we saw back in, I would say the early days, it's not really, so let's say 10 years ago before real frameworks were out there, guidance, there were some. Right? That organizations that were thinking ahead were coming with their own bespoke frameworks. And the reality is it does matter what you're focused on, how you're organized matters. But the more important key is that you are organized. You have a way to ingest and manage and measure the information you have coming at you. And it's surprising that we're here in security, because in risk management, this is normal in risk management. We just saw with SVB, right? Silicon Valley Bank, they ran without a chief risk officer for six years, and now we see why that's important. A certain percentage of their deposits were, well, 90% and above and maybe 95, 96% above, were well above the FDIC cap.

They also were heavily tech-based, right? Naturally by their depositors, which means at any given time somebody can come in immediately withdraw all of their cash in an instant. And by the way, that happened at what? I think today, the Wall Street Journal quoted it as a million dollars a minute for 10 hours.

Kip Boyle: It was crazy.

Ryan Leirvik: But this is the thing. This is-

Kip Boyle: Fueled by social media.

Ryan Leirvik: Well, yeah. But this is it. It's understanding the world you live in, right? It's not that different in security. Instead of money or withdrawal or capital that we're sitting on by our depositors that you then deploy in other places. In security it's the assets we have and the ability for somebody with malicious intent or just somebody doing something poorly. Putting those assets out into the world where anybody can see them. We live in a time now where all that stuff is available. Think about how many cloud instances you see that are swiped and put on that, well, I'm not going to name names at this point, but we forget to actually turn off public access, right?

Jake Bernstein: It happens all the time.

Ryan Leirvik: That's right. It's the same thing. And so without a categorical way of looking at the risk to like, okay, what's the real risk and then managing it, right? We're just going to still play Whac-A-Mole.

Jake Bernstein: There's something here, back to the script briefly, that I want-

Kip Boyle: I want to know the answer to this question.

Jake Bernstein: I want to wrap up on this. So since we've been talking about risk assessment, I'm curious, I want to ask a question about something that Kip and I talked about with Karen Worstell, back in episode 126. And Donn Parker, one of the godfathers of information security in communications of the ACM volume 50 issue three, March, 2007 said, risk assessment is hughie. And the full quote is, "Information security is an open-ended art, material unknowns predominate, not a science. There are no statistically valid adversity data with margin of error because we don't know what we don't know. And this will never change as long as victims and potential and actual perpetrators have the right and motivation to withhold the information. And we have insufficient funding to dig for that data. I believe this based on having interviewed more than 200 perpetrators and their victims."

As a result, Donn says, the only reasonable risk management approach is, quote, demonstrating due diligence to a defensible standard of care. Two questions. Have you heard this position before? And then what do you think of it?

Ryan Leirvik: Great. Yes. And it's a very reasonable position, right? Because in our world you're never 100% secure unless you turn everything off and shut off access. It just can't be.

Kip Boyle: 300% out of business.

Ryan Leirvik: That's right. Exactly. Which is totally against the point of having an IT infrastructure in the first place. So yes, there is a level of, having a standard of care for the information in which you are an owner that is in your authority to take possession of is really where the line is. And this is where we get the thresholds. Everybody loves the terms. Not everybody. Sorry. We hear a lot the term what's your risk threshold? But this is where it sounds like-

Jake Bernstein: Or appetite is another one.

Ryan Leirvik: Yeah. Perfect. Risk appetite, risk threshold. Where is that line for us? And this is where this quote is so perfectly astute or at least the astuteness behind it, because it's like, where is that level of care? We're above that, we are just going to manage it, we're going to manage the problem. Below it we should have mitigated the risk that brought it. And this is where strategic measures can really help identify where that line is. Because you can start to say, percentage of employees demonstrating poor security. All right, how many employees out of our hole is enough to realize, is enough where we've got repeat offenders on our phishing campaigns that also have DLP triggers, how many is enough? Right?

Because we're not going to get to zero, it's just not going to happen. But how many is enough? So we realize that at least the vast majority of our employees are demonstrating good security behavior. So now we know where to look. It takes the problem and brings it down to something manageable. And that is really where that do care comes into play. And professionally I think that's a good definition.

Jake Bernstein: It is. And-

Kip Boyle: Donn was actually saying that you could just follow a standard due care and never do risk assessment. Just don't even do it. Right? Just don't even go there. Don't even mention words like likelihood and threat and vulnerability in terms of trying to define a framework of risk.

Jake Bernstein: I still think that's going too far, but Kip knows that.

Kip Boyle: Well, I want to know what Ryan thinks. Donn was one of my mentors, I worked with him. And his perspective on this influenced the work that I did when I was at Stanford research. Because guess what? He was in charge. He was in charge. And so he was the one telling us, hey, when you go out and serve customers, focus on due care. Don't focus on all this hughie. Anyway, so I'm interested in what Ryan thinks.

Ryan Leirvik: I think it's right. Establishing what that due care looks like is important. This is where definitions matter. The question there is, in the world we live in now, what assessments do you need? I will actually pose it that most risk assessments are hughie because you're going to spend more time analyzing what the risk looks like versus spending that time from a resource standpoint on protecting what's most important. So the thread there is, rather than assessing things constantly, focus on the mitigation aspects and where that threshold is. But assessments themselves if done right have that due care focus. Where are the gaps in our due care that we think we've got but we don't? And this is where really good assessments come in. There's a whole world out there that are providing like push button pen tests that like-

Jake Bernstein: Oh god, I hate that. Yes, it's a pen test. No, no, that was just a vulnerability scan.

Ryan Leirvik: Exactly. And that's-

Kip Boyle: Very barebone ones at that.

Ryan Leirvik: Precisely. And that's where the-

Jake Bernstein: Isn't that what we're doing? When we do our cyber risk management action plan, we are focusing on, it's a form of gap analysis, risk assessment on what we're doing, what our mitigations are, what our strategy is. It's not about taking and looking at the threats. And that's one thing before we end, and this is going to go a little long, but that's okay, because this is phenomenal episode which people will hear. This concept of starting with the threat, it's so dangerous for a couple of reasons. It's disorienting, as you say, it's distracting for security defenders. It also leads to this absolutely unreasonable position of, oh, no one's out to get us, so we must be fine. Just a-

Kip Boyle: Right. And then the whole conversation collapses.

Jake Bernstein: The whole misunderstanding of, well, I don't see any threats that are going to target me, so I'm not going to worry about information security. And it's just as bad the other direction, you might think, oh my gosh, the threat is nation state, military special forces. There's no way I can possibly defend against them, so I'm not even going to try. Right? Either way, you get to the same place by either saying there are no threats, or by saying the threat is indefensible because it's so huge. Both of those are completely wrong. And I'm curious what your take is on that.

Ryan Leirvik: 100%. If you're chasing the threat, you're giving the power to somebody else. Think about that. You're spending resources on what everybody else is doing and you can't chase them. But to flip that, if you're focused on what the impact to the business is, what is really going to hurt my business? And focus on protecting that, this is the due care process. Now, depending on which threat actor shows up, you at least know what you're protecting and your focus is on protecting that particular asset class or classes. It's the proverbial, we spent all of our time reinforcing the house. I hate analogy, sorry, but I'm going to use it. We reinforce everything. We got double bolts on everything. Actually, you know what? I'm going to use a better example that just came to mind.

So we've all spent time in classified spaces. I was taking a firefighter through a class outside of one once, and it was a 9/11, NY Fire Department, one of the chiefs walking through a space just outside of a classified space once. And I said, look at us. He goes, what's all those dials and stuff? Super secret stuff back there. He goes, you know what I would do? I'd just take a battleax right through the door, right to the side, I go right through. And that's why you don't focus on the threats, right? It's like if you're trying to protect at all and you're not-

Jake Bernstein: You're like, hey, no. You're not supposed to. That's not playing the game. You're not doing it right. You can't do that.

Ryan Leirvik: So we didn't think through what the possibility. Like actual threat modeling is really helpful, because now forget about who the threat is, but if you think about different ways like the battleax through the door, how somebody could use it in a way, now the attention is focused on the critical assets and that due care versus the threat itself.

Kip Boyle: And now here's a great opportunity for me to name-drop Adam Shostack, and if you ever want to do threat modeling, he's got a wonderful book that you can buy for a reasonable amount of money. He's got even better training that you can go to. I've gone to his training, I've read his book If you want to do threat modeling, he's the guy. He's the dude. Go see him. Okay, so there's his free commercial because he never paid me to say that, just so you know. Okay. So we are coming up on time here, and I want to respect the fact that people only want these episodes to go so far. Somebody told me one time, they're like, when your episodes go too far, I have to sit in my car. I get to work, you're not done talking yet, and I have to sit there and wait for you to stop talking. So please be kind.

Ryan Leirvik: Sorry for everybody listening. Hope you-

Kip Boyle: We're going to be kind. Okay. So Ryan, if somebody wants to reach out and connect with you, learn more about you and your work, how should they do that?

Ryan Leirvik: Terrific. Neuvik.com. Neuvik.com. Just hit us up there or LinkedIn or anywhere else, and we're pretty easily defined. Thank you.

Kip Boyle: Okay, fantastic. Ryan. Thanks so much for being here. That wraps up this episode of the Cyber Risk Management podcast. Today we talked about measuring cyber risk with our guest, Ryan Leirvik, who's the CEO of a company called Neuvik Solutions, and he's the author of a book called Understand, Manage, and Measure Cyber Risk. We're so glad you were here. We'll see you next time everybody.

Jake Bernstein: See you next time.

Speaker 1: Thanks for joining us today on the Cyber Risk Management podcast. If you need to overcome a cybersecurity hurdle that's keeping you from growing your business profitably, then please visit us at cr-map.com. Thanks for tuning in. See you next time.