Speaker 1: Welcome to the Cyber Risk Management Podcast. Our mission is to help executives thrive as cyber risk managers. Your hosts are Kip Boyle, virtual chief information security officer at Cyber Risk Opportunities and Jake Bernstein, partner at the law firm of K&L Gates. Visit them at cr-map.com and klgates.com.

Jake: So Kip, what are we going to talk about today on episode 107 of the Cyber Risk Management Podcast?

Kip Boyle: I just love that we're numbering the podcast. I think I've said that seven times now.

Jake: I think, yes.

Kip Boyle: Sorry.

Jake: I think you have.

Kip Boyle: I'll stop saying it. It's cool. Well, episode 107 is going to go down in Cyber Risk Management Podcast history. First of all, because we're going to discuss something called the response side of vendor due diligence and-

Jake: I'm already sold. This can be great.

Kip Boyle: Yep. Yep. And because we're going to do it with a guest, Caroline McCaffery, and also because Caroline actually works at a company called ClearOPS. She's the co-founder and CEO, and she's also our episode sponsor. And I think this is the first time we've talked with somebody from a privacy tech company. So there's a lot going on here.

Jake: There is. And we should just mention and explain what we mean by episode sponsor, because I don't think we've done one of those before.

Kip Boyle: We haven't. Can you believe it? We're 107 episodes into our podcast and we've never had a sponsor. So this is something new. Jake and I have talked a lot about should we have sponsors? Other podcasts have sponsors. Shouldn't we be like other podcasts? Whatever, whatever. And this is not something we've ever felt comfortable trying, but then we met Caroline and we thought, all right, let's give it a shot. Right?

Now listen, everybody, we're not going to be doing this every episode. We're just maybe do it once in a while. And, Jake, as a former assistant attorney general for the state of Washington, who's very, very dialed in on the topics like this. Would you please tell our audience how sponsorships work on the Cyber Risk Management Podcast?

Jake: You bet. So first our guest is a subject matter expert on the topic. And as we just mentioned, ClearOPS is a privacy tech company and we'll be talking about the response side of vendor due diligence, which I am very curious about.

Second, we're only going to take on a sponsor if we think the episode will be just as relevant in depth and informative as any non-sponsored episode we'd set out to make.

And then third, we're going to be talking about the product in a way that helps illuminate interesting approaches for solving the problem that we're discussing. And just to add, this should not be seen as endorsement or anything like that. We haven't tested the product. I can't say, we don't know. We want to learn about the underlying technology and these concepts. That's what we're doing here.

Kip Boyle: Right. And just so everybody knows, we do have a vetting process and a prep process, which included a tour of the ClearOPS product. We do understand how it works and it seems very good as far as we can tell, but that's not what we're here to really talk about. All right, but let's talk about response side vendor due diligence. So-

Jake: What is that?

Kip Boyle: ... Caroline. Yeah, Caroline. Hi, welcome. Please tell us.

Caroline McCaffery: Hi. All right. Thank you. Thank you so much for having me. It's an honor to be here and especially to be your first sponsor. So response side vendor due diligence, specifically focusing on privacy and security. When I say vendor due diligence, basically I'll take you back to the first time I had to deal with what I call a security questionnaire. I think a lot of people call it vendor risk assessment, vendor security assessments. There's a lot of names for them, but I was an attorney working as the general counsel at a marketing technology company. And we received one of these very long questionnaires on behalf of very large enterprise company. And since I worked with the sales team very closely, I became the project manager for this questionnaire, which became the most incredibly painful process I have had to deal with. And I've dealt with a lot of inaudible processes that stressed me.

Kip Boyle: Yeah, I've done many of those more than I can count.

Caroline McCaffery: Yeah. So it's usually an Excel spreadsheet these days getting closer to more portals that contains all sorts of questions about the network and the infrastructure and the application, the authentication of the vendor's services. And in this first instance that I'm talking about, I went to my VP of engineering and I said, "Who can help me answer this?" And he was like, "Well, let's let you and me sit down and we'll take a shot at it." And for an hour, and we got through about 15 questions, I was like, this thing is a nightmare.

Jake: I'm guessing it was a lot more than 15 questions long.

Caroline McCaffery: Yes, it was about 150 questions long at that time. And it was awful and it took us months and it was a large deal. So of course there were other things going on, but that questionnaire certainly slowed down the deal by a significant amount of time. And I actually suspect the reason Enterprise sales has slowed down over the last five to seven years is because of these security due diligence and privacy due diligence processes that the vendors-

Kip Boyle: That's a lot of friction.

Caroline McCaffery: Yeah.

Kip Boyle: It's a lot of friction in the sales process. On the response side, I do a ton of work on the response side, helping our customers respond to these questionnaires. And actually, it's cute that you say was 150 question spreadsheet because these days, I'm dealing with 300 question spreadsheets, right? I'm sure you've probably seen those since you did your first one too.

Caroline McCaffery: Oh yeah. I've seen up to 1,100 of them.

Kip Boyle: 1000? Oh, my God. No.

Caroline McCaffery: Yes.

Jake: I think this is super helpful. It's super relevant. And I think what we can do is we can say response side vendor due diligence is really... We can say it's a vendor problem, right? It's a problem for vendors who need to do business with companies that are asking them to fill out these enormous questionnaires. Kip, this seems to be happening more and more, which is not surprising given that this is episode 107, but I feel like we've talked about this issue in the past. I am sure that we have. Maybe we can put something in the show notes at some point about a link to the other episodes. But my recollection here, one of the things that we talked about is the economic disconnect that is created by some of these vendor due diligence questionnaires.

I think Caroline, what you mentioned was at least you are in-house council and able to work with your VP of engineering on that. But we have done it and I've done it as outside council. And, I mean, the amount of money that gets spent to go over and review these questionnaires, it's crazy. And sometimes, and tell me if you haven't seen this before, although I'm sure you see it all the time, the cost of filling out the questionnaire, just doesn't line up with the value of the contract. And I'm really curious to hear what you have to say about... I assume that this is one of the founding reasons or the reasons for the founding of ClearOPS.

Caroline McCaffery: Yes. Yeah. Well, yes, absolutely. And I mean 100%. So, in terms of the incentives, one of the things that I often joke about is sales is commissioned, right? They get a commission for getting a sale. And so their incentives are to get these questioners done and done quickly. But the minute they kick it over to the engineering team or the security team, if the company happens to have one or even the outside consultant, the security consultant who may be no commission is going to be sent over to those people first. So they have no incentive to answer these things. In fact, they're almost disincentivized to answer these things because a lot of the questions, and Jake this is being attorney, they feel like they are going to trap you and cause liability. And you just don't even know because you don't know what the purpose is of the questionnaire.

Jake: Absolutely. I mean, I think it's a really difficult issue. I think we're seeing it now, the several year, really up until a year or two ago, well, even more, probably even less than that up to last 6 to 12 months, insurance questionnaires were really simple. They were really straightforward and simple.

Caroline McCaffery: Yeah.

Jake: Now, insurance questionnaires are starting to resemble these 150 to 1000 question, security due diligence and these questionnaires and that's just to buy insurance, which maybe that makes sense. But I think one of the things that's missing is, what is this? What's the value in many cases of these? And I feel for a lot of buyers, basically the questionnaires it's legal cover and that's about it. And I'm curious, I mean, as a lawyer yourself, what's your kind of perspective on that issue? Why are you being asked to do this? What's the security value or is it really a contract value?

Caroline McCaffery: Yeah, I think it was just purely a contract value before, sort of a check the box type of exercise shifting the liability. But now, because of the way the regulations are being written, when you look at CPRA, specifically CCPA, New York SHIELD Act, the Virginia law, the Colorado law, I'm pretty sure that I looked at the Utah law. It says the same thing, due diligence, what they call reasonable due diligence, which I put in quotes and with my hands. And obviously you can't see, but the reasonable due diligence security due diligence is a requirement of these laws. So now buyers are not only facing liability from a contract perspective, but also potentially regulatory fines. And some of these fines go up to $10,000 an instance. So it can really add up.

Jake: Yeah, for sure. And I think the regulations that include this type of vendor due diligence requirement are just proliferating at a rapid pace. The updates to the Gramm-Leach-Bliley Safeguards Rule that the FTC spent two years doing and are taking, will take effect this coming December and December, 2022, that has even more detail on it. The SEC has proposed Cybersecurity Risk Management Rules for under the 1940 acts, those that also contain specific fender due diligence, regulatory requirements. So, yeah, no, you're totally right. But that just makes it even more critical to get perhaps even more detail out of these. So, I think we've painted potentially depending on your point of view. Well, maybe just always a kind of horrifying picture of the situation. So, what are the challenges that you see vendors taking? We talked about the security questionnaires. Are there other ones though?

Caroline McCaffery: Yes. So, in the world of third party risk management from the vendor perspective, the security questionnaires were basically the enterprise buyer solution to the fact that the vendor didn't have, let's say a security audit completed. So a SOC 2, ISO 27001, one of those types of audits, which are very expensive. So as the enterprise kind of pushed the due diligence process from other enterprise vendors that they were working with into SMB vendors that they were working with, they said, "Okay, well, the SMB don't have these security audits, so we're going to now push the questionnaires." But then also you have public due diligence that they're doing. So they're googling the vendor and making sure that there's certain things in place. So vendors are really... There's information that the fire is getting in order to conduct their due diligence that sometimes the vendor may not even be aware of.

Jake: Which I mean, that's difficult to manage when that's the case for sure. Okay. So, how common are these... I think the security questionnaire was largely enough, but all this takes a lot of energy and a lot of work. So, what are vendors facing? How far behind are they in their ability to respond? Because if you think about it, it's really easy to send an Excel sheet and say, "Fill this out." Right? Just go ahead and do it.

Caroline McCaffery: Yep.

Jake: And that's from the buy side, but as you kind of hinted that, it's not the reality for the recipient of the questionnaire.

Caroline McCaffery: Yeah. No, it's not. I mean, there's so many ways to answer this question, but the one thing, I think a point that has been really driven home to me recently in talking to clients and prospects, is that a lot of times when you get a questionnaire, you're talking about the insurance questionnaires, a lot of times when you get these things, you answer them from what's in your head. So, you'll open it up and you'll just start to attack it from what you have in your memory. There's a lot of issues with that. But then if you also take into account that a lot of companies who are vendors don't have maybe a security team, so they don't have the resources who are dedicated to security. So, now you're relying on the information that's in the head of multiple people around the organization. So, now you're adding in layers of misinformation and that also is-

Jake: Not intentionally so, just-

Caroline McCaffery: No, right.

Jake: ... It's just memory, right? These are not like no one's out there trying to lie on these things. I think that would be a tremendously bad idea.

Caroline McCaffery: Yes, exactly. 100%, no one's trying to lie. Everyone's trying to answer them and get them done. But the lack of organization certainly can lead to cross purposes in terms of the information that's being disclosed.

Jake: Yep.

Caroline McCaffery: And then of course it leads to the actual lack of efficiency in the process. And I think when it comes to... What a lot of vendors are doing is they're saying, "Well, let me go see if there's a more efficient way to do this. Maybe I'll go get the security audit. So I'll spend now 40, 50, $60,000 on getting a security audit in the hopes that this avoids the questionnaires." And then they fall into the trap of, "Oh, actually now I have." I have to give up the audit, I still have to do the questionnaire. And, I've just now created a much more robust response process, which means I actually need to go higher in security. And as we all know, cybersecurity has a talent shortage.

Jake: Well, and I think, this is the part I recall talking about, but I think maybe it's good to get your perspective too. Even though I think it's quite fair to say that nobody is ever lying on these things, they do face a really tough situation, right? All the businesses that have to fill these things out, because the reality is that, particularly for these smaller mid-size businesses, they've got very much a chicken and egg problem going on here. Doing all the things substantively in a questionnaire, let alone, just filling it out is a lot of effort.

Caroline McCaffery: Yes.

Jake: Being able to do everything in the questionnaire could be really, really expensive and way, way, way more effort. And there is kind of this, you're stuck between this rock and a hard place, and they're closing together, because that's your funding runway.

Caroline McCaffery: Yeah.

Jake: The money is going away and you can't really move. And in order to get money to solve any of these things, you have to sell the product, but you can't sell the product unless you do these things. And I wonder if we are... I mean, yeah. I think that kind of setting that illustrates the problem right there. So yeah, I think-

Caroline McCaffery: inaudible.

Jake: ... we've illustrated the problem space quite well. Let's start out with the security questionnaire itself. I have a lot of experience representing the buyer. I've done both for sure. I think Kip has probably worked with more sellers and I've probably worked with more buyers and-

Kip Boyle: Yeah, that's true for me.

Jake: Yeah. I think that's true. And when I'm representing a buyer, I'm like, "Yeah, we have to do this due diligence." And you know what? You do. You have to. As we kind of mentioned previously, not only their liability and contract reasons, but there's regulatory reasons now as well.

There's a lot of risk to not just sensitive data, but the reality is that I think everyone knows this now, so many data breaches and cybersecurity failures and events come through the cyber supply chain via vendors. Right?

Caroline McCaffery: Mm-hmm.

Jake: So there's also, I mean, it would just be very risky to not do these types of things. So, I guess Caroline, why shouldn't all these vendors fill out these applications?

Caroline McCaffery: I mean, I think they're here to stay, unfortunately, for better or worse. I think you can even look back at financial services, auditing financial audits and see that the same process has been happening for a lot longer than security questionnaires came around. But why shouldn't they? I guess, to what you're saying before, a lot of people in the industry do think of them as a little bit of security theater. Like you said, the pressure in the questionnaire, a lot of these questionnaires are yes, no, not applicable, which means you have to make a binary choice. Do I answer this? Do I tick this box or do I not tick this box? And if I have-

Jake: Yeah.

Caroline McCaffery: ... a gray area response, I don't have anywhere to put that. And if you get to-

Jake: And-

Caroline McCaffery: ... yeah, go ahead.

Jake: Maybe I should clarify my question. I think it's probably better to say it seems to make sense that we should expect vendors to fill out these questionnaires completely-

Caroline McCaffery: Yes.

Jake: ... and accurately. I think you're right. They're not going anywhere, but this is... I mean, gosh, it's such a hard problem space.

Caroline McCaffery: Yeah.

Kip Boyle: Well, so on the buyer's side, right, you've got this situation where you want complete and accurate answers. But how do you know you're getting complete and accurate answers? Unless you go out and look, but that's why you did the questionnaire because you don't have time to go out and look, right?

Jake: Yeah.

Kip Boyle: So the buyers are in this Catch-22 sort of a situation. Well, and the sellers are also in a Catch-22 because, Jake, you just said it a moment ago, that it can be very difficult for a small company trying to provide services to a very large financial services, for example, for them to do everything, to protect the data in the way that the large financial service would.

And so, on the seller side, there's a temptation to just tell the buyer what they think the buyer wants to hear, because the chances of being discovered are, quite frankly, very low. You could get a buyer with a false sense of security and then you could get a seller that is misrepresenting themselves with a very low likelihood of being caught. And it's crazy because I've seen it where this mismatch is just incredible. Where the number of people working in the FinTech, because I work with some FinTechs, right? The total workforce of the FinTech is smaller than just the security department in the buyer.

Caroline McCaffery: Yes.

Kip Boyle: And they have no idea. Some of these questionnaires are just absurd. So for example, I've got this one FinTech customer where their cloud first, everything's in AWS, everything. They have no legacy data centers or co-hosting or anything like that. It's all in the cloud. But these questionnaires show up and these big buyers are just assuming that the FinTech is exactly like them.

Jake: It's lack of empathy, really.

Kip Boyle: Yeah. There's no empathy at all. It's just one size fit's all, right. And my company, right? I'm the owner of a small company, Cyber Risk Opportunities. I have six people and I just completed a vendor due diligence with this enormous insurance broker that wants help from us. And they sent us, guess what? A 200-question questionnaire. And I would say the vast majority of the answers that I gave were either not applicable or I'm too small to do that.

Caroline McCaffery: Yep. You're literally preaching to the choir on this, but it's so true. Vendors are inundated with questionnaires. Like I said, it's gone from-

Jake: Inundated.

Caroline McCaffery: Yes. It's gone from enterprise down to SMB, it's there... It's a horizontal problem straight across the board. Everybody's getting them. And no one at least, well, much reason clear up I'll do that little plug. But the whole point of why I got into this business is because I've thought to myself, there has got to be a better process. We need to make sure that we're tracking the answers. Because as a lawyer, if you responded to a questionnaire last year and you're responding to it this year, you need to know what you said last year, because you could be-

Jake: Oh, yeah.

Caroline McCaffery: ... caught in a lie that you didn't even realize because the person who answered it last year, left the company six months ago and they promised to SOC 2 audit that no one else knew about and you haven't gotten it yet. Right?

Jake: Yep. Oh my God. I think my head's going to explode. Well, I mean, it's not only that, but a single company just needs to have a consistent answer. Right?

Caroline McCaffery: Yeah.

Jake: I mean, tell me if you've seen this one before. A salesperson will end up filling out a questionnaire.

Caroline McCaffery: Yes.

Jake: Because they want to get it done.

Caroline McCaffery: Yes.

Jake: And they don't know... I understand, but at the same time, the amount of risk that creates for their company is massive. And I feel for this, it's the CEO basically needs to say, "Okay, here are the answers that you are all authorized to use. Thou shall not vary from these responses." And I think we're getting really close to what ClearOPS is designed to do.

Caroline McCaffery: Yeah. That's exactly what I've heard so many in the security industry, complain about how they can tell when a sales or marketing person actually-

Jake: Oh, yeah, that's totally true.

Caroline McCaffery: ... answers the questionnaire. And so exactly that, you have a system of approved answers. So at least they can take the first pass-

Jake: Yeah.

Caroline McCaffery: ... save you that administrative time, but still put the control of being able to review the answers in the person's hands that actually ultimately will take liability for it.

Kip Boyle: Yeah. One of the techniques that I've used with customers in order to decrease the cycle time and the resources of responding to these questionnaires is we have a template response. So, we've got a spreadsheet that contains the most frequently asked questions and the official response to any question that looks like that. So when we're doing them, that's one of the first places we go to is this answer bank, if you will.

Caroline McCaffery: Yep.

Kip Boyle: And it just kind of expedites things. Another thing we do is we actually have a cap on the amount of pre-sales of buyer due diligence that will participate in for free, because one of the advantages of being a FinTech and for a buyer to be getting services from a FinTech is that it's a pretty low cost point for whatever it is that you're buying. And there just isn't a lot of margin in most cases to do what sometimes feels like infinite presales due diligence. And so we actually put a cap on it, and then we say to the buyer, "Hey, we're happy to do more due diligence if you'd like, but you're going to need to pay for it."

Jake: Well, I think what's so interesting, Caroline, and I know that's not really in the script to ask you about your background, although we probably should have put that in the script. Because I think it's fascinating, but lawyers do need to be involved. I think that lawyers need to be involved in filling out these things, given the risk and the liability they create. And as we mentioned earlier, Kip, forget the margin. You could easily lose money on a contract through this process and I think that's the height of insanity in so many ways. And I'm curious, we could probably make this episode go for an hour and a half, but I do want to get to Caroline's additional content here.

And I'm curious about the second challenge security audits and specifically like SOC 2 is a fascinating one because I think it used to be really important, but used to use the phrase security theater a few minutes ago-

Kip Boyle: Love that.

Jake: ... and I do too, but I've noticed that with SOC 2, certain companies out there and I've specifically seen very, very large tech companies. They've just become skeptical of SOC 2.

Caroline McCaffery: Yeah.

Jake: They've been commoditized and they're like-

Caroline McCaffery: Yes.

Jake: ... and I'm just kind of curious if you can talk about what you've seen with respect to that. And how do you think the challenge can be dealt with and all that stuff, what your product does?

Caroline McCaffery: Yeah. I mean, in the SOC 2 space, we've seen a very quick and a lot of money flow into the SOC 2 prep companies, software companies. And I think it's been around for a long time, SOC 2 prep, usually done by, I think what is becoming a popular term, the virtual CISO or CSO, however you say that acronym, has been conducting preaudit prep for a long time, and then they kick it over to the auditors to actually conduct the audit. And now we're seeing software that is trying to take over that particular role. I have talked to a lot of virtual CISOs and I think there is some skepticism, but also embracing of the software.

But because of that, there has been this strong movement from the vendor side of, "Oh, I can just go get my SOC 2 in a couple weeks." So why would I go through a security questionnaire over and over and over again, this really painful process nobody wants to do when I can just go pay whatever it is that they're charging and get my SOC 2 in a couple weeks. And then that way, instead of answering the security question, I can just go right around and say, "Well, I have my SOC 2, here take this instead."

Kip Boyle: Yeah.

Caroline McCaffery: Unfortunately, as I have seen over and over again, SOC 2 is only one question on the questionnaire.

Kip Boyle: Yep.

Caroline McCaffery: Awkward only actually satisfies.

Kip Boyle: Yeah.

Caroline McCaffery: You just paid a lot of money for one question, one answer.

Jake: Congratulations, you've got 199 to go.

Caroline McCaffery: But it does work in some cases. But as you can imagine with the enterprise, enterprise being as smart as it is, they're seeing this too. And they're seeing how SOC 2s are becoming more prevalent in their vendors. And they're thinking, "Well, if it's easier to get, then the quality must be..." Well, maybe I won't go with quality, but the quantity certainly starts to make them think, "Well, then I need to increase the quantity somewhere else. And so-

Jake: Yeah.

Caroline McCaffery: ... instead of my questionnaire being 300 questions, I'll go to 1100 questions.

Kip Boyle: Geez. That's taken everything in the wrong direction.

Jake: Yes.

Kip Boyle: The problem I have was-

Jake: Nobody could see the facial expression I just made, but it was...

Kip Boyle: Sour lemon face.

Jake: Sour. Yeah. Like what's going on here?

Kip Boyle: One of the things about the SOC 2 that I find inherently difficult is that management chooses the controls that'll be tested.

Caroline McCaffery: Yes.

Kip Boyle: And I can tell you, as somebody who's worked in a service organization before, management has every incentive to choose controls that are easy to test, that they're particularly good at. And they have no incentive to choose controls that are extremely meaningful for their buyers, but are difficult for them to do.

Caroline McCaffery: Yes.

Kip Boyle: Does that make sense?

Caroline McCaffery: I mean, yeah, it does make sense. I actually end up having a lot of calls just to help people out. They're like free, it's nothing really to do even with ClearOPS where I talk about how to determine your SOC 2, type 1 and type 2 audit. And one of the things I say to everyone, and I'll give you advice here too, since I give it for free is talk to a few customers, ask them how many of the control criteria, how many of the controls should they be testing? If there's five, should they be getting all five or is the common control of security okay? Is that sufficient for purposes of getting past that enterprise security departments review of their SOC 2? Because the last thing you want to do is go get a SOC 2 just on the common control for security. And you're like, "Woo, God got my SOC 2." And then you get to your first CISO at a major bank and they're like, "That's great, but where's the other four?"

Kip Boyle: Right. Right. One thing I tell my customers is before you choose the criteria that you're going to put in scope for your SOC 2, go talk to your biggest customers and ask them what they're looking for.

Caroline McCaffery: Yep.

Kip Boyle: Because of exactly that issue that you brought up, Caroline, which thank you. That's great. There's a third challenge, Caroline, that you outlined, which is public data that is available that ostensibly tells the buyer about the seller's security posture. So, in your work, since this is what you're focused on all the time, what are you seeing?

Caroline McCaffery: Yeah, this is such a fascinating area. So when we first started looking at what public data was available about an entity, like a domain, right? You think about a website domain, and you think about what they're doing and what their marketing department is doing. When I was at a couple startups, my marketing department was like Wild Wild West. I had no idea that when they were throwing events, they were taking the email contact list and sharing it with their sponsors. That's just part of the industry. These are data privacy issues-

Jake: Oh, yes.

Caroline McCaffery: ... and practices. And if you review some of the things that are on a website, not even just the cookie notice, but all the third party code, everything that is on the way, that's all in the control of the marketing department. I'm not trying to say blame the marketing new department. That's not what I'm trying to do, but I'm just saying that what your security engineer is doing in terms of making sure there's authentication, that's really locked down and making sure there's all sorts of protections in the actual backend the databases, whatever, the Amazon instance, and then you go to the marketing department and there's a website vulnerability, all that work the security engineer is doing.

And no one on the other side of the house has any idea of anything to do with security. And so what do companies do to see if you're eating your own dog food? Well, they'll Google you. They will look at your website domain. One of the things I talk about all the time and no one actually really thinks about this, but in my mind, it's a big point of due diligence, is when's the last time you updated your privacy policy. Because I can tell you if you haven't updated it since GDPR went to effect, I don't trust you.

Jake: That's a really, really good point. And there are so many, I mean, maybe we'll have to have Caroline back just to talk about her background, but I mean, I think it's such a complex set of problems with marketing and these types of privacy issues, but you're 100%, right. All of that goes into the level of trust and isn't that really what this comes down to? If we think about it and Kip, you and I have had this discussion with a buy side client of ours, you can't do a penetration test on every vendor.

Kip Boyle: Right.

Jake: You can't. And I think what all of this is really coming down to at the end of the day is do I, the buyer, trust that you, the vendor, are taking reasonable steps to maintain security. That's the entirety of it. And I don't think that this current reality is particularly sustainable for all the reasons that we just spent the last 30 or so minutes-

Kip Boyle: Yeah.

Jake: ... discussing. And I want to hear from Caroline, we've really, I think, developed the problem. Is there a solution?

Caroline McCaffery: I mean, it's what I'm working on, right? So what Kip said earlier about having an answer bank? Well, we wanted to take that one step further and say... Because I did the same thing, have the answer bank, but then have use some of the new technology to make that easier to populate that answer bank into the new questionnaire. Right? So the AI does that. You could of course review it. I mean, you always have to review the stuff, but I've done the math, answering a question on average takes four minutes in a security questionnaire. You got to digest the question. Sometimes the questions are weird. You don't even understand them that you sometimes have to find the source of information. So four minutes per question, I already know-

Jake: That's like the fastest you could possibly do it. Right?

Kip Boyle: That is the fastest because sometimes it takes me 10 minutes on average or 15 minutes on average, depending on how obtuse the questions are.

Jake: Right.

Caroline McCaffery: Yep.

Jake: And maybe just to be clear, when you say four minutes, it's almost assuming that you have the answer handy. This is just the amount of time it takes to answer the question, assuming you have the answer relatively nearby. It just takes time.

Caroline McCaffery: Yes. That's to that point, that's assuming you have an answer bank, right? So you're looking for your-

Jake: Yeah. That's what I wanted to make sure of.

Caroline McCaffery: Yep. Four minutes. With a prepopulated function, I can take that down manually a little bit by doing some fancy copy paste, find search things, but using a prepopulate, I can take that average down to at most two minutes. I've even been able to take it down to almost as little as 30 seconds, because-

Jake: Wow.

Caroline McCaffery: ... I can look at the question. I can see the other question that was in the answer bank and I can be like, "And that's the right answer because the questions are so similar."

Jake: Yeah. That's really useful.

Caroline McCaffery: So, that's one thing.

Kip Boyle: Yeah. So what's-

Caroline McCaffery: Yeah. Yeah. Go ahead.

Kip Boyle: Oh, sorry, Caroline. But I wanted to tell everybody in the audience that this is one of the reasons why we thought Caroline would make a great guest and guest not just as a sponsor because I really feel Caroline is a thought leader in this space and she's choosing to take her thought leadership and pour it into a product. Now, whether you buy her product or not it doesn't matter to me. I know that matters to Caroline, but I love the thought leadership that she's bringing to us right now. And so I just want to highlight that and say how much I appreciate.

Jake: Well, and one other thing too, is from my perspective, I think what I love about ClearOPS as a concept is that I'm sure you yourself will get these security questionnaires, but the irony is you shouldn't really need to do them because what you're offering people is a tool, right? It's a tool. Yes, there is some like AI and some machine learning and some fanciness going on there. I don't want to understate what you've done at all, but I think you probably yourself recognize that conceptually, it's not that complicated, but nobody has done this.

Kip Boyle: Yeah.

Jake: And the amount of time saved is massive. I mean, it's just enormous. And I think that it pays substantive dividends because in consistency. It's not just speed, right? It's consistency, it's accuracy. It's the ability to have an approved bank of answers for the questions. In a way, it's a part of a governance, like program for a vendor to be able to have this. It's so much more than just a way to quickly populate an Excel sheet, right? I mean, that's how I look at it.

Caroline McCaffery: Yep. And I will add to both of your points, which is, it is just a tool, right? I mean, we definitely don't believe in replacing the substantive work of risk assessment. A tool shouldn't be doing that. That's a human capability, but this rote exercise of doing a search, find, copy, paste job from one spreadsheet to another, everyone tries to hire an intern to do this job because it's just so painful. And so let's use tools to take away the pain, the objective of just automate the stuff that should be automated and let you actually spend the time doing the substantive work that all security professionals want to do, which is evaluate the business and what data is collecting, how much data protection needs to happen, what security should be in place and how risky this really is for it on the buy side, how risky is this on the vendor? How much risk am I putting my business in by not having the security in place that I should have?

Jake: Yeah. And I just think that there are such dividends to be gained by using tools like this. And while it's totally the case, it is quote, "just a tool." What you can end up... It also feels like way more than the sum of its parts at the end of the day, right? Because of these benefits around accuracy and consistency and authorization to provide answers. I think it strikes me as something that is very, very important. And I wonder too, if there are ways that you could build in even more expand the use of the tool, particularly for vendors as it does kind of become a centralized governance point.

Caroline McCaffery: Yeah.

Jake: My point here is that, I look around and out of all the assessments and the risk management that we do, none of it require... you could do it all with a piece of paper and a pen, hypothetically. Because it's really a thought process, right? It's an intellectual exercise. It's not programming, right? It's not making a widget. It's none of those things. It's weird. And the tools do matter as Kip and I have said many times, tools matter. There's a reason Excel is so popular.

Kip Boyle: Yeah.

Jake: But the technology is pushing things forward and it's not replacing the thinking at all, but it's making you better at it-

Caroline McCaffery: Yeah.

Jake: ... actually. So I think it's really cool.

Caroline McCaffery: Thank you.

Kip Boyle: I like any tool that's going to let me spend less time on rote tasks and more time on analysis and prioritization and that sort of thing and risk assessment. I'm in favor of that. And just to be clear, we haven't used the ClearOPS tool. We have seen it demoed, but again, I think people should check it out. If nothing else, just to see some thought leadership in this space in action. And Caroline, thank you so much for coming onto the show, sharing with us what you know and for being our sponsor. And I just want to ask you, as we close out this episode, where can our listeners go to find out more?

Caroline McCaffery: Well, thank you again for having me. So you can visit us on our website, which is www.clearops.io, which is the only thing you got to remember, we are a .io company. You can also email us. It goes directly to my inbox at info@clearops.io, more than happy to take direct emails as well. And yeah, thanks again for having me. It's been a ton of fun. It's funny how you can take a boring topic like security questionnaires, and really turn it into something that's interesting.

Jake: It really is. And I know it is funny, I'd almost be willing to... This is the type of thing where once you start to really think about it, you're like, "Yes. It's just a tool, but at the same time, it's for the vendor, it's also more risk management." Right?

Caroline McCaffery: Yes.

Jake: It just goes so far beyond the surface level of how you could describe it. I think it's just very fascinating.

Kip Boyle: Yeah. Well, that wraps up this episode of the Cyber Risk Management Podcast. Today, we discussed response side vendor due diligence with our guest and episode sponsor, Caroline McCaffery, who is the co-founder and the CEO of a privacy tech company called ClearOPS. Thanks everybody so much. We'll see you next time.

Jake: See you next time.

Speaker 1: Thanks for joining us today on the Cyber Risk Management Podcast. If you need to overcome a cyber security hurdle, that's keeping you from growing your business profitably, then please visit us at cr-map.com. Thanks for tuning in. See you next time.