Skip to main content

When Metrics Go Awry in Commercial Banks

PrecisionLender's founder Carl Ryden likes to say "Data that doesn't lead to action is just trivia." Banks are great at measuring all sorts of things, but which metrics are really helping them improve performance and which ones are just ... trivia? ​ 

  

Helpful Links

Questions? Comments? Email Jim Young at jim.young@q2.com

Transcript:

Jim Young:
Hi, and welcome to the Purposeful Banker, the podcast brought to you by PrecisionLender. We discuss the big topics on the minds of today's best bankers. I'm your host, Jim Young, Director of Content of PrecisionLender, joined again today by Dallas Wells, our EVP of Strategy.
 
First off, an apology for missing our regularly scheduled episode on October 18th. So it's a rarity for us. I think we probably broke some sort of Lou Gehrig/Cal Ripken type streak with it. And I know for many of you, it left a void that was almost unfillable for two weeks in your life. Hopefully, we can redeem ourselves with this episode. I won't get into the dirty details of what happened. Just know that it was almost entirely Dallas' fault.
 
So moving onward though, today's episode was actually inspired by a client who was quoting Carl Ryden. So here's some quick context. Andy Max of First National Bank of Omaha was a panelist on our recent fireside chat on investing in privacy. We're going to soon have a blog post recap of that discussion up on the site if we don't already, by the time this podcast drops. So just be on the lookout for it.
 
Anyway, the conversation turned to, "Well, how do you measure privacy? What's the data you're using and how are you measuring it?" And that's when Andy dropped this Carlism, "Data that doesn't lead to action is just trivia." So that got us to thinking, what are some of the things you might be meticulously measuring right now at your bank, these metrics out there, that aren't really moving the needle? So in other words, what are some metrics that may have devolved into just trivia? So Dallas, welcome back to the show, prodigal podcaster.
 
Dallas Wells:
Yeah. Thanks for agreeing to let me come back after causing the missed episodes. Appreciate that.
 
Jim Young:
The key to being the guy that does the intro is your first to throw the other guy under the bus. It's really one of the big advantages to being the the host.
 
Dallas Wells:
I noticed. Yeah.
 
Jim Young:
All right. So let's dive in here. I've got a list of metrics that you provided me ahead of time, that you believe are often turning into trivia at banks. So I want you to tell me why that's happened at some of the banks. And then I will probably argue with you on almost all of them. So first up here, overhead costs, keeping track of costs seems like a fairly important thing to do, Dallas. So how could this possibly be trivia?
 
Dallas Wells:
So it is important. I won't say that knowing how much something costs isn't important. I guess the question, and to tie it back to the way Andy Max was describing it is, so what? What are you going to do with that information? And there are plenty of folks at the bank who have plenty to do with that. That's literally their job, is to figure out how to squeeze incremental costs out of every function, process across the entire bank. But if you have commercial relationship managers, whose job is to generate new business and to just do it at kind of, maximizing the spreads and revenue that they can generate on this new business, is the cost, the overhead cost, of that deal really the thing that you want to spend so much time drilling into their brand brain? So you should have some threshold over which the deal has to be above this, or we don't make any money. They should know where that line in the sand is. And there's lots of ways to communicate that.
 
But what we find is that relationship managers are obsessed with this number. And it's because it's all that anyone at the bank wants to talk about. And so when they are structuring and pricing these deals, as you measure that with some sort of a pricing tool, the number that inevitably gets zeroed in on is, well, how much overhead cost am I being allocated on this deal? And that's the one that the bankers want to pick apart. Well, you're charging me for the branch network, or for the health insurance, or you're not charging me for those things and so we don't really know how profitable this deal is. And that's where 90% of the debate about profitability of a deal resides, is in overhead costs. It's the one that the banker can't do a thing about. And should that really be, should the health insurance cost of the bank be what decides whether or not you book a new come commercial transaction with a commercial customer?
 
Or should you be driving that decision a different way? So it takes up way too much time and attention and frankly, heartache. And all of those debates and all that brain power would be much better used on a hundred other aspects of that deal that actually can be changed, can be in impacted by that banker, and how they put that deal together with that customer. So that's the so-what part of it, is, yeah, we know where the costs are, but if you can't do anything about it, it doesn't need to be item 1A on everybody's list.
 
Jim Young:
Gotcha. And I'm guessing also that the level of calculation of it and focus on it probably gets to the point where it's, again, we've used that phrase, it goes from directionally accurate to, in this case, trivial differences in terms of it.
 
Dallas Wells:
Yeah. This is one where it's always wrong. So we know the sum total of the overhead costs. That one's pretty easy. You know the checks that get written and you know what the total expenses are. But do we know what the share of the cost for that $200,000 revolving line of credit that you just did for Jim's Muffler Shop? No. It's all going to be a modeling exercise. They're all going to be wrong. They'll be a general directionally correct look at it. So we want to make sure that we kind of, again, in aggregate, we're covering those expenses, but especially things for like fixed costs and don't be making an individual transaction decision based on that. If anything, remove that entirely and just look at the marginal cost of doing that deal. How much are we going to spend to originate this? Do we cover that cost? Then we're good to go. We're net accretive here on all the risk-adjusted ways that we look at that. So that's the key part of it.
 
Jim Young:
Okay. All right. Moving on to the next one. I'm going to try not to take this one personally, but you're going to detect it's creeped into the tone of my voice here. But the next one you had listed was spreads. And I feel like, yeah, every month Anna-Fah Lohn and I go through the commercial loan pricing market updates and we take a look at spreads. And we monitor how they're doing, and what that means, and why it's something's moving up and why it's moving down and all that sort of thing. Are you, basically, invalidating all the work that Anna-Fay Lohn and I have been doing?
 
Dallas Wells:
Wow, this one did get personal, didn't it? As opposed to overhead costs, if you did have to rank the metrics, the numbers to pay attention to as you're working on the commercial side of the business, credit spreads should be at the very top. So your work is not in vain. But we see some, and it's typically because of incentive plans, frankly, but we see some myopic focus on credit spreads. So as we are configuring and rolling out PrecisionLender for a new bank, I have personally been involved with this exact discussion more than once of the banker saying, "Well, look, that's great that you show all the risk-adjusted net income and the returns on the various capital allocations, but can you just get rid of all that stuff and just show me what the spread is?" And the answer's always, "Well, no, that would be really stupid. We're not going to do that to you."
 
And so the oversimplified reason why, okay, I want to know that the credit spreads on this, the spread over my index, is 300 basis points. All right, is that good or bad? Is it enough? Or is it not? Are you doing unsecured credit card style lending? 300 basis points is not enough. Are you doing a secured loan to Microsoft? That's more than enough. And, in fact, it's going to be so much that you're not going to win that deal. So credit spreads are missing all kinds of context and really important context. And so they are a vital, really important ingredient. Banks that generate good, solid credit spreads are going to be more profitable than those banks that do not, as a general rule. But there's lots of other pieces that go into that.
 
One of our most profitable clients actually has some of the lowest credit spreads across our entire book of business. And it's because they have, essentially, near zero credit risk. They lend to rock solid customers. So on a risk-adjusted basis it's very profitable. And they tend to use these credit facilities as not quite loss leaders, but it's the least profitable part of their business. And then they go sell them very profitable deposit and fee based business, which is where they make the majority of their returns, especially on a risk-adjusted basis as returns on capital.
 
So they take their little bit of capital and they extend that capital in the form of credit to these rock solid customers. And then they don't have to extend new capital to do things like treasury services, that is wildly profitable for them. So is credit spread the most important number them? It shouldn't be. And they need to know what it is. By all means, it should be measured. But you also need all that other context to know, is it appropriate or not?
 
Jim Young:
Yeah. So in fairness to that one, maybe not trivia, but maybe not, like you said, as be-all-end-all as the way maybe it's treated at places, so.
 
Dallas Wells:
Right.
 
Jim Young:
All right. So the next one, you've done a good job here. I don't have a witty retort on this one, because I'm not quite sure what you mean. So the next one you have on here is market share. It's kind of a broad term. So what are you referring to here? And where is this one, again, something where it's a metric that can go awry?
 
Dallas Wells:
So it's a broad term because banks use it all over the place. So they do measure market share of deposits, of loans in particular geographic markets or in particular industries. I stuck this one on here because, and maybe I'm just a glutton for punishment and like to relive terrible things, but I read the story of Washington Mutual and it's sort of massive growth and then ultimate failure and seizure by the FDIC. Their most important metric through their big growth rate phase was market share. They were obsessed with what share of new mortgage originations did they win. That's kind of the easiest example of why market share and a business like banking maybe isn't the right number to measure, because they hit their number. They wanted to be the leaders. They picked a few geographic markets and they said, "We want to be the biggest originator in these markets." And they did it. And then it exploded them.
 
So similar things on a less disastrous scale can happen when you're measuring market share even of deposits. It's not that hard to win a bunch of deposits, if you just overpay for them. It's fairly simple. Just pay way above market interest rates and you're going to get a bunch of deposits. Congratulations, you earned a bunch of market share. And on the lending side, you can take more risk. You can charge less. If you're gaining too much market share, a lot of times it's a sign that you're doing it wrong in the banking business.
 
There are places where that can be obviously, an exception, but as a general rule, if you're making massive gains in market share, you need to be triple checking yourself. What are you missing? What does everyone else see that you're not seeing? And so that's one where, again, if you incentivize your market president of a metro area based on what's the market share across the various lines of business, you're going to get some consequences there that you probably don't like.
 
Jim Young:
Yeah. So again, and I'm now revising the title of this podcast again, as we go through it, because again, that's one that it's not trivia, because it's leading to action. It's just leading to the wrong action. Okay. All right, but now you've once again, got me shaking my head here with your next one, which is asset and loan portfolio size. I mean, you just take this one away, because I'm like, "All right, how can this possibly be a metric that could go awry?"
 
Dallas Wells:
It's related to the first one. So still, the most common way of incentivizing a relationship manager or a loan officer is by the size of their portfolio. And so even when banks try to intentionally get away from that and they create these incentive plans, a lot of times they make them way too complicated. They're like, "Well, it's not just going to be the based on portfolio size. There's actually these 35 different metrics that we're going to put in this different weighting." And it turns into this God awful spreadsheet that nobody really knows how it works. And so you effectively, have no incentive comp plan, because how could you possibly drive behavior in something so complex?
 
But even if you get that part right, when the competing banks in town come knocking on their door and the recruiters are calling, they're going to want to know, what's the size of that book? That's still how people are backing into kind of, what's the value of this relationship manager? Do you carry a 20 million portfolio? Or $200 million portfolio? Now, that size is correlated to how profitable that employee is and how profitable that book of business is. There is a positive correlation there. It is not one to one. And there are plenty of clients in our database where the most profitable relationship manager does not have the largest portfolio. That is very, very common. I'm willing to go a rough guess off the top of my head and say it's maybe half. In half the institutions it's whoever has the biggest portfolio also is the most profitable. But in the other half, that's not the case.
 
So again, is it a number that matter? Yes, of course it is. It's one of the inputs that will determine how much returns you're actually making. It is the outsized importance that's placed on it that causes it to maybe be potentially troublesome where again, you're incenting the wrong behavior. You're booking business, no matter if it's too risky or no matter if it's profitable enough. And what you're really trying to do as a financial institution is you're trying to take the precious resource that you have, which is capital. You only have so much capital to go around. You only have so much capital who invest in these deals. So it really should be how much income are you generating from the capital that you've extended out? From the capital that we've put in your pocket to be able to do this book of business, how much money are we getting back on that?
 
So if you have this massive loan portfolio, but it also soaks up a ton of the capital and you're only making a 6 or 7% return on that capital, which is fairly common with some of these big portfolios, you're doing it wrong. Again, you're not optimizing for the right thing. You're just going for size instead of quality.
 
Jim Young:
I think I've used this analogy before, because in a previous podcast, but it's the world I understand, which is sports. But I've had coaches. When I've interviewed them before, we talk about 4.4 speed and speed in the 40. And they'll say, "Well, 4.4 speed is great, as long as it's going in the right direction!" If it's not, if you're going in the wrong direction, it's actually really damaging. And I can see it like one portfolio size is great as long as that's a big profitable portfolio versus a big, not-very-profitable portfolio. And at that point it's doing damage, so. All right. I think you might have touched on this a little bit here, with the next one, with return on assets. So is some of what you were just discussing kind of, does that sort of bleed into sort of your thoughts on how return on assets as a measurement, as a metric, maybe can go a little awry?
 
Dallas Wells:
Yeah. Again, it's the same concept there, where asset-based decisions and the size of the assets and the returns on those assets, there will be a correlation there, but it's not the right number. It's not the way to look at it. And again, we have lots of clients, and they have their logic for it, and sometimes I follow it and, and yes, ROA is a metric that you should be aware of. But we have clients who want to default that and say, "We want people to make decisions based on the return on assets of a particular deal." But the assets are not, that's not the scarce part of your business. That's not the constraint.
 
The constraint is the capital. That's what you will run out of. That's what limits your growth. That is what always you have to make the best use of. It is, when you get right down to it, that is the money entrusted to you by shareholders to make a return on. And so you need to generate the best returns possible on that. And kind of how much you leverage that and how big the asset size turns into because of that is going to be part of the strategy. That's part of the financial discussion that's happening there is, how do we leverage this money and where do we best allocate it? That's going to determine how big the assets are on what the ROA is. But would you measure the return on assets of a bond portfolio the same as you would alone portfolio? Those are very different risk profiles there.
 
And again, they're using up different amounts of capital. You have very little capital that you have to put against a bond portfolio, depending on what kinds of bonds, of course. But because there's this very little risk there. So you can take this little bitty sliver of capital and you generate a tiny ROA on it, but you can leverage it more because it is safe, versus loans, you just can't do that kind of leverage. So ROA is sort of like credit spreads. You're just missing so much of the important context there.
 
Jim Young:
Okay. All right. So you softened me up here. You've got me nodding my head along with you and thinking that we're totally on the same page here. And then you hit me with system adoption. And I mean, I can't tell you the number of times that we tell banks, every day, that their systems don't matter if people don't use them. But now, are you telling me that tracking adoption doesn't matter, Dallas Wells?
 
Dallas Wells:
So adoption does ... Well, it sort of matters. So it's another one again, where I get why this gets measured. It's sort of like in the marketing world, why you would measure traffic counts on a blog post. Is it important to have more traffic instead of less traffic? Generally, yes. But it should also be the right kind of traffic. System adoption is kind of the same thing. And this is another one that can kind of create some perverse incentives.
 
So you'll see, as folks are rolling out technology platforms, one of the ways that they measure success is, well, how much time do users spend inside the platform? Well, they're spending six hours a day in there. This is great. It's widely adopted. It's widely used. We've got everybody in there. This was a successful rollout. Okay. Is that the best use of those six hours? Or could the job that needs to be done there, could it be done in 10 minutes if they didn't have to get inside this clunky, obtuse system to try to navigate their way around. So time on the system seems like a, if you're trying to be effective and efficient, it seems like a backwards of measuring that.
 
Same thing, even really with logins. Hey, people are logging into the system 10 times a day. Okay, 10 separate times they've got to go in and log into that system to check for something. Wouldn't it be better to put that information where they need it, when they need it? And especially now the way that technology has evolved, a really effective enterprise tool, sometimes you don't even know that it exists, because it's surfacing things through APIs or it's sort of routing information inside of another platform.
 
So Salesforce is a really good of this. It's a true platform on which other systems are now bolted. So you may be doing your loan origination system on top of the Force.com system. It's a very common setup. And so what your users are doing in there is loan origination stuff. Do they know that they're logging into Salesforce? Is it ever branded as Salesforce? Does that matter? So at PrecisionLender, we've built integrations into Salesforce. So the users are inside Salesforce, because that's where a lot of these things are built. They push pricing in PrecisionLender and it pops open a window, which is technically our system. They do what they need to do. They hit save and it updates everything in Salesforce. If you ask that banker, "Hey, do you like using PrecisionLender?" We know this, because we did these surveys and they're like, "I don't know what PrecisionLender is. Never heard of it."
 
And at first we were like, "Oh crap!" We were measuring these sort of metrics and we're like, "This is a disaster. They don't even know what this is." And then we're like, "Actually, I think maybe that's just a really well designed and well built integration. It's smooth. It's seamless. They never know it happens. They just assume it's a widget or a window inside of Salesforce. It all happens in one place. There's no separate login. There's no jarring transition from one user interface to the other. It's all smooth and makes sense and is easy." So there are lots of tools like that now that users are using. And you may never actually even capture that they logged in.
 
And people may be seeing outputs from your system. Executives may be seeing dashboards that they never log in for, but there's data being piped from your system to their eyeballs. Is that valuable? Yeah. Do they log in to get it? Not necessarily. Does that mean it wasn't useful to you? So you can start to get distracted by the noise of those things. And instead of finding what, granted, is much more difficult to measure, but you're actually trying to measure value and effectiveness there. And you've got to find proxies for that. And frankly, logins and time in the system are just the laziest proxies you can possibly come up with. And a lot of times they don't tell you what you really need to know.
 
Jim Young:
All right. So I mean, to put a bow on that one, saying that it's measuring system adoption might still be valuable, but the way you're measuring it maybe isn't, because... I mean, I get the point, but yeah.
 
Dallas Wells:
I think that's right. Yeah.
 
Jim Young:
Okay. All right. One final question here. Sorry, not question. My final measurement, I should say, but I will have a question with it, which is branch traffic and transaction accounts. Is this just simply a, hey, the whole world is digital, so brick and mortar stuff doesn't matter sort of statement you're making here? Or what are you getting at with that one?
 
Dallas Wells:
No, I think it might be the opposite. So all banks since just about forever have kind of measured transaction counts at branches. And for the last several decades, you can guess at the trend line there. It's been a steady down and to the right transactions per employee, per branch, per just about any way you measure it. And what happens is you've got now some branches that used to maybe do, I'll make up a number, 50,000 transactions a day. And it would be super high on Fridays, because you'd have lines around the corner to your drive through where people were coming and depositing and cashing payroll checks.
 
And now, all that stuff happens electronically. So instead of 50,000 transactions a day, you might have days where there's 10 transactions. And so the temptation is to look at that and say, "Well, traffic counts are way down. We need to close that branch." The problem is, is that you're not comparing apples to apples. Those transactions are no longer cash checks for the most part. They're going to be much more high-impact kind of interactions. So it's an issue that needs to be solved, that somebody wants to do face to face. Or it is for that customer, maybe a highly emotional discussion that they need to have, that they don't feel comfortable having over the phone or it's complicated, they can't get it done online. It's something where they need to come in and actually deal with another human to get something complex done, or to have a sales discussion about some product or another.
 
So the transaction counts are way down. But that doesn't mean that the impact of that branch is down. So it's another one that's hard to measure of like, well, what was the impact of those 10 interactions you had inside the building versus the 50,000 that you had if few years ago? And is that branch really less valuable? Maybe or maybe not. I don't think the transaction counts, or the traffic, or the length of the drive through line are not a good indicator of that. It's actually going to be much more difficult to figure out than it used to be.
And even how you staff that branch is more difficult than it used to be. It used to be a pretty simple formula. You could actually measure the lines and there's a whole science around queuing, and how long the lines are, and how much staff do you need to handle that and you had a whole team at the bank that handled that kind of stuff. This metric is probably a relic from that era that just is not as important anymore. And instead you need to figure out a different way to measure that impact.
 
Jim Young:
Okay. All right. Well, you have successfully made it through the gauntlet of these questions. You've done, I got to admit, a pretty solid job of defending your position on here. And to be fair, the position again on this is not, these are all stupid and I think you should never measure them. It's really more of, these are the things if you're measuring them, you're not getting the full story or you might be overusing them or over valuing them. Does that seem fair?
 
Dallas Wells:
Yes. I think that's right. I think it is. Look, banking's a complicated business and the strive to find the one metric to rule them all, there's no such thing. And I think the other thing that we didn't explicitly mention, but you probably heard traces of it through there, is banks are really good at finding new metrics and new reports that are needed. They're really terrible at getting rid of ones that are no longer useful. So a lot of this stuff is, at one time had a solid purpose and it was necessary. And now, it shows up in your inbox and you're like, "What am I supposed to do with this?" It's noise, or trivia as Andy and Carl put it.
 
But the ones that you should pay attention to are, again, the ones that help you make a decision and the ones that actually drive some action. Those are useful metrics. If the metric you're looking at doesn't do either one of those things, it's probably time for the scrap heap or least to filter it to a different folder in your inbox. It doesn't need to be clouding your brain space the way it might be.
 
Jim Young:
All right. Well that will do it for this week's show. Dallas, thanks again for coming on.
 
Dallas Wells:
You bet. Thank you.
 
Jim Young:
And thanks so much for listening. Now, for a few friendly reminders. If you want to listen to more podcasts, check out more of our content, you can visit the resource page at PrecisionLender.com or head over to our homepage to learn more about the company behind the content. If you like what you've been hearing, please make sure to subscribe to the feed on Apple Podcasts, Google Player, Stitcher. We love to get ratings and feedback on any of those platforms. Till next time, this is Jim Young and Dallas Wells, and you've been listening to Purposeful Banker.