How Can AI Deliver Real Value for Banking?

Cheryl Brown

In a first for The Purposeful Banker, Alex Habet hosts a livestream discussion with Carl Ryden and Corey Gross exploring AI’s role in banking and lessons learned from Q2’s Andi Copilot for bankers.

Listen

Watch

Transcript

Alex Habet

Hi, and welcome to a special livestreamed episode of The Purposeful Banker, the leading commercial banking podcast, brought to you by Q2 PrecisionLender, where we discuss the big topics on the minds of today's best bankers. I'm your host, Alex Habet.

So this is the first time we've ever recorded the show in front of a live audience, but I think we chose the perfect occasion to do so because today's topic, how AI can deliver real value for banking, is probably one of the hottest topics out there right now, certainly in the space that we cover on the show, day in and day out. But we wanted to set a clear intention upfront about what today's conversation is really about. Our goal here today is to help you all make better sense of the world around you, specifically centered around some of these more recent advances and innovations.

Because the opportunity in front of banking is wide, it's vast, it can also be equal parts precarious. And, of course, it's not immune to over-exaggerated hype. So in the spirit of that, we're not going to complicate your lives further by continuing to add to the noise. You're not going to see detailed specs of our products, nor will you even see a demo today. We look forward to showing you all that kind of stuff in other meetings and venues. But today, in the spirit of making sense of it all, we thought we'd bring in two individuals who are uniquely positioned to speak to the opportunities ahead, not only because they're leaders in AI here at Q2, but because they've lived their own versions of this already and have gained valuable experiences getting to this point today.

Today, Q2 proudly has one of the deepest, most reputable approaches to AI anywhere in financial services. AI is core to the DNA of several Q2 products, but getting there was a bigger journey, one that's lasted nearly a decade in some cases. My two guests were both CEOs of companies which were eventually acquired by Q2, but these weren't just any companies. These companies function with AI at the heart of the operation. Carl Ryden is currently executive fellow here at Q2 and formerly the founder and CEO of PrecisionLender. PrecisionLender has set the standard in commercial relationship pricing technology across the world since 2009. I'm also joined by Corey Gross, who leads Q2's overall approach to AI, both in terms of our internal use, meaning how employees like me use AI every day, but also how the technologies themselves are embedded in the products that we offer to our clients. Corey was formerly the founder and CEO of Sensibill. Sensibill allows financial services companies to collect, synthesize, and action everyday customer spend data in order to personalize at scale.

So Carl, I'd like to get things going this afternoon with you. So I'd like to take you back to around the time where you and I probably met, it was a little under 10 years ago, but I'd like to understand at that time as you were contemplating what are ways you can take a powerful platform like PrecisionLender and make it even more valuable to banks, take us back to what were you thinking about. What made you determine what to build and ship and how has that influenced your thinking even up to this point today?

Carl Ryden

Yeah, I'll even go back further than when we first met. When we first started with PrecisionLender, it was really about, we saw a lot of solutions at banks and a lot of relationship managers were struggling because what they were having to do was fill out a form, hit next, fill another form, hit next, fill another form, hit next on some web-based application. They'd get to that last stage, it would say, "I'm sorry, that deal doesn't work. Try again." And they'd go back, back, back and the screen would blow up and all those other things. And it was a task, not a tool. It wasn't something that was helping them do their job better. It was something that they had to do to put data into a database so that somebody one day could generate a report and tell them, "Hey, you're not doing as well as we thought you could do. Why aren't you?"

And the frustration on the RM’s mind at that time was like, well, because I spend most of my day entering stuff into this damn system that you've given me. And so the idea we started with was how do we build something that does everything that the computer can do really, really well so that the human can do the thing that the human does really well, even better? It built tools, not tasks, right? How do we humanize the machine as opposed to mechanizing the human? How do we, instead of forcing them to act more like computers, how do we make the computers act more human and allow them to have a ... You hire a relationship manager to manage relationships, to have empathy, understand those customers, have those rich conversations, take all the products and capabilities of the bank and find a way to help that customer or prospect make progress if we can in their business. And sadly, a lot of RMs don't spend their day doing that.

So we started with that simple idea: Do everything the computer can do well so that the human can do the things they really can do better. And it started with a simple idea: Instead of fill out a screen, hit next, fill out a screen, hit next, and then you give them not the answer, you give them the, "Hey, you didn't work. Try again," the computer's perfectly capable of understanding that answer. We know the answer. It's just math about is this deal profitable enough. Does it meet the bank's objectives for the risk and other things? Why don't we take all that information instead of hiding it from them and springing on them at the last minute, how do we take that and move it forward into that conversation where they can have a better, more productive conversation with that customer, with that relationship that the bank's trying to build?

And we did that, and we did it with this real simple thing we call the Profitability Wizard, that as you are, instead of treating the screen as a form the customer's filling out, we treat it as a search, like Google. As you type things in, you're not trying to enter stuff into a database, you're trying to search for a solution that works for this customer. So how do we inject answers into that conversation to say, "Well, here's a way you can make the deal work. Here's another way you can make the deal work. Here's something for a customer like this for the relationship like this at a bank like this, here's the things we can do to help them. Make sure you talk to them about those." And it worked really well.

And we had these little dots we would pop up on the screen and we were dumb enough because we called it the Profitability Wizard, which was a completely really silly name to call it. And none of our customers actually bought into that name. They called it the magic dots. So these little dots would pop up on the screen and kind of coach them on ways to do things better. And one day we had this bank we worked with, First National Bank of Omaha, which is a wonderful bank with really great people. And we worked with a guy named Andy Max at this bank. Andy was the guy who had been tasked with the folks at the bank, the ownership of the bank, to build something that works for the bank and helping them manage pricing and profitability. And Andy saw what we had built, and he went to the higher ups at the bank and he said, "If I could build anything, I would build this, but we can't build this. Let's just buy their stuff." And they did, luckily for us, and I think for them as well.

And Andy, as he rolled it out to his bankers, he did this magical thing that a lot of the great people do. He would sit with the RMs and kind of walk a mile in their moccasins and kind of have empathy for what they're trying to do. And he would say, "Hey Joe, I see you're doing a deal like this to a customer like this. Last week, John did a deal just like that. Let me show you what he did." And then Andy would call us up. He'd do that several times and he'd call us up and he'd say, "Hey, Carl, I've done this two or three times. Could you put a magic dot on the screen that when you see them doing something like this, you offer that suggestion?" And we're like, well, that's an amazing idea. So we did that, and we started adding these magic dots and we called this a story of Andy and the Magic Dots back in the early days of PL.

And I was sitting in the room one day with the latest round of suggestions from Andy and talking to a developer and I said, "Hey Marcy, here's the things that Andy said we should do, and I think it's really great." And we were both lamenting, wouldn't it be great if every bank had an Andy? Andy makes our product 100 times more valuable at that bank because of that coaching he provides in the moment to the RMs, that guidance, that support, that additional bit of intelligence and awareness of the whole world around them to help them make that deal better. And the idea was like, wouldn't it be great if every bank had an Andy? And then we said, well shucks, why don't we just do that?

And so we actually created Andi, and we rebranded the Profitability Wizard and restructured how we thought about it to actually give every bank an Andi and give the banks the ability to deploy these things we call skills into the workflow that as they're doing the thing they're doing, as they're having those conversations, as they're working through ways to make that deal work, they had Andi over their shoulder who could see what they're doing, who saw every deal at the bank, who saw what was working, knew what the bank's strategy was, knew what the bank's products were, and could actually offer suggestions to make that deal better for that customer in that moment.

And this was, gosh, I think this was like, when was this? Alex, maybe you remember the date? I want to say it was like 20 ... It was eight years ago or so, nine years ago that we rolled that out, and we rolled it out at our Client Advisory Board meeting and said, "Hey, here's what we're going to do, guys, give us your feedback." The Client Advisory Board absolutely loved it. It was actually really wonderful because we rolled it out, Andy was in the room, Andy Max, and I literally told him we named it after him, except we changed the “y” to an “I” just so he wouldn't sue us. And he got a chuckle out of that. And really that has animated our journey ever since about how do we take all of the knowledge that exists within the bank and take it and translate it into that moment and translate it into a specific moment for a specific banker doing a specific task with a specific customer and target it as narrowly as possible.

And when you do that, you don't have to give people a book. You can give them a short checklist of here's the top three things that you can do to make this better. And that little help, that little boost, it turned out to be really, really powerful. And I'll talk more about that. But that's where we got started. And then when we built it, we first said, well, Andi's going to be able to see what's happening, like Andi's looking over their shoulder. But then we saw the chance of, well, we had some customers said, "Well, I want to ask a question." So we said, well shucks, we'll make it so Andi can kind of hear, and I'll put air quotes around "hear," but really as you can type a question into the Andi box and ask Andi a question. And that was the first chatbot.

And that turned out to be incredibly powerful, as well, because they would ask things even about, not about loan pricing. They would ask, how do I get a deal like this approved? What are the steps? What are the processes? And we had deployed a language model at the time for how do we understand what they're saying. It was called a language understanding model and it was the best Microsoft had to offer after we researched everything. It was really quite good, at that. But it pales in comparison to what people have today. And now with the advent of large language models and folks who spent hundreds of millions of dollars to build and train models to not only understand language, but now who can generate language back and translate that knowledge into that moment more effectively, I joke with you and I joke with Corey about we kind of built the Formula One car ready to take the Ferrari engine, but we didn't have the Ferrari engine at the time.

And as a small company, we were never going to spend a hundred million dollars to train an AI model, but we knew it was going to happen. And now those things have come to us and I think we're in a really good position now to take all the infrastructure we built around that, the ability to have skills that are trusted and secure, that connect to your bank's actual data in a secure, reliable, safe way, and use the power of language model to translate that into this moment. And at first I want to do it, we're going to do it within Andi, within the back of the bank into helping employees better serve customers. But eventually, I hope we see it. I think we're going to see this move all the way into the front of the bank where it's actually helping customers directly in the same way.

Alex Habet

Without a doubt. I remember those days back then early on when you first unveiled the skill's framework, and this memory vividly sticks out to me. We came as a small contingent from one of the banks to check out what PrecisionLender was up to around the time. And when we entered the headquarters there in Cary, North Carolina, you had the conference room on the left and then straight ahead was the kitchen and there was this device that I had never seen before. It was an Amazon Echo speaker, but the original ones that looked like a Pringles can, and I’d never seen one before. And I was kind of curious just looking at that thing, and we started asking questions and you and the team had started educating us about what Amazon had built with this thing and how much of an influence it almost had to the creation of the infrastructure that powers a lot of the Andi skills framework today.

I guess one question I have, one follow-up question, are you having any experiences with large language models today outside of the Q2 world, whether it's personal life or for some hobbies that are inspiring you to think about how you bring some of this stuff into a banking experience or a banking copilot experience one day?

Carl Ryden

Oh, yeah. I mean, so this is something that by the way, I want to go back and maybe correct you a little bit when you said I was part of a small contingent that came to visit you guys in Cary, when you came, you were part of one of the top banks in the U.S., and I think you guys outnumbered us when you showed up in our cafeteria there. But it was a great fun and great partnership. This is something that's kind of been near and dear to my heart for a long time. I'm an electrical engineer by background, couple undergrad, graduate degree, and actually had done machine learning stuff back before it was really cool, back in the day.

And you can always see this idea of how do we use data in a way that's much more reactive and better than take data, put it in the database, take the data out of that database, generate some reports into an Excel file, take the Excel file, boil it down to a PowerPoint, the PowerPoint then lights up a wall in a conference room or on a Zoom call, and then hope that somehow all of that leads to us doing stuff better. That is kind of a long feedback loop that is fraught with peril, and how do you actually close that? And one of the things we would talk about a lot was when folks would ask us for a report out of PrecisionLender, bankers have a way they love reports and love dashboards and that sort of thing. And I always kind of thought like, OK, when you have a report …

I actually was a consultant with Bain and Company for a long time. We had a partner there who was really, really great that he would have some young analyst who had done some fantastic analysis, and he would say, "If I'm running this business and knowing this perfectly well from this report, what would you have me do differently Monday morning now that I know this?" And that I think is one of these things that draws out kind of the action that you want to translate that data and that insight into an action. And so then you say, well hey, if I generate the report and you generate the PowerPoint and have a meeting and discuss it, what actions would you like to be taken based on that? And often, that is the place, that last mile where it would all fall down where you couldn't translate it into action.

And so what happens is with Andi, we were able to actually, OK, instead of waiting for something to go wrong, instead of mispricing a deal, having it get booked, having it show up on the accounts, then generate a report of, "Here's the deals we booked, and oh gosh, these suck. Now what are we going to do about it?" we can actually get in front of that and say, "Hey, as you're pricing that deal, here's what you can do to make it better." And we actually close that loop and close it in a way that you can actually get in front of it and be proactive about guiding things to a better place instead of reactive around, "Hey, you did something wrong."

I joke all the time about the army of people that banks hire that I think their job is to kind of walk around with reports, printed out reports rolled up and smack people on the head and the hands with those reports and tell them to do better. And I would say, the thing I always leaned into is this idea of coaching. How do you coach people in the moment to do better? And that's the best place to do it in the moment when they're doing something to say, "Hey, you might think about adjusting your feet on your back swing there because this is going to help you this way." Right? Before they do it, as opposed to after the hole's over you go, "I saw you were doing that wrong two holes ago, you should have done better."

Coaching is in the moment, constructive, positive and drives toward different actions. I would say my analogy I always give folks is just yelling to do better is like the fan with the stats who's like, "Score more touchdowns," yelling at the team. It may feel good that he's doing that, but he's not actually changing the outcome. And in fact, he's probably hurting the outcome by doing that if people even listen to them. So the skills framework, we did borrow that, or steal that I would say, from Amazon because it worked really well for them. The idea of how do you extend these things—because that's something banks always wanted to do—is how do we extend it to where we can actually have it reflect our strategies and our personality and our way of approaching our customers and our way of serving our customers? And that idea of how do you handcraft the experience at scale that you want your employees to be able to deliver to your customers was something that was kind of core to what we did.

Alex Habet

It's an amazing journey and certainly the one that a lot of customers out there really appreciate, especially with the ambient value that's generated. Corey, what about you? Tell us about your journey starting from back in the Sensibill days up to today. I imagine you're someone who's fairly used to a rapid pace of innovation, having run a company like Sensibill, employing kind of the bleeding edge technologies along the way. Tell us what you learned along and how is all that work culminating to what you're doing here at Q2 today?

Corey Gross

Yeah, so in a lot of ways, similar to the beginning of how Carl was articulating, you don't just want a human being to fill in data and do work that a computer can do if it's capable of doing it, if you give it the tools to be able to do that. And so in the early days of Sensibill, we were inspired by personal financial management applications, which always promised this. It probably checks all the Carl Ryden boxes. Number one, forcing a human being to do a lot of manual data entry because it couldn't quite categorize transactions properly, so a human was forced to go back and fix all of that, input cash transactions, do all this manual work. And then for all of your hard work, you would get a report which would tell you how terrible you are at managing your money. So it was the double whammy.

And so what we were inspired by in Sensibill is what if we could automate the collection of that data upfront and just do better classifying transactions, classifying the transaction, not just at a high level because people all have a fundamental understanding of where their money goes—25% may go to living expenses, my mortgage, groceries, all of the categorical buckets people already kind of know. And so telling them again what they already know wasn't useful and forcing them to give you extra data to tell them what they already know is annoying. So what Sensibill aspired to do is use deep learning machine learning techniques, training a computer to read data from different places as accurately as could be, and then instead of basically telling someone to do better or to, "Hey, stop spending money on groceries because you're overspending," what can I do with that?

It goes back to, what do I do with the information that you've given me? You could probably coach people along the way, as we learned in partnering with our financial institutions that utilize Sensibill's platform, be able to help people in the moment find solutions, services, financial products that might be a better fit for them based on their own spending patterns and spending behavior. And if you're a small business who used our solution, it was coaching people where they might be able to make adjustments along the way to be able to optimize their business' cashflow and their business' spending. So familiarity with offloading tasks that would drag down a human being to a computer.

So when we got to Q2 and ultimately when the opportunity to start taking some of the techniques and taking some of the learnings that our team had acquired over the course of the Sensibill journey and finding a place to plug in and where it can truly be a force multiplier, we've applied Sensibill tech in multiple places at Q2. And the latest being whether it's we were really good at reading financial documents, reading checks, reading invoices, reading receipts, and so that has great application to things like fraud, check fraud, which we've had experience now doing at Q2. But then when we were thinking of some of the broader applications for this technology in the context of what something like Andi can do.

Well, we've given Andi ears to hear when you type and talk to Andi through a chat interface. We allow Andi to see when it's watching what an RM does through the PrecisionLender interface. What if Andi could see more things? What if we want to give Andi exposure to different types of documents and different types of data and information so that it could better assist RMs throughout the entire workflow of what they need to accomplish? And so we've been able to take some of that Sensibill capability and embed it into Andi's skills toolkit, so to speak, so that we give Andi more opportunity to read and more opportunity to assist from beyond what it's able to do today. So it's been a collaboration between the Sensibill team and the skills and knowledge that it's acquired through its journey, and the PL team for what it's done so magically for so long with Andi and PL.

Alex Habet

It's fair to say then that the combination of the original Andi plus the technologies that Sensibill brought in is essentially providing benefit to the institution by increasing the surface area of where a copilot could help ease the burden of getting the job done, from a banker's perspective, right? You can now pull information from more sources, organize that information, store it, and make use of it in new ways that are only possible through the combinations of some of these new added capabilities. Is that kind of a fair way to put it?

Corey Gross

Right. Yeah. And as Carl said, now we have this gift of large language models that we didn't even have to invest harder in R and D dollars to be able to deploy. And so it's that combination of the framework that Andi had built in order to serve specific needs that is extensible to other applications as we've now utilized at Q2. What Sensibill has been able to build, which allows it exactly as you called out, Alex, to expand the service area of data and of knowledge that we can accumulate to help support an RM through their journey. And then the ability to improve the experience of talking human to computer translation, and then generating from a computer back to human language that can resonate, that a human being can understand and take action on, versus having to go through their own translation of what the computer said back to them. And that's just the real miracle of LLMs, is that now you can ask clarifying questions, you can have context for a conversation, versus just getting a very rudimentary response.

Alex Habet

A couple of days ago, OpenAI had some pretty big announcements, and I'm just singling them out because they're top of mind in AI these days, for good reason. But a lot of people are reacting to this announcement a couple of days ago because here you have the breakthrough moment, which was in end of November 2022 with the initial launch of ChatGPT.

Now the power, it seems to be expanding at a rapid clip on top of that, with their new APIs for developers to build their own agent-like experiences in their own apps and tap into that. You have GPT-4 Turbo, which is a smarter and now cheaper model to leverage. They're adding the ability to, without any need for any code, to create AI agents to tailor to individual needs. And the use case that Sam Altman, the CEO of OpenAI, used was in his travels around the world, he gets often asked by people advice, especially from people who are running early startups, they're asking for advice, "Hey, how do I get my product off the ground? How do I get it to market?" Things like that. And he basically mumbled on stage, said, "Well, that would be a great task that I would love to just have a GPT trained that can help these individuals out."

They also did some new stuff in audio and text to speech, which again, these might be little buzzy things and in some cases they might not apply to banking, but they do potentially increase more surface area of interactions. So help this audience understand, because things are just rapidly coming out now, how does the team quickly weed out the stuff that's useful or weed out the stuff that's not as useful versus the stuff that is, and how does it ultimately become part of the collection of capabilities that would manifest itself as a product that Q2 offers to the clients? Could you just give a quick primer on that evaluation?

Carl Ryden

Yeah, so if I'm OK, Alex, that's good?

Alex Habet

Yeah, yeah, absolutely.

Carl Ryden

So I think a lot of these things will be incredibly useful, and I think I want to put them in the bucket of reducing the friction. And Corey touched on a little bit about all the things you ask a human to do to get value out of a computer. You've got to put the stuff in the system to get the report out so you can then see how you're doing. Asking people to, assigning them task and widening that envelope, the cycle from when you invest in it to when you get value out of it, both as a company and as an individual, is a lot. And a lot of things die under that weight of that waiting for that loop to close. And I think a lot of bankers on the call have probably felt this and IT projects that roll out and they just don't ever ... Where's the ROI? I know the I, I just haven't seen the R yet come back around.

But a lot of those things like text to speech and speech to text and providing more natural human machine interfaces and more conforming to human existence is really wonderful, and that will be valuable. It is absolutely great, but it is not sufficient. And what I mean by sufficient is for a bank, you're in the trust business, so you actually need, I don't need a best-efforts guess at what the answer might be. What I need to know about balance, I need to know about balance. I need to be able to connect these things and ground them into actual data. And so when we first started with Andi back in the day and building skills, we thought the most valuable skills, and a lot of the things a lot of banks got excited about, were these machine learning skills about predicting the utilization of a line of credit, predicting the draw schedule on a construction loan, predicting the next best product and all these things.

And they were valuable, but what turned out to be incredibly valuable were simple heuristics. If you see them doing a deal like this, then suggest they do this. Just reminding folks, and I always used to joke is, a lot of folks thought PrecisionLender, the value of PrecisionLender, was taking the big Excel model they had created to price their deals and moving it into a web-based application. And I said it is that. There's some of that, and that's kind of necessary but not sufficient. But where the real value gets unlocked is taking the PowerPoint that you actually present to your RMs when you have your beginning of the year sales kickoff, off-site, whatever you call it, and you say, "This year to grow the bank, we're going to focus on this, we're going to focus on treasury services and deposits and other things."

That thing, when you get all the RMs in a room at some hotel lobby convention center or whatever, half of them are a little bit of sleep, you tell them this, they're all cheering up and then they go away and they forget it. How do we take that PowerPoint and translate that into the moment? How do we actually remind them of here's the things, here's how we win? And then not just that shared knowledge we get out of that, but also being able to continually harvest that and see what do the best RMs do in a deal like this and look at that and then pattern match that and say, "Why don't I suggest that to this other RM who's pricing a deal?"

And so this idea of a coaching network where you can see what the best people do when they're doing a thing and then harvest that, codify it, learn from it, and then translate it to the rest to where it can actually create different actions and different behaviors, is something that I think is incredibly powerful and was at the core of Andi. We did that a lot with simple heuristics where we had people who'd say, "Hey, look, if they're pricing a deal and it's got this and it's got these particular attributes to it, just do this." And those turn out to be incredibly powerful.

And this is something I tell folks a lot is, I don't think anyone actually really wakes up in the morning and goes, "I want artificial intelligence." I just want intelligence. And if it's got to be artificial, fine. If that's the only way I can get it, then I'll do it artificial. If that's the most scalable, effective way of doing it, I'll do it. I just want intelligence delivered in a way that that intelligence can get translated into differential actions and things that people do differently to win a better deal, to build a better relationship, to strengthen the brand of the bank, to help make our employees more efficient and more effective and help them connect more to their job.

And this is the thing, when I've simplified a lot is I take all these technologies and you put them in the category of, yeah, those are good, and they'll create a sexy demo that will get people excited and there's value in that, but then what's really necessary to deploy this and make it effective within my bank? And I think it is that connection to the knowledge model, connection to the knowledge of the bank, connection to your strategies, your policies, your procedures, your portfolio, your goals, your risk appetite. Styles make fights, right? This is the thing that really matters. How do you take all of that information securely, safely in a way that people can translate that into the moment where it can be most effective and compliant and all those other things?

The other thing I tell folks is this, is the best place, I gave a talk some time ago at BankOnPurpose called "Build Iron Man Suits, Not Terminators." We focus a lot on how do we replace humans, like whole humans? And what a human does throughout a day is their job consists of multiple tasks. There's multiple tasks. And I think trying to replace a whole human is a really hard thing and a risky thing to do, and don't think of it that way. Build an Ironman suit. How do I augment that human and make them better? How do I replace the things?

And the simple way I describe this to a lot of folks is if you look at your people at your bank and you say, for any job like a relationship manager, what do I truly pay them to do? Really what do I pay them to do? What is the highest job that I want them doing? And a lot of times that is managing relationship, having empathy, connecting with customers, having those detailed conversations and connections to the customers where you can actually understand their situation. But then you say, well, now what do they spend their day doing? And what you find more and more, and we've seen this across big banks, small banks, everything, whenever they do a poll of the relationship managers—from the commercial side is the one I'm most familiar with—I think in a lot of areas, is what do you spend your day doing and how much of your day do you spend on non-sales, non-client facing relationship building activities? That number has gone up and up and up and up and up. For some banks, it's like 60-some percent. Two thirds of their time is spent doing things that are non-client-value-added, right? It's internal paperwork for the bank.

And so I always say, what do you truly hire people to do? And then what do they spend their day doing? And what you'll find is they often spend their day doing something other than the thing you truly hired them to do. And how do we offload that to an AI assistant? How do we build them an Ironman suit to lift the heavy things so that they cannot actually think through what needs to happen? And what you end up with is you end up really enhancing the humanity of what you are, which is I think the great way to think about it. But I always say, how do we build something that offloads those things?

And I don't want it to get about saving mouse clicks or because we often get in that kind of silly place, but it really is about not just about saving the mouse clicks where two is less than three, but it's really about opening up the envelope so they can spend more of their day on the things we truly hire them to do, connecting with customers, building personal deep relationships with those customers and understanding of not just their business, but what the business means for their personal wealth and financial wellbeing, and how do we help them make progress in their life.

And I think that's the thing where it is, this is a place where community banks have often carved out a really powerful niche. And I think one of the things that I think is a threat to those folks is the new AI models and things allowed us to ... are going to allow personalization at a level at scale we haven't seen in a long time. And there is a, can the small banks use cloud technology and other things to kind of extend their advantage in this personal relationship business before the large banks can use AI and other things to kind of eat away at their perceived personalization? Which is an interesting kind of race that's going on.

Alex Habet

So just to double back on the question. So we meet with banks and credit unions all the time and we talk about, "Hey, here's our approach, here's how we think that the interactions should be for a banker," and we'll typically get the boilerplate question of, "Well, how is my data being stored? Is it secured?" And that's usually question number one that we get, and it's a given. I mean, the industry that we serve is highly regulated. You can't have an answer that says, "Well, it's loosey goosey security." We have the thoughtful way and approach to storing the information and making sure it doesn't get out of where it's supposed to be. But what are the higher order questions beyond that first one? And maybe that question shouldn't even be the first one. What would you recommend to all those who are listening right now who are probably going to get onto another sizzle demo from a vendor, maybe it's us, maybe it's another one? What question should they ask besides, is my data going to be secure?

Carl Ryden

I think it's also how does this help me compete? And it's not, when I talk about styles make fights, understanding your policies, your procedures, your risk appetite, your portfolio goals, your community, the industries you serve, the thing that I've realized in AI, machine learning and data is as you narrow the context, things get a lot easier. And so the generic bits of... Language models are really good at generically translating one set of language to another set of language, but if you don't start with the right set of language or if you're leaking it from one customer's data to another customer's data or one bank's to another bank's, not only does it make it harder, it also makes it less effective.

Alex Habet

And so how do you make sure that the trust barrier that's so critical to translating knowledge to language?

Carl Ryden

So one of the things that, Corey, you were talking about Sensibill, that I think has been really wonderful to watch with the PL team working with Sensibill something we always talked about when we gave them the nudge and Andi gave them a suggestion, we needed to be able to underwrite that suggestion and back it up and footnote it and say, because ultimately it needs to be trusted. And so, one of the things we did, even at the very beginning with PL with the math is all the math was open. We didn't want to hide it from the RM, right? We don't want to force it upon them, but they need to be able to see and trust that isn't ... there's not somebody behind the scenes with their thumb on the scale kind of making this stuff up, right? There needs to be trust and transparency in that.

It's something that I think Sensibill does really well in that demos one day folks will see is when it actually makes a suggestion, that suggestion, you can actually see where it came from. When it says, "Hey, I read this and your policy says you can't do this sort of thing or you need to do it this way," it will actually reference the section in that policy so you can actually go straight to it and ground it out. And building trust in that interaction is really, really key. And it's actually really hard.

That's kind of the long tail problem of the large language models, is that they're really good at translating generic text and they have generic knowledge about things encoded into their models. But really what you need to do is connect that directly to the facts that are relevant to your particular circumstance and do that reliably, securely, and not just securely like it doesn't leak from one customer to another.

Even within the bank, we have deals at big banks that are material and non-public transactions that can't be seen by other RMs at the bank, who only the deal team can see those. And you have to have this awareness of the promises that we make in terms of accuracy, traceability, trustability, transparency. Because if you don't have that, it quickly erodes. The first wrong answer, all of a sudden.

And what's interesting is not the wrong answer, it's the first wrong answer, of course erodes trust, but what happens is, I know this from the pricing world, it's most valuable when it's at least a little bit controversial. If I tell you to do the thing you were already going to do, it's not very valuable. If I tell you to do the thing you're never going to do, it's not that valuable. But if I tell you to do, "Hey, here's something a little bit outside of what you're going to do."

Alex Habet

"Have you thought of this?"

Carl Ryden

"Have you thought of this?" And you go yes, and you do it, and that gives you the benefit of what you get, then you build that trust and you accumulate that trust. And in some ways that is, I think, at the heart of the experience of banking. When you have a bank that you love, that bank through a million little interactions has kind of built up that trust with you over time. And in some ways, we do that with Andi. You've done it with Sensibill. So when you guys translate receipts or scanned in documents, not only do you tell you here's what it is, but you can trace it back to here's where it came from and you can actually see that it's right. And that is, there's little things you add into that user experience, whether it's internal user, external user, that allows them to ground their trust and build real trust in the things, is probably the most important piece.

It's the stake behind the sexy, it's the actual hard part. And believe it or not, that's something that took us a lot of effort and time to build within Andi and PL, but it is the hard part. And it's the skills, the infrastructure for deploying the skills, for monitoring the skills for, measuring that they're still doing what they should be doing, it's for who gets to use them and how the rights model behind that. And this is kind of all core to kind of both PL, Sensibill, and very much Q2 in the rights model of who can see what data when, and making sure you honor those promises throughout the entire process is super important.

Corey Gross

And so, one of the things that I think just to layer onto what Carl's saying here, in the early days of determining what the FI should be asking as it relates to uses of LLMs or generative AI, it starts with what is the right application of this technology? Because I remember Carl, when we were just kicking off the idea of bringing Andi outside of PL and how would that work and what would it look like, it's really anchoring on use cases that LLMs can effectively support, versus if you want them to play the starring role, you could erode that trust if you don't have a high degree of confidence or if what's required in terms of the output requires a high degree of confidence in order for it to be right in order for it to be a repeatable use case.

And so I think what we're seeing a lot of in the early days, you mentioned kind of off the top, Alex, in a technology hype cycle, great demos are going to happen and a lot of it is going to create this pressure to GPT something. And I think instead of starting with the technology, it's anchoring on what are the problems that we're trying to solve, what are the workflows that could be improved through applications of this? And Andi isn't just an LLM thrown at a problem. It is a framework. It is a collection of technologies and capabilities that an LLM can enhance to make the experience of using Andi better.

Carl Ryden

I always call Andi the intelligence delivery mechanism. It moves intelligence from wherever it exists within the bank to where it can be most valuable, and both in time and space. In this moment for this RM pricing, this deal, this relationship, here's the most valuable bit of knowledge we have within this bank that we can inject into that moment. And here's how we're going to say that it actually creates the most opportunity for it to be adopted and used.

This is something folks have probably heard me say this throughout and use the word “translate.” Now, I say the word “translate” a lot because what I think the way a transformer model works, which is what all these things are under the covers, it is fundamentally a translator. It translates words to words. It can translate English to French or French to German or French to Python code or whatever. It's really good at translating. It has some knowledge embedded into it, a lot of knowledge embedded into it, generic knowledge, because you have to have that knowledge to wield language, but it's really a language translation.

And so, one of the things I like to do is when you think about how to deploy these things, language models in particular, always think about it as you try to restructure the problem as a translation problem. And so if you're using the language model, if you go ask GPT, "Who is president of the U.S.," it's going to give you the wrong answer because when it was trained, somebody else was president, right? That's not up to date. But what happens is when you, and you're seeing OpenAI and others do this too with their version of skills, which are tools and other things, where you have to connect that ability to translate and use language to a source of knowledge, trusted knowledge. And that is kind of the really important part of it is, how do I take this bit of knowledge and translate it to this person in this moment with this problem who's trying to make this progress? And this is where I was like, Andi is really a delivery mechanism where it can deliver that stuff to where it's most valuable within the organization at exactly the right time.

Alex Habet

So Corey, to ask you as you're kind of leading the charge working with some banks to bring the new Andi Copilot to life, the one that has decidedly moved out of just living in PrecisionLender and now can sit across the ecosystems. You might notice for those of you out there that a lot of copilots are popping up. Essentially, a lot of these applications are for the first time getting acclimated with the use of copilots just within their application. This is something that our team has been doing for a long time, and we're kind of onto the next issue, which is how do you have it live across apps? And this is pretty greenfield type of work right now, so help the audience out there understand what's kind of the thought process with working with institutions to bring these technologies to life? How are we partnering with some of these forward-thinking institutions these days?

Corey Gross

Yeah, so as we had mentioned when we announced the Copilot platform launch, we're piloting this with a handful of institutions that are Q2 customers, and we're going to be working collaboratively with them to make sure that what we're delivering, as far as what are the workflows that Andi can actually assist with, map to those problems that are most germane for the relationship manager. We don't just want to have a generic GPT. Andi is not a generic GPT that sits in a browser that you can ask questions of and it'll give you answers based on its limited information set. We want Andi to be able to function in much the same way that it functions for PL customers, which is as an expert, as a coach, as an assistant that can guide a relationship manager throughout a given workflow. And so our process is to spend a lot of time working with relationship managers and the folks who manage and coach relationship managers to make sure that Andi could be a trusted assistant in that capacity.

So that's going to run with product managers and our team working very closely with them into the early part of 2024, so that as we start to iterate, as we learn what use cases are most appropriate for Andi to be able to address, and as we connect it to more sources of knowledge in order to meet those use cases, we'll be readying ourselves to get this rolled out to more and more customers gradually. I think what we want to be doing is taking advantage of the partnerships that we already have to make sure that we're addressing real problems with what we're good at.

This is not going to be a boil-the-ocean approach. This is going to be very targeted toward use cases for certain roles so that we can build them their specific Ironman suits and then earn the right to move to different roles throughout the institution starting within commercial banking, relationship management, move toward the back of the house, move to retail banking, and then on. As Carl said, the ambition here is to get to the front of the experience with account holders themselves. But it's going to be a journey. And as part of that journey, it's about making sure that we're building trust along the way.

Alex Habet

So how should people out there who are interested, who are kind of seeing the light and are interested in reaching out … we do have, of course, beta partners, but betas fill up. How can others stay engaged most effectively with our team?

Corey Gross

Yeah, we're not going to be shy about, so first and foremost, there's a site that I think is somewhere to reference maybe in a chat where folks can fill in details to get more information about the status of Andi Copilot and put themselves on a waiting list for when we open up to more institutions. But we certainly want to share progress of how the pilot is unfolding and share our learnings with our initial rollout. So we're not going to be shy about doing that. And I think that's been the right approach for other copilots that have recently come out. Microsoft has been … they demo, they collect feedback from a handful of customers that they've rolled out their copilot to, and then they put out blog posts and information to Microsoft customers about how the pilots and the work that they're doing with their customers is being trending.

And we will act no different because we'll want to be getting feedback from the market to know that we're doing the right things right, both to earn the trust of folks that we're implementing AI copilots for the first time, but also to make sure that we're covering off the highest-value jobs that we want machines to be able to handle so that ultimately our code, as Carl said, is very clear. We want to make sure that humans are handling the jobs that they're uniquely qualified to execute so that we can hand over the more machine-like tasks to computers. And so the job for us is to roll this out, see that we're doing that capably, and then add more and more to Andi's skillset.

Alex Habet

Alright. Well, you heard Corey mention a few moments ago, you should be seeing in the console, there should be a link where you can sign up, where you can basically stay in touch with what we're up to, get some regular updates and stay engaged with the team. I want to thank both of you for coming on here to share from your experiences. I'm pretty sure that this huge turnout of people are finding it extremely valuable. I also do want to give you guys one more opportunity to provide some parting thoughts in the last couple of minutes that we have here. So Carl, in your infinite wisdom, any last words that you'd like to leave our audience with before we sign off?

Carl Ryden

Yeah. Well, I mean, I do think we focus a lot on how this is going to affect the internal operations of the bank, but I actually think, I don't think it is hyperbole to say that the ability to transform energy into thought and language and intelligence is revolutionary and it's going to change the shape of our society in ways that I don't know that we fully understand. It's going to be a hell of a ride, but it's going to be ... The machine age, if you look at, for thousands of years, GDP per person was flat, and then we basically created machines and we had access to abundant energy to power those machines. So now, the productive power of humans was not limited by the calories we can intake and the sweat of our brow. We could now actually use machines to amplify our effort. And if you look at that, what happened when that took place?

If you look at the plot of GDP for thousands of years, the last 5,000 years, it was flat for 4,000 and then it went vertical. And over the last 100 years, it has gone absolutely vertical in terms of progress. I think right now we have the ability, just like transforming energy into machines, transform energy into physical work, which allows us to amplify the impact humans can have on the world. Now we can transform energy into intellectual work. And I think it's going to have a similar impact on the world. And I say we talk about it from internal to a bank, but the types of companies that banks are going to fund, the types of companies that banks are going to ... how capital is used is going to change. And I think it's going to transform a lot, and I think it's worthwhile to do this just to kind of get a sense and a read on the future that's kind of coming our way because we need to get comfortable with it kind of sooner rather than later.

Alex Habet

I promised infinite wisdom. Carl delivers infinite wisdom. Corey, what about you? Any last words?

Corey Gross

I feel like I shouldn't steal the impact of that wisdom.

Alex Habet

Alright, well, then we'll leave it at that. I want to thank everyone out there who joined us today. Again, it's an incredible turnout. If you submitted a question, we'll reply with a response from one of our experts after the show. Who knows, we may even include some of those questions and answers as part of the podcast replay. If you've enjoyed today's livestream, consider subscribing to The Purposeful Banker wherever you like to listen to your podcasts, including Apple, Spotify, Stitcher, and iHeartRadio. Just search for commercial banking, and we're going to be in the top results. You can also find some QR codes on the screen. That makes it one step easier.

If you're like me and enjoy a video element to podcasts, you can also catch our show on YouTube, which we recently launched with all the episodes from the past year and a half. So there's plenty to binge on, if that's your thing. And if you have a moment to spare, let us know what you think in the comments. Until next time, this is Alex Habet signing off, and thank you for watching today.