Why Off-the-Shelf AI Doesn't Understand Money — Udi Menkes, Intuit

AI Engineer · 19 min · 190 sentences · from YouTube's caption track

Each timecode opens YouTube at the start of that sentence. Line anchors (#s42) are the cue ids in the WebVTT, and every line carries its start and end seconds. All transcripts has every talk, and the whole corpus as one file.

00:12So, I have a three-year-old daughter and she's abs absolutely adorable and the parents here in the room know how insightful that age can be and she has a complete theory
00:25about money by now.
00:27And I'll give you an example.
00:28So, a couple weeks ago, I was driving the car.
00:31I was coming into park and there was something in my dead end and I scratched the car.
00:36I went out.
00:37I'm like, "Oh man, I can't believe I scratched the car."
00:39And then I hear my daughter from the back and she was like, "Daddy, what's happened?"
00:44And, you know, I'm explaining it to her and then she says, "What's the problem?
00:49Just buy another one."
00:52So anyway, um I want to ask you today with a raise of hand, who uses LLMs, has used LLMs for getting financial advice, a recommendation on something in the financial world.
01:08Great.
01:09Almost everyone.
01:10Wait, keep your hand up if you trusted the answer and you actually followed the advice.
01:19Okay.
01:20a lot of hands are going down and that's the core problem.
01:25So I had the same thing a couple a couple uh months ago.
01:28I had a big decision I was looking to take.
01:31Should I invest in you know real estate niche or in the stock market niche on a specific area and you know I do AI for finance for a living.
01:41So I went the full-blown way.
01:44context, brain, um, the books, the knowledge, all my finances combined, all the latest models, and it gave me a recommendation.
01:53You should do A with great reasoning.
01:56And then I changed just a little bit, one of the assumptions, and it completely flipped.
02:01You should do B, never do A. And then I tweaked one more small thing, and it went all the way back to A. And at that moment, I understood
02:12that the advice sounds good.
02:14It sounds sound, but I can't really trust it.
02:18And by the end of the talk today, you will understand why off-the-shelf LLMs don't understand money and what you need to do about it.
02:32So, I'm going to show you a couple of real examples from a study we're doing at into it on thousands and thousands of businesses around a 100,000 situations and time frames.
02:44This is an example of a small business, a new landlord that is building a rental property business.
02:50His first property and he's down.
02:53He's he's in negative cash flow.
02:55there's a open loan and the profit is basically trending into the red and a question comes up.
03:02How do I improve my profit?
03:04And a frontier model gives the following response.
03:09Go and acquire a second rental property because that'll bring more income and compensate for the deficit.
03:17And that model had all of the business's data.
03:21Now that's very risky for someone in the negative in the red to be doing.
03:27On the other hand, a model that is grounded in real outcomes and what I mean by that is a model that has seen similar situations of such businesses what they did and what was the outcome
03:40actually recommended to raise prices on the existing tenant by 5 to 10%.
03:46And to do it before the renewal.
03:49Now, some of you are thinking that it's just a matter of context.
03:54Just give it more context.
03:56And the thing is that this advice is coming based off on real situations of similar businesses and would actually move them into profitability in this case.
04:10And this is not a one-off.
04:11I can go on and on showing you a lot of examples.
04:13This is second one.
04:14This is an egg supplier where one customer is 70% of the revenue and one vendor is almost all of its cost.
04:23Same question, how do I improve my profit?
04:26A frontier model says raise your prices 15 to 20%.
04:34Now you understand that is very risky because you might lose almost all your revenue.
04:40On the other hand, the same grounded model in real outcomes went actually to the cost site and recommended to negotiate um negotiate the vendor cost pricing for a 5 to 10% reduction.
04:55So it went to the cost side.
04:56It took into account the constraints.
04:59So what we saw are two examples of frontiers.
05:03I'm talking about leading models in the world today that had all the business context actually give advice that could be very harmful for the business.
05:14And that's what I call the fluent bluff.
05:19The fluent bluff is a generic fluent and confident answer that frontier LLMs can give you around money because of what they learned on the internet, blogs, books, advice columns,
05:35what people wrote about money, but not based on what actually happened.
05:42And I would argue that almost every answer that you see related to money and finances is such and we're soon going to release re the research I talked about but I'll give you
05:56a highlight from there across these 100,000 businesses in time frames 40% of the time the essence of the advice that the frontier models were giving was acquire a new customer
06:11which which is you know everyone would wish they could do that and 14 more additional percent were increase basically the revenue from your product.
06:23So combined more than half of the essence of the advice that was given by the frontier models was acquire new customers and try to increase revenue from the profit.
06:36And this is not just me saying this is a very interesting research coming that just came out a couple weeks ago from researchers at Princeton.
06:44What they tried to do is simulate and answer the question can the leading models drive long horizon business decisions.
06:54And what they did is they gave the models a harness with tools and data and everything they would need to take decisions across a simulation of 500 days.
07:04Can they turn a profit?
07:06Each model got a million dollars to start with and guess what happened?
07:13Most of the models drove the company bankrupt and it didn't even take 500 days.
07:20And the interesting part is that they also ran a simple rules-based system and that rules-based system outbeat almost all of the models.
07:31Even the very very few models that were able to generate some profit, it was in a specific in instance and when you rerun that they also actually drove bankruptcy.
07:43So how can it be that a simple rule-based system outbeats the frontier models today on real business decisions?
07:55And here's what I want you to think about.
07:58A frontier model has read about money, but a grounded model in real outcome has actually watched what happens.
08:10And let me be precise with my argue here.
08:12Even if you take all of a company's data and for example we at into it have all the financial data from QuickBooks for example the general ledger the P&L the cash flows everything the invoices of the business
08:24and you give it to a frontier LLM it's still just one group of data points on a company and that's a difference between sounding right and actually being right
08:38so I'm Udy Menquez and I've been in the AI and finance world for the past 15 years.
08:44I started in the AI science world leading AI and data teams and shifted into product management becoming an about four years ago an AI princ AI product manager at in it long before by the way it was cool to become an AIPM
09:00um and today I'm a principal product manager at into it.
09:03lead financial intelligence and advisory systems that help into its customers to take better decisions and grow their business.
09:12And the question that I fixate on on a daily basis is not which model is the best now that I I can use.
09:21It's what do we fundamentally have that no model access can replicate and how can we can I transform that into AI native transformative and delightful experiences for our customers.
09:37Now I want to develop some intuition for from three different angles on why these models bluff.
09:43So the first angle is around context is not experience.
09:48And I'll give you another real example.
09:50This is an apparel company where 80% of the cost is coming from this one vendor.
09:58And the textbook answer is go cut your biggest cost.
10:02Right?
10:02Textbook book answer.
10:04But the issue here is you cut this cost.
10:07That same vendor was actually enabling the generation of 97% of that company's revenue.
10:14So cut that biggest cost and you lose almost all the revenue.
10:19And that can be great margins on zero dollars.
10:25And think about it.
10:27If I give you an option to work with two two different adviserss, one adviser is a very experienced one, years of experience working with businesses guiding them and another
10:40adviser which is very very smart.
10:43They know all the textbook.
10:44They're fresh.
10:45They read everything.
10:46They know AI in and out.
10:48but you don't have experience, I would bet you would always go with the experienced one.
10:54And that's the same thing with the models in what I just showed you.
10:59And it turns out experience is very hard to measure.
11:03And I'll give you an example.
11:04So let's look at a restaurant.
11:06A restaurant, let's say, raises price and after 6 months becomes much more profitable.
11:13Is it because they raise prices or is it because they're just naturally successful?
11:20And the challenge is obviously you can't run the business twice, right?
11:24So what you do is you take two groups of similar companies, similar businesses that have the same propensity to raise prices, the same likelihood to raise prices.
11:35One group raised prices while the other didn't.
11:39And then we look after some time at the results.
11:41So, the group that raised prices actually gained $4,200 a day profit.
11:47And the group that didn't raise prices actually gained $2,800 a day.
11:53So, the question is, what is the impact of raising prices?
11:58So, a naive answer would be the difference, right?
12:011,400 is the impact of raising prices.
12:04But actually, you need to account for the fact that the companies that raise prices are actually naturally more successful businesses, which is also why they could raise the prices.
12:14So, the real difference is more like $1,150 for this illustration.
12:21And we measure the impact of actions on the outcome through a measure called Kate, conditional average treatment error, which looks at that connection.
12:33And here's what I want all the AI and finance leaders here in the room to pay attention to.
12:38So find where you can see a lot of different situations across entities that you have in your data.
12:45In our case, it's businesses what they did and verify the outcomes and how things turned out.
12:53If you can see that in the data because that's the one thing that Frontier off-the-shelf models do not have.
13:01And don't get me wrong, Frontier models are amazing and we actually use them.
13:07And I'll I'll show you how we use them.
13:09So, we use them to generate hypothesis candidates for actions we would suggest a business to do.
13:17But then we would use a model that we trained using reinforcement learning in order to figure out which one of those is actually a right move to do versus a mistake that could drive the business
13:30down.
13:31Now how we do it a little bit into our approach is we look at what we call we actually create from the data millions of business trajectories.
13:41So we have data add into it across our products QuickBooks, Turboax, Credit Karma, Mailchimp.
13:48So think about a business and the financial data.
13:50There's the general ledger, the P&L, the cash flow statements, the invoices like I mentioned before.
13:55So we take all of that data and we create what we call business states.
13:59A state of a business is think about a very detailed summary at a given point of time and then we derive actions.
14:08So for example in your general ledger I can look and see that you have invested in a campaign in a marketing campaign or you paid someone.
14:17So I know you're paying payroll.
14:18I know how much um um your hiring costs are and so on.
14:22So we derive all of these actions and we look at what are the outcomes in different time frames and outcomes can be increase in profit and revenue in cash flow
14:33in time combination of those.
14:36So we create millions of vectors of state, action and outcome and then we train an R model to be able to understand in given situations of similar businesses which actions
14:49lead to the best outcomes.
14:52And then the third step is we actually train an LLM to be able to generate that better advice.
15:02That's where opinions are going in and evidence is going out.
15:06So the model you saw in the examples at the beginning were actually a model that we developed with researchers at into it across millions of small and medium businesses.
15:17And we actually tested it head-to-head with all of the leading models in the world.
15:22And we were able with a midsize cheaper model to outperform the frontier models because of the grounding that I just showed you.
15:33And the interesting part as a product person you would think that it's all about the model size and the bigger and better model obviously I would have a lot better chance.
15:41But it doesn't turn out to be true.
15:43And the moat here is that it's not about the model access, it's about the data itself that you have.
15:53And then we went ahead and built an experience out of it.
15:56And this is an AI business advisor that is currently in beta with research in a research preview with our customers where we use the LLM that I just described that we created
16:07to proactively raise opportunities for businesses at every given point of time.
16:12Here's what you should do.
16:13Here is why.
16:14Grounded in who is like you, who's similar to you, what they did, and why we're actually recommending you to do it.
16:21And you can drill down into it, understand, and create action plans that will lead to your business actually growing in the right direction.
16:31Now, I want to take a step back and zoom out because this isn't just about money.
16:37We are entering the era of outcomedriven AI.
16:41And the question stops being which model is better and starts becoming how can we steer AI to actually make it achieve the outcomes we want in our domains.
16:53And it doesn't matter if you're building an anti-fraud system or a health care system, logistics developer tools.
17:02The winners in my opinion are going to be those with the best system of records, creating unique data sets out of them and then training the models to achieve
17:13the outcomes.
17:17And as the product person here, it's not just about the science.
17:20The science is very important.
17:22But a great adviser, think about the great adviserss and mentors that you had in your life.
17:27They understand you, right?
17:30They understand your preferences, what you like, what you don't like.
17:33So, a great advisory experience needs to have two things.
17:38It needs to have great grounded science, the best science, but also it needs to understand you, what you prefer, and it needs to even make you feel as if you were part of the decision
17:50to create a trusted experience.
17:55So, three things I want you to remember today.
17:58Every model has read about your domain, but none has actually watched the plays and the moves and their outcomes.
18:09And that gap is the whole game.
18:12Now, in coding agents and coding models, we're seeing it very advanced.
18:16A lot of verified outcomes and creating models that actually lead to better outcomes in coding, but it's very much unexplored in the financial domain and in other domains as well.
18:27And you don't close the gap with bigger models.
18:31You close the gap with experience, embedding experience into the model by looking at verified outcomes in your data.
18:40What actually worked at scale.
18:44So off-the-shelf models don't understand money, but grounded in real outcomes, it does.
18:52And that's what we were able to figure out.
18:56So, here's the one thing I want you to do tomorrow.
19:00Well, actually, you know what?
19:01Go ahead and enjoy Fourth of July weekend.
19:03But right after that, look in your data where you can see situations across entities and outcomes you can verify.
19:15Get deep into that data and start grounding your AI in that.
19:19Think about those angles and that's for you to build.
19:24Thank you very much.
19:26Thank you for listening to me.
19:28Happy to connect LinkedIn, Twitter, in the hallway.
19:31Thank you very much.