WEBVTT

NOTE Sentence-level transcript of https://www.youtube.com/watch?v=T-42NrYjwxM

NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/T-42NrYjwxM.html. A cue ends where the next begins, or 2 s after its last word.

s1
00:00:12.720 --> 00:00:15.560
Hey everyone, I'm Alex from Ramp.

s2
00:00:15.560 --> 00:00:20.600
Um I'll be speaking about chasing capabilities while waiting for the frontier to advance.

s3
00:00:20.600 --> 00:00:22.720
So, quick uh intro about myself.

s4
00:00:22.720 --> 00:00:34.960
I joined Ramp around 3 years ago uh on the broader applied AI team working on a bunch of different surfaces and products applying LLMs and ML to make a great product even better.

s5
00:00:35.120 --> 00:00:43.040
Uh to and to to caveat my presentation on Ramp Labs, Ramp is an extremely bitter lesson filled company.

s6
00:00:43.040 --> 00:00:48.680
Over the last year, the number of use cases that use LLMs has increased 123%.

s7
00:00:48.720 --> 00:01:00.760
While at the same time, a lot of our top features are growing in token spend, but on a unit economic basis, they're actually becoming cheaper because there are frontier models that are being released

s8
00:01:00.760 --> 00:01:05.840
uh all the time now, and these models are cheaper, faster, uh and better.

s9
00:01:05.840 --> 00:01:11.440
And so, we're able to provide the same level of accuracy at a much lower cost.

s10
00:01:11.440 --> 00:01:15.640
And so, this is a really bitter lesson filled uh pattern.

s11
00:01:15.640 --> 00:01:22.000
And so, given all this, a question that I sometimes get, why not just wait?

s12
00:01:22.000 --> 00:01:29.360
Why have an applied research function inside of a company like Ramp, an app application layer company, right?

s13
00:01:29.360 --> 00:01:35.320
Um and to answer that, I would like to draw draw a parallel to 2023.

s14
00:01:35.400 --> 00:01:42.800
Back in 2023, uh you had access to kind of like GPT-4, and creating an agent was a lot of work.

s15
00:01:42.800 --> 00:01:47.040
Um you would have to create very, very strict guardrails.

s16
00:01:47.040 --> 00:01:49.000
Models weren't allowed to backtrack.

s17
00:01:49.000 --> 00:01:50.440
They didn't have thinking tokens.

s18
00:01:50.440 --> 00:01:57.480
And so, if it went in the wrong direction, it would just kind of charge ahead and not realize that it made mistakes.

s19
00:01:57.480 --> 00:02:09.972
You had to sweat over all of the prompts where to the point where um s- some synonyms could change the behavior of of your agent because the they weren't RL'd properly on

s20
00:02:09.972 --> 00:02:12.452
generalizing to the user's intent.

s21
00:02:12.452 --> 00:02:20.772
You had to reduce and scope down the context as much as possible because giving too much context to the agent would just confuse it.

s22
00:02:20.772 --> 00:02:25.732
And even back then, we decided, well, we need to ship these things.

s23
00:02:25.732 --> 00:02:29.732
We can't wait for GPT-4.5 or GPT-5 to come out.

s24
00:02:29.732 --> 00:02:32.812
We need to serve our financial professionals now.

s25
00:02:32.812 --> 00:02:39.532
And we shipped things like Ramp Tour Guide and many other agents even though it was not the easy thing to do.

s26
00:02:39.532 --> 00:02:43.572
And fast forward now to 2026, building an agent is a lot easier.

s27
00:02:43.572 --> 00:02:54.692
The models have gotten so much better that you can kind of give them MCPs and access to kind of any context and they do a very very good job of understanding what they need to do given a task.

s28
00:02:54.692 --> 00:03:02.252
But, I still believe that we can dig a layer down and create even better experiences for our users.

s29
00:03:02.252 --> 00:03:06.372
It's just it requires a bit more effort now.

s30
00:03:06.452 --> 00:03:16.372
So, to present this uh some of our work from Ramp Labs, three specific cases of how we've applied research into Ramp itself.

s31
00:03:16.372 --> 00:03:20.732
Starting with creating a specialist model.

s32
00:03:20.732 --> 00:03:28.412
So, in November of last year, we launched Ramp Sheets, which was one of the really early uh spreadsheet agents.

s33
00:03:28.412 --> 00:03:35.492
This was before the two frontier labs had launched their versions of spreadsheet agents and we had a very good launch in November.

s34
00:03:35.492 --> 00:03:39.852
A bunch of users from great companies had signed up and started using it.

s35
00:03:39.852 --> 00:03:46.372
And we realized that well, this is a very general purpose spreadsheet handling agent.

s36
00:03:46.372 --> 00:03:57.652
And a lot of the work that people do on Ramp both inside of the company, but also all of our customers accounting and finance professionals are spending most of their days inside of spreadsheets.

s37
00:03:57.652 --> 00:04:08.574
So, we can kind of take the spread the harness and the brains behind Ramp Sheets and added into a bunch of other um surfaces within Ramp.

s38
00:04:08.574 --> 00:04:18.094
And as we were building this out, we realized, well, we can probably apply some research uh from Ramp Labs to make this even better.

s39
00:04:18.094 --> 00:04:22.934
And we sat down and we started looking, where do models fall short on this?

s40
00:04:22.934 --> 00:04:37.374
And back then, this was like Opus 4.6 and the place that had the most friction was when you would drop in a lot of spreadsheets into Ram Sheets and you would ask it to build a DCF or or model some financial scenario,

s41
00:04:37.374 --> 00:04:47.614
it would spend a lot of time creating a plan at first and then researching all of the sheets that it had access to to try to get a good understanding of and lay of the land.

s42
00:04:47.614 --> 00:04:54.854
And Opus 4.6 is a great model, but it was like pretty slow and the thinking tokens are also pretty expensive.

s43
00:04:54.854 --> 00:05:05.414
And so, we we took a step back and and decided to train a retrieval model or kind of like a financial question-answering model that would while still keeping these frontier models

s44
00:05:05.414 --> 00:05:16.414
as orchestrators, would then be handed off to the this much faster model that would do uh financial question-answering for that planning agent.

s45
00:05:16.414 --> 00:05:30.974
And so, we partnered with the company Prime Intellect to uh post-train a Qwen 3.5 uh 35 billion um model and we actually got above Opus 4.6, which was the frontier back then

s46
00:05:30.974 --> 00:05:37.534
uh exact task accuracy, while also being able to serve it at high cool uh level latencies.

s47
00:05:37.534 --> 00:05:47.734
So, a much faster, but just as smart model that could then be used in uh in tandem with the really smart frontier orchestrator.

s48
00:05:48.014 --> 00:05:57.974
Um and so, this was like a very successful um example of post-training that we had actually sent to prod as part of our Ramp Labs effort.

s49
00:05:57.974 --> 00:06:12.015
Uh the second uh big thing that we are actually exploring up to this day is uh KV cache tricks and particularly cartridges are really interesting um family of techniques where

s50
00:06:12.015 --> 00:06:21.775
you're able to compress a lot of context up to 20 to 30 times more context into the KV cash offline and serve it for your agent.

s51
00:06:21.775 --> 00:06:30.495
And so if you compare it to the RAM sheets example, for example, um uh there's a lot of tasks that you know are going to happen at some point, right?

s52
00:06:30.495 --> 00:06:39.215
Like month-end close, a month finishes and on the first of the next month a financial professional will go and try to reconcile and close their books.

s53
00:06:39.215 --> 00:06:44.055
You know kind of the ledger, the aging report and the documents that they will need to be retrieving.

s54
00:06:44.055 --> 00:06:53.015
And so you could prepare them as context, but the agent will still need to agentically retrieve and and create a good picture and understanding of them.

s55
00:06:53.015 --> 00:07:03.135
With cartridges, we're able to compress all of that work and build one overnight before the the financial professional starts their work on the first of the month.

s56
00:07:03.135 --> 00:07:12.295
And once they open up the the the agent, it knows exactly what it needs to do because it has all of the the context already uh built in.

s57
00:07:12.295 --> 00:07:20.055
Cartridges are also a much lower lift in post training because the way that they do self-play is kind of a lot more generalizable.

s58
00:07:20.055 --> 00:07:29.415
The caveat with stuff like this is that compared to post training, this is something that the labs might ship at some point is in their APIs and their foray into

s59
00:07:29.415 --> 00:07:31.335
uh continuous learning.

s60
00:07:31.335 --> 00:07:39.455
And even if they were to do it in the API, we would actually be happy because we are developing a lot of this tribal knowledge inside of the company

s61
00:07:39.455 --> 00:07:42.215
um and just learning how best to apply these techniques.

s62
00:07:42.215 --> 00:07:49.455
So even if it does ship as an API, we can just hit the ground running on day one knowing how to use them.

s63
00:07:50.735 --> 00:08:02.455
And finally, the third example uh of applied research that we are sending to prod is our work on uh agentic map reduce and kind of agentic swarms.

s64
00:08:02.455 --> 00:08:12.697
So more concretely, we published an article on the RAMP Labs uh blog on how we uh used 10,000 parallel agents to pen test all of Ramp.

s65
00:08:12.697 --> 00:08:23.697
Um so at Ramp we have a uh cloud agent called Ramp Inspect, which helps out with setting up all of the infrastructure and plumbing for something as ambitious as this.

s66
00:08:23.697 --> 00:08:36.937
And we were able to spin up 10,000 parallel agents and then kind of map this work out, try to find any vulnerabilities, and then also agentically reduce on the tickets that they would create.

s67
00:08:36.937 --> 00:08:46.017
So deduplicate them with LLMs and then try to reproduce them adversarially with LLMs to try to make sure that the vulnerabilities are real and important.

s68
00:08:46.017 --> 00:08:52.657
At the end of all of this, we have actually found seven high vulner- high severity vulnerabilities that we all patched.

s69
00:08:52.657 --> 00:09:01.177
Interestingly, one of the um vulnerabilities that we describe in that article is a Lua script injection that you could send over via email.

s70
00:09:01.177 --> 00:09:05.297
And no single component of that stack was unsafe.

s71
00:09:05.297 --> 00:09:15.657
And the kind of swarm had to patch together multiple components and understanding of multiple components end-to-end in order to be able to figure this out.

s72
00:09:15.657 --> 00:09:19.977
And the other nice thing with this is that this is all running automa- automatically.

s73
00:09:19.977 --> 00:09:29.617
You don't need to spend anyone's time, it's just you start it and the only thing that uh failed failed passes that get filtered out is the cost of the tokens.

s74
00:09:30.337 --> 00:09:41.857
The other thing that's interesting outside of just building out all of the infrastructure to support these swarms is that we do research into how to best use models as part of these swarms.

s75
00:09:41.857 --> 00:09:52.817
One of the interesting insights from that same article that we published is that you can actually use open weight models that are up to five times cheaper compared to the frontier reasoning.

s76
00:09:52.817 --> 00:09:58.217
And because you're set- setting it up as a swarm, the detection rate is not five times lower.

s77
00:09:58.217 --> 00:10:05.977
So you can kind of spend more on these cheaper tokens and have almost as good of a detection rate.

s78
00:10:07.577 --> 00:10:19.018
Um if you follow through all of these examples, you kind of notice a pattern of a lot of them is attempts at saving on time and saving on money.

s79
00:10:19.018 --> 00:10:23.578
So, reducing latency, which is time, and then reducing on token cost, which is money.

s80
00:10:23.578 --> 00:10:28.178
That's actually Ramp's uh slogan, uh saving our customers time and money.

s81
00:10:28.178 --> 00:10:36.738
Um and so, I'd like to to point to two uh products that we actually offer from Ramp that have been um influenced by Ramp Labs' research.

s82
00:10:36.738 --> 00:10:47.018
The first one is token spend management that connects to a bunch of uh your API keys and is able to give you a really good picture of how LLMs are being used inside of your company.

s83
00:10:47.018 --> 00:11:01.018
Not only token spend on an API key basis or a user basis, but digging deeper, accounting categories, um and even things like uh tax credits that you can get by

s84
00:11:01.018 --> 00:11:03.178
uh analyzing the spend.

s85
00:11:03.178 --> 00:11:18.738
It also goes deeper and we create a taxonomy that uh contextually analyzes and semantically analyzes the LLM usage, so you can actually understand how LLMs are being used outside of just attributing it to a team or a person inside of your company.

s86
00:11:18.738 --> 00:11:29.978
And then the second one is Ramp Router, which also leans into improving your the latency of your agent, where we have a bunch of routing strategies that we work with Ramp Labs to develop,

s87
00:11:29.978 --> 00:11:40.378
uh things like Nvidia Switchyard, things like smart flex offloading of work to both improve latency and um cost per tokens that you spend.

s88
00:11:40.378 --> 00:11:47.498
And there's a bunch of different strategies already available for the the for routing, and even more coming out later.

s89
00:11:47.498 --> 00:11:54.738
So, if you're interested in any of these two products, you can take a photo uh of the URLs or or scan the QR codes there.

s90
00:11:54.738 --> 00:11:56.978
Give you a second.

s91
00:12:02.658 --> 00:12:03.818
Awesome.

s92
00:12:03.818 --> 00:12:15.819
And then I I'd also invite you to check out our Ramp Labs uh blog, where we publish um a bunch of really interesting articles that I haven't had the chance to cover as part of this presentation.

s93
00:12:15.819 --> 00:12:24.659
Um notably one of the really interesting ones is portal where we figured out a way of uh transferring learnings from one post train to another.

s94
00:12:24.659 --> 00:12:34.979
So, you could do something like a Laura uh post train on one model, so like the Quen 3.5 that I was describing, but then transfer it to newer versions of Quen

s95
00:12:34.979 --> 00:12:37.259
or even across different families.

s96
00:12:37.259 --> 00:12:47.059
So, you're post training with this latent that then can be projected onto different uh models and that really amortizes the cost that you have to spend on post training.

s97
00:12:47.059 --> 00:13:00.739
Uh more blog posts on agentic map produce, so not only the pen testing stuff that we did, but also things like um analyzing and creating taxonomies of LLM usage, which is stuff that has gone into the token spend product.

s98
00:13:00.739 --> 00:13:14.739
Um things like self-improving monitoring where we use uh LLMs to automatically um create monitors with DataDog on a lot of our product services and then as the words come in, improve those monitors automatically

s99
00:13:14.739 --> 00:13:21.619
without a person in in the loop to try to basically create super super robust and resilient software.

s100
00:13:21.619 --> 00:13:32.899
Um and finally semantic tagging, which is this interesting uh foray into creating this um agentic map produce over a bunch of user sessions where we can annotate and create taxonomies

s101
00:13:32.899 --> 00:13:34.499
that are open-ended.

s102
00:13:34.499 --> 00:13:46.619
So, not knowing um ahead of time which what are the clusters of LLM usage and trying to create a taxonomy that can then be reconciled against the spend inside of the companies.

s103
00:13:46.619 --> 00:13:55.859
So, if uh if you're interested in this uh type of research, x.com/ramplabs where you can uh go and and check out more of our research.

s104
00:13:56.419 --> 00:14:06.379
And uh there's my Twitter if you have any other questions or uh want to talk about some use case as part of your applied research.

s105
00:14:06.499 --> 00:14:07.379
And that's it.

s106
00:14:07.379 --> 00:14:09.544
Thank you.

s107
00:14:09.544 --> 00:14:11.544
[applause]