WEBVTT

NOTE Sentence-level transcript of https://www.youtube.com/watch?v=VjEP0xqTUI0

NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/VjEP0xqTUI0.html. A cue ends where the next begins, or 2 s after its last word.

s1
00:00:01.309 --> 00:00:03.309
[music]

s2
00:00:12.920 --> 00:00:14.680
Yeah, I really appreciate everybody showing up.

s3
00:00:14.680 --> 00:00:17.200
Uh, as Madhu mentioned, my name's Armon.

s4
00:00:17.200 --> 00:00:22.080
I lead our product and sales-led growth engineering teams at Ramp.

s5
00:00:22.080 --> 00:00:27.720
Um, and today I'm going to talk to you about, uh, the building blocks of go-to-market orchestration.

s6
00:00:27.720 --> 00:00:33.160
And um, to kick it off, like, what do I mean by go-to-market orchestration?

s7
00:00:33.160 --> 00:00:39.040
Effectively, like, what we're building towards is the ability to just describe a motion, right?

s8
00:00:39.040 --> 00:00:44.440
Whether it's like playbooks or experiments or like evergreen campaigns that you want to run.

s9
00:00:44.440 --> 00:00:50.800
And how those get like distributed across the channels through which you actually execute your go-to-market, right?

s10
00:00:50.800 --> 00:00:54.160
Whether it's outbound or ads or web or whatever.

s11
00:00:54.160 --> 00:01:00.000
Um, we want the ability to kind of describe this and automate that output.

s12
00:01:00.000 --> 00:01:10.200
And this really started a few years ago where we kind of noticed that uh, there's a ton of great ideas, you know, like everybody across product and data and engineering and go-to-market

s13
00:01:10.200 --> 00:01:13.080
have like really good ideas for things that they want to do.

s14
00:01:13.080 --> 00:01:16.400
And the bottleneck is kind of like everything after that, right?

s15
00:01:16.400 --> 00:01:18.560
How do you go pull an audience to go and target?

s16
00:01:18.560 --> 00:01:27.520
How do you go and convince a bunch of people to like um, abide by whatever strategy that you've come up with or playbooks or enablement materials.

s17
00:01:27.520 --> 00:01:32.280
Um, and we wanted to try to aim to uh, reduce that coordination cost.

s18
00:01:32.280 --> 00:01:43.920
So, there's parts of this where we could see it as like an engineering problem, even like a few years ago, just go and create like a consistent data substrate, go and like federate that across the different systems through which you,

s19
00:01:43.920 --> 00:01:45.840
uh, run your go-to-market.

s20
00:01:45.840 --> 00:01:54.680
And obviously in the last few years, agents have really like deepened our ability to go and like push the level of automation that you can do on behalf of operators

s21
00:01:54.680 --> 00:01:59.000
like as close as possible to those points of execution.

s22
00:01:59.000 --> 00:02:03.000
Um So, like really specifically, uh I'm a golfer.

s23
00:02:03.000 --> 00:02:11.480
Suppose I want to offer golfers at uh East Coast construction companies an incentive to like try Ramp, talk to sales, whatever.

s24
00:02:11.480 --> 00:02:18.760
Uh and we want to be able to go and spin up an audience of uh golfers at East Coast construction companies, spin up like an incentive.

s25
00:02:18.760 --> 00:02:29.280
Let's go offer like some Pro V1 golf balls to uh these people, go create like outbound sequences, generate the copy, generate uh creative for paid ads and for web,

s26
00:02:29.280 --> 00:02:37.000
maybe show some in-app notifications for your customers, and do all of that seamlessly by just describing the intent, right?

s27
00:02:37.000 --> 00:02:40.320
And probably more than just this one sentence.

s28
00:02:40.320 --> 00:02:45.760
Um So, a few years ago, we kind of identified uh a few fundamental challenges here.

s29
00:02:45.760 --> 00:02:52.880
Um As was previously mentioned, uh the necessary data for this was just messy, inconsistent across systems, right?

s30
00:02:52.880 --> 00:03:04.320
Everybody's operating off of a different uh source of truth, and that makes it like effectively impossible to go and distribute some coordinated action across these different go-to-market teams and channels.

s31
00:03:04.320 --> 00:03:08.519
Uh The next is that like reps were just buried in busywork, right?

s32
00:03:08.519 --> 00:03:14.680
Even if like you have the best intentions, I want to go and like run this campaign, uh I want your help doing it.

s33
00:03:14.680 --> 00:03:19.280
The reality is that like uh our sales teams are in back-to-back-to-back-to-back meetings all day.

s34
00:03:19.280 --> 00:03:30.840
They're outbounding, they're selling, and um the operational burden of like doing everything in between sales was just really high, uh which made kind of like really scaling out experimentation

s35
00:03:30.840 --> 00:03:33.360
and creativity challenging.

s36
00:03:33.360 --> 00:03:38.040
Uh And similar to that, just the coordination and distribution are expensive, right?

s37
00:03:38.040 --> 00:03:39.720
If you're like, "I have this idea.

s38
00:03:39.720 --> 00:03:42.280
I'm going to go write this like proposal, this enablement material.

s39
00:03:42.280 --> 00:03:45.880
I'm going to go try to like convince a bunch of people to go and use all of this."

s40
00:03:45.880 --> 00:03:52.720
That's just like a really challenging thing to do on any like pace that's not on the order of like months.

s41
00:03:52.720 --> 00:03:59.640
Um So, um over the last few years we've been trying to solve this problem from the ground up, right?

s42
00:03:59.640 --> 00:04:09.320
How can we start with that uh ingestion and consistency problem uh and data quality, which is just like, you know, on the road map every quarter.

s43
00:04:09.320 --> 00:04:22.800
Um how can we then go build those vertical efficiency and growth levers, uh saving people time uh in like managing operations and execution, uh as well as like how can we improve conversion rates, make people more performant

s44
00:04:22.800 --> 00:04:28.680
by being able to kind of scale some of these more like uh informed and personalized and creative strategies.

s45
00:04:28.680 --> 00:04:31.240
And then how can we extend this horizontally, right?

s46
00:04:31.240 --> 00:04:35.000
Teams have very common workflows at some level, right?

s47
00:04:35.000 --> 00:04:37.760
Everybody wants to outbound, everybody has meetings.

s48
00:04:37.760 --> 00:04:42.919
Uh how can we go take the patterns that we build for one team and start to just mirror it to others?

s49
00:04:42.919 --> 00:04:47.720
Uh and now kind of where we're at is like this distribution and coordination problem, right?

s50
00:04:47.720 --> 00:04:55.520
How can you go and execute across multiple channels simultaneously through just like the description of intent?

s51
00:04:56.080 --> 00:04:59.960
So, yeah, I'll get into the building blocks.

s52
00:04:59.960 --> 00:05:03.880
Really broadly, uh go-to-market agents are complicated.

s53
00:05:03.880 --> 00:05:07.160
Um in order to do this effectively, right?

s54
00:05:07.160 --> 00:05:17.960
Your agents have to understand pretty much the entirety of your company, how you go to market, why products are useful, uh how to kind of like segment your buyers, your prospects, your customers

s55
00:05:17.960 --> 00:05:27.840
from people who have like never heard about you and you have like no information on them and they have no information on you, all the way to like customers who are actively using your products who have like

s56
00:05:27.840 --> 00:05:33.120
a totally different set of um you know, problems that you have to work with.

s57
00:05:33.120 --> 00:05:44.000
Um And to just start to get a little technical here, um we started with like this consistent data foundation uh problem.

s58
00:05:44.000 --> 00:05:47.600
And if you're looking at this and you're like, that looks like a CDP."

s59
00:05:47.600 --> 00:05:49.800
Uh yeah, you're you're right.

s60
00:05:49.800 --> 00:05:59.240
Uh we effectively went and built um an internal customer data platform at Ramp uh where we're effectively doing your very traditional things.

s61
00:05:59.240 --> 00:06:14.720
We're going to take CRM data, product data, uh enrichment data, um web data, buying signals, you know, whether it's things that are internally modeled like um I don't know, we think that this customer has a high propensity to attach to

s62
00:06:14.720 --> 00:06:23.480
procurement or treasury uh all the way to things that are like external signals like funding announcements, um as well as like interaction data, right?

s63
00:06:23.480 --> 00:06:30.960
Emails, meetings, calls, uh page views, um and on the signal side of this, right?

s64
00:06:30.960 --> 00:06:40.760
We have some set of real-time events that are coming in, uh things like emails, you can go and pipe them onto a Kafka topic, consume them, uh and then funnel them back into

s65
00:06:40.760 --> 00:06:53.000
uh both like we have like a Postgres database that backs all of this, it enables us to maintain like transactional guarantees, referential integrity between the entities that exist and the different entities that exist, right?

s66
00:06:53.000 --> 00:07:04.960
Between your CRM, between your product, between third parties, um and attribute everything to the right level of detail, which we found to be like a pretty important problem, as well as all the associated metadata

s67
00:07:04.960 --> 00:07:07.360
around capturing like where did this come from?

s68
00:07:07.360 --> 00:07:09.160
When did it, you know, come in?

s69
00:07:09.160 --> 00:07:13.560
Um as well as starting to embed a lot of this data, right?

s70
00:07:13.560 --> 00:07:18.280
So much sales data is just inherently um unstructured, right?

s71
00:07:18.280 --> 00:07:24.919
You have like call transcripts, you have emails, you have notes, and the ability to kind of search across that is really valuable.

s72
00:07:24.919 --> 00:07:31.600
Uh we have a set of online batch jobs, which are really just calling a lot of APIs uh for the most part.

s73
00:07:31.600 --> 00:07:38.600
Uh Ramp's addressable market is pretty much like the entire US um and now expanding internationally.

s74
00:07:38.600 --> 00:07:50.040
So, being able to kind of like pre-compute, pre-process, pre-ingest like all this enrichment data about who we can sell to and who we're already selling to is um really important for us.

s75
00:07:50.040 --> 00:08:01.360
And then um as previously mentioned, a ton of work has gone into the offline piece of this with uh DBT, Snowflake, pulling everything into our warehouse, doing a lot of offline batch compute,

s76
00:08:01.360 --> 00:08:06.280
and then piping that in via reverse ETL back into the same layer.

s77
00:08:07.520 --> 00:08:16.240
Uh next, more tactically, the way we tend to approach these problems is solve for one team first, then scale horizontally.

s78
00:08:16.240 --> 00:08:21.680
Um as I mentioned before, you have like a very overlapping set of problems that exist, right?

s79
00:08:21.680 --> 00:08:24.160
Everybody wants to do automated outbound.

s80
00:08:24.160 --> 00:08:26.440
Uh everybody wants to prepare for meetings.

s81
00:08:26.440 --> 00:08:34.400
Uh whereas certain teams may have like problems or like things that they do that are isolated to them, like QBR generation.

s82
00:08:34.400 --> 00:08:42.760
Um and to get into an example, like one of the things that we shipped is like pre-meeting briefs, right?

s83
00:08:42.760 --> 00:08:47.760
Um for AMs, AMs are like count account account managers.

s84
00:08:47.760 --> 00:08:55.680
Uh they kind of manage the customer relationships that exist, trying to ensure that customers are using Ramp uh as best as possible.

s85
00:08:55.680 --> 00:09:01.640
And um there's a lot of like important context that goes into like uh a meeting, right?

s86
00:09:01.640 --> 00:09:03.440
It's like what are we talking about?

s87
00:09:03.440 --> 00:09:04.920
Who are we meeting with?

s88
00:09:04.920 --> 00:09:07.360
Um what is the AM trying to do?

s89
00:09:07.360 --> 00:09:10.240
Like what are the product usage information?

s90
00:09:10.240 --> 00:09:11.520
What are the account vitals?

s91
00:09:11.520 --> 00:09:13.480
What's the agenda that we want to tackle?

s92
00:09:13.480 --> 00:09:16.320
And similarly, like what is the customer trying to do, right?

s93
00:09:16.320 --> 00:09:19.200
Do they have open tickets that they're trying to address?

s94
00:09:19.200 --> 00:09:23.640
Did they like email us saying that there is like a specific thing they're trying to talk about?

s95
00:09:23.640 --> 00:09:31.880
And how can we pull this together for AMs so that they can go in prepared uh and kind of manage the uh operational piece of just being in back-to-back-to-back

s96
00:09:31.880 --> 00:09:33.680
meetings all day.

s97
00:09:33.680 --> 00:09:45.640
Um again, technically, uh the place to start with this is obviously if we're trying to generate a pre-meeting brief, we need to know what these meetings are, uh so we can pipe in meeting events,

s98
00:09:45.640 --> 00:09:54.840
uh do some hydration, map uh things like attendee emails, meeting titles uh back to the accounts that we're meeting with.

s99
00:09:54.840 --> 00:10:05.960
This is like a sneaky hard problem at Ramp because you have the same emails that can work on behalf of multiple businesses, so it's kind of like a fuzzy match, and we can go and persist that, so that way every downstream consumer

s100
00:10:05.960 --> 00:10:07.880
of like, "Hey, I care about this meeting."

s101
00:10:07.880 --> 00:10:12.200
doesn't have to go and like recompute this from the ground up.

s102
00:10:12.360 --> 00:10:19.560
And also, as mentioned in the previous talk, uh we've also built a system around durable execution, right?

s103
00:10:19.560 --> 00:10:22.680
That's pretty agnostic to the trigger that comes in.

s104
00:10:22.680 --> 00:10:30.200
Everything is represented as a durable thread built around Temporal, representing each tool call and model call as an activity.

s105
00:10:30.200 --> 00:10:40.200
That way, if uh you know, like a worker goes out for some reason, it can resume uh execution from where it left off, uh pulling together all the state that had accumulated

s106
00:10:40.200 --> 00:10:49.000
at that point in time, instead of starting back from like the beginning of the thread and trying to reprocess everything, which would be very inefficient and slow.

s107
00:10:49.000 --> 00:11:00.280
Um there's also like great out-of-the-box capabilities for things like config-scoped tool calls, uh different agents are going to have access to different uh sets of tools, which give them access to different information,

s108
00:11:00.280 --> 00:11:06.000
different integrations, uh and different skills that might be necessary to actually perform the work.

s109
00:11:06.000 --> 00:11:14.120
And similarly, there's things like uh human-in-the-loop uh tooling to just pause execution, get input, resume.

s110
00:11:14.120 --> 00:11:26.400
Um And then getting to the uh unstructured piece of this, as I mentioned, like unstructured information is probably like the most valuable thing you're sitting on uh within your

s111
00:11:26.400 --> 00:11:30.520
um warehouse or your notes or wherever you store this today.

s112
00:11:30.520 --> 00:11:35.520
Uh so, we have some set of real time data coming in, um, meeting transcripts, emails.

s113
00:11:35.520 --> 00:11:46.480
We have some sort of like uh, batch jobs that are kind of pulling in like enablement materials, product knowledge, playbooks, um, chunking them, embedding them, putting them in Turbo Buffer,

s114
00:11:46.480 --> 00:11:50.720
and allows you to kind of or allows agents to go and search like what do I care about?

s115
00:11:50.720 --> 00:11:52.320
What am I trying to answer right now?

s116
00:11:52.320 --> 00:12:03.400
And doing some combination of like uh, vector search, attribute search, keyword search in order to pull information scoped to like a specific account, for example, uh, without having to pull in like

s117
00:12:03.400 --> 00:12:11.040
the full raw corpus into agent context, um, which would also be very inefficient, very expensive.

s118
00:12:11.680 --> 00:12:16.080
And similarly, we've gone and built a skill library to allow people to customize their agents, right?

s119
00:12:16.080 --> 00:12:21.160
Getting back to the meeting brief example, different people have different formats that they care about.

s120
00:12:21.160 --> 00:12:33.360
They have different information that they care about, um, and allowing them to kind of represent that, uh, in text, giving that to the agent to pull it together, uh, has been like very valuable for getting adoption.

s121
00:12:33.360 --> 00:12:37.240
And putting all this together, you get an operational background agent, right?

s122
00:12:37.240 --> 00:12:45.640
You have like every night we're going to go and generate these things, fan out a set of agents that are going to go and compute, uh, per account, uh, meeting prep,

s123
00:12:45.640 --> 00:12:56.680
uh, which gives, uh, or which use some set of tools giving them access to like uh, that online CDP in Postgres I mentioned, the vector database, uh, meeting prep skills that we own at the system level,

s124
00:12:56.680 --> 00:13:01.560
as well as like custom instructions that users are providing themselves.

s125
00:13:02.960 --> 00:13:09.839
And getting into the extending the blocks, um, the goal is for these foundations to speed up the next thing, right?

s126
00:13:09.839 --> 00:13:11.040
Meetings are super important.

s127
00:13:11.040 --> 00:13:17.040
We want to be able to generate things like post-meeting follow-ups and things like automatic CRM updates, right?

s128
00:13:17.040 --> 00:13:22.160
Which can pull in the transcript and say like, "Hey, we discussed this potential expansion opportunity.

s129
00:13:22.160 --> 00:13:29.160
Let me go and pre-fill all the information needed to create that opportunity, get a thumbs up from my rep, and just make it happen.

s130
00:13:29.160 --> 00:13:33.320
Um and similarly, we want to extend it horizontally to other teams, right?

s131
00:13:33.320 --> 00:13:46.440
Which is mainly an exercise of creating specific skills, data integrations, um and like just data ingestion itself, where we can say like, "Okay, email, call transcript embeddings, custom instructions,

s132
00:13:46.440 --> 00:13:58.880
generalizable, but if we're building this for AEs, we're hand- handling like pre-sales, um opportunities, we need to go and focus more on like third-party data instead of a bunch of product data that we have already,

s133
00:13:58.880 --> 00:14:02.720
and that needs to be uh incorporated into our customer data platform.

s134
00:14:02.720 --> 00:14:10.800
The skills need to go and reference kind of like a different set of uh information that we have on the people that we're trying to sell to.

s135
00:14:11.360 --> 00:14:19.440
And similarly, uh we've built this in a way where employees have access to the same tools and skills that are being used for the background agents that we're creating,

s136
00:14:19.440 --> 00:14:19.640
right?

s137
00:14:19.640 --> 00:14:32.839
We set up a what we call like our GT MCP, uh and this is basically just like uh a window into the same exact tools that we've set up for these background agents, so that way the things that we build are just kind of automatically

s138
00:14:32.839 --> 00:14:35.800
federated out to people who want to go and build their own agents.

s139
00:14:35.800 --> 00:14:41.320
They want to go chat with the information that we're setting up, uh and build their own automations.

s140
00:14:41.800 --> 00:14:43.160
And they're building a ton of them.

s141
00:14:43.160 --> 00:14:55.920
Uh this is just like uh a glimpse into some of the analytics that we've uh done taking the reasoning generated by uh the MCP uh tool calls, you know, that we've uh that are being executed

s142
00:14:55.920 --> 00:14:57.280
uh remotely.

s143
00:14:57.280 --> 00:15:06.560
And this compounds because like when people go and build their own thing and they go and connect to our MCP, they're basically telling us like, "Here is a problem that I have.

s144
00:15:06.560 --> 00:15:08.560
Here's how I'm trying to solve this problem."

s145
00:15:08.560 --> 00:15:16.200
And we can go and work with them to be like, "Okay, we can just go and productionize this, uh distribute this to everybody who probably has similar problems."

s146
00:15:16.200 --> 00:15:26.760
And they give us the prompts and the skills and the like, you know, even applications that they're vibe coding, uh to just like really simplify our ability to just go and productionize

s147
00:15:26.760 --> 00:15:30.280
um like these use cases.

s148
00:15:30.760 --> 00:15:36.800
So now you're probably wondering uh what about that golf example that I had mentioned at the beginning?

s149
00:15:36.800 --> 00:15:39.400
Um the orchestration problem.

s150
00:15:39.400 --> 00:15:50.200
Um the the point that I'm trying to convey by talking about all these specific things that we're doing is that these vertical builds that we're creating are the foundation of like

s151
00:15:50.200 --> 00:15:54.480
uh multi-team, multi-channel like distribution.

s152
00:15:54.480 --> 00:15:58.200
Um if we want to be able to say like here is a playbook.

s153
00:15:58.200 --> 00:15:59.480
Here's how you sell procurement.

s154
00:15:59.480 --> 00:16:01.160
Here's how you sell to construction.

s155
00:16:01.160 --> 00:16:11.280
Or here like wacky experiment ideas that we have uh like offering Pro V1s to golfers, which is actually like uh it works really well.

s156
00:16:11.280 --> 00:16:22.120
Um we need to be able to say like uh take in that corpus of information of things that people are trying to do and federate that out through the background agents that are actually creating these artifacts

s157
00:16:22.120 --> 00:16:27.880
that people are like using to operationalize like go to market and execute.

s158
00:16:27.880 --> 00:16:44.000
So for my Pro V1 golf example, um the goal is to funnel this into ramp revenue, uh the internal application that we have built um and go and like effectively like funnel this into some of these vertical solutions that we've created, right?

s159
00:16:44.000 --> 00:16:49.600
So you can say like for SDRs, we want to go and create an audience of here are the golfers that we want to send things to.

s160
00:16:49.600 --> 00:16:53.640
We can go and generate like personalized copy and sequences that they can go and send.

s161
00:16:53.640 --> 00:17:01.440
Maybe we want to go and create web landing pages and spin up the uh images and the creative that will point these uh email sequences to.

s162
00:17:01.440 --> 00:17:05.720
And we can do all of that through just like the description of like here's my intent.

s163
00:17:05.720 --> 00:17:08.959
Get the people who own these channels to review them and sign off.

s164
00:17:08.959 --> 00:17:16.520
And really allow us to just like move a lot quicker in how we uh ship and like scale creatively.

s165
00:17:16.520 --> 00:17:20.040
Um across all these different go-to-market channels.

s166
00:17:20.040 --> 00:17:33.000
So, the goal of this is to ship faster, ship safer, um scale our teams, become more efficient, and um with these campaigns, we can go and execute them across like multiple channels

s167
00:17:33.000 --> 00:17:40.240
with consistent audience targeting, um agents can go and hold context on multiple things that are like options, right?

s168
00:17:40.240 --> 00:17:54.320
We can go and execute this campaign or that campaign or that experiment and balance the like traditional multi-armed bandit problem of like exploring like new possibilities versus like being safe and like going into just known returns.

s169
00:17:54.320 --> 00:18:07.760
Um and then we can build in guardrails as well to go and um effectively like manage compliance rules, rules of engagement, and being context aware, making sure we're not doing the same thing over and over again.

s170
00:18:07.760 --> 00:18:12.760
Um and yeah, just do this on behalf of everybody.

s171
00:18:13.560 --> 00:18:16.360
And those are the building blocks of go-to-market orchestration.

s172
00:18:16.360 --> 00:18:18.447
Thank you,

s173
00:18:18.447 --> 00:18:20.447
[applause]

s174
00:18:20.640 --> 00:18:24.360
We have probably time for one question.

s175
00:18:24.360 --> 00:18:27.000
Hey, there we go.

s176
00:18:32.080 --> 00:18:32.857
Hey.

s177
00:18:32.857 --> 00:18:33.440
[clears throat]

s178
00:18:33.440 --> 00:18:41.920
So, just curious um if how would you approach building something like this for a smaller company or for a company that's that's just getting started?

s179
00:18:41.920 --> 00:18:51.280
Yeah, I think a few people before have like mentioned something similar, but I would go and like find the very specific use cases that you can build automation around

s180
00:18:51.280 --> 00:18:55.400
and just like solve really specific problems that exist first.

s181
00:18:55.400 --> 00:19:01.320
Um like 3 years ago, there was two of us and we were building like automated outbound, right?

s182
00:19:01.320 --> 00:19:11.240
So, like we're just trying to figure out like how can we go and use GPT 3.5 and like put personalized copy uh into some sequences and go and like

s183
00:19:11.240 --> 00:19:14.760
pull data from uh wherever to go and generate that.

s184
00:19:14.760 --> 00:19:21.800
And by doing these things and solving these problems, you get like a really good understanding of how this works, how it could extend to other teams.

s185
00:19:21.800 --> 00:19:25.200
Um and solving like real problems as you go.

s186
00:19:25.200 --> 00:19:33.120
The reality is that like you can't spend like a year going and building like some really complicated system architecture that like is perfect.

s187
00:19:33.120 --> 00:19:38.840
So, you have to like piece together the vertical solutions and then stick them together.

s188
00:19:50.803 --> 00:19:52.803
[music]