The Building Blocks of GTM Orchestration — Arman Vaziri, Ramp
https://www.youtube.com/watch?v=VjEP0xqTUI0
AI Engineer · 19 min · 188 sentences · sentence-level transcript from YouTube's caption track; each line begins at the time shown

[00:01] [music]
[00:12] Yeah, I really appreciate everybody showing up.
[00:14] Uh, as Madhu mentioned, my name's Armon.
[00:17] I lead our product and sales-led growth engineering teams at Ramp.
[00:22] Um, and today I'm going to talk to you about, uh, the building blocks of go-to-market orchestration.
[00:27] And um, to kick it off, like, what do I mean by go-to-market orchestration?
[00:33] Effectively, like, what we're building towards is the ability to just describe a motion, right?
[00:39] Whether it's like playbooks or experiments or like evergreen campaigns that you want to run.
[00:44] And how those get like distributed across the channels through which you actually execute your go-to-market, right?
[00:50] Whether it's outbound or ads or web or whatever.
[00:54] Um, we want the ability to kind of describe this and automate that output.
[01:00] And this really started a few years ago where we kind of noticed that uh, there's a ton of great ideas, you know, like everybody across product and data and engineering and go-to-market
[01:10] have like really good ideas for things that they want to do.
[01:13] And the bottleneck is kind of like everything after that, right?
[01:16] How do you go pull an audience to go and target?
[01:18] How do you go and convince a bunch of people to like um, abide by whatever strategy that you've come up with or playbooks or enablement materials.
[01:27] Um, and we wanted to try to aim to uh, reduce that coordination cost.
[01:32] So, there's parts of this where we could see it as like an engineering problem, even like a few years ago, just go and create like a consistent data substrate, go and like federate that across the different systems through which you,
[01:43] uh, run your go-to-market.
[01:45] And obviously in the last few years, agents have really like deepened our ability to go and like push the level of automation that you can do on behalf of operators
[01:54] like as close as possible to those points of execution.
[01:59] Um So, like really specifically, uh I'm a golfer.
[02:03] Suppose I want to offer golfers at uh East Coast construction companies an incentive to like try Ramp, talk to sales, whatever.
[02:11] Uh and we want to be able to go and spin up an audience of uh golfers at East Coast construction companies, spin up like an incentive.
[02:18] Let's go offer like some Pro V1 golf balls to uh these people, go create like outbound sequences, generate the copy, generate uh creative for paid ads and for web,
[02:29] maybe show some in-app notifications for your customers, and do all of that seamlessly by just describing the intent, right?
[02:37] And probably more than just this one sentence.
[02:40] Um So, a few years ago, we kind of identified uh a few fundamental challenges here.
[02:45] Um As was previously mentioned, uh the necessary data for this was just messy, inconsistent across systems, right?
[02:52] Everybody's operating off of a different uh source of truth, and that makes it like effectively impossible to go and distribute some coordinated action across these different go-to-market teams and channels.
[03:04] Uh The next is that like reps were just buried in busywork, right?
[03:08] Even if like you have the best intentions, I want to go and like run this campaign, uh I want your help doing it.
[03:14] The reality is that like uh our sales teams are in back-to-back-to-back-to-back meetings all day.
[03:19] They're outbounding, they're selling, and um the operational burden of like doing everything in between sales was just really high, uh which made kind of like really scaling out experimentation
[03:30] and creativity challenging.
[03:33] Uh And similar to that, just the coordination and distribution are expensive, right?
[03:38] If you're like, "I have this idea.
[03:39] I'm going to go write this like proposal, this enablement material.
[03:42] I'm going to go try to like convince a bunch of people to go and use all of this."
[03:45] That's just like a really challenging thing to do on any like pace that's not on the order of like months.
[03:52] Um So, um over the last few years we've been trying to solve this problem from the ground up, right?
[03:59] How can we start with that uh ingestion and consistency problem uh and data quality, which is just like, you know, on the road map every quarter.
[04:09] Um how can we then go build those vertical efficiency and growth levers, uh saving people time uh in like managing operations and execution, uh as well as like how can we improve conversion rates, make people more performant
[04:22] by being able to kind of scale some of these more like uh informed and personalized and creative strategies.
[04:28] And then how can we extend this horizontally, right?
[04:31] Teams have very common workflows at some level, right?
[04:35] Everybody wants to outbound, everybody has meetings.
[04:37] Uh how can we go take the patterns that we build for one team and start to just mirror it to others?
[04:42] Uh and now kind of where we're at is like this distribution and coordination problem, right?
[04:47] How can you go and execute across multiple channels simultaneously through just like the description of intent?
[04:56] So, yeah, I'll get into the building blocks.
[04:59] Really broadly, uh go-to-market agents are complicated.
[05:03] Um in order to do this effectively, right?
[05:07] Your agents have to understand pretty much the entirety of your company, how you go to market, why products are useful, uh how to kind of like segment your buyers, your prospects, your customers
[05:17] from people who have like never heard about you and you have like no information on them and they have no information on you, all the way to like customers who are actively using your products who have like
[05:27] a totally different set of um you know, problems that you have to work with.
[05:33] Um And to just start to get a little technical here, um we started with like this consistent data foundation uh problem.
[05:44] And if you're looking at this and you're like, that looks like a CDP."
[05:47] Uh yeah, you're you're right.
[05:49] Uh we effectively went and built um an internal customer data platform at Ramp uh where we're effectively doing your very traditional things.
[05:59] We're going to take CRM data, product data, uh enrichment data, um web data, buying signals, you know, whether it's things that are internally modeled like um I don't know, we think that this customer has a high propensity to attach to
[06:14] procurement or treasury uh all the way to things that are like external signals like funding announcements, um as well as like interaction data, right?
[06:23] Emails, meetings, calls, uh page views, um and on the signal side of this, right?
[06:30] We have some set of real-time events that are coming in, uh things like emails, you can go and pipe them onto a Kafka topic, consume them, uh and then funnel them back into
[06:40] uh both like we have like a Postgres database that backs all of this, it enables us to maintain like transactional guarantees, referential integrity between the entities that exist and the different entities that exist, right?
[06:53] Between your CRM, between your product, between third parties, um and attribute everything to the right level of detail, which we found to be like a pretty important problem, as well as all the associated metadata
[07:04] around capturing like where did this come from?
[07:07] When did it, you know, come in?
[07:09] Um as well as starting to embed a lot of this data, right?
[07:13] So much sales data is just inherently um unstructured, right?
[07:18] You have like call transcripts, you have emails, you have notes, and the ability to kind of search across that is really valuable.
[07:24] Uh we have a set of online batch jobs, which are really just calling a lot of APIs uh for the most part.
[07:31] Uh Ramp's addressable market is pretty much like the entire US um and now expanding internationally.
[07:38] So, being able to kind of like pre-compute, pre-process, pre-ingest like all this enrichment data about who we can sell to and who we're already selling to is um really important for us.
[07:50] And then um as previously mentioned, a ton of work has gone into the offline piece of this with uh DBT, Snowflake, pulling everything into our warehouse, doing a lot of offline batch compute,
[08:01] and then piping that in via reverse ETL back into the same layer.
[08:07] Uh next, more tactically, the way we tend to approach these problems is solve for one team first, then scale horizontally.
[08:16] Um as I mentioned before, you have like a very overlapping set of problems that exist, right?
[08:21] Everybody wants to do automated outbound.
[08:24] Uh everybody wants to prepare for meetings.
[08:26] Uh whereas certain teams may have like problems or like things that they do that are isolated to them, like QBR generation.
[08:34] Um and to get into an example, like one of the things that we shipped is like pre-meeting briefs, right?
[08:42] Um for AMs, AMs are like count account account managers.
[08:47] Uh they kind of manage the customer relationships that exist, trying to ensure that customers are using Ramp uh as best as possible.
[08:55] And um there's a lot of like important context that goes into like uh a meeting, right?
[09:01] It's like what are we talking about?
[09:03] Who are we meeting with?
[09:04] Um what is the AM trying to do?
[09:07] Like what are the product usage information?
[09:10] What are the account vitals?
[09:11] What's the agenda that we want to tackle?
[09:13] And similarly, like what is the customer trying to do, right?
[09:16] Do they have open tickets that they're trying to address?
[09:19] Did they like email us saying that there is like a specific thing they're trying to talk about?
[09:23] And how can we pull this together for AMs so that they can go in prepared uh and kind of manage the uh operational piece of just being in back-to-back-to-back
[09:31] meetings all day.
[09:33] Um again, technically, uh the place to start with this is obviously if we're trying to generate a pre-meeting brief, we need to know what these meetings are, uh so we can pipe in meeting events,
[09:45] uh do some hydration, map uh things like attendee emails, meeting titles uh back to the accounts that we're meeting with.
[09:54] This is like a sneaky hard problem at Ramp because you have the same emails that can work on behalf of multiple businesses, so it's kind of like a fuzzy match, and we can go and persist that, so that way every downstream consumer
[10:05] of like, "Hey, I care about this meeting."
[10:07] doesn't have to go and like recompute this from the ground up.
[10:12] And also, as mentioned in the previous talk, uh we've also built a system around durable execution, right?
[10:19] That's pretty agnostic to the trigger that comes in.
[10:22] Everything is represented as a durable thread built around Temporal, representing each tool call and model call as an activity.
[10:30] That way, if uh you know, like a worker goes out for some reason, it can resume uh execution from where it left off, uh pulling together all the state that had accumulated
[10:40] at that point in time, instead of starting back from like the beginning of the thread and trying to reprocess everything, which would be very inefficient and slow.
[10:49] Um there's also like great out-of-the-box capabilities for things like config-scoped tool calls, uh different agents are going to have access to different uh sets of tools, which give them access to different information,
[11:00] different integrations, uh and different skills that might be necessary to actually perform the work.
[11:06] And similarly, there's things like uh human-in-the-loop uh tooling to just pause execution, get input, resume.
[11:14] Um And then getting to the uh unstructured piece of this, as I mentioned, like unstructured information is probably like the most valuable thing you're sitting on uh within your
[11:26] um warehouse or your notes or wherever you store this today.
[11:30] Uh so, we have some set of real time data coming in, um, meeting transcripts, emails.
[11:35] We have some sort of like uh, batch jobs that are kind of pulling in like enablement materials, product knowledge, playbooks, um, chunking them, embedding them, putting them in Turbo Buffer,
[11:46] and allows you to kind of or allows agents to go and search like what do I care about?
[11:50] What am I trying to answer right now?
[11:52] And doing some combination of like uh, vector search, attribute search, keyword search in order to pull information scoped to like a specific account, for example, uh, without having to pull in like
[12:03] the full raw corpus into agent context, um, which would also be very inefficient, very expensive.
[12:11] And similarly, we've gone and built a skill library to allow people to customize their agents, right?
[12:16] Getting back to the meeting brief example, different people have different formats that they care about.
[12:21] They have different information that they care about, um, and allowing them to kind of represent that, uh, in text, giving that to the agent to pull it together, uh, has been like very valuable for getting adoption.
[12:33] And putting all this together, you get an operational background agent, right?
[12:37] You have like every night we're going to go and generate these things, fan out a set of agents that are going to go and compute, uh, per account, uh, meeting prep,
[12:45] uh, which gives, uh, or which use some set of tools giving them access to like uh, that online CDP in Postgres I mentioned, the vector database, uh, meeting prep skills that we own at the system level,
[12:56] as well as like custom instructions that users are providing themselves.
[13:02] And getting into the extending the blocks, um, the goal is for these foundations to speed up the next thing, right?
[13:09] Meetings are super important.
[13:11] We want to be able to generate things like post-meeting follow-ups and things like automatic CRM updates, right?
[13:17] Which can pull in the transcript and say like, "Hey, we discussed this potential expansion opportunity.
[13:22] Let me go and pre-fill all the information needed to create that opportunity, get a thumbs up from my rep, and just make it happen.
[13:29] Um and similarly, we want to extend it horizontally to other teams, right?
[13:33] Which is mainly an exercise of creating specific skills, data integrations, um and like just data ingestion itself, where we can say like, "Okay, email, call transcript embeddings, custom instructions,
[13:46] generalizable, but if we're building this for AEs, we're hand- handling like pre-sales, um opportunities, we need to go and focus more on like third-party data instead of a bunch of product data that we have already,
[13:58] and that needs to be uh incorporated into our customer data platform.
[14:02] The skills need to go and reference kind of like a different set of uh information that we have on the people that we're trying to sell to.
[14:11] And similarly, uh we've built this in a way where employees have access to the same tools and skills that are being used for the background agents that we're creating,
[14:19] right?
[14:19] We set up a what we call like our GT MCP, uh and this is basically just like uh a window into the same exact tools that we've set up for these background agents, so that way the things that we build are just kind of automatically
[14:32] federated out to people who want to go and build their own agents.
[14:35] They want to go chat with the information that we're setting up, uh and build their own automations.
[14:41] And they're building a ton of them.
[14:43] Uh this is just like uh a glimpse into some of the analytics that we've uh done taking the reasoning generated by uh the MCP uh tool calls, you know, that we've uh that are being executed
[14:55] uh remotely.
[14:57] And this compounds because like when people go and build their own thing and they go and connect to our MCP, they're basically telling us like, "Here is a problem that I have.
[15:06] Here's how I'm trying to solve this problem."
[15:08] And we can go and work with them to be like, "Okay, we can just go and productionize this, uh distribute this to everybody who probably has similar problems."
[15:16] And they give us the prompts and the skills and the like, you know, even applications that they're vibe coding, uh to just like really simplify our ability to just go and productionize
[15:26] um like these use cases.
[15:30] So now you're probably wondering uh what about that golf example that I had mentioned at the beginning?
[15:36] Um the orchestration problem.
[15:39] Um the the point that I'm trying to convey by talking about all these specific things that we're doing is that these vertical builds that we're creating are the foundation of like
[15:50] uh multi-team, multi-channel like distribution.
[15:54] Um if we want to be able to say like here is a playbook.
[15:58] Here's how you sell procurement.
[15:59] Here's how you sell to construction.
[16:01] Or here like wacky experiment ideas that we have uh like offering Pro V1s to golfers, which is actually like uh it works really well.
[16:11] Um we need to be able to say like uh take in that corpus of information of things that people are trying to do and federate that out through the background agents that are actually creating these artifacts
[16:22] that people are like using to operationalize like go to market and execute.
[16:27] So for my Pro V1 golf example, um the goal is to funnel this into ramp revenue, uh the internal application that we have built um and go and like effectively like funnel this into some of these vertical solutions that we've created, right?
[16:44] So you can say like for SDRs, we want to go and create an audience of here are the golfers that we want to send things to.
[16:49] We can go and generate like personalized copy and sequences that they can go and send.
[16:53] Maybe we want to go and create web landing pages and spin up the uh images and the creative that will point these uh email sequences to.
[17:01] And we can do all of that through just like the description of like here's my intent.
[17:05] Get the people who own these channels to review them and sign off.
[17:08] And really allow us to just like move a lot quicker in how we uh ship and like scale creatively.
[17:16] Um across all these different go-to-market channels.
[17:20] So, the goal of this is to ship faster, ship safer, um scale our teams, become more efficient, and um with these campaigns, we can go and execute them across like multiple channels
[17:33] with consistent audience targeting, um agents can go and hold context on multiple things that are like options, right?
[17:40] We can go and execute this campaign or that campaign or that experiment and balance the like traditional multi-armed bandit problem of like exploring like new possibilities versus like being safe and like going into just known returns.
[17:54] Um and then we can build in guardrails as well to go and um effectively like manage compliance rules, rules of engagement, and being context aware, making sure we're not doing the same thing over and over again.
[18:07] Um and yeah, just do this on behalf of everybody.
[18:13] And those are the building blocks of go-to-market orchestration.
[18:16] Thank you,
[18:18] [applause]
[18:20] We have probably time for one question.
[18:24] Hey, there we go.
[18:32] Hey.
[18:32] [clears throat]
[18:33] So, just curious um if how would you approach building something like this for a smaller company or for a company that's that's just getting started?
[18:41] Yeah, I think a few people before have like mentioned something similar, but I would go and like find the very specific use cases that you can build automation around
[18:51] and just like solve really specific problems that exist first.
[18:55] Um like 3 years ago, there was two of us and we were building like automated outbound, right?
[19:01] So, like we're just trying to figure out like how can we go and use GPT 3.5 and like put personalized copy uh into some sequences and go and like
[19:11] pull data from uh wherever to go and generate that.
[19:14] And by doing these things and solving these problems, you get like a really good understanding of how this works, how it could extend to other teams.
[19:21] Um and solving like real problems as you go.
[19:25] The reality is that like you can't spend like a year going and building like some really complicated system architecture that like is perfect.
[19:33] So, you have to like piece together the vertical solutions and then stick them together.
[19:50] [music]