The Building Blocks of GTM Orchestration — Arman Vaziri, Ramp

AI Engineer · 19 min · 188 sentences · from YouTube's caption track

Each timecode opens YouTube at the start of that sentence. Line anchors (#s42) are the cue ids in the WebVTT, and every line carries its start and end seconds. All transcripts has every talk, and the whole corpus as one file.

00:01[music]
00:12Yeah, I really appreciate everybody showing up.
00:14Uh, as Madhu mentioned, my name's Armon.
00:17I lead our product and sales-led growth engineering teams at Ramp.
00:22Um, and today I'm going to talk to you about, uh, the building blocks of go-to-market orchestration.
00:27And um, to kick it off, like, what do I mean by go-to-market orchestration?
00:33Effectively, like, what we're building towards is the ability to just describe a motion, right?
00:39Whether it's like playbooks or experiments or like evergreen campaigns that you want to run.
00:44And how those get like distributed across the channels through which you actually execute your go-to-market, right?
00:50Whether it's outbound or ads or web or whatever.
00:54Um, we want the ability to kind of describe this and automate that output.
01:00And this really started a few years ago where we kind of noticed that uh, there's a ton of great ideas, you know, like everybody across product and data and engineering and go-to-market
01:10have like really good ideas for things that they want to do.
01:13And the bottleneck is kind of like everything after that, right?
01:16How do you go pull an audience to go and target?
01:18How do you go and convince a bunch of people to like um, abide by whatever strategy that you've come up with or playbooks or enablement materials.
01:27Um, and we wanted to try to aim to uh, reduce that coordination cost.
01:32So, there's parts of this where we could see it as like an engineering problem, even like a few years ago, just go and create like a consistent data substrate, go and like federate that across the different systems through which you,
01:43uh, run your go-to-market.
01:45And obviously in the last few years, agents have really like deepened our ability to go and like push the level of automation that you can do on behalf of operators
01:54like as close as possible to those points of execution.
01:59Um So, like really specifically, uh I'm a golfer.
02:03Suppose I want to offer golfers at uh East Coast construction companies an incentive to like try Ramp, talk to sales, whatever.
02:11Uh and we want to be able to go and spin up an audience of uh golfers at East Coast construction companies, spin up like an incentive.
02:18Let's go offer like some Pro V1 golf balls to uh these people, go create like outbound sequences, generate the copy, generate uh creative for paid ads and for web,
02:29maybe show some in-app notifications for your customers, and do all of that seamlessly by just describing the intent, right?
02:37And probably more than just this one sentence.
02:40Um So, a few years ago, we kind of identified uh a few fundamental challenges here.
02:45Um As was previously mentioned, uh the necessary data for this was just messy, inconsistent across systems, right?
02:52Everybody's operating off of a different uh source of truth, and that makes it like effectively impossible to go and distribute some coordinated action across these different go-to-market teams and channels.
03:04Uh The next is that like reps were just buried in busywork, right?
03:08Even if like you have the best intentions, I want to go and like run this campaign, uh I want your help doing it.
03:14The reality is that like uh our sales teams are in back-to-back-to-back-to-back meetings all day.
03:19They're outbounding, they're selling, and um the operational burden of like doing everything in between sales was just really high, uh which made kind of like really scaling out experimentation
03:30and creativity challenging.
03:33Uh And similar to that, just the coordination and distribution are expensive, right?
03:38If you're like, "I have this idea.
03:39I'm going to go write this like proposal, this enablement material.
03:42I'm going to go try to like convince a bunch of people to go and use all of this."
03:45That's just like a really challenging thing to do on any like pace that's not on the order of like months.
03:52Um So, um over the last few years we've been trying to solve this problem from the ground up, right?
03:59How can we start with that uh ingestion and consistency problem uh and data quality, which is just like, you know, on the road map every quarter.
04:09Um how can we then go build those vertical efficiency and growth levers, uh saving people time uh in like managing operations and execution, uh as well as like how can we improve conversion rates, make people more performant
04:22by being able to kind of scale some of these more like uh informed and personalized and creative strategies.
04:28And then how can we extend this horizontally, right?
04:31Teams have very common workflows at some level, right?
04:35Everybody wants to outbound, everybody has meetings.
04:37Uh how can we go take the patterns that we build for one team and start to just mirror it to others?
04:42Uh and now kind of where we're at is like this distribution and coordination problem, right?
04:47How can you go and execute across multiple channels simultaneously through just like the description of intent?
04:56So, yeah, I'll get into the building blocks.
04:59Really broadly, uh go-to-market agents are complicated.
05:03Um in order to do this effectively, right?
05:07Your agents have to understand pretty much the entirety of your company, how you go to market, why products are useful, uh how to kind of like segment your buyers, your prospects, your customers
05:17from people who have like never heard about you and you have like no information on them and they have no information on you, all the way to like customers who are actively using your products who have like
05:27a totally different set of um you know, problems that you have to work with.
05:33Um And to just start to get a little technical here, um we started with like this consistent data foundation uh problem.
05:44And if you're looking at this and you're like, that looks like a CDP."
05:47Uh yeah, you're you're right.
05:49Uh we effectively went and built um an internal customer data platform at Ramp uh where we're effectively doing your very traditional things.
05:59We're going to take CRM data, product data, uh enrichment data, um web data, buying signals, you know, whether it's things that are internally modeled like um I don't know, we think that this customer has a high propensity to attach to
06:14procurement or treasury uh all the way to things that are like external signals like funding announcements, um as well as like interaction data, right?
06:23Emails, meetings, calls, uh page views, um and on the signal side of this, right?
06:30We have some set of real-time events that are coming in, uh things like emails, you can go and pipe them onto a Kafka topic, consume them, uh and then funnel them back into
06:40uh both like we have like a Postgres database that backs all of this, it enables us to maintain like transactional guarantees, referential integrity between the entities that exist and the different entities that exist, right?
06:53Between your CRM, between your product, between third parties, um and attribute everything to the right level of detail, which we found to be like a pretty important problem, as well as all the associated metadata
07:04around capturing like where did this come from?
07:07When did it, you know, come in?
07:09Um as well as starting to embed a lot of this data, right?
07:13So much sales data is just inherently um unstructured, right?
07:18You have like call transcripts, you have emails, you have notes, and the ability to kind of search across that is really valuable.
07:24Uh we have a set of online batch jobs, which are really just calling a lot of APIs uh for the most part.
07:31Uh Ramp's addressable market is pretty much like the entire US um and now expanding internationally.
07:38So, being able to kind of like pre-compute, pre-process, pre-ingest like all this enrichment data about who we can sell to and who we're already selling to is um really important for us.
07:50And then um as previously mentioned, a ton of work has gone into the offline piece of this with uh DBT, Snowflake, pulling everything into our warehouse, doing a lot of offline batch compute,
08:01and then piping that in via reverse ETL back into the same layer.
08:07Uh next, more tactically, the way we tend to approach these problems is solve for one team first, then scale horizontally.
08:16Um as I mentioned before, you have like a very overlapping set of problems that exist, right?
08:21Everybody wants to do automated outbound.
08:24Uh everybody wants to prepare for meetings.
08:26Uh whereas certain teams may have like problems or like things that they do that are isolated to them, like QBR generation.
08:34Um and to get into an example, like one of the things that we shipped is like pre-meeting briefs, right?
08:42Um for AMs, AMs are like count account account managers.
08:47Uh they kind of manage the customer relationships that exist, trying to ensure that customers are using Ramp uh as best as possible.
08:55And um there's a lot of like important context that goes into like uh a meeting, right?
09:01It's like what are we talking about?
09:03Who are we meeting with?
09:04Um what is the AM trying to do?
09:07Like what are the product usage information?
09:10What are the account vitals?
09:11What's the agenda that we want to tackle?
09:13And similarly, like what is the customer trying to do, right?
09:16Do they have open tickets that they're trying to address?
09:19Did they like email us saying that there is like a specific thing they're trying to talk about?
09:23And how can we pull this together for AMs so that they can go in prepared uh and kind of manage the uh operational piece of just being in back-to-back-to-back
09:31meetings all day.
09:33Um again, technically, uh the place to start with this is obviously if we're trying to generate a pre-meeting brief, we need to know what these meetings are, uh so we can pipe in meeting events,
09:45uh do some hydration, map uh things like attendee emails, meeting titles uh back to the accounts that we're meeting with.
09:54This is like a sneaky hard problem at Ramp because you have the same emails that can work on behalf of multiple businesses, so it's kind of like a fuzzy match, and we can go and persist that, so that way every downstream consumer
10:05of like, "Hey, I care about this meeting."
10:07doesn't have to go and like recompute this from the ground up.
10:12And also, as mentioned in the previous talk, uh we've also built a system around durable execution, right?
10:19That's pretty agnostic to the trigger that comes in.
10:22Everything is represented as a durable thread built around Temporal, representing each tool call and model call as an activity.
10:30That way, if uh you know, like a worker goes out for some reason, it can resume uh execution from where it left off, uh pulling together all the state that had accumulated
10:40at that point in time, instead of starting back from like the beginning of the thread and trying to reprocess everything, which would be very inefficient and slow.
10:49Um there's also like great out-of-the-box capabilities for things like config-scoped tool calls, uh different agents are going to have access to different uh sets of tools, which give them access to different information,
11:00different integrations, uh and different skills that might be necessary to actually perform the work.
11:06And similarly, there's things like uh human-in-the-loop uh tooling to just pause execution, get input, resume.
11:14Um And then getting to the uh unstructured piece of this, as I mentioned, like unstructured information is probably like the most valuable thing you're sitting on uh within your
11:26um warehouse or your notes or wherever you store this today.
11:30Uh so, we have some set of real time data coming in, um, meeting transcripts, emails.
11:35We have some sort of like uh, batch jobs that are kind of pulling in like enablement materials, product knowledge, playbooks, um, chunking them, embedding them, putting them in Turbo Buffer,
11:46and allows you to kind of or allows agents to go and search like what do I care about?
11:50What am I trying to answer right now?
11:52And doing some combination of like uh, vector search, attribute search, keyword search in order to pull information scoped to like a specific account, for example, uh, without having to pull in like
12:03the full raw corpus into agent context, um, which would also be very inefficient, very expensive.
12:11And similarly, we've gone and built a skill library to allow people to customize their agents, right?
12:16Getting back to the meeting brief example, different people have different formats that they care about.
12:21They have different information that they care about, um, and allowing them to kind of represent that, uh, in text, giving that to the agent to pull it together, uh, has been like very valuable for getting adoption.
12:33And putting all this together, you get an operational background agent, right?
12:37You have like every night we're going to go and generate these things, fan out a set of agents that are going to go and compute, uh, per account, uh, meeting prep,
12:45uh, which gives, uh, or which use some set of tools giving them access to like uh, that online CDP in Postgres I mentioned, the vector database, uh, meeting prep skills that we own at the system level,
12:56as well as like custom instructions that users are providing themselves.
13:02And getting into the extending the blocks, um, the goal is for these foundations to speed up the next thing, right?
13:09Meetings are super important.
13:11We want to be able to generate things like post-meeting follow-ups and things like automatic CRM updates, right?
13:17Which can pull in the transcript and say like, "Hey, we discussed this potential expansion opportunity.
13:22Let me go and pre-fill all the information needed to create that opportunity, get a thumbs up from my rep, and just make it happen.
13:29Um and similarly, we want to extend it horizontally to other teams, right?
13:33Which is mainly an exercise of creating specific skills, data integrations, um and like just data ingestion itself, where we can say like, "Okay, email, call transcript embeddings, custom instructions,
13:46generalizable, but if we're building this for AEs, we're hand- handling like pre-sales, um opportunities, we need to go and focus more on like third-party data instead of a bunch of product data that we have already,
13:58and that needs to be uh incorporated into our customer data platform.
14:02The skills need to go and reference kind of like a different set of uh information that we have on the people that we're trying to sell to.
14:11And similarly, uh we've built this in a way where employees have access to the same tools and skills that are being used for the background agents that we're creating,
14:19right?
14:19We set up a what we call like our GT MCP, uh and this is basically just like uh a window into the same exact tools that we've set up for these background agents, so that way the things that we build are just kind of automatically
14:32federated out to people who want to go and build their own agents.
14:35They want to go chat with the information that we're setting up, uh and build their own automations.
14:41And they're building a ton of them.
14:43Uh this is just like uh a glimpse into some of the analytics that we've uh done taking the reasoning generated by uh the MCP uh tool calls, you know, that we've uh that are being executed
14:55uh remotely.
14:57And this compounds because like when people go and build their own thing and they go and connect to our MCP, they're basically telling us like, "Here is a problem that I have.
15:06Here's how I'm trying to solve this problem."
15:08And we can go and work with them to be like, "Okay, we can just go and productionize this, uh distribute this to everybody who probably has similar problems."
15:16And they give us the prompts and the skills and the like, you know, even applications that they're vibe coding, uh to just like really simplify our ability to just go and productionize
15:26um like these use cases.
15:30So now you're probably wondering uh what about that golf example that I had mentioned at the beginning?
15:36Um the orchestration problem.
15:39Um the the point that I'm trying to convey by talking about all these specific things that we're doing is that these vertical builds that we're creating are the foundation of like
15:50uh multi-team, multi-channel like distribution.
15:54Um if we want to be able to say like here is a playbook.
15:58Here's how you sell procurement.
15:59Here's how you sell to construction.
16:01Or here like wacky experiment ideas that we have uh like offering Pro V1s to golfers, which is actually like uh it works really well.
16:11Um we need to be able to say like uh take in that corpus of information of things that people are trying to do and federate that out through the background agents that are actually creating these artifacts
16:22that people are like using to operationalize like go to market and execute.
16:27So for my Pro V1 golf example, um the goal is to funnel this into ramp revenue, uh the internal application that we have built um and go and like effectively like funnel this into some of these vertical solutions that we've created, right?
16:44So you can say like for SDRs, we want to go and create an audience of here are the golfers that we want to send things to.
16:49We can go and generate like personalized copy and sequences that they can go and send.
16:53Maybe we want to go and create web landing pages and spin up the uh images and the creative that will point these uh email sequences to.
17:01And we can do all of that through just like the description of like here's my intent.
17:05Get the people who own these channels to review them and sign off.
17:08And really allow us to just like move a lot quicker in how we uh ship and like scale creatively.
17:16Um across all these different go-to-market channels.
17:20So, the goal of this is to ship faster, ship safer, um scale our teams, become more efficient, and um with these campaigns, we can go and execute them across like multiple channels
17:33with consistent audience targeting, um agents can go and hold context on multiple things that are like options, right?
17:40We can go and execute this campaign or that campaign or that experiment and balance the like traditional multi-armed bandit problem of like exploring like new possibilities versus like being safe and like going into just known returns.
17:54Um and then we can build in guardrails as well to go and um effectively like manage compliance rules, rules of engagement, and being context aware, making sure we're not doing the same thing over and over again.
18:07Um and yeah, just do this on behalf of everybody.
18:13And those are the building blocks of go-to-market orchestration.
18:16Thank you,
18:18[applause]
18:20We have probably time for one question.
18:24Hey, there we go.
18:32Hey.
18:32[clears throat]
18:33So, just curious um if how would you approach building something like this for a smaller company or for a company that's that's just getting started?
18:41Yeah, I think a few people before have like mentioned something similar, but I would go and like find the very specific use cases that you can build automation around
18:51and just like solve really specific problems that exist first.
18:55Um like 3 years ago, there was two of us and we were building like automated outbound, right?
19:01So, like we're just trying to figure out like how can we go and use GPT 3.5 and like put personalized copy uh into some sequences and go and like
19:11pull data from uh wherever to go and generate that.
19:14And by doing these things and solving these problems, you get like a really good understanding of how this works, how it could extend to other teams.
19:21Um and solving like real problems as you go.
19:25The reality is that like you can't spend like a year going and building like some really complicated system architecture that like is perfect.
19:33So, you have to like piece together the vertical solutions and then stick them together.
19:50[music]