How AI Agents Let GTM Teams Scale — Justin Joyce, Cloudflare
https://www.youtube.com/watch?v=Qw_tC68KKes
AI Engineer · 19 min · 156 sentences · sentence-level transcript from YouTube's caption track; each line begins at the time shown

[00:01] [music]
[00:12] Well, thank you everyone for joining.
[00:13] I'm hoping that you guys had a great time so far at this conference and you guys have a lot of takeaways back, you know, to your company.
[00:20] Um, I don't know if any of you guys saw the uh AGI pills downstairs.
[00:24] Yeah, well, I just took some.
[00:26] So, if I say any phrases about that's the uh the the gun What's that What's that the phrase?
[00:32] That's the uh burning gun or the smoking gun or I start hallucinating in general, well, please snap me back.
[00:38] That's probably just the AGI pills.
[00:41] All right.
[00:42] So, without further ado, let's get started.
[00:45] Uh my name is Justin Joyce.
[00:46] I'm a principal sales operations and strategy manager at Cloudflare.
[00:50] Um and I work with the go-to-market team as part of the revenue operations organization, specifically on the teams that uh produce leads for the sales teams, as well as the customer experience team, which works on
[01:04] uh the customer experience after the sale the sales have been done.
[01:09] And uh just a little background about me.
[01:13] As Moda said, I started in sales operations uh and sales and then I moved to sales uh to the machine learning side about the last 7 years at Grainger.
[01:23] And I really wanted to do that to be able to learn how to uh have prescriptive analysis and prescriptive uh prediction so I can help the business better to
[01:33] make decisions and to understand what's the next step next best step.
[01:39] So, uh 6 months ago, I had an opportunity to move back into sales operations uh because I really wanted to use all the skills that I'd been learning from machine learning as well as from sales
[01:52] operations in general.
[01:57] So, what's the general problem?
[01:59] The general problem is that traditional go-to-market does not scale.
[02:04] There's a few um facets to this.
[02:07] The first facet is that usually teams on the back office side, uh they're either doing work in Excel or sheets at worst, um building analysis each week, multiple hours a week,
[02:20] and as they take on multiple projects, it gets exponentially long with how many of those analysis that they're doing.
[02:26] Uh at best, they are producing dashboards, um providing information to the uh leadership and executive team, um which, you know, meets the needs of most teams, um but not all of them.
[02:39] And that needs meets that needs uh means that in general, not all the requirements of the go-to-market teams are met.
[02:47] They're not able to really provide all the information that the teams need when um they need it.
[02:55] The second problem, which is more on the go-to-market side with the sales and uh the sales teams and the other teams that I mentioned that I support, is they have two gaps.
[03:06] Essentially, the first gap is the context gap, meaning when a salesperson is uh talking to a prospect um one call and talking to a current customer in the next call, or
[03:18] um talking at an adoption conversation the call after that, they have to constantly switch contexts and they have to gather information for those specific calls, which is good.
[03:28] They really need to get that information to have those calls and understand how to approach the situation, but they have to do all that work in between.
[03:37] Um the second one what is like what I like to call is the expert gap, which is the gap between how your expert salesperson or expert go-to-market um sales individual,
[03:49] how he would approach a situation, how he would talk to a prospect, how he would uh work on adoption call, how he would handle a customer satisfaction issue, and
[03:59] between also the a new salesperson or someone that's just ramping up.
[04:04] So, ideally like everyone working at the same operational level, so you have consistency in execution, consistency in messaging of how to assess a a customer's problems and how your product can help fit that portfolio.
[04:17] So, with these two problems with manual work as well as uh salespeople not having enough information and having the gap of having to get all the information they need
[04:29] and gather that as well as not be able to execute the same level, it really creates an inefficiency in the go-to-market organization.
[04:37] And so, for the last 6 months or so since I've joined Cloudflare, I've been really focusing on how can I make this operation efficient from back to front and
[04:47] there's a lot of great things that we've been doing at our company in general that's helped me enable that and I really want to share some of those findings with you.
[04:56] So, the framework that I'm proposing here, which I think is is it's going to really be effective in as we flesh this out in the future is a three-pillar
[05:06] approach.
[05:07] The first pillar approach is how can we scale analysis and the ability of operations team to meet the data needs of executives, leadership, as well as be able to build applications
[05:20] using business context.
[05:21] How can they take things that would take 2 hours to do down to 5 minutes?
[05:26] Um back in well, not back in January of this year, I after I joined I joined the company, I had built these skills and I'd started asking questions of the data directly
[05:37] and I was able to get answers immediately while doing other things and I saw the huge power of how if we can scale the analysis and the operations of the teams, we can actually focus on the second part of my job, which is strategy.
[05:50] The second pillar is how can is to scale insight.
[05:54] There's story in the data, and how can we provide that to the team?
[05:58] How can we provide that team at, you know, the weekly level, at the different levels of management?
[06:02] How can we provide that information, that insight, that story to every customer that the sales teams are talking to?
[06:11] And the third one, which is arguably the biggest one, is uh to provide self-service capabilities to the go-to-market team.
[06:19] When these sales individuals, when they're talking to a customer, how can they get the expert-level information that they need to interact with that customer and to best assess, you know, how they should approach the situation, how to handle rejections,
[06:34] how to upsell them, and how to handle customer satisfaction issues.
[06:39] Uh this is a huge part of what I've seen we've done at Cloudflare, and I'll share a little bit what that looks like.
[06:47] So, as it relates to scaling the analytical capability, the back-office operations, what we have done as we have built role-specific skill files, which have the context of the business information tying it to the data.
[07:01] This is for both technical and non-technical users.
[07:04] Technical users, you could say the ones who are building SQL and being able to data engineer a lot of solutions.
[07:10] And then the non-technical people would be more individuals who are closer to the business with the sales people who may not know how to write SQL.
[07:18] And so, we have skill files that they're able to use to ask questions of the data to get answers fairly quickly while doing other tasks.
[07:27] And one example here is in those skill files, we also, through testing, we've included the types of questions that the business would ask of the data.
[07:38] In this case, looking at closed date changes and opportunities, as well as uh changes in the amount of the opportunities, so that we can answer essentially 80% or or more of the questions,
[07:49] where the other 20% might be more uh complex strategic questions.
[07:54] And so, overall, this allows the teams to be able to embed all of the logic into the skill files and get answers uh fairly quickly.
[08:02] So, I've seen users who do not know any SQL, and the essentially their request in the past we bottleneck to someone who knows data and can write SQL for complex queries, be able to just ask questions of the data
[08:14] and get answers.
[08:15] And this is very useful also for what I show later on on the third pillar is for building skills for these go-to-market teams, so that they um can and ask questions of their data
[08:28] and and get answers.
[08:30] Also, uh in our team we've used these same skill files to build multiple applications.
[08:34] Uh when usually, you know, that is uh done in IT and bottlenecked in those areas, we're able to use the semantic information about the business um knowledge, as well as the columns table to build these applications
[08:48] rather quickly.
[08:48] So, this allows us to free up our time, so that we can focus on strategy and enablement.
[08:57] All right, for the second pillar, uh what I mentioned earlier, there's a story in the data and they really shouldn't have to search for it.
[09:03] Uh what I'm showing you here is synthetic data on the right.
[09:07] Uh we have a weekly summary that goes out, which highlights how the business is doing, how they're pacing to their goals, and then highlighting um trends, uh standouts, as well as watches.
[09:18] So, this uh we provide this information to the business, so they can, as you just like you can open your phone now uh on Gemini, if you have it, and you can see your notes for the day or the things that you need to do,
[09:30] giving that same level of information to the go-to-market team, so they can just go along their day, and if they do need to look at some of the reports or dashboards, they can
[09:39] to drill in, um but we bring the story to them.
[09:42] And I'll pull this together why I think this is really important.
[09:46] Um you know, of course there's a place for dashboards and standard information, but there's different level of adoption of the KPI metrics at any given company.
[09:54] You're going to have people who are love dashboards, people are never going to look at them.
[09:58] So, I think you really need to have a way to scaffold that across um the business.
[10:03] So, how do we do this automated analysis?
[10:05] So, a big part of this is simplifying the data so that the AI agents can actually um analyze the data in a very consistent and clean way.
[10:14] Here what we do is we transform the data by the dimension of time, also slice of the logical part of the business, which is manager, theater, and finally the metric.
[10:24] Here we have data uh that is wide.
[10:26] You could also go um from wide to long.
[10:29] Uh our trend information that I showed you, um uh that data is long, and then we do some pre-processing on that data um to then highlight trends.
[10:38] So, the the uh embedding of the logic of how you would filter this data to even analyze it, um as well as the logical um aggregations the business want to see is all engineered
[10:49] up front.
[10:50] This from my experience, this handles uh 80 or plus percent of the requests is just getting information about the performance of the teams and how they're doing.
[10:59] You can always go down to the raw data, but this last uh pillar, which I'll go over in a minute, um allows them to do that.
[11:05] To be able to orchestrate this uh effectively and be able to rely on it, we have a multi-agent workflow where we first get the data, and then we do a first pass draft on the data calling our MCPs,
[11:18] um and then this we have a second reviewer agent who checks the veracity of the data, and then we have a third um agent, which is a tone agent, who using a multi-shot prompt
[11:27] um is able to just craft the message and highlight the risks and opportunities um equally.
[11:33] And with every run, we have observability into each of the LLM calls, so we can see what is passed and what is the response that is going on there.
[11:41] And so, this architecture we tested for about 2-3 months and, you know, looking every single run to see what is going wrong.
[11:50] And this is the the model that we had set up that is really working for us.
[11:54] And we really hope to expand this beyond just what I've shown you for multiple teams, but also down to the customer level like I was just talking to you about.
[12:03] The third part is the self-service model.
[12:06] And what I'm showing you here is our internal tool called Cloudflare OS, which is an agentic workspace that is running on Cloudflare where the go-to-market team can come in here.
[12:17] It spins up their own compute and their own persistent environment using Cloudflare workers as well as durable objects, which is basically a storage sort of like S3.
[12:29] And so, the sales people can come in here and get the data they need it when they need it.
[12:36] So, some use cases that these teams are using it for is doing a forecast brief, building QBR decks, building a purchase deck on what the customer that they're onboarding has purchased,
[12:46] doing account planning, general queries of the data, as well as renewal preparation.
[12:51] How are they going to look at what the customer has used and either upsell them or figure out how they can get them adopting their product more.
[13:01] So, just a little bit more into that Cloudflare S setup that I just showed you.
[13:05] The AI agent workspace is where that screen I was I was showing you.
[13:09] And through the the three-part piece of the skills in the lower left, which is our like expert level information, as well as the MCP connection and the AI gateway,
[13:22] they're able to have conversations in this agentic workspace to pull data they need and using expert-level skills, which is curated, um so that they're able to execute the jobs that they need to do when they need to do it.
[13:36] And so, a little more information about the skill repository, uh we have a central alias where skills are presented uh to uh the central team, curated by the uh go-to-market team, as well as by operations team, and they're reviewed, so we can make sure that we're not having a proliferation
[13:52] of skills, and we have an expert-level knowledge skill at every level, so that they can really get all the information they need for how to approach uh any customer situation.
[14:04] And so, just a few more uh um images here of uh them using it.
[14:07] Here, uh we have them building a prescriptive plan for their daily work.
[14:12] They're asking a question, it's their um you can see the agent is responding by looking into the MCP and starting to pull the data together.
[14:20] And related to the QBR deck, uh here's a slide of generating a custom slide deck for a customer call.
[14:29] And so, this has really, I think, unlocked the ability of the go-to-market teams to be able to really have all their information uh serviced to them.
[14:39] And so, bring it together with the three pillars that I talked about.
[14:42] Um if you really don't have all these, I think you have issues with serving the go-to-market needs uh in terms of uh using optimizing the use of Agentyc uh systems.
[14:53] With self-service, you allow them to be able to pull data when they need it for whatever situation they need it with the expert-level information.
[14:59] The the second is by pushing the uh the insights to the business, you're able to surface the generalized standardized information of how these teams are doing, and also um
[15:11] yeah, so so that there's no and also uh so they're aligning with source of truth on performance.
[15:15] Um and then, the the third one is the scaling of the analytical team for them to be able to answer queries and build applications for the teams, which really unlocks a lot
[15:25] um because the opportunity cost of that team being overloaded and being able to help is that the meet the needs of the go-to-market team um is not met.
[15:34] So, some findings um and the future.
[15:37] So, uh the first thing is skill curation is the basis for all of this agentic workforce.
[15:43] If you're able to embed the knowledge of the business into the skill files as well as the skills uh uh for the an analyst to be able to build
[15:52] uh answer questions or build applications as well as the skills that I showed you in the Cloudflare OS, you're really able to uh give them the ability to use the agentic systems in a more predictable and deterministic way so that they can execute
[16:07] um evenly across the board.
[16:09] The second thing is through this whole process, the feedback loop is very important.
[16:14] Just like uh a company would try to sell a product externally and get feedback, with these internal teams, uh the feedback loop is very important to be able to see is is what you're building is it actually useful?
[16:25] What are some issues that they're having?
[16:26] And how can you make this uh work more efficiently?
[16:30] And the third thing is the layering of those uh three pillars that I talked about.
[16:35] Being able to answer questions where the team comes to you, some of the go-to-market team, that's how they like to interface with the operations team is to be able to ask questions.
[16:43] Um and then the pushing of information and then self-serviceability.
[16:47] Through that, you're able to uh interweave all the needs of the team to be able to be met by this uh agentic run uh team.
[16:55] So, through all these uh different pillars that I talked about, we've really been able to 2x our efficiency and be able to serve the teams as well as allowing them to be able to get the information that they need to do their job.
[17:08] And some some things um that I see going into the future.
[17:11] Number one is a deeper integration with uh our systems that we work in.
[17:16] So, for example, those QBR decks and um renewal call skills, uh how can we set up meetings for the go-to-market team and embed those artifacts in those meetings so they don't have to actually pull them.
[17:30] We can allow them to that self-service portal to be more ad hoc in what they need.
[17:35] But, that requires some information or some security setup and how can we do that?
[17:40] And then also, another example is getting meeting notes from those calls, which you have to set up that across the board.
[17:48] So, there's some system side thing that we have to work on there.
[17:51] The second thing is harder problems around quoting and approvals and updating the CRM itself.
[17:59] Uh we use Salesforce and we're just in the midst of building the connections and the ability for us to update Salesforce with these Agenty systems.
[18:08] And I see that being um set up in a way that I set up with that automated analysis where you have workflows uh to just make sure that everything is getting
[18:19] um done right.
[18:20] And the second thing is we're sort of reached the Cambrian stage of using Agenty systems, which means there's an explosion of excitement and skills and finding out ways to solve anything with AI.
[18:32] But, I see as we get to this uh fuller integration and standardization, we're going to uh want to come back and and and not really limit, but just figure out a really strategic approach for allowing each team to use the Agenty system so that the source of truth in all the systems are aligning.
[18:50] All right.
[18:51] Well, thank you for joining this talk and I appreciate you, um you know, coming here.
[18:55] Hope you have a great conference.
[18:57] [applause]
[19:12] [music]