WEBVTT

NOTE Sentence-level transcript of https://www.youtube.com/watch?v=Qw_tC68KKes

NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/Qw_tC68KKes.html. A cue ends where the next begins, or 2 s after its last word.

s1
00:00:01.309 --> 00:00:03.309
[music]

s2
00:00:12.640 --> 00:00:13.760
Well, thank you everyone for joining.

s3
00:00:13.760 --> 00:00:20.920
I'm hoping that you guys had a great time so far at this conference and you guys have a lot of takeaways back, you know, to your company.

s4
00:00:20.920 --> 00:00:24.720
Um, I don't know if any of you guys saw the uh AGI pills downstairs.

s5
00:00:24.720 --> 00:00:26.200
Yeah, well, I just took some.

s6
00:00:26.200 --> 00:00:32.680
So, if I say any phrases about that's the uh the the gun What's that What's that the phrase?

s7
00:00:32.680 --> 00:00:38.360
That's the uh burning gun or the smoking gun or I start hallucinating in general, well, please snap me back.

s8
00:00:38.360 --> 00:00:41.040
That's probably just the AGI pills.

s9
00:00:41.040 --> 00:00:42.040
All right.

s10
00:00:42.040 --> 00:00:45.000
So, without further ado, let's get started.

s11
00:00:45.000 --> 00:00:46.160
Uh my name is Justin Joyce.

s12
00:00:46.160 --> 00:00:50.440
I'm a principal sales operations and strategy manager at Cloudflare.

s13
00:00:50.440 --> 00:01:04.960
Um and I work with the go-to-market team as part of the revenue operations organization, specifically on the teams that uh produce leads for the sales teams, as well as the customer experience team, which works on

s14
00:01:04.960 --> 00:01:09.840
uh the customer experience after the sale the sales have been done.

s15
00:01:09.840 --> 00:01:13.680
And uh just a little background about me.

s16
00:01:13.680 --> 00:01:23.760
As Moda said, I started in sales operations uh and sales and then I moved to sales uh to the machine learning side about the last 7 years at Grainger.

s17
00:01:23.760 --> 00:01:33.800
And I really wanted to do that to be able to learn how to uh have prescriptive analysis and prescriptive uh prediction so I can help the business better to

s18
00:01:33.800 --> 00:01:39.520
make decisions and to understand what's the next step next best step.

s19
00:01:39.520 --> 00:01:52.560
So, uh 6 months ago, I had an opportunity to move back into sales operations uh because I really wanted to use all the skills that I'd been learning from machine learning as well as from sales

s20
00:01:52.560 --> 00:01:55.880
operations in general.

s21
00:01:57.800 --> 00:01:59.360
So, what's the general problem?

s22
00:01:59.360 --> 00:02:04.720
The general problem is that traditional go-to-market does not scale.

s23
00:02:04.720 --> 00:02:07.960
There's a few um facets to this.

s24
00:02:07.960 --> 00:02:20.400
The first facet is that usually teams on the back office side, uh they're either doing work in Excel or sheets at worst, um building analysis each week, multiple hours a week,

s25
00:02:20.400 --> 00:02:26.560
and as they take on multiple projects, it gets exponentially long with how many of those analysis that they're doing.

s26
00:02:26.560 --> 00:02:39.240
Uh at best, they are producing dashboards, um providing information to the uh leadership and executive team, um which, you know, meets the needs of most teams, um but not all of them.

s27
00:02:39.240 --> 00:02:47.280
And that needs meets that needs uh means that in general, not all the requirements of the go-to-market teams are met.

s28
00:02:47.280 --> 00:02:54.680
They're not able to really provide all the information that the teams need when um they need it.

s29
00:02:55.120 --> 00:03:06.760
The second problem, which is more on the go-to-market side with the sales and uh the sales teams and the other teams that I mentioned that I support, is they have two gaps.

s30
00:03:06.760 --> 00:03:18.959
Essentially, the first gap is the context gap, meaning when a salesperson is uh talking to a prospect um one call and talking to a current customer in the next call, or

s31
00:03:18.959 --> 00:03:28.959
um talking at an adoption conversation the call after that, they have to constantly switch contexts and they have to gather information for those specific calls, which is good.

s32
00:03:28.959 --> 00:03:37.560
They really need to get that information to have those calls and understand how to approach the situation, but they have to do all that work in between.

s33
00:03:37.560 --> 00:03:49.239
Um the second one what is like what I like to call is the expert gap, which is the gap between how your expert salesperson or expert go-to-market um sales individual,

s34
00:03:49.239 --> 00:03:59.640
how he would approach a situation, how he would talk to a prospect, how he would uh work on adoption call, how he would handle a customer satisfaction issue, and

s35
00:03:59.640 --> 00:04:04.240
between also the a new salesperson or someone that's just ramping up.

s36
00:04:04.240 --> 00:04:17.959
So, ideally like everyone working at the same operational level, so you have consistency in execution, consistency in messaging of how to assess a a customer's problems and how your product can help fit that portfolio.

s37
00:04:17.959 --> 00:04:29.840
So, with these two problems with manual work as well as uh salespeople not having enough information and having the gap of having to get all the information they need

s38
00:04:29.840 --> 00:04:37.120
and gather that as well as not be able to execute the same level, it really creates an inefficiency in the go-to-market organization.

s39
00:04:37.120 --> 00:04:47.320
And so, for the last 6 months or so since I've joined Cloudflare, I've been really focusing on how can I make this operation efficient from back to front and

s40
00:04:47.320 --> 00:04:56.000
there's a lot of great things that we've been doing at our company in general that's helped me enable that and I really want to share some of those findings with you.

s41
00:04:56.000 --> 00:05:06.600
So, the framework that I'm proposing here, which I think is is it's going to really be effective in as we flesh this out in the future is a three-pillar

s42
00:05:06.600 --> 00:05:07.919
approach.

s43
00:05:07.919 --> 00:05:20.000
The first pillar approach is how can we scale analysis and the ability of operations team to meet the data needs of executives, leadership, as well as be able to build applications

s44
00:05:20.000 --> 00:05:21.760
using business context.

s45
00:05:21.760 --> 00:05:26.160
How can they take things that would take 2 hours to do down to 5 minutes?

s46
00:05:26.160 --> 00:05:37.280
Um back in well, not back in January of this year, I after I joined I joined the company, I had built these skills and I'd started asking questions of the data directly

s47
00:05:37.280 --> 00:05:50.400
and I was able to get answers immediately while doing other things and I saw the huge power of how if we can scale the analysis and the operations of the teams, we can actually focus on the second part of my job, which is strategy.

s48
00:05:50.400 --> 00:05:54.680
The second pillar is how can is to scale insight.

s49
00:05:54.680 --> 00:05:58.320
There's story in the data, and how can we provide that to the team?

s50
00:05:58.320 --> 00:06:02.760
How can we provide that team at, you know, the weekly level, at the different levels of management?

s51
00:06:02.760 --> 00:06:11.280
How can we provide that information, that insight, that story to every customer that the sales teams are talking to?

s52
00:06:11.840 --> 00:06:19.720
And the third one, which is arguably the biggest one, is uh to provide self-service capabilities to the go-to-market team.

s53
00:06:19.720 --> 00:06:34.720
When these sales individuals, when they're talking to a customer, how can they get the expert-level information that they need to interact with that customer and to best assess, you know, how they should approach the situation, how to handle rejections,

s54
00:06:34.720 --> 00:06:39.600
how to upsell them, and how to handle customer satisfaction issues.

s55
00:06:39.600 --> 00:06:47.000
Uh this is a huge part of what I've seen we've done at Cloudflare, and I'll share a little bit what that looks like.

s56
00:06:47.080 --> 00:07:01.360
So, as it relates to scaling the analytical capability, the back-office operations, what we have done as we have built role-specific skill files, which have the context of the business information tying it to the data.

s57
00:07:01.360 --> 00:07:04.320
This is for both technical and non-technical users.

s58
00:07:04.320 --> 00:07:10.480
Technical users, you could say the ones who are building SQL and being able to data engineer a lot of solutions.

s59
00:07:10.480 --> 00:07:18.960
And then the non-technical people would be more individuals who are closer to the business with the sales people who may not know how to write SQL.

s60
00:07:18.960 --> 00:07:27.760
And so, we have skill files that they're able to use to ask questions of the data to get answers fairly quickly while doing other tasks.

s61
00:07:27.919 --> 00:07:38.000
And one example here is in those skill files, we also, through testing, we've included the types of questions that the business would ask of the data.

s62
00:07:38.000 --> 00:07:49.080
In this case, looking at closed date changes and opportunities, as well as uh changes in the amount of the opportunities, so that we can answer essentially 80% or or more of the questions,

s63
00:07:49.080 --> 00:07:54.400
where the other 20% might be more uh complex strategic questions.

s64
00:07:54.400 --> 00:08:02.840
And so, overall, this allows the teams to be able to embed all of the logic into the skill files and get answers uh fairly quickly.

s65
00:08:02.840 --> 00:08:14.480
So, I've seen users who do not know any SQL, and the essentially their request in the past we bottleneck to someone who knows data and can write SQL for complex queries, be able to just ask questions of the data

s66
00:08:14.480 --> 00:08:15.919
and get answers.

s67
00:08:15.919 --> 00:08:28.280
And this is very useful also for what I show later on on the third pillar is for building skills for these go-to-market teams, so that they um can and ask questions of their data

s68
00:08:28.280 --> 00:08:30.000
and and get answers.

s69
00:08:30.000 --> 00:08:34.719
Also, uh in our team we've used these same skill files to build multiple applications.

s70
00:08:34.719 --> 00:08:48.240
Uh when usually, you know, that is uh done in IT and bottlenecked in those areas, we're able to use the semantic information about the business um knowledge, as well as the columns table to build these applications

s71
00:08:48.240 --> 00:08:48.839
rather quickly.

s72
00:08:48.839 --> 00:08:54.880
So, this allows us to free up our time, so that we can focus on strategy and enablement.

s73
00:08:57.520 --> 00:09:03.920
All right, for the second pillar, uh what I mentioned earlier, there's a story in the data and they really shouldn't have to search for it.

s74
00:09:03.920 --> 00:09:07.160
Uh what I'm showing you here is synthetic data on the right.

s75
00:09:07.160 --> 00:09:18.839
Uh we have a weekly summary that goes out, which highlights how the business is doing, how they're pacing to their goals, and then highlighting um trends, uh standouts, as well as watches.

s76
00:09:18.839 --> 00:09:30.280
So, this uh we provide this information to the business, so they can, as you just like you can open your phone now uh on Gemini, if you have it, and you can see your notes for the day or the things that you need to do,

s77
00:09:30.280 --> 00:09:39.160
giving that same level of information to the go-to-market team, so they can just go along their day, and if they do need to look at some of the reports or dashboards, they can

s78
00:09:39.160 --> 00:09:42.960
to drill in, um but we bring the story to them.

s79
00:09:42.960 --> 00:09:46.000
And I'll pull this together why I think this is really important.

s80
00:09:46.000 --> 00:09:54.240
Um you know, of course there's a place for dashboards and standard information, but there's different level of adoption of the KPI metrics at any given company.

s81
00:09:54.240 --> 00:09:58.120
You're going to have people who are love dashboards, people are never going to look at them.

s82
00:09:58.120 --> 00:10:03.080
So, I think you really need to have a way to scaffold that across um the business.

s83
00:10:03.080 --> 00:10:05.560
So, how do we do this automated analysis?

s84
00:10:05.560 --> 00:10:14.480
So, a big part of this is simplifying the data so that the AI agents can actually um analyze the data in a very consistent and clean way.

s85
00:10:14.480 --> 00:10:24.680
Here what we do is we transform the data by the dimension of time, also slice of the logical part of the business, which is manager, theater, and finally the metric.

s86
00:10:24.680 --> 00:10:26.880
Here we have data uh that is wide.

s87
00:10:26.880 --> 00:10:29.560
You could also go um from wide to long.

s88
00:10:29.560 --> 00:10:38.400
Uh our trend information that I showed you, um uh that data is long, and then we do some pre-processing on that data um to then highlight trends.

s89
00:10:38.400 --> 00:10:49.880
So, the the uh embedding of the logic of how you would filter this data to even analyze it, um as well as the logical um aggregations the business want to see is all engineered

s90
00:10:49.880 --> 00:10:50.920
up front.

s91
00:10:50.920 --> 00:10:59.960
This from my experience, this handles uh 80 or plus percent of the requests is just getting information about the performance of the teams and how they're doing.

s92
00:10:59.960 --> 00:11:05.920
You can always go down to the raw data, but this last uh pillar, which I'll go over in a minute, um allows them to do that.

s93
00:11:05.920 --> 00:11:18.080
To be able to orchestrate this uh effectively and be able to rely on it, we have a multi-agent workflow where we first get the data, and then we do a first pass draft on the data calling our MCPs,

s94
00:11:18.080 --> 00:11:27.800
um and then this we have a second reviewer agent who checks the veracity of the data, and then we have a third um agent, which is a tone agent, who using a multi-shot prompt

s95
00:11:27.800 --> 00:11:33.280
um is able to just craft the message and highlight the risks and opportunities um equally.

s96
00:11:33.280 --> 00:11:41.880
And with every run, we have observability into each of the LLM calls, so we can see what is passed and what is the response that is going on there.

s97
00:11:41.880 --> 00:11:50.240
And so, this architecture we tested for about 2-3 months and, you know, looking every single run to see what is going wrong.

s98
00:11:50.240 --> 00:11:54.560
And this is the the model that we had set up that is really working for us.

s99
00:11:54.560 --> 00:12:03.640
And we really hope to expand this beyond just what I've shown you for multiple teams, but also down to the customer level like I was just talking to you about.

s100
00:12:03.640 --> 00:12:06.480
The third part is the self-service model.

s101
00:12:06.480 --> 00:12:17.080
And what I'm showing you here is our internal tool called Cloudflare OS, which is an agentic workspace that is running on Cloudflare where the go-to-market team can come in here.

s102
00:12:17.080 --> 00:12:29.320
It spins up their own compute and their own persistent environment using Cloudflare workers as well as durable objects, which is basically a storage sort of like S3.

s103
00:12:29.320 --> 00:12:36.000
And so, the sales people can come in here and get the data they need it when they need it.

s104
00:12:36.640 --> 00:12:46.760
So, some use cases that these teams are using it for is doing a forecast brief, building QBR decks, building a purchase deck on what the customer that they're onboarding has purchased,

s105
00:12:46.760 --> 00:12:51.880
doing account planning, general queries of the data, as well as renewal preparation.

s106
00:12:51.880 --> 00:13:01.120
How are they going to look at what the customer has used and either upsell them or figure out how they can get them adopting their product more.

s107
00:13:01.480 --> 00:13:05.920
So, just a little bit more into that Cloudflare S setup that I just showed you.

s108
00:13:05.920 --> 00:13:09.640
The AI agent workspace is where that screen I was I was showing you.

s109
00:13:09.640 --> 00:13:22.760
And through the the three-part piece of the skills in the lower left, which is our like expert level information, as well as the MCP connection and the AI gateway,

s110
00:13:22.760 --> 00:13:36.400
they're able to have conversations in this agentic workspace to pull data they need and using expert-level skills, which is curated, um so that they're able to execute the jobs that they need to do when they need to do it.

s111
00:13:36.400 --> 00:13:52.320
And so, a little more information about the skill repository, uh we have a central alias where skills are presented uh to uh the central team, curated by the uh go-to-market team, as well as by operations team, and they're reviewed, so we can make sure that we're not having a proliferation

s112
00:13:52.320 --> 00:14:03.480
of skills, and we have an expert-level knowledge skill at every level, so that they can really get all the information they need for how to approach uh any customer situation.

s113
00:14:04.360 --> 00:14:07.880
And so, just a few more uh um images here of uh them using it.

s114
00:14:07.880 --> 00:14:12.560
Here, uh we have them building a prescriptive plan for their daily work.

s115
00:14:12.560 --> 00:14:20.520
They're asking a question, it's their um you can see the agent is responding by looking into the MCP and starting to pull the data together.

s116
00:14:20.520 --> 00:14:29.040
And related to the QBR deck, uh here's a slide of generating a custom slide deck for a customer call.

s117
00:14:29.360 --> 00:14:38.839
And so, this has really, I think, unlocked the ability of the go-to-market teams to be able to really have all their information uh serviced to them.

s118
00:14:39.040 --> 00:14:42.320
And so, bring it together with the three pillars that I talked about.

s119
00:14:42.320 --> 00:14:53.400
Um if you really don't have all these, I think you have issues with serving the go-to-market needs uh in terms of uh using optimizing the use of Agentyc uh systems.

s120
00:14:53.400 --> 00:14:59.600
With self-service, you allow them to be able to pull data when they need it for whatever situation they need it with the expert-level information.

s121
00:14:59.600 --> 00:15:11.080
The the second is by pushing the uh the insights to the business, you're able to surface the generalized standardized information of how these teams are doing, and also um

s122
00:15:11.080 --> 00:15:15.760
yeah, so so that there's no and also uh so they're aligning with source of truth on performance.

s123
00:15:15.760 --> 00:15:25.280
Um and then, the the third one is the scaling of the analytical team for them to be able to answer queries and build applications for the teams, which really unlocks a lot

s124
00:15:25.280 --> 00:15:34.440
um because the opportunity cost of that team being overloaded and being able to help is that the meet the needs of the go-to-market team um is not met.

s125
00:15:34.440 --> 00:15:37.520
So, some findings um and the future.

s126
00:15:37.520 --> 00:15:43.520
So, uh the first thing is skill curation is the basis for all of this agentic workforce.

s127
00:15:43.520 --> 00:15:52.720
If you're able to embed the knowledge of the business into the skill files as well as the skills uh uh for the an analyst to be able to build

s128
00:15:52.720 --> 00:16:07.600
uh answer questions or build applications as well as the skills that I showed you in the Cloudflare OS, you're really able to uh give them the ability to use the agentic systems in a more predictable and deterministic way so that they can execute

s129
00:16:07.600 --> 00:16:09.560
um evenly across the board.

s130
00:16:09.560 --> 00:16:14.760
The second thing is through this whole process, the feedback loop is very important.

s131
00:16:14.760 --> 00:16:25.280
Just like uh a company would try to sell a product externally and get feedback, with these internal teams, uh the feedback loop is very important to be able to see is is what you're building is it actually useful?

s132
00:16:25.280 --> 00:16:26.839
What are some issues that they're having?

s133
00:16:26.839 --> 00:16:30.320
And how can you make this uh work more efficiently?

s134
00:16:30.320 --> 00:16:35.520
And the third thing is the layering of those uh three pillars that I talked about.

s135
00:16:35.520 --> 00:16:43.880
Being able to answer questions where the team comes to you, some of the go-to-market team, that's how they like to interface with the operations team is to be able to ask questions.

s136
00:16:43.880 --> 00:16:47.440
Um and then the pushing of information and then self-serviceability.

s137
00:16:47.440 --> 00:16:55.880
Through that, you're able to uh interweave all the needs of the team to be able to be met by this uh agentic run uh team.

s138
00:16:55.880 --> 00:17:08.959
So, through all these uh different pillars that I talked about, we've really been able to 2x our efficiency and be able to serve the teams as well as allowing them to be able to get the information that they need to do their job.

s139
00:17:08.959 --> 00:17:11.920
And some some things um that I see going into the future.

s140
00:17:11.920 --> 00:17:16.040
Number one is a deeper integration with uh our systems that we work in.

s141
00:17:16.040 --> 00:17:30.600
So, for example, those QBR decks and um renewal call skills, uh how can we set up meetings for the go-to-market team and embed those artifacts in those meetings so they don't have to actually pull them.

s142
00:17:30.600 --> 00:17:35.760
We can allow them to that self-service portal to be more ad hoc in what they need.

s143
00:17:35.760 --> 00:17:40.080
But, that requires some information or some security setup and how can we do that?

s144
00:17:40.080 --> 00:17:48.160
And then also, another example is getting meeting notes from those calls, which you have to set up that across the board.

s145
00:17:48.160 --> 00:17:51.640
So, there's some system side thing that we have to work on there.

s146
00:17:51.640 --> 00:17:59.720
The second thing is harder problems around quoting and approvals and updating the CRM itself.

s147
00:17:59.720 --> 00:18:08.920
Uh we use Salesforce and we're just in the midst of building the connections and the ability for us to update Salesforce with these Agenty systems.

s148
00:18:08.920 --> 00:18:19.160
And I see that being um set up in a way that I set up with that automated analysis where you have workflows uh to just make sure that everything is getting

s149
00:18:19.160 --> 00:18:20.280
um done right.

s150
00:18:20.280 --> 00:18:32.000
And the second thing is we're sort of reached the Cambrian stage of using Agenty systems, which means there's an explosion of excitement and skills and finding out ways to solve anything with AI.

s151
00:18:32.000 --> 00:18:50.760
But, I see as we get to this uh fuller integration and standardization, we're going to uh want to come back and and and not really limit, but just figure out a really strategic approach for allowing each team to use the Agenty system so that the source of truth in all the systems are aligning.

s152
00:18:50.760 --> 00:18:51.040
All right.

s153
00:18:51.040 --> 00:18:55.760
Well, thank you for joining this talk and I appreciate you, um you know, coming here.

s154
00:18:55.760 --> 00:18:57.093
Hope you have a great conference.

s155
00:18:57.093 --> 00:18:59.093
[applause]

s156
00:19:12.813 --> 00:19:14.813
[music]