WEBVTT

NOTE Sentence-level transcript of https://www.youtube.com/watch?v=b9UhZkKjX_A

NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/b9UhZkKjX_A.html. A cue ends where the next begins, or 2 s after its last word.

s1
00:00:01.309 --> 00:00:03.309
[music]

s2
00:00:12.960 --> 00:00:14.560
Hey everybody, thanks for coming.

s3
00:00:14.560 --> 00:00:15.960
My name's Jose Palafox.

s4
00:00:15.960 --> 00:00:18.320
I've been at GitHub for 6 and 1/2 years.

s5
00:00:18.320 --> 00:00:26.400
So, I joined the team right after the Microsoft acquisition of GitHub and then shortly after GitHub acquired a tool called Semmel.

s6
00:00:26.400 --> 00:00:28.600
We made an acquisition of a security company.

s7
00:00:28.600 --> 00:00:34.200
And so, my role at GitHub really has been growing our security business for the last five or six years.

s8
00:00:34.200 --> 00:00:45.880
In the last year, I really kind of focused on AI, but my role is to work in the field working with customers on how do they adopt our technology, how to think about scaling our technology inside of their organization.

s9
00:00:45.880 --> 00:00:55.120
So, today what we're going to talk about is really just like an introduction to agents on the platform and there's sort of two different modes that I like to think of agents in.

s10
00:00:55.120 --> 00:01:02.720
There's like autonomous agents that I have off running in the pipeline and then I have local agents that I'm using to do various bits of work.

s11
00:01:02.720 --> 00:01:11.320
So, I'm going to kind of walk you through sort of how to build those, how to think about them, how to think about scaling them, and think about different workflows that are possible using agents.

s12
00:01:11.320 --> 00:01:13.320
This talk is going to be fairly high-level.

s13
00:01:13.320 --> 00:01:17.200
So, if you have additional questions afterwards, you want to chat, feel free to grab me.

s14
00:01:17.200 --> 00:01:20.160
I'll kind of hang out right after the talk in the back of the room.

s15
00:01:20.160 --> 00:01:22.600
You can feel free to come by and chat with me afterwards.

s16
00:01:22.600 --> 00:01:25.000
We can talk a little bit deeper about your use cases.

s17
00:01:25.000 --> 00:01:28.960
This is a quick preview of just kind of things that are available on the platform.

s18
00:01:28.960 --> 00:01:38.320
Just getting us started, I always like to baseline with sort of the overall data flow of how Copilot works on the platform because a lot of people aren't familiar with how

s19
00:01:38.320 --> 00:01:42.520
Copilot works under the hood and and maybe aren't familiar with all of our different interfaces.

s20
00:01:42.520 --> 00:01:51.000
So, just to give you a quick preview of what the product has, you've got a handful of different interfaces that you can use to interact with the coding agent.

s21
00:01:51.000 --> 00:01:56.840
You've got the IDE, you've got the GitHub CLI, which is sort of a lightweight interface that sits on your desktop.

s22
00:01:56.840 --> 00:02:00.600
You can take any session from the CLI and you can stream it.

s23
00:02:00.600 --> 00:02:10.320
So, if you want to you know, close your laptop or walk away from your laptop for a period of time, you can either delegate the agent to github.com and have it go run in the background

s24
00:02:10.320 --> 00:02:18.640
or you can stream it using a command {slash} remote and it'll let you log in through your mobile device and actually steer the agent from your mobile device.

s25
00:02:18.640 --> 00:02:20.600
You've also got the SDK.

s26
00:02:20.600 --> 00:02:30.680
So, if you want to create agents that live inside of Slack or inside of Teams, I'm exposed some individual functionality to a set of team members that maybe don't have a GitHub license,

s27
00:02:30.680 --> 00:02:33.240
you can do that pretty easily through the through the SDK.

s28
00:02:33.240 --> 00:02:37.440
So, you can build applications using the CLI or the SDK.

s29
00:02:37.440 --> 00:02:46.520
Regardless of what door you go through on the top, you're going to hit this blue box here and this is what we call Cappy or the Copilot API service and

s30
00:02:46.520 --> 00:02:48.920
this is also referred to as like the harness, right?

s31
00:02:48.920 --> 00:03:00.160
This is the thing that does all the difficult engineering tasks taking your input and chunking them and sending them to the models making sure that you're not repeating any content from open source providers,

s32
00:03:00.160 --> 00:03:08.160
dealing with authentication for you, dealing with metrics logging, dealing with audit logging, all of the kind of like to-dos of the infrastructure of the platform.

s33
00:03:08.160 --> 00:03:12.959
And then of course, once you send a message in, we take that and we pass it to the model providers.

s34
00:03:12.959 --> 00:03:20.560
So, regardless of which door you go into, you hit the harness and then we take your output and we send it to the model providers.

s35
00:03:20.560 --> 00:03:24.320
Once we get the output back, we filter it and then display that back to you.

s36
00:03:24.320 --> 00:03:33.080
So, that's sort of under the hood how the product works just so you kind of have like a high-level understanding of the different doors you can go through and and where you can touch it.

s37
00:03:33.080 --> 00:03:42.680
Now, if you're starting in the CLI for example, you have a handful of embedded agents and I think it's useful to kind of look at the embedded agents just to understand

s38
00:03:42.680 --> 00:03:52.200
what agents do because there's like a lot of different you know, potential ways that you could implement whether it's an agent or a skill or you know, it all just ends up being marked down at the end of the day.

s39
00:03:52.200 --> 00:03:58.680
So, it's useful to kind of see how um we're implementing agents in the platform in a way that uh you can utilize.

s40
00:03:58.680 --> 00:04:05.440
So, I just type {slash} agents, and all this does is drop up the um the default agents that are part of the CLI harness here.

s41
00:04:05.440 --> 00:04:07.200
You can see I have a handful of them, right?

s42
00:04:07.200 --> 00:04:14.880
So, if I want to explore a code base, for example, I don't necessarily want to use the biggest model possible uh in order to do that.

s43
00:04:14.880 --> 00:04:24.080
So, we have a specialized research agent, which is our explorer agent, um that you can call or invoke uh through a keyword or actually invoking like the {slash} command

s44
00:04:24.080 --> 00:04:31.200
um in in in the CLI, and it'll spin up, you know, a small model using Haiku, um and go and investigate a code base, right?

s45
00:04:31.200 --> 00:04:35.160
If you um you know, have a plan that you've done, right?

s46
00:04:35.160 --> 00:04:43.320
And you've written out a planning document, and you want a different model provider to come and audit the plan because it has different training data and different model weighting,

s47
00:04:43.320 --> 00:04:44.760
uh then fine, you can do that.

s48
00:04:44.760 --> 00:04:52.320
Rubber duck is like a embedded agent that will validate a plan that we know was written by Opus with, you know, a pass from GPT or something like that.

s49
00:04:52.320 --> 00:05:00.560
Um you can see in all of these models they specify or in all of these agents they specify the model, and that's one of the options that you have when you're building custom agents.

s50
00:05:00.560 --> 00:05:05.520
You can always control the model that you're using, and then you control the instructions that you're giving it.

s51
00:05:05.520 --> 00:05:10.400
So, if these model or these um agents aren't doing, you know, something that you need done, right?

s52
00:05:10.400 --> 00:05:22.400
If you have a specific task or a review or you have a compliance or um regulatory requirement that you have to meet, and maybe you have a tool that you want to run or um a series of checks that you want to have that are custom to you.

s53
00:05:22.400 --> 00:05:24.400
May maybe you want to write an agent.

s54
00:05:24.400 --> 00:05:36.120
And you know, if you uh want, you can you can just use the agent command here, and it will um set you through like a a work uh a workflow to add an agent into your your CLI environment.

s55
00:05:36.120 --> 00:05:48.280
So, here I'm getting to pick, you know, do I want this on the project level like in the Git project that I'm working on, or do I want this as like a um an agent that's available regardless of which project I'm working on at the user level, right?

s56
00:05:48.280 --> 00:05:54.360
Um so, maybe I'll say I'm out of that at the at the org level or at my my my individual user level.

s57
00:05:54.360 --> 00:05:56.440
And then I go ahead and create this with Copilot.

s58
00:05:56.440 --> 00:05:58.760
Copilot will do all the scaffolding for me.

s59
00:05:58.760 --> 00:06:10.680
So, um if I uh want, I can generate an agent that maybe uh you know, looks at the GitHub change log and it um you know, will summarize all of the latest features or something that that Copilot did.

s60
00:06:10.680 --> 00:06:17.840
So, I did that earlier today and just generated a sample agent, you know, using this uh agent builder that's built into the platform, right?

s61
00:06:17.840 --> 00:06:20.120
And this is available in all of the interfaces.

s62
00:06:20.120 --> 00:06:25.960
Um you know, if you have the IDE, you have an agents interface there as well that will do the same kind of build build process with you.

s63
00:06:25.960 --> 00:06:28.080
But, what's nice is it scaffolds everything out.

s64
00:06:28.080 --> 00:06:33.480
So, from like a basic one-line prompt that just says, "Hey, I want to go to the change log and summarize all the changes."

s65
00:06:33.480 --> 00:06:34.840
It'll write the description for me.

s66
00:06:34.840 --> 00:06:36.200
It'll format all the front matter.

s67
00:06:36.200 --> 00:06:38.680
It will um structure success criteria.

s68
00:06:38.680 --> 00:06:41.080
It'll take an attempt at structuring methodology.

s69
00:06:41.080 --> 00:06:43.440
Like, it kind of knows how to write these files.

s70
00:06:43.440 --> 00:06:46.520
So, you get to be uh you know, somewhat lazy with it, right?

s71
00:06:46.520 --> 00:06:49.160
You you you just write a general description of what you want to get done.

s72
00:06:49.160 --> 00:06:51.080
We'll fill in all the specifics for you.

s73
00:06:51.080 --> 00:06:53.680
You can of course come in here and edit all of this, right?

s74
00:06:53.680 --> 00:07:00.360
So, if you don't like uh the order that it's um picking, you know, sources in this case, um we could change that, right?

s75
00:07:00.360 --> 00:07:07.200
Like, this one wants to start with, you know, use the GitHub roadmap, but maybe you want it just to jump straight to the change log, so you're not getting roadmap items.

s76
00:07:07.200 --> 00:07:08.560
You know, you can just edit it here, right?

s77
00:07:08.560 --> 00:07:09.720
There's no no big deal.

s78
00:07:09.720 --> 00:07:12.880
But, this does all the scaffolding work for you.

s79
00:07:12.880 --> 00:07:16.000
What's nice now is you've created like an agent that works in the harness.

s80
00:07:16.000 --> 00:07:21.240
So, if you have repeat tasks that you have to do, you can call this agent to do those tasks for you, right?

s81
00:07:21.240 --> 00:07:23.000
Um that that's kind of step one.

s82
00:07:23.000 --> 00:07:27.520
Uh but, once you do that, you're kind of like, "What's the next thing that I got to go do?"

s83
00:07:27.520 --> 00:07:31.160
And I think um you know, the next thing is how do I share this agent, right?

s84
00:07:31.160 --> 00:07:33.640
How do other people uh take advantage of it?

s85
00:07:33.640 --> 00:07:39.400
And the easiest way to do that inside of Copilot is to do that um by making a marketplace.

s86
00:07:39.400 --> 00:07:47.000
So, a marketplace is just a repository with a little bit of extra uh JSON in one of the directories that says, "Hey, this is a marketplace."

s87
00:07:47.000 --> 00:07:51.600
And what it does is it gives you some light package management capabilities inside the repo.

s88
00:07:51.600 --> 00:07:57.000
So, if I subscribe to a marketplace, then I can update all the agents simultaneously.

s89
00:07:57.000 --> 00:07:58.640
I can version them, things like that.

s90
00:07:58.640 --> 00:08:00.960
I can notify people of security vulnerabilities.

s91
00:08:00.960 --> 00:08:05.400
You know, there's features built in that allow me to communicate with the users that are downstream for me.

s92
00:08:05.400 --> 00:08:12.440
So, creating a marketplace is oftentimes one of the first things that I work with companies on when I go in and try to talk with them about how to scale

s93
00:08:12.440 --> 00:08:15.000
Copilot adoption inside their organization.

s94
00:08:15.000 --> 00:08:19.520
A lot of times there's not a central repository for teams to even contribute agents or skills to.

s95
00:08:19.520 --> 00:08:23.600
So, if you don't have a central location, you should generate one, right?

s96
00:08:23.600 --> 00:08:31.080
And there is a capability basically to sync those with the the local client device and then pushing and out different agents.

s97
00:08:31.080 --> 00:08:38.159
You can also use a specific directory inside of your organization or specific repo.

s98
00:08:38.159 --> 00:08:45.720
So, if you're inside of GitHub, I'm at the org level here, and then I went into a repository, and it has to be named GitHub private, right?

s99
00:08:45.720 --> 00:08:52.760
This is a naming convention that we've used, but if you have a GitHub private repository, you'll be able to expose agent files.

s100
00:08:52.760 --> 00:09:01.200
So, in this case, I have three agents here, and you can see from the cloud agent, I'm actually able to call that specific agent on the platform.

s101
00:09:01.200 --> 00:09:06.640
So, I showed you earlier in the diagram, you know, you can you can interact with all of these from different surface areas.

s102
00:09:06.640 --> 00:09:08.200
This is the cloud interaction, right?

s103
00:09:08.200 --> 00:09:12.120
If I want to delegate to a cloud agent that's hosted on the platform, I can do that here.

s104
00:09:12.120 --> 00:09:19.160
But what's nice is this compliance bot that I'm showing you here in the the drop-down is the same compliance bot that's in the repository, right?

s105
00:09:19.160 --> 00:09:23.200
So, this is a way of distributing agents out to the team, right?

s106
00:09:23.200 --> 00:09:30.480
So, if I write a handy agent that checks the change log for me, then I can publish it here, and I can share that with everybody else, right?

s107
00:09:30.480 --> 00:09:37.760
So, this will push to everyone's IDE environment or push to their CLI environment for them, and then they can just call the same agent that I'm working on.

s108
00:09:37.760 --> 00:09:39.840
So, now I have some reuse, right?

s109
00:09:39.840 --> 00:09:47.000
Like I can publish them in a marketplace, I can publish them directly as um assets inside of my business that anybody can call, right?

s110
00:09:47.000 --> 00:09:50.000
And this is what's allowing me to do standardization, right?

s111
00:09:50.000 --> 00:09:55.480
If you look at the cost model for all of these products, the cheapest way to use them is to get cash hits, right?

s112
00:09:55.480 --> 00:09:59.440
You want exact matches between turns on common data.

s113
00:09:59.440 --> 00:10:02.760
And having reusable components will drive up your cash rate, right?

s114
00:10:02.760 --> 00:10:16.880
So, if cost is a concern, um building reusable agents and making sure that people are solving problems in the same way, uh rather than reinventing the wheel every time, is a good way to drive down your your overall bill, drive up your cash rate.

s115
00:10:16.880 --> 00:10:23.760
So, again, this um dot GitHub private directory, this is how you would share an agent internally to everyone inside the org, right?

s116
00:10:23.760 --> 00:10:29.200
So, you can make a marketplace that will work across organizations, that could even work, you know, externally, right?

s117
00:10:29.200 --> 00:10:34.400
So, we have a repository called awesome co-pilot that has a bunch of um you know, example agents and things.

s118
00:10:34.400 --> 00:10:36.120
You could register that as a marketplace.

s119
00:10:36.120 --> 00:10:37.680
This is purely internal, right?

s120
00:10:37.680 --> 00:10:41.680
And this is like an admin can push this out.

s121
00:10:41.680 --> 00:10:50.560
The last thing that I want to do sometimes um when I'm working uh with an agent is I want to be able to use um agents across repositories.

s122
00:10:50.560 --> 00:10:58.480
And the biggest challenge with these uh compliance agents, for example, is that they're always starting in the repo where the agent lives.

s123
00:10:58.480 --> 00:11:05.400
So, if I have agents living in this repository, um they maybe can't take action in other repositories, right?

s124
00:11:05.400 --> 00:11:10.640
I need to start the agent in the other repo so that it has permissions in that repository.

s125
00:11:10.640 --> 00:11:14.760
Um inside of the co-pilot app, there's a command called orchestrate.

s126
00:11:14.760 --> 00:11:19.280
Um and orchestrate is our horizontal scaling for across repo.

s127
00:11:19.280 --> 00:11:28.680
Uh so, this will allow you to set a series of GitHub issues as uh goals for the agent, and then spin up sub agents to address all of those tasks.

s128
00:11:28.680 --> 00:11:33.680
But, what's nice is that it will spin up agents in the repository that it's supposed to be working in.

s129
00:11:33.680 --> 00:11:44.640
So, if you want to take a task inside of a microservice where you're like modifying an API and then you have to make a change somewhere downstream because of that, Orchestrate will allow you to set up several agents that are going to make

s130
00:11:44.640 --> 00:11:46.600
cross-repo changes simultaneously.

s131
00:11:46.600 --> 00:11:49.840
So, I can use the the pre-canned agents, right?

s132
00:11:49.840 --> 00:11:58.280
If they're available to me in those other marketplaces or being pushed to me by my org admin, I can kind of summon them here to go do work in individual repositories

s133
00:11:58.280 --> 00:12:01.520
and keep them sort of on task using this this Orchestrate tool.

s134
00:12:01.520 --> 00:12:06.760
And that's really like three layers of how do I use agents locally, right?

s135
00:12:06.760 --> 00:12:11.040
Like in in using agents locally has a lot of utility, right?

s136
00:12:11.040 --> 00:12:13.960
But sometimes I don't want to run an agent locally.

s137
00:12:13.960 --> 00:12:22.520
I actually want the agent to be run as a part of the repository and certain agents maybe are going to be performing like maintenance tasks in the repo.

s138
00:12:22.520 --> 00:12:28.480
And so, I want to run them in the repo on that repo's budget, not on my budget, right?

s139
00:12:28.480 --> 00:12:32.040
Or I want to run it using the repo identity, not my identity, right?

s140
00:12:32.040 --> 00:12:35.480
Not every agent needs to be me taking an action.

s141
00:12:35.480 --> 00:12:40.480
You know, it's it's maybe a maintenance task that has to be done on behalf of of the project.

s142
00:12:40.480 --> 00:12:45.720
When we're working in that model, we have a framework that you can use called agentic workflows.

s143
00:12:45.720 --> 00:12:52.360
And agentic workflows is a pretty simple concept like the Copilot CLI can be used in a headless mode.

s144
00:12:52.360 --> 00:13:00.680
So, I can call the Copilot CLI, I can use the dash P command and basically pass it a single prompt one-shot prompt for it to go do.

s145
00:13:00.680 --> 00:13:08.040
And then if I take that and I run that inside of a GitHub Actions runner or in some CI system, then I have an agent that's running in a pipeline for me, right?

s146
00:13:08.040 --> 00:13:11.200
So, I can trigger it on events that are happening in the repo.

s147
00:13:11.200 --> 00:13:15.200
Like on a release, I want to update I want to update documentation.

s148
00:13:15.200 --> 00:13:18.120
Or every week, I want to update my context files.

s149
00:13:18.120 --> 00:13:23.040
Or, you know, in maybe a model like this, I have various maintenance tasks that I want to go do.

s150
00:13:23.040 --> 00:13:31.800
Like one of the biggest problems today that's happening in repositories where a lot of agentic development is happening is there's lots of spaghetti code getting added, lots of duplicate code.

s151
00:13:31.800 --> 00:13:38.000
Like the agents are lazy about implementing, you know, functions and they're really lazy about namespacing.

s152
00:13:38.000 --> 00:13:41.560
So, they'll just like create constant duplication inside the code base.

s153
00:13:41.560 --> 00:13:52.080
So, maybe you want an agent that looks inside the code base, identifies opportunities for deduplication of code, and then creates a series of issues for you.

s154
00:13:52.080 --> 00:14:05.200
And if you use this framework, this agentic workflows framework, what's nice is we'll we'll build in human in the loop triggers for you so that you can review and decide when you're actually going to spend money on side of the next phase of the flow.

s155
00:14:05.200 --> 00:14:09.640
So, in this flow here, you see I have the first agent maybe running on a cron schedule.

s156
00:14:09.640 --> 00:14:10.760
It's looking in the repo.

s157
00:14:10.760 --> 00:14:15.680
It identifies, you know, three items where there's opportunity for me to make improvement in the code base.

s158
00:14:15.680 --> 00:14:16.480
Okay, great.

s159
00:14:16.480 --> 00:14:21.480
The developer reads that and then makes a decision about whether or not those things are worth fixing.

s160
00:14:21.480 --> 00:14:26.640
And they express that through a slash command that they can make as an issue comment, right?

s161
00:14:26.640 --> 00:14:31.680
So, they they go to the issue and they say, "Okay, advance this issue to the next phase of the process, right?"

s162
00:14:31.680 --> 00:14:33.680
For us, that's maybe a planning phase, right?

s163
00:14:33.680 --> 00:14:36.720
So, maybe I'm going to call another pipeline agent, right?

s164
00:14:36.720 --> 00:14:39.960
Which is the PRD generating product manager agent, right?

s165
00:14:39.960 --> 00:14:41.920
That's going to scope the feature for me.

s166
00:14:41.920 --> 00:14:43.360
It generates a plan.

s167
00:14:43.360 --> 00:14:46.839
It hands it off to co-pilot in the platform for implementation.

s168
00:14:46.839 --> 00:14:49.839
And then the developer comes back in the loop maybe at an acceptance phase, right?

s169
00:14:49.839 --> 00:14:53.440
Let's look at the code and evaluate whether or not we actually want to take it in the project.

s170
00:14:53.440 --> 00:14:55.520
Now I've got pipeline agents.

s171
00:14:55.520 --> 00:15:00.360
And what's really cool about the pipeline agents is how they scale.

s172
00:15:00.360 --> 00:15:06.280
You know, the the the the challenge with working locally is you're on like a single session or you're on your device.

s173
00:15:06.280 --> 00:15:11.040
If your device goes down or you lose network activity, you know, your your workflow is interrupted.

s174
00:15:11.040 --> 00:15:15.600
These are running in the platform, so they're running autonomously from you.

s175
00:15:15.600 --> 00:15:23.880
If you want to see like a maximalist approach to how this works, the GitHub AW project itself, like the thing that is the framework for building all these agents,

s176
00:15:23.880 --> 00:15:26.480
it is built entirely by agents.

s177
00:15:26.480 --> 00:15:36.320
And this team has sort of an internal goal that they're only going to do their development on their mobile devices because they interact entirely with this application through those slash commands, right?

s178
00:15:36.320 --> 00:15:43.880
So just to give you a feel of like what a fully, you know, agent-led development project looks like, this is this is our our our framework, right?

s179
00:15:43.880 --> 00:15:51.240
And it has I don't know, 200 agents in here that are all running different parts of the pipeline, right?

s180
00:15:51.240 --> 00:15:57.640
And all of these follow a similar workflow that we just showed on that on that slide, right?

s181
00:15:57.640 --> 00:15:59.880
That architecture is reusable.

s182
00:15:59.880 --> 00:16:02.880
So let me see if I can just pick one of these.

s183
00:16:02.880 --> 00:16:06.920
I'm a little bit picking at random here, but let's see what this one does, right?

s184
00:16:06.920 --> 00:16:11.160
So this is checking daily reports on other agents, right?

s185
00:16:11.160 --> 00:16:13.360
So this is a meta agent that's looking on things.

s186
00:16:13.360 --> 00:16:19.840
It's running on a daily schedule and it has, you know, its agent.md file that's working away here, right?

s187
00:16:19.840 --> 00:16:28.680
So every time this thing runs, it's generating some markdown report or some action or input for another agent, and then we're just continuing to trigger it.

s188
00:16:28.680 --> 00:16:30.560
You can feel free to take a look through here.

s189
00:16:30.560 --> 00:16:33.600
There's hundreds of these that are creating this project.

s190
00:16:33.600 --> 00:16:37.000
GetHub also has a bunch of them that we've bundled up.

s191
00:16:37.000 --> 00:16:41.240
So inside of our GetHub next organization, there's this project called Agentics.

s192
00:16:41.240 --> 00:16:48.200
Agentics has 30 or so different pipeline agents that you may be interested in just pulling off the shelf and running in your project.

s193
00:16:48.200 --> 00:16:51.480
Some of these will do the code deduplication that we talked about.

s194
00:16:51.480 --> 00:16:59.680
Some of them are different types of reviewers, so if you need a QA reviewer or, you know, you want one that's funny and just makes jokes, like there's all kinds of

s195
00:16:59.680 --> 00:17:02.960
different sample agents here that you can pull and drop into the pipeline.

s196
00:17:02.960 --> 00:17:06.800
But what's nice about this is it gets the workload off of your machine.

s197
00:17:06.800 --> 00:17:14.000
And I think when we shift back to thinking about cost, getting the workload off of your machine is key for optimizing cost, right?

s198
00:17:14.000 --> 00:17:22.720
If I'm having everyone in my organization work with AI and I'm telling them to go use it, but I can't see how they're actually working with the tool.

s199
00:17:22.720 --> 00:17:24.800
I don't have any opportunity to optimize, right?

s200
00:17:24.800 --> 00:17:31.840
I can't actually see whether or not the agents that they're using are too verbose or are even achieving the task.

s201
00:17:31.840 --> 00:17:36.520
I can't tell when tool calls are happening if they're successful or unsuccessful.

s202
00:17:36.520 --> 00:17:47.040
I I I can't tell if they're using agent reasoning for all types of things or if they're you know correctly building little bash scripts to actually go and do uh different search functions and things like that.

s203
00:17:47.040 --> 00:17:52.240
Like there's lots of optimization opportunities for me, but if I can't see it, I can't do any of that optimization.

s204
00:17:52.240 --> 00:17:58.640
So, by getting the workload off of the laptop and moving it into CI, I get to see what's happening, right?

s205
00:17:58.640 --> 00:18:02.600
I have every single agent running and it's instrumented with hotel.

s206
00:18:02.600 --> 00:18:11.400
So, I can go to a SIM or some kind of logging platform and look, does my code duplication agent agent, how does it run across repositories?

s207
00:18:11.400 --> 00:18:12.840
How much does it cost me, right?

s208
00:18:12.840 --> 00:18:15.080
I can pivot on average token use.

s209
00:18:15.080 --> 00:18:25.760
I can also do things like say, "Hey, every time that an agent exceeds its P90 for what I expect it to cost me to run, go investigate why it exceeded its its threshold, right?

s210
00:18:25.760 --> 00:18:33.000
I want it to tell me what debugging information I can I can take to evaluate it further or to improve it, right?

s211
00:18:33.000 --> 00:18:45.160
Or if I see that a tool call's failing over and over, maybe I want to AB test different front matter or maybe I want to AB test different models and see if they produce like the the right output for me so I can pick a cheaper model at some point, right?

s212
00:18:45.160 --> 00:18:50.560
All of this stuff is possible once I get the agents into the pipeline.

s213
00:18:50.560 --> 00:18:55.000
Anyhow, so this was a quick and dirty kind of tour of agents in the platform, right?

s214
00:18:55.000 --> 00:19:02.240
We talked through building an agent locally, sharing agents through the platform, and then eventually moving those agents into the pipeline.

s215
00:19:02.240 --> 00:19:05.320
If you want to talk more about any part of that, I'll be in the back.

s216
00:19:05.320 --> 00:19:09.400
Feel free to come up and grab me or you can add me on LinkedIn and ping me.

s217
00:19:09.400 --> 00:19:11.520
I'm happy to continue the conversation.

s218
00:19:11.520 --> 00:19:17.240
Um, I work at GitHub in a field capacity, so if you know your GitHub AE and you want to chat with me, uh, feel free to reach out with them.

s219
00:19:17.240 --> 00:19:20.200
Uh, we we can get in touch and and have a quick conversation as well.

s220
00:19:20.200 --> 00:19:23.040
Thanks so much for your time today.