WEBVTT

NOTE Sentence-level transcript of https://www.youtube.com/watch?v=48YUYDjwfYY

NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/48YUYDjwfYY.html. A cue ends where the next begins, or 2 s after its last word.

s1
00:00:13.280 --> 00:00:14.559
All right, guys.

s2
00:00:14.559 --> 00:00:16.000
Very nice to meet you all.

s3
00:00:16.000 --> 00:00:16.880
My name is Laura.

s4
00:00:16.880 --> 00:00:24.240
I'm the co-founder and president of Castai, who is the inventor of Kimchi, which I'm going to show you in a few seconds.

s5
00:00:24.240 --> 00:00:35.120
Um the talk is going to be about preparatory models for edge case but use open source for the rest and you're going to see some great result we have with that with the coding agent called Kimchi.

s6
00:00:35.120 --> 00:00:38.160
I have the pleasure to be here with Zilinas.

s7
00:00:38.160 --> 00:00:39.440
Hey guys, I am Julianas.

s8
00:00:39.440 --> 00:00:42.320
I am leading engineering for Kimchi coding platform.

s9
00:00:42.320 --> 00:00:45.920
So you have Zilinas who run the entire engineering team for Kimchi.

s10
00:00:45.920 --> 00:00:48.160
I'm so excited to be here with you.

s11
00:00:48.160 --> 00:00:48.960
All right, let's go.

s12
00:00:48.960 --> 00:00:56.879
So um you you may have seen this in the news uh two two piece of information that came up in the last few weeks.

s13
00:00:56.879 --> 00:01:08.479
The first one is there's a company in India that spent $500 million on anthropic alone in one months which is a kind of a good example of tokens that are completely out of control.

s14
00:01:08.479 --> 00:01:20.560
The other example is a famous tweet from Uber, the CTO of Uber, who say that in the last four months he actually used the entire budget of anthropic for the entire year.

s15
00:01:20.560 --> 00:01:28.159
And this is how much the token um the token maxing or the token cost is a thing for the enterprises.

s16
00:01:28.159 --> 00:01:30.960
And you're going to see how we fix this.

s17
00:01:30.960 --> 00:01:34.960
Companies responded this by limiting the use of token.

s18
00:01:34.960 --> 00:01:45.200
You may have heard this many many times with our friends developers and as a developer when you limit the amount of token it feels like this.

s19
00:01:45.200 --> 00:01:54.720
It's like hey you can use your laptop you have an hour battery and you can only charge once once once a day.

s20
00:01:54.720 --> 00:02:00.240
That's how it feels to limit the amount of token for a developer.

s21
00:02:00.240 --> 00:02:03.439
We at Kimchi think this is wrong.

s22
00:02:03.439 --> 00:02:17.840
Our jobs as manager is to do the opposite is to ensure that we want to make the tokens unlimited and inexpensive.

s23
00:02:17.840 --> 00:02:21.360
Our job is not to prevent the developer to use coding agent.

s24
00:02:21.360 --> 00:02:29.040
Our job is to make sure they can use it as much as they want for as long as they want in a completely unlimited fashion.

s25
00:02:29.040 --> 00:02:32.400
This is why we started Kimchi.

s26
00:02:32.400 --> 00:02:35.680
At Kimchi, we have 300 developers.

s27
00:02:35.680 --> 00:02:36.239
Oh, sorry.

s28
00:02:36.239 --> 00:02:38.480
300 employees.

s29
00:02:38.480 --> 00:02:40.560
Twothird of those are developers.

s30
00:02:40.560 --> 00:02:42.560
The Zeven has run that.

s31
00:02:42.560 --> 00:02:46.480
And we've been using this coding agent for the last three months.

s32
00:02:46.480 --> 00:02:52.879
I'm going to show you what we built, how we built it, and the result of it.

s33
00:02:52.879 --> 00:02:59.840
And then Zilvenat will show you a few more things that make the coding really, really cool.

s34
00:03:00.400 --> 00:03:03.040
It's coming now.

s35
00:03:03.120 --> 00:03:08.720
We built an entire token optimization based on this idea.

s36
00:03:08.720 --> 00:03:15.680
What you see on screen is the true cost of LLM model for the same task.

s37
00:03:15.680 --> 00:03:19.840
It was made by the famous white paper in a university.

s38
00:03:19.840 --> 00:03:21.840
And here's what they say.

s39
00:03:21.840 --> 00:03:25.440
On one side, you see the cost per token.

s40
00:03:25.440 --> 00:03:32.319
On the other side, you see the cost per task for the same task with the same quality.

s41
00:03:32.319 --> 00:03:33.440
That is the important thing.

s42
00:03:33.440 --> 00:03:35.599
Otherwise, you cannot compare Apple with Apple.

s43
00:03:35.599 --> 00:03:37.120
And here's what is written here.

s44
00:03:37.120 --> 00:03:44.640
First, they are all different, but also look at Gemini 3 Flash, our friends at Google.

s45
00:03:44.640 --> 00:03:51.200
What it seems to be a relatively inexpensive model at $3.5 per million token.

s46
00:03:51.200 --> 00:04:02.959
It's a blended average at the time this study was made, but actually the task to complete with Geminina 3 was 305, sorry, $75.

s47
00:04:03.360 --> 00:04:04.480
And look at the others.

s48
00:04:04.480 --> 00:04:07.680
Minimax 2.7 is one of my favorite.

s49
00:04:07.680 --> 00:04:15.680
The cost per token is 1.5 and the task the t the cost for the task is $148.

s50
00:04:15.680 --> 00:04:18.400
It means the models are not the same.

s51
00:04:18.400 --> 00:04:23.919
They don't cost the same, but they also not the same per task.

s52
00:04:23.919 --> 00:04:27.919
And that's why we built our coding agent.

s53
00:04:27.919 --> 00:04:39.360
Would it be nice that you have an automated fashion, automated engine, automated harness that will select the right model for the task at the right time based on the outcome

s54
00:04:39.360 --> 00:04:41.440
of the task.

s55
00:04:41.440 --> 00:04:44.639
That is what we did with Kimy Coding.

s56
00:04:44.639 --> 00:04:48.639
That is what Zilmanas did with his team.

s57
00:04:48.639 --> 00:04:50.000
And here's the results.

s58
00:04:50.000 --> 00:04:52.880
We've been using it for three months.

s59
00:04:52.880 --> 00:04:58.960
2.5 times in the amount of savings we have overloaded for the period.

s60
00:04:58.960 --> 00:05:05.440
This is a graph of what would have been our cloud bill by day.

s61
00:05:05.440 --> 00:05:12.000
The purple one on top is what would have been cloud cost over time.

s62
00:05:12.000 --> 00:05:16.400
The blue line is what we paid.

s63
00:05:17.039 --> 00:05:19.120
same outcome.

s64
00:05:19.120 --> 00:05:20.160
But look at this.

s65
00:05:20.160 --> 00:05:26.080
The number of tokens increased by 1.5 times.

s66
00:05:26.080 --> 00:05:28.720
Okay, we all experienced that.

s67
00:05:28.720 --> 00:05:34.080
The cost to code decreased by 1.5 times.

s68
00:05:34.400 --> 00:05:36.320
This is our job as managers.

s69
00:05:36.320 --> 00:05:52.560
We provide to our team a coding agent that is essentially unlimited because of that coding agent is growing 3x less 2.5 time less than the gross of token for the task.

s70
00:05:52.720 --> 00:05:57.280
There's another result the multimodel I showed you before.

s71
00:05:57.280 --> 00:06:05.520
I'm going to show you the evolution of the selection of the model over time by the harness.

s72
00:06:05.520 --> 00:06:14.639
And here it looks like you see there is a lot of orange early June and there's a lot of yellow late June.

s73
00:06:14.639 --> 00:06:19.039
There's a complete shift meaning the harness is selecting a very different model.

s74
00:06:19.039 --> 00:06:23.440
It happened the 12th of June.

s75
00:06:23.840 --> 00:06:34.639
And this is what happened the 12th of The 3rd of June Kim Kim 2.6 is winning by far the most popular of the harness.

s76
00:06:34.639 --> 00:06:46.800
21st of June the one winning is Minimat 3. Us as human probably we can find and understand and test and be comfortable with a new model coming in.

s77
00:06:46.800 --> 00:06:50.080
Do you think we look at this as human?

s78
00:06:50.080 --> 00:06:51.520
Absolutely not.

s79
00:06:51.520 --> 00:07:02.800
But an autonomous harness, an automated coding agent that is obsessed with cost is going to do just that.

s80
00:07:03.759 --> 00:07:16.000
That is what I mean when I say our responsibility is to think about providing our developers with the right cost per token for the same outcome.

s81
00:07:16.000 --> 00:07:19.599
It is our responsibility to do it.

s82
00:07:19.599 --> 00:07:27.440
Zinas is going to tell you a little bit more of how it works because whatever you're going to see it's open source.

s83
00:07:27.440 --> 00:07:29.759
Go Zas.

s84
00:07:30.720 --> 00:07:31.120
Okay.

s85
00:07:31.120 --> 00:07:32.720
So one more thing to mention.

s86
00:07:32.720 --> 00:07:51.440
So Laura mentioned that token maxing is no longer an option for everyone and uh as a manager at the company as most of you guys probably heard the having unlimited amount of tokens to spend is no longer looking to be like a case and uh we had to build this hardness for ourselves.

s87
00:07:51.440 --> 00:07:55.280
So meaning like we ended up spending too much on cloud code like half a year ago.

s88
00:07:55.280 --> 00:07:59.599
We saw this uh cloud bill skyrocketing and we started working on our own hardness.

s89
00:07:59.599 --> 00:08:09.199
behind the scenes it's using like pono SSDK but uh the goal of that is like to take the open source experience still contribute back to open source by providing

s90
00:08:09.199 --> 00:08:21.360
our own harnesses open source building constructs like ferment which fits into the kimchi theme which is meant for longunning tasks where on average we estimate the human envelope for these tasks is uh

s91
00:08:21.360 --> 00:08:30.000
typically happening only like every two hours every three hours and etc. So the goal is uh you use Kim to harness you run a firmment process.

s92
00:08:30.000 --> 00:08:35.919
It asks you a bunch of questions and autonomously runs a coding task for like more than two hours.

s93
00:08:35.919 --> 00:08:39.440
The goal of that is that because it breaks it into milestones.

s94
00:08:39.440 --> 00:08:44.080
It knows how to implement it and at the end of the cycle you would get scoring for your output.

s95
00:08:44.080 --> 00:08:46.320
So you would actually know what's happening.

s96
00:08:46.320 --> 00:08:57.839
And going back to previously discussed like the amount of models used with uh Kimchi harness the scoring is there to prove you that whatever model we will pick for you you will always have like

s97
00:08:57.839 --> 00:09:01.600
good enough scoring for the output on the task you're implementing.

s98
00:09:01.600 --> 00:09:07.920
The way whole Kimchi platform works the goal is like uh we are not just providing you the coding harness.

s99
00:09:07.920 --> 00:09:11.920
We want to provide you the full software development life cycle solution.

s100
00:09:11.920 --> 00:09:15.279
So you do a change it runs a build for you.

s101
00:09:15.279 --> 00:09:16.480
It checks if something broke.

s102
00:09:16.480 --> 00:09:20.560
If something broke, it will go back, do a change again, rebuild again.

s103
00:09:20.560 --> 00:09:23.600
And that's uh how this fermentation process looks.

s104
00:09:23.600 --> 00:09:27.600
If everything is complete, uh there is no extra limitation needs to be done.

s105
00:09:27.600 --> 00:09:31.200
The changes will be deployed to the staging environment.

s106
00:09:31.200 --> 00:09:44.240
And you see what what I like with the way explain it is the harness select the model fment is constantly checking the quality of the code and going back

s107
00:09:44.240 --> 00:09:54.399
meaning there is a a higher order model that is checking that quality and thinking that quality is not good enough please do it again this is what I was saying earlier by

s108
00:09:54.399 --> 00:10:02.160
it looks at the outcome and it's optimizing the token based on the quality of the outcome.

s109
00:10:02.160 --> 00:10:04.320
That is what is super cool with FET.

s110
00:10:04.320 --> 00:10:04.720
Yep.

s111
00:10:04.720 --> 00:10:11.839
And artifact is only considered complete once like the output of the scoring like at least B as a score.

s112
00:10:11.839 --> 00:10:20.880
And even for that like if you don't like the score you can ask agent to go back fix it up and like it will try to achieve the A as a scoring.

s113
00:10:21.279 --> 00:10:25.680
The intention of stuffing at the staging is currently by design.

s114
00:10:25.680 --> 00:10:29.760
We still believe there's like some little human that needs to be in the loop before going to the production.

s115
00:10:29.760 --> 00:10:34.640
At least at this point we all know that the ultimate f future is to avoid that.

s116
00:10:34.640 --> 00:10:41.600
You go to production, you look at the metrics, you look at the observability data, the solo metrics you have in your environment.

s117
00:10:41.600 --> 00:10:46.800
If you're running on Kubernetes, you want to check if the pots are not crashing, if application health checks are passing.

s118
00:10:46.800 --> 00:10:51.680
So all of these things need to be done before it ship it shipped to production automatically.

s119
00:10:51.680 --> 00:10:53.839
So this is like one of the things that are coming.

s120
00:10:53.839 --> 00:11:04.399
So the current limitation we stop at staging but uh we'll go to production and and further on that what we believe is that reading code is not enough anymore.

s121
00:11:04.399 --> 00:11:23.279
So like uh probably most of you noticed like across this year like uh the more vibe engineering or agentic engineering we do in our coding environments the more code changes we get in our chain sets and it kind of becomes unbearable to read especially as you keep bringing like a little bit of a less technical personnel into

s122
00:11:23.279 --> 00:11:24.240
your pipelines.

s123
00:11:24.240 --> 00:11:41.760
So imagine like product manager doing the coding or my dear colleague Laura starting to do some coding and suddenly he comes up with a PR which is like 2,000 lines long and you need to actually review not just the differences but the actual intent which he wanted to achieve and the specifications

s124
00:11:41.760 --> 00:11:44.240
which he provided to the coding agent.

s125
00:11:44.240 --> 00:11:52.480
So yeah moving on uh one thing we built more on top of that is the thing we called kimchi teleport.

s126
00:11:52.480 --> 00:11:54.800
So Kimchi teleport allows you to close your laptop.

s127
00:11:54.800 --> 00:11:57.600
It spins up sandbox in the remote environment.

s128
00:11:57.600 --> 00:12:00.240
It's a secure sandbox where you don't need to care about anything.

s129
00:12:00.240 --> 00:12:02.880
We can do that both on our SAS platform.

s130
00:12:02.880 --> 00:12:09.120
We can ship it to your on premise deployments if you're like a financial technology company and uh things like that.

s131
00:12:09.120 --> 00:12:11.600
So we will manage this agent sessions for you.

s132
00:12:11.600 --> 00:12:12.720
We will make sure they are running.

s133
00:12:12.720 --> 00:12:17.360
We are accessible both from your laptop, from mobile device if we are running in the background.

s134
00:12:17.360 --> 00:12:24.240
So it goes back and ties to the story where I mentioned that the human in the loop is only appearing like every two or three hours.

s135
00:12:24.240 --> 00:12:26.880
So teleport emphasizes that.

s136
00:12:26.880 --> 00:12:30.480
You run your process, you teleported to the remote environment.

s137
00:12:30.480 --> 00:12:32.000
The y spins up in the background.

s138
00:12:32.000 --> 00:12:33.279
You don't need to care about it.

s139
00:12:33.279 --> 00:12:37.680
We will make sure it's the right size, it's the right security requirements and everything else.

s140
00:12:37.680 --> 00:12:43.200
And it will keep on running until the task is done or until the human in the loop needs to be involved.

s141
00:12:43.200 --> 00:12:48.320
Get notifications for some clarification of specs and things like that.

s142
00:12:48.320 --> 00:12:50.480
Guys, I really like this idea.

s143
00:12:50.480 --> 00:12:54.880
Look at the this this is a screenshot of teleport.

s144
00:12:54.880 --> 00:13:00.160
I am puzzled that sometimes innovations are always the simplest one.

s145
00:13:00.160 --> 00:13:08.880
Like this was born out of the idea that one of us was in the plane coding and then Wi-Fi breaks.

s146
00:13:08.880 --> 00:13:10.800
And what do you do when Wi-Fi breaks?

s147
00:13:10.800 --> 00:13:15.680
It breaks the coding because it lives inside your thing.

s148
00:13:15.680 --> 00:13:17.120
No, I think it was not the Wi-Fi.

s149
00:13:17.120 --> 00:13:20.639
It was the battery that broke, not the Wi-Fi.

s150
00:13:20.639 --> 00:13:21.519
So you lost.

s151
00:13:21.519 --> 00:13:22.480
So he That's it.

s152
00:13:22.480 --> 00:13:25.120
The guy was stopped there.

s153
00:13:25.120 --> 00:13:28.480
What teleport does it it takes all your environment.

s154
00:13:28.480 --> 00:13:33.680
It synchronizes with a container that live inside one of the hyperscaler.

s155
00:13:33.680 --> 00:13:38.000
In our case, it's Google and locally on your laptop.

s156
00:13:38.000 --> 00:13:39.760
It feels exactly the same.

s157
00:13:39.760 --> 00:13:42.399
It just doesn't run inside your laptop.

s158
00:13:42.399 --> 00:13:45.200
It feels like it.

s159
00:13:45.279 --> 00:13:57.519
It runs somewhere else in your environment in a container in a Kubernetes cluster inside the place at Google that will continue running even if you close your laptop.

s160
00:13:57.519 --> 00:14:00.000
how simple that is.

s161
00:14:00.000 --> 00:14:03.680
And yet our favorite tool is this one.

s162
00:14:03.680 --> 00:14:12.399
I think the the last one I I I looked at it, 62% of our engineers are only using teleport when they are coding now for that problem.

s163
00:14:12.399 --> 00:14:14.320
They go home, it continues.

s164
00:14:14.320 --> 00:14:15.279
They're in transit.

s165
00:14:15.279 --> 00:14:17.120
It continues.

s166
00:14:17.120 --> 00:14:19.920
They on holiday, they don't look at it anymore.

s167
00:14:19.920 --> 00:14:22.160
It continues.

s168
00:14:22.160 --> 00:14:23.279
That's so cool, huh?

s169
00:14:23.279 --> 00:14:24.560
Yep.

s170
00:14:24.560 --> 00:14:26.480
So very smart to that.

s171
00:14:26.480 --> 00:14:29.440
So we built another tool called Kimchi studio.

s172
00:14:29.440 --> 00:14:36.399
If that teleport is something typically individual uses, studio is kind of like a teleport for teams and enterprises.

s173
00:14:36.399 --> 00:14:41.279
The goal behind the studio is uh that you have as kind of like board on your environment.

s174
00:14:41.279 --> 00:14:45.120
It again runs in the teleport sandboxes in the secure environment.

s175
00:14:45.120 --> 00:14:54.959
The thing here is like you're supposed to share your tasks and work in progress and plans which are made by the Kimchi coding harness with open source or proprietary models with your teams.

s176
00:14:54.959 --> 00:15:08.880
So for example imagine you as an engineer you are like working on a task uh you started implementing asking for a plan the plan can be easily reviewed by your colleagues like by your pe engineer by your product manager or someone else who cares what's happening and

s177
00:15:08.880 --> 00:15:19.120
this is how you kind of work with your whole team on collaborating on this agentic engineering journey instead of like saying oh I'm going to type this prompt I'm going to try to execute it to the production

s178
00:15:19.120 --> 00:15:28.639
only later on to learn that the specification and requirements were incorrect the output is not what was actually wanted by the task and things like that.

s179
00:15:28.639 --> 00:15:37.040
And this is yet another thing we built for ourself, we find it so cool, we decided to make it a product.

s180
00:15:37.120 --> 00:15:40.160
How do you work with your colleague?

s181
00:15:40.160 --> 00:15:47.519
If what you have is a C is a is a terminal, a CLI, how do you share that with the others?

s182
00:15:47.519 --> 00:15:49.440
Well, first he has to live somewhere else.

s183
00:15:49.440 --> 00:16:01.519
So teleport is the base of what we call studio and studio is a very nice way to visualize all the sessions that are running including those that are asking

s184
00:16:01.519 --> 00:16:11.360
for a review and then someone in the teams can say I take it and I answer the uh I I answer the question that is for review.

s185
00:16:11.360 --> 00:16:12.240
Then you answer it.

s186
00:16:12.240 --> 00:16:17.600
It goes back to in progress until the next one goes into review.

s187
00:16:17.600 --> 00:16:24.720
And you can have many many of those in a team in a pizza team, a team of five to 10 guys.

s188
00:16:24.720 --> 00:16:25.440
Think about it.

s189
00:16:25.440 --> 00:16:33.360
Yet yet a very simple innovation that is so essential in how our engineers are coding.

s190
00:16:33.360 --> 00:16:36.320
They can now share in a very simple way, right?

s191
00:16:36.320 --> 00:16:37.759
It's a it's a browser.

s192
00:16:37.759 --> 00:16:38.800
It's graphical.

s193
00:16:38.800 --> 00:16:48.240
They can share and they can take over the task according to the comban style of what is in review, what is in progress.

s194
00:16:48.240 --> 00:16:50.480
They can start new ones with backlog.

s195
00:16:50.480 --> 00:16:51.600
And you know what?

s196
00:16:51.600 --> 00:16:53.199
If they close the laptop,

s197
00:16:53.199 --> 00:16:55.920
it's continues.

s198
00:16:55.920 --> 00:16:57.120
This is so cool.

s199
00:16:57.120 --> 00:16:58.480
Yep.

s200
00:16:58.480 --> 00:16:59.440
This is really cool.

s201
00:16:59.440 --> 00:17:01.519
Thank you.

s202
00:17:01.519 --> 00:17:03.360
Yeah, we forgot to show.

s203
00:17:03.360 --> 00:17:06.240
Oh, this is this is the one that we took over, right?

s204
00:17:06.240 --> 00:17:07.760
That's the one,

s205
00:17:07.760 --> 00:17:08.160
guys.

s206
00:17:08.160 --> 00:17:12.799
You can see this at our booth if you want to uh to see more of it.

s207
00:17:12.799 --> 00:17:13.919
We are in U4.

s208
00:17:13.919 --> 00:17:17.120
It's exactly this way across a few streets.

s209
00:17:17.120 --> 00:17:18.319
But have a look at it.

s210
00:17:18.319 --> 00:17:19.120
It's really good.

s211
00:17:19.120 --> 00:17:20.559
It is open source.

s212
00:17:20.559 --> 00:17:24.079
Most of it at least the harness is entirely open source.

s213
00:17:24.079 --> 00:17:25.439
Uh studio and teleport.

s214
00:17:25.439 --> 00:17:26.559
You need a Google account.

s215
00:17:26.559 --> 00:17:32.000
It's a little bit more com uh more complicated, but it's a a five minute install.

s216
00:17:32.000 --> 00:17:43.919
And then you get your teleport setup as many sessions as you want inside your Google account and then you can share with share it with your colleagues with Kimchi Studio.

s217
00:17:43.919 --> 00:17:45.360
So come see us.

s218
00:17:45.360 --> 00:17:47.919
We'll be delighted to speak to you.

s219
00:17:47.919 --> 00:17:48.640
Thank you guys.

s220
00:17:48.640 --> 00:17:51.039
Thank you guys.