WEBVTT

NOTE Sentence-level transcript of https://www.youtube.com/watch?v=lCBf9slCanI

NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/lCBf9slCanI.html. A cue ends where the next begins, or 2 s after its last word.

s1
00:00:13.200 --> 00:00:15.519
Hello everyone.

s2
00:00:15.519 --> 00:00:16.960
It's great to meet you all.

s3
00:00:16.960 --> 00:00:17.920
I'm Tais.

s4
00:00:17.920 --> 00:00:20.800
I'm the founder of Taste Labs.

s5
00:00:20.800 --> 00:00:23.920
For those of you who don't know us, we came out of Stealth a few weeks ago.

s6
00:00:23.920 --> 00:00:27.760
Uh, and our whole mission is basically how do we end AI slop?

s7
00:00:27.760 --> 00:00:34.320
And we believe that to really solve this problem, we have to first decompose and understand subjective domains.

s8
00:00:34.320 --> 00:00:34.480
Right?

s9
00:00:34.480 --> 00:00:38.559
I think as probably all of you know, AI has gotten quite good at things like coding and math.

s10
00:00:38.559 --> 00:00:44.239
Uh but it's still super behind on things like design, creative writing, personality, emotional intelligence.

s11
00:00:44.239 --> 00:00:50.879
And to understand these domains, I think we have to take a little bit of a different approach than we do with um with objective ones.

s12
00:00:50.879 --> 00:00:59.359
So our idea is like how do we become this data and infrastructure layer to really uh understand these problems and to become the solution for them across the stack.

s13
00:00:59.359 --> 00:01:04.320
So from the foundation model layer all the way to how do we build solutions for agents as well.

s14
00:01:04.320 --> 00:01:07.200
So we work primarily in two ways.

s15
00:01:07.200 --> 00:01:18.560
We work with the top frontier labs on how do we evaluate benchmark their models understand where they're breaking uh understand how we can fix them and how do we determine also which problem is better fixed through each method.

s16
00:01:18.560 --> 00:01:23.920
So what things should be turned into RL environments, which things should be turned into post- training data problems.

s17
00:01:23.920 --> 00:01:35.040
Um, but then we also go and work with a lot of agent and application layer companies on what are things that we actually don't believe should be solved at the foundation model layer and that might be better solved through methods like context

s18
00:01:35.040 --> 00:01:36.720
uh or understanding user intent.

s19
00:01:36.720 --> 00:01:36.880
Right?

s20
00:01:36.880 --> 00:01:43.520
We're basically betting on a world where suddenly you're going to have billions of people creating uh that are not necessarily experts.

s21
00:01:43.520 --> 00:01:49.680
So this understanding of user intent and contest and context is equally as important as how do we get these models to improve.

s22
00:01:49.680 --> 00:01:51.680
For today I'm going to focus on the model training part.

s23
00:01:51.680 --> 00:01:58.960
For those that are here tomorrow I'll also be giving a chat on the design track where I'll cover more on what we're doing on the agent side of the house.

s24
00:01:58.960 --> 00:02:01.040
But there we go.

s25
00:02:01.040 --> 00:02:01.840
Okay.

s26
00:02:01.840 --> 00:02:04.000
So most of the world is subjective as I was mentioning.

s27
00:02:04.000 --> 00:02:05.600
Uh a lot of the world is subjective right?

s28
00:02:05.600 --> 00:02:22.400
If we talk about these domains of writing even workflows within companies right of sales marketing uh a lot of the times there's this multitude of answers there's not one clear right answer and it's very hard to define what great even means I think at the end of the day like those are why these domains are so difficult

s29
00:02:22.400 --> 00:02:40.080
um and we oftentimes I would say forget to mention like why uh we treat for example the fact that code is verifiable and measurable as something that is a property about models and models are great at at coding

s30
00:02:40.080 --> 00:02:44.480
um because we've made them great at coding but realistically it's actually a fact about code.

s31
00:02:44.480 --> 00:02:54.400
Code is something that decomposes, it verifies, it executes and so it makes it a lot easier for us to be able to train on these domains for something like design or writing like how do you decompose it?

s32
00:02:54.400 --> 00:02:55.440
How do you verify it?

s33
00:02:55.440 --> 00:02:57.519
How do you judge if it's actually good?

s34
00:02:57.519 --> 00:02:59.519
So that's why they become so difficult.

s35
00:02:59.519 --> 00:03:05.599
Um, so there's two characteristics that I want to touch on today on why fundamentally subjective domains are harder.

s36
00:03:05.599 --> 00:03:08.159
One is that capability follows measurability.

s37
00:03:08.159 --> 00:03:14.159
So if we can solve the measurability problem or at least part of it, then we can solve a big portion of these domains.

s38
00:03:14.159 --> 00:03:24.080
Uh the second which I'll touch on later is basically this like collapse to the mean and why the mean is not necessarily optimal in subjective domains.

s39
00:03:25.840 --> 00:03:27.840
Okay.

s40
00:03:32.000 --> 00:03:40.959
So to really start solving this problem, we have to turn something that feels fuzzy like if I ask you what is great design into something that is more verifiable.

s41
00:03:40.959 --> 00:03:42.400
So there's a few questions here, right?

s42
00:03:42.400 --> 00:03:52.799
Because if I ask you this of what is great design um you could ask yourself, okay, um do you mean great for which type of person, for which type of taste, for which situation?

s43
00:03:52.799 --> 00:03:54.480
The same slide could be amazing.

s44
00:03:54.480 --> 00:03:59.360
for example, if you are a startup and completely inappropriate if you are a finance firm.

s45
00:03:59.360 --> 00:04:01.280
So it's contextual first of all.

s46
00:04:01.280 --> 00:04:06.159
Second of all, it has this property that it changes over time which is different from other domains.

s47
00:04:06.159 --> 00:04:10.480
What is considered good today is different than five years ago and different than five years from now.

s48
00:04:10.480 --> 00:04:12.640
In code that's not necessarily true or in math, right?

s49
00:04:12.640 --> 00:04:15.200
That's something that is way more consistent over time.

s50
00:04:15.200 --> 00:04:27.040
So our ability to again decompose it and understand how is this good for a specific audience, how is this good today, how is this good in context um is some of the things that we've been thinking about in terms of how to how to break this down.

s51
00:04:27.040 --> 00:04:30.960
But I want to give you a very specific example because of course this can mean many things.

s52
00:04:30.960 --> 00:04:32.320
So let's talk about brand.

s53
00:04:32.320 --> 00:04:37.520
Um if you're at a company and you've used coding agents, you've probably shipped an internal dashboard.

s54
00:04:37.520 --> 00:04:40.240
You've probably shipped an internal like landing page.

s55
00:04:40.240 --> 00:04:46.240
And oftentimes you might wonder, okay, how do I determine if this is slop, if this is actually good?

s56
00:04:46.240 --> 00:04:53.600
And you have kind of this secret weapon at your disposal, which is really all the work that probably designers at your companies, for example, put into defining your brand.

s57
00:04:53.600 --> 00:04:56.160
A brand to define takes a lot of effort, takes a lot of care.

s58
00:04:56.160 --> 00:05:03.440
You're defining all these components about it, when it's good, why you're choosing certain combinations of colors, of typography, of spacing, of texture.

s59
00:05:03.440 --> 00:05:07.520
Um, but if I just ask you to be like, okay, create something great, that's very hard.

s60
00:05:07.520 --> 00:05:14.560
But suddenly if I'm like okay make something that is on brand that is a much easier problem to define and a brand is something that can become decomposable.

s61
00:05:14.560 --> 00:05:19.520
Uh so for example if we I'm using the reduct brand as an example here because I I I like their website.

s62
00:05:19.520 --> 00:05:26.880
Um let's say that we decompose this brand into the colors, the typography, the motion, the animation, the textures.

s63
00:05:26.880 --> 00:05:30.320
Suddenly you have these very codified things that you can verify against.

s64
00:05:30.320 --> 00:05:36.800
Verifying in general if something's on brand and you can try this uh by prompting an LLM as a judge to do it is quite hard.

s65
00:05:36.800 --> 00:05:46.560
But once you start picking apart the exact elements that represent what great is, then it suddenly becomes the shape of something that is codifiable and verifiable.

s66
00:05:46.560 --> 00:05:53.120
So if you want to turn this into a shape of an RL environment, for example, right, how would you train a model for a capability like brand adurance?

s67
00:05:53.120 --> 00:05:59.520
Uh LLM LLM as a judge might not necessarily always be the best method.

s68
00:05:59.520 --> 00:06:01.039
We know that there's a lot of reward hacking.

s69
00:06:01.039 --> 00:06:04.639
We know that there's uh interesting hallucination patterns there too.

s70
00:06:04.639 --> 00:06:14.000
And so we oftentimes try to create methods of basically how do we turn a task that feels fuzzy into one where there's a clear ground truth so that it can become the shape of an environment.

s71
00:06:14.000 --> 00:06:23.759
So in this case the task design itself is really kind of the hardest part of the problem of how do you turn something that appears very fuzzy into something that actually can be arled.

s72
00:06:23.759 --> 00:06:27.199
Um and so in this case that decomposition that I mentioned becomes the ground truth.

s73
00:06:27.199 --> 00:06:34.800
So let's say that you start by tasking an agent to create a new page that is going to adhere to the reductal brand but be completely net new and different.

s74
00:06:34.800 --> 00:06:40.880
Um you would want that output to not only be graded versus the original but to be graded on this ground truth, right?

s75
00:06:40.880 --> 00:06:47.199
Because it could come up with completely new ways of using these components that are still valid but are different from the original.

s76
00:06:47.199 --> 00:06:50.319
So you don't necessarily want to just see if it's replicating the original.

s77
00:06:50.319 --> 00:06:57.919
So this is one example of like how to turn this into a problem of environment shaped um so that we can make it more verifiable.

s78
00:06:57.919 --> 00:07:06.240
But in a way all of these things are I would say like spectrums right you have uh in a problem like design you have these elements of things like vision

s79
00:07:06.240 --> 00:07:24.960
uh alignment typography that are closer to objective once you start moving up that scale onto things like style fit creativity how do you judge and measure something like creativity right that's much harder and so you kind of need to think of this as like a routing problem of how do you understand this like vast fuzzy problem break

s80
00:07:24.960 --> 00:07:30.479
it down into smaller components and what is the best solution for each of these components.

s81
00:07:30.479 --> 00:07:32.240
So why does something feel like slop?

s82
00:07:32.240 --> 00:07:35.120
For example, when we're talking about something like creativity, right?

s83
00:07:35.120 --> 00:07:45.039
I think this is the second reason why um subductive domains are so much harder to solve because if we're talking about the properties of models, they're basically predicting what's the most likely outcome

s84
00:07:45.039 --> 00:07:46.000
to show up next.

s85
00:07:46.000 --> 00:07:48.880
And they assume that that outcome is the ideal outcome.

s86
00:07:48.880 --> 00:07:51.440
And for something like math encoding, that is true, right?

s87
00:07:51.440 --> 00:07:55.440
You want the answer that your model gives you to be the average answer.

s88
00:07:55.440 --> 00:08:00.400
If you're asking what 2 plus 2 is, uh, which also happens to be the right answer and the optimal answer.

s89
00:08:00.400 --> 00:08:04.400
But for something like writing or design, you don't necessarily want the average answer, right?

s90
00:08:04.400 --> 00:08:09.599
The average meaning the most likely does not necessarily coincide with like the optimal.

s91
00:08:09.599 --> 00:08:16.080
Uh, a lot of like what I describe it is a lot of greatness and creativity happens actually at the ends of the distribution.

s92
00:08:16.080 --> 00:08:17.360
It's not the most likely outcome.

s93
00:08:17.360 --> 00:08:24.479
It's when you actually actively break from rules and actively break from patterns that you can create things that are subjective and and great.

s94
00:08:24.479 --> 00:08:34.080
Um, and so the reason why this feels like slop and that we have this feeling that we're surrounded by by slop is exactly because of this collapse to the mean and this repetition.

s95
00:08:34.080 --> 00:08:37.200
And so we have to find ways of okay, how do we break these patterns?

s96
00:08:37.200 --> 00:08:40.560
How do we break from the mean uh but in a way that's also intentional.

s97
00:08:40.560 --> 00:08:44.800
So then you kind of shift the problem onto things that are not so easily maybe verifiable.

s98
00:08:44.800 --> 00:08:51.760
Uh but that are more questions of human preference and judgment and that might be better solved by data for example than by environments.

s99
00:08:51.760 --> 00:08:56.160
And so again this kind of like mode collapse is is really the thing that we're trying to solve.

s100
00:08:56.160 --> 00:08:58.399
And how do we force that distribution back?

s101
00:08:58.399 --> 00:09:12.480
Uh, oftentimes, by the way, we we work, for example, with a community of designers, um, over like a thousand experts that are experts in different types of medium, different styles, and we purposely want to force that distribution when we're breaking down the problem.

s102
00:09:12.480 --> 00:09:12.880
Exactly.

s103
00:09:12.880 --> 00:09:17.120
So, we don't end up in this mode collapse, but there's of course a lot of other pieces of that puzzle.

s104
00:09:17.120 --> 00:09:26.560
Uh, but I think this is an interesting framework is like the closer you are to something that it becomes verifiable, especially programmatically, the better for something like RL, right?

s105
00:09:26.560 --> 00:09:35.600
And I think the challenge is how do we turn things that feel fuzzy into things that become more verifiable by establishing this ground truth and designing tasks in a way that allow for that.

s106
00:09:35.600 --> 00:09:43.360
But then the more that it does shift to things that are contextual or that depend on that time element that we talked about or this like distinction in preference,

s107
00:09:43.360 --> 00:09:47.680
the more this shifts towards something that requires human judgment.

s108
00:09:48.080 --> 00:09:54.240
So uh I I I like this analogy of basically kind of pulling things toward verification.

s109
00:09:54.240 --> 00:10:02.880
So brand adherance on its own would be something that's very hard and that is probably better judged by a human than for example by an LLM as a judge or something deterministic.

s110
00:10:02.880 --> 00:10:12.880
But by codifying it by understanding which pieces matter and how do I turn that into something that is observable and measurable we can kind of pull it into this realm of verification.

s111
00:10:12.880 --> 00:10:23.760
uh so this routing logic is I would say if you have one takeaway uh take this away it's like how do we break down this problem into something that you understand what's actually the best method to solve it

s112
00:10:23.760 --> 00:10:40.720
and so when we are talking about these things that are more subjective uh we do require human judgment I think this is something that we um we believe human judgment is still at a much higher level than any LLM as a judge and this like human taste is really how do we encapsulate this in a way that

s113
00:10:40.720 --> 00:10:46.079
uh can be turned into high quality data so that we can train these models better.

s114
00:10:46.079 --> 00:11:01.040
And I think one of the tricky things here is often times in the past you had kind of this uh collection of preference data that would collapse again to the mean because you would collect it from a bunch of different people without necessarily understanding who they are, what they like, why they like it, when they like it.

s115
00:11:01.040 --> 00:11:06.800
And if you don't break up that problem accordingly, you then end up again with preferences that kind of don't agree with each other.

s116
00:11:06.800 --> 00:11:08.480
Because that naturally happens in the world, right?

s117
00:11:08.480 --> 00:11:11.360
I bet that some of you might like one style better than another.

s118
00:11:11.360 --> 00:11:17.440
And that doesn't mean either of those things are wrong or it doesn't mean that the best answer is the average of what two people might like.

s119
00:11:17.440 --> 00:11:23.519
It means that we need to fundamentally understand that the world is multi-preference and how do we do that matching accordingly.

s120
00:11:23.519 --> 00:11:32.320
So this almost like understanding of how do we create like a preference vector let's say for someone and attach that to even something like preference data can help us to

s121
00:11:32.320 --> 00:11:40.079
train in a way that allows for that more pluralism of preferences intentionally instead of that data being turned into something that's noisy.

s122
00:11:40.720 --> 00:11:50.079
when we talk about data quality as well for these domains, I think it becomes very uh I don't know if any of you have bought data for example for these domains or or have tried to curate data yourselves.

s123
00:11:50.079 --> 00:11:55.120
Um but there's it's very hard to define okay like now I'm realming to the side of data.

s124
00:11:55.120 --> 00:11:56.000
What is actually good?

s125
00:11:56.000 --> 00:11:58.480
What is actually going to be helpful to my model?

s126
00:11:58.480 --> 00:12:07.680
Um and there's obviously a few things that are harder to define but a few that I think become more controllable and how do we actually understand patterns of quality so that we can measure them.

s127
00:12:07.680 --> 00:12:09.760
So one I would say is that problem decomposition.

s128
00:12:09.760 --> 00:12:12.560
How do we force like true distributions of what you see in the world?

s129
00:12:12.560 --> 00:12:16.800
How do you force true like expert selection across these u different buckets?

s130
00:12:16.800 --> 00:12:18.800
And these are things that you can totally control, right?

s131
00:12:18.800 --> 00:12:20.320
You can see who those experts are.

s132
00:12:20.320 --> 00:12:29.519
You can do a selection pattern that is very strict um and you can decompose that problem and this is something that is completely in your control and that totally helps with the

s133
00:12:29.519 --> 00:12:33.839
um results let's say of the experiment being being good.

s134
00:12:33.839 --> 00:12:40.639
Um the second is I would say that like flow of like how do you determine then the right problem routeed to the right solution.

s135
00:12:40.639 --> 00:12:52.160
Um we also I would say do a lot of what we call essentially like QA on this data and I think there's two ways to do that is understanding what are properties about that data point that

s136
00:12:52.160 --> 00:12:54.880
correlate with being it being rich and high signal.

s137
00:12:54.880 --> 00:13:06.639
So for example, specificity when you're trying to ask an expert to define is this good, is this bad or put reasoning behind it or create a whole observation system around

s138
00:13:06.639 --> 00:13:08.959
uh how they would judge an asset.

s139
00:13:08.959 --> 00:13:17.279
The specificity of their language and of how precise they're being able to be with how they're doing that description is what will determine that data quality.

s140
00:13:17.279 --> 00:13:22.800
Or for example, if you can tie their commentary with actually which piece in the code does this relate to.

s141
00:13:22.800 --> 00:13:26.000
Like let's say you have an expert that's judging a landing page.

s142
00:13:26.000 --> 00:13:29.680
Uh they might be able to just like write a paragraph describing this.

s143
00:13:29.680 --> 00:13:35.120
But we know that models have a tricky time kind of actually connecting the piece of the code to the visual.

s144
00:13:35.120 --> 00:13:44.480
And so if you can find for example a method to tie that exact code component to the commentary of the expert, suddenly you have data that is way less noisy and way more clear.

s145
00:13:44.480 --> 00:13:59.600
So these are some things that we can do both in terms of the flow of how you connect the data uh of how you collect the data but also in these like QA checks of how do you determine characteristics about it that will uh correlate highly let's say with with valuable data and same with human QA.

s146
00:13:59.600 --> 00:14:02.800
I think by the way human QA is tricky here because there's two sides to it.

s147
00:14:02.800 --> 00:14:11.199
There is a side of for example let's say you're collecting uh a bunch of preference data about slide design.

s148
00:14:11.199 --> 00:14:21.440
Um you could have human QA be like okay is this again high quality data is the following the specs which most people would agree on but suddenly if you ask for expert consensus and you try to have

s149
00:14:21.440 --> 00:14:31.920
another expert see if they agree with the initial designers like votes you might start seeing some disagreement there and you I think the key is understanding is that disagreement something that's actually

s150
00:14:31.920 --> 00:14:44.160
a flaw in the data meaning are they for example disagreeing on something that they should be agreeing on such as alignment like alignment is something that's pretty objective So it would be kind of odd to see experts disagreeing on that front.

s151
00:14:44.160 --> 00:14:52.000
But suddenly if they're disagreeing on things like sty style or um aesthetics that is not necessarily bad data that's actually good data.

s152
00:14:52.000 --> 00:14:55.040
It shows you that there is a distinction for what people like.

s153
00:14:55.040 --> 00:15:06.399
And so uh this almost like analysis of how do you run human QA in a way that both screens for kind of the fundamentals but then when you use consensus I think in an intentional way is another big big piece of this.

s154
00:15:06.399 --> 00:15:10.720
Um and then obviously kind of seeing actually how this data is is interacting with models.

s155
00:15:10.720 --> 00:15:12.959
Uh we run a lot of research on our side.

s156
00:15:12.959 --> 00:15:27.600
Obviously when you're interacting with labs as we do um it there's a lot more involved and we oftentimes don't get that feedback loop of exactly what affected this cause which is why we're so adamant on focusing on these things we can control in terms of data quality because these are things that

s157
00:15:27.600 --> 00:15:30.399
uh we can completely measure on on our side.

s158
00:15:30.399 --> 00:15:44.320
Um so I think one other uh piece of advice or message of the day is I think especially when it comes to subjective domains I would advocate for a quality over quantity approach.

s159
00:15:44.320 --> 00:15:47.120
I think creating high quality data is expensive.

s160
00:15:47.120 --> 00:15:47.760
It's difficult.

s161
00:15:47.760 --> 00:15:51.199
It takes a lot of understanding and depth around a specific domain.

s162
00:15:51.199 --> 00:16:06.079
and having that be incredibly high quality done by people that also are incredibly high taste or whatever you want to call it in that domain uh yield far better results than getting a bunch of noisy data or a bunch of messy data uh that was not necessarily intentional or didn't have all those things we talked about of

s163
00:16:06.079 --> 00:16:07.600
like the problem breakdown.

s164
00:16:07.600 --> 00:16:10.800
So that is my my message of of the day.

s165
00:16:10.800 --> 00:16:12.959
Thank you.

s166
00:16:27.751 --> 00:16:29.751
[music]