WEBVTT

NOTE Sentence-level transcript of https://www.youtube.com/watch?v=sDMGWK4wZ_w

NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/sDMGWK4wZ_w.html. A cue ends where the next begins, or 2 s after its last word.

s1
00:00:01.309 --> 00:00:03.309
[music]

s2
00:00:12.520 --> 00:00:13.360
Test.

s3
00:00:13.360 --> 00:00:15.080
Okay, amazing.

s4
00:00:15.080 --> 00:00:16.040
It's great to meet everyone.

s5
00:00:16.040 --> 00:00:16.920
I'm Taís.

s6
00:00:16.920 --> 00:00:19.040
I'm the founder of Taste Labs.

s7
00:00:19.040 --> 00:00:25.680
Uh for those of you who don't know us, we came out of stealth a few weeks ago and our whole mission is basically how do we end AI slop?

s8
00:00:25.680 --> 00:00:27.800
I that's my personal enemy.

s9
00:00:27.800 --> 00:00:34.400
Um and so we really believe that to solve this problem of slop, we have to like decode subjective domains.

s10
00:00:34.400 --> 00:00:39.920
Uh there's been so much effort being put into getting models and agents amazing at things like coding and math.

s11
00:00:39.920 --> 00:00:44.560
Uh and it's time that we put all that same effort into making them great at things like design uh and writing.

s12
00:00:44.560 --> 00:00:49.040
And so design is this first pillar that we're starting with and it's been it's been incredibly exciting.

s13
00:00:49.040 --> 00:00:51.800
Um We work primarily in two ways.

s14
00:00:51.800 --> 00:01:04.400
So we work a lot with the frontier labs on how do we evaluate their models, understand where they're breaking, understand what could be better about them, and then construct the right either post-training data or RL environments to basically fix that problem.

s15
00:01:04.400 --> 00:01:13.120
And part of this is like how do you take something as fuzzy and large as design and break it down to a level that you can identify what is best solved through each method.

s16
00:01:13.120 --> 00:01:19.240
What are elements of design that are almost like once you kind of boil down the problem becomes so specific that they almost become deterministic.

s17
00:01:19.240 --> 00:01:29.760
So for example, uh if you're trying to train a model to be good at selecting color palettes or have contrast or alignment, those are things that if you define the problem and the context in a specific enough way,

s18
00:01:29.760 --> 00:01:33.840
uh you you can get to an answer that's like pretty objective or that at least most experts would agree to.

s19
00:01:33.840 --> 00:01:38.520
But maybe other things like uh aesthetics, you naturally will see this expert disagreement.

s20
00:01:38.520 --> 00:01:41.280
And so then you want to lean on to things that are closer to to data.

s21
00:01:41.280 --> 00:01:44.240
So anyway, we spend a lot of time thinking about all those problems.

s22
00:01:44.240 --> 00:01:47.960
Uh but on the other side is also without even touching the model layer, right?

s23
00:01:47.960 --> 00:01:52.080
How do we actually help agents and app layer companies produce better things?

s24
00:01:52.080 --> 00:01:53.720
And there's a lot that goes into that, right?

s25
00:01:53.720 --> 00:02:01.840
You have these different sets of problems at the application layer because you're using an off-the-shelf model that tends to collapse in terms uh uh of style tends to collapse to the mean.

s26
00:02:01.840 --> 00:02:04.400
So, how do we force that creativity back to the system?

s27
00:02:04.400 --> 00:02:07.960
How do we avoid these patterns of slop, which we'll talk about a lot today?

s28
00:02:07.960 --> 00:02:15.160
Uh how do you understand like user preferences or brand preferences preference so that you can uh maintain endurance to that style?

s29
00:02:15.160 --> 00:02:25.280
Uh so, there's lots of things that are actually need to be solved as context or judgment or verification at the app layer, which is why we kind of work across both.

s30
00:02:25.720 --> 00:02:30.240
Maybe I'll start with more of a a philosophical question of like how how do you define something that is great?

s31
00:02:30.240 --> 00:02:32.320
Like how do you define greatness?

s32
00:02:32.320 --> 00:02:34.920
And for something like math, it's easier, right?

s33
00:02:34.920 --> 00:02:39.840
Because there's kind of one objective answer, and uh great is the same as correct.

s34
00:02:39.840 --> 00:02:43.680
But then for something like writing or design, it's much harder, right?

s35
00:02:43.680 --> 00:02:49.160
Like how do you define what's like a great tweet or what's a great art piece or what's a great website?

s36
00:02:49.160 --> 00:02:56.959
Um I don't know what's the last time that you interacted with a poem or walked into a coffee shop and for some reason it kind of like hit different and it felt

s37
00:02:56.959 --> 00:02:58.000
very special.

s38
00:02:58.000 --> 00:03:01.320
Uh but probably it's a combination of things that it it felt very unique.

s39
00:03:01.320 --> 00:03:02.480
It felt almost a little different.

s40
00:03:02.480 --> 00:03:03.920
It kind of called your attention.

s41
00:03:03.920 --> 00:03:11.120
Uh it felt like there it was made with a lot of care and attention to to detail and craft, and it almost had the sense of of like authenticity.

s42
00:03:11.120 --> 00:03:13.080
Um and I think that's a lot of what AI is missing today.

s43
00:03:13.080 --> 00:03:16.800
It's like how do we take uh things that are not necessarily average, right?

s44
00:03:16.800 --> 00:03:19.920
How do we produce things that are purposely like out of distribution?

s45
00:03:19.920 --> 00:03:22.160
Um and slop is kind of the opposite of that, right?

s46
00:03:22.160 --> 00:03:28.840
I think it is hard to define what is great sometimes, but I think it's pretty pretty easy to define what is slop in the sense that most people would agree.

s47
00:03:28.840 --> 00:03:36.840
I think the sense of like repetition of kind of soullessness is something that all of us feel right now when using AI, and I think it's quite magical, by the way, that

s48
00:03:36.840 --> 00:03:46.280
AI has gotten to a point that any human on the planet that is not even a designer, that is not an engineer, can click a button and suddenly make an entire PowerPoint or make a website or make a web app.

s49
00:03:46.280 --> 00:03:47.519
That's pretty cool.

s50
00:03:47.519 --> 00:03:49.160
But it comes with consequences, right?

s51
00:03:49.160 --> 00:03:54.760
Uh it comes with consequences of suddenly now the cost of generation is basically going to zero.

s52
00:03:54.760 --> 00:03:58.520
Uh and But the average person hasn't necessarily honed their taste.

s53
00:03:58.520 --> 00:04:04.720
Like I does think about the amount of effort and work that a designer puts in throughout their life to like build up their taste, right?

s54
00:04:04.720 --> 00:04:15.959
Like there's all this process of like getting exposed to many things and learning to like spot patterns and learning to develop a point of view and like kind of do things in a in a courageous way that maybe are a little bit against the norm.

s55
00:04:15.959 --> 00:04:19.560
Learning what not to do and how to like have restraint and that's very hard.

s56
00:04:19.560 --> 00:04:26.240
Like the average person doesn't necessarily have the the time nor the skills to go and develop taste in everything, let's say in design.

s57
00:04:26.240 --> 00:04:32.640
And so um I think it would be a bad case scenario for us to just like be like, "Okay, the way to fix slop is for everyone to have taste."

s58
00:04:32.640 --> 00:04:35.160
cuz I don't think that's necessarily realistic.

s59
00:04:35.160 --> 00:04:43.680
Um I think how do we how can we understand this better so that we can make even for the average person the ability to create something great and to understand maybe their own taste

s60
00:04:43.680 --> 00:04:45.800
um easy more more more easy.

s61
00:04:45.800 --> 00:04:49.040
So that's that's a lot of what we're we're focusing on.

s62
00:04:49.040 --> 00:04:52.720
Um So yeah, I mean this phenomenon of slop, by the way, is not new.

s63
00:04:52.720 --> 00:04:58.240
Uh if you were in the internet uh as social media emerged, you probably saw a lot of slop before that.

s64
00:04:58.240 --> 00:05:01.160
But I do think that AI has been this kind of like accelerating force, right?

s65
00:05:01.160 --> 00:05:05.640
Of like being able to create things very easily uh with a click of a button and that like thoughtlessness around it.

s66
00:05:05.640 --> 00:05:10.320
And there's kind of these three characteristics that I I would say repeat in slop.

s67
00:05:10.320 --> 00:05:11.480
Uh so A, repetition.

s68
00:05:11.480 --> 00:05:15.120
So you start seeing the same thing many many many times.

s69
00:05:15.120 --> 00:05:18.040
Um the second is lack of fit, which I actually think is is very related.

s70
00:05:18.040 --> 00:05:23.200
So fit is kind of this ability for something to feel correct for a specific context, right?

s71
00:05:23.200 --> 00:05:25.720
For a specific moment in time, for a specific person.

s72
00:05:25.720 --> 00:05:37.840
Uh but suddenly if you have repetition and let's say one person asks for a website for their pet shop and the other one asks for a website for their finance firm and somehow those designs converge and look the same.

s73
00:05:37.840 --> 00:05:38.760
That's quite odd, right?

s74
00:05:38.760 --> 00:05:44.160
Like if that was in if you were actually crafting that with care, that wouldn't you wouldn't converge necessarily on those things.

s75
00:05:44.160 --> 00:05:49.720
And so this lack of fit and lack of understanding of context is actually huge problem that like leads to slop.

s76
00:05:49.720 --> 00:05:56.840
Um and the third is maybe low intent, which is probably a mix of Yeah, you're going to have a bunch of people prompting really quickly and maybe just wanting to one shot something.

s77
00:05:56.840 --> 00:06:01.000
But I think there's actually this like intent interpretation piece that's missing in the systems that we're building.

s78
00:06:01.000 --> 00:06:02.920
Like how can you help your user, right?

s79
00:06:02.920 --> 00:06:11.680
Like how can you help them better understand the intent that they have um so that you can add more color and add more context on onto what you're trying to create.

s80
00:06:12.160 --> 00:06:12.600
Okay.

s81
00:06:12.600 --> 00:06:19.400
And I I'm a big believer by the way that you in order to fix something first have to measure it and you first have to understand it.

s82
00:06:19.400 --> 00:06:27.760
I think that's exactly why we're so focused on like how do we uh turn these domains into something a bit more verifiable so that we can attach a measure to it.

s83
00:06:27.760 --> 00:06:34.280
So uh you'll you'll go on a little bit of a research journey with me here now, but we basically wanted to figure out can we measure slop?

s84
00:06:34.280 --> 00:06:40.760
Like can we actually measure this quantitatively and spot this and what does that like look like?

s85
00:06:41.160 --> 00:06:49.120
So we analyzed over 2 million websites from the past like 10 years kind of like way back machine style to try to understand all the trends across like design,

s86
00:06:49.120 --> 00:06:53.960
how is the internet changing, uh how are how is like design changing over time?

s87
00:06:53.960 --> 00:06:55.520
And two things were interesting.

s88
00:06:55.520 --> 00:07:06.640
And we also met by the way then kind of synthetically generated a set of uh design websites so we could kind of like compare like how does human-made sites compare to AI-generated

s89
00:07:06.640 --> 00:07:07.280
ones?

s90
00:07:07.280 --> 00:07:08.440
And there were a few things that were interesting.

s91
00:07:08.440 --> 00:07:15.600
So one was that you already kind of saw a a bit of like a collapse uh on the internet before even AI.

s92
00:07:15.600 --> 00:07:27.760
So you saw kind of the internet becoming more homogeneous, using more similar color palettes, using more similar layouts, uh which is probably a function of more uh I would say this like kind of trends spreading more and more and more quickly, let's say.

s93
00:07:27.760 --> 00:07:34.040
Uh but with AI I think you saw this repetition happening a lot more and being almost more like um identified kind of regardless of context.

s94
00:07:34.040 --> 00:07:37.919
So even in completely different buckets you saw patterns that were very similar.

s95
00:07:37.919 --> 00:07:42.320
So we we built this I I call this probes, but basically we uh we did two things.

s96
00:07:42.320 --> 00:07:47.560
So we did this like pattern mining on all this data to understand like what are features that we can extract from all these sites.

s97
00:07:47.560 --> 00:07:50.760
What are all these characteristics that we can make more objective, right?

s98
00:07:50.760 --> 00:07:53.200
Colors, typography, layout, audience.

s99
00:07:53.200 --> 00:07:57.200
Like, how can we like distill this down into things that become almost like uh structured?

s100
00:07:57.200 --> 00:07:59.720
And then how do we uh train up these like probes?

s101
00:07:59.720 --> 00:08:01.960
So, think of these as like baby classifiers.

s102
00:08:01.960 --> 00:08:05.880
Like, how do we uh train the ability to spot this one characteristic?

s103
00:08:05.880 --> 00:08:14.800
And for all these slop sites, we identify we started identifying like what are the probes that basically mean the site is very likely to be AI slop.

s104
00:08:14.800 --> 00:08:23.240
Um and especially when you start combining them use and you see the frequency of multiple of these happening at once, it became very likely that you could actually like measure

s105
00:08:23.240 --> 00:08:24.320
uh and predict slop.

s106
00:08:24.320 --> 00:08:29.520
And we saw a a super high basically ability to do that prediction, which was really cool to see.

s107
00:08:29.520 --> 00:08:37.840
This performed better, by the way, than like most LLM as a judge methods of like asking an LLM to like judge if that uh is like great human quality versus like AI-generated

s108
00:08:37.840 --> 00:08:38.560
slop.

s109
00:08:38.560 --> 00:08:39.520
Uh so, that was pretty cool to see.

s110
00:08:39.520 --> 00:08:49.080
And I think kind of shows this pattern that we see in AI really being uh an actually quantitative thing that we can see in slop, uh which I find really cool.

s111
00:08:49.080 --> 00:08:50.760
But, obviously, we don't want to stop there, right?

s112
00:08:50.760 --> 00:08:52.280
We don't want to just measure slop.

s113
00:08:52.280 --> 00:08:53.200
We want to also solve it.

s114
00:08:53.200 --> 00:09:02.040
And so, um there's a few I I think I mentioned this before, but like the as the cost of production basically goes to zero, I think the thing that becomes

s115
00:09:02.040 --> 00:09:05.920
expensive and matters more than ever is judgment.

s116
00:09:05.920 --> 00:09:07.839
Um I don't even want to use the word taste here.

s117
00:09:07.839 --> 00:09:08.360
Is judgment.

s118
00:09:08.360 --> 00:09:10.720
I think it's this ability to discern what's right.

s119
00:09:10.720 --> 00:09:15.240
It's this ability to break down a problem so that you can actually understand it and create solutions for it.

s120
00:09:15.240 --> 00:09:19.760
And so, yes, there's the side of judgment that is human judgment that I actually think is more valuable than ever.

s121
00:09:19.760 --> 00:09:26.280
But, there's also the side of like how do we build the right tools and systems to like fix pieces of this problem, right?

s122
00:09:27.480 --> 00:09:30.560
So, yeah, how do we how do we fight slop, my my enemy?

s123
00:09:30.560 --> 00:09:37.520
Um And, by the way, I think there's there's a lot of conversation going around how do you fight slop at the model layer?

s124
00:09:37.520 --> 00:09:39.160
Like, how do we make models better?

s125
00:09:39.160 --> 00:09:40.720
How do we make models have a higher bar?

s126
00:09:40.720 --> 00:09:44.680
Which don't get me wrong, it has to be solved and we're working very hard to solve that, too.

s127
00:09:44.680 --> 00:09:49.200
But I actually think this problem of inference time is equally, if not even more important.

s128
00:09:49.200 --> 00:09:52.520
Because that's actually when you interact with the end user.

s129
00:09:52.520 --> 00:09:57.120
And this kind of back and forth of how do you understand this context and intent happens at the moment of inference time.

s130
00:09:57.120 --> 00:10:02.480
So, I don't think that we can ignore and just make models better and not solve this, otherwise slop will keep existing.

s131
00:10:02.480 --> 00:10:10.040
Um so, maybe breaking down a few of those pieces and kind of um a few of the ways that we've thought about solving this or a few solutions that we built to solve this.

s132
00:10:10.040 --> 00:10:17.880
But I think, for example, for something like repetition, one of the things that we're working on is I I've nicknamed it, I don't know if that's going to be the official name, but like the creativity API.

s133
00:10:17.880 --> 00:10:22.280
How can we create a system that almost becomes an inspiration machine for your agent?

s134
00:10:22.280 --> 00:10:30.920
So that it can produce something that's actually out of distribution instead of something that is in that same average and kind of mean that we're seeing happen with like the slop sites.

s135
00:10:30.920 --> 00:10:38.480
Um so, this is one of the ways that practically, if we can intentionally produce something that's out of distribution, you can improve this like overall uh quality.

s136
00:10:38.480 --> 00:10:43.320
And by the way, I I don't think that this can be something just like randomness.

s137
00:10:43.320 --> 00:10:47.600
So, it's not just about like turning up the temperature of the model and and kind of fingers crossed hoping for the best.

s138
00:10:47.600 --> 00:10:53.839
I think it's much more like how do we understand um even like what are rules or expectations in specific domains?

s139
00:10:53.839 --> 00:10:57.880
Like let's say that you asked for a slide deck for for the pitch of your startup.

s140
00:10:57.880 --> 00:11:00.839
Like what is a what does a good pitch deck look like?

s141
00:11:00.839 --> 00:11:05.400
And then how do you almost like intentionally break rules uh to create things that are more creative, right?

s142
00:11:05.400 --> 00:11:11.280
Because usually creativity isn't like randomness, isn't doing something that completely feels off for that situation.

s143
00:11:11.280 --> 00:11:20.920
It's like you intentionally maybe diverge on a couple of things while maintaining kind of um adherence to to expectations of that category, let's say for for others.

s144
00:11:20.920 --> 00:11:22.520
So, that's one of the things we're working on.

s145
00:11:22.520 --> 00:11:31.520
The second one of this problem of fit, I think um it's interesting, but brands, as probably a lot of you who are designers know, takes so much effort to create great brands.

s146
00:11:31.520 --> 00:11:39.080
Like great brands are the work of dozens of designers putting in a lot of like craft and thought and care.

s147
00:11:39.080 --> 00:11:44.960
And so we've almost like already pre-done the work of defining what is great for that specific company and then we're not using it well.

s148
00:11:44.960 --> 00:11:51.840
So this like brand endurance actually think is a huge problem and one of the things that can very more easily let's say like raise that bar of quality.

s149
00:11:51.840 --> 00:11:54.560
So I'll I'll touch on an example on this one specifically.

s150
00:11:54.560 --> 00:11:58.560
And then same with like intent and judgment I think the baby classifiers was a good example.

s151
00:11:58.560 --> 00:12:06.640
Um like how it how we can actually like use this to be even become a gate for slop and and not let your agent ship slop.

s152
00:12:06.640 --> 00:12:09.760
But so the brand API is the first product that we're releasing to to the public.

s153
00:12:09.760 --> 00:12:13.880
This is already in in beta testing with a bunch of our design partners.

s154
00:12:13.880 --> 00:12:22.400
And essentially what it does is it can take let's say a brand URL and extract this into like very specific components that are good for an agent to follow.

s155
00:12:22.400 --> 00:12:30.920
So basically how do we turn something as fuzzy as a brand into something so structured that it becomes easy to for your agent to follow that but also for you to judge against it, right?

s156
00:12:30.920 --> 00:12:35.080
Because I think the piece that we can't forget here is this judgment and verification.

s157
00:12:35.080 --> 00:12:38.720
So yes, this goes and helps your agent to produce something better.

s158
00:12:38.720 --> 00:12:43.280
But how can we also add a way for you to judge okay, is the agent actually staying on track?

s159
00:12:43.280 --> 00:12:47.880
Is it actually performing well to adhere to this brand or how is it failing or where is it failing?

s160
00:12:47.880 --> 00:12:53.400
So this is the first flow I would say that we we're seeing that is really helping to improve quality.

s161
00:12:53.400 --> 00:12:56.720
And what's cool is of course we're talking here about an example of a brand that already exists.

s162
00:12:56.720 --> 00:13:03.160
But let's say you have an agent or you have an app and the person that is using your app actually doesn't have a brand.

s163
00:13:03.160 --> 00:13:04.760
Let's say they're an average consumer.

s164
00:13:04.760 --> 00:13:14.760
Can we actually One of the things that we're creating is basically like a repository, like an index of brands of pre almost like pre-created brand system so that if they want something that feels dreamy,

s165
00:13:14.760 --> 00:13:26.800
why not retrieve a dreamy brand system that already has been thought out to be cohesive instead of doing like a generative approach the moment of that might end up not so great or might end up again in those pillars of slop.

s166
00:13:26.800 --> 00:13:28.600
And I want to show you a real example of this in action.

s167
00:13:28.600 --> 00:13:32.440
So, um there's this company that I think is awesome called the General Intelligence Company of New York.

s168
00:13:32.440 --> 00:13:34.440
They have a sick website, you guys should check it out.

s169
00:13:34.440 --> 00:13:41.760
Um but basically, if you ask Cloud Design to create a slide deck uh in their branding, the the middle one is basically what it comes up with.

s170
00:13:41.760 --> 00:13:44.440
So, the one on the left is is the original brand.

s171
00:13:44.440 --> 00:13:46.640
Uh this is the kind of the the default.

s172
00:13:46.640 --> 00:13:52.880
And if you kind of use this extraction actually in the process, it creates something that's way more high fidelity with the original.

s173
00:13:52.880 --> 00:13:56.440
Um and that even like in the details, I would say like feels right.

s174
00:13:56.440 --> 00:13:59.160
So, this is just to show an example of it in in action.

s175
00:13:59.160 --> 00:14:11.960
Um but yeah, I think we I think all of us would agree that like human human taste and kind of the peak of human craft is always going to be like deeply valuable.

s176
00:14:11.960 --> 00:14:21.800
And that right now, I think the challenge is we are almost even not earning the right to debate this like how can we have uh models like reach this like pinnacle of taste.

s177
00:14:21.800 --> 00:14:22.760
I don't think it's about that at all.

s178
00:14:22.760 --> 00:14:25.480
It's like how do we first just like raise the bar.

s179
00:14:25.480 --> 00:14:27.600
Like the bar is currently, I would say, on the ground.

s180
00:14:27.600 --> 00:14:36.000
And so, I think all of this work that we're putting into like how do we decompose a problem and how do we measure it is exactly so that we can at least like improve this bar

s181
00:14:36.000 --> 00:14:36.800
um of quality.

s182
00:14:36.800 --> 00:14:39.760
And I think we have to start with that.

s183
00:14:41.240 --> 00:14:41.800
That's it.

s184
00:14:41.800 --> 00:14:43.960
Uh thank you very much for for the time.

s185
00:14:43.960 --> 00:14:46.225
Uh this is this is awesome.

s186
00:14:46.225 --> 00:14:48.225
[applause]

s187
00:15:03.910 --> 00:15:05.910
[music]