WEBVTT

NOTE Sentence-level transcript of https://www.youtube.com/watch?v=QrMcNe2jjt8

NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/QrMcNe2jjt8.html. A cue ends where the next begins, or 2 s after its last word.

s1
00:00:01.309 --> 00:00:03.309
[music]

s2
00:00:12.560 --> 00:00:14.520
How's it going?

s3
00:00:14.520 --> 00:00:18.120
The end of uh the end of the conference, how's everybody feeling?

s4
00:00:18.120 --> 00:00:21.160
Tired, drinking from the fire hose as well?

s5
00:00:21.160 --> 00:00:31.520
Are you guys a little bit tired of hearing loop engineering, harness engineering, software factory, evolves, pre-training, post-training data, and whatnot?

s6
00:00:31.520 --> 00:00:34.680
Uh but anyways, those topics were more than valid, right?

s7
00:00:34.680 --> 00:00:35.960
Uh hi.

s8
00:00:35.960 --> 00:00:38.680
Some uh really uh common faces here.

s9
00:00:38.680 --> 00:00:43.680
By the way, I'm super excited to be here and talk to you guys about this title.

s10
00:00:43.680 --> 00:00:52.760
And And I'm sorry, when I was when I submitted the application, I was thinking of potentially a catchy title, but I don't like this at all.

s11
00:00:52.760 --> 00:00:56.640
So, with all due respect to the organizers, I'll have to make a change.

s12
00:00:56.640 --> 00:01:08.320
Uh and the actual topic that I want to focus here today is lessons learned uh from a team that is building proper generative uh UX and UI.

s13
00:01:08.320 --> 00:01:13.240
And I was going to uh touch on agentic orchestration, but come on.

s14
00:01:13.240 --> 00:01:17.160
Uh over the last 3 days, this is what we heard all the time.

s15
00:01:17.160 --> 00:01:29.680
So, I'd rather focus on what I didn't hear enough about here in the conference, and hopefully you walk away, if not with something very tangible, but with a new mental model that can spark meaningful discussions down the road.

s16
00:01:29.680 --> 00:01:31.080
Sound good?

s17
00:01:31.080 --> 00:01:32.160
All right.

s18
00:01:32.160 --> 00:01:33.320
Very good.

s19
00:01:33.320 --> 00:01:34.480
So, hi everybody.

s20
00:01:34.480 --> 00:01:35.200
Uh I'm Gus.

s21
00:01:35.200 --> 00:01:37.440
I'm a general manager at commercetools.

s22
00:01:37.440 --> 00:01:41.040
I lead product, UX, and engineering for zero-to-one products.

s23
00:01:41.040 --> 00:01:46.560
I tell people I'm in a very privileged position because we get to build really cool stuff.

s24
00:01:46.560 --> 00:01:49.960
So, we cook really interesting stuff at commercetools.

s25
00:01:49.960 --> 00:01:50.600
Fantastic.

s26
00:01:50.600 --> 00:01:54.800
And this is what you guys can expect uh at least during the presentation.

s27
00:01:54.800 --> 00:02:05.800
Uh I'd like to make sure that we're on the same page with respect to the problem uh the problem space uh followed by a quick demo of the product because I'm not sure if you guys agree, an image speaks more than 1,000 words.

s28
00:02:05.800 --> 00:02:10.080
So, it would just uh make it more tangible for uh everybody here.

s29
00:02:10.080 --> 00:02:17.120
Followed by a rapid discussion on the emergence of UI protocols, and I'm not sure if you guys joined maybe some of the talks here.

s30
00:02:17.120 --> 00:02:22.520
Even the founder of some of these protocols were here uh this week, and that was really cool.

s31
00:02:22.520 --> 00:02:28.360
Followed by and last but not least, the challenges that uh my team uh and I faced and we still face.

s32
00:02:28.360 --> 00:02:33.240
And some of the mitigation tactics uh that we put in place to overcome some of these challenges.

s33
00:02:33.240 --> 00:02:36.000
So, with that let's continue.

s34
00:02:36.000 --> 00:02:37.800
The problem space.

s35
00:02:37.800 --> 00:02:46.680
This is This is more like a a a statement um and we're still adapt to the software that we ship, not the other way around.

s36
00:02:46.680 --> 00:02:47.080
Right?

s37
00:02:47.080 --> 00:03:00.480
And then uh although you could argue when GPT came out, this was November uh 2022, we had a really good glimpse of uh real personalization, but everything else uh remained static.

s38
00:03:00.480 --> 00:03:08.239
And even with AI, we keep shipping a lot of stuff much faster, but to a significant extent, it is still static.

s39
00:03:08.239 --> 00:03:11.959
And my question is, why?

s40
00:03:11.959 --> 00:03:18.280
So, for the uh over the last 40 years, uh we kept shipping a static experiences.

s41
00:03:18.280 --> 00:03:35.200
And if I put it myself in the shoes of uh some of uh my customers, they need several SaaS applications for uh the day-to-day work, and each with its own mental model and its own way to get anything done.

s42
00:03:35.200 --> 00:03:40.680
And over time, it just kept getting worse, just accruing uh debt.

s43
00:03:40.680 --> 00:03:47.519
The cognitive load that we wanted to remove and that we wanted to transfer to the machine, uh it's on us.

s44
00:03:47.519 --> 00:03:52.160
And now with AI, things can be different.

s45
00:03:52.160 --> 00:03:55.880
I don't know if you guys agree, but this is uh what we what we think.

s46
00:03:55.880 --> 00:03:57.880
And I do have a couple of examples.

s47
00:03:57.880 --> 00:04:03.040
I'm not going to say out loud the name of these apps, but let's take a look.

s48
00:04:03.040 --> 00:04:03.880
Here.

s49
00:04:03.880 --> 00:04:05.200
First one.

s50
00:04:05.200 --> 00:04:07.040
You know, that icon is very well known.

s51
00:04:07.040 --> 00:04:11.200
This is probably the most famous CRM of all times.

s52
00:04:11.200 --> 00:04:15.080
But when I look at this screen, there's a lot going on.

s53
00:04:15.080 --> 00:04:16.560
I don't even know where to start.

s54
00:04:16.560 --> 00:04:17.160
Right?

s55
00:04:17.160 --> 00:04:24.440
Not only the information overload, how many features how many teams do you think are somewhat involved just to ship this?

s56
00:04:24.440 --> 00:04:25.960
Many, probably.

s57
00:04:25.960 --> 00:04:26.280
Right?

s58
00:04:26.280 --> 00:04:27.680
And this is just one.

s59
00:04:27.680 --> 00:04:28.400
Let's have a look.

s60
00:04:28.400 --> 00:04:30.720
I have a couple of other examples.

s61
00:04:30.720 --> 00:04:33.120
So, this here.

s62
00:04:33.120 --> 00:04:34.560
Look at that.

s63
00:04:34.560 --> 00:04:36.520
Oh my god, this is a fancy table.

s64
00:04:36.520 --> 00:04:38.000
I don't even know where to start.

s65
00:04:38.000 --> 00:04:40.800
But anyways, one more.

s66
00:04:40.800 --> 00:04:43.600
Does everybody know this one here?

s67
00:04:43.600 --> 00:04:44.640
Beautiful.

s68
00:04:44.640 --> 00:04:46.200
Beautiful UI.

s69
00:04:46.200 --> 00:04:46.480
Right?

s70
00:04:46.480 --> 00:04:52.880
It's a fantastic, super intuitive, and something else just to highlight.

s71
00:04:52.880 --> 00:04:58.360
Do you guys know how much time these companies need to invest in onboarding people?

s72
00:04:58.360 --> 00:05:00.520
So, that was the the trade-off.

s73
00:05:00.520 --> 00:05:00.800
Right?

s74
00:05:00.800 --> 00:05:06.680
So, you got to allocate a lot of time for a lot of people just to onboard newcomers.

s75
00:05:06.680 --> 00:05:12.320
As a result of this complexity that has been introduced over time.

s76
00:05:12.320 --> 00:05:16.040
So, this is just at least the hardcore evidence.

s77
00:05:16.040 --> 00:05:20.120
So, different apps, different logic every single time.

s78
00:05:20.120 --> 00:05:22.200
And then more apps are coming out.

s79
00:05:22.200 --> 00:05:28.800
So, imagine just put yourself in the shoes of average user, and then oh, now I have five five apps.

s80
00:05:28.800 --> 00:05:33.400
And then every single one I need to learn how to navigate, how to browse, and so on and so forth.

s81
00:05:33.400 --> 00:05:37.040
And then this has been the history up until now.

s82
00:05:37.040 --> 00:05:41.600
But then this was August last year, I sat down with my boss.

s83
00:05:41.600 --> 00:05:45.040
It happens to be the founder of the company, so big shout out to my boss.

s84
00:05:45.040 --> 00:05:51.960
And then we asked this uh question because at Commerce Tools, we are an API first company, 300 plus still counting.

s85
00:05:51.960 --> 00:06:03.080
And we ask this posing question, through the lens of artificial intelligence, what are the foundational shifts that could be made if we could change drastically the way that we interact with software?

s86
00:06:03.080 --> 00:06:05.680
Not in a uh static uh fashion.

s87
00:06:05.680 --> 00:06:10.480
And the answer to that question led to the product that I'm going to demo right now.

s88
00:06:10.480 --> 00:06:11.600
Let's go.

s89
00:06:11.600 --> 00:06:12.800
How about quick demo?

s90
00:06:12.800 --> 00:06:14.280
Guys like the idea?

s91
00:06:14.280 --> 00:06:15.280
Give me a thumbs up.

s92
00:06:15.280 --> 00:06:15.520
Bye.

s93
00:06:15.520 --> 00:06:16.600
I know everybody's tired.

s94
00:06:16.600 --> 00:06:17.320
Let's go.

s95
00:06:17.320 --> 00:06:18.440
Yeah.

s96
00:06:18.440 --> 00:06:19.440
All right.

s97
00:06:19.440 --> 00:06:20.440
Cool.

s98
00:06:20.440 --> 00:06:21.320
Look at this.

s99
00:06:21.320 --> 00:06:22.440
I know, I'm going to zoom in.

s100
00:06:22.440 --> 00:06:23.200
No worries.

s101
00:06:23.200 --> 00:06:25.680
So, I have this query here.

s102
00:06:25.680 --> 00:06:28.880
Create a sales report for Q1.

s103
00:06:28.880 --> 00:06:41.560
And the UX side of me, when I look at uh what was generated, and by the way, everything here on the right side has been auto-generated, guided by us, but this is AI, right?

s104
00:06:41.560 --> 00:06:52.720
Deciding on the placement, on the information architecture, deciding which uh components had to be actually retrieved from the catalog, but I don't like it at all.

s105
00:06:52.720 --> 00:06:52.960
Right?

s106
00:06:52.960 --> 00:07:03.800
It's a Even if you don't know a lot uh about UX, just let's, you know, I have at least four different variations because those were four different terms for the same query.

s107
00:07:03.800 --> 00:07:05.120
Let's check it out.

s108
00:07:05.120 --> 00:07:09.760
First one here, it was Q1 uh Urban Thread, whatever that is.

s109
00:07:09.760 --> 00:07:12.680
Uh Q1, I see all of these KPI cards.

s110
00:07:12.680 --> 00:07:14.480
There's a lot going on here.

s111
00:07:14.480 --> 00:07:18.960
And then my intuition tells me, man, this doesn't add up.

s112
00:07:18.960 --> 00:07:19.760
Okay.

s113
00:07:19.760 --> 00:07:22.560
Second one, it's not Q1 anymore.

s114
00:07:22.560 --> 00:07:24.480
Now this is January and March.

s115
00:07:24.480 --> 00:07:26.280
There's no consistency.

s116
00:07:26.280 --> 00:07:27.160
Does that help?

s117
00:07:27.160 --> 00:07:29.160
Yes or no?

s118
00:07:29.160 --> 00:07:29.880
No, right?

s119
00:07:29.880 --> 00:07:32.520
This is This is You only create confusion.

s120
00:07:32.520 --> 00:07:42.120
If this is a heavy personalized experience for the user, imagine if every single time you need to prompt, and then uh at least the model will output something different.

s121
00:07:42.120 --> 00:07:44.960
This is not good, and this is just the second turn.

s122
00:07:44.960 --> 00:07:47.640
Let's have a look at the third one.

s123
00:07:47.640 --> 00:07:51.680
Oh my god, now there is even more stuff here on the right side.

s124
00:07:51.680 --> 00:08:02.960
But so, this is I have this component, the KPI cards, a bunch of text, a bunch of charts, and then this this was the beginning of a journey.

s125
00:08:02.960 --> 00:08:04.200
One more?

s126
00:08:04.200 --> 00:08:04.760
Yes.

s127
00:08:04.760 --> 00:08:11.000
Okay, now it's still Q1, but still for me this still it's confusing.

s128
00:08:11.000 --> 00:08:18.840
And because this has been a very experimental journey, at least okay, let's let's move on here.

s129
00:08:18.840 --> 00:08:28.160
My feedback to the team and to myself was no, no, and no. There's no way that I would ship this to to prod whatsoever, right?

s130
00:08:28.160 --> 00:08:32.680
And then I have my colleagues here just to confirm what I just said.

s131
00:08:32.680 --> 00:08:37.719
But then things evolved, and I'll like to demo the current state of the product.

s132
00:08:37.719 --> 00:08:40.680
It's much more sophisticated, and let's have a look.

s133
00:08:40.680 --> 00:08:53.120
I I'll like to plan a campaign, and for what it's worth, I'm going to save you from all the nitty-gritty details for everything that is domain specific, but I'm going to pick this query here,

s134
00:08:53.120 --> 00:08:55.080
and then let's see what happens.

s135
00:08:55.080 --> 00:08:58.320
And this is the the agentic orchestration part that I was going to highlight.

s136
00:08:58.320 --> 00:09:04.040
Underneath the hood we have the orchestrator, and the orchestrator can then just extract the intent of the query.

s137
00:09:04.040 --> 00:09:07.520
Based off of the intent of the query, it can locate the tools, right?

s138
00:09:07.520 --> 00:09:19.720
Those can be first party, third party tools, and the outputs of this different it could be agents on MCP servers, combined will give enough what I call ammunition and context

s139
00:09:19.720 --> 00:09:24.760
for the UX agent to eventually render something that we call meaningful.

s140
00:09:24.760 --> 00:09:30.000
So, compared to the previous turns, this is this is decent, right?

s141
00:09:30.000 --> 00:09:38.920
I want to just remove my bias, but the overall aesthetics, the look and feel of this query, uh it it resonates with me.

s142
00:09:38.920 --> 00:09:39.640
Would you agree?

s143
00:09:39.640 --> 00:09:42.160
Give me a thumbs up if you like if you agree.

s144
00:09:42.160 --> 00:09:45.120
Okay, at least um the vast majority here.

s145
00:09:45.120 --> 00:09:47.320
And then see, it is decent.

s146
00:09:47.320 --> 00:09:48.880
And let me just continue here.

s147
00:09:48.880 --> 00:09:50.720
And then once again, right?

s148
00:09:50.720 --> 00:09:54.800
This was decided by AI guided by us.

s149
00:09:54.800 --> 00:09:57.560
I would just want to make make that clear here.

s150
00:09:57.560 --> 00:10:02.280
And then I'm going to touch on the on the UI protocols and then how you can make this happen.

s151
00:10:02.280 --> 00:10:03.360
But okay, let's see.

s152
00:10:03.360 --> 00:10:06.160
If I approve here, and then this is already live.

s153
00:10:06.160 --> 00:10:07.440
And this is pre-prod, right?

s154
00:10:07.440 --> 00:10:09.080
So, uh this is great.

s155
00:10:09.080 --> 00:10:11.880
Let me go back to my presentation.

s156
00:10:12.240 --> 00:10:12.880
Perfect.

s157
00:10:12.880 --> 00:10:14.160
I have one more question.

s158
00:10:14.160 --> 00:10:17.040
Are you guys as skeptical that this is possible?

s159
00:10:17.040 --> 00:10:19.200
Because I can tell you this it is possible.

s160
00:10:19.200 --> 00:10:20.800
If you're still skeptical, don't worry.

s161
00:10:20.800 --> 00:10:30.360
I have all of these guys uh here uh also uh every day just looking at me and then challenging whether uh this uh can be made possible at scale, right?

s162
00:10:30.360 --> 00:10:31.680
And then uh okay.

s163
00:10:31.680 --> 00:10:39.720
And then this is the part that I'll like just to touch uh touch base on the uh on the three ways that you can render what you just saw.

s164
00:10:39.720 --> 00:10:41.800
And they're different UI protocols.

s165
00:10:41.800 --> 00:10:44.760
Are you guys familiar with uh generative uh UI?

s166
00:10:44.760 --> 00:10:46.240
Have you guys played with it?

s167
00:10:46.240 --> 00:10:47.720
Let me see here.

s168
00:10:47.720 --> 00:10:49.320
Okay, well, that's really cool.

s169
00:10:49.320 --> 00:10:50.240
Uh okay.

s170
00:10:50.240 --> 00:10:58.520
So, uh what I would like just to share with you guys it's all about how much control you want to exercise over the experience.

s171
00:10:58.520 --> 00:11:07.360
And this matters a lot because uh you know, as a non-deterministic solution, uh you can decide if you want something I really um like this.

s172
00:11:07.360 --> 00:11:08.960
So, let's have a look.

s173
00:11:08.960 --> 00:11:11.720
Here, this is uh ChatGPT.

s174
00:11:11.720 --> 00:11:18.480
And my query was help me find a Japanese restaurant uh in SF today.

s175
00:11:18.480 --> 00:11:23.480
If you guys see here, this component this component is very opinionated.

s176
00:11:23.480 --> 00:11:24.760
Would you agree with that?

s177
00:11:24.760 --> 00:11:25.839
Here?

s178
00:11:25.839 --> 00:11:26.120
Right?

s179
00:11:26.120 --> 00:11:31.200
So, uh you can have complete control over this uh component.

s180
00:11:31.200 --> 00:11:36.760
So, depending upon the nature of your business, this works really well, right?

s181
00:11:36.760 --> 00:11:39.640
And for that, let me just go back here.

s182
00:11:39.640 --> 00:11:41.960
And then this is what I call a control.

s183
00:11:41.960 --> 00:11:45.000
Essentially, you ship the component as it is.

s184
00:11:45.000 --> 00:11:49.520
The agent will pick and it will display exactly the way that you describe.

s185
00:11:49.520 --> 00:12:01.320
However, depending upon the nature of your business, at least for us, right, at my company, where a B2B SaaS, the there's so much configuration that we don't want to be over uh prescriptive

s186
00:12:01.320 --> 00:12:06.200
because uh the feedback that I keep getting from my customers, "Oh, the flows are so confusing.

s187
00:12:06.200 --> 00:12:07.880
There's so much configuration.

s188
00:12:07.880 --> 00:12:11.400
How can you remove the cognitive load uh for me?"

s189
00:12:11.400 --> 00:12:16.520
But uh if you're like Booking, for example, this approach uh works uh really well.

s190
00:12:16.520 --> 00:12:18.560
And then let's see uh how it works.

s191
00:12:18.560 --> 00:12:22.160
So, essentially, you have the agent.

s192
00:12:22.160 --> 00:12:28.320
The agent will just pick the component from your catalog and then it will render uh as it is, right?

s193
00:12:28.320 --> 00:12:36.200
And uh in the interest uh of time, I'm not going to touch base on the code snippets uh that I have for this uh three different types.

s194
00:12:36.200 --> 00:12:41.680
But afterwards, if you guys are interested, uh I can share the presentation and then you can have a look, all right?

s195
00:12:41.680 --> 00:12:42.520
Very good.

s196
00:12:42.520 --> 00:12:45.360
This is uh at least on the left side.

s197
00:12:45.360 --> 00:12:50.800
Let's discuss a little bit on the right side because this is when you give full autonomy to the LLM.

s198
00:12:50.800 --> 00:12:55.839
If you guys remember uh at least the previous attempts for my product, this exactly what we did.

s199
00:12:55.839 --> 00:13:03.120
So, we just said, "So, hey LLM, how uh how would you compose this experience knowing that you have these components?"

s200
00:13:03.120 --> 00:13:03.440
Right?

s201
00:13:03.440 --> 00:13:12.760
But uh that was uh that was a little bit uh a little bit of our opinion because uh it has uh it had uh some of the components available.

s202
00:13:12.760 --> 00:13:18.080
But it could happen that uh you can just delegate fully to the LLM.

s203
00:13:18.080 --> 00:13:23.760
And then right now, I'm here on Claude and I asked Claude, "So, hey, create an org chart with three levels."

s204
00:13:23.760 --> 00:13:24.760
That was it.

s205
00:13:24.760 --> 00:13:30.720
And then Claude just rendered this diagram and it works really well.

s206
00:13:30.720 --> 00:13:31.040
Right?

s207
00:13:31.040 --> 00:13:37.280
But here, if I put myself in the shoes of a company, I'm not sure I would delegate fully to the LLM.

s208
00:13:37.280 --> 00:13:42.320
Because I cannot control at least the app the output and the outcome.

s209
00:13:42.320 --> 00:13:49.800
And me personally, me guys, as a UX leader, the UX side of me will always say no. You got to be in control.

s210
00:13:49.800 --> 00:13:56.160
There's been a couple of talks here at least this week on design, on taste, and judgment.

s211
00:13:56.160 --> 00:13:57.880
And this matters a lot.

s212
00:13:57.880 --> 00:14:05.960
If you guys want to embark on this journey of leveraging these protocols, you don't want to delegate too much of the actual experience to the LLM.

s213
00:14:05.960 --> 00:14:10.600
You got to find alternatives and I'm going to touch on that in just a little bit.

s214
00:14:10.600 --> 00:14:11.480
Okay, cool.

s215
00:14:11.480 --> 00:14:15.600
And then this is how the open-ended approach works.

s216
00:14:15.600 --> 00:14:30.160
So essentially there there's going to be an MCP tool and then this will literally ship the HTML and then in a sandbox I frame environment, this will be rendered in the host of your choice.

s217
00:14:30.160 --> 00:14:31.200
But it can be a chat.

s218
00:14:31.200 --> 00:14:33.120
It can be this cloud.

s219
00:14:33.120 --> 00:14:34.280
It could be a chat GPT.

s220
00:14:34.280 --> 00:14:37.839
It could be perplexity or it could be any other chat.

s221
00:14:37.839 --> 00:14:42.120
If you're willing just to to give full control to the LLM, good luck.

s222
00:14:42.120 --> 00:14:50.120
But the one that I would like to highlight is this here and this was our choice that we call the declarative.

s223
00:14:50.120 --> 00:14:52.839
Declarative is in the middle.

s224
00:14:52.839 --> 00:15:00.240
If you guys heard some of the protocols and I don't want to get into the specifics of each because they have different characteristics.

s225
00:15:00.240 --> 00:15:11.640
But HTMX from Google, JSON render from Vercel, OpenUI by Thesis, those are some of the protocols that will give you this in between here.

s226
00:15:11.640 --> 00:15:13.640
And let let's have a look, right?

s227
00:15:13.640 --> 00:15:25.360
So at least for my product, the one that I just showed, the orchestrator agent will eventually, if you think of the whole traversal, the user will enter the query

s228
00:15:25.360 --> 00:15:27.600
and then there's going to be the intent classification.

s229
00:15:27.600 --> 00:15:41.320
Based off of the intent classification, then the tools will be invoked, the data will be retrieved, and somewhat somewhat in between Alice, there will be the mapping of the eligible components from your catalog

s230
00:15:41.320 --> 00:15:45.880
to the entities of of the tools, right?

s231
00:15:45.880 --> 00:15:49.280
And then the orchestrator will just broadcast this UI description.

s232
00:15:49.280 --> 00:15:50.800
It's like a UI spec.

s233
00:15:50.800 --> 00:15:58.800
This UI spec will be also we have this component catalog here and we use the Zod schema and then you got to be compliant with this protocols.

s234
00:15:58.800 --> 00:16:02.600
This is just one of the requirements and then you just render that.

s235
00:16:02.600 --> 00:16:03.280
Right?

s236
00:16:03.280 --> 00:16:09.080
And then their final output will be the native UI, in this case the React components.

s237
00:16:09.080 --> 00:16:10.040
This is it.

s238
00:16:10.040 --> 00:16:18.240
The good thing about the declarative approach is it will be compliant with your design system everywhere.

s239
00:16:18.240 --> 00:16:19.200
This matters a lot.

s240
00:16:19.200 --> 00:16:28.160
So, in our case, we did not want to delegate to the LLM because you guys saw over there you can't change the copy, right?

s241
00:16:28.160 --> 00:16:29.640
So, it's not key one.

s242
00:16:29.640 --> 00:16:33.080
Sometimes it's going to be March January to March.

s243
00:16:33.080 --> 00:16:34.120
It matters a lot.

s244
00:16:34.120 --> 00:16:37.360
So, within UX we have different factors, right?

s245
00:16:37.360 --> 00:16:41.280
We have the actual UX's if you think of the overall experience.

s246
00:16:41.280 --> 00:16:46.280
There is UI, there's also copy, UX writing, and so on and so forth.

s247
00:16:46.280 --> 00:16:49.280
But this approach gives us this in between.

s248
00:16:49.280 --> 00:16:58.200
It is less deterministic and I think this is a really good segue to some of the challenges because imagine, I'm going to use my example once again, the orchestrator

s249
00:16:58.200 --> 00:17:07.120
will just fetch the eligible components for that query, but then it's up to the LLM how to place in the UI.

s250
00:17:07.120 --> 00:17:14.640
And this can get really really messy, but and those are the challenges that I'll like to share with you guys here.

s251
00:17:14.640 --> 00:17:18.480
I got to be careful because I've got only 30 minutes, but let's go.

s252
00:17:18.480 --> 00:17:26.800
So, the first challenge is if the agent picks the components, who's in charge or what entity is in charge of arranging them?

s253
00:17:26.800 --> 00:17:29.240
And this is information architecture, right?

s254
00:17:29.240 --> 00:17:32.200
This is a critical aspect of UX.

s255
00:17:32.200 --> 00:17:37.360
And then once again, if you put yourself in the shoes of the average customer, it matters a lot.

s256
00:17:37.360 --> 00:17:41.240
So, if you're just left alone, the placement can be totally random.

s257
00:17:41.240 --> 00:17:47.680
So, at least in my team, we borrow this concept of atomic design.

s258
00:17:47.680 --> 00:17:50.080
And atomic design, it goes like this.

s259
00:17:50.080 --> 00:17:51.920
Let me just change here.

s260
00:17:51.920 --> 00:17:52.840
Uh yeah.

s261
00:17:52.840 --> 00:18:05.000
So, as you can see, atomic design, and then I have the definition, is a methodology composed of five distinct stages working together to create interface design systems in a more deliberate

s262
00:18:05.000 --> 00:18:06.400
and hierarchical manner.

s263
00:18:06.400 --> 00:18:07.480
This helps a lot.

s264
00:18:07.480 --> 00:18:07.680
Why?

s265
00:18:07.680 --> 00:18:16.920
Because those are the individual elements, and if you think of the overall structure of the page, it gives me the ability to steer it as I see fit.

s266
00:18:16.920 --> 00:18:22.880
And what we've done in my team, so this UX agent, we harnessed this UX agent.

s267
00:18:22.880 --> 00:18:35.200
So, we eventually taught this this UX agent what good looks like, what is the optimal layout for a given situation, and we have a catalog of different templates.

s268
00:18:35.200 --> 00:18:41.560
But I'll like to show you at least this because this detail is very important.

s269
00:18:41.560 --> 00:18:45.240
Here, it is the overall hierarchy hierarchy.

s270
00:18:45.240 --> 00:18:58.680
So, if you remember part of the orchestrator, the orchestrator will eventually just fetch the eligible components to accomplish the query of the user, but then the next big question is, how do we arrange that?

s271
00:18:58.680 --> 00:19:01.800
The approach that we used was think of this hierarchy.

s272
00:19:01.800 --> 00:19:05.200
So, you have the overall page, the layout.

s273
00:19:05.200 --> 00:19:08.040
The layout will contain different slots.

s274
00:19:08.040 --> 00:19:10.240
So, think of this one here, the header.

s275
00:19:10.240 --> 00:19:14.120
You can have the main, and then you can have sub sub slots.

s276
00:19:14.120 --> 00:19:16.320
Sub slots can have sub slots.

s277
00:19:16.320 --> 00:19:20.680
And within the sub slots, you can have eligible component categories.

s278
00:19:20.680 --> 00:19:31.600
And then this will allow us to steer eventually the the the optimal placement of the components that have been retrieved by the orchestrator.

s279
00:19:31.600 --> 00:19:38.120
So this is literally us codifying our UX knowledge into this agent.

s280
00:19:38.120 --> 00:19:40.680
So the next time, you know, it doesn't matter.

s281
00:19:40.680 --> 00:19:42.880
We'll just follow the same approach.

s282
00:19:42.880 --> 00:19:44.800
And then this is the hierarchy that we're using.

s283
00:19:44.800 --> 00:19:48.280
So layout to slot to sub slot to components.

s284
00:19:48.280 --> 00:19:52.720
However, because of the orchestrator, we we flipped the order.

s285
00:19:52.720 --> 00:19:58.720
So from components, components will map to sub slot, sub slots to slots, and slots to templates.

s286
00:19:58.720 --> 00:20:02.560
And then we can just arrange as needed.

s287
00:20:02.560 --> 00:20:04.440
So this was a hell of a challenge.

s288
00:20:04.440 --> 00:20:16.240
It is still a challenge, by the way, and it's a really good segue to the second one, the second challenge, which is the actual design, your design system and your catalog.

s289
00:20:16.240 --> 00:20:19.360
This becomes the heartbeat of the whole thing, right?

s290
00:20:19.360 --> 00:20:27.960
I cannot stress enough, if you guys see the potential of leveraging this UI protocols for your product, this is going to be a big deal, right?

s291
00:20:27.960 --> 00:20:38.040
So we're pushing the boundaries and then we're testing and we keep on testing these different protocols that I mentioned, ATUI, JSON Render, Open UI, and so on and so forth.

s292
00:20:38.040 --> 00:20:45.520
But this has been a quite challenging because the the catalog is the contract between the agent and the UI.

s293
00:20:45.520 --> 00:20:48.280
So every property mat- matters.

s294
00:20:48.280 --> 00:20:51.720
And not only for the catalog, but for the layout as well.

s295
00:20:51.720 --> 00:20:57.360
So if you remember, the layout has its own components, the slots and the sub slots.

s296
00:20:57.360 --> 00:21:01.200
Each of those components will have its own attributes.

s297
00:21:01.200 --> 00:21:05.480
And all of that, this curation, let me just encapsulate into curation.

s298
00:21:05.480 --> 00:21:10.600
This curation is absolutely needed so that you can deliver something meaningful.

s299
00:21:10.600 --> 00:21:14.200
Not as some demo that you will see out there for the sake of demo, right?

s300
00:21:14.200 --> 00:21:20.440
So, this this this will give you a control will allow you to steer from UX perspective.

s301
00:21:20.440 --> 00:21:21.200
Very good.

s302
00:21:21.200 --> 00:21:29.640
And last but not least, one significant challenge that we had is that my teams do not design the pixel anymore.

s303
00:21:29.640 --> 00:21:31.280
I don't know if you could see that, right?

s304
00:21:31.280 --> 00:21:34.000
So, we're not here designing at the entire flow.

s305
00:21:34.000 --> 00:21:47.160
Now, AI can dictate that to a significant extent, but the nature of the work shifted quite a bit and it's been an interesting journey to say the least, a really good one, but even for the non-technical

s306
00:21:47.160 --> 00:21:54.280
PMs and UX designers, it it was a big hit because right now we talk about the schema.

s307
00:21:54.280 --> 00:21:57.240
Let's talk about this curation of the catalog.

s308
00:21:57.240 --> 00:21:58.880
Let's talk about the rules.

s309
00:21:58.880 --> 00:22:02.280
Let's talk about the synthetic data that we can generate.

s310
00:22:02.280 --> 00:22:08.680
How can we generate the queries that will map to a given component as part of this mapping logic?

s311
00:22:08.680 --> 00:22:11.080
Let's talk about interaction patterns.

s312
00:22:11.080 --> 00:22:18.400
So, this if once again, if you guys are going to embark on this journey, be aware that the people element is very important.

s313
00:22:18.400 --> 00:22:24.680
When I talk to other leaders, I I talk about the three P's: people, product, and process, right?

s314
00:22:24.680 --> 00:22:28.480
And then a very lightweight process, but this matters a lot.

s315
00:22:28.480 --> 00:22:33.440
And with that, I'm going to leave a couple of resources here.

s316
00:22:33.440 --> 00:22:38.280
So, and by the way, those are talks from AIE for what it's worth.

s317
00:22:38.280 --> 00:22:43.000
So, yeah, people that have been talking about this protocols over and over and over.

s318
00:22:43.000 --> 00:22:46.880
So, please just take advantage, take a screenshot.

s319
00:22:46.880 --> 00:22:52.600
And if you guys want to connect with me here, yeah, my LinkedIn or just take a screenshot.

s320
00:22:52.600 --> 00:22:56.200
I'll love to talk more about the topic.

s321
00:22:56.200 --> 00:22:58.000
I can tell this is just a matter of time, right?

s322
00:22:58.000 --> 00:22:58.920
So, this is coming.

s323
00:22:58.920 --> 00:23:01.680
So, thank you so much.

s324
00:23:02.720 --> 00:23:04.880
I know.

s325
00:23:15.166 --> 00:23:17.166
[music]