WEBVTT

NOTE Sentence-level transcript of https://www.youtube.com/watch?v=ZJjRgvjhq3M

NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/ZJjRgvjhq3M.html. A cue ends where the next begins, or 2 s after its last word.

s1
00:00:01.309 --> 00:00:03.309
[music]

s2
00:00:12.719 --> 00:00:14.240
So hello everyone.

s3
00:00:14.240 --> 00:00:19.279
Uh glad that you know there's quite a crowd here even if it's holiday.

s4
00:00:19.279 --> 00:00:20.160
It's Monday morning.

s5
00:00:20.160 --> 00:00:22.640
It's 9:00 a.m. So I'm Stephan Fabri.

s6
00:00:22.640 --> 00:00:34.480
I'm part of the open AAI the applied AI engineering team alongside Oliver and also we have in a crowd another part of uh our team Dom.

s7
00:00:34.480 --> 00:00:43.440
So feel free during the session you know if you don't want to ask ask question we also have a third man that you can leverage.

s8
00:00:43.440 --> 00:00:54.640
So today's session is going to be a two-hour workshop so it's going to be interactive you are going to use chib work and codeex more into the detail later

s9
00:00:54.640 --> 00:01:02.879
and we really want to kind of show you the capabilities um of like our product within the financial services space.

s10
00:01:02.879 --> 00:01:15.600
How can you merge, you know, financial data, you know, LLM, AI, intelligence, and the right output to put everything together to help like your day-to-day business.

s11
00:01:15.600 --> 00:01:25.200
So, today we're going to focus more on a wealth management case given like you know, we know in the crowd there are a lot of people coming from asset manager, investment banks.

s12
00:01:25.200 --> 00:01:29.200
So, we thought that um this could be like an interesting case.

s13
00:01:29.200 --> 00:01:36.720
So to give you a little bit of a background like uh our team focus mostly on interacting with clients.

s14
00:01:36.720 --> 00:01:52.159
So we spend all our days working with like some of you guys helping really to understand you know how can I transform like put this model into production at my company and extract the maximum possible value.

s15
00:01:52.159 --> 00:01:59.759
So for each dollar that you invest with us, our task is really to help you maximize the value extract from it.

s16
00:01:59.759 --> 00:02:10.800
So no matter if it's operation, if it's an agent, if it's like a code coding productivity, that's more or less our day-to-day and our team covers mostly like the financial institution here in New York.

s17
00:02:10.800 --> 00:02:14.000
So that's about me, Oliver.

s18
00:02:14.000 --> 00:02:23.520
Yeah, I'm working in the same team also long history in financial services, asset managers and also investor in banking but I'm a builder at heart.

s19
00:02:23.520 --> 00:02:33.280
So today we want to talk about also how to build with the platform um and show you the use case here but yeah happy to help um I work with big customers

s20
00:02:33.280 --> 00:02:36.879
um in the room probably a couple of people I have seen already.

s21
00:02:36.879 --> 00:02:44.959
So if you approach us want to solve a problem with the API or with codeex yeah we are the right people.

s22
00:02:45.519 --> 00:02:56.720
So before continuing I can I ask you to scan this QR code and put your uh information is mostly the email address.

s23
00:02:56.720 --> 00:03:04.640
The reason is because we want to give you access to a specific works uh workspace um within OpenAI.

s24
00:03:04.640 --> 00:03:10.159
So you're going to get access to the financial data and you're going to have unlimited token for the session.

s25
00:03:10.159 --> 00:03:14.159
Plus we also enable the ultraast mode.

s26
00:03:14.159 --> 00:03:19.360
So is a specific like mode for like our latest model.

s27
00:03:19.360 --> 00:03:27.200
So 6.1 soul and six Astra that reaches like a 300 token per second on average.

s28
00:03:27.200 --> 00:03:32.319
So I feel that like for a workshop makes it easier to like you know create something in a faster way.

s29
00:03:32.319 --> 00:03:41.840
So I'm going to show the QR code later but please uh it's really important otherwise you're not going to get access.

s30
00:03:41.840 --> 00:03:47.519
So just maybe our colleague in the room also if you need it now you come later maybe just people sitting there.

s31
00:03:47.519 --> 00:03:52.799
So we need that to get access to the sh GBT work website basically.

s32
00:03:52.799 --> 00:03:55.680
So that's what we need.

s33
00:03:56.159 --> 00:04:01.439
So um just kind of a preview of the session today.

s34
00:04:01.439 --> 00:04:03.680
This is kind of like a tentative schedule.

s35
00:04:03.680 --> 00:04:07.519
So probably we're going to like go over and mix and match a little bit.

s36
00:04:07.519 --> 00:04:17.280
But we're going to give you a quick introduction uh on like chibi for work uh work for financial services and like kind of like more the ecosystem.

s37
00:04:17.280 --> 00:04:26.720
Then we're going to spend 10 minutes focusing a little bit more on the setup making sure that all of you guys can access um judg work for financial services

s38
00:04:26.720 --> 00:04:35.520
and then we're going to have kind of like two main session that I'm going to be led by Oliver like the first one where we're going to focus more on

s39
00:04:35.520 --> 00:04:55.600
work and then we're going to move to codeex and finally um we kept like 20 minutes mostly like for question and kind of like general feedback and like about uh you know what's going on in the space and you know whatever question that you might have like both on the coding and also like more on the industry

s40
00:04:55.600 --> 00:04:56.960
wise.

s41
00:04:56.960 --> 00:05:02.160
So let's now go to a quick introduction on our services.

s42
00:05:02.160 --> 00:05:08.720
So as you probably know we have three main interfaces three main surfaces that we use.

s43
00:05:08.720 --> 00:05:21.759
Chajibbt that is like you know the original one we have chibbity work that is basically something that it's more focused for knowledge workers so it's like you know it can run agent

s44
00:05:21.759 --> 00:05:29.680
it can work it works with plug-in but it's not really like codeex that is more meant for like engineering type of task.

s45
00:05:29.680 --> 00:05:40.560
So these are the three surfaces that right now we are offering and as already mentioned we're going to focus on chargibility work first and then codeex.

s46
00:05:40.560 --> 00:05:57.199
So what we did is basically within the charge work and charge financial services we tried to create a product that is enterprise focused that basically it's fine-tuned like the model is the same but the product itself is fine-tuned

s47
00:05:57.199 --> 00:06:00.080
for the specific like financial services type of user.

s48
00:06:00.080 --> 00:06:03.919
So there are like three main component to this product.

s49
00:06:03.919 --> 00:06:05.520
So first of all the data.

s50
00:06:05.520 --> 00:06:20.319
So it's we have some like built-in data that I think it's they're high quite high quality but also we you can access to other third party data vendor directly

s51
00:06:20.319 --> 00:06:29.680
within uh chachbt so the agent can interact directly with the data sources without any um overhead.

s52
00:06:29.680 --> 00:06:39.440
Then second we really like pushed [clears throat] our model and trained them on specific financial um type of work.

s53
00:06:39.440 --> 00:06:44.800
So like you know we can think about like you know M&A equity research and so on so forth.

s54
00:06:44.800 --> 00:06:47.440
And third component is the are the artifacts.

s55
00:06:47.440 --> 00:07:00.880
So basically we really pushed in the in the recent months the capability of our model to actually build presentation build spreadsheet and also being able to leverage like reference template

s56
00:07:00.880 --> 00:07:06.960
that from like you know historical example you know and reproduce them like with the for the current analysis.

s57
00:07:06.960 --> 00:07:09.199
So yeah maybe just to add to it.

s58
00:07:09.199 --> 00:07:17.199
So in the in the labs what we do now the first part will really focus on this plug-in share GBT for financial services.

s59
00:07:17.199 --> 00:07:29.680
So it sits in the web surface the website basically and then we show all the feature because how to use a financial data is one thing that's like a one benefit here but also

s60
00:07:29.680 --> 00:07:40.800
there's certain ways or actually in instructs the prompt to present it the information in a chart or use also some verification and also have some annotations where the data comes from.

s61
00:07:40.800 --> 00:07:50.319
So all that if you hang out for the lab will be very important to see and that's sort of the domain specific solution we to showcase.

s62
00:07:51.919 --> 00:08:03.680
So let's start a little bit like with um the building data and that's why we want to on board you to this specific version because charge for financial services is only available like on the enterprise side.

s63
00:08:03.680 --> 00:08:08.240
So these are some of the vendor that are built in within the product.

s64
00:08:08.240 --> 00:08:15.680
So we like had partnership with like the Lupa, Elsag, Pitchbook, Crunchbase and other vendor.

s65
00:08:15.680 --> 00:08:26.479
And basically to give you an idea like of like what's available out of the box, we have like SEC feelings, we have market data.

s66
00:08:26.479 --> 00:08:28.400
Market data of course are delayed.

s67
00:08:28.400 --> 00:08:33.680
So uh I think there is a 15 minutes delayed but still you have like real-time market data.

s68
00:08:33.680 --> 00:08:47.839
you have news and like we also have like you know private company profiles and earning calls transcript and the main difference here is that these data sources are like

s69
00:08:47.839 --> 00:08:49.839
really piped into the product.

s70
00:08:49.839 --> 00:09:02.560
So it's not just like the row data that is entering but like we did like a specific fine-tuning of of the databases to make sure that like this these data are easy to to

s71
00:09:02.560 --> 00:09:04.240
be consumed by the model.

s72
00:09:04.240 --> 00:09:16.080
So what we've seen is like like a nice improvement on like how the model is actually able to ingest this data because we kind of like wrap it in in a format that is like

s73
00:09:16.080 --> 00:09:21.680
not not just like you know the row data set but it's more um fitted for the LLM.

s74
00:09:21.680 --> 00:09:22.160
Yeah.

s75
00:09:22.160 --> 00:09:23.760
So maybe just one thing to add.

s76
00:09:23.760 --> 00:09:36.320
I mean I have done something similar myself actually inside the company and it's quite hard if you pull down all the SEC files and you have it scale the data you have to make sure you grab the new information

s77
00:09:36.320 --> 00:09:44.959
I mean I have done it in a in an asset management case but I think what's really here unique is basically you you have those agents running collecting this data

s78
00:09:44.959 --> 00:09:56.720
we have the financial data pro providers that work with us but also you have to way have a context layer behind it that actually is efficient enough to serve it and that's often a technical challenge.

s79
00:09:56.720 --> 00:09:58.959
So that's what we can show you also later.

s80
00:09:58.959 --> 00:10:00.880
That's a sort of what the product has inside.

s81
00:10:00.880 --> 00:10:13.680
Now it's there's some technical solution for providing that at speed and then like moving a little bit more on like you know how to leverage this data to create analysis like

s82
00:10:13.680 --> 00:10:19.760
what our team our product team did is like build several plugins specifically for financial tasks.

s83
00:10:19.760 --> 00:10:27.680
So specifically we can talk about the uh the investment banking one and autary research one.

s84
00:10:27.680 --> 00:10:42.560
So within these plugins there that is like what we define uh at open eye as a plug-in is kind of like a library of skills plus MCPS and what we built is like we we partner with like top investment banking,

s85
00:10:42.560 --> 00:10:54.560
top asset management, top equity researcher to build the kind of like most flexible but also most like realistic plugins like skills that you can actually leverage as a baseline

s86
00:10:54.560 --> 00:10:55.680
for your analysis.

s87
00:10:55.680 --> 00:11:06.640
So what we've seen not only in the area of like financial services but what we when is that when we provide specific skills these skills present like a good baseline.

s88
00:11:06.640 --> 00:11:18.640
So let's say that you are an equity research it wants to streamline uh his [clears throat] them their job what you can do is literally like take the the skill out of the box.

s89
00:11:18.640 --> 00:11:29.519
there's been a lot of work behind it and then based on using that skill to create like you know a schedule job to create like basically a pipeline that try to reproduce

s90
00:11:29.519 --> 00:11:42.823
what you're aim to and you use that as a baseline and on top of that you try to improve you try to customize to your taste and adding all the beats that are currently missing but what we've seen is that like these type of plugins

s91
00:11:42.823 --> 00:11:43.200
[snorts]

s92
00:11:43.200 --> 00:11:57.279
provide like a strong blaze line so you don't have to start from zero you start from like something that is built out of industry knowledge and then let's say that the 80% is there and then you add your 20% customization

s93
00:11:57.279 --> 00:12:04.160
based on like your specific needs, your specific like view and so on so forth.

s94
00:12:04.720 --> 00:12:11.839
So finally again these are like some of the use cases that are already out of the box.

s95
00:12:11.839 --> 00:12:15.839
You know these are a little bit more skewed on the investment bank.

s96
00:12:15.839 --> 00:12:18.399
We're going to work today more on the asset management.

s97
00:12:18.399 --> 00:12:35.279
If you think about you know pitchbook prep uh preparation, earning analysis, valuation, LBO modeling, uh everything comes out of the box and it's uh quite interesting because even myself like I I come from like a background in

s98
00:12:35.279 --> 00:12:47.920
in capital markets and I had like to like interact with few people from investment banking talking about valu valuation and just like leveraging this plug-in like running few example locally

s99
00:12:47.920 --> 00:12:57.519
was able like to bring me a little bit up to speed to understand um you know the product the the the problem behind um what the clients were talking about.

s100
00:12:57.519 --> 00:13:07.200
So let's this is kind of like a little bit like the initial presentation about like you know what's the effort on uh the financial services.

s101
00:13:07.200 --> 00:13:11.200
We are now going to move to the more like building part.

s102
00:13:11.200 --> 00:13:15.680
And as a final reminder please add your details here.

s103
00:13:15.680 --> 00:13:24.160
It's really really important and I'm gonna stay here for a minute making sure that everyone has it.

s104
00:13:24.160 --> 00:13:33.279
Yeah, maybe I think it would be good point um when we switch I will show you a little bit what we're doing hands on now that you can just confirm you can access

s105
00:13:33.279 --> 00:13:37.839
the site and then also be able to log in into shb work.

s106
00:13:37.839 --> 00:13:38.079
Yeah.

s107
00:13:38.079 --> 00:13:44.320
Anyone has having problem putting the data in like the email address?

s108
00:13:44.320 --> 00:13:46.880
I think it is a good sign.

s109
00:13:46.880 --> 00:13:51.360
So we can now switch to the fun part.

s110
00:13:51.360 --> 00:13:56.160
Um so this is actually the website we're going to use for the building part.

s111
00:13:56.160 --> 00:14:11.279
So please um um feel free to scan it and if the it ask for a code the code is there but it should be in theory that you write QR.

s112
00:14:11.279 --> 00:14:15.279
Yeah maybe we wait a little bit before we switch because we have a lot of time today.

s113
00:14:15.279 --> 00:14:25.680
Um maybe just make sure everyone can access it and then I can talk a little bit also about the content and I show you also some um technical details.

s114
00:14:25.680 --> 00:14:33.360
So what we'll do we we saw some of the nice out ofthe-box features in um shb work for financial service.

s115
00:14:33.360 --> 00:14:37.760
So you have data but also sort of customized use cases.

s116
00:14:37.760 --> 00:14:46.880
So the first part we thought what would be the benefit for a wealth advisor business with shbd for financial service.

s117
00:14:46.880 --> 00:14:56.560
So I think what we want to do and show here how can you actually make this process more efficient because at the end of the day you want to talk

s118
00:14:56.560 --> 00:14:58.480
talk to your advisor more.

s119
00:14:58.480 --> 00:15:15.760
Now, you don't want to uh have a team that's working, preparing, analyzing portfolios, figuring out um what happened, explaining it, preparing the meeting, um preparing um maybe also for the customer a deck that explains

s120
00:15:15.760 --> 00:15:20.959
why were those changes happening in the portfolio, what needs to be done to bring it back in balance.

s121
00:15:20.959 --> 00:15:23.680
So, the first part is really the use case.

s122
00:15:23.680 --> 00:15:33.360
You bring this product shed GBT for financial service to your wealth advisor teams and they can basically accelerate your workflows.

s123
00:15:33.360 --> 00:15:36.240
So it's an enduser tool in the team.

s124
00:15:36.240 --> 00:15:42.720
I mean often in my experience also have very different work workflows or different way of working.

s125
00:15:42.720 --> 00:15:47.199
So it's customizable and we walk you through this use case.

s126
00:15:47.199 --> 00:16:00.880
a customer there's a portfolio change happening and you want to go all the way preparing um the P&L or the portfolio attribution and then showing the customer what to do to bring it back into balance.

s127
00:16:00.880 --> 00:16:11.120
So that's the use case number one and um we provide a couple of prompts there and then I don't know who in the room is oh is a question

s128
00:16:21.360 --> 00:16:22.800
So yeah, we will fix that.

s129
00:16:22.800 --> 00:16:23.839
Um, that's normal.

s130
00:16:23.839 --> 00:16:25.839
Yeah.

s131
00:16:28.320 --> 00:16:30.560
Let's go.

s132
00:16:31.120 --> 00:16:33.120
So, uh, I'm going to work on it.

s133
00:16:33.120 --> 00:16:34.079
You can keep going.

s134
00:16:34.079 --> 00:16:36.720
So, like is the website like the the latest slide, right?

s135
00:16:36.720 --> 00:16:38.480
The shbt work you I think.

s136
00:16:38.480 --> 00:16:38.720
Yeah.

s137
00:16:38.720 --> 00:16:42.720
So, my colleague work on it and I will do a little bit more to explain it.

s138
00:16:42.720 --> 00:16:46.240
So, and maybe for the people in the room here, I don't know.

s139
00:16:46.240 --> 00:16:47.440
We have a couple people.

s140
00:16:47.440 --> 00:16:48.639
What's what's your background?

s141
00:16:48.639 --> 00:16:57.600
I mean, you're like a developer, builder, engineer, an AI exp who is like more in a technical role, developing, building something in in the room.

s142
00:16:57.600 --> 00:17:02.320
So, we have quite I can't see it fully, but yeah, it's a half maybe.

s143
00:17:02.320 --> 00:17:12.000
So I think for those people that are more hands-on and technical now and I want to explain also the second part of the workshop once we get your access

s144
00:17:12.000 --> 00:17:25.199
is actually in our service in the codec side the desktop solution the codeex would be also it can be used by end users business users knowledge workers but this in this case we want to do a little bit more

s145
00:17:25.199 --> 00:17:35.919
customization of it because let's assume you you deploy CHB financial service in your company successfully and it meets the business users need.

s146
00:17:35.919 --> 00:17:39.039
But then the question is how can I extend it?

s147
00:17:39.039 --> 00:17:41.840
How can I make my data coming with it?

s148
00:17:41.840 --> 00:17:44.559
How can I customize it?

s149
00:17:44.559 --> 00:17:50.720
So the second part of the workshop we thought there are two there basically two ways of doing it.

s150
00:17:50.720 --> 00:17:54.160
You could say let's build a custom application.

s151
00:17:54.160 --> 00:17:57.280
I get an open AI API key.

s152
00:17:57.280 --> 00:17:58.960
Let's do some web UI.

s153
00:17:58.960 --> 00:18:01.200
It's easy to do also with codex and web UI.

s154
00:18:01.200 --> 00:18:15.280
I deployed in my company and then yeah I have the typical software development life cycle or is this path A or path B is you basically think about it

s155
00:18:15.280 --> 00:18:29.039
oh my users have already in the enterprise shed GBT for work they already have codeex most of them why don't you keep sort of this need that I have for my business

s156
00:18:29.039 --> 00:18:44.400
inside this ecosystem So like we explore in the second part basically what would you do as an internal developer using codeex to create a skill a plug in and a plug-in extension.

s157
00:18:44.400 --> 00:18:53.200
I explain what it is in a bit to actually implement a full application that could sit inside the surface.

s158
00:18:53.200 --> 00:18:58.640
So basically with that with those things I will walk you through what those things are.

s159
00:18:58.640 --> 00:19:02.480
you can customize basically the workflows.

s160
00:19:02.480 --> 00:19:05.520
I mean, everyone I think probably knows now what a skill is.

s161
00:19:05.520 --> 00:19:12.480
Um, it's a markup that provides a little bit more customization for you and reusable.

s162
00:19:12.480 --> 00:19:19.600
But then once you actually want to connect your data, you need an MCP, you need some tools, you need to define all that.

s163
00:19:19.600 --> 00:19:21.679
And then that's where you have a plugin.

s164
00:19:21.679 --> 00:19:25.679
Now the plugin is sort of a little bit heavier and you can customize it around it.

s165
00:19:25.679 --> 00:19:40.559
So we create a plugin there and then with a plugin it can lifts on a desktop it can live also in the website and then what also we just recently announced is actually on dev day plug-in extensions.

s166
00:19:40.559 --> 00:19:50.640
So what we do at the end we also actually customize the UI a little bit so you can customize the surface to fit your use case.

s167
00:19:50.640 --> 00:20:02.640
So we have an advisor that would use shbdt for financial service in the web but the workflow you implement can actually also be sitting in the in the desktop application

s168
00:20:02.640 --> 00:20:04.720
and then so we exploring that

s169
00:20:04.720 --> 00:20:06.960
one second

s170
00:20:07.280 --> 00:20:09.120
no

s171
00:20:09.120 --> 00:20:09.919
okay

s172
00:20:09.919 --> 00:20:11.108
so we get there

s173
00:20:11.108 --> 00:20:11.200
[laughter]

s174
00:20:11.200 --> 00:20:20.799
so so I have to talk more and probably I can talk you through it so maybe can you do it um on a um I can show maybe the content also Yeah.

s175
00:20:20.799 --> 00:20:21.280
Yeah.

s176
00:20:21.280 --> 00:20:22.160
Yeah.

s177
00:20:22.160 --> 00:20:33.280
So, but just I think the interesting part is we did do in this workshop once we get access to it to you and don't worry if you can't get it out we the website is available later also.

s178
00:20:33.280 --> 00:20:48.720
It's really what would I do as an developer with a company an enterprise that already has this platforms inside the company and can I avoid a little bit my work or reduce my work to deploy applications?

s179
00:20:48.720 --> 00:20:53.280
No, that's a neat idea I think that becomes possible.

s180
00:20:53.280 --> 00:21:03.919
So, just a question uh about the uh built-in tools you mentioned like uh transcripts and filings and all that sort of uh other data that you have there and that you

s181
00:21:03.919 --> 00:21:05.600
I guess the harness has some awareness.

s182
00:21:05.600 --> 00:21:10.720
Is it just that the the harness and skills are there or is there something that you did to the model?

s183
00:21:10.720 --> 00:21:15.919
Just any kind of curious what what is makes it so much better.

s184
00:21:18.960 --> 00:21:21.039
The party mic is on as well.

s185
00:21:21.039 --> 00:21:21.520
Okay.

s186
00:21:21.520 --> 00:21:28.159
So, just for the people that have not the full link, um that's a workshop access code.

s187
00:21:28.159 --> 00:21:30.720
Can you get to it?

s188
00:21:37.440 --> 00:21:40.559
So, someone just confirm it works.

s189
00:21:40.559 --> 00:21:40.880
Okay.

s190
00:21:40.880 --> 00:21:42.154
You see, fix it.

s191
00:21:42.154 --> 00:21:44.154
[laughter]

s192
00:21:50.159 --> 00:21:54.960
So maybe also explaining um what we use under the hood for the workshop here.

s193
00:21:54.960 --> 00:22:05.760
So that's actually um the sites feature the capability to host a website and it's quite u easy and in this case it's a public veite actually.

s194
00:22:05.760 --> 00:22:12.159
So what's very neat um to create nice looking um functional UIs actually.

s195
00:22:12.159 --> 00:22:20.480
So just I just wait a little bit that everyone is um can get to the workshop setup.

s196
00:22:20.480 --> 00:22:30.720
We are also sending the emails for um accessing the the workspace as mentioned before it should come up.

s197
00:22:30.720 --> 00:22:30.960
Yeah.

s198
00:22:30.960 --> 00:22:43.280
So, while we do it, um I want to also I think we have little um like bottles or something for everyone here and um not for everyone, for the people that participate today.

s199
00:22:43.280 --> 00:22:53.679
And to make it a little bit fun um we actually once you're able to be um signed up in the room and I see we already have quite a few people there.

s200
00:22:53.679 --> 00:22:56.960
Um so this is also created with um the sites application.

s201
00:22:56.960 --> 00:23:04.480
So, everyone who was able to get into the um website um got 100,000 virtual money already.

s202
00:23:04.480 --> 00:23:12.799
So, to make it a little fun and of course we all know the market always moves up and down and it goes up uh maybe a little bit down now.

s203
00:23:12.799 --> 00:23:21.520
But if you complete the steps um you get additional funding and then um at the end of the day we want at the end of the workshop we want to see

s204
00:23:21.520 --> 00:23:22.640
um the top three people.

s205
00:23:22.640 --> 00:23:23.120
Yeah.

s206
00:23:23.120 --> 00:23:26.080
And um they get some bottles for sure.

s207
00:23:26.080 --> 00:23:30.480
So to make a little fun, it's a little bit uh to keep it interesting.

s208
00:23:30.480 --> 00:23:37.039
But um so for for the website now because I think we see people are in there now.

s209
00:23:37.039 --> 00:23:52.320
So what I want to um basically point out so the first thing we really need to have access to shed GBT for financial services and um that's basically the website here and you can see

s210
00:23:52.320 --> 00:23:57.760
you have financial data and you can see the plug-in um shed GBD for financial services.

s211
00:23:57.760 --> 00:24:09.039
So that's sort of the thing my colleague works on onboards you and when you have access to this website which sht for financial services and in the financial data

s212
00:24:09.039 --> 00:24:16.960
um that we showed in the presentation um you are free to go and follow the instructions under core capabilities.

s213
00:24:16.960 --> 00:24:30.640
No. So you can say setup once and um we have a little bit a zip file to have some data there for the customer example and etc. So it's really the wealth management use case.

s214
00:24:30.640 --> 00:24:41.200
How can I help my advisor to be faster and then I definitely would recommend um each of the subsections you can do on your own.

s215
00:24:41.200 --> 00:24:55.440
So if you're interested more portfolio questions, uh research or charting, um feel free to start one of those sections yourself and then um when you do it and have any question, please raise your hand also in the room.

s216
00:24:55.440 --> 00:24:57.520
We want to help you there.

s217
00:24:57.520 --> 00:25:02.799
Um and it shows really the highlights of the the main features of the product here.

s218
00:25:02.799 --> 00:25:13.760
And then for the people that are more interested in in sort of building um I think it is a question you have to have codecs on your desktop.

s219
00:25:13.760 --> 00:25:17.440
No. So so for the people that aren't you can install it.

s220
00:25:17.440 --> 00:25:21.919
Um I think that's the second part and I I think it feel free.

s221
00:25:21.919 --> 00:25:26.880
I mean if you rather want to envision that product more what is the product doing?

s222
00:25:26.880 --> 00:25:28.720
What are the out of the box capabilities?

s223
00:25:28.720 --> 00:25:43.520
stay in the core capabilities once you have access the website but with codeex um I think that's the idea what I was saying earlier how can we develop maybe with an AI native tool

s224
00:25:43.520 --> 00:26:01.919
in an enterprise applications now workflows etc so and then here um we walks you through the skill plugin um we're doing also an MCP use case here to augment this workflow we do on the first side

s225
00:26:01.919 --> 00:26:13.679
and then um I definitely want to highlight try out the plug-in extension and I think what's a plug-in extension I just want to explain it a little bit um I think we

s226
00:26:13.679 --> 00:26:25.919
I think people know now there are ways of extending um AI harnesses with skills and the plug-in is sort of little bit heavier and you see the structure it has skills

s227
00:26:25.919 --> 00:26:35.600
and has an MCP server It has a tool configuration and then you also have an UI probably to surface it in a way how you want it.

s228
00:26:35.600 --> 00:26:36.559
That's cool.

s229
00:26:36.559 --> 00:26:42.400
That's the one thing I mentioned but it's sort of what I want to say is also possible.

s230
00:26:42.400 --> 00:26:51.760
Now it's basically you have the shed GBT or the desktop the CEX application and you see this highlighted areas here.

s231
00:26:51.760 --> 00:27:01.200
You could actually customize it depending on the conversation that maybe your wealth management business has with codeex or chat GBT.

s232
00:27:01.200 --> 00:27:13.600
No. So you could say if I type in a certain thing and it's a certain file type suddenly it pops out a UI an extension that you control and can deploy

s233
00:27:13.600 --> 00:27:15.840
and make it more custom.

s234
00:27:15.840 --> 00:27:18.240
No. And there are some examples actually.

s235
00:27:18.240 --> 00:27:24.480
So yeah, I want to highlight those are the things um we could um if feel free to proceed there.

s236
00:27:24.480 --> 00:27:38.080
So the people that have already codex start with setup once and the people that still get access to the website um please start with um the setup once there.

s237
00:27:38.080 --> 00:27:41.840
Yeah, there was a question actually which is important.

s238
00:27:41.840 --> 00:27:52.400
So, so shedgbt for financial services lives exclusively in our shedgbt work web interface.

s239
00:27:52.400 --> 00:27:59.039
No, so that might could change but right now it's the only way to access that.

s240
00:27:59.039 --> 00:28:11.679
No. So you basically I mean you have sort of I mean I have seen enterprises that have the webs GBT work, they have the codec, they have both and even knowledge who users have both.

s241
00:28:11.679 --> 00:28:13.919
Um so you have to patch it together a little bit.

s242
00:28:13.919 --> 00:28:19.840
It might change in the future but right now it sits there but it doesn't it doesn't matter.

s243
00:28:19.840 --> 00:28:26.000
You can actually extend plugins on a desktop but also on the share GBT work site.

s244
00:28:26.000 --> 00:28:28.000
Now the plugins live there too.

s245
00:28:28.000 --> 00:28:47.520
So you have the options there and currently plug-in extensions the customization of the UI actually lives only on the desktop side now so you cannot do it yet in the web but it will be probably it's actually on our website explained that is coming up

s246
00:28:47.520 --> 00:29:02.559
um and so just be aware of it um it's all pretty new here and um the surfaces provide different capabilities just be aware of it and right now if you like a real builder and say hey

s247
00:29:02.559 --> 00:29:15.279
I want to get the financial data with a CLI now because Codex has also like CLI you want to put it in a pipeline you're not able to actually access the data now yeah

s248
00:29:15.279 --> 00:29:28.799
or it's basically not provided yet so as mentioned before here whilst you're in charge you have you can select the model we suggest you to go with either Astra

s249
00:29:28.799 --> 00:29:45.520
or 6.1 Soul and if you can click on the on the speed you can pick ultra fast that is going to make like your life much better so and yeah it's just the promps you follow the promps

s250
00:29:45.520 --> 00:29:57.440
and I mean you're learning by reading a little bit what the product actually provides for you so I mean the shibbity for financial services I think when you go on this market

s251
00:29:57.440 --> 00:30:11.039
the the historical market data and also research you see some of the key features now that sort of as Stefano said um we focus on to get that in the product more tuned

s252
00:30:11.039 --> 00:30:32.000
not fine-tuned but tuned to your use cases now so everyone has the email has logged into the correct workspace and can use like ch work fantastic went almost as as mostly as I would have.

s253
00:30:32.000 --> 00:30:32.559
Yeah.

s254
00:30:32.559 --> 00:30:39.919
So I mean I I maybe for the people that just want to listen I can do it in parallel also a couple things.

s255
00:30:39.919 --> 00:30:46.000
Um but any question I think we rather want to help you with questions so feel free.

s256
00:30:46.000 --> 00:30:47.200
Yeah

s257
00:30:47.200 --> 00:30:51.279
you mentioned the specific setup for financial aid.

s258
00:30:51.279 --> 00:30:59.360
uh you had mentioned uh with the specific setup for financial services has does it have another ontology layer like what else have you built?

s259
00:30:59.360 --> 00:31:07.520
I think there was a question asked earlier I didn't quite get the get the answer what specific things you had done beyond the model itself.

s260
00:31:07.919 --> 00:31:12.320
So I think like what I mentioned before is really like the data ontology.

s261
00:31:12.320 --> 00:31:19.200
So it's not just like connecting the data sources as if you know it would be like you know an API call.

s262
00:31:19.200 --> 00:31:32.960
We actually the data ontology has been changed in order to make it like extremely easy for the model to get extract the right information and also there is like kind of a context of like for each type of data source what's relevant what's not.

s263
00:31:32.960 --> 00:31:41.679
So if you think about 10ks or 10 QS like is not just like providing the file but is also explaining you know what is what within the file.

s264
00:31:41.679 --> 00:31:58.559
I think that's kind of the on the data point that's the kind of like key differentiator and also when when like not just from the integrated data sources but also from like the plug-in that we partner with like uh data provider that you can login into if you have access

s265
00:31:58.559 --> 00:31:59.760
that's the same concept.

s266
00:31:59.760 --> 00:32:05.440
So it's not just getting the data is actually getting data with the right contents in the right format.

s267
00:32:05.440 --> 00:32:05.840
Yeah.

s268
00:32:05.840 --> 00:32:18.000
And also when we talk about plugins um these plugins is not of course like at the end like if you look at the skills are kind of like some markdown files some templates but

s269
00:32:18.000 --> 00:32:28.320
there's been like there's actual expert knowledge we've been like partnering with financial institution for quite a long time and when we built this product we had like a selected number of partner

s270
00:32:28.320 --> 00:32:34.480
like top investment banking and so on so forth that we worked with in order to validate the quality of the product.

s271
00:32:34.480 --> 00:32:46.240
So it's is literally like the the the kind of knowledge that we build to put to transform like like the model into a product that can be used by like financial experts.

s272
00:32:46.240 --> 00:32:47.039
So

s273
00:32:47.039 --> 00:32:47.519
yeah.

s274
00:32:47.519 --> 00:32:47.840
Yeah.

s275
00:32:47.840 --> 00:32:59.200
So I I think um there I mean if I've just talked about it before if you do it yourself I mean you say I want to do the context layer the ontology

s276
00:32:59.200 --> 00:33:11.440
myself at scale and it's quite large data now we have to ingest now so you can do it in an scalable product with agents like doing public data but also have the data providers

s277
00:33:11.440 --> 00:33:15.919
and there's a lot of ways of making sure it finds the right data.

s278
00:33:15.919 --> 00:33:22.000
So I think um we actually have a couple um blog post how our data architecture is behind.

s279
00:33:22.000 --> 00:33:33.120
Now there's something interesting what services we use open source services but I think it shows you there's a lot of know how we build actually for billion users basically

s280
00:33:33.120 --> 00:33:45.279
like a context efficiently no and so that's the same thing that's under the hood powering this product no so I think it's maybe the engineering side now and the context engineering

s281
00:33:45.279 --> 00:33:54.640
um right now it's it's an early product and I think you will see how we involve it now but also lives on GBT work for now not via CLI

s282
00:33:54.640 --> 00:33:57.679
and yeah let's see how the world goes here

s283
00:33:57.679 --> 00:34:06.880
is there any way to access some of these more primitive data structures that you guys have developed like if you have ontologies or like a graph

s284
00:34:06.880 --> 00:34:20.079
so like no what's under the hood it's not really accessible uh because this is like a product meant a little bit more for the knowledge user not the pure engineer

s285
00:34:20.079 --> 00:34:38.159
but uh I think it's without enter too many the details is literally like each data set was like on boarded with the kind of like provider to really understand like what's the best way to serve it and also it's it's mostly like the understanding of the data set.

s286
00:34:38.159 --> 00:34:43.040
So it's not just like it's literally like you know what's the information like what's the relevant information.

s287
00:34:43.040 --> 00:34:57.200
So how you know like a simple example like you know when you on board market prices versus return like explaining all these detail on like you know if you want to do this operation if you want to extract this information

s288
00:34:57.200 --> 00:35:01.920
don't like how to avoid like basic uh misconception and so on so forth.

s289
00:35:01.920 --> 00:35:02.320
So

s290
00:35:02.320 --> 00:35:13.680
did you did you guys when you were talking about 10K and 10 Q sort of embedding did you guys have like custom embeddings and stuff is that is that at the back end of this?

s291
00:35:13.680 --> 00:35:25.599
No no no no it's it's literally like we create the data set and like pro pre-process the data in a way that the model can access easily like think about like a kind of a tool call

s292
00:35:25.599 --> 00:35:29.760
but like a really customized tool call for that data set.

s293
00:35:29.760 --> 00:35:32.880
So, no worries.

s294
00:35:32.880 --> 00:35:33.200
Yeah.

s295
00:35:33.200 --> 00:35:34.160
I mean, I'm just showing you.

s296
00:35:34.160 --> 00:35:35.680
I mean, I'm doing now the poms.

s297
00:35:35.680 --> 00:35:43.200
Now, if you're on the website, you just copy the pom, put it in the product and um and then also when you when you see now the icons here.

s298
00:35:43.200 --> 00:35:48.079
So, you see a little bit um is actually picked up the plugin.

s299
00:35:48.079 --> 00:35:48.880
So, think about it.

s300
00:35:48.880 --> 00:35:53.440
The plugin is a way that's hinted in the prompt.

s301
00:35:53.440 --> 00:35:57.440
Um I hinted in the prompt to use shbt for financial services.

s302
00:35:57.440 --> 00:36:09.520
So it follows a plug-in concept currently and then um you can see actually the financial data that's under the hood used and the internal tool calls basically in this plug-in happening.

s303
00:36:09.520 --> 00:36:19.280
So it's really um like a plug-in concept right now and then um yeah under the hood we provides a back end basically that's scalable.

s304
00:36:19.280 --> 00:36:23.119
So I would say like to give you like the really high level view of this product.

s305
00:36:23.119 --> 00:36:33.359
I think it's really cool because you know this is basically the interface that like every like knowledge user like the equity research analyst is going to use.

s306
00:36:33.359 --> 00:36:43.359
But what we are also going to do in the second part is like thinking about how can you add plugins like uh we all know that you know these data sources are not enough.

s307
00:36:43.359 --> 00:36:45.920
They're like you're probably gonna have like internal tools.

s308
00:36:45.920 --> 00:36:48.160
You're gonna have internal data set.

s309
00:36:48.160 --> 00:36:57.119
You can build your own plug-in with your own skill like especially as we mentioning before like you want to do the last layer of customization.

s310
00:36:57.119 --> 00:37:06.160
So as an engineer as like you can build this tool that then like you can use as a plug-in in charge for work.

s311
00:37:06.160 --> 00:37:10.480
So your end user your knowledge user is going to have this as a base.

s312
00:37:10.480 --> 00:37:22.240
So let's say this is like a really strong baseline on how to interact with data, how to build like specific financial analysis, how to produce like output like Excel, PowerPoint,

s313
00:37:22.240 --> 00:37:32.240
but you can add a customization layer that your company or like with like the specific tools with specific data set with specific logic is going to make it unique.

s314
00:37:32.240 --> 00:37:33.520
So

s315
00:37:33.520 --> 00:37:47.520
yeah, so just point out I mean you you do it yourself also um but I think it shows you the underlying tool calls happening in the sharp financial service you don't have the full access to the sort of the context there

s316
00:37:47.520 --> 00:38:01.520
but the second part is um this customizable UI so so there I mean we had just recently intelligent UI basically yeah now when you get in response you basically have a more

s317
00:38:01.520 --> 00:38:14.480
um tuned UI answer basically through the response now and this is one example here and that is something that the plug-in provides too and so it has this context when I ask questions about revenue

s318
00:38:14.480 --> 00:38:22.800
that you want to see it in a certain way and maybe as a table and the chart so those are the things that are addition so if you use a

s319
00:38:22.800 --> 00:38:32.480
the Astra model out of the box it would not do it necessarily automatically like that now you have to basically do it yourself so I just want to point All those little things

s320
00:38:32.480 --> 00:38:42.000
that make a difference um because it's already a little bit um not it's not fine-tuned model but it's sort of has context how to present data.

s321
00:38:42.000 --> 00:38:44.000
Yeah.

s322
00:38:44.621 --> 00:38:46.621
[snorts]

s323
00:38:54.480 --> 00:38:57.920
We check the pogus here.

s324
00:38:58.640 --> 00:38:58.960
Okay.

s325
00:38:58.960 --> 00:39:02.962
I see Mike, Zack, and Cool Deep

s326
00:39:02.962 --> 00:39:04.079
[laughter]

s327
00:39:04.079 --> 00:39:06.400
doing well.

s328
00:39:07.760 --> 00:39:14.160
Oh, and this one is also accessible by mobile in case uh you know you want to do use a different interface.

s329
00:39:14.160 --> 00:39:15.920
This is not just like desktop.

s330
00:39:15.920 --> 00:39:19.599
It's is also from like the mobile app works.

s331
00:39:19.599 --> 00:39:20.079
Yeah.

s332
00:39:20.079 --> 00:39:23.280
So, I think that's like the one thing I want to point out.

s333
00:39:23.280 --> 00:39:25.232
market prices would be awesome because

s334
00:39:25.232 --> 00:39:25.359
[laughter]

s335
00:39:25.359 --> 00:39:31.839
I know I'm I'm basically uh right now you have to be in in the surface here.

s336
00:39:31.839 --> 00:39:44.400
So if I ask the question now give me the historical market prices and um yeah I think that would be really neat to have an API but as is always like with data providers you have to be careful

s337
00:39:44.400 --> 00:39:55.839
uh how the licensing works here but yeah in this case um it's basically um like a very important thing you want to know the market prices in an easy way and there would be also an opportunity

s338
00:39:55.839 --> 00:40:08.880
if I built it in-house um to take that on because you all have those internal systems and then have an extension a plug-in sorry a plugin that sits maybe in chat GBT work

s339
00:40:08.880 --> 00:40:19.839
and then also on the desktop side and does exactly that now because you have already databases access to internal data and then yeah maybe use the same um visualizations

s340
00:40:19.839 --> 00:40:38.160
and then um provide an extension besides a product that's out of the Yeah, it's um it takes a little bit.

s341
00:40:38.160 --> 00:40:40.400
So even the little pumps here a couple minutes.

s342
00:40:40.400 --> 00:40:56.000
So if you impatient um and you want to go a little faster uh you know how to change the model or you basically the actually you know what you can show like if you click on this uh little thunder on the left

s343
00:40:56.000 --> 00:41:01.440
you can go oh doesn't oh doesn't

s344
00:41:01.839 --> 00:41:12.480
no oh only on 61 soul if you go on 61 soul I thought like ultra fast was allowed here.

s345
00:41:12.480 --> 00:41:12.880
Okay.

s346
00:41:12.880 --> 00:41:14.160
Fantastic.

s347
00:41:14.160 --> 00:41:14.720
Use that.

s348
00:41:14.720 --> 00:41:15.599
It's really beautiful.

s349
00:41:15.599 --> 00:41:16.560
Yeah.

s350
00:41:16.560 --> 00:41:16.880
Yeah.

s351
00:41:16.880 --> 00:41:21.823
I I mean the the benefit of working at Open you can use ultra fast.

s352
00:41:21.823 --> 00:41:23.823
[laughter]

s353
00:41:24.319 --> 00:41:29.520
No, it should be working also on the web.

s354
00:41:29.520 --> 00:41:31.520
Yeah.

s355
00:41:31.599 --> 00:41:32.240
Oh yeah.

s356
00:41:32.240 --> 00:41:34.319
I mean I I did the right model now.

s357
00:41:34.319 --> 00:41:35.119
So yeah.

s358
00:41:35.119 --> 00:41:37.200
So yeah, do ultra.

s359
00:41:37.200 --> 00:41:52.880
Um and also I think I'm just what I said the benefit of working in openi you can actually use ultra for your normal work a lot um and it's actually I think it's a day and night I think um if if this becomes sort of

s360
00:41:52.880 --> 00:42:03.760
more accessible for everyone um it's just I mean I mean how many people probably waited sometimes for some agent work and then you do some couple things in parallel

s361
00:42:03.760 --> 00:42:12.800
but now now I think yeah I I really appre appreciate it and um and I think um want to point out it's it's it's very helpful doing your work.

s362
00:42:12.800 --> 00:42:13.680
Yeah,

s363
00:42:13.680 --> 00:42:26.319
I would say like on this topic uh bringing a lot of like discussion that we had with clients I think like token efficiency and token cost is like I think for the last three months has been like probably the main priority.

s364
00:42:26.319 --> 00:42:38.880
So of course right now you know we are talking about Astra so that is like our top performing model but what you observe that is that of course if you're building something completely new a new codebase

s365
00:42:38.880 --> 00:42:52.640
you're doing research using the most powerful model gives you a hedge but when you're building something like more like a repeating uh repeatable task you can get top level top model performance

s366
00:42:52.640 --> 00:42:54.800
if you build the right context.

s367
00:42:54.800 --> 00:43:07.599
So if you build the right skill that explains more in details you know what you expect to the model uh then even leveraging model like Luna or even like 61

s368
00:43:07.599 --> 00:43:12.720
6.1 sold is actually is extremely good can really give you the top of of the performance.

s369
00:43:12.720 --> 00:43:18.079
So you don't have to go for like the most expensive model always the state-of-the-art model.

s370
00:43:18.079 --> 00:43:29.680
What we've seen as a common pattern is really like you can think about like building the skills in a proper way that is going to maximize [clears throat] the performance and this is like a common

s371
00:43:29.680 --> 00:43:41.680
or if you're working in more like multi- aent setup what we've seen is that the orchestrator needs to be like probably like the smartest of the agent but the single sub agent

s372
00:43:41.680 --> 00:43:52.160
don't need to be like the Astra you can switch to something like Terra something like Luna and still get like basically 95% of the overall uh result.

s373
00:43:52.160 --> 00:44:02.640
So of course to do all this measurement it's about the use cases about like kind of like building evals is about actually measuring the performance of what you're doing but

s374
00:44:02.640 --> 00:44:18.480
you know Astra is more like for what is like completely new research new codebase but also the other models especially even Luna that is our cheapest like I don't remember like out of my like

s375
00:44:18.480 --> 00:44:31.359
by heart like the the price but is is extremely cheap We've seen it like that with the right skills can be leveraged and produce extremely high quality content from like you know reports to analysis.

s376
00:44:31.359 --> 00:44:38.800
It's just that like you need to use the right skill and make sure that with the right skill the performance is actually matching your expectation.

s377
00:44:38.800 --> 00:44:45.839
But that's a great strategy that we've seen working across the board you know for actually optimizing the cost.

s378
00:44:45.839 --> 00:44:57.119
Especially if you think that like let's say that you are running like this like wealth management conser concurge on like your kind of like entire universe of client that

s379
00:44:57.119 --> 00:45:08.400
scales up pretty quickly both in terms of like cost and also in time time needed like using models like Luna that with the right set of skills can really like help you

s380
00:45:08.400 --> 00:45:11.040
uh mitigate this kind of problem.

s381
00:45:11.040 --> 00:45:11.839
Sorry.

s382
00:45:11.839 --> 00:45:12.079
Yeah.

s383
00:45:12.079 --> 00:45:26.480
No, I think yeah, I think the point about I mean the models are changing quickly and there are more models coming but yeah it's often I mean every time we we have a model also I think you have to figure out what's your effort level sort of now because the capabilities

s384
00:45:26.480 --> 00:45:42.000
sometimes sit differently now the price point versus um the capabilities so I think I'm I'm basically in my day right now I'm using a lot 6.1 soul and then I can have the luxury to use ultra of

s385
00:45:42.000 --> 00:45:55.280
And then but I think ever sometimes when I created the workshop actually the website um so this all was created of course with Astra and um even the more advanced

s386
00:45:55.280 --> 00:45:57.280
live room view I think.

s387
00:45:57.280 --> 00:46:00.560
Yeah, those are the things I handed off to Astra basically.

s388
00:46:00.560 --> 00:46:01.680
Um, I let it run.

s389
00:46:01.680 --> 00:46:03.280
I I was on Friday.

s390
00:46:03.280 --> 00:46:12.160
I let it run and it came back after that and then I just fine-tune it a little bit more the website with um soul because it was quicker and more efficient

s391
00:46:12.160 --> 00:46:18.319
and yeah, I think it's very important every time a new model comes figure out the capabilities, the effort level.

s392
00:46:18.319 --> 00:46:20.960
Um, and yeah, it's it's always changing.

s393
00:46:20.960 --> 00:46:23.440
I cannot say we can't

s394
00:46:23.440 --> 00:46:35.599
I would say like let's while you know while you go through um the various exercises like the question for you would be like any initial thoughts on like you know how are you using

s395
00:46:35.599 --> 00:46:49.760
this model right now this type of product and models within your like day-to-day and what do you think are the limitation or the potential anyone wants to like hot takes

s396
00:46:49.760 --> 00:46:54.000
on um using this type of product.

s397
00:46:54.960 --> 00:46:57.200
No one.

s398
00:46:58.560 --> 00:46:59.680
Yeah.

s399
00:46:59.680 --> 00:47:00.720
Okay.

s400
00:47:00.720 --> 00:47:03.119
We'll have later a little bit more questions.

s401
00:47:03.119 --> 00:47:09.760
But so, so I think when I when I saw the first time the product, I mean, we have it seen internally earlier.

s402
00:47:09.760 --> 00:47:11.597
Um, I said, "Okay, great.

s403
00:47:11.597 --> 00:47:12.160
[laughter]

s404
00:47:12.160 --> 00:47:13.599
But I need a little bit more."

s405
00:47:13.599 --> 00:47:15.760
So maybe we can pivot.

s406
00:47:15.760 --> 00:47:21.280
Um, this product is there and it's I think definitely geared to the knowledge worker.

s407
00:47:21.280 --> 00:47:23.599
But let's think about the builders.

s408
00:47:23.599 --> 00:47:36.160
So I have a little passion for it because my experience is definitely in the building side also and um um I I want to go a little bit um on on the plugins

s409
00:47:36.160 --> 00:47:36.880
again.

s410
00:47:36.880 --> 00:47:44.640
So so I think the plug-in architecture sort of the folder structure what's inside and plug in um is nicely described here.

s411
00:47:44.640 --> 00:47:56.319
So you see the skills MCP server which connects to something outside also via tools and then you have some UI components that could be rendered doing usage of the plug-in and JGBT

s412
00:47:56.319 --> 00:47:58.560
work for financial service is a plug-in.

s413
00:47:58.560 --> 00:48:09.920
So, and then um the the thing that you can do then with an architecture like that and that's maybe a segue now to go more on the builder side

s414
00:48:09.920 --> 00:48:20.240
is come up with an architecture that leverage all that in a in a in a way that sort of brings all those different pieces together.

s415
00:48:20.240 --> 00:48:26.559
So, this is sort of an architecture diagram um because you have to ask the question what's your surface.

s416
00:48:26.559 --> 00:48:29.760
So when I say service, we have shed GBT work.

s417
00:48:29.760 --> 00:48:31.359
We have which sits in the website.

s418
00:48:31.359 --> 00:48:36.640
We have codeex which sits locally or we have also the shed locally in the desktop.

s419
00:48:36.640 --> 00:48:50.480
Um and then as a builder I mean we all have now access usually with codecs but I see this actually coming to the knowledge workers more and then there will be probably more similarity in the UI for both um use cases in the future.

s420
00:48:50.480 --> 00:48:58.480
So you have it on your desktop and you can do a lot more in codeex because now you can install a local MCP actually.

s421
00:48:58.480 --> 00:49:14.079
Now you can create with code very with promps an MCP server that maybe goes to your internal security portfolio database or client database maybe or Salesforce could be also there used etc. But then

s422
00:49:14.079 --> 00:49:15.839
you build it there.

s423
00:49:15.839 --> 00:49:19.599
You have a pro you have a prototype usually as a developer there.

s424
00:49:19.599 --> 00:49:32.400
Now you basically connect to your local website sort of and you have a codex running and it connects to the MCP but um you you also want to use a plug-in infrastructure.

s425
00:49:32.400 --> 00:49:36.960
So the plug-in infrastructure sits in sht managed.

s426
00:49:36.960 --> 00:49:39.359
So I also maybe just point it out.

s427
00:49:39.359 --> 00:49:50.240
So if you want to get now your own plugins in shed GBT work um you can install a plug-in creator.

s428
00:49:53.440 --> 00:49:53.680
Yeah.

s429
00:49:53.680 --> 00:50:04.559
So you can install this plugin and when you install this plugin you can also install plugins in the JGBT workite.

s430
00:50:04.559 --> 00:50:10.559
So that's another way of customization it and the same thing you can do on the desktop side.

s431
00:50:10.559 --> 00:50:11.920
I I don't know if I confuse you.

s432
00:50:11.920 --> 00:50:19.040
It's but it's basically think about the surface and you have different avenues to customize it.

s433
00:50:19.040 --> 00:50:30.319
So just be aware the website plug-in creator is one option and then in your desktop of course you can install and plug in and if you have users you could

s434
00:50:30.319 --> 00:50:42.960
figure out an enterprise mechanism to deploy those plugins with your codeex deployment no so we have um like a management business everyone gets codex and also access to shbt work

s435
00:50:42.960 --> 00:50:52.240
you can figure out what's the folder structure with the plugins that's managed by the enterprise and then deploy through your existing mechanism and that that's a huge benefit.

s436
00:50:52.240 --> 00:51:02.720
I mean if I would build now an application myself I have to worry about how to log in how to make sure my access control is there.

s437
00:51:02.720 --> 00:51:08.960
So if you build an application on the side in enterprise it takes quite a time to get it out and approved.

s438
00:51:08.960 --> 00:51:17.040
But if you build it in an in surface that's already approved and running and managed you can hook in.

s439
00:51:17.040 --> 00:51:20.240
No. So that's sort of the plug here.

s440
00:51:20.240 --> 00:51:27.839
So yeah, we would then use some of the OpenAI plugin um plug-in infrastructure.

s441
00:51:27.839 --> 00:51:37.119
In this case, the SHBT financial services access to our context data where we have financial data etc. It still sits in the web service only.

s442
00:51:37.119 --> 00:51:39.200
You can't use the CLI.

s443
00:51:39.200 --> 00:51:47.119
And then the next thing is you would connect in your plug-in with an tool called your databases.

s444
00:51:47.119 --> 00:51:57.440
and that would be another MCP or you do it um like in a different way and you can use authentication etc and making sure it's sort of an enterprise

s445
00:51:57.440 --> 00:52:03.680
managed way of connecting it to the OpenAI hosted environment and all that can be done.

s446
00:52:03.680 --> 00:52:15.520
So this would be sort of one way of doing a little bit more customization but you stay in this thing and I I think I I can just I'm very excited about all those options now.

s447
00:52:15.520 --> 00:52:28.960
I mean I I love to build agents myself and host it, but to get it out approved in an enterprise is often very hard and if you can leverage something

s448
00:52:28.960 --> 00:52:38.240
and also um if you have it in codex there are some um guard rails policies that can actually prevent that a plugin can go outside of the sandbox.

s449
00:52:38.240 --> 00:52:48.880
Now the sandbox is sits in the codeex you have sort of controls there and then if I deploy code I just have to figure out an enterprise approved way

s450
00:52:48.880 --> 00:52:51.599
to get my plug-in deployed.

s451
00:52:51.599 --> 00:52:59.680
No, so that's sort of the I think it's a neat idea to explore and it can only get better in the future.

s452
00:52:59.680 --> 00:52:59.920
Yeah.

s453
00:52:59.920 --> 00:53:05.956
And I and I think that like when we talk about enterprise control like staying like within

s454
00:53:05.956 --> 00:53:06.000
[snorts]

s455
00:53:06.000 --> 00:53:13.920
you know this surface of chibbity work and like which plug-in can be deployed to which employee is like really common request.

s456
00:53:13.920 --> 00:53:23.520
So all the enterprise control here available so you can really control the access control who access the data who access the plug-in and so on so forth.

s457
00:53:23.520 --> 00:53:34.800
So this I feel I feel that like what Oliver was mentioning is really like here especially if charg work is your final surface for your users the like transforming

s458
00:53:34.800 --> 00:53:47.680
like an idea a prototype into a kind of like deployed plug-in that can actually be used by your end your end product um end user it's like much faster than what like we're used to.

s459
00:53:47.680 --> 00:53:49.280
We have a question.

s460
00:53:49.280 --> 00:53:52.960
So you have to have server.

s461
00:53:53.772 --> 00:53:55.680
[sighs]

s462
00:53:55.680 --> 00:53:57.920
So depends on the plug-in.

s463
00:53:57.920 --> 00:54:08.800
So it depends on the plug-in depends like what you need to do like a plugins could even be like a a set of skills that you know.

s464
00:54:14.640 --> 00:54:15.280
Yeah.

s465
00:54:15.280 --> 00:54:20.000
Then then you will have to create like a plugin like an MCPlay.

s466
00:54:23.040 --> 00:54:24.400
Exactly.

s467
00:54:24.400 --> 00:54:27.040
And also to manage the authentic authentication.

s468
00:54:27.040 --> 00:54:27.920
Yeah.

s469
00:54:27.920 --> 00:54:28.640
Yeah.

s470
00:54:28.640 --> 00:54:37.839
So so so maybe because now think about in this world where this is a platform which I think it is already what to do when in those terms.

s471
00:54:37.839 --> 00:54:41.200
So so first of all a skill.

s472
00:54:41.200 --> 00:54:43.599
So we have like some markup files.

s473
00:54:43.599 --> 00:54:56.319
So I would create in natural language um like maybe a couple markup files what to do and I think also it's like quite easy now to create a skill with some help from codeex also

s474
00:54:56.319 --> 00:55:09.200
and then it's sort of it's a reusable u method of reusing prompts I would say or knowledge no and and I think that's available currently on the codec side

s475
00:55:09.200 --> 00:55:17.280
so if you have a developer we're using skills usually to develop faster It's not actually available in the web side right now.

s476
00:55:17.280 --> 00:55:24.480
But then the interesting part is a plugin is what you said is with a skill I can't do really an enterprise integration.

s477
00:55:24.480 --> 00:55:31.119
I have some reusability um in the workflow that I can scale and then can share it across the company.

s478
00:55:31.119 --> 00:55:41.200
But the plugin is where oh now I have to authenticate systemto system or maybe I want to do on behalf of the user.

s479
00:55:41.200 --> 00:55:48.079
No, you have to sort of authenticate um for the user that's actually logged in into your codeex side.

s480
00:55:48.079 --> 00:55:56.559
So, so that's why the plugins will be a package where you have the capabilities maybe also some skills that you want to reuse.

s481
00:55:56.559 --> 00:56:07.920
But then to your point, I need some code, I need some logic, I need to have some way of getting my credentials for authentication, etc. So, that's a plug in here.

s482
00:56:07.920 --> 00:56:08.640
That's a plugin.

s483
00:56:08.640 --> 00:56:11.200
And you saw the folder structure before.

s484
00:56:11.200 --> 00:56:17.520
And then it uses MCP connections and there's also uh a UI component part of it.

s485
00:56:17.520 --> 00:56:28.960
There's an app UI standard like sort of a way of doing all UI components with a plug-in and then all those things can be zipped up basically in a in a package and that's a plug-in.

s486
00:56:28.960 --> 00:56:36.319
So it's really I would say the plug-in are most powerful because they contain the super set of everything.

s487
00:56:36.319 --> 00:56:50.000
And then what I mentioned before is um once you actually in the surface so if I go back to the plug-in extensions and I think I want to give you I think this is I think a good example

s488
00:56:50.000 --> 00:57:06.079
to understand it because I had also a hard time to understand what is an extension but here it's maybe more visually so I think this an example so I mean you see that's sort of the surface you're familiar with GBT but on the bottom left.

s489
00:57:06.079 --> 00:57:12.799
You see there's a way of adding your own icon, your own app hook basically.

s490
00:57:12.799 --> 00:57:18.799
And then once you click it, now this is all now managed by a plug-in that you can control.

s491
00:57:18.799 --> 00:57:23.760
So, and the beauty is it's just not just this little icon there you can customize.

s492
00:57:23.760 --> 00:57:29.200
You can customize uh different parts of the surface.

s493
00:57:29.200 --> 00:57:37.200
So we can customize the settings, we can customize the navigation and then also think about it.

s494
00:57:37.200 --> 00:57:44.880
I mean it's I if I go now in in in an enterprise and I have to go to five different websites to my job.

s495
00:57:44.880 --> 00:57:46.880
I don't want to do that actually.

s496
00:57:46.880 --> 00:57:58.559
I want to stay in my favorite tool which is AI powered has a very good frontier model hopefully um and I can have my conversation for my workflow and I don't have to leave it.

s497
00:57:58.559 --> 00:58:06.880
So what I would like to do is I have my I'm doing my work as a end user but then it pops up it knows a context.

s498
00:58:06.880 --> 00:58:18.720
Oh I have now something specialty around getting um maybe an ad u like a client meeting like a client meeting and have to prepare.

s499
00:58:18.720 --> 00:58:26.240
So it knows in the conversation and it will pop up my customized UI for that.

s500
00:58:26.240 --> 00:58:38.559
Now maybe the that your W business asked you about a certain thing you want to do quicker or you want to see a certain metric from an internal system for this customer profile.

s501
00:58:38.559 --> 00:58:42.960
That's what you can do now and you don't have to leave the surface.

s502
00:58:42.960 --> 00:58:55.920
So that's a I think I want to give you sort of an idea where those concepts sit and um if you take that away from the session that would be already awesome.

s503
00:58:55.920 --> 00:58:57.200
Yeah.

s504
00:58:57.200 --> 00:58:59.760
So any questions?

s505
00:59:01.119 --> 00:59:01.359
Yeah.

s506
00:59:01.359 --> 00:59:05.200
So maybe I switch in in my visualization I set up the codec side.

s507
00:59:05.200 --> 00:59:07.760
So would be a question also in the room.

s508
00:59:07.760 --> 00:59:14.400
Um, who has uh Codex running locally already?

s509
00:59:14.400 --> 00:59:16.400
Okay, I see.

s510
00:59:16.400 --> 00:59:18.640
Yeah, like a fair amount.

s511
00:59:18.640 --> 00:59:24.400
Yeah, codex of CHP app right now where it they're the same surface.

s512
00:59:27.839 --> 00:59:37.280
So, yeah.

s513
00:59:37.280 --> 00:59:44.400
So I will just play along a little bit because now it's more um building a plugin for me.

s514
00:59:59.599 --> 00:59:59.839
Yeah.

s515
00:59:59.839 --> 01:00:02.319
And everything becomes just a problem nowadays.

s516
01:00:02.319 --> 01:00:33.440
So, Yeah, I think it will take some time.

s517
01:00:33.440 --> 01:00:43.119
So even with Ultra because I think even if you use Ultra, you have to build code and compile or not compile but run it on locally.

s518
01:00:43.119 --> 01:00:44.720
Yeah.

s519
01:00:44.720 --> 01:00:51.200
So maybe that's also one thing um because I'm doing a lot like this of course for like workshops like that.

s520
01:00:51.200 --> 01:00:56.640
It's basically um sort of I'm on working on my desktop now.

s521
01:00:56.640 --> 01:01:04.160
So my I can have a couple more shed conversations open and do another project with maybe create another plugin or something like that.

s522
01:01:04.160 --> 01:01:15.119
But yeah, I think you really want to scale um basically where where basically the uh the agent runs or basically you want to have a cloud environment.

s523
01:01:15.119 --> 01:01:17.280
So that's another thing that's very useful.

s524
01:01:17.280 --> 01:01:23.520
Um I'm doing also for financial service a lot of um this appsac security scanning right now.

s525
01:01:23.520 --> 01:01:33.040
So it's a very common use case and yeah it takes even on a desktop you have occupied a large repo and you do like sort of a appseac inspection

s526
01:01:33.040 --> 01:01:49.359
it clocks your compute locally now so you want to move that sort of to a so remote site now so this case also if if you have a lot of tasks running um I think yeah we have a couple capabilities out of the box there but also you can customize it where actually the compute or the codeex

s527
01:01:49.359 --> 01:01:53.599
runs now in this case the codex runs locally which takes even if the model as fast.

s528
01:01:53.599 --> 01:01:57.599
It takes time or it's it takes as much time as my laptop has.

s529
01:01:57.599 --> 01:02:07.760
No. So, but yeah, it would be good to to have those infrastructure components also inside an enterprise where you basically can easily spin and spawn compute.

s530
01:02:07.760 --> 01:02:15.760
No, and then um you hand over task and um it's basically um you come back when the work is done.

s531
01:02:36.240 --> 01:02:38.240
Yeah.

s532
01:02:42.319 --> 01:02:44.319
Yeah.

s533
01:02:56.079 --> 01:02:57.359
Yeah.

s534
01:02:57.359 --> 01:03:01.599
So the there are a couple of couple of angles here.

s535
01:03:01.599 --> 01:03:15.039
Yes, the idea is really especially with plugins and like you know and the skills underneath is really like to create not not to have everyone with different skill but to actually build like kind of a marketplace

s536
01:03:15.039 --> 01:03:22.640
and share them across teams and that works also with what we were mentioning before that is more like the templates.

s537
01:03:22.640 --> 01:03:27.920
So artifact templates could be like a website, could be a PowerPoint presentation, it could be an Excel.

s538
01:03:27.920 --> 01:03:38.240
What we've seen is that like you want to have like we even within the plugins you can build some like add some kind of historical example as files and

s539
01:03:38.240 --> 01:03:41.680
then like the the plug-in will use that as a reference.

s540
01:03:41.680 --> 01:03:52.319
So let's say that we are in the context of like you know piece of research that needs to be created for this case like the your you know private

s541
01:03:52.319 --> 01:04:03.680
you know private wealth clients what you can do is actually maybe provide like few historical example and then like ask like within the skill to use that as a reference

s542
01:04:03.680 --> 01:04:09.839
then of course you can modify them but like the idea is really like to be as as reusable as possible.

s543
01:04:09.839 --> 01:04:22.240
So because like we we know that like format matters a lot and that's why like uh even if doesn't really like connects to the financial aspect of it like we we put a lot of effort like in artifact creation.

s544
01:04:22.240 --> 01:04:33.520
So if you probably seen like over the last six months the capabilities of our model to produce not just correct you know a spreadsheet but to actually format spreadsheet

s545
01:04:33.520 --> 01:04:40.559
in like highest possible level of like formatting that like is matching like investment banking standard really went up.

s546
01:04:40.559 --> 01:04:47.200
So we have like several benchmark we can show you but I think that like the best example is like try it yourself.

s547
01:04:47.200 --> 01:05:02.400
If you have like some template or spreadsheet presentation PDF that you want that you have and just try to say like okay with this new piece of analysis like build it in in this output you're going to see that probably

s548
01:05:02.400 --> 01:05:14.559
uh the output is going to be quite strong and then of course that's the last mile where you probably going to have like to customize the skill a little bit to say like hey these are couple of details that are not correct

s549
01:05:14.559 --> 01:05:19.119
but like if you mention it in the skill then at the next iteration are going to be correct.

s550
01:05:19.119 --> 01:05:26.000
So yes, the best part is like having sharable skills because you also want to kind of keep fine-tuning them.

s551
01:05:26.000 --> 01:05:37.039
So another like interesting point is that are the skill reusable between different type of model different families of model like the answer is that like if we move within charge

s552
01:05:37.039 --> 01:05:38.559
families yes.

s553
01:05:38.559 --> 01:05:53.520
So the idea is that probably you know if you move from like you know five to six so like a year a year uh kind of like jump you need some adjustment but in theory like between incremental

s554
01:05:53.520 --> 01:05:55.590
updates like the

s555
01:05:55.590 --> 01:05:56.160
[snorts]

s556
01:05:56.160 --> 01:06:07.119
what we've seen is that usually you need less the specification because the model are getting smarter and smarter but these skills tends to be quite reusable and like the cost of maintenance is

s557
01:06:07.119 --> 01:06:08.319
is kind of reasonable.

s558
01:06:08.319 --> 01:06:09.280
not to elevate it.

s559
01:06:09.280 --> 01:06:10.559
So um

s560
01:06:10.559 --> 01:06:25.200
yeah I think yeah maybe the skill can be a rabbit hole also actually so because yeah I think the skill reusability and then also maintain it um I have seen some customers go a little wild a lot of skills and then the teams

s561
01:06:25.200 --> 01:06:35.599
I mean it's difficult to share and you have to standardize it sort of so this is more like an organizational problem but I think if you want to like say you like an app builder

s562
01:06:35.599 --> 01:06:41.839
I would rather lean to plug in because you control also as a deployment and can package it a little bit more.

s563
01:06:41.839 --> 01:06:50.480
If I I mean we're sitting on a font office or like some quant I mean they have everyone has skills on their on the on their codeex basically.

s564
01:06:50.480 --> 01:06:59.680
So I think the question is then if two people in your team do similar things, they're creating a skill, how can you make sure it's like reusable, you know?

s565
01:06:59.680 --> 01:07:10.880
So I would say um if it's a sort of when you have a couple team members and they're figuring out actually do a similar task maybe with a little bit

s566
01:07:10.880 --> 01:07:17.599
optionality, I think it's highly recommended to actually remove some skills in your team because it's difficult to maintain.

s567
01:07:17.599 --> 01:07:22.480
And I I actually advised a customer recently that I think you have too many skills.

s568
01:07:22.480 --> 01:07:29.839
I mean maybe just try to come up with a good like like like a definition of the core capabilities.

s569
01:07:29.839 --> 01:07:40.720
If you do like an portfolio analyszis or you do some trading analyszis thing um keep that as a skill but not for everything in your workflow because it becomes kind of hard to maintain

s570
01:07:40.720 --> 01:07:50.400
and also the point what you had is um you still have to retest with the models now because um what we what is one benefit maybe if you work with us

s571
01:07:50.400 --> 01:08:02.079
is also you can access to the early preview models now so when you have a relationship for us because then you can see also what's impacting your business when you have certain promps and and also

s572
01:08:02.079 --> 01:08:13.440
and you can test it early now and make sure when it's live then you actually make sure that it's the same expected behavior now but yeah my advice is if you want to control it

s573
01:08:13.440 --> 01:08:27.679
stick to skills you have a builder that actually thought about it what should be like encapsulated encapsulated no and then if you see too many people have too many skills it can also slow down actually the processing

s574
01:08:27.679 --> 01:08:33.520
so Um it's not as like a straight answer there but you have to manage it.

s575
01:08:33.520 --> 01:08:35.199
It's my recommendation.

s576
01:08:35.199 --> 01:08:35.600
Yeah.

s577
01:08:35.600 --> 01:08:43.199
I think as a as a general trend that we've seen is that uh um so our team mostly focuses on API use cases.

s578
01:08:43.199 --> 01:08:48.319
So uh use cases that like where you know you're going to use response API.

s579
01:08:48.319 --> 01:08:53.199
So um eating like a different an like a fourth surface.

s580
01:08:53.199 --> 01:08:58.319
But I've been an open eye for like few months now.

s581
01:08:58.319 --> 01:09:07.920
And what you've seen already in like this kind of a short period is that lot of use cases that before were only able we were only able to package

s582
01:09:07.920 --> 01:09:13.520
if you were able if you were like actually writing code using API and API keys.

s583
01:09:13.520 --> 01:09:19.359
Now they can be perfectly recreated like um within uh with the plug-in.

s584
01:09:19.359 --> 01:09:24.960
So we to give an example guys is a little bit outside of the world management.

s585
01:09:24.960 --> 01:09:33.199
We built few months ago like an internal use case of like how to to create an advanced piece of financial research.

s586
01:09:33.199 --> 01:09:42.080
So what you can think about like as a deep research but specifically for finan for financial type of problems using uh the API.

s587
01:09:42.080 --> 01:09:45.920
So it was like kind of an orchestration of like multiple agents.

s588
01:09:45.920 --> 01:09:49.920
So the like the schematics was pretty simple like on a high level.

s589
01:09:49.920 --> 01:09:52.159
So you have what we call a planner.

s590
01:09:52.159 --> 01:10:04.480
So basically an agent that ingest the question and split the question between like sub sub question and for each sub question create basically a sub agent that is going to investigate

s591
01:10:04.480 --> 01:10:15.280
in an independent manner and then there's going to be kind of like what we call a critique agent a feedback agent that is basically taking all the output from these different independent

s592
01:10:15.280 --> 01:10:23.840
uh research putting them putting them together validating that there is no not conflict if there is conflict between different findings.

s593
01:10:23.840 --> 01:10:36.640
It uh creates automatically new sub agent to investigate this like this like kind of contradiction you know till a moment where like we the final report reaches like the quality and answers all the question

s594
01:10:36.640 --> 01:10:37.840
that you want to cover.

s595
01:10:37.840 --> 01:10:46.239
So really think about like an orchestration of sub agent and like kind of like a general orchestration that is really dynamic.

s596
01:10:46.239 --> 01:11:03.360
Right now you can do we like actually Dom recreated that same use case using a plug-in talking about the deep research and that is way more usable for clients because first of all you know the end user doesn't have to

s597
01:11:03.360 --> 01:11:18.080
leave the the interface to actually like you know enter like a different type of product that it can just stay in codeex it can actually visualize like what are the sub agents doing, what are investigating,

s598
01:11:18.080 --> 01:11:24.560
you can create even a specific uh interface for the user to like understand what's going on.

s599
01:11:24.560 --> 01:11:37.520
And with the current like multi- aent capabilities like you can really see everything in Codex, do everything in CEX and deploy you know instead of having like creating a new codebase

s600
01:11:37.520 --> 01:11:41.040
just create a plug-in and ship it into Codex.

s601
01:11:41.040 --> 01:11:41.520
So

s602
01:11:41.520 --> 01:11:52.159
why would why would you want to So I think we can ask the author.

s603
01:11:52.159 --> 01:11:52.880
Yeah.

s604
01:11:52.880 --> 01:11:53.120
Yeah.

s605
01:11:53.120 --> 01:11:55.600
And also there's a Matthew and Zack and Cohen.

s606
01:11:55.600 --> 01:11:56.800
I think you guys are leading here.

s607
01:11:56.800 --> 01:11:58.080
I mean it's a fun thing.

s608
01:11:58.080 --> 01:12:00.400
I mean let's see at the end.

s609
01:12:00.400 --> 01:12:02.560
So but it's a tough race here.

s610
01:12:02.560 --> 01:12:03.199
Uh yeah.

s611
01:12:03.199 --> 01:12:16.080
So when it comes to where and why and when we would use plug-in capabilities typically the the original example of that the fundamental deep deep research for equities uh was an application that I built about how to do deep parallel

s612
01:12:16.080 --> 01:12:19.600
validated work in terms of that space out of professional quality.

s613
01:12:19.600 --> 01:12:22.400
It ended up leveraging a significant amount of orchestration.

s614
01:12:22.400 --> 01:12:28.080
It meant like a lot of deterministic nondeterministic workloads within that process and it was a standalone application.

s615
01:12:28.080 --> 01:12:35.040
What started to make a lot of sense is that a lot of the people doing that work weren't going to standalone individual one-off applications to do that work.

s616
01:12:35.040 --> 01:12:39.760
They started consolidating where that work happened and that happened to be in codeex or jetp work.

s617
01:12:39.760 --> 01:12:54.560
So it makes a lot of sense to bring that surface area and then the choice of how that how we want to like rebuild that came down to the fact of can I leverage the capabilities of codeex including its sub aent capabilities including its ability to direct dynamic cap uh agent structures

s618
01:12:54.560 --> 01:13:04.400
to allow plugins and connected to different data sources instead of building these bespoke connections and it started to make a lot of sense to say hey this works as a plug-in specifically focused on

s619
01:13:04.400 --> 01:13:11.199
utilizing and reusing the LM capabilities to handle things like dynamic tool calling specifically that in line with how models do that.

s620
01:13:11.199 --> 01:13:13.520
So we call that ind in distribution tool calling.

s621
01:13:13.520 --> 01:13:17.120
It's one of the things that makes things faster and more accurate in terms of how those tools happen.

s622
01:13:17.120 --> 01:13:32.239
So all these things happen to line up to like make sense of like instead of building the standalone agent, build this functionality into a set of a of a workflow, a hard or soft workflow depending on the gates you want to build as and then use codecs as the main surface area, the product area for that.

s623
01:13:32.239 --> 01:13:44.159
So I was basically taking an old school style really manually tuned application that was using large language models in some nodes and traditional terministic workloads others and consolidating them into a codeex native process

s624
01:13:44.159 --> 01:13:50.400
which specifically for the use and ease of the user which is you don't have to go to this one hosted site.

s625
01:13:50.400 --> 01:13:52.000
It becomes locally generated.

s626
01:13:52.000 --> 01:13:54.159
It becomes customizable based on their needs.

s627
01:13:54.159 --> 01:13:57.040
They can basically add hey these are the data sources I want to use instead.

s628
01:13:57.040 --> 01:13:59.760
These are the types of gates so I have like quality bar gates.

s629
01:13:59.760 --> 01:14:08.880
So it started to become less of a black box in this predefined application and more this evolving process that made sense to live alongside the consumer.

s630
01:14:10.000 --> 01:14:10.960
Thanks D.

s631
01:14:10.960 --> 01:14:11.520
Yeah.

s632
01:14:11.520 --> 01:14:15.600
So so I just have to maybe add to the builders here also a little bit.

s633
01:14:15.600 --> 01:14:24.640
So I think under the hood I think we have the codex harness which is open source here basically.

s634
01:14:24.640 --> 01:14:37.760
So that's also a big competitive advantage I think because you have basically the full access to the source code which running basically what I showed today here and then you run it on the desktop

s635
01:14:37.760 --> 01:14:52.640
that's one option with a manage product and that's what you see here it's basically but then also you can take that and move it with an SDK to the server side or an appunner there's a way of doing that

s636
01:14:52.640 --> 01:14:57.520
programmatically And then of course you have the same builder capabilities.

s637
01:14:57.520 --> 01:15:10.560
You have you see you have the plugins that it's basically um becomes a way where you can plug hook into an existing system that definitely has a lot of traction now

s638
01:15:10.560 --> 01:15:18.560
and um gives you multiple choices when you have this research workflow what my colleague just mentioned maybe created very quickly.

s639
01:15:18.560 --> 01:15:23.520
Maybe a business tech or quant or a researcher has be able to do it.

s640
01:15:23.520 --> 01:15:27.199
But then you want to scale it to an application on the server side.

s641
01:15:27.199 --> 01:15:28.080
What do you do?

s642
01:15:28.080 --> 01:15:31.600
How can you take the work you did already and scale it to the team?

s643
01:15:31.600 --> 01:15:39.440
So you can take that and with the codex SDK you can hook it up in a real interface also if you need to.

s644
01:15:39.440 --> 01:15:42.719
No, but you don't it's just a way that you think about it.

s645
01:15:42.719 --> 01:15:49.040
It becomes very fluid where you can reuse your work and where you host it in the future.

s646
01:15:49.040 --> 01:15:51.040
Yeah.

s647
01:15:54.480 --> 01:15:56.640
So, let's quick check time.

s648
01:15:56.640 --> 01:15:59.440
We have 40 minutes.

s649
01:15:59.440 --> 01:15:59.679
Yeah.

s650
01:15:59.679 --> 01:16:01.760
Maybe 50 minutes at the end question or

s651
01:16:01.760 --> 01:16:11.120
Yeah, we're going to have like 50 minutes Q&A at the end, but feel free to interrupt at us any time.

s652
01:16:13.199 --> 01:16:22.159
So I would you know keep talking a little bit on Codex just to give a little bit more context on like why we think it's a great product and

s653
01:16:22.159 --> 01:16:35.280
as as we mentioned like the backbone of Codex so the uprunner and uh is a completely open source so even for more like to deploy this product within enterprise

s654
01:16:35.280 --> 01:16:46.480
it gives like a lot of transparency to see exactly you know the first party app of of course is proprietary but like the engine that is run running behind is fully open source.

s655
01:16:46.480 --> 01:16:50.800
So yeah it's not just transparent but it's also fully customizable.

s656
01:16:50.800 --> 01:16:56.719
So uh I think that's kind of like really interesting um angle.

s657
01:16:56.719 --> 01:17:06.159
So within enterprise you can actually choose to customize the uprunner that then is going to leverage the first party interface to your needs.

s658
01:17:06.159 --> 01:17:09.760
If you need any level of a customization it's there.

s659
01:17:09.760 --> 01:17:21.040
Additionally like what we think compared to you know other type of interfaces is literally that um our model are trained within with this harness.

s660
01:17:21.040 --> 01:17:31.280
So our model you know you know there's a lot of pre-training and post- trainining but when we think about post- trainining the postraining is is leveraging like our own harness.

s661
01:17:31.280 --> 01:17:44.800
So the our model basically are trained to work you know within like multiple harnesses but specifically on this one they're going to squeeze more performance for this kind of like trivial reason because

s662
01:17:44.800 --> 01:17:49.920
they like within the training phase they're already exposed to to this codebase.

s663
01:17:49.920 --> 01:18:05.600
So especially like recently if we think about all the multi- aent capabilities so what Dom was mentioning so more lots of like workflows the most advanced one are leveraging like sub agents and the basically

s664
01:18:05.600 --> 01:18:10.800
we the agent itself is taking care of like deciding when a sub agent is needed and when it's not.

s665
01:18:10.800 --> 01:18:24.000
Of course, if you move like uh within what we call the ultra, not ultra fast, the ultra mode, basically there like we are pushing the agent to leverage a little bit more these sub agent uh spudging capabilities.

s666
01:18:24.000 --> 01:18:36.800
But especially when working with sub agents working sub aent codeex provides like a a much better performance and on top of that what you have on on the background is a compaction mode.

s667
01:18:36.800 --> 01:18:51.199
So compaction mode is basically nothing but like a secondary model that is going to take the context of the current conversation and keep it compacting in order like to remove like the redundancies

s668
01:18:51.199 --> 01:18:56.800
keep that the um the size you know within the the limits and it's a really high quality model.

s669
01:18:56.800 --> 01:19:07.360
So what we've seen and this is a little bit more in what we've seen kind of like in the research type of task where like an extremely long context is needed.

s670
01:19:07.360 --> 01:19:14.800
Compaction mode really allowed us to solve problem where like much longer context uh were required.

s671
01:19:14.800 --> 01:19:21.679
So this basically allowed us to perform extremely well in what we call the long running task.

s672
01:19:21.679 --> 01:19:28.480
So um like we as a team we will frequently work with hedge funds and prop trading firms.

s673
01:19:28.480 --> 01:19:39.760
So as an example I that I run myself is that I took like one of the papers from like quant finance like you know brownie emotion and like option pricing

s674
01:19:39.760 --> 01:19:53.679
and I literally asked the model and we are not even talking about Astro it was like few months ago so it was like 5.5 to within codeex just replicate the the codebase of the mo of the paper so basically

s675
01:19:53.679 --> 01:20:01.360
replicate the codebase of the paper generate the numerical result and try to see if they match what this what the paper mentioned.

s676
01:20:01.360 --> 01:20:12.400
So there is like a task that you know it I think it ran for a couple of hours and where like it would have been impossible to run without compaction mode

s677
01:20:12.400 --> 01:20:22.080
and then for example after two hours I was kind of kind of impressed that the model was able to fully reproduce this like you know really quant heavy codebase

s678
01:20:22.080 --> 01:20:35.840
with like you know stoastic barium motion simulation calibration of models on top of that and that was like basically a task that without compaction mode is not achievable and compaction mode is already available

s679
01:20:35.840 --> 01:20:54.480
first also via API but you know if you use the codeex arness is available in the arness and you don't even have to think about it yeah so I mean I'm running in par I think the promps some of people have done it um yeah it's running um creating the basically the plug-in extension right now

s680
01:20:54.480 --> 01:21:05.120
um yeah it's basically um you see on the right side the way of in this example, my customized UI could be hooked up into the surface.

s681
01:21:05.120 --> 01:21:10.719
So, it's like a small example only, but yeah, it's it's pretty easy, of course, just to prompt.

s682
01:21:10.719 --> 01:21:14.159
Um, but yeah, it shows you some UI options there.

s683
01:21:14.159 --> 01:21:24.719
And then, um, if you look at our website, there are a lot more examples actually how our partners like Figma or Adobe have created a little bit more an app now where you can do a more complex thing.

s684
01:21:24.719 --> 01:21:31.199
But I I really think I mean I have said it many times is I think that's where probably the industry is moving to now.

s685
01:21:31.199 --> 01:21:41.360
You think about this more you have an AI platform and I want to plug in in that and then it's basically a marketplace and enterprises have their own inside

s686
01:21:41.360 --> 01:21:53.840
and I think that's my personal view of course um but I see a strong resemblance here and um yeah it's I think it's a builder you have to just take as an opportunity you know to take advantage of.

s687
01:21:53.840 --> 01:21:55.600
Yeah.

s688
01:21:55.600 --> 01:21:55.840
Yeah.

s689
01:21:55.840 --> 01:22:09.199
And I I would say that uh when when I joined OpenAI, a lot of people were still using Codex CLI as like you know mo most of people like you know coming from an engineering background like we're all used to CLI

s690
01:22:09.199 --> 01:22:20.560
and I've seen that the percentage of like extremely technical people moving to the first party app and almost never leaving codeex as an app is getting like bigger and bigger.

s691
01:22:20.560 --> 01:22:34.159
So uh as a former [clears throat] quant I still do around some experiment and right now like I used to have like my Jupyter notebook always open right now like is also already like within codex.

s692
01:22:34.159 --> 01:22:43.040
So what we really think and I think one of the message that we want to pass is really like even as a technical person codeex is an amazing surface

s693
01:22:43.040 --> 01:22:58.639
and it's getting like all the you know kind of surface within codeex that you need and then you can as a technical person create plugins and ship those plugins to your like

s694
01:22:58.639 --> 01:23:13.360
knowledge but non-technical users and the the way that you can actually put put this plug-in in production is can be quite stream streamlined thanks to enterprise control and you can even you can give like really powerful tool

s695
01:23:13.360 --> 01:23:26.719
to your end user in like weeks instead of month and I think that especially for financial services where like lot of data ingestion is needed every time there's maybe like a new data source

s696
01:23:26.719 --> 01:23:39.199
something that needs to be changed like a new analysis needs to be done this this new approach this new like AI based approach is going to like really help maximize the productivity

s697
01:23:39.199 --> 01:23:50.480
in a way that we've seen you know starting in 2025 for coding so that the famous like we all all heard like 10x 100x productivity on the engineering side

s698
01:23:50.480 --> 01:24:05.120
our objective here is like to bring that booster also to the knowledge workers so that's kind of I think the the vision behind the product and financial services thanks to this like mix of like artifacts,

s699
01:24:05.120 --> 01:24:11.600
data sources like and technical analysis feels like um kind of like an immediate match.

s700
01:24:11.600 --> 01:24:15.040
But while sorry

s701
01:24:15.040 --> 01:24:19.920
any question on all this topic that we covered so far

s702
01:24:23.679 --> 01:24:28.800
I know we are here to discuss financial services but do you see that happening in other industries?

s703
01:24:28.800 --> 01:24:33.120
Is it the financial services because the data is quite structured?

s704
01:24:33.120 --> 01:24:35.040
You build the ontology layer.

s705
01:24:35.040 --> 01:24:37.360
Is open AI doing that for healthcare?

s706
01:24:37.360 --> 01:24:39.199
Is something else happening?

s707
01:24:39.199 --> 01:24:39.840
So,

s708
01:24:39.840 --> 01:24:43.440
so the answer Okay, the answer is yes.

s709
01:24:43.440 --> 01:24:57.120
So, um we are working on multiple uh verticals and healthcare is definitely um uh one of the most priority the biggest priorities also in terms of like research.

s710
01:24:57.120 --> 01:25:01.199
Um I would say that the other one that we get exposed a lot is cyber security.

s711
01:25:01.199 --> 01:25:13.760
cyber security is change is dra drastically changing and in the space for example aside of like the more the more like cyber security specific model that we offer like our offering is called daybreak

s712
01:25:13.760 --> 01:25:25.199
um is also we also like basically build again a plug-in and what and again it's the same concept as as before the idea is that like your cyber security team

s713
01:25:25.199 --> 01:25:36.639
doesn't need to be an AI expert team or at least at the beginning so we bas Basically, you know, we provide you the model, we provide the the ARS, the open source ARS

s714
01:25:36.639 --> 01:25:48.560
uh to work with, but we also provide you like a plug-in for like, for example, for doing like assets uh static asset scanning that is going to is tuned on our model.

s715
01:25:48.560 --> 01:25:58.880
So, it's going to help you create that baseline in order that like you know out of the box you can like models, harness, plugins to run everything.

s716
01:25:58.880 --> 01:26:06.880
Actually we have like a library of skills that helps you to do everything from the scanning to the the final like review and patching.

s717
01:26:06.880 --> 01:26:08.664
But you know this is not

s718
01:26:08.664 --> 01:26:08.800
[laughter]

s719
01:26:08.800 --> 01:26:14.159
we're focusing more financial services but our focus is really like across industries.

s720
01:26:14.159 --> 01:26:27.440
Of course there are industry where like the benefit is like easy to measure finance cyber security healthcare research in general and others where like use cases like are not as broad.

s721
01:26:27.440 --> 01:26:29.440
So

s722
01:26:30.560 --> 01:26:35.920
so I think it's for the extending platform builder.

s723
01:26:35.920 --> 01:26:46.719
It's more I think our digital native businesses or like you would say sort of software providers that you usually look for new avenues to sort of distribute also their solutions now.

s724
01:26:46.719 --> 01:26:52.320
So I think our team works sort of with them a lot more.

s725
01:26:52.320 --> 01:26:56.480
I think what we are here because we in New York also met financial services.

s726
01:26:56.480 --> 01:27:04.000
I think it's what you say it's a structured data is it it lends to itself to use actually AI.

s727
01:27:04.000 --> 01:27:19.679
Now AI is sort of um with their data and I think it's kind of always hard to sort of have those data silos in those companies and then how can you actually leverage all this data in a context which is not there.

s728
01:27:19.679 --> 01:27:22.960
It's not ontologies etc. is not there.

s729
01:27:22.960 --> 01:27:39.840
So sometimes I think what I see as a strategy is sort of you have a it's it will take so much time to proper context layer across all the data stores but why don't we start from the bottom up too now so bottom up would be

s730
01:27:39.840 --> 01:27:49.440
you have sort of the shb work of codex and then you have teams that basically connecting bottom up to all the data sources and build sort of this context

s731
01:27:49.440 --> 01:27:54.560
over time while you modernize your data architecture over time.

s732
01:27:54.560 --> 01:28:00.560
So it's sort of the AI would be super efficient and I think we have the luxury internally.

s733
01:28:00.560 --> 01:28:01.920
We don't have those legacy things.

s734
01:28:01.920 --> 01:28:03.280
We have this context layer already.

s735
01:28:03.280 --> 01:28:08.639
It's it's amazing what you can do then as a workforce if it's actually unlocked.

s736
01:28:08.639 --> 01:28:15.679
But you have to bottom up and then top down will take more time for you actually to do it at scale.

s737
01:28:15.679 --> 01:28:17.679
Yeah.

s738
01:28:20.320 --> 01:28:21.440
So let's see.

s739
01:28:21.440 --> 01:28:32.560
um co Matthew and Zakia I think we have this my screen saver I want to talk one more thing I I think um it's probably to to mention also

s740
01:28:32.560 --> 01:28:47.280
um so when you install like sort of any extension inside a codeex no I think there's also you have to maybe you actually have some code running then on the desktop of someone else no

s741
01:28:47.280 --> 01:28:57.600
so or you have it in the website no so in our security teams we have a lot of discussion with them and um how can you make sure they don't do any harm

s742
01:28:57.600 --> 01:29:10.880
I mean there must be some sandbox and I think um maybe just sandboxing with codeex is a very important concept to understand because it's another benefit to go actually go this way

s743
01:29:10.880 --> 01:29:22.239
because let's say you have a plugin that has to do a tool call so you want to limit where the tool can go from an URL you You always have to have multiple

s744
01:29:22.239 --> 01:29:22.880
controls.

s745
01:29:22.880 --> 01:29:34.000
You want to start with your enterprise controls, networking security, endpoint inspections, but once you have a an application on a platform, you want to have also controls inside there.

s746
01:29:34.000 --> 01:29:46.320
So the codeex provides actually uh policies um the config files toml that are managed on an administrator site as an enterprise but also push down when you do a deployment

s747
01:29:46.320 --> 01:29:56.480
and in those sides you can decide okay I I don't want to get that this plugin can go to a certain website no and I want to make sure

s748
01:29:56.480 --> 01:30:07.600
certain things can be only done also on my workspace let's say a team develops inside an enterprise something but it could not go like to maybe like an sensitive project folder.

s749
01:30:07.600 --> 01:30:15.040
Now, so you can limit actually um with policies inside codeex an enterprise policy.

s750
01:30:15.040 --> 01:30:27.520
No, what is can be accessed locally via the codeex client and also what network what URLs um maybe what even plugins certain people can do.

s751
01:30:27.520 --> 01:30:30.239
And I think that's important to understand.

s752
01:30:30.239 --> 01:30:39.199
There's a sophisticated way to do that and and that's basically the configuration that you can see also here.

s753
01:30:39.199 --> 01:30:41.840
It's the custom config toml.

s754
01:30:41.840 --> 01:30:47.440
So this is something your enterprise administrator would define once on an enterprise level for different teams.

s755
01:30:47.440 --> 01:30:50.639
It pushes it down and it has all those things I just mentioned.

s756
01:30:50.639 --> 01:31:05.520
the security sandboxing because it will run whatever it's hosted inside the codeex client will run inside that you still have to protect tools etc yourself but I think that's already a benefit

s757
01:31:05.520 --> 01:31:18.800
um for this architecture and then the next thing is we all know that it's sort of difficult to I mean the security team goes all the way left we want to we want to

s758
01:31:18.800 --> 01:31:32.000
we we want to do um um like any sensitive operations but then the business user says hey it's so slow I always have to click please go to this plugin go to this website and and I think

s759
01:31:32.000 --> 01:31:46.480
um what also it's unique here we have a uh like the way of doing a guardian like we can actually um automate a little bit the decision- making when it's critical or not critical and this is sort of a way also to make sure

s760
01:31:46.480 --> 01:32:07.600
when you use a codex setup it's sort of balancing what security needs as a policy but also what's convenient or comfortable for the end user and this helps to make sure when it's sensitive and when it's not sensitive do I need to go to this workspace folder or not so I think that's very important also to highlight

s761
01:32:07.600 --> 01:32:14.000
because I think this is the thing you have to also explain your users now when you build plugins

s762
01:32:14.000 --> 01:32:23.679
no yeah like and basically every every big institution we work with they will customize uh the configuration for approval permission.

s763
01:32:23.679 --> 01:32:35.840
So that's like a really like usually when we talk about like deploying codex across the enterprise that's the first like point that we discuss about and like again there is a big flexibility there.

s764
01:32:35.840 --> 01:32:38.239
Any other question?

s765
01:32:38.239 --> 01:32:41.120
So, so maybe maybe almost the end.

s766
01:32:41.120 --> 01:32:44.320
I think should we

s767
01:32:44.320 --> 01:33:00.009
get our invite and then maybe ask some question if you uh co Matthew and Zack I think uh the leading as every good tech company we brought a little bit of merch.

s768
01:33:00.009 --> 01:33:02.009
[laughter]

s769
01:33:02.480 --> 01:33:15.074
So, like should we say that Co, Matthew, and Zach are the one getting an amazing water bottle branded OpenAI

s770
01:33:15.074 --> 01:33:17.074
[laughter]

s771
01:33:17.120 --> 01:33:19.360
picked this morning from the office.

s772
01:33:19.360 --> 01:33:20.314
So,

s773
01:33:20.314 --> 01:33:20.800
[laughter]

s774
01:33:20.800 --> 01:33:23.280
if you if you're still here, let's see.

s775
01:33:23.280 --> 01:33:29.920
Otherwise, so let's start like uh Coin.

s776
01:33:29.920 --> 01:33:31.440
I don't know how to pronounce it correctly.

s777
01:33:31.440 --> 01:33:31.679
Yeah.

s778
01:33:31.679 --> 01:33:32.080
Come up.

s779
01:33:32.080 --> 01:33:32.719
Come up here.

s780
01:33:32.719 --> 01:33:34.719
Yeah.

s781
01:33:36.239 --> 01:33:36.560
Yeah.

s782
01:33:36.560 --> 01:33:38.560
Thanks for answering the question.

s783
01:33:38.560 --> 01:33:39.760
It was not too hard.

s784
01:33:39.760 --> 01:33:41.679
So

s785
01:33:41.679 --> 01:33:41.920
yeah.

s786
01:33:41.920 --> 01:33:42.880
So maybe

s787
01:33:42.880 --> 01:33:44.800
tell me about you.

s788
01:33:44.800 --> 01:33:45.120
Yeah.

s789
01:33:45.120 --> 01:33:51.199
So uh my name is Um I'm working as a data engineer in the Netherlands.

s790
01:33:51.199 --> 01:33:56.080
Um so it's pretty interesting to see also the crowd over here.

s791
01:33:56.080 --> 01:33:58.560
I've been to AI engineer in London.

s792
01:33:58.560 --> 01:34:05.920
Um and looking forward also to meet a lot of people over here and to see what people are doing here especially in the financial industry.

s793
01:34:05.920 --> 01:34:07.760
Um that's about me.

s794
01:34:07.760 --> 01:34:08.080
Yeah.

s795
01:34:08.080 --> 01:34:11.199
And you are from company

s796
01:34:12.080 --> 01:34:12.639
data.

s797
01:34:12.639 --> 01:34:22.639
So we are consultant company uh yeah working worldwide and I'm working for the for the Dutch uh section.

s798
01:34:22.719 --> 01:34:24.880
Thank you.

s799
01:34:25.199 --> 01:34:27.920
Then we have Matthew.

s800
01:34:31.440 --> 01:34:31.679
Yeah.

s801
01:34:31.679 --> 01:34:32.000
Awesome.

s802
01:34:32.000 --> 01:34:32.239
Yeah.

s803
01:34:32.239 --> 01:34:33.675
You have to also say something now.

s804
01:34:33.675 --> 01:34:34.400
[laughter]

s805
01:34:34.400 --> 01:34:35.840
I hope it was fun.

s806
01:34:35.840 --> 01:34:36.960
Yeah.

s807
01:34:36.960 --> 01:34:37.440
Yeah.

s808
01:34:37.440 --> 01:34:38.080
Oh, thank you.

s809
01:34:38.080 --> 01:34:39.280
This was this was awesome.

s810
01:34:39.280 --> 01:34:40.639
Uh my name is Matt.

s811
01:34:40.639 --> 01:34:44.639
Uh company's x1wealth.com and we're building an AI native family office.

s812
01:34:44.639 --> 01:34:48.239
So, I'm doing a ton with codeex and this is very helpful.

s813
01:34:48.239 --> 01:34:49.280
Appreciate it.

s814
01:34:49.280 --> 01:34:51.520
I think an AI native sounds very good.

s815
01:34:51.520 --> 01:34:52.015
I think I

s816
01:34:52.015 --> 01:34:52.159
[laughter]

s817
01:34:52.159 --> 01:34:55.199
would invest in that.

s818
01:34:55.199 --> 01:34:55.520
Yeah.

s819
01:34:55.520 --> 01:34:55.760
Cool.

s820
01:34:55.760 --> 01:34:55.920
Yeah.

s821
01:34:55.920 --> 01:34:56.239
Thank you.

s822
01:34:56.239 --> 01:34:59.199
Then last one, Zach.

s823
01:34:59.360 --> 01:35:01.360
Okay.

s824
01:35:01.440 --> 01:35:02.159
Yeah.

s825
01:35:02.159 --> 01:35:04.320
Come up.

s826
01:35:05.600 --> 01:35:06.080
Hi.

s827
01:35:06.080 --> 01:35:06.960
So, I'm Zach.

s828
01:35:06.960 --> 01:35:14.800
I'm the founder of the Clockworks and I build apps that use AI to explore new things that we haven't really been able to do before.

s829
01:35:14.800 --> 01:35:19.920
I focus mostly on development topics, but you know, I'm always looking for new opportunities.

s830
01:35:19.920 --> 01:35:21.600
So,

s831
01:35:21.600 --> 01:35:22.239
thank you.

s832
01:35:22.239 --> 01:35:23.440
I have a Yeah, good.

s833
01:35:23.440 --> 01:35:24.400
Yeah, I have an idea.

s834
01:35:24.400 --> 01:35:26.731
So I think the homework is for you by

s835
01:35:26.731 --> 01:35:27.360
[laughter]

s836
01:35:27.360 --> 01:35:32.080
to build a plug-in extension and put it on the marketplace or plug in.

s837
01:35:32.080 --> 01:35:33.040
Yeah.

s838
01:35:33.040 --> 01:35:36.639
Do you know anyone that can help us push it through?

s839
01:35:37.920 --> 01:35:39.520
Oh, you already there.

s840
01:35:39.520 --> 01:35:41.520
So

s841
01:35:44.560 --> 01:35:47.120
yeah, he's he's he gets it.

s842
01:35:47.120 --> 01:35:47.360
Yeah.

s843
01:35:47.360 --> 01:35:48.719
So yeah, this is also another thing.

s844
01:35:48.719 --> 01:35:59.679
So I mean with all the um marketplaces now so there's a little bit you have to submit it because it can go also to external customers but I think

s845
01:35:59.679 --> 01:36:11.280
if if you see it like this this is I think a signal for people I mean this is maybe B to B2B outside marketplace now but also don't forget the internal

s846
01:36:11.280 --> 01:36:22.880
enterprise thing is the same concept yeah but I think it's really a signal where you have to leverage the distribution channels that are sort of emerging.

s847
01:36:22.880 --> 01:36:29.199
So the emerging distribution channels and and we have seen it all with the app store and all how viral it is now.

s848
01:36:29.199 --> 01:36:41.199
So basically think it is another way and of course the SM review if it's public now so and we just have to make sure it's smooth and safe and secure and it fits the right category now but yeah it's a

s849
01:36:41.199 --> 01:36:43.366
I think yeah just it's a signal that's

s850
01:36:43.366 --> 01:36:43.520
[laughter]

s851
01:36:43.520 --> 01:36:44.239
what I want to say.

s852
01:36:44.239 --> 01:36:45.520
Yeah.

s853
01:36:45.520 --> 01:36:45.679
Yeah.

s854
01:36:45.679 --> 01:36:56.480
I know that's I think is that's also the beauty of an open source code base and I would say that like again today we cover more like you know our product surface so codeex

s855
01:36:56.480 --> 01:37:09.520
charg work but we do also a lot of interaction with clients when working on bespoke use cases and specifically like on uh API use cases like of course that's a little bit more complicated

s856
01:37:09.520 --> 01:37:25.360
and there's a lot of topics that we can discuss but like a lot of the engagement that we do is actually understanding uh the use cases helping like you know bring best breast practices because we are the first user internal user of our product

s857
01:37:25.360 --> 01:37:27.520
we do speak with a lot of people in the industry.

s858
01:37:27.520 --> 01:37:34.719
So for example as we like mentioned before this magic word evals I think is like the most important one.

s859
01:37:34.719 --> 01:37:47.360
So literally like helping clients you know understanding you know how to design the right eval for the right problem and the question is always like you know we talk about you know boosting productivity

s860
01:37:47.360 --> 01:37:58.880
but especially when you want to go to production you need to have a way to measure like actually the performance of your product and it's you know again when we're talking about like specific

s861
01:37:58.880 --> 01:38:10.239
you know chat thread like there you don't need any valves but when you build your agents then it's needed And for example you know best practices there on like how to build evals

s862
01:38:10.239 --> 01:38:22.320
understanding like out of an evals data set the performance why the performance is not meeting the requirement you know what the kind of like you know probably like it's not about the model it's about

s863
01:38:22.320 --> 01:38:34.719
how to improve the arness the prompting the connection with the tools and sometimes it's also like a model limitation so we also connect back to research to understand like where our model are hitting the limits where they're not

s864
01:38:34.719 --> 01:38:45.199
behaving even if in the perfect environment as expected and ideally you know thanks to your feedback at the next model iteration that is is kind of fixed or improved.

s865
01:38:45.199 --> 01:38:46.719
We have a question sorry.

s866
01:38:46.719 --> 01:38:47.119
Yes.

s867
01:38:47.119 --> 01:38:58.320
So on the topic of evals, I was quite curious to see where what your perspective is in terms of the market's maturity in terms of adoption of evals and understanding of the fundamentals

s868
01:38:58.320 --> 01:39:04.320
and even going into advanced topics or more advanced topics like auto research type of workflows.

s869
01:39:04.320 --> 01:39:06.400
Okay, so this is a pretty big question.

s870
01:39:06.400 --> 01:39:08.000
Let's start with the evals.

s871
01:39:08.000 --> 01:39:17.520
Uh I would say that evils are the most usually the blocker because building evils and this is my personal perspective.

s872
01:39:17.520 --> 01:39:20.880
So happy to uh hear like different thoughts.

s873
01:39:20.880 --> 01:39:27.679
The complexity comes from the fact that evals usually incorporates a lot of like business business knowledge.

s874
01:39:27.679 --> 01:39:34.400
So coming from uh SMMES so subject matter experts and actually they are built by engineers.

s875
01:39:34.400 --> 01:39:45.199
So uh it's having you know the intersection of like a team that has like subject matter experts and engineers to build evals is not trivial.

s876
01:39:45.199 --> 01:39:59.360
So usually I always and again here there is a bias because in my previous job I was like what is called a quantitative researcher like for me whenever you build an agent evals is the most important piece because

s877
01:39:59.360 --> 01:40:10.880
you cannot do optimization if you don't do not measure the performance and it's not just you know like the one line of like oh I'm you know I have my evils and it's like 70%

s878
01:40:10.880 --> 01:40:21.520
there but it's also understanding the underlying metrics of like you you know first of all like you know where are we performing well where are we are not performing well and so on so forth so

s879
01:40:21.520 --> 01:40:33.520
I think like evas is like a really hard part but it's not because it's a difficult technology is because like you need to basically put like expert like expert expertise

s880
01:40:33.520 --> 01:40:46.960
within like kind of like an engineering created data set so then best practices there like to keep it short I think like always like start simple Don't don't start with the LLM as a judge.

s881
01:40:46.960 --> 01:40:58.800
Start simple and then add complexity because otherwise like you'd never understand if like the quality of your evals is not good enough or the quality of like the agent is not good enough.

s882
01:40:58.800 --> 01:41:18.639
So that's for me like the like kind of like best practice and one kind of cheat code that we always suggest when you use LLM as an agent don't ask to rank one to five to score one to five always ask yes or no it's much better try to decompose that one to five

s883
01:41:18.639 --> 01:41:37.119
into a true or false and you know like right now like doing some promotion we also release like the decision API so you can have literally a model that you know it's quite good is based on out of Luna and it's going to tell you like you know you know quickly super quickly yes or no and the

s884
01:41:37.119 --> 01:41:49.679
probability and the kind of the confidence around that answer so I see that like a lot of possible application for a decision API uh could be actually within the evils so that's the first part second part is auto research

s885
01:41:49.679 --> 01:42:03.679
it is like a really big topic it is um something that uh I think really started significant this here and uh what I've seen is that like and there are benchmarks so like

s886
01:42:03.679 --> 01:42:17.280
we reach really good coding capabilities already last year but like our model specifically really like improved on the mathematical capabilities more recently especially with like Astra and 5.6 six.

s887
01:42:17.280 --> 01:42:27.440
So I think like especially from a kind of quant perspective like the intersection of coding and math research really allows you for this like more auto research task.

s888
01:42:27.440 --> 01:42:35.520
So I would suggest like for you for example like one of the auto research task was what I mentioned before replicating a paper.

s889
01:42:35.520 --> 01:42:47.600
So of course like there are different level of like complexity there but what we have especially if you think about one specific metric that you want to improve to

s890
01:42:47.600 --> 01:42:53.360
hill climb as we say in AI terms like we have what is called the goal mode in codeex.

s891
01:42:53.360 --> 01:43:03.840
So basically a goal mode is basic you have to specify you know what's the target and it's better to have like kind of a target that is like a number a score.

s892
01:43:03.840 --> 01:43:18.880
Uh one example to keep it simple is you know like try you have this agent you have this EVAS data set this benchmark try to calibrate the prompt in order to maximize the performance of the

s893
01:43:18.880 --> 01:43:28.639
of the agent and what happens there is that the agent looks at the target and then keeps iterating on like new ideas and new hypothesis.

s894
01:43:28.639 --> 01:43:41.199
Of course, this is kind of like the the kind of like most s simplistic kind of like auto research mode, but like some of the result that we published also like in terms of mathematics early this year

s895
01:43:41.199 --> 01:43:49.440
were you were created using goal mode in uh in codeex actually for one of those we also published the prompt.

s896
01:43:49.440 --> 01:44:00.080
So I think like if you're interested in the space reading actually the public prompt that we gave to Codex in order to prove that theorem it was a conjecture

s897
01:44:00.080 --> 01:44:04.880
uh there is a different sorry for the mathematician in the room.

s898
01:44:04.880 --> 01:44:16.320
Um I think that's the most interesting like really interesting piece of content because like shows you a little bit more the logic of like this because like multi- aent and sub aent

s899
01:44:16.320 --> 01:44:16.960
approaches.

s900
01:44:16.960 --> 01:44:28.960
So um like for example like especially when you do research task you're you're probably going to use ultra uh mode then you're going to really see that the agent is going to leverage like

s901
01:44:28.960 --> 01:44:36.400
the sub agent automatically to actually like run some independent piece of research in parallel then like merge them and iterate.

s902
01:44:36.400 --> 01:44:48.080
So I would say that you know these are the two really good entry point for evils and auto research and I don't want to understate like even if it's simple

s903
01:44:48.080 --> 01:44:52.080
like we actually use goal mode to achieve like significant results.

s904
01:44:52.080 --> 01:44:55.679
So um it's quite powerful.

s905
01:44:56.400 --> 01:44:56.880
Yeah.

s906
01:44:56.880 --> 01:44:58.159
Plus one on goal mode.

s907
01:44:58.159 --> 01:44:58.320
Yeah.

s908
01:44:58.320 --> 01:45:07.360
I think I think yeah just half a year ago you had to worry about the prompt and you had to worry about how to orchestrate the agents and all this stuff.

s909
01:45:07.360 --> 01:45:21.119
So now I you move back now you you really have to think about what should the agent do as a goal and I think that's very powerful because I mean the models will be more capable to actually do all that than

s910
01:45:21.119 --> 01:45:34.560
automatically what we used to do what we do still partially manually and I think I highly recommend plus one on the goal mode and also I think today we had chativity financial services

s911
01:45:34.560 --> 01:45:52.320
plugin basically shown but I think our team really I mean also my colleague Dom we're focusing with financial service customers or any fintech etc to actually implement it at scale and also those difficult problems and there's a lot of nuances in the evaluations

s912
01:45:52.320 --> 01:46:02.800
that our team brings so we're not going away we just want to make sure you leverage our models and the platform at the scale I mean this was more simpler today but approach us

s913
01:46:02.800 --> 01:46:06.000
if you have any use cases happy to work with you

s914
01:46:06.000 --> 01:46:11.199
yeah know I really want to stress the fact that it's kind of like a twod directional conversation.

s915
01:46:11.199 --> 01:46:23.840
So, you know, if you already have a relationship with OpenAI, please, you know, hit your account director like feedbacks are more than welcome and also it's part of our job really making sure that,

s916
01:46:23.840 --> 01:46:29.280
you know, it's not just nice presentation, but actually it's going to work at your own companies.

s917
01:46:29.280 --> 01:46:33.920
So, um that's really like, you know, we are here.

s918
01:46:33.920 --> 01:46:36.719
But any other question?

s919
01:46:37.119 --> 01:46:37.920
Okay.

s920
01:46:37.920 --> 01:46:41.040
One of the questions I have.

s921
01:46:41.760 --> 01:46:42.320
Thank you.

s922
01:46:42.320 --> 01:46:55.280
Uh, one of the questions I have is uh during the presentation there was a lot of um uh readonly skills and plugin curious for uh the clients or any skills that you've seen that actually writes back into

s923
01:46:55.280 --> 01:47:01.440
the system or like system of records that uh the team has been building with.

s924
01:47:01.440 --> 01:47:03.280
That's a good question.

s925
01:47:03.280 --> 01:47:06.400
I think Dom has a has a even better answer.

s926
01:47:06.400 --> 01:47:06.639
Yeah.

s927
01:47:06.639 --> 01:47:07.760
Hey, I'll come up here.

s928
01:47:07.760 --> 01:47:08.480
Uh, yeah.

s929
01:47:08.480 --> 01:47:17.360
So, when it comes to plug-in structures, uh, and how they can and should be interacting with larger systems, there's a couple ways to do that.

s930
01:47:17.360 --> 01:47:20.239
Uh, and it depends on like what you're trying to build.

s931
01:47:20.239 --> 01:47:32.480
So if I'm building a fullblown like plug-in extension, I'm really just orchestr I'm using the codeex as the orchestration system, but those tools and processes can absolutely store to remote locations

s932
01:47:32.480 --> 01:47:33.679
and to interact with those systems.

s933
01:47:33.679 --> 01:47:36.719
The easiest way to do that is to interact with the local file system.

s934
01:47:36.719 --> 01:47:49.040
So if you tell it, hey, within this project where I'm scoping this out, it can it can write back to these processes or you give it instructions for the model to actually like the output of the tool is this and the output of the or the instruction of a skill set is to say, hey, persist this in

s935
01:47:49.040 --> 01:47:50.000
this location.

s936
01:47:50.000 --> 01:47:53.280
Uh you'll see a lot of persistent structuring uh within codeex.

s937
01:47:53.280 --> 01:48:00.000
It's gotten really really good at handling long context by effectively utilizing its local file system as like that point of record.

s938
01:48:00.000 --> 01:48:14.159
That being said, once you move from a traditional like readonly plugin or skill set into a full-blown extension oper operation, like I tell it to go talk to the central server, I tell it to go spin up a database or at least connect to the specific database as part of a tool.

s939
01:48:14.159 --> 01:48:23.280
So like the a lot of the use cases that we do, we just did one around proxy voting, being able to actually go define your proxy voting systems, what it actually had what it sent to.

s940
01:48:23.280 --> 01:48:35.520
was like even though the plugin in the surface area was codeex the actual source of record was this app server that was connected via one of the plug-in extensions it says hey write this report write the findings write the date write the the sources

s941
01:48:35.520 --> 01:48:44.560
back to this end of it database and that database feeds the like their production application system which is outside of codeex so operating system from a user perspective is codeex

s942
01:48:44.560 --> 01:48:56.480
uh operating system from an administration perspective and from the legal review perspective was outside of codeex so the extensions MCPS allow you to be flexible where the those different operating systems are and where the systems record are.

s943
01:48:56.480 --> 01:49:08.800
Uh and you can either offload that to the codeex and keep things local or you can build the tools and build those avenues into particular like MCP and OOTH related you know permission sets by pretty easily dropping an MCP where you need to.

s944
01:49:08.800 --> 01:49:20.560
Yeah, I would say like another like common use there is really like to especially when you're doing like specific product um processes that needs to be kind of like audited

s945
01:49:20.560 --> 01:49:39.119
like he was mentioning proxy voting having like kind of like a standardized like what we frequently build like kind of a standardized like logs that will allow like you know a technical user to understand you know not just the end of the analysis but how every decision every like piece of was done that's for example Like this

s946
01:49:39.119 --> 01:49:53.600
extended log is something that we tend to store frequently and especially you know in a highly regulated framework like um industry like financial services being able to back trace every decision to the data source and the logic behind

s947
01:49:53.600 --> 01:50:07.760
it was kind of like kind of a crucial and they like usually for example when we produce this artifact as output we also store like logs that are fully auditable for this reason as part of like the process and the plug-in.

s948
01:50:07.760 --> 01:50:10.159
One last question maybe.

s949
01:50:10.159 --> 01:50:12.159
I think we almost at the end now or

s950
01:50:12.159 --> 01:50:14.320
Yeah, we are like five minutes out.

s951
01:50:14.320 --> 01:50:15.600
Yeah, thank you.

s952
01:50:15.600 --> 01:50:17.360
Thank you for spending time with us now.

s953
01:50:17.360 --> 01:50:24.560
And then please reach out for us and thanks have a good good uh conference and thanks for coming.

s954
01:50:24.560 --> 01:50:27.199
Thank you so much guys.