Build Your Own Finance Agent Inside ChatGPT - Stefano Fabbri & Oliver Steffmann, OpenAI

AI Engineer · 110 min · 954 sentences · from YouTube's caption track

Each timecode opens YouTube at the start of that sentence. Line anchors (#s42) are the cue ids in the WebVTT, and every line carries its start and end seconds. All transcripts has every talk, and the whole corpus as one file.

1. 00:01[music]
2. 00:12So hello everyone.
3. 00:14Uh glad that you know there's quite a crowd here even if it's holiday.
4. 00:19It's Monday morning.
5. 00:20It's 9:00 a.m. So I'm Stephan Fabri.
6. 00:22I'm part of the open AAI the applied AI engineering team alongside Oliver and also we have in a crowd another part of uh our team Dom.
7. 00:34So feel free during the session you know if you don't want to ask ask question we also have a third man that you can leverage.
8. 00:43So today's session is going to be a two-hour workshop so it's going to be interactive you are going to use chib work and codeex more into the detail later
9. 00:54and we really want to kind of show you the capabilities um of like our product within the financial services space.
10. 01:02How can you merge, you know, financial data, you know, LLM, AI, intelligence, and the right output to put everything together to help like your day-to-day business.
11. 01:15So, today we're going to focus more on a wealth management case given like you know, we know in the crowd there are a lot of people coming from asset manager, investment banks.
12. 01:25So, we thought that um this could be like an interesting case.
13. 01:29So to give you a little bit of a background like uh our team focus mostly on interacting with clients.
14. 01:36So we spend all our days working with like some of you guys helping really to understand you know how can I transform like put this model into production at my company and extract the maximum possible value.
15. 01:52So for each dollar that you invest with us, our task is really to help you maximize the value extract from it.
16. 01:59So no matter if it's operation, if it's an agent, if it's like a code coding productivity, that's more or less our day-to-day and our team covers mostly like the financial institution here in New York.
17. 02:10So that's about me, Oliver.
18. 02:14Yeah, I'm working in the same team also long history in financial services, asset managers and also investor in banking but I'm a builder at heart.
19. 02:23So today we want to talk about also how to build with the platform um and show you the use case here but yeah happy to help um I work with big customers
20. 02:33um in the room probably a couple of people I have seen already.
21. 02:36So if you approach us want to solve a problem with the API or with codeex yeah we are the right people.
22. 02:45So before continuing I can I ask you to scan this QR code and put your uh information is mostly the email address.
23. 02:56The reason is because we want to give you access to a specific works uh workspace um within OpenAI.
24. 03:04So you're going to get access to the financial data and you're going to have unlimited token for the session.
25. 03:10Plus we also enable the ultraast mode.
26. 03:14So is a specific like mode for like our latest model.
27. 03:19So 6.1 soul and six Astra that reaches like a 300 token per second on average.
28. 03:27So I feel that like for a workshop makes it easier to like you know create something in a faster way.
29. 03:32So I'm going to show the QR code later but please uh it's really important otherwise you're not going to get access.
30. 03:41So just maybe our colleague in the room also if you need it now you come later maybe just people sitting there.
31. 03:47So we need that to get access to the sh GBT work website basically.
32. 03:52So that's what we need.
33. 03:56So um just kind of a preview of the session today.
34. 04:01This is kind of like a tentative schedule.
35. 04:03So probably we're going to like go over and mix and match a little bit.
36. 04:07But we're going to give you a quick introduction uh on like chibi for work uh work for financial services and like kind of like more the ecosystem.
37. 04:17Then we're going to spend 10 minutes focusing a little bit more on the setup making sure that all of you guys can access um judg work for financial services
38. 04:26and then we're going to have kind of like two main session that I'm going to be led by Oliver like the first one where we're going to focus more on
39. 04:35work and then we're going to move to codeex and finally um we kept like 20 minutes mostly like for question and kind of like general feedback and like about uh you know what's going on in the space and you know whatever question that you might have like both on the coding and also like more on the industry
40. 04:55wise.
41. 04:56So let's now go to a quick introduction on our services.
42. 05:02So as you probably know we have three main interfaces three main surfaces that we use.
43. 05:08Chajibbt that is like you know the original one we have chibbity work that is basically something that it's more focused for knowledge workers so it's like you know it can run agent
44. 05:21it can work it works with plug-in but it's not really like codeex that is more meant for like engineering type of task.
45. 05:29So these are the three surfaces that right now we are offering and as already mentioned we're going to focus on chargibility work first and then codeex.
46. 05:40So what we did is basically within the charge work and charge financial services we tried to create a product that is enterprise focused that basically it's fine-tuned like the model is the same but the product itself is fine-tuned
47. 05:57for the specific like financial services type of user.
48. 06:00So there are like three main component to this product.
49. 06:03So first of all the data.
50. 06:05So it's we have some like built-in data that I think it's they're high quite high quality but also we you can access to other third party data vendor directly
51. 06:20within uh chachbt so the agent can interact directly with the data sources without any um overhead.
52. 06:29Then second we really like pushed [clears throat] our model and trained them on specific financial um type of work.
53. 06:39So like you know we can think about like you know M&A equity research and so on so forth.
54. 06:44And third component is the are the artifacts.
55. 06:47So basically we really pushed in the in the recent months the capability of our model to actually build presentation build spreadsheet and also being able to leverage like reference template
56. 07:00that from like you know historical example you know and reproduce them like with the for the current analysis.
57. 07:06So yeah maybe just to add to it.
58. 07:09So in the in the labs what we do now the first part will really focus on this plug-in share GBT for financial services.
59. 07:17So it sits in the web surface the website basically and then we show all the feature because how to use a financial data is one thing that's like a one benefit here but also
60. 07:29there's certain ways or actually in instructs the prompt to present it the information in a chart or use also some verification and also have some annotations where the data comes from.
61. 07:40So all that if you hang out for the lab will be very important to see and that's sort of the domain specific solution we to showcase.
62. 07:51So let's start a little bit like with um the building data and that's why we want to on board you to this specific version because charge for financial services is only available like on the enterprise side.
63. 08:03So these are some of the vendor that are built in within the product.
64. 08:08So we like had partnership with like the Lupa, Elsag, Pitchbook, Crunchbase and other vendor.
65. 08:15And basically to give you an idea like of like what's available out of the box, we have like SEC feelings, we have market data.
66. 08:26Market data of course are delayed.
67. 08:28So uh I think there is a 15 minutes delayed but still you have like real-time market data.
68. 08:33you have news and like we also have like you know private company profiles and earning calls transcript and the main difference here is that these data sources are like
69. 08:47really piped into the product.
70. 08:49So it's not just like the row data that is entering but like we did like a specific fine-tuning of of the databases to make sure that like this these data are easy to to
71. 09:02be consumed by the model.
72. 09:04So what we've seen is like like a nice improvement on like how the model is actually able to ingest this data because we kind of like wrap it in in a format that is like
73. 09:16not not just like you know the row data set but it's more um fitted for the LLM.
74. 09:21Yeah.
75. 09:22So maybe just one thing to add.
76. 09:23I mean I have done something similar myself actually inside the company and it's quite hard if you pull down all the SEC files and you have it scale the data you have to make sure you grab the new information
77. 09:36I mean I have done it in a in an asset management case but I think what's really here unique is basically you you have those agents running collecting this data
78. 09:44we have the financial data pro providers that work with us but also you have to way have a context layer behind it that actually is efficient enough to serve it and that's often a technical challenge.
79. 09:56So that's what we can show you also later.
80. 09:58That's a sort of what the product has inside.
81. 10:00Now it's there's some technical solution for providing that at speed and then like moving a little bit more on like you know how to leverage this data to create analysis like
82. 10:13what our team our product team did is like build several plugins specifically for financial tasks.
83. 10:19So specifically we can talk about the uh the investment banking one and autary research one.
84. 10:27So within these plugins there that is like what we define uh at open eye as a plug-in is kind of like a library of skills plus MCPS and what we built is like we we partner with like top investment banking,
85. 10:42top asset management, top equity researcher to build the kind of like most flexible but also most like realistic plugins like skills that you can actually leverage as a baseline
86. 10:54for your analysis.
87. 10:55So what we've seen not only in the area of like financial services but what we when is that when we provide specific skills these skills present like a good baseline.
88. 11:06So let's say that you are an equity research it wants to streamline uh his [clears throat] them their job what you can do is literally like take the the skill out of the box.
89. 11:18there's been a lot of work behind it and then based on using that skill to create like you know a schedule job to create like basically a pipeline that try to reproduce
90. 11:29what you're aim to and you use that as a baseline and on top of that you try to improve you try to customize to your taste and adding all the beats that are currently missing but what we've seen is that like these type of plugins
91. 11:42[snorts]
92. 11:43provide like a strong blaze line so you don't have to start from zero you start from like something that is built out of industry knowledge and then let's say that the 80% is there and then you add your 20% customization
93. 11:57based on like your specific needs, your specific like view and so on so forth.
94. 12:04So finally again these are like some of the use cases that are already out of the box.
95. 12:11You know these are a little bit more skewed on the investment bank.
96. 12:15We're going to work today more on the asset management.
97. 12:18If you think about you know pitchbook prep uh preparation, earning analysis, valuation, LBO modeling, uh everything comes out of the box and it's uh quite interesting because even myself like I I come from like a background in
98. 12:35in capital markets and I had like to like interact with few people from investment banking talking about valu valuation and just like leveraging this plug-in like running few example locally
99. 12:47was able like to bring me a little bit up to speed to understand um you know the product the the the problem behind um what the clients were talking about.
100. 12:57So let's this is kind of like a little bit like the initial presentation about like you know what's the effort on uh the financial services.
101. 13:07We are now going to move to the more like building part.
102. 13:11And as a final reminder please add your details here.
103. 13:15It's really really important and I'm gonna stay here for a minute making sure that everyone has it.
104. 13:24Yeah, maybe I think it would be good point um when we switch I will show you a little bit what we're doing hands on now that you can just confirm you can access
105. 13:33the site and then also be able to log in into shb work.
106. 13:37Yeah.
107. 13:38Anyone has having problem putting the data in like the email address?
108. 13:44I think it is a good sign.
109. 13:46So we can now switch to the fun part.
110. 13:51Um so this is actually the website we're going to use for the building part.
111. 13:56So please um um feel free to scan it and if the it ask for a code the code is there but it should be in theory that you write QR.
112. 14:11Yeah maybe we wait a little bit before we switch because we have a lot of time today.
113. 14:15Um maybe just make sure everyone can access it and then I can talk a little bit also about the content and I show you also some um technical details.
114. 14:25So what we'll do we we saw some of the nice out ofthe-box features in um shb work for financial service.
115. 14:33So you have data but also sort of customized use cases.
116. 14:37So the first part we thought what would be the benefit for a wealth advisor business with shbd for financial service.
117. 14:46So I think what we want to do and show here how can you actually make this process more efficient because at the end of the day you want to talk
118. 14:56talk to your advisor more.
119. 14:58Now, you don't want to uh have a team that's working, preparing, analyzing portfolios, figuring out um what happened, explaining it, preparing the meeting, um preparing um maybe also for the customer a deck that explains
120. 15:15why were those changes happening in the portfolio, what needs to be done to bring it back in balance.
121. 15:20So, the first part is really the use case.
122. 15:23You bring this product shed GBT for financial service to your wealth advisor teams and they can basically accelerate your workflows.
123. 15:33So it's an enduser tool in the team.
124. 15:36I mean often in my experience also have very different work workflows or different way of working.
125. 15:42So it's customizable and we walk you through this use case.
126. 15:47a customer there's a portfolio change happening and you want to go all the way preparing um the P&L or the portfolio attribution and then showing the customer what to do to bring it back into balance.
127. 16:00So that's the use case number one and um we provide a couple of prompts there and then I don't know who in the room is oh is a question
128. 16:21So yeah, we will fix that.
129. 16:22Um, that's normal.
130. 16:23Yeah.
131. 16:28Let's go.
132. 16:31So, uh, I'm going to work on it.
133. 16:33You can keep going.
134. 16:34So, like is the website like the the latest slide, right?
135. 16:36The shbt work you I think.
136. 16:38Yeah.
137. 16:38So, my colleague work on it and I will do a little bit more to explain it.
138. 16:42So, and maybe for the people in the room here, I don't know.
139. 16:46We have a couple people.
140. 16:47What's what's your background?
141. 16:48I mean, you're like a developer, builder, engineer, an AI exp who is like more in a technical role, developing, building something in in the room.
142. 16:57So, we have quite I can't see it fully, but yeah, it's a half maybe.
143. 17:02So I think for those people that are more hands-on and technical now and I want to explain also the second part of the workshop once we get your access
144. 17:12is actually in our service in the codec side the desktop solution the codeex would be also it can be used by end users business users knowledge workers but this in this case we want to do a little bit more
145. 17:25customization of it because let's assume you you deploy CHB financial service in your company successfully and it meets the business users need.
146. 17:35But then the question is how can I extend it?
147. 17:39How can I make my data coming with it?
148. 17:41How can I customize it?
149. 17:44So the second part of the workshop we thought there are two there basically two ways of doing it.
150. 17:50You could say let's build a custom application.
151. 17:54I get an open AI API key.
152. 17:57Let's do some web UI.
153. 17:58It's easy to do also with codex and web UI.
154. 18:01I deployed in my company and then yeah I have the typical software development life cycle or is this path A or path B is you basically think about it
155. 18:15oh my users have already in the enterprise shed GBT for work they already have codeex most of them why don't you keep sort of this need that I have for my business
156. 18:29inside this ecosystem So like we explore in the second part basically what would you do as an internal developer using codeex to create a skill a plug in and a plug-in extension.
157. 18:44I explain what it is in a bit to actually implement a full application that could sit inside the surface.
158. 18:53So basically with that with those things I will walk you through what those things are.
159. 18:58you can customize basically the workflows.
160. 19:02I mean, everyone I think probably knows now what a skill is.
161. 19:05Um, it's a markup that provides a little bit more customization for you and reusable.
162. 19:12But then once you actually want to connect your data, you need an MCP, you need some tools, you need to define all that.
163. 19:19And then that's where you have a plugin.
164. 19:21Now the plugin is sort of a little bit heavier and you can customize it around it.
165. 19:25So we create a plugin there and then with a plugin it can lifts on a desktop it can live also in the website and then what also we just recently announced is actually on dev day plug-in extensions.
166. 19:40So what we do at the end we also actually customize the UI a little bit so you can customize the surface to fit your use case.
167. 19:50So we have an advisor that would use shbdt for financial service in the web but the workflow you implement can actually also be sitting in the in the desktop application
168. 20:02and then so we exploring that
169. 20:04one second
170. 20:07no
171. 20:09okay
172. 20:09so we get there
173. 20:11[laughter]
174. 20:11so so I have to talk more and probably I can talk you through it so maybe can you do it um on a um I can show maybe the content also Yeah.
175. 20:20Yeah.
176. 20:21Yeah.
177. 20:22So, but just I think the interesting part is we did do in this workshop once we get access to it to you and don't worry if you can't get it out we the website is available later also.
178. 20:33It's really what would I do as an developer with a company an enterprise that already has this platforms inside the company and can I avoid a little bit my work or reduce my work to deploy applications?
179. 20:48No, that's a neat idea I think that becomes possible.
180. 20:53So, just a question uh about the uh built-in tools you mentioned like uh transcripts and filings and all that sort of uh other data that you have there and that you
181. 21:03I guess the harness has some awareness.
182. 21:05Is it just that the the harness and skills are there or is there something that you did to the model?
183. 21:10Just any kind of curious what what is makes it so much better.
184. 21:18The party mic is on as well.
185. 21:21Okay.
186. 21:21So, just for the people that have not the full link, um that's a workshop access code.
187. 21:28Can you get to it?
188. 21:37So, someone just confirm it works.
189. 21:40Okay.
190. 21:40You see, fix it.
191. 21:42[laughter]
192. 21:50So maybe also explaining um what we use under the hood for the workshop here.
193. 21:54So that's actually um the sites feature the capability to host a website and it's quite u easy and in this case it's a public veite actually.
194. 22:05So what's very neat um to create nice looking um functional UIs actually.
195. 22:12So just I just wait a little bit that everyone is um can get to the workshop setup.
196. 22:20We are also sending the emails for um accessing the the workspace as mentioned before it should come up.
197. 22:30Yeah.
198. 22:30So, while we do it, um I want to also I think we have little um like bottles or something for everyone here and um not for everyone, for the people that participate today.
199. 22:43And to make it a little bit fun um we actually once you're able to be um signed up in the room and I see we already have quite a few people there.
200. 22:53Um so this is also created with um the sites application.
201. 22:56So, everyone who was able to get into the um website um got 100,000 virtual money already.
202. 23:04So, to make it a little fun and of course we all know the market always moves up and down and it goes up uh maybe a little bit down now.
203. 23:12But if you complete the steps um you get additional funding and then um at the end of the day we want at the end of the workshop we want to see
204. 23:21um the top three people.
205. 23:22Yeah.
206. 23:23And um they get some bottles for sure.
207. 23:26So to make a little fun, it's a little bit uh to keep it interesting.
208. 23:30But um so for for the website now because I think we see people are in there now.
209. 23:37So what I want to um basically point out so the first thing we really need to have access to shed GBT for financial services and um that's basically the website here and you can see
210. 23:52you have financial data and you can see the plug-in um shed GBD for financial services.
211. 23:57So that's sort of the thing my colleague works on onboards you and when you have access to this website which sht for financial services and in the financial data
212. 24:09um that we showed in the presentation um you are free to go and follow the instructions under core capabilities.
213. 24:16No. So you can say setup once and um we have a little bit a zip file to have some data there for the customer example and etc. So it's really the wealth management use case.
214. 24:30How can I help my advisor to be faster and then I definitely would recommend um each of the subsections you can do on your own.
215. 24:41So if you're interested more portfolio questions, uh research or charting, um feel free to start one of those sections yourself and then um when you do it and have any question, please raise your hand also in the room.
216. 24:55We want to help you there.
217. 24:57Um and it shows really the highlights of the the main features of the product here.
218. 25:02And then for the people that are more interested in in sort of building um I think it is a question you have to have codecs on your desktop.
219. 25:13No. So so for the people that aren't you can install it.
220. 25:17Um I think that's the second part and I I think it feel free.
221. 25:21I mean if you rather want to envision that product more what is the product doing?
222. 25:26What are the out of the box capabilities?
223. 25:28stay in the core capabilities once you have access the website but with codeex um I think that's the idea what I was saying earlier how can we develop maybe with an AI native tool
224. 25:43in an enterprise applications now workflows etc so and then here um we walks you through the skill plugin um we're doing also an MCP use case here to augment this workflow we do on the first side
225. 26:01and then um I definitely want to highlight try out the plug-in extension and I think what's a plug-in extension I just want to explain it a little bit um I think we
226. 26:13I think people know now there are ways of extending um AI harnesses with skills and the plug-in is sort of little bit heavier and you see the structure it has skills
227. 26:25and has an MCP server It has a tool configuration and then you also have an UI probably to surface it in a way how you want it.
228. 26:35That's cool.
229. 26:36That's the one thing I mentioned but it's sort of what I want to say is also possible.
230. 26:42Now it's basically you have the shed GBT or the desktop the CEX application and you see this highlighted areas here.
231. 26:51You could actually customize it depending on the conversation that maybe your wealth management business has with codeex or chat GBT.
232. 27:01No. So you could say if I type in a certain thing and it's a certain file type suddenly it pops out a UI an extension that you control and can deploy
233. 27:13and make it more custom.
234. 27:15No. And there are some examples actually.
235. 27:18So yeah, I want to highlight those are the things um we could um if feel free to proceed there.
236. 27:24So the people that have already codex start with setup once and the people that still get access to the website um please start with um the setup once there.
237. 27:38Yeah, there was a question actually which is important.
238. 27:41So, so shedgbt for financial services lives exclusively in our shedgbt work web interface.
239. 27:52No, so that might could change but right now it's the only way to access that.
240. 27:59No. So you basically I mean you have sort of I mean I have seen enterprises that have the webs GBT work, they have the codec, they have both and even knowledge who users have both.
241. 28:11Um so you have to patch it together a little bit.
242. 28:13It might change in the future but right now it sits there but it doesn't it doesn't matter.
243. 28:19You can actually extend plugins on a desktop but also on the share GBT work site.
244. 28:26Now the plugins live there too.
245. 28:28So you have the options there and currently plug-in extensions the customization of the UI actually lives only on the desktop side now so you cannot do it yet in the web but it will be probably it's actually on our website explained that is coming up
246. 28:47um and so just be aware of it um it's all pretty new here and um the surfaces provide different capabilities just be aware of it and right now if you like a real builder and say hey
247. 29:02I want to get the financial data with a CLI now because Codex has also like CLI you want to put it in a pipeline you're not able to actually access the data now yeah
248. 29:15or it's basically not provided yet so as mentioned before here whilst you're in charge you have you can select the model we suggest you to go with either Astra
249. 29:28or 6.1 Soul and if you can click on the on the speed you can pick ultra fast that is going to make like your life much better so and yeah it's just the promps you follow the promps
250. 29:45and I mean you're learning by reading a little bit what the product actually provides for you so I mean the shibbity for financial services I think when you go on this market
251. 29:57the the historical market data and also research you see some of the key features now that sort of as Stefano said um we focus on to get that in the product more tuned
252. 30:11not fine-tuned but tuned to your use cases now so everyone has the email has logged into the correct workspace and can use like ch work fantastic went almost as as mostly as I would have.
253. 30:32Yeah.
254. 30:32So I mean I I maybe for the people that just want to listen I can do it in parallel also a couple things.
255. 30:39Um but any question I think we rather want to help you with questions so feel free.
256. 30:46Yeah
257. 30:47you mentioned the specific setup for financial aid.
258. 30:51uh you had mentioned uh with the specific setup for financial services has does it have another ontology layer like what else have you built?
259. 30:59I think there was a question asked earlier I didn't quite get the get the answer what specific things you had done beyond the model itself.
260. 31:07So I think like what I mentioned before is really like the data ontology.
261. 31:12So it's not just like connecting the data sources as if you know it would be like you know an API call.
262. 31:19We actually the data ontology has been changed in order to make it like extremely easy for the model to get extract the right information and also there is like kind of a context of like for each type of data source what's relevant what's not.
263. 31:32So if you think about 10ks or 10 QS like is not just like providing the file but is also explaining you know what is what within the file.
264. 31:41I think that's kind of the on the data point that's the kind of like key differentiator and also when when like not just from the integrated data sources but also from like the plug-in that we partner with like uh data provider that you can login into if you have access
265. 31:58that's the same concept.
266. 31:59So it's not just getting the data is actually getting data with the right contents in the right format.
267. 32:05Yeah.
268. 32:05And also when we talk about plugins um these plugins is not of course like at the end like if you look at the skills are kind of like some markdown files some templates but
269. 32:18there's been like there's actual expert knowledge we've been like partnering with financial institution for quite a long time and when we built this product we had like a selected number of partner
270. 32:28like top investment banking and so on so forth that we worked with in order to validate the quality of the product.
271. 32:34So it's is literally like the the the kind of knowledge that we build to put to transform like like the model into a product that can be used by like financial experts.
272. 32:46So
273. 32:47yeah.
274. 32:47Yeah.
275. 32:47So I I think um there I mean if I've just talked about it before if you do it yourself I mean you say I want to do the context layer the ontology
276. 32:59myself at scale and it's quite large data now we have to ingest now so you can do it in an scalable product with agents like doing public data but also have the data providers
277. 33:11and there's a lot of ways of making sure it finds the right data.
278. 33:15So I think um we actually have a couple um blog post how our data architecture is behind.
279. 33:22Now there's something interesting what services we use open source services but I think it shows you there's a lot of know how we build actually for billion users basically
280. 33:33like a context efficiently no and so that's the same thing that's under the hood powering this product no so I think it's maybe the engineering side now and the context engineering
281. 33:45um right now it's it's an early product and I think you will see how we involve it now but also lives on GBT work for now not via CLI
282. 33:54and yeah let's see how the world goes here
283. 33:57is there any way to access some of these more primitive data structures that you guys have developed like if you have ontologies or like a graph
284. 34:06so like no what's under the hood it's not really accessible uh because this is like a product meant a little bit more for the knowledge user not the pure engineer
285. 34:20but uh I think it's without enter too many the details is literally like each data set was like on boarded with the kind of like provider to really understand like what's the best way to serve it and also it's it's mostly like the understanding of the data set.
286. 34:38So it's not just like it's literally like you know what's the information like what's the relevant information.
287. 34:43So how you know like a simple example like you know when you on board market prices versus return like explaining all these detail on like you know if you want to do this operation if you want to extract this information
288. 34:57don't like how to avoid like basic uh misconception and so on so forth.
289. 35:01So
290. 35:02did you did you guys when you were talking about 10K and 10 Q sort of embedding did you guys have like custom embeddings and stuff is that is that at the back end of this?
291. 35:13No no no no it's it's literally like we create the data set and like pro pre-process the data in a way that the model can access easily like think about like a kind of a tool call
292. 35:25but like a really customized tool call for that data set.
293. 35:29So, no worries.
294. 35:32Yeah.
295. 35:33I mean, I'm just showing you.
296. 35:34I mean, I'm doing now the poms.
297. 35:35Now, if you're on the website, you just copy the pom, put it in the product and um and then also when you when you see now the icons here.
298. 35:43So, you see a little bit um is actually picked up the plugin.
299. 35:48So, think about it.
300. 35:48The plugin is a way that's hinted in the prompt.
301. 35:53Um I hinted in the prompt to use shbt for financial services.
302. 35:57So it follows a plug-in concept currently and then um you can see actually the financial data that's under the hood used and the internal tool calls basically in this plug-in happening.
303. 36:09So it's really um like a plug-in concept right now and then um yeah under the hood we provides a back end basically that's scalable.
304. 36:19So I would say like to give you like the really high level view of this product.
305. 36:23I think it's really cool because you know this is basically the interface that like every like knowledge user like the equity research analyst is going to use.
306. 36:33But what we are also going to do in the second part is like thinking about how can you add plugins like uh we all know that you know these data sources are not enough.
307. 36:43They're like you're probably gonna have like internal tools.
308. 36:45You're gonna have internal data set.
309. 36:48You can build your own plug-in with your own skill like especially as we mentioning before like you want to do the last layer of customization.
310. 36:57So as an engineer as like you can build this tool that then like you can use as a plug-in in charge for work.
311. 37:06So your end user your knowledge user is going to have this as a base.
312. 37:10So let's say this is like a really strong baseline on how to interact with data, how to build like specific financial analysis, how to produce like output like Excel, PowerPoint,
313. 37:22but you can add a customization layer that your company or like with like the specific tools with specific data set with specific logic is going to make it unique.
314. 37:32So
315. 37:33yeah, so just point out I mean you you do it yourself also um but I think it shows you the underlying tool calls happening in the sharp financial service you don't have the full access to the sort of the context there
316. 37:47but the second part is um this customizable UI so so there I mean we had just recently intelligent UI basically yeah now when you get in response you basically have a more
317. 38:01um tuned UI answer basically through the response now and this is one example here and that is something that the plug-in provides too and so it has this context when I ask questions about revenue
318. 38:14that you want to see it in a certain way and maybe as a table and the chart so those are the things that are addition so if you use a
319. 38:22the Astra model out of the box it would not do it necessarily automatically like that now you have to basically do it yourself so I just want to point All those little things
320. 38:32that make a difference um because it's already a little bit um not it's not fine-tuned model but it's sort of has context how to present data.
321. 38:42Yeah.
322. 38:44[snorts]
323. 38:54We check the pogus here.
324. 38:58Okay.
325. 38:58I see Mike, Zack, and Cool Deep
326. 39:02[laughter]
327. 39:04doing well.
328. 39:07Oh, and this one is also accessible by mobile in case uh you know you want to do use a different interface.
329. 39:14This is not just like desktop.
330. 39:15It's is also from like the mobile app works.
331. 39:19Yeah.
332. 39:20So, I think that's like the one thing I want to point out.
333. 39:23market prices would be awesome because
334. 39:25[laughter]
335. 39:25I know I'm I'm basically uh right now you have to be in in the surface here.
336. 39:31So if I ask the question now give me the historical market prices and um yeah I think that would be really neat to have an API but as is always like with data providers you have to be careful
337. 39:44uh how the licensing works here but yeah in this case um it's basically um like a very important thing you want to know the market prices in an easy way and there would be also an opportunity
338. 39:55if I built it in-house um to take that on because you all have those internal systems and then have an extension a plug-in sorry a plugin that sits maybe in chat GBT work
339. 40:08and then also on the desktop side and does exactly that now because you have already databases access to internal data and then yeah maybe use the same um visualizations
340. 40:19and then um provide an extension besides a product that's out of the Yeah, it's um it takes a little bit.
341. 40:38So even the little pumps here a couple minutes.
342. 40:40So if you impatient um and you want to go a little faster uh you know how to change the model or you basically the actually you know what you can show like if you click on this uh little thunder on the left
343. 40:56you can go oh doesn't oh doesn't
344. 41:01no oh only on 61 soul if you go on 61 soul I thought like ultra fast was allowed here.
345. 41:12Okay.
346. 41:12Fantastic.
347. 41:14Use that.
348. 41:14It's really beautiful.
349. 41:15Yeah.
350. 41:16Yeah.
351. 41:16I I mean the the benefit of working at Open you can use ultra fast.
352. 41:21[laughter]
353. 41:24No, it should be working also on the web.
354. 41:29Yeah.
355. 41:31Oh yeah.
356. 41:32I mean I I did the right model now.
357. 41:34So yeah.
358. 41:35So yeah, do ultra.
359. 41:37Um and also I think I'm just what I said the benefit of working in openi you can actually use ultra for your normal work a lot um and it's actually I think it's a day and night I think um if if this becomes sort of
360. 41:52more accessible for everyone um it's just I mean I mean how many people probably waited sometimes for some agent work and then you do some couple things in parallel
361. 42:03but now now I think yeah I I really appre appreciate it and um and I think um want to point out it's it's it's very helpful doing your work.
362. 42:12Yeah,
363. 42:13I would say like on this topic uh bringing a lot of like discussion that we had with clients I think like token efficiency and token cost is like I think for the last three months has been like probably the main priority.
364. 42:26So of course right now you know we are talking about Astra so that is like our top performing model but what you observe that is that of course if you're building something completely new a new codebase
365. 42:38you're doing research using the most powerful model gives you a hedge but when you're building something like more like a repeating uh repeatable task you can get top level top model performance
366. 42:52if you build the right context.
367. 42:54So if you build the right skill that explains more in details you know what you expect to the model uh then even leveraging model like Luna or even like 61
368. 43:076.1 sold is actually is extremely good can really give you the top of of the performance.
369. 43:12So you don't have to go for like the most expensive model always the state-of-the-art model.
370. 43:18What we've seen as a common pattern is really like you can think about like building the skills in a proper way that is going to maximize [clears throat] the performance and this is like a common
371. 43:29or if you're working in more like multi- aent setup what we've seen is that the orchestrator needs to be like probably like the smartest of the agent but the single sub agent
372. 43:41don't need to be like the Astra you can switch to something like Terra something like Luna and still get like basically 95% of the overall uh result.
373. 43:52So of course to do all this measurement it's about the use cases about like kind of like building evals is about actually measuring the performance of what you're doing but
374. 44:02you know Astra is more like for what is like completely new research new codebase but also the other models especially even Luna that is our cheapest like I don't remember like out of my like
375. 44:18by heart like the the price but is is extremely cheap We've seen it like that with the right skills can be leveraged and produce extremely high quality content from like you know reports to analysis.
376. 44:31It's just that like you need to use the right skill and make sure that with the right skill the performance is actually matching your expectation.
377. 44:38But that's a great strategy that we've seen working across the board you know for actually optimizing the cost.
378. 44:45Especially if you think that like let's say that you are running like this like wealth management conser concurge on like your kind of like entire universe of client that
379. 44:57scales up pretty quickly both in terms of like cost and also in time time needed like using models like Luna that with the right set of skills can really like help you
380. 45:08uh mitigate this kind of problem.
381. 45:11Sorry.
382. 45:11Yeah.
383. 45:12No, I think yeah, I think the point about I mean the models are changing quickly and there are more models coming but yeah it's often I mean every time we we have a model also I think you have to figure out what's your effort level sort of now because the capabilities
384. 45:26sometimes sit differently now the price point versus um the capabilities so I think I'm I'm basically in my day right now I'm using a lot 6.1 soul and then I can have the luxury to use ultra of
385. 45:42And then but I think ever sometimes when I created the workshop actually the website um so this all was created of course with Astra and um even the more advanced
386. 45:55live room view I think.
387. 45:57Yeah, those are the things I handed off to Astra basically.
388. 46:00Um, I let it run.
389. 46:01I I was on Friday.
390. 46:03I let it run and it came back after that and then I just fine-tune it a little bit more the website with um soul because it was quicker and more efficient
391. 46:12and yeah, I think it's very important every time a new model comes figure out the capabilities, the effort level.
392. 46:18Um, and yeah, it's it's always changing.
393. 46:20I cannot say we can't
394. 46:23I would say like let's while you know while you go through um the various exercises like the question for you would be like any initial thoughts on like you know how are you using
395. 46:35this model right now this type of product and models within your like day-to-day and what do you think are the limitation or the potential anyone wants to like hot takes
396. 46:49on um using this type of product.
397. 46:54No one.
398. 46:58Yeah.
399. 46:59Okay.
400. 47:00We'll have later a little bit more questions.
401. 47:03But so, so I think when I when I saw the first time the product, I mean, we have it seen internally earlier.
402. 47:09Um, I said, "Okay, great.
403. 47:11[laughter]
404. 47:12But I need a little bit more."
405. 47:13So maybe we can pivot.
406. 47:15Um, this product is there and it's I think definitely geared to the knowledge worker.
407. 47:21But let's think about the builders.
408. 47:23So I have a little passion for it because my experience is definitely in the building side also and um um I I want to go a little bit um on on the plugins
409. 47:36again.
410. 47:36So so I think the plug-in architecture sort of the folder structure what's inside and plug in um is nicely described here.
411. 47:44So you see the skills MCP server which connects to something outside also via tools and then you have some UI components that could be rendered doing usage of the plug-in and JGBT
412. 47:56work for financial service is a plug-in.
413. 47:58So, and then um the the thing that you can do then with an architecture like that and that's maybe a segue now to go more on the builder side
414. 48:09is come up with an architecture that leverage all that in a in a in a way that sort of brings all those different pieces together.
415. 48:20So, this is sort of an architecture diagram um because you have to ask the question what's your surface.
416. 48:26So when I say service, we have shed GBT work.
417. 48:29We have which sits in the website.
418. 48:31We have codeex which sits locally or we have also the shed locally in the desktop.
419. 48:36Um and then as a builder I mean we all have now access usually with codecs but I see this actually coming to the knowledge workers more and then there will be probably more similarity in the UI for both um use cases in the future.
420. 48:50So you have it on your desktop and you can do a lot more in codeex because now you can install a local MCP actually.
421. 48:58Now you can create with code very with promps an MCP server that maybe goes to your internal security portfolio database or client database maybe or Salesforce could be also there used etc. But then
422. 49:14you build it there.
423. 49:15You have a pro you have a prototype usually as a developer there.
424. 49:19Now you basically connect to your local website sort of and you have a codex running and it connects to the MCP but um you you also want to use a plug-in infrastructure.
425. 49:32So the plug-in infrastructure sits in sht managed.
426. 49:36So I also maybe just point it out.
427. 49:39So if you want to get now your own plugins in shed GBT work um you can install a plug-in creator.
428. 49:53Yeah.
429. 49:53So you can install this plugin and when you install this plugin you can also install plugins in the JGBT workite.
430. 50:04So that's another way of customization it and the same thing you can do on the desktop side.
431. 50:10I I don't know if I confuse you.
432. 50:11It's but it's basically think about the surface and you have different avenues to customize it.
433. 50:19So just be aware the website plug-in creator is one option and then in your desktop of course you can install and plug in and if you have users you could
434. 50:30figure out an enterprise mechanism to deploy those plugins with your codeex deployment no so we have um like a management business everyone gets codex and also access to shbt work
435. 50:42you can figure out what's the folder structure with the plugins that's managed by the enterprise and then deploy through your existing mechanism and that that's a huge benefit.
436. 50:52I mean if I would build now an application myself I have to worry about how to log in how to make sure my access control is there.
437. 51:02So if you build an application on the side in enterprise it takes quite a time to get it out and approved.
438. 51:08But if you build it in an in surface that's already approved and running and managed you can hook in.
439. 51:17No. So that's sort of the plug here.
440. 51:20So yeah, we would then use some of the OpenAI plugin um plug-in infrastructure.
441. 51:27In this case, the SHBT financial services access to our context data where we have financial data etc. It still sits in the web service only.
442. 51:37You can't use the CLI.
443. 51:39And then the next thing is you would connect in your plug-in with an tool called your databases.
444. 51:47and that would be another MCP or you do it um like in a different way and you can use authentication etc and making sure it's sort of an enterprise
445. 51:57managed way of connecting it to the OpenAI hosted environment and all that can be done.
446. 52:03So this would be sort of one way of doing a little bit more customization but you stay in this thing and I I think I I can just I'm very excited about all those options now.
447. 52:15I mean I I love to build agents myself and host it, but to get it out approved in an enterprise is often very hard and if you can leverage something
448. 52:28and also um if you have it in codex there are some um guard rails policies that can actually prevent that a plugin can go outside of the sandbox.
449. 52:38Now the sandbox is sits in the codeex you have sort of controls there and then if I deploy code I just have to figure out an enterprise approved way
450. 52:48to get my plug-in deployed.
451. 52:51No, so that's sort of the I think it's a neat idea to explore and it can only get better in the future.
452. 52:59Yeah.
453. 52:59And I and I think that like when we talk about enterprise control like staying like within
454. 53:05[snorts]
455. 53:06you know this surface of chibbity work and like which plug-in can be deployed to which employee is like really common request.
456. 53:13So all the enterprise control here available so you can really control the access control who access the data who access the plug-in and so on so forth.
457. 53:23So this I feel I feel that like what Oliver was mentioning is really like here especially if charg work is your final surface for your users the like transforming
458. 53:34like an idea a prototype into a kind of like deployed plug-in that can actually be used by your end your end product um end user it's like much faster than what like we're used to.
459. 53:47We have a question.
460. 53:49So you have to have server.
461. 53:53[sighs]
462. 53:55So depends on the plug-in.
463. 53:57So it depends on the plug-in depends like what you need to do like a plugins could even be like a a set of skills that you know.
464. 54:14Yeah.
465. 54:15Then then you will have to create like a plugin like an MCPlay.
466. 54:23Exactly.
467. 54:24And also to manage the authentic authentication.
468. 54:27Yeah.
469. 54:27Yeah.
470. 54:28So so so maybe because now think about in this world where this is a platform which I think it is already what to do when in those terms.
471. 54:37So so first of all a skill.
472. 54:41So we have like some markup files.
473. 54:43So I would create in natural language um like maybe a couple markup files what to do and I think also it's like quite easy now to create a skill with some help from codeex also
474. 54:56and then it's sort of it's a reusable u method of reusing prompts I would say or knowledge no and and I think that's available currently on the codec side
475. 55:09so if you have a developer we're using skills usually to develop faster It's not actually available in the web side right now.
476. 55:17But then the interesting part is a plugin is what you said is with a skill I can't do really an enterprise integration.
477. 55:24I have some reusability um in the workflow that I can scale and then can share it across the company.
478. 55:31But the plugin is where oh now I have to authenticate systemto system or maybe I want to do on behalf of the user.
479. 55:41No, you have to sort of authenticate um for the user that's actually logged in into your codeex side.
480. 55:48So, so that's why the plugins will be a package where you have the capabilities maybe also some skills that you want to reuse.
481. 55:56But then to your point, I need some code, I need some logic, I need to have some way of getting my credentials for authentication, etc. So, that's a plug in here.
482. 56:07That's a plugin.
483. 56:08And you saw the folder structure before.
484. 56:11And then it uses MCP connections and there's also uh a UI component part of it.
485. 56:17There's an app UI standard like sort of a way of doing all UI components with a plug-in and then all those things can be zipped up basically in a in a package and that's a plug-in.
486. 56:28So it's really I would say the plug-in are most powerful because they contain the super set of everything.
487. 56:36And then what I mentioned before is um once you actually in the surface so if I go back to the plug-in extensions and I think I want to give you I think this is I think a good example
488. 56:50to understand it because I had also a hard time to understand what is an extension but here it's maybe more visually so I think this an example so I mean you see that's sort of the surface you're familiar with GBT but on the bottom left.
489. 57:06You see there's a way of adding your own icon, your own app hook basically.
490. 57:12And then once you click it, now this is all now managed by a plug-in that you can control.
491. 57:18So, and the beauty is it's just not just this little icon there you can customize.
492. 57:23You can customize uh different parts of the surface.
493. 57:29So we can customize the settings, we can customize the navigation and then also think about it.
494. 57:37I mean it's I if I go now in in in an enterprise and I have to go to five different websites to my job.
495. 57:44I don't want to do that actually.
496. 57:46I want to stay in my favorite tool which is AI powered has a very good frontier model hopefully um and I can have my conversation for my workflow and I don't have to leave it.
497. 57:58So what I would like to do is I have my I'm doing my work as a end user but then it pops up it knows a context.
498. 58:06Oh I have now something specialty around getting um maybe an ad u like a client meeting like a client meeting and have to prepare.
499. 58:18So it knows in the conversation and it will pop up my customized UI for that.
500. 58:26Now maybe the that your W business asked you about a certain thing you want to do quicker or you want to see a certain metric from an internal system for this customer profile.
501. 58:38That's what you can do now and you don't have to leave the surface.
502. 58:42So that's a I think I want to give you sort of an idea where those concepts sit and um if you take that away from the session that would be already awesome.
503. 58:55Yeah.
504. 58:57So any questions?
505. 59:01Yeah.
506. 59:01So maybe I switch in in my visualization I set up the codec side.
507. 59:05So would be a question also in the room.
508. 59:07Um, who has uh Codex running locally already?
509. 59:14Okay, I see.
510. 59:16Yeah, like a fair amount.
511. 59:18Yeah, codex of CHP app right now where it they're the same surface.
512. 59:27So, yeah.
513. 59:37So I will just play along a little bit because now it's more um building a plugin for me.
514. 59:59Yeah.
515. 59:59And everything becomes just a problem nowadays.
516. 60:02So, Yeah, I think it will take some time.
517. 60:33So even with Ultra because I think even if you use Ultra, you have to build code and compile or not compile but run it on locally.
518. 60:43Yeah.
519. 60:44So maybe that's also one thing um because I'm doing a lot like this of course for like workshops like that.
520. 60:51It's basically um sort of I'm on working on my desktop now.
521. 60:56So my I can have a couple more shed conversations open and do another project with maybe create another plugin or something like that.
522. 61:04But yeah, I think you really want to scale um basically where where basically the uh the agent runs or basically you want to have a cloud environment.
523. 61:15So that's another thing that's very useful.
524. 61:17Um I'm doing also for financial service a lot of um this appsac security scanning right now.
525. 61:23So it's a very common use case and yeah it takes even on a desktop you have occupied a large repo and you do like sort of a appseac inspection
526. 61:33it clocks your compute locally now so you want to move that sort of to a so remote site now so this case also if if you have a lot of tasks running um I think yeah we have a couple capabilities out of the box there but also you can customize it where actually the compute or the codeex
527. 61:49runs now in this case the codex runs locally which takes even if the model as fast.
528. 61:53It takes time or it's it takes as much time as my laptop has.
529. 61:57No. So, but yeah, it would be good to to have those infrastructure components also inside an enterprise where you basically can easily spin and spawn compute.
530. 62:07No, and then um you hand over task and um it's basically um you come back when the work is done.
531. 62:36Yeah.
532. 62:42Yeah.
533. 62:56Yeah.
534. 62:57So the there are a couple of couple of angles here.
535. 63:01Yes, the idea is really especially with plugins and like you know and the skills underneath is really like to create not not to have everyone with different skill but to actually build like kind of a marketplace
536. 63:15and share them across teams and that works also with what we were mentioning before that is more like the templates.
537. 63:22So artifact templates could be like a website, could be a PowerPoint presentation, it could be an Excel.
538. 63:27What we've seen is that like you want to have like we even within the plugins you can build some like add some kind of historical example as files and
539. 63:38then like the the plug-in will use that as a reference.
540. 63:41So let's say that we are in the context of like you know piece of research that needs to be created for this case like the your you know private
541. 63:52you know private wealth clients what you can do is actually maybe provide like few historical example and then like ask like within the skill to use that as a reference
542. 64:03then of course you can modify them but like the idea is really like to be as as reusable as possible.
543. 64:09So because like we we know that like format matters a lot and that's why like uh even if doesn't really like connects to the financial aspect of it like we we put a lot of effort like in artifact creation.
544. 64:22So if you probably seen like over the last six months the capabilities of our model to produce not just correct you know a spreadsheet but to actually format spreadsheet
545. 64:33in like highest possible level of like formatting that like is matching like investment banking standard really went up.
546. 64:40So we have like several benchmark we can show you but I think that like the best example is like try it yourself.
547. 64:47If you have like some template or spreadsheet presentation PDF that you want that you have and just try to say like okay with this new piece of analysis like build it in in this output you're going to see that probably
548. 65:02uh the output is going to be quite strong and then of course that's the last mile where you probably going to have like to customize the skill a little bit to say like hey these are couple of details that are not correct
549. 65:14but like if you mention it in the skill then at the next iteration are going to be correct.
550. 65:19So yes, the best part is like having sharable skills because you also want to kind of keep fine-tuning them.
551. 65:26So another like interesting point is that are the skill reusable between different type of model different families of model like the answer is that like if we move within charge
552. 65:37families yes.
553. 65:38So the idea is that probably you know if you move from like you know five to six so like a year a year uh kind of like jump you need some adjustment but in theory like between incremental
554. 65:53updates like the
555. 65:55[snorts]
556. 65:56what we've seen is that usually you need less the specification because the model are getting smarter and smarter but these skills tends to be quite reusable and like the cost of maintenance is
557. 66:07is kind of reasonable.
558. 66:08not to elevate it.
559. 66:09So um
560. 66:10yeah I think yeah maybe the skill can be a rabbit hole also actually so because yeah I think the skill reusability and then also maintain it um I have seen some customers go a little wild a lot of skills and then the teams
561. 66:25I mean it's difficult to share and you have to standardize it sort of so this is more like an organizational problem but I think if you want to like say you like an app builder
562. 66:35I would rather lean to plug in because you control also as a deployment and can package it a little bit more.
563. 66:41If I I mean we're sitting on a font office or like some quant I mean they have everyone has skills on their on the on their codeex basically.
564. 66:50So I think the question is then if two people in your team do similar things, they're creating a skill, how can you make sure it's like reusable, you know?
565. 66:59So I would say um if it's a sort of when you have a couple team members and they're figuring out actually do a similar task maybe with a little bit
566. 67:10optionality, I think it's highly recommended to actually remove some skills in your team because it's difficult to maintain.
567. 67:17And I I actually advised a customer recently that I think you have too many skills.
568. 67:22I mean maybe just try to come up with a good like like like a definition of the core capabilities.
569. 67:29If you do like an portfolio analyszis or you do some trading analyszis thing um keep that as a skill but not for everything in your workflow because it becomes kind of hard to maintain
570. 67:40and also the point what you had is um you still have to retest with the models now because um what we what is one benefit maybe if you work with us
571. 67:50is also you can access to the early preview models now so when you have a relationship for us because then you can see also what's impacting your business when you have certain promps and and also
572. 68:02and you can test it early now and make sure when it's live then you actually make sure that it's the same expected behavior now but yeah my advice is if you want to control it
573. 68:13stick to skills you have a builder that actually thought about it what should be like encapsulated encapsulated no and then if you see too many people have too many skills it can also slow down actually the processing
574. 68:27so Um it's not as like a straight answer there but you have to manage it.
575. 68:33It's my recommendation.
576. 68:35Yeah.
577. 68:35I think as a as a general trend that we've seen is that uh um so our team mostly focuses on API use cases.
578. 68:43So uh use cases that like where you know you're going to use response API.
579. 68:48So um eating like a different an like a fourth surface.
580. 68:53But I've been an open eye for like few months now.
581. 68:58And what you've seen already in like this kind of a short period is that lot of use cases that before were only able we were only able to package
582. 69:07if you were able if you were like actually writing code using API and API keys.
583. 69:13Now they can be perfectly recreated like um within uh with the plug-in.
584. 69:19So we to give an example guys is a little bit outside of the world management.
585. 69:24We built few months ago like an internal use case of like how to to create an advanced piece of financial research.
586. 69:33So what you can think about like as a deep research but specifically for finan for financial type of problems using uh the API.
587. 69:42So it was like kind of an orchestration of like multiple agents.
588. 69:45So the like the schematics was pretty simple like on a high level.
589. 69:49So you have what we call a planner.
590. 69:52So basically an agent that ingest the question and split the question between like sub sub question and for each sub question create basically a sub agent that is going to investigate
591. 70:04in an independent manner and then there's going to be kind of like what we call a critique agent a feedback agent that is basically taking all the output from these different independent
592. 70:15uh research putting them putting them together validating that there is no not conflict if there is conflict between different findings.
593. 70:23It uh creates automatically new sub agent to investigate this like this like kind of contradiction you know till a moment where like we the final report reaches like the quality and answers all the question
594. 70:36that you want to cover.
595. 70:37So really think about like an orchestration of sub agent and like kind of like a general orchestration that is really dynamic.
596. 70:46Right now you can do we like actually Dom recreated that same use case using a plug-in talking about the deep research and that is way more usable for clients because first of all you know the end user doesn't have to
597. 71:03leave the the interface to actually like you know enter like a different type of product that it can just stay in codeex it can actually visualize like what are the sub agents doing, what are investigating,
598. 71:18you can create even a specific uh interface for the user to like understand what's going on.
599. 71:24And with the current like multi- aent capabilities like you can really see everything in Codex, do everything in CEX and deploy you know instead of having like creating a new codebase
600. 71:37just create a plug-in and ship it into Codex.
601. 71:41So
602. 71:41why would why would you want to So I think we can ask the author.
603. 71:52Yeah.
604. 71:52Yeah.
605. 71:53And also there's a Matthew and Zack and Cohen.
606. 71:55I think you guys are leading here.
607. 71:56I mean it's a fun thing.
608. 71:58I mean let's see at the end.
609. 72:00So but it's a tough race here.
610. 72:02Uh yeah.
611. 72:03So when it comes to where and why and when we would use plug-in capabilities typically the the original example of that the fundamental deep deep research for equities uh was an application that I built about how to do deep parallel
612. 72:16validated work in terms of that space out of professional quality.
613. 72:19It ended up leveraging a significant amount of orchestration.
614. 72:22It meant like a lot of deterministic nondeterministic workloads within that process and it was a standalone application.
615. 72:28What started to make a lot of sense is that a lot of the people doing that work weren't going to standalone individual one-off applications to do that work.
616. 72:35They started consolidating where that work happened and that happened to be in codeex or jetp work.
617. 72:39So it makes a lot of sense to bring that surface area and then the choice of how that how we want to like rebuild that came down to the fact of can I leverage the capabilities of codeex including its sub aent capabilities including its ability to direct dynamic cap uh agent structures
618. 72:54to allow plugins and connected to different data sources instead of building these bespoke connections and it started to make a lot of sense to say hey this works as a plug-in specifically focused on
619. 73:04utilizing and reusing the LM capabilities to handle things like dynamic tool calling specifically that in line with how models do that.
620. 73:11So we call that ind in distribution tool calling.
621. 73:13It's one of the things that makes things faster and more accurate in terms of how those tools happen.
622. 73:17So all these things happen to line up to like make sense of like instead of building the standalone agent, build this functionality into a set of a of a workflow, a hard or soft workflow depending on the gates you want to build as and then use codecs as the main surface area, the product area for that.
623. 73:32So I was basically taking an old school style really manually tuned application that was using large language models in some nodes and traditional terministic workloads others and consolidating them into a codeex native process
624. 73:44which specifically for the use and ease of the user which is you don't have to go to this one hosted site.
625. 73:50It becomes locally generated.
626. 73:52It becomes customizable based on their needs.
627. 73:54They can basically add hey these are the data sources I want to use instead.
628. 73:57These are the types of gates so I have like quality bar gates.
629. 73:59So it started to become less of a black box in this predefined application and more this evolving process that made sense to live alongside the consumer.
630. 74:10Thanks D.
631. 74:10Yeah.
632. 74:11So so I just have to maybe add to the builders here also a little bit.
633. 74:15So I think under the hood I think we have the codex harness which is open source here basically.
634. 74:24So that's also a big competitive advantage I think because you have basically the full access to the source code which running basically what I showed today here and then you run it on the desktop
635. 74:37that's one option with a manage product and that's what you see here it's basically but then also you can take that and move it with an SDK to the server side or an appunner there's a way of doing that
636. 74:52programmatically And then of course you have the same builder capabilities.
637. 74:57You have you see you have the plugins that it's basically um becomes a way where you can plug hook into an existing system that definitely has a lot of traction now
638. 75:10and um gives you multiple choices when you have this research workflow what my colleague just mentioned maybe created very quickly.
639. 75:18Maybe a business tech or quant or a researcher has be able to do it.
640. 75:23But then you want to scale it to an application on the server side.
641. 75:27What do you do?
642. 75:28How can you take the work you did already and scale it to the team?
643. 75:31So you can take that and with the codex SDK you can hook it up in a real interface also if you need to.
644. 75:39No, but you don't it's just a way that you think about it.
645. 75:42It becomes very fluid where you can reuse your work and where you host it in the future.
646. 75:49Yeah.
647. 75:54So, let's quick check time.
648. 75:56We have 40 minutes.
649. 75:59Yeah.
650. 75:59Maybe 50 minutes at the end question or
651. 76:01Yeah, we're going to have like 50 minutes Q&A at the end, but feel free to interrupt at us any time.
652. 76:13So I would you know keep talking a little bit on Codex just to give a little bit more context on like why we think it's a great product and
653. 76:22as as we mentioned like the backbone of Codex so the uprunner and uh is a completely open source so even for more like to deploy this product within enterprise
654. 76:35it gives like a lot of transparency to see exactly you know the first party app of of course is proprietary but like the engine that is run running behind is fully open source.
655. 76:46So yeah it's not just transparent but it's also fully customizable.
656. 76:50So uh I think that's kind of like really interesting um angle.
657. 76:56So within enterprise you can actually choose to customize the uprunner that then is going to leverage the first party interface to your needs.
658. 77:06If you need any level of a customization it's there.
659. 77:09Additionally like what we think compared to you know other type of interfaces is literally that um our model are trained within with this harness.
660. 77:21So our model you know you know there's a lot of pre-training and post- trainining but when we think about post- trainining the postraining is is leveraging like our own harness.
661. 77:31So the our model basically are trained to work you know within like multiple harnesses but specifically on this one they're going to squeeze more performance for this kind of like trivial reason because
662. 77:44they like within the training phase they're already exposed to to this codebase.
663. 77:49So especially like recently if we think about all the multi- aent capabilities so what Dom was mentioning so more lots of like workflows the most advanced one are leveraging like sub agents and the basically
664. 78:05we the agent itself is taking care of like deciding when a sub agent is needed and when it's not.
665. 78:10Of course, if you move like uh within what we call the ultra, not ultra fast, the ultra mode, basically there like we are pushing the agent to leverage a little bit more these sub agent uh spudging capabilities.
666. 78:24But especially when working with sub agents working sub aent codeex provides like a a much better performance and on top of that what you have on on the background is a compaction mode.
667. 78:36So compaction mode is basically nothing but like a secondary model that is going to take the context of the current conversation and keep it compacting in order like to remove like the redundancies
668. 78:51keep that the um the size you know within the the limits and it's a really high quality model.
669. 78:56So what we've seen and this is a little bit more in what we've seen kind of like in the research type of task where like an extremely long context is needed.
670. 79:07Compaction mode really allowed us to solve problem where like much longer context uh were required.
671. 79:14So this basically allowed us to perform extremely well in what we call the long running task.
672. 79:21So um like we as a team we will frequently work with hedge funds and prop trading firms.
673. 79:28So as an example I that I run myself is that I took like one of the papers from like quant finance like you know brownie emotion and like option pricing
674. 79:39and I literally asked the model and we are not even talking about Astro it was like few months ago so it was like 5.5 to within codeex just replicate the the codebase of the mo of the paper so basically
675. 79:53replicate the codebase of the paper generate the numerical result and try to see if they match what this what the paper mentioned.
676. 80:01So there is like a task that you know it I think it ran for a couple of hours and where like it would have been impossible to run without compaction mode
677. 80:12and then for example after two hours I was kind of kind of impressed that the model was able to fully reproduce this like you know really quant heavy codebase
678. 80:22with like you know stoastic barium motion simulation calibration of models on top of that and that was like basically a task that without compaction mode is not achievable and compaction mode is already available
679. 80:35first also via API but you know if you use the codeex arness is available in the arness and you don't even have to think about it yeah so I mean I'm running in par I think the promps some of people have done it um yeah it's running um creating the basically the plug-in extension right now
680. 80:54um yeah it's basically um you see on the right side the way of in this example, my customized UI could be hooked up into the surface.
681. 81:05So, it's like a small example only, but yeah, it's it's pretty easy, of course, just to prompt.
682. 81:10Um, but yeah, it shows you some UI options there.
683. 81:14And then, um, if you look at our website, there are a lot more examples actually how our partners like Figma or Adobe have created a little bit more an app now where you can do a more complex thing.
684. 81:24But I I really think I mean I have said it many times is I think that's where probably the industry is moving to now.
685. 81:31You think about this more you have an AI platform and I want to plug in in that and then it's basically a marketplace and enterprises have their own inside
686. 81:41and I think that's my personal view of course um but I see a strong resemblance here and um yeah it's I think it's a builder you have to just take as an opportunity you know to take advantage of.
687. 81:53Yeah.
688. 81:55Yeah.
689. 81:55And I I would say that uh when when I joined OpenAI, a lot of people were still using Codex CLI as like you know mo most of people like you know coming from an engineering background like we're all used to CLI
690. 82:09and I've seen that the percentage of like extremely technical people moving to the first party app and almost never leaving codeex as an app is getting like bigger and bigger.
691. 82:20So uh as a former [clears throat] quant I still do around some experiment and right now like I used to have like my Jupyter notebook always open right now like is also already like within codex.
692. 82:34So what we really think and I think one of the message that we want to pass is really like even as a technical person codeex is an amazing surface
693. 82:43and it's getting like all the you know kind of surface within codeex that you need and then you can as a technical person create plugins and ship those plugins to your like
694. 82:58knowledge but non-technical users and the the way that you can actually put put this plug-in in production is can be quite stream streamlined thanks to enterprise control and you can even you can give like really powerful tool
695. 83:13to your end user in like weeks instead of month and I think that especially for financial services where like lot of data ingestion is needed every time there's maybe like a new data source
696. 83:26something that needs to be changed like a new analysis needs to be done this this new approach this new like AI based approach is going to like really help maximize the productivity
697. 83:39in a way that we've seen you know starting in 2025 for coding so that the famous like we all all heard like 10x 100x productivity on the engineering side
698. 83:50our objective here is like to bring that booster also to the knowledge workers so that's kind of I think the the vision behind the product and financial services thanks to this like mix of like artifacts,
699. 84:05data sources like and technical analysis feels like um kind of like an immediate match.
700. 84:11But while sorry
701. 84:15any question on all this topic that we covered so far
702. 84:23I know we are here to discuss financial services but do you see that happening in other industries?
703. 84:28Is it the financial services because the data is quite structured?
704. 84:33You build the ontology layer.
705. 84:35Is open AI doing that for healthcare?
706. 84:37Is something else happening?
707. 84:39So,
708. 84:39so the answer Okay, the answer is yes.
709. 84:43So, um we are working on multiple uh verticals and healthcare is definitely um uh one of the most priority the biggest priorities also in terms of like research.
710. 84:57Um I would say that the other one that we get exposed a lot is cyber security.
711. 85:01cyber security is change is dra drastically changing and in the space for example aside of like the more the more like cyber security specific model that we offer like our offering is called daybreak
712. 85:13um is also we also like basically build again a plug-in and what and again it's the same concept as as before the idea is that like your cyber security team
713. 85:25doesn't need to be an AI expert team or at least at the beginning so we bas Basically, you know, we provide you the model, we provide the the ARS, the open source ARS
714. 85:36uh to work with, but we also provide you like a plug-in for like, for example, for doing like assets uh static asset scanning that is going to is tuned on our model.
715. 85:48So, it's going to help you create that baseline in order that like you know out of the box you can like models, harness, plugins to run everything.
716. 85:58Actually we have like a library of skills that helps you to do everything from the scanning to the the final like review and patching.
717. 86:06But you know this is not
718. 86:08[laughter]
719. 86:08we're focusing more financial services but our focus is really like across industries.
720. 86:14Of course there are industry where like the benefit is like easy to measure finance cyber security healthcare research in general and others where like use cases like are not as broad.
721. 86:27So
722. 86:30so I think it's for the extending platform builder.
723. 86:35It's more I think our digital native businesses or like you would say sort of software providers that you usually look for new avenues to sort of distribute also their solutions now.
724. 86:46So I think our team works sort of with them a lot more.
725. 86:52I think what we are here because we in New York also met financial services.
726. 86:56I think it's what you say it's a structured data is it it lends to itself to use actually AI.
727. 87:04Now AI is sort of um with their data and I think it's kind of always hard to sort of have those data silos in those companies and then how can you actually leverage all this data in a context which is not there.
728. 87:19It's not ontologies etc. is not there.
729. 87:22So sometimes I think what I see as a strategy is sort of you have a it's it will take so much time to proper context layer across all the data stores but why don't we start from the bottom up too now so bottom up would be
730. 87:39you have sort of the shb work of codex and then you have teams that basically connecting bottom up to all the data sources and build sort of this context
731. 87:49over time while you modernize your data architecture over time.
732. 87:54So it's sort of the AI would be super efficient and I think we have the luxury internally.
733. 88:00We don't have those legacy things.
734. 88:01We have this context layer already.
735. 88:03It's it's amazing what you can do then as a workforce if it's actually unlocked.
736. 88:08But you have to bottom up and then top down will take more time for you actually to do it at scale.
737. 88:15Yeah.
738. 88:20So let's see.
739. 88:21um co Matthew and Zakia I think we have this my screen saver I want to talk one more thing I I think um it's probably to to mention also
740. 88:32um so when you install like sort of any extension inside a codeex no I think there's also you have to maybe you actually have some code running then on the desktop of someone else no
741. 88:47so or you have it in the website no so in our security teams we have a lot of discussion with them and um how can you make sure they don't do any harm
742. 88:57I mean there must be some sandbox and I think um maybe just sandboxing with codeex is a very important concept to understand because it's another benefit to go actually go this way
743. 89:10because let's say you have a plugin that has to do a tool call so you want to limit where the tool can go from an URL you You always have to have multiple
744. 89:22controls.
745. 89:22You want to start with your enterprise controls, networking security, endpoint inspections, but once you have a an application on a platform, you want to have also controls inside there.
746. 89:34So the codeex provides actually uh policies um the config files toml that are managed on an administrator site as an enterprise but also push down when you do a deployment
747. 89:46and in those sides you can decide okay I I don't want to get that this plugin can go to a certain website no and I want to make sure
748. 89:56certain things can be only done also on my workspace let's say a team develops inside an enterprise something but it could not go like to maybe like an sensitive project folder.
749. 90:07Now, so you can limit actually um with policies inside codeex an enterprise policy.
750. 90:15No, what is can be accessed locally via the codeex client and also what network what URLs um maybe what even plugins certain people can do.
751. 90:27And I think that's important to understand.
752. 90:30There's a sophisticated way to do that and and that's basically the configuration that you can see also here.
753. 90:39It's the custom config toml.
754. 90:41So this is something your enterprise administrator would define once on an enterprise level for different teams.
755. 90:47It pushes it down and it has all those things I just mentioned.
756. 90:50the security sandboxing because it will run whatever it's hosted inside the codeex client will run inside that you still have to protect tools etc yourself but I think that's already a benefit
757. 91:05um for this architecture and then the next thing is we all know that it's sort of difficult to I mean the security team goes all the way left we want to we want to
758. 91:18we we want to do um um like any sensitive operations but then the business user says hey it's so slow I always have to click please go to this plugin go to this website and and I think
759. 91:32um what also it's unique here we have a uh like the way of doing a guardian like we can actually um automate a little bit the decision- making when it's critical or not critical and this is sort of a way also to make sure
760. 91:46when you use a codex setup it's sort of balancing what security needs as a policy but also what's convenient or comfortable for the end user and this helps to make sure when it's sensitive and when it's not sensitive do I need to go to this workspace folder or not so I think that's very important also to highlight
761. 92:07because I think this is the thing you have to also explain your users now when you build plugins
762. 92:14no yeah like and basically every every big institution we work with they will customize uh the configuration for approval permission.
763. 92:23So that's like a really like usually when we talk about like deploying codex across the enterprise that's the first like point that we discuss about and like again there is a big flexibility there.
764. 92:35Any other question?
765. 92:38So, so maybe maybe almost the end.
766. 92:41I think should we
767. 92:44get our invite and then maybe ask some question if you uh co Matthew and Zack I think uh the leading as every good tech company we brought a little bit of merch.
768. 93:00[laughter]
769. 93:02So, like should we say that Co, Matthew, and Zach are the one getting an amazing water bottle branded OpenAI
770. 93:15[laughter]
771. 93:17picked this morning from the office.
772. 93:19So,
773. 93:20[laughter]
774. 93:20if you if you're still here, let's see.
775. 93:23Otherwise, so let's start like uh Coin.
776. 93:29I don't know how to pronounce it correctly.
777. 93:31Yeah.
778. 93:31Come up.
779. 93:32Come up here.
780. 93:32Yeah.
781. 93:36Yeah.
782. 93:36Thanks for answering the question.
783. 93:38It was not too hard.
784. 93:39So
785. 93:41yeah.
786. 93:41So maybe
787. 93:42tell me about you.
788. 93:44Yeah.
789. 93:45So uh my name is Um I'm working as a data engineer in the Netherlands.
790. 93:51Um so it's pretty interesting to see also the crowd over here.
791. 93:56I've been to AI engineer in London.
792. 93:58Um and looking forward also to meet a lot of people over here and to see what people are doing here especially in the financial industry.
793. 94:05Um that's about me.
794. 94:07Yeah.
795. 94:08And you are from company
796. 94:12data.
797. 94:12So we are consultant company uh yeah working worldwide and I'm working for the for the Dutch uh section.
798. 94:22Thank you.
799. 94:25Then we have Matthew.
800. 94:31Yeah.
801. 94:31Awesome.
802. 94:32Yeah.
803. 94:32You have to also say something now.
804. 94:33[laughter]
805. 94:34I hope it was fun.
806. 94:35Yeah.
807. 94:36Yeah.
808. 94:37Oh, thank you.
809. 94:38This was this was awesome.
810. 94:39Uh my name is Matt.
811. 94:40Uh company's x1wealth.com and we're building an AI native family office.
812. 94:44So, I'm doing a ton with codeex and this is very helpful.
813. 94:48Appreciate it.
814. 94:49I think an AI native sounds very good.
815. 94:51I think I
816. 94:52[laughter]
817. 94:52would invest in that.
818. 94:55Yeah.
819. 94:55Cool.
820. 94:55Yeah.
821. 94:55Thank you.
822. 94:56Then last one, Zach.
823. 94:59Okay.
824. 95:01Yeah.
825. 95:02Come up.
826. 95:05Hi.
827. 95:06So, I'm Zach.
828. 95:06I'm the founder of the Clockworks and I build apps that use AI to explore new things that we haven't really been able to do before.
829. 95:14I focus mostly on development topics, but you know, I'm always looking for new opportunities.
830. 95:19So,
831. 95:21thank you.
832. 95:22I have a Yeah, good.
833. 95:23Yeah, I have an idea.
834. 95:24So I think the homework is for you by
835. 95:26[laughter]
836. 95:27to build a plug-in extension and put it on the marketplace or plug in.
837. 95:32Yeah.
838. 95:33Do you know anyone that can help us push it through?
839. 95:37Oh, you already there.
840. 95:39So
841. 95:44yeah, he's he's he gets it.
842. 95:47Yeah.
843. 95:47So yeah, this is also another thing.
844. 95:48So I mean with all the um marketplaces now so there's a little bit you have to submit it because it can go also to external customers but I think
845. 95:59if if you see it like this this is I think a signal for people I mean this is maybe B to B2B outside marketplace now but also don't forget the internal
846. 96:11enterprise thing is the same concept yeah but I think it's really a signal where you have to leverage the distribution channels that are sort of emerging.
847. 96:22So the emerging distribution channels and and we have seen it all with the app store and all how viral it is now.
848. 96:29So basically think it is another way and of course the SM review if it's public now so and we just have to make sure it's smooth and safe and secure and it fits the right category now but yeah it's a
849. 96:41I think yeah just it's a signal that's
850. 96:43[laughter]
851. 96:43what I want to say.
852. 96:44Yeah.
853. 96:45Yeah.
854. 96:45I know that's I think is that's also the beauty of an open source code base and I would say that like again today we cover more like you know our product surface so codeex
855. 96:56charg work but we do also a lot of interaction with clients when working on bespoke use cases and specifically like on uh API use cases like of course that's a little bit more complicated
856. 97:09and there's a lot of topics that we can discuss but like a lot of the engagement that we do is actually understanding uh the use cases helping like you know bring best breast practices because we are the first user internal user of our product
857. 97:25we do speak with a lot of people in the industry.
858. 97:27So for example as we like mentioned before this magic word evals I think is like the most important one.
859. 97:34So literally like helping clients you know understanding you know how to design the right eval for the right problem and the question is always like you know we talk about you know boosting productivity
860. 97:47but especially when you want to go to production you need to have a way to measure like actually the performance of your product and it's you know again when we're talking about like specific
861. 97:58you know chat thread like there you don't need any valves but when you build your agents then it's needed And for example you know best practices there on like how to build evals
862. 98:10understanding like out of an evals data set the performance why the performance is not meeting the requirement you know what the kind of like you know probably like it's not about the model it's about
863. 98:22how to improve the arness the prompting the connection with the tools and sometimes it's also like a model limitation so we also connect back to research to understand like where our model are hitting the limits where they're not
864. 98:34behaving even if in the perfect environment as expected and ideally you know thanks to your feedback at the next model iteration that is is kind of fixed or improved.
865. 98:45We have a question sorry.
866. 98:46Yes.
867. 98:47So on the topic of evals, I was quite curious to see where what your perspective is in terms of the market's maturity in terms of adoption of evals and understanding of the fundamentals
868. 98:58and even going into advanced topics or more advanced topics like auto research type of workflows.
869. 99:04Okay, so this is a pretty big question.
870. 99:06Let's start with the evals.
871. 99:08Uh I would say that evils are the most usually the blocker because building evils and this is my personal perspective.
872. 99:17So happy to uh hear like different thoughts.
873. 99:20The complexity comes from the fact that evals usually incorporates a lot of like business business knowledge.
874. 99:27So coming from uh SMMES so subject matter experts and actually they are built by engineers.
875. 99:34So uh it's having you know the intersection of like a team that has like subject matter experts and engineers to build evals is not trivial.
876. 99:45So usually I always and again here there is a bias because in my previous job I was like what is called a quantitative researcher like for me whenever you build an agent evals is the most important piece because
877. 99:59you cannot do optimization if you don't do not measure the performance and it's not just you know like the one line of like oh I'm you know I have my evils and it's like 70%
878. 100:10there but it's also understanding the underlying metrics of like you you know first of all like you know where are we performing well where are we are not performing well and so on so forth so
879. 100:21I think like evas is like a really hard part but it's not because it's a difficult technology is because like you need to basically put like expert like expert expertise
880. 100:33within like kind of like an engineering created data set so then best practices there like to keep it short I think like always like start simple Don't don't start with the LLM as a judge.
881. 100:46Start simple and then add complexity because otherwise like you'd never understand if like the quality of your evals is not good enough or the quality of like the agent is not good enough.
882. 100:58So that's for me like the like kind of like best practice and one kind of cheat code that we always suggest when you use LLM as an agent don't ask to rank one to five to score one to five always ask yes or no it's much better try to decompose that one to five
883. 101:18into a true or false and you know like right now like doing some promotion we also release like the decision API so you can have literally a model that you know it's quite good is based on out of Luna and it's going to tell you like you know you know quickly super quickly yes or no and the
884. 101:37probability and the kind of the confidence around that answer so I see that like a lot of possible application for a decision API uh could be actually within the evils so that's the first part second part is auto research
885. 101:49it is like a really big topic it is um something that uh I think really started significant this here and uh what I've seen is that like and there are benchmarks so like
886. 102:03we reach really good coding capabilities already last year but like our model specifically really like improved on the mathematical capabilities more recently especially with like Astra and 5.6 six.
887. 102:17So I think like especially from a kind of quant perspective like the intersection of coding and math research really allows you for this like more auto research task.
888. 102:27So I would suggest like for you for example like one of the auto research task was what I mentioned before replicating a paper.
889. 102:35So of course like there are different level of like complexity there but what we have especially if you think about one specific metric that you want to improve to
890. 102:47hill climb as we say in AI terms like we have what is called the goal mode in codeex.
891. 102:53So basically a goal mode is basic you have to specify you know what's the target and it's better to have like kind of a target that is like a number a score.
892. 103:03Uh one example to keep it simple is you know like try you have this agent you have this EVAS data set this benchmark try to calibrate the prompt in order to maximize the performance of the
893. 103:18of the agent and what happens there is that the agent looks at the target and then keeps iterating on like new ideas and new hypothesis.
894. 103:28Of course, this is kind of like the the kind of like most s simplistic kind of like auto research mode, but like some of the result that we published also like in terms of mathematics early this year
895. 103:41were you were created using goal mode in uh in codeex actually for one of those we also published the prompt.
896. 103:49So I think like if you're interested in the space reading actually the public prompt that we gave to Codex in order to prove that theorem it was a conjecture
897. 104:00uh there is a different sorry for the mathematician in the room.
898. 104:04Um I think that's the most interesting like really interesting piece of content because like shows you a little bit more the logic of like this because like multi- aent and sub aent
899. 104:16approaches.
900. 104:16So um like for example like especially when you do research task you're you're probably going to use ultra uh mode then you're going to really see that the agent is going to leverage like
901. 104:28the sub agent automatically to actually like run some independent piece of research in parallel then like merge them and iterate.
902. 104:36So I would say that you know these are the two really good entry point for evils and auto research and I don't want to understate like even if it's simple
903. 104:48like we actually use goal mode to achieve like significant results.
904. 104:52So um it's quite powerful.
905. 104:56Yeah.
906. 104:56Plus one on goal mode.
907. 104:58Yeah.
908. 104:58I think I think yeah just half a year ago you had to worry about the prompt and you had to worry about how to orchestrate the agents and all this stuff.
909. 105:07So now I you move back now you you really have to think about what should the agent do as a goal and I think that's very powerful because I mean the models will be more capable to actually do all that than
910. 105:21automatically what we used to do what we do still partially manually and I think I highly recommend plus one on the goal mode and also I think today we had chativity financial services
911. 105:34plugin basically shown but I think our team really I mean also my colleague Dom we're focusing with financial service customers or any fintech etc to actually implement it at scale and also those difficult problems and there's a lot of nuances in the evaluations
912. 105:52that our team brings so we're not going away we just want to make sure you leverage our models and the platform at the scale I mean this was more simpler today but approach us
913. 106:02if you have any use cases happy to work with you
914. 106:06yeah know I really want to stress the fact that it's kind of like a twod directional conversation.
915. 106:11So, you know, if you already have a relationship with OpenAI, please, you know, hit your account director like feedbacks are more than welcome and also it's part of our job really making sure that,
916. 106:23you know, it's not just nice presentation, but actually it's going to work at your own companies.
917. 106:29So, um that's really like, you know, we are here.
918. 106:33But any other question?
919. 106:37Okay.
920. 106:37One of the questions I have.
921. 106:41Thank you.
922. 106:42Uh, one of the questions I have is uh during the presentation there was a lot of um uh readonly skills and plugin curious for uh the clients or any skills that you've seen that actually writes back into
923. 106:55the system or like system of records that uh the team has been building with.
924. 107:01That's a good question.
925. 107:03I think Dom has a has a even better answer.
926. 107:06Yeah.
927. 107:06Hey, I'll come up here.
928. 107:07Uh, yeah.
929. 107:08So, when it comes to plug-in structures, uh, and how they can and should be interacting with larger systems, there's a couple ways to do that.
930. 107:17Uh, and it depends on like what you're trying to build.
931. 107:20So if I'm building a fullblown like plug-in extension, I'm really just orchestr I'm using the codeex as the orchestration system, but those tools and processes can absolutely store to remote locations
932. 107:32and to interact with those systems.
933. 107:33The easiest way to do that is to interact with the local file system.
934. 107:36So if you tell it, hey, within this project where I'm scoping this out, it can it can write back to these processes or you give it instructions for the model to actually like the output of the tool is this and the output of the or the instruction of a skill set is to say, hey, persist this in
935. 107:49this location.
936. 107:50Uh you'll see a lot of persistent structuring uh within codeex.
937. 107:53It's gotten really really good at handling long context by effectively utilizing its local file system as like that point of record.
938. 108:00That being said, once you move from a traditional like readonly plugin or skill set into a full-blown extension oper operation, like I tell it to go talk to the central server, I tell it to go spin up a database or at least connect to the specific database as part of a tool.
939. 108:14So like the a lot of the use cases that we do, we just did one around proxy voting, being able to actually go define your proxy voting systems, what it actually had what it sent to.
940. 108:23was like even though the plugin in the surface area was codeex the actual source of record was this app server that was connected via one of the plug-in extensions it says hey write this report write the findings write the date write the the sources
941. 108:35back to this end of it database and that database feeds the like their production application system which is outside of codeex so operating system from a user perspective is codeex
942. 108:44uh operating system from an administration perspective and from the legal review perspective was outside of codeex so the extensions MCPS allow you to be flexible where the those different operating systems are and where the systems record are.
943. 108:56Uh and you can either offload that to the codeex and keep things local or you can build the tools and build those avenues into particular like MCP and OOTH related you know permission sets by pretty easily dropping an MCP where you need to.
944. 109:08Yeah, I would say like another like common use there is really like to especially when you're doing like specific product um processes that needs to be kind of like audited
945. 109:20like he was mentioning proxy voting having like kind of like a standardized like what we frequently build like kind of a standardized like logs that will allow like you know a technical user to understand you know not just the end of the analysis but how every decision every like piece of was done that's for example Like this
946. 109:39extended log is something that we tend to store frequently and especially you know in a highly regulated framework like um industry like financial services being able to back trace every decision to the data source and the logic behind
947. 109:53it was kind of like kind of a crucial and they like usually for example when we produce this artifact as output we also store like logs that are fully auditable for this reason as part of like the process and the plug-in.
948. 110:07One last question maybe.
949. 110:10I think we almost at the end now or
950. 110:12Yeah, we are like five minutes out.
951. 110:14Yeah, thank you.
952. 110:15Thank you for spending time with us now.
953. 110:17And then please reach out for us and thanks have a good good uh conference and thanks for coming.
954. 110:24Thank you so much guys.