WEBVTT NOTE Sentence-level transcript of https://www.youtube.com/watch?v=0PqGT2L6cZM NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/0PqGT2L6cZM.html. A cue ends where the next begins, or 2 s after its last word. s1 00:00:01.309 --> 00:00:03.309 [music] s2 00:00:12.440 --> 00:00:14.320 I'm personally really excited about this talk. s3 00:00:14.320 --> 00:00:23.280 Uh, just by way of quick intro, I'm Adee, I'm co-founder and CEO at Reducto, uh, but most importantly, we have Raj here who is the global chief technology officer at Datasite. s4 00:00:23.280 --> 00:00:32.840 Um, the reason why I'm excited, uh, is I think Raj has probably one of the most interesting viewpoints for AI in that he founded company very early on in the vertical AI space. s5 00:00:32.840 --> 00:00:39.960 Um, got acquired into Datasite and then got to see both like subsidiary within Datasite to think about Datasite as a whole. s6 00:00:39.960 --> 00:00:49.320 Um, I got some of Raj's stats before this talk, um, where something like 60% of all transactions over $100 million happen through Datasite. s7 00:00:49.320 --> 00:00:50.360 That's correct, yeah. s8 00:00:50.360 --> 00:00:51.000 Okay. s9 00:00:51.000 --> 00:00:58.640 Um, if we're making a leaderboard, trillion plus tokens per year, I'm sure you have plenty of other cool numbers, um, but the goal for today is to make this conversation. s10 00:00:58.640 --> 00:01:05.440 We'll talk a bit about Raj's journey, um, some of the problems he's gone through, and then we'll try to open it up for Q&A as well for whatever you all are curious about. s11 00:01:05.440 --> 00:01:06.720 Sound good? s12 00:01:06.720 --> 00:01:07.640 Cool. s13 00:01:07.640 --> 00:01:12.360 Um, maybe to start, do you want to just walk through that arc and what it looked like behind the scenes? s14 00:01:12.360 --> 00:01:13.800 Yeah, for sure, appreciate that. s15 00:01:13.800 --> 00:01:16.640 Um, good to meet everyone, Raj Bakhru. s16 00:01:16.640 --> 00:01:27.000 Uh, my background's as a technologist, so I spent my entire career in development, started my career at Goldman, Highbridge, Kepos Capital, was building research and trading systems for quant funds and all sorts of fun things. s17 00:01:27.000 --> 00:01:32.480 Started a security firm that I sold to a private equity back firm, ended up running M&A and corporate strategy for them. s18 00:01:32.480 --> 00:01:42.360 And then started this vertical AI solution called Blue Flame in the M&A space, um, because we sort of knew that the impact to unstructured data was going to be pretty pretty big. s19 00:01:42.360 --> 00:01:48.520 Um, so we were building out for about a year, year and a half, and then Datasite approached us to acquire us. s20 00:01:48.520 --> 00:02:00.452 Um, and we started working with Reducto actually while we were separate, um, so before Datasite had acquired us, um, we were small startup, we were scrappy, uh, we built a lot of our own parsing s21 00:02:00.452 --> 00:02:03.932 and Doc Pipeline and ingestion pipeline. s22 00:02:03.932 --> 00:02:07.332 Um, had a great team, a great CTO, co-founders. s23 00:02:07.332 --> 00:02:09.332 We were trying to save every dollar we could. s24 00:02:09.332 --> 00:02:12.732 Um, so we when we built all of that, we're like, "This is great. s25 00:02:12.732 --> 00:02:13.532 It's working." s26 00:02:13.532 --> 00:02:14.452 Until it did. s27 00:02:14.452 --> 00:02:16.652 I mean, I'll talk a little bit about that journey. s28 00:02:16.652 --> 00:02:21.652 Um, and then as we got brought into Datasite, we were acquired obviously very early on. s29 00:02:21.652 --> 00:02:24.372 Our team was about 50 people at the time we got acquired. s30 00:02:24.372 --> 00:02:29.252 Um, the first thing that Datasite wanted to do with us was bring us into all of the data rooms. s31 00:02:29.252 --> 00:02:37.052 And if you guys aren't familiar with the data room concept, um, data rooms are basically where all of the transaction documents get stored when you're selling a company. s32 00:02:37.052 --> 00:02:44.332 So, as I deep mention, something like 60% of all transactions above a reasonable size threshold, 100 million, happen on Datasite. s33 00:02:44.332 --> 00:02:50.292 It's a place where people, bankers primarily, are sharing all of that data with private equity firms. s34 00:02:50.292 --> 00:02:54.492 Um, so it means that there's a massive unstructured data on Datasite, right? s35 00:02:54.492 --> 00:02:56.772 Billions of pages of content. s36 00:02:56.772 --> 00:02:58.492 Every major bank is using it. s37 00:02:58.492 --> 00:03:00.092 Every private equity firm is using it. s38 00:03:00.092 --> 00:03:02.132 Every diligence provider is using it. s39 00:03:02.132 --> 00:03:06.492 Over 500,000 monthly active users on Datasite accessing that content. s40 00:03:06.492 --> 00:03:13.132 Um, and that meant that adding AI solutions was obviously top of mind for the board and everyone else who was accessing the platform. s41 00:03:13.132 --> 00:03:16.412 Users were asking for it all uh day and day out. s42 00:03:16.412 --> 00:03:24.132 We had a great data science team um from the days where my CTO co-founder and I sort of scrapily built the ingestion pipeline. s43 00:03:24.132 --> 00:03:33.852 We had obviously moved on to Reducto, recognizing that we weren't going to have the level of quality, speed, um, recall, cost that we could get with Reducto. s44 00:03:33.852 --> 00:03:37.412 And I can talk about some of the numbers that led us to that that decision. s45 00:03:37.412 --> 00:03:41.732 Um, and why that panned out today with Datasite scale. s46 00:03:41.732 --> 00:03:42.852 But we needed to build fast. s47 00:03:42.852 --> 00:03:56.092 We needed to get all of the data rooms that Datasite had, 55,000 data rooms a year, um, parsed, ingested into our vector database so that we could build out our graph rank platform and answer questions within the data room s48 00:03:56.092 --> 00:03:58.532 uh that users were submitting at scale. s49 00:03:58.532 --> 00:03:59.972 Um, so that was a problem at hand. s50 00:03:59.972 --> 00:04:02.094 How How How do we build this into the VDR? s51 00:04:02.094 --> 00:04:04.214 They'd never done anything like this. s52 00:04:04.214 --> 00:04:08.654 They'd never taken their data their content out of their Azure platform. s53 00:04:08.654 --> 00:04:13.654 Historically, it's you know the walls were high around the data rooms as you can imagine. s54 00:04:13.654 --> 00:04:15.534 It's the world's most sensitive data. s55 00:04:15.534 --> 00:04:21.814 All of the most material material non-public info out there sitting in data sites virtual data rooms. s56 00:04:21.814 --> 00:04:23.894 Uh so we had to figure out a way to do this securely. s57 00:04:23.894 --> 00:04:33.814 We had to figure out a way to do it um in a place that it uh complied with all the regulatory compliance regimes that our clients have and pass on to us. s58 00:04:33.814 --> 00:04:38.134 All of the certifications, the ISO certifications and so on that that we had. s59 00:04:38.134 --> 00:04:45.174 Um so we spent a lot of time with the Deeds team in figuring out how to get there, how to make sure that it all of the content remained s60 00:04:45.174 --> 00:04:51.894 protected um and secured and get us all comfortable with leveraging uh Reducto's tooling. s61 00:04:51.894 --> 00:05:02.054 The reason that we went with Reducto's tooling as I sort of alluded to is we basically had to one move fast, but two we did a whole bunch of benchmarking actually with our data science team around s62 00:05:02.054 --> 00:05:06.494 sort of our scrappily built tooling for extraction and parsing and ingestion. s63 00:05:06.494 --> 00:05:11.414 And what I'll say about sort of that landscape is that um it's all edge cases. s64 00:05:11.414 --> 00:05:13.574 Like literally is everything is an edge case. s65 00:05:13.574 --> 00:05:23.174 I remember getting, you know, after we had built this we were working with some of the world's largest private equity firms and they would send to us um examples where our reg platform got a question wrong. s66 00:05:23.174 --> 00:05:26.193 And actually Carlos here will will remember some of these. s67 00:05:26.193 --> 00:05:26.414 [laughter] s68 00:05:26.414 --> 00:05:39.494 Um you'll dive into it and be like, well I got it wrong because the shade of blue on the legend of this pie chart didn't correlate perfect perfectly and the LLM just read it wrong even though we used the best vision model in the world. s69 00:05:39.494 --> 00:05:46.134 And actually if you were to pass that same thing as a pie chart or that pie chart um as a bar chart to that same model it would have gotten it right. s70 00:05:46.134 --> 00:05:50.054 It just the devil's in the details on all of this stuff. s71 00:05:50.054 --> 00:05:52.454 Um on Excel sheets, we had to deal with merged cells. s72 00:05:52.454 --> 00:05:55.894 And as you guys know, finance spreadsheets are the most complicated documents in the world. s73 00:05:55.894 --> 00:05:59.094 Like it is hard for a human to understand these things. s74 00:05:59.094 --> 00:06:07.575 Um so getting the models to actually understand the way to parse this this content out and understand the content was very very difficult. s75 00:06:07.575 --> 00:06:12.455 Um so we found that we were just spending an enormous amount of support time dealing with all these edge cases. s76 00:06:12.455 --> 00:06:19.375 And if I sort of think about the buy versus build trade-off, what do I want my limited set of high-quality engineers working on? s77 00:06:19.375 --> 00:06:27.695 It is not the support cases of making sure that we're extracting extracting content properly um when that's already been solved. s78 00:06:27.695 --> 00:06:30.615 Like you know, rebuilding the wheel doesn't make sense. s79 00:06:30.615 --> 00:06:37.055 Um so we we got a tremendous amount of benefit and leverage out of saying let's let's pass that on to Reducto. s80 00:06:37.055 --> 00:06:41.735 Let's let our engineers work on value-add revenue-generating business problems. s81 00:06:41.735 --> 00:06:48.215 Focus on building agents that our clients care about executing on the interface and the rag platform. s82 00:06:48.215 --> 00:06:53.535 Um building out graph rag was actually a big thing for us and we leveraged the outputs of Reducto in that process. s83 00:06:53.535 --> 00:07:02.735 So we wanted our data science team focused on that, not on how do I get the right data out of a spreadsheet um and dealing with all of the pains that we experienced on that. s84 00:07:02.735 --> 00:07:11.695 Then you you run into the pains of like if it's an XLSB file um macro-enabled, if it's a template file, it just it's endless. s85 00:07:11.695 --> 00:07:13.695 Literally the end edge cases are endless. s86 00:07:13.695 --> 00:07:17.655 So we're very thankful to Reducto for solving a lot of those problems for us. s87 00:07:17.655 --> 00:07:18.455 I appreciate that. s88 00:07:18.455 --> 00:07:27.615 Um at this point we've had like a year of context, but I think one of the most interesting things about your journey is that you were very early to the vertical AI app space. s89 00:07:27.615 --> 00:07:35.335 Um and the problems that you work on are hard um for many of the reasons that you mentioned, but also like things beyond that. s90 00:07:35.335 --> 00:07:41.535 Um and especially earlier with the model landscape as it existed, I'm sure it felt even more insurmountable. s91 00:07:41.535 --> 00:07:49.055 Um so I'd love to start maybe just by hearing a bit more of the pre-data site journey of like how you got started with Blue Flame, what you were hoping to achieve, s92 00:07:49.055 --> 00:07:52.095 um and how joining Data Site has kind of changed that. s93 00:07:52.095 --> 00:07:52.535 Yeah. s94 00:07:52.535 --> 00:07:59.535 Um so when when we launched our our startup startup, we had started we had sold the startup very early on before. s95 00:07:59.535 --> 00:08:02.935 The last startup, cybersecurity startup, we sold a year in. s96 00:08:02.935 --> 00:08:05.375 And we started this one saying we are not going to sell quickly. s97 00:08:05.375 --> 00:08:06.670 We are going to see this out. s98 00:08:06.670 --> 00:08:07.255 [laughter] s99 00:08:07.255 --> 00:08:12.097 And it's going to be a long time and we're going to be a unicorn and do all those fun things that we wanted to do. s100 00:08:12.097 --> 00:08:15.577 Um So, that was that was the the initial state. s101 00:08:15.577 --> 00:08:24.097 I brought in a lot of folks that we'd worked in worked with before and that's one of the nice things of being a second time founder is you kind of know the network, you know the people you want to bring in. s102 00:08:24.097 --> 00:08:32.457 So, we were very fortunate to bring in a lot of those people and start with a lot of the people that we knew and trusted and we could yell at each other and still be really good friends at the end of the day. s103 00:08:32.457 --> 00:08:34.697 Um and we knew each other's families and all that. s104 00:08:34.697 --> 00:08:38.110 So, that really nice experience to start as a second time. s105 00:08:38.110 --> 00:08:38.217 [snorts] s106 00:08:38.217 --> 00:08:43.097 We also had the ability thankfully to fund ourselves to a large extent this this second time. s107 00:08:43.097 --> 00:08:45.537 So, we bootstrapped for a while and started building. s108 00:08:45.537 --> 00:08:53.737 Now, that again made us do some things that might have been a little excessively cost-sensitive and not the most optimal decisions, but that's what you do as a startup s109 00:08:53.737 --> 00:08:55.817 and when it's your own money especially. s110 00:08:55.817 --> 00:08:58.257 So, we had a really great experience building that out. s111 00:08:58.257 --> 00:09:00.257 Uh we were actually going to do our first VC round. s112 00:09:00.257 --> 00:09:04.137 We had our LOI signed, term sheet signed. s113 00:09:04.137 --> 00:09:16.217 We had done sort of the dating process with all the VCs out there, the Sand Hill Road walk and all the New York ones and we were you know, a week away from sort of final final signatures s114 00:09:16.217 --> 00:09:26.497 and there was additional capacity on our round and they said you should go you know, see if these folks are interested and so we we approached Data Science and we're like, do you guys want to put in 5 million in this round? s115 00:09:26.497 --> 00:09:32.737 And we demoed and showed them a bit of what we were doing and they said to us actually, we just want to buy the whole company. s116 00:09:32.737 --> 00:09:35.217 We're like, okay and literally happened over a dinner. s117 00:09:35.217 --> 00:09:41.017 Um Like whatever meeting in the office, subsequent dinner that night and then they told us that. s118 00:09:41.017 --> 00:09:44.057 So, we we very quickly sort of figured it out. s119 00:09:44.057 --> 00:09:49.857 And the reason that we thought it was really compelling was if you know, this was about a year and a half ago now. s120 00:09:49.857 --> 00:09:54.617 At that point sort Cloud and Office Suite hadn't taken off. s121 00:09:54.617 --> 00:09:57.017 You sort of have to go back to remember where we were a year and a half ago. s122 00:09:57.017 --> 00:10:00.417 It feels like forever ago and the world was entirely different. s123 00:10:00.417 --> 00:10:03.497 Agentic coding hadn't taken off to the level it has now. s124 00:10:03.497 --> 00:10:06.657 Um So, we knew that we had to raise a massive amount of money. s125 00:10:06.657 --> 00:10:14.018 Our competitors in the vertical AI space, Um, of whom worked with Reducto, had raised hundreds of millions of dollars, literally. s126 00:10:14.018 --> 00:10:21.738 Um, so we kind of knew that if we're going to remain competitive, we were going to be raising hundreds of millions of dollars, we would be taking masses mass dilution in doing that. s127 00:10:21.738 --> 00:10:31.058 Um, we also knew that staying competitive against the frontier models was going to be increasingly difficult, particularly as they were starting to launch vertical strategies. s128 00:10:31.058 --> 00:10:36.658 And at that point, sort of OpenAI for FS and Cloud for FS hadn't been fully announced, but there were some rumors around it. s129 00:10:36.658 --> 00:10:39.618 They'd made some some ins- insinuations about it. s130 00:10:39.618 --> 00:10:47.418 Um, and we knew that just to remain competitive and differentiated, that we would have to bring in new types of data. s131 00:10:47.418 --> 00:10:57.378 And to us, the VDRs were this incredible valuable resource of prop data that had never been opened up to anyone in the world, and we would be the only ones with access to it at that time. s132 00:10:57.378 --> 00:10:58.938 So, that was super compelling. s133 00:10:58.938 --> 00:11:03.258 Um, Datasite also owns a private company database firm called Grata. s134 00:11:03.258 --> 00:11:14.138 Has about 20 million private companies in its database, has transaction data, contacts, etc. Used very heavily in private equity and banking worlds for, um, sourcing and business development and origination. s135 00:11:14.138 --> 00:11:18.098 So, how do I find add-ons for my platform portfolio company? s136 00:11:18.098 --> 00:11:21.098 Uh, very classic Grata type query. s137 00:11:21.098 --> 00:11:23.378 That data would be brought into a BlueFlame as well. s138 00:11:23.378 --> 00:11:30.298 So, now we had these two really nice proprietary data sets feeding us in a vertical AI space where no one else would have that. s139 00:11:30.298 --> 00:11:31.978 Really nice and differentiated. s140 00:11:31.978 --> 00:11:33.338 And that's played out really nicely. s141 00:11:33.338 --> 00:11:39.548 So, over the over the course of the last year and a half, we've been bringing those really tightly together and um, that's s142 00:11:39.548 --> 00:11:39.738 [snorts] s143 00:11:39.738 --> 00:11:45.098 that's been a really nice story for us to combine engineering teams, combine data, combine solutions. s144 00:11:45.098 --> 00:11:46.498 Um, and then we actually benchmark that. s145 00:11:46.498 --> 00:11:57.258 Like, the the ability or the the fact that we brought the data closer to the models saves on token costs, it saves on run run times, um, and increases quality of what we're doing. s146 00:11:57.258 --> 00:12:01.778 So, that's been a that's been, um, quite a nice thesis to see pan out. s147 00:12:01.778 --> 00:12:05.578 Um, all of that data, obviously, requires scale. s148 00:12:05.578 --> 00:12:09.738 So, that's that's again where Reducto comes in and helps us with a lot of that. s149 00:12:09.738 --> 00:12:12.058 Ingestion processing and and so on. s150 00:12:12.058 --> 00:12:18.899 Datasite over the course of the last year and a half since we've joined them has grown, I want to say about 40% top line. s151 00:12:18.899 --> 00:12:24.739 Um so just the amount of content that's coming into the platform is is growing pretty quickly and pretty massively. s152 00:12:24.739 --> 00:12:28.219 Um the demand on our AI solutions have gone up. s153 00:12:28.219 --> 00:12:38.099 So now people aren't just saying I want to ask questions against documents, the agents are obviously iterating against those documents, doing much more fanciful diligence analysis, or generating PowerPoint decks, s154 00:12:38.099 --> 00:12:40.059 IC memos, and so on. s155 00:12:40.059 --> 00:12:44.579 Um so it's pretty fullsome suite of what they can what they can accomplish with the solution together. s156 00:12:44.579 --> 00:12:45.339 Yeah. s157 00:12:45.339 --> 00:12:51.979 Um it's an interesting position to be in in that I I I think you have one of the most compelling data sets in the industry. s158 00:12:51.979 --> 00:12:55.819 Um you're in a position now where obviously it's a global workforce. s159 00:12:55.819 --> 00:12:58.499 We were talking about all the different offices and all of that. s160 00:12:58.499 --> 00:13:01.419 Um and model capabilities have gotten a lot better. s161 00:13:01.419 --> 00:13:08.379 Um so more broadly like how do you think about what you invest in internally versus what you build uh and buy externally as well? s162 00:13:08.379 --> 00:13:13.859 Um and also just like what sorts of things are top of mind as you think about the coming year? s163 00:13:13.859 --> 00:13:14.579 Yeah. s164 00:13:14.579 --> 00:13:15.539 Um great question. s165 00:13:15.539 --> 00:13:17.659 So we think a lot about the build versus buy, right? s166 00:13:17.659 --> 00:13:19.819 We we work with a lot of vendors. s167 00:13:19.819 --> 00:13:21.939 We spend a lot with a lot of vendors. s168 00:13:21.939 --> 00:13:24.539 Um and that's an intentional decision. s169 00:13:24.539 --> 00:13:25.659 Um if s170 00:13:25.659 --> 00:13:25.699 [snorts] s171 00:13:25.699 --> 00:13:35.379 I think about what I want my my most talented people working on, it's not things that vendors can easily handle and are dedicated to handling and solving every day. s172 00:13:35.379 --> 00:13:39.299 Um it's really hard to find really high quality good engineers, right? s173 00:13:39.299 --> 00:13:49.939 The fact of the matter is where everyone in market is trying to hire the top tier talent of engineering talent right now because what we're seeing, what you guys are all probably seeing it is with AI coding, s174 00:13:49.939 --> 00:13:59.459 the folks who get the most leverage and value out of AI coding are the folks who have really strong mental models of code bases, really understand engineering principles, have done it for a long time. s175 00:13:59.459 --> 00:14:07.059 Um so they've all been 10xed, you know, over their normal 10x uh with AI with agentic coding. s176 00:14:07.059 --> 00:14:09.299 The middle layer, so-so. s177 00:14:09.299 --> 00:14:17.381 Um the bottom layer is actually getting a getting a lot of leverage and now they don't they don't sort of uh have the burden of trying to figure out documentation that doesn't exist in all of that. s178 00:14:17.381 --> 00:14:19.821 The LLMs can build that dynamically for them. s179 00:14:19.821 --> 00:14:27.381 Um so we're seeing sort of a barbell effect where we want to hire really really strong talent and really junior talent and build that up. s180 00:14:27.381 --> 00:14:31.381 Um where I want those people working is obviously not on support cases. s181 00:14:31.381 --> 00:14:33.581 It's not on things that are already solved. s182 00:14:33.581 --> 00:14:37.701 Um it's on the hardest and most pressing problems for the business. s183 00:14:37.701 --> 00:14:45.621 At our scale, so Data sites not a billion-dollar top line, but general range of a billion-dollar top line. s184 00:14:45.621 --> 00:14:55.181 Um at our scale to grow 20% a year, which is our ambition or board's ambition, uh we need to be generating $200 million of revenue incremental revenue every year. s185 00:14:55.181 --> 00:15:03.661 And if you ask yourself how you're going to go do that, um it's going to be by putting new AI products out in market, leveraging, getting on top of token flow, s186 00:15:03.661 --> 00:15:07.101 expanding solution set that you're delivering to your clients. s187 00:15:07.101 --> 00:15:11.621 Um reinventing the wheel is certainly going to get in the way of that. s188 00:15:11.621 --> 00:15:21.701 Um having internal teams trying to uh iterate on some of the existing solutions that are that are already out there and deal with the support burden of those is not the best use of my talent. s189 00:15:21.701 --> 00:15:24.141 I have about 250 engineers in the org. s190 00:15:24.141 --> 00:15:28.941 So if I allocate out where I want those 250 focus, we have five major initiatives across the firm. s191 00:15:28.941 --> 00:15:38.901 Um those are five revenue-generating projects that we're putting out, new new builds or consolidations of tools, new commercial or monetization models. s192 00:15:38.901 --> 00:15:47.221 Um I want those 250 people split across those five and I want as few working on the layers underneath that as possible. s193 00:15:47.221 --> 00:15:54.661 And if I can outsource everything on the under the layers underneath that, that's an ideal um ideal split of time for my my team. s194 00:15:54.661 --> 00:15:55.821 Yeah. s195 00:15:55.821 --> 00:15:58.501 Uh before I turn over for questions, I realize we have 2 minutes. s196 00:15:58.501 --> 00:16:01.581 I guess maybe one last thing is like where do you find friction today? s197 00:16:01.581 --> 00:16:05.061 Um what are problems that you would hope people in the audience would solve for you? s198 00:16:05.061 --> 00:16:11.341 Um we still struggle with getting the models to do exactly what we want all the time. s199 00:16:11.341 --> 00:16:22.822 Um we do a lot of working of all of the models and actually, you know, if you you back to pre-Reducto world, we wasted a lot of time on trying to figure out the right model for the right type of chart for the right type of s200 00:16:22.822 --> 00:16:24.222 uh document that we're passing in. s201 00:16:24.222 --> 00:16:33.262 So, we've we've gotten rid of a lot of that, but we still deal with it on the agent front, trying to benchmark out, you know, why is 61 Soul tool spam doing tool call spamming s202 00:16:33.262 --> 00:16:40.742 when Opus 55 is not, but then 61 Soul is so much cheaper that we want to move in that direction, and we're trying to get it to go to go there. s203 00:16:40.742 --> 00:16:47.382 Um so, we do a lot of sort of the optimization of the three factors we care about, which is cost, speed, and quality. s204 00:16:47.382 --> 00:16:51.782 Quality is top top of mind for everyone in our space, right? s205 00:16:51.782 --> 00:16:56.262 Like, any finance user will pay any amount of money to have the best answer. s206 00:16:56.262 --> 00:17:00.502 Um so, we need to make sure we're always optimizing on quality while trying to solve the other two. s207 00:17:00.502 --> 00:17:01.382 Yeah. s208 00:17:01.382 --> 00:17:02.382 Makes sense. s209 00:17:02.382 --> 00:17:03.342 Any questions from the room? s210 00:17:03.342 --> 00:17:03.862 Yeah. s211 00:17:03.862 --> 00:17:09.862 Can you zoom a little bit about the integration between Data Science and Reducto? s212 00:17:09.862 --> 00:17:15.822 Yeah, so we have an environment where Reducto's um operating in our environment against all of our content. s213 00:17:15.822 --> 00:17:17.902 We have bill- billions of pages of content. s214 00:17:17.902 --> 00:17:25.902 We have very big spurts of content when bakers are dropping in large data sets, potentially up to a terabyte of data can get dropped in on a moment's notice. s215 00:17:25.902 --> 00:17:29.422 Uh so, we have an ingestion pipeline that takes all of that, passes it through Reducto. s216 00:17:29.422 --> 00:17:30.622 We get that back. s217 00:17:30.622 --> 00:17:35.862 We We leverage Turbo Buffer as our vector database, so we're passing it off to Turbo Buffer. s218 00:17:35.862 --> 00:17:38.022 Um and then we build our graph rag on top of that. s219 00:17:38.022 --> 00:17:44.782 We're executing all of our graph rag queries using Prop Python code against all of that. s220 00:17:45.142 --> 00:17:46.462 Yeah. s221 00:17:46.462 --> 00:17:49.142 Um I'm on the sell side of things. s222 00:17:49.142 --> 00:17:52.902 Um we're we are actually customers of Data Science. s223 00:17:52.902 --> 00:17:53.557 sell side is. s224 00:17:53.557 --> 00:17:54.462 [laughter] s225 00:17:54.462 --> 00:17:57.222 Um curious about Reducto. s226 00:17:57.222 --> 00:18:07.502 So, we're sitting on a transaction like, we have a file system with a ton of transactions, and we want to make that available to agents, but there's a way of s227 00:18:07.502 --> 00:18:08.822 obviously it's MNPI. s228 00:18:08.822 --> 00:18:13.022 We want to anonymize that before it makes its way into something like Reducto. s229 00:18:13.022 --> 00:18:13.342 Hm. s230 00:18:13.342 --> 00:18:18.022 We're going to either build this or buy this, so very helpful conversation. s231 00:18:18.022 --> 00:18:25.543 That being said, giving it to the agents You You mentioned before that you built a custom eval. s232 00:18:25.543 --> 00:18:30.103 And so I I'm curious about how Y'all went about this eval. s233 00:18:30.103 --> 00:18:30.463 Yep. s234 00:18:30.463 --> 00:18:32.983 Potentially open sourced. s235 00:18:32.983 --> 00:18:36.263 Um but like curious about how you guys s236 00:18:36.263 --> 00:18:37.743 Yeah. s237 00:18:37.743 --> 00:18:45.103 So our custom eval that's related to We have a few custom evals, but the one that's most relevant to Reducto is what we call VDR bench. s238 00:18:45.103 --> 00:18:46.023 We haven't open sourced it. s239 00:18:46.023 --> 00:18:50.263 We're actually thinking about doing that, thinking about partnering with OpenAI on a paper around it. s240 00:18:50.263 --> 00:19:02.663 Um what we're doing with VDR bench is we basically take a common virtual data room of hundreds of documents, um and we ask a few hundred questions against that data room with the known set of answers against those. s241 00:19:02.663 --> 00:19:06.263 Um we run that every time we make changes to to pretty much anything. s242 00:19:06.263 --> 00:19:12.383 Um the scores on that we expect to be higher than any of the uh frontier labs directly. s243 00:19:12.383 --> 00:19:18.423 So if you were to, for example, drop that same set of files into cloud projects or OpenAI projects, we need to make sure that we're beating that. s244 00:19:18.423 --> 00:19:26.303 Um and a lot of the reason that we beat that our data science team benchmarks of our recall rate at, you know, improving 22% because of our Reducto parsing. s245 00:19:26.303 --> 00:19:29.943 Um you know, a lot of that's because of the the way we pull out the content. s246 00:19:29.943 --> 00:19:31.503 It's actually not the other side of it. s247 00:19:31.503 --> 00:19:38.943 A lot of people think that sort of all the work needs to be done on the other side, the answering side, and most of the work actually is on the extraction side. s248 00:19:38.943 --> 00:19:42.743 You get that right, and everything else can follow.