WEBVTT NOTE Sentence-level transcript of https://www.youtube.com/watch?v=W4pnzOURwHQ NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/W4pnzOURwHQ.html. A cue ends where the next begins, or 2 s after its last word. s1 00:00:01.309 --> 00:00:03.309 [music] s2 00:00:12.600 --> 00:00:16.480 All right, thank you for joining the last workshop of the day. s3 00:00:16.480 --> 00:00:17.440 So, I'm Dennis. s4 00:00:17.440 --> 00:00:24.720 I'll be sharing a little bit about insurance claims data and how powerful it can be for enterprise-wide AI programs. s5 00:00:24.720 --> 00:00:27.080 So, let's get started. s6 00:00:27.400 --> 00:00:30.400 So, who here works in insurance? s7 00:00:30.480 --> 00:00:31.280 Few people. s8 00:00:31.280 --> 00:00:33.800 Who here wants to work in insurance? s9 00:00:33.800 --> 00:00:35.040 Okay, interesting. s10 00:00:35.040 --> 00:00:35.440 Few people. s11 00:00:35.440 --> 00:00:36.960 All right. s12 00:00:36.960 --> 00:00:47.960 So, insurance is a very interesting sort of sector of our economy where people sort of find their way working there, but it powers such a important part of our ecosystem. s13 00:00:47.960 --> 00:00:51.560 So, we're going to talk a little bit about insurance as a whole. s14 00:00:51.560 --> 00:00:55.360 And we're ultimately going to focus this talk on how do we improve our lives in general? s15 00:00:55.360 --> 00:01:03.000 So, we'll focus on medical claims, we'll improve the claim and patient outcomes as a focus of this talk, and I'll keep bringing that back. s16 00:01:03.000 --> 00:01:11.440 I think often times in a lot of fintech sort of finance insurance discussions, sometimes things can get a little bit too business focused. s17 00:01:11.440 --> 00:01:15.400 So, I always want everybody to to focus on this aspect. s18 00:01:15.400 --> 00:01:16.640 So, a little bit about myself. s19 00:01:16.640 --> 00:01:17.320 I'm Dennis. s20 00:01:17.320 --> 00:01:19.640 I lead the AI program at Wisdocs. s21 00:01:19.640 --> 00:01:23.080 We help companies with their complex medical claim processing. s22 00:01:23.080 --> 00:01:29.560 Been with the role in in the role for 2 years now, and we've seen a huge amount of transformation since 2024. s23 00:01:29.880 --> 00:01:34.680 So, you might have seen some of the news coverage in terms of medical coverage and insurance in general. s24 00:01:34.680 --> 00:01:43.760 The whole GLP-1 transformation in our economy, in our health care environment has changed how insurance is pricing things as well. s25 00:01:43.760 --> 00:01:55.000 We also see that spouse insurance and broadly insurance coverage has continued to evolve with increasing costs and other factors that change the the approach of insurance. s26 00:01:55.040 --> 00:01:57.920 So, the talk is going to focus on three main chapters. s27 00:01:57.920 --> 00:02:00.960 The first one is on the technology transformation that we have. s28 00:02:00.960 --> 00:02:08.560 The second is I'll go a little bit into the insurance space in terms of some of the call it nitty-gritty, but sort of a a brief introduction there. s29 00:02:08.560 --> 00:02:14.720 And then also I'll talk about how AI is going to help make insurance and the patient experience even better. s30 00:02:14.720 --> 00:02:17.560 Let's talk about technology transformation. s31 00:02:17.720 --> 00:02:26.880 So, we all probably in our work have started some form of digital transformation within our companies, whether we work in smaller startups or with in larger enterprises. s32 00:02:26.880 --> 00:02:30.800 Now, the challenge with that is that we never finished this transformation. s33 00:02:30.800 --> 00:02:40.840 AI hit us in 2023 and 2024, but most companies have not finished their cloud migrations, have not finished leveraging big data to its full potential, have not deployed a clean data lake, right? s34 00:02:40.840 --> 00:02:44.440 Who here has a data lake they're proud of? s35 00:02:44.560 --> 00:02:45.040 All right. s36 00:02:45.040 --> 00:02:46.120 Very very few people. s37 00:02:46.120 --> 00:02:49.160 Is it even possible to have a data lake you're proud of, right? s38 00:02:49.160 --> 00:02:57.880 And on top of that, we're now getting all this hype about AI, and we need to really make sure that we have the other prerequisites down before we can actually deploy s39 00:02:57.880 --> 00:03:00.120 real applications. s40 00:03:00.920 --> 00:03:06.920 So, in insurance and many other industries, paper was the big way of processing things, right? s41 00:03:06.920 --> 00:03:09.600 You'd get a stack of papers of going through a medical claim. s42 00:03:09.600 --> 00:03:10.680 You'd organize it. s43 00:03:10.680 --> 00:03:12.920 You'd have these giant tables full of them. s44 00:03:12.920 --> 00:03:15.920 You'd use cards to put in certain sections. s45 00:03:15.920 --> 00:03:24.480 And this is very difficult to actually get any anything outside of the claim itself, because how do you do cross-claim analysis when everything is paper, right? s46 00:03:24.480 --> 00:03:27.640 You could put as many folders as you want, but it's very difficult. s47 00:03:27.640 --> 00:03:29.760 Now, we still have not exited the paper era. s48 00:03:29.760 --> 00:03:38.840 There are still a number of practices that they use paper around the world, whether it's in the doctor's office, in in legal departments, or within insurance companies themselves. s49 00:03:38.840 --> 00:03:42.680 So, we're still not over paper, but we're getting close. s50 00:03:43.120 --> 00:03:46.840 Now, in terms of digitizing, we are still also not over the fax machine. s51 00:03:46.840 --> 00:03:48.920 The The fax market is continuing to grow. s52 00:03:48.920 --> 00:03:52.440 It's over $7 billion globally, which is always surprising. s53 00:03:52.440 --> 00:03:58.240 I referenced this in my talk at An Engineer SF last year and it has not changed. s54 00:03:58.240 --> 00:03:59.920 Fax is still being used. s55 00:03:59.920 --> 00:04:13.320 Now, what we're doing now is we're jerry-rigging faxes into our digital systems where we can convert them in a fairly clean format, but it still does not solve the problem of how we actually exchange information in the health care system. s56 00:04:13.320 --> 00:04:24.200 Now, if we think about how quickly technology has been adopted, EHRs, electronic health records, uh we're only at 8% adoption in 2008. s57 00:04:24.200 --> 00:04:30.440 And it only with the the COVID pandemic did we actually hit the '80s and '90s in EHR adoption. s58 00:04:30.440 --> 00:04:32.880 And again, many hospitals still run on paper. s59 00:04:32.880 --> 00:04:40.240 So, we're still not fully through this digitization, digital transformation era, and yet everybody wants to do AI. s60 00:04:41.440 --> 00:04:52.680 If we think about the types of forms people receive when when doing claims processing, these are some three sample forms from three large states for workers compensation. s61 00:04:52.680 --> 00:05:01.480 So, workers compensation is if you get hurt at work, you file a claim saying, "Hey, I hurt my arm falling down or I was driving a forklift and then something went wrong." s62 00:05:01.480 --> 00:05:03.520 And none of these forms look the same, right? s63 00:05:03.520 --> 00:05:08.440 So, within 50 different states in the US, you have 50 different types of forms generally. s64 00:05:08.440 --> 00:05:10.000 They all ask for different information. s65 00:05:10.000 --> 00:05:12.040 They're not standardized. s66 00:05:12.040 --> 00:05:20.680 They're fairly standardized in that every form that comes in looks the same, so you can have some determinism there, but even then the information is different. s67 00:05:20.680 --> 00:05:31.040 So, even as a country in the US, let alone any other country in the world, we don't capture information in the same way and these versions sometimes change as well. s68 00:05:32.000 --> 00:05:34.053 If we talk about what people s69 00:05:34.053 --> 00:05:34.200 [snorts] s70 00:05:34.200 --> 00:05:37.000 care about in the insurance space is a loss ratio, right? s71 00:05:37.000 --> 00:05:41.120 So, how many premiums are you collecting and how much money are you paying out? s72 00:05:41.120 --> 00:05:49.720 So, I'll talk a little bit more about the these numbers here and what they mean, but in general, this is what drives the bottom line of an organization, right? s73 00:05:49.720 --> 00:05:57.480 When you report a quarterly earnings or an annual earnings, you're going to be emphasizing what is your loss ratio, how well can you predict risk, how well can you underwrite, s74 00:05:57.480 --> 00:06:01.400 how how well are you managing fraud, how well are you managing your expenses. s75 00:06:01.400 --> 00:06:04.080 And so companies want to keep costs down and make more money, right? s76 00:06:04.080 --> 00:06:05.920 That's just capitalism in general. s77 00:06:05.920 --> 00:06:16.880 Um and if we think about some of the statistics here, we we generally uh hover around the 50% um loss ratio here in terms of the amount of money we have to pay out for insurable claims s78 00:06:16.880 --> 00:06:20.360 uh compared to the premiums we get. s79 00:06:21.760 --> 00:06:28.520 Now I'm I'm introducing all of these sort of pieces of context and I've only said AI once so far in my talk. s80 00:06:28.520 --> 00:06:36.000 And it's really important again to think about the problem that we're trying to solve and then only choose AI if it's the right solution. s81 00:06:37.080 --> 00:06:42.320 Now for AI uh to get proper AI solutions, we need to go through the right amount of transformation, right? s82 00:06:42.320 --> 00:06:44.520 We need to have the right goals in mind. s83 00:06:44.520 --> 00:06:47.520 We need to have a team that can actually articulate what we want to do with AI. s84 00:06:47.520 --> 00:06:52.080 We can't just have people get on stage and say AI AI AI, right? s85 00:06:52.080 --> 00:07:01.040 We need the right people skills to actually accomplish this transformation and we need the right frameworks in place, whether it's governance, security, infrastructure management, and data. s86 00:07:01.040 --> 00:07:02.440 Now there's a question here. s87 00:07:02.440 --> 00:07:11.360 Um for those working at an enterprise context, who here could get a database or a VM within a week if they requested it? s88 00:07:11.360 --> 00:07:11.760 Okay. s89 00:07:11.760 --> 00:07:13.560 Who could not? s90 00:07:13.560 --> 00:07:15.040 Okay, that's progress. s91 00:07:15.040 --> 00:07:19.400 Who here could deploy an application to production in a week? s92 00:07:19.640 --> 00:07:21.560 A new application. s93 00:07:21.560 --> 00:07:21.720 Okay. s94 00:07:21.720 --> 00:07:23.240 Who could not? s95 00:07:23.240 --> 00:07:23.440 Okay. s96 00:07:23.440 --> 00:07:25.440 So So we we still need some progress. s97 00:07:25.440 --> 00:07:28.080 Who can choose a new AI vendor in a week? s98 00:07:28.080 --> 00:07:28.800 Ah, there we go. s99 00:07:28.800 --> 00:07:30.360 No hands. s100 00:07:30.360 --> 00:07:30.560 Right. s101 00:07:30.560 --> 00:07:34.600 So it still takes time for us to go through all these different processes. s102 00:07:34.600 --> 00:07:42.240 Uh they're certainly better than they were 10, 20 years ago in the digital transformation component, but a lot of companies still get stuck in governance and security. s103 00:07:42.240 --> 00:07:46.480 And a lot of those have to do with with the factors here. s104 00:07:47.280 --> 00:07:51.680 Who here has been asked for some kind of query like this from an executive? s105 00:07:51.680 --> 00:07:55.120 Give me some data set that is perfect data. s106 00:07:55.120 --> 00:07:55.520 Right? s107 00:07:55.520 --> 00:07:55.539 A s108 00:07:55.539 --> 00:07:55.560 [laughter] s109 00:07:55.560 --> 00:07:56.240 few people, right? s110 00:07:56.240 --> 00:07:57.280 We we we've seen this before. s111 00:07:57.280 --> 00:07:59.600 Hey, can you pull that report for me quickly? s112 00:07:59.600 --> 00:08:05.040 Um it's much easier now with AI, but you need to assume that that data exists, right? s113 00:08:05.040 --> 00:08:08.680 So, if we come up with a more complex query here, right? s114 00:08:08.680 --> 00:08:17.360 Uh let's say that we we have a claim and for a given patient and their condition, uh in this case, uh an ankle injury, we want to make sure that s115 00:08:17.360 --> 00:08:24.280 we can assign them to a doctor who has top 50% tile uh patient outcomes, right? s116 00:08:24.280 --> 00:08:27.440 Within some recent time frame. s117 00:08:27.760 --> 00:08:33.960 Now, this query is fairly complex, but if you could actually just write this query on your data set and get that result, right? s118 00:08:33.960 --> 00:08:35.200 I'd be amazed, right? s119 00:08:35.200 --> 00:08:39.039 Data is very very challenging in an enterprise environment. s120 00:08:39.039 --> 00:08:49.480 So, for us to get these kind of outcomes and get these kinds of insights, we have to structure our data systems in a in a in a very well-defined way. s121 00:08:50.040 --> 00:08:53.320 So, let's talk a little bit about about medical claims. s122 00:08:53.320 --> 00:08:59.360 So, in general, I've grouped uh insurance based on the the NAIC uh classifications. s123 00:08:59.360 --> 00:09:04.720 We have property and casualty, life, accident, and health, and sort of broader health uh insurance. s124 00:09:04.720 --> 00:09:10.520 I'll actually be talking about property and casualty, since there the claims are typically the the most complex. s125 00:09:10.600 --> 00:09:14.320 If you think about P&C, um I've mainly worked on the casualty side. s126 00:09:14.320 --> 00:09:21.680 So, uh auto uh insurance, uh various types of liability, general liability, product liability, workers' compensation. s127 00:09:21.680 --> 00:09:26.560 There's also the whole property side, uh which is its own uh set of complexities. s128 00:09:26.560 --> 00:09:29.760 But, generally, this is a $1 trillion market, right? s129 00:09:29.760 --> 00:09:34.480 Uh based on on premiums, and reinsurance is its own kind of forms. s130 00:09:34.480 --> 00:09:39.120 So, if we We about these blue boxes, many of these handle complex medical claims. s131 00:09:39.120 --> 00:09:46.400 Somebody gets hurt because a product is defective, somebody gets into a car accident, uh somebody gets hurt at work. s132 00:09:47.200 --> 00:09:52.600 Now, adjudicating a claim and getting a response back to a patient or a claimant takes a long time. s133 00:09:52.600 --> 00:09:54.840 In the best case scenario, it takes a few weeks. s134 00:09:54.840 --> 00:09:57.920 In the worst case scenario, it can drag on for a few years. s135 00:09:57.920 --> 00:10:05.520 Depending on complex the the scenario is, depending on complex the regulatory environment is, depending on how litigated the environment is. s136 00:10:05.520 --> 00:10:11.480 And even once you get the the medical services delivered, the claim can keep dragging on and on. s137 00:10:11.480 --> 00:10:20.440 So, this is not a simple process, and I want all of us to imagine how much more effective the the healthcare system could be if we could shrink this by a factor of 10, right? s138 00:10:20.440 --> 00:10:29.080 You get hurt at work, you you submit the application, uh you go through that process, and everything is digital, and everything is smooth. s139 00:10:30.560 --> 00:10:33.240 So, why does it take so long? s140 00:10:33.240 --> 00:10:40.560 There's a lot of handoff points with getting even the medical side, not accounting for the for legal side, the actual insurance side as well, right? s141 00:10:40.560 --> 00:10:49.640 So, when you when you get hurt uh in one of these scenarios and have to be insured for a complex claim, you have to get the the diagnosis, you have to go to various specialists. s142 00:10:49.640 --> 00:10:51.680 Sometimes specialists disagree, right? s143 00:10:51.680 --> 00:10:58.280 Then you have to make sure that you have the right coverage, uh whether it's primary care or one of these uh additional forms of liability. s144 00:10:58.280 --> 00:11:03.000 You have utilization reviews where doctors have to agree with one another that a procedure is necessary. s145 00:11:03.000 --> 00:11:05.160 Uh we have treatments, we have billing. s146 00:11:05.160 --> 00:11:06.760 You try to figure out what your bill means, right? s147 00:11:06.760 --> 00:11:08.880 With the explanation of benefits and coverage. s148 00:11:08.880 --> 00:11:11.520 You get the payments, and you have continued treatments. s149 00:11:11.520 --> 00:11:23.600 Uh there's so many additional bubbles here that can be added and forks in the road, um depending on the steps, but in general, it it is not a simple process. s150 00:11:24.920 --> 00:11:27.160 And along the way, lots of things can go wrong. s151 00:11:27.160 --> 00:11:39.480 Uh I just listed around 15 that that come to mind immediately, but this has to do with the HR situations and insurance companies with claim adjusters having high turnover, high retirement rates, has to do with technology s152 00:11:39.480 --> 00:11:42.960 of people reprocessing the same data over and over again. s153 00:11:42.960 --> 00:11:54.560 Has to do with the types of situations we end up with where claimants are are stressed and sometimes make mistakes and are in the position where they're not acting rationally, right? s154 00:11:54.560 --> 00:11:59.440 Cuz if you're you've gone through a traumatic car accident, you're not going to be the most rational person. s155 00:11:59.440 --> 00:12:02.800 So, let's talk about how technology can help us. s156 00:12:05.040 --> 00:12:09.160 So, let's go through a workers' compensation claim. s157 00:12:09.160 --> 00:12:12.040 Has anybody seen a workers' comp claim before? s158 00:12:12.040 --> 00:12:13.680 Few people, okay. s159 00:12:13.680 --> 00:12:20.960 So, typically when somebody gets injured, they have to report it within the workplace and then make a filing of that as well. s160 00:12:20.960 --> 00:12:26.440 Those three forms I showed, the Texas, New York, and California ones, those are typical intake forms. s161 00:12:26.440 --> 00:12:33.840 And you go through this process of actually visiting various people, whether it's an emergency room, whether you have additional treatments, and so forth. s162 00:12:33.840 --> 00:12:39.600 And you don't always have your claim reviewed until you get claims later on. s163 00:12:39.600 --> 00:12:42.800 Now, as you go through that, there's various types of claims. s164 00:12:42.800 --> 00:12:50.360 Here I generated some synthetic ones as an example, but they're they're fairly close to what you have in in production. s165 00:12:50.360 --> 00:12:59.120 So, you go through you go through all these types of treatments, you get different types of procedures, you might be in network, you might be out of network, you might be traveling for these, s166 00:12:59.120 --> 00:13:03.400 and sometimes you have conflicting information as well. s167 00:13:03.640 --> 00:13:08.280 And as we go through each of these processes, all the documents look different. s168 00:13:08.280 --> 00:13:12.760 Every hospital has its own way of structuring a bill, right? s169 00:13:12.760 --> 00:13:15.720 Every doctor has a different way to write a progress note. s170 00:13:15.720 --> 00:13:18.440 It it's not consistent in any way, shape, or form. s171 00:13:18.440 --> 00:13:23.240 So, this is why the data is unstructured and many times it's not in an electronic feed. s172 00:13:23.240 --> 00:13:25.680 It's a PDF, it's an email, it's a fax. s173 00:13:25.680 --> 00:13:32.520 So, you have all these technology complexities that layer on top of all the business complexity that lies underneath. s174 00:13:33.600 --> 00:13:35.120 Then you have something like this, right? s175 00:13:35.120 --> 00:13:39.720 Somebody does an assessment and there's three different results, right? s176 00:13:39.720 --> 00:13:43.440 For let's say the range of motion that somebody has. s177 00:13:43.440 --> 00:13:48.040 How are you going to resolve this in order to adjudicate the claim? s178 00:13:48.880 --> 00:13:52.200 Now we haven't even layered the the other portion on top of this. s179 00:13:52.200 --> 00:13:56.960 While these medical processes are going through, there's also the risk of litigation. s180 00:13:56.960 --> 00:14:04.440 So with an insurance, typically insurance is on the the defendant side of a litigated claim and you have the plaintiff side. s181 00:14:04.440 --> 00:14:11.000 Now the plaintiff side is incentivized to handle as much volume as possible and to get as big of a settlement as possible. s182 00:14:11.000 --> 00:14:15.960 So you actually get into a situation where you have two different incentive structures. s183 00:14:15.960 --> 00:14:22.840 So on the plaintiff side, go as quickly as possible, make the most the biggest claims possible, put up giant billboards everywhere. s184 00:14:22.840 --> 00:14:25.040 We saw those when we were flying in probably, right? s185 00:14:25.040 --> 00:14:26.600 all over the place. s186 00:14:26.600 --> 00:14:32.680 And the plaintiff side has actually adopted AI a lot more quickly compared to the insurance or defendant side. s187 00:14:32.680 --> 00:14:37.680 Because you get a percentage of how fast you move and how big your claim is. s188 00:14:37.680 --> 00:14:48.000 Now the range of the actual percentage of compensation for for lawyers depending on your jurisdiction can range from 9 to 15% if you're in California for workers comp fees. s189 00:14:48.000 --> 00:14:52.600 In other jurisdictions, you get into that 1/3 territory or even 40%. s190 00:14:52.600 --> 00:15:07.080 So now on top of healthcare, on top of insurance, on top of technology challenges, on top of administration, you now have litigation and insurance companies settle aside a lot of money for for litigated claims. s191 00:15:07.080 --> 00:15:15.320 Now the challenge here is that if you have not gone through a proper digital transformation, you are especially vulnerable to to bad actors, right? s192 00:15:15.320 --> 00:15:20.640 So you can be sent repeated claims, you can be sent fraudulent claims as making that even easier to do right now. s193 00:15:20.640 --> 00:15:28.120 So if you don't have the ability to actually handle those and process things efficiently, you get more and more fraud in the system, right? s194 00:15:28.120 --> 00:15:31.360 It's similar to if your website is easy to DDoS, right? s195 00:15:31.360 --> 00:15:34.560 People are going to keep DDoS-ing you. s196 00:15:36.520 --> 00:15:40.920 And we forget about what matters most as we start solving all these sub-problems, right? s197 00:15:40.920 --> 00:15:49.160 So, these are the things that matter most in terms of the quality of somebody's care, reducing costs, making sure it's a quick process, and it's a good experience. s198 00:15:50.800 --> 00:15:52.600 So, let's go through and and solve some of these problems. s199 00:15:52.600 --> 00:15:55.320 I'll talk about three different phases and how they can help. s200 00:15:55.320 --> 00:15:59.040 So, number one, let's talk about clean data, how we can accomplish that. s201 00:15:59.040 --> 00:16:05.320 Number two, let's talk about within document analysis, and then let's talk about cross-claim factors. s202 00:16:06.880 --> 00:16:08.000 Okay, clean data. s203 00:16:08.000 --> 00:16:13.160 So, has anybody here worked with claims data before or any kind of insurance data? s204 00:16:13.160 --> 00:16:13.960 Few folks. s205 00:16:13.960 --> 00:16:14.920 Okay. s206 00:16:14.920 --> 00:16:20.640 How many duplicate records sort of have have you processed or have you seen? s207 00:16:21.040 --> 00:16:21.440 Many? s208 00:16:21.440 --> 00:16:22.880 Yeah, millions, right? s209 00:16:22.880 --> 00:16:29.480 Uh depending on the jurisdiction that you work with, they can be as much as 50% or even higher, right? s210 00:16:29.480 --> 00:16:33.520 So, you have so many duplicates coming in, and sometimes this is intentional, right? s211 00:16:33.520 --> 00:16:37.040 Sometimes to get a claims processed, you actually have to keep pinging people, right? s212 00:16:37.040 --> 00:16:39.960 Following up with my claim, following up with my claim, right? s213 00:16:39.960 --> 00:16:46.160 Uh so, because people are slow, you get into this sort of cascading failure effect that we see in technology. s214 00:16:46.160 --> 00:16:49.080 Commingled records with other patients, right? s215 00:16:49.080 --> 00:16:52.000 You have Bob Joe and you have Bob Joe Jr. s216 00:16:52.000 --> 00:16:52.400 Right? s217 00:16:52.400 --> 00:16:56.640 Those medical records get mixed in a at a at a fairly often occurrence. s218 00:16:56.640 --> 00:16:57.800 It violates HIPAA. s219 00:16:57.800 --> 00:17:03.240 There's a whole bunch of regulatory risk for that, but the system and the digital transformation is not set up for that. s220 00:17:03.240 --> 00:17:08.480 And decisions can actually be made when other people's medical records are are mixed in. s221 00:17:08.480 --> 00:17:08.720 Right? s222 00:17:08.720 --> 00:17:14.800 So, this is especially sensitive when there's a high uh complexity claim where things can be missed. s223 00:17:14.800 --> 00:17:21.920 Somebody's record gets mixed in by a doctor upstream or administrator and suddenly the whole claim falls apart. s224 00:17:21.920 --> 00:17:26.400 Then we have the the more expected items of poorly scanned or undigitized data. s225 00:17:26.400 --> 00:17:27.040 Right? s226 00:17:27.040 --> 00:17:31.800 You have some some handwriting that you can't understand, let alone doctor's handwriting, right? s227 00:17:31.800 --> 00:17:34.520 Some things come in on paper. s228 00:17:34.520 --> 00:17:41.160 So, we we keep getting these poorly uh clean data examples and they happen for many reasons, right? s229 00:17:41.160 --> 00:17:47.240 So, some of this happens because uh you have a lot of copy pasting that happens in the medical system, right? s230 00:17:47.240 --> 00:17:51.680 You go through uh doctors or nurses as as they're updating the medical records, right? s231 00:17:51.680 --> 00:17:53.640 They'll copy paste between patients. s232 00:17:53.640 --> 00:17:55.840 They'll go through and they'll copy paste between visits. s233 00:17:55.840 --> 00:18:00.160 Uh you might have seen that personally, but it happens a lot uh in the space. s234 00:18:00.160 --> 00:18:03.560 Uh EMRs are actually not set up consistently. s235 00:18:03.560 --> 00:18:06.800 So, has anybody here seen an EMR before? s236 00:18:06.800 --> 00:18:08.560 Their own medical record, right? s237 00:18:08.560 --> 00:18:11.960 Anybody here moved across providers or networks? s238 00:18:11.960 --> 00:18:12.400 Yeah. s239 00:18:12.400 --> 00:18:14.080 Do they look the same? s240 00:18:14.080 --> 00:18:15.400 No, they don't look the same. s241 00:18:15.400 --> 00:18:23.560 Um I'm from Toronto, Canada and many of our hospitals do fresh implementations every time they set up an EMR. s242 00:18:23.560 --> 00:18:28.272 And this often happens both due to the initial processes that exist within a hospital, s243 00:18:28.272 --> 00:18:28.840 [snorts] s244 00:18:28.840 --> 00:18:30.880 it happens across the network as well. s245 00:18:30.880 --> 00:18:35.040 There's very interesting dynamics that happen on why it's set up a certain way. s246 00:18:35.040 --> 00:18:41.440 And if you want to talk about data cleanliness, people do some very strange stuff in EMRs with nested fields. s247 00:18:41.960 --> 00:18:51.080 So, even though we do have electronic health records and medical systems, they're all different, they're structured weirdly, so you do have to do some some transformations on it as well. s248 00:18:51.080 --> 00:18:57.720 And often times it's it's difficult to pull data out, so people just choose to to send faxes or PDFs at the end of the day. s249 00:18:57.720 --> 00:19:00.520 So, solving clean data is not an easy problem. s250 00:19:00.520 --> 00:19:07.040 Uh AI can help us, digital transformation can help us, but upstream there's a lot of factors that go into it. s251 00:19:07.320 --> 00:19:11.080 So, uh this is an example of some illustrative data. s252 00:19:11.080 --> 00:19:12.480 Uh of duplicate records. s253 00:19:12.480 --> 00:19:17.360 Uh real data looks quite similar, as well, with a high number of duplicates across certain networks. s254 00:19:17.360 --> 00:19:21.800 And this is actually very frustrating if you are on the receiving end of these duplicate records, right? s255 00:19:21.800 --> 00:19:27.160 So, we have a number of customers who are like, "Can this organization stop sending us so many duplicates, right?" s256 00:19:27.160 --> 00:19:33.000 And unless you have good systems to detect this, you got you just keep getting the same bad quality data. s257 00:19:33.000 --> 00:19:43.520 So, it's really important to have those technical implementations and detection systems to actually make sure that you can safeguard your your own data pool uh data um cleanliness. s258 00:19:43.520 --> 00:19:49.000 Now, once you have this data, there there's mechanisms of of actually fixing things, right? s259 00:19:49.000 --> 00:19:52.440 For regulatory or uh interpersonal reasons, right? s260 00:19:52.440 --> 00:19:56.840 But you still have this messy data that exists as a baseline. s261 00:19:56.840 --> 00:20:08.560 And once you can detect things like duplications, uh again, seems like a solved problem, uh you no longer incentivize things like sending claims multiple times with with different dates or the same claim s262 00:20:08.560 --> 00:20:13.840 uh or just other folks have to clean up their data, as well. s263 00:20:14.560 --> 00:20:19.600 The other part that, beyond just the cleanliness of the data, is that these claims are are really large. s264 00:20:19.600 --> 00:20:27.800 Uh when I first started working and saw within our database that there was a PDF that was 8 GB in size, that was very surprising. s265 00:20:27.800 --> 00:20:32.520 Um so, it these claims have anything and everything within them. s266 00:20:32.520 --> 00:20:35.120 They have uh images, right? s267 00:20:35.120 --> 00:20:37.080 Uh if you think about auto accidents, right? s268 00:20:37.080 --> 00:20:41.320 There are their images uh within the site of the injury, collisions, and such. s269 00:20:41.320 --> 00:20:45.920 There's x-ray reports, uh ultrasounds, everything in between. s270 00:20:45.920 --> 00:20:49.040 And sometimes this just crashes a lot of PDF viewers. s271 00:20:49.040 --> 00:20:55.720 Uh if you think about some of the old computers that you might have, trying to open up an 8 GB uh byte PDF will be will be quite challenging. s272 00:20:55.720 --> 00:20:57.880 Uh and even on the technology systems, right? s273 00:20:57.880 --> 00:20:59.720 We talk about context windows all the time. s274 00:20:59.720 --> 00:21:03.480 How are you going to fit a context uh into a context window in 8 GB PDF, right? s275 00:21:03.480 --> 00:21:12.000 It's it's not as easy as saying, "Hey Claude, take a look uh at my 10-page PDF and and give me a response or hey chat GPT or Gemini, right? s276 00:21:12.000 --> 00:21:13.240 Not that easy. s277 00:21:13.240 --> 00:21:18.360 Um and on top of that, actually going through and working through a case requires a significant amount of time. s278 00:21:18.360 --> 00:21:24.880 So, either the technical challenges or the business challenges that result in very complex technical challenges. s279 00:21:25.720 --> 00:21:31.760 Now, if we work through and we're adding more and more complexity into here, a lot of these claims also have irregularities. s280 00:21:31.760 --> 00:21:39.840 So, you might have some progress notes, for example, of a earlier patient report that says, "Hey, here is this x-ray that was booked for this date." s281 00:21:39.840 --> 00:21:43.400 You receive the claimant package and that x-ray is not there. s282 00:21:43.400 --> 00:21:43.680 Right? s283 00:21:43.680 --> 00:21:48.040 And it's very easy to miss that without actually having this highlighted in some sort of way. s284 00:21:48.040 --> 00:21:57.160 Um and without that, again, the whole trajectory of a case can change whether there was adequate care given, uh whether the person actually went to the the the care provided, s285 00:21:57.160 --> 00:22:01.680 uh and how do you actually cross-reference this with the billing side of things? s286 00:22:01.680 --> 00:22:09.120 So, this is a very very common scenario where upstream, again, because the data is messy, you actually have to ask repeated times, "Hey, can you send me this record?" s287 00:22:09.120 --> 00:22:10.560 "Oh, we missed faxing this." s288 00:22:10.560 --> 00:22:11.920 "Oh, the fax glitched." s289 00:22:11.920 --> 00:22:14.000 "Oh, we changed our back-end system." s290 00:22:14.000 --> 00:22:14.240 Right? s291 00:22:14.240 --> 00:22:20.960 So, with as an organization, you have to have these lines of defense in in order to make sure that these get flagged. s292 00:22:20.960 --> 00:22:25.440 And as a claims adjuster, a claims adjuster might be processing 200 claims at a time. s293 00:22:25.440 --> 00:22:32.040 So, they're not going to be able to keep everything within their minds saying like, "Oh yeah, I need to follow up on on this x-ray or not." s294 00:22:32.040 --> 00:22:32.240 Right? s295 00:22:32.240 --> 00:22:35.600 They can take notes, they can move forward, but things are going to get lost. s296 00:22:35.600 --> 00:22:41.960 So, this is again where technology can be a great lift uh to these kinds of problems. s297 00:22:43.200 --> 00:22:45.360 Then you get conflicting diagnoses. s298 00:22:45.360 --> 00:22:47.040 This happens fairly often as well. s299 00:22:47.040 --> 00:22:51.320 So, who here has sought a second opinion for a medical condition? s300 00:22:51.320 --> 00:22:53.080 Few folks, right? s301 00:22:53.080 --> 00:22:57.400 Um sometimes we we seek them out um to get another perspective. s302 00:22:57.400 --> 00:22:59.720 Sometimes you get completely different answers, right? s303 00:22:59.720 --> 00:23:05.680 And the more complex the claim is, the more likely there is to be some some kind of conflict. s304 00:23:05.680 --> 00:23:21.720 So, in this case, what often happens is when you adjudicate a medical claim and you see some conflicting diagnosis or you want to double-check how much you should set aside to actually pay out for the claim, you send it to an IME or an independent independent medical evaluation. s305 00:23:21.720 --> 00:23:26.280 Then you have another doctor who performs their their review of the file. s306 00:23:26.280 --> 00:23:31.200 And then you have to manage those doctors and the provider networks and the IME themselves. s307 00:23:31.200 --> 00:23:32.600 So, now we're stacking again. s308 00:23:32.600 --> 00:23:36.280 More people into the ecosystem, more complexity. s309 00:23:37.440 --> 00:23:45.000 Now, AI can be quite useful here and it would it would be, I guess, poor taste to give a talk in 2026 without talking about evals. s310 00:23:45.000 --> 00:23:52.680 But we did use AI to to try to solve these issues in some ways, the conflicting diagnosis, the missing records and a number of other ones. s311 00:23:52.680 --> 00:24:02.720 So, we built this benchmark called MLCR with artificial analysis to to measure the effectiveness of AI in actually answering these questions. s312 00:24:03.160 --> 00:24:11.120 So, flip over to our blog post that we talked about and I'll scroll down to what this benchmark looks like. s313 00:24:11.120 --> 00:24:15.280 So, we released six different tiers of questions related to medical claims. s314 00:24:15.280 --> 00:24:20.440 Some of these are very simple, like when did this injury happen and what was the injury, right? s315 00:24:20.440 --> 00:24:22.200 Then they get a little bit more complex. s316 00:24:22.200 --> 00:24:24.360 How many surgeries get performed? s317 00:24:24.360 --> 00:24:27.640 How much capacity does the this patient have? s318 00:24:27.640 --> 00:24:33.640 And eventually we get into more and more complex questions that are even harder to to resolve. s319 00:24:33.640 --> 00:24:42.600 And oftentimes, the way that you ask a question as a claim adjuster, a doctor, an administrator, you don't have full context of what's happening. s320 00:24:42.600 --> 00:24:49.280 So, this is where we see hallucinations show up where people assume something is in a medical record, but it's not. s321 00:24:50.320 --> 00:24:51.800 So, let's go a little bit more to the results. s322 00:24:51.800 --> 00:24:55.480 So, this is available on the artificial analysis website. s323 00:24:55.480 --> 00:25:02.480 If you want to take a look, uh, so we released an open source, uh, data set for the first three tiers of questions that we have, the easy ones. s324 00:25:02.480 --> 00:25:09.400 And we have a private set that we host with artificial analysis for, um, to stop benchmarking to to a certain extent. s325 00:25:09.400 --> 00:25:11.800 So, it's very interesting to see how how models perform. s326 00:25:11.800 --> 00:25:16.120 Uh, we actually measure this on three different, uh, axes. s327 00:25:16.120 --> 00:25:18.880 So, let me flip back to that slide. s328 00:25:18.880 --> 00:25:21.160 We measure this on conciseness. s329 00:25:21.160 --> 00:25:23.080 So, how concise is a response? s330 00:25:23.080 --> 00:25:27.320 Uh, and we measure this against a human, uh, expert validated answer. s331 00:25:27.320 --> 00:25:28.800 We then go through accuracy. s332 00:25:28.800 --> 00:25:30.520 Is that Is the answer correct? s333 00:25:30.520 --> 00:25:32.560 Uh, and the final one is completeness. s334 00:25:32.560 --> 00:25:39.320 Are all the relevant, uh, informations, uh, piece of information answered, uh, in the question? s335 00:25:40.200 --> 00:25:41.080 So, we go back here. s336 00:25:41.080 --> 00:25:46.160 Um, the frontier has continued to evolve very, very quickly, and this is where having evals is really important. s337 00:25:46.160 --> 00:25:53.880 Uh, when we released the eval, uh, the highest accuracy that that we saw on the levels of reasoning that that we had was 20%. s338 00:25:53.880 --> 00:25:57.760 A few months later, we're in the 75% with with the very high reasoning levels. s339 00:25:57.760 --> 00:25:57.960 Right? s340 00:25:57.960 --> 00:26:06.080 So, models move very quickly, um, but this is great because it means that we can start productizing a number of the more complex questions. s341 00:26:06.080 --> 00:26:11.320 And we have more certainty of the confidence that somebody has to review this, uh, in a system. s342 00:26:11.320 --> 00:26:13.920 So, within our system, there's always a human in the loop. s343 00:26:13.920 --> 00:26:16.240 Uh, so that's really important. s344 00:26:16.240 --> 00:26:23.760 You also start getting the the cost curves here on the Pareto frontiers, on the cost to actually ask this question and get a suitable answer. s345 00:26:23.760 --> 00:26:27.960 And this has been improving through the past few months as well. s346 00:26:27.960 --> 00:26:29.360 And you get some of these tradeoffs as well. s347 00:26:29.360 --> 00:26:39.960 So, one of the interesting things when we were running these benchmarks on, uh, document analysis was that the GPT models were were excellent at accuracy, but they omitted some key pieces of information. s348 00:26:39.960 --> 00:26:46.280 Uh, Anthropic's models were were quite good, um, and they're much more complete on the amount of information. s349 00:26:46.280 --> 00:26:50.040 Um, they they do have slightly lower accuracy here as well. s350 00:26:50.040 --> 00:26:51.400 Can I ask a question? s351 00:26:51.400 --> 00:26:52.080 Sure. s352 00:26:52.080 --> 00:26:55.360 Accuracy, how do you measure it compared to the human factor? s353 00:26:55.360 --> 00:27:00.160 Yes, compared to to human answer uh and then we do the triple judge uh verification. s354 00:27:00.160 --> 00:27:01.600 So, yeah. s355 00:27:01.600 --> 00:27:02.600 So, we have accuracy here. s356 00:27:02.600 --> 00:27:11.880 You can see how uh the curve looks like and we have completeness uh which is uh is still not solved across most model providers. s357 00:27:11.880 --> 00:27:18.280 And yes, to to the earlier question, uh this is based on a human verified answer and on a calibrated set of judges. s358 00:27:18.280 --> 00:27:27.280 Uh within our blog post, we describe the the methods we have here uh and we released a follow-up as well on on how we did this calibration. s359 00:27:27.520 --> 00:27:36.960 So, even though we're using AI here to structure our data and extract important insights, we want to make sure that each step uh of that process is well calibrated because s360 00:27:36.960 --> 00:27:40.920 if upstream uh your data is incorrect and you start using it downstream, right? s361 00:27:40.920 --> 00:27:43.440 You haven't gained anything. s362 00:27:43.600 --> 00:27:52.120 So, again, really important as you're going through the claims uh uh process to to calibrate your judges, to build good evals. s363 00:27:52.120 --> 00:27:55.480 Uh I think that's a recurring theme at Ed Most Talks. s364 00:27:55.480 --> 00:28:02.160 And you want to make sure you have those experts who can actually verify this information, so you work across teams to to get there. s365 00:28:03.600 --> 00:28:04.160 Okay. s366 00:28:04.160 --> 00:28:05.400 So, that's within a claim. s367 00:28:05.400 --> 00:28:10.600 Uh we know that with some amount of certainty, we can extract some types of insights. s368 00:28:10.600 --> 00:28:12.200 We can flag conflicting diagnosis. s369 00:28:12.200 --> 00:28:13.240 We can deduplicate. s370 00:28:13.240 --> 00:28:16.040 We can search for missing records. s371 00:28:16.040 --> 00:28:19.400 Let's go to something more interesting, which is cross claims. s372 00:28:19.400 --> 00:28:19.680 Right? s373 00:28:19.680 --> 00:28:29.160 So, if you are a mid-size uh insurance company, you might be collecting $1 billion in premiums within a uh portfolio of lines of businesses. s374 00:28:29.160 --> 00:28:37.880 And you might be paying out, let's say, 20,000 uh claims uh at a time per year uh or even more, depending on what line of business you deal with. s375 00:28:37.880 --> 00:28:42.800 Now, you want more insight of where you actually like what's affecting your loss ratio. s376 00:28:42.800 --> 00:28:45.000 What kind of claims are people making? s377 00:28:45.000 --> 00:28:46.320 In which geographies? s378 00:28:46.320 --> 00:28:47.640 What type of injuries? s379 00:28:47.640 --> 00:28:51.800 and what type of doctors are being involved. s380 00:28:52.440 --> 00:28:59.840 This is where I think it gets quite interesting as well as actually being able to, once you have that data extracted from the claim, you can start doing interesting visualizations, s381 00:28:59.840 --> 00:29:01.640 regressions, and work as well. s382 00:29:01.640 --> 00:29:01.880 Right? s383 00:29:01.880 --> 00:29:11.200 So, two of the questions that we we've been asked and we've worked on answering is sort of how much are people traveling within a network, right? s384 00:29:11.200 --> 00:29:14.400 So, if a patient is, let's say, in San Francisco or New York, right? s385 00:29:14.400 --> 00:29:16.520 Are they getting treatment in their local area? s386 00:29:16.520 --> 00:29:19.440 And is that the right person to to be giving the treatment? s387 00:29:19.440 --> 00:29:21.800 We also have these networks of who's actually working together. s388 00:29:21.800 --> 00:29:32.680 So, this is some redacted data, a subset from from California, and you can actually track to see which lawyers are working together, which doctors are working together, which chiropractors, s389 00:29:32.680 --> 00:29:39.280 physiotherapists, and what is the actual working relationship look like and the story within the claim itself. s390 00:29:39.280 --> 00:29:46.560 So, this is that sort of higher-tier insight that you can get within extracting data from your claim. s391 00:29:47.240 --> 00:29:49.480 You then get into some interesting medical outcomes, right? s392 00:29:49.480 --> 00:29:51.720 So, which conditions co-occur together? s393 00:29:51.720 --> 00:29:54.960 So, if somebody has a back injury, as an example, right? s394 00:29:54.960 --> 00:30:06.280 You might have a series of different diagnoses that could be mapped to ICD codes that coexist together, and you might see, okay, why does this exist from a certain s395 00:30:06.280 --> 00:30:09.400 doctor's write-up versus another's? s396 00:30:09.400 --> 00:30:17.560 And in this case, again, you you only get these capabilities if you can solve the data extraction problem within one claim, and then roll it up into your broader s397 00:30:17.560 --> 00:30:19.960 insurance portfolio. s398 00:30:21.840 --> 00:30:23.440 Now, we get to the money portion, right? s399 00:30:23.440 --> 00:30:31.640 So, pulling a small subsegment here of two treatments of physiotherapy, different doctors charge different amounts, right? s400 00:30:31.640 --> 00:30:33.280 We've probably seen that. s401 00:30:33.280 --> 00:30:34.440 And sometimes that's fine. s402 00:30:34.440 --> 00:30:36.920 If you're a better doctor, you should be able to charge more, right? s403 00:30:36.920 --> 00:30:42.720 If you can get somebody to recover as quickly as possible in five treatments instead of 10, you should be able to charge more, right? s404 00:30:42.720 --> 00:30:53.280 So, uh but it's really important to understand costs and and focus on minimizing them because as as soon as there's cost inflation through different parts of the medical system, it keeps compounding, right? s405 00:30:53.280 --> 00:30:59.040 Premiums increase, uh the fees that we are charged increases, uh and we get less for our money. s406 00:30:59.040 --> 00:31:03.880 So, within this analysis, uh you can see across claims, is there a consistent factor? s407 00:31:03.880 --> 00:31:07.040 What what is the amount of coverage that you should be providing? s408 00:31:07.040 --> 00:31:11.560 How How you should you be underwriting these costs for claims as well. s409 00:31:11.560 --> 00:31:13.800 And you can estimate how things are going to change over time, right? s410 00:31:13.800 --> 00:31:19.760 So, year over year, as an example here, uh there was a 10% increase for this type of treatment for a given doctor, right? s411 00:31:19.760 --> 00:31:23.040 So, is that going to happen across uh the jurisdiction? s412 00:31:23.040 --> 00:31:27.880 Is that going to happen across the the industry in general? s413 00:31:30.240 --> 00:31:41.640 So, then you can start doing some counterfactual analysis or or changes in a in analysis to say, if we change our network, if we start directing folks into a certain direction, if we change how we how we bill, right? s414 00:31:41.640 --> 00:31:43.840 What are the potential savings that exist? s415 00:31:43.840 --> 00:31:49.480 And what kind of procedures should people be having based on the standard of care that exists? s416 00:31:49.480 --> 00:31:57.960 Um there is there's recently an article about how as anybody see it seen at the overbilling article about $1 billion of additional billings that have come in? s417 00:31:57.960 --> 00:31:59.000 Have you seen that? s418 00:31:59.000 --> 00:32:14.000 Um so, those are examples where again, if you cannot process data effectively, people are going to find ways to to to go through and and submit additional records uh or do some some shadier behavior as well. s419 00:32:14.000 --> 00:32:19.160 So, all this to say is that once you have good data, you can start making these decisions. s420 00:32:21.720 --> 00:32:27.760 And then once you have this decision, uh this claims information, you can start writing it into a broader system, right? s421 00:32:27.760 --> 00:32:38.520 So, you can start using this in in the other lines of business of saying, if you know these are the trends in auto injury or workers' compensation, are we seeing the same thing in general liability? s422 00:32:38.520 --> 00:32:48.480 If you know that your underwriting model is missing certain trends or you don't have as much explain ability there, can we make the model more explainable with what we're seeing? s423 00:32:48.480 --> 00:32:55.240 How do we provide this sort of magical data table that we wish that that existed within our data lake? s424 00:32:55.240 --> 00:33:04.680 So again, the more data you have, the more complete of a picture you you can make and the better you can underwrite and price the product at the end of the day for the consumer. s425 00:33:04.680 --> 00:33:16.040 And this is information you need to actually power the the effective AI applications, whether they're on on the underwriting side, on the fraud side, on the claim side as well. s426 00:33:16.960 --> 00:33:19.520 So if you think about like a blue sky world, right? s427 00:33:19.520 --> 00:33:22.200 We we want to reduce something more broadly. s428 00:33:22.200 --> 00:33:25.320 So if we want to reduce costs for everybody, right? s429 00:33:25.320 --> 00:33:27.400 Taking into account that patient experience. s430 00:33:27.400 --> 00:33:30.040 If we want to reduce the time that a claim is open. s431 00:33:30.040 --> 00:33:37.680 If we want to improve how quickly people get back to work and the the treatment they get, we need to make sure we have the right data and the right digital transformation s432 00:33:37.680 --> 00:33:39.160 to do that. s433 00:33:39.160 --> 00:33:44.840 So this is an illustrative example in the work that we've done within WiseDocs as well. s434 00:33:44.840 --> 00:33:47.840 We've seen significant savings as well. s435 00:33:47.840 --> 00:33:53.880 So going through this digital transformation process and having the right data is really really important. s436 00:33:54.120 --> 00:33:55.760 So I'll wrap up the workshop here. s437 00:33:55.760 --> 00:34:00.600 I did want to leave quite a bit of time for questions cuz it is pretty comprehensive topic. s438 00:34:00.600 --> 00:34:04.360 So thank you for listening and happy to take some questions. s439 00:34:05.588 --> 00:34:07.588 [applause] s440 00:34:09.480 --> 00:34:11.679 Yes, I'll I'll share the blog post. s441 00:34:11.679 --> 00:34:12.879 Yes, question over here. s442 00:34:12.879 --> 00:34:18.840 Yeah, so so the question was about how do we actually structure the data to build um all this information. s443 00:34:18.840 --> 00:34:21.879 So it goes through a number of different levels. s444 00:34:21.879 --> 00:34:24.120 I'll share some talks that I've given on the topic as well. s445 00:34:24.120 --> 00:34:30.879 The baseline is we we use OCR and images to to extract information from the underlying records. s446 00:34:30.879 --> 00:34:33.000 Sort of standard pre-processing. s447 00:34:33.000 --> 00:34:36.560 Pre-processing PDFs is not an easy task as well if anybody has worked with them. s448 00:34:36.560 --> 00:34:42.440 Uh those 8 gigabyte ones have some some weird one in a million examples in them. s449 00:34:42.440 --> 00:34:48.919 So, once you have the the data, the text and images, we have a series of models on top, um NLP models, extraction models. s450 00:34:48.919 --> 00:34:55.120 We use VLMs uh to structure it sometimes, like a a more standard entity extraction model works well. s451 00:34:55.120 --> 00:34:57.680 We have various classifiers that we build on top. s452 00:34:57.680 --> 00:35:01.880 Um we then have a summary layer that sits on top of that to summarize the documents. s453 00:35:01.880 --> 00:35:05.400 Uh that is then human verified to make sure that the information is correct. s454 00:35:05.400 --> 00:35:08.400 And finally, we run those additional layers of analysis on top of that. s455 00:35:08.400 --> 00:35:19.600 So, we have this abstraction pattern that sort of concentrates the the density of information as you go up, um but it's verified at various points in time with various models and and human intervention as well. s456 00:35:19.600 --> 00:35:24.000 So, that means once you get to the top in the highest value items, that's when AI can be effective. s457 00:35:24.000 --> 00:35:28.920 And those are sort of those MLCR uh type questions that can have the right answer. s458 00:35:28.920 --> 00:35:32.800 So, within MLCR, this actually uses clean data to ask these questions. s459 00:35:32.800 --> 00:35:38.920 Real-world data is not clean, so you have to have various steps ahead of that to to clean up the data to get there. s460 00:35:38.920 --> 00:35:50.440 Yeah, so typically our our customers So, bigger customers have multiple lines of business internally and they have different uh rules I would say across sharing across different teams. s461 00:35:50.440 --> 00:35:52.960 Uh other customers just work in one line of business. s462 00:35:52.960 --> 00:35:57.200 So, they might just work in auto, they might just work in disability, they might just work in workers' comp. s463 00:35:57.200 --> 00:36:01.040 Um but we take the trends uh across the board to try to figure out and flag those. s464 00:36:01.040 --> 00:36:05.680 So, there's the question that within sort of an office, right? s465 00:36:05.680 --> 00:36:09.960 How do you surface some of those insights uh or or sort of geography? s466 00:36:09.960 --> 00:36:19.210 Then you have the the layer on top, which might be um the broader company, but comp- insurance companies also have a collection of companies within them sometimes uh as separate businesses. s467 00:36:19.210 --> 00:36:23.680 [clears throat] Uh and then how do you actually feed that into a centralized data lake uh as well. s468 00:36:23.680 --> 00:36:30.680 So, again, every company is a a bit different, but we we go as fast as our partners want us to. s469 00:36:30.680 --> 00:36:31.520 We live in claims. s470 00:36:31.520 --> 00:36:32.720 So so we focus on claims. s471 00:36:32.720 --> 00:36:36.520 So claims are our our bread and butter, specifically complex claims. s472 00:36:36.520 --> 00:36:40.200 That's where our whole AI pipeline is focused on. s473 00:36:40.200 --> 00:36:51.840 But as we sort of evolve and go and solve the sort of easier challenges, I would say, with adjudicating a claim, processing a claim, extracting that information, you start seeing these more sort of insightful s474 00:36:51.840 --> 00:36:56.040 aspects that help organizations make portfolio level decisions. s475 00:36:56.040 --> 00:36:56.400 Yeah. s476 00:36:56.400 --> 00:36:57.520 Yeah, so that's a good question. s477 00:36:57.520 --> 00:37:01.400 So the question is how do we onboard somebody when there there's a lot of these challenges? s478 00:37:01.400 --> 00:37:06.400 So typically we work with the most important thing is a willingness to change, right? s479 00:37:06.400 --> 00:37:10.960 So we we've all probably worked with teams or organizations who want to change versus those who don't. s480 00:37:10.960 --> 00:37:13.520 Um very very different experience. s481 00:37:13.520 --> 00:37:20.480 So assuming it's an organization that wants to change, there'll be a few sample files typically that we'll work through in various formats. s482 00:37:20.480 --> 00:37:26.520 Typically people will send us either things they're immediately working on or have or are basically they're trying to back test. s483 00:37:26.520 --> 00:37:33.560 Uh and then we go through a review process to see what did you extract, what did our system extract, how does it align? s484 00:37:33.560 --> 00:37:35.840 Does it match the information you want to see? s485 00:37:35.840 --> 00:37:38.120 Do you have different types of standards? s486 00:37:38.120 --> 00:37:46.240 So if we go back, for example, uh to the different types of documents here, let's just go back to the intake form. s487 00:37:46.240 --> 00:37:48.640 There might be various dates on this form. s488 00:37:48.640 --> 00:37:53.080 Do you want the date of injury or do you want the the report date when it was submitted, right? s489 00:37:53.080 --> 00:37:56.160 So these are things of alignment of business objectives. s490 00:37:56.160 --> 00:37:58.160 Go through process a few standard cases. s491 00:37:58.160 --> 00:38:09.520 If things work well, then we start sort of a broader pilot and then gets into the fun integration aspect of integrating with with upstream providers apart from just like hand uploaded documents, API integrations, s492 00:38:09.520 --> 00:38:11.240 like dealing with all all the messiness. s493 00:38:11.240 --> 00:38:18.400 So as long as there's a a clear business outcome and transformation goal, then it's it's a fairly straightforward process. s494 00:38:18.400 --> 00:38:27.680 And again, as as we go through that journey, and sometimes it's like we don't have the capabilities out of the box, sometimes we do, sometimes there's multiple integrations on internal systems as well. s495 00:38:27.680 --> 00:38:29.240 So, yeah. s496 00:38:29.240 --> 00:38:33.320 So, let's start with the first part of the question, which is handling regulations and how do we work with partners? s497 00:38:33.320 --> 00:38:38.080 So, we it depends on how how quickly people are are moving. s498 00:38:38.080 --> 00:38:40.040 Uh and also depends how quickly regulations are moving. s499 00:38:40.040 --> 00:38:46.800 So, the the way we've set up the system right now is that it always requires a human in the loop to make a decision. s500 00:38:46.800 --> 00:38:58.200 Uh so, we surface these insights as an example for things like a conflicting diagnosis, we'll see, "Hey, on page two on page 20, there there's a difference of diagnosis between these two doctors, you should take a look, right?" s501 00:38:58.200 --> 00:39:06.400 So, as long as there's a human in the loop, I'd say it it removes like 70% of the the regulatory concerns that people have. s502 00:39:06.400 --> 00:39:16.560 Uh regulation is also still catching up, so uh there's a lot of things to read about like the NAIC reports that I mentioned, they that organization publishes a number of different reports. s503 00:39:16.560 --> 00:39:18.600 There's the guidelines that exist for that. s504 00:39:18.600 --> 00:39:22.240 We talk to the internal governments and regulatory teams as well. s505 00:39:22.240 --> 00:39:27.920 Uh and if they want to see some pieces of information, like we will share with them like this is how the AI pipeline works, right? s506 00:39:27.920 --> 00:39:30.000 This is the type of data that's being used. s507 00:39:30.000 --> 00:39:39.360 Um but at the end of the day, as long as you don't build a black box and things are auditable uh and there's a human in the loop, that take cares uh takes care of many things. s508 00:39:39.360 --> 00:39:45.080 The regulatory side, there's like a whole other like business decision decision aspect, like how to approach nuclear verdicts. s509 00:39:45.080 --> 00:39:59.760 Uh for uh nuclear verdicts are um very large uh awards or um claims that that are $10 million or thermonuclear ones which are $100 million, which really reallocate uh businesses' reserves. s510 00:39:59.760 --> 00:40:04.440 So, for those ones, typically people come with like an approach on how they want to handle them. s511 00:40:04.440 --> 00:40:08.160 And within the platform, you can say, "I'm going to handle my most complex claims first." s512 00:40:08.160 --> 00:40:10.400 Or I'm going to handle the one that was submitted first. s513 00:40:10.400 --> 00:40:13.440 Um and those are all sort of business decisions people can make. s514 00:40:13.440 --> 00:40:18.800 Uh the second part of the question on like people's internal like organizations' internal goals. s515 00:40:18.800 --> 00:40:29.000 Um I mean, we want to work with partners that share the same sort of mindset about improving patient care, uh reducing backlogs uh for claims, providing the best decisions. s516 00:40:29.000 --> 00:40:38.000 A lot of times our customers are not even thinking too much about like the the cost of health care itself is everything around it, right? s517 00:40:38.000 --> 00:40:44.520 So, uh if you're a large insurance company, you actually spend hundreds of millions of dollars on legal fees. s518 00:40:44.520 --> 00:40:44.720 Right? s519 00:40:44.720 --> 00:40:52.560 So, that is something that is not even tied to the patient experience that there's a large amount of optimization you can do there and to speed things up. s520 00:40:52.560 --> 00:40:53.080 Right? s521 00:40:53.080 --> 00:40:58.240 Um there's regulatory concerns that in certain jurisdictions like California, things are very litigated. s522 00:40:58.240 --> 00:41:01.080 Um you want to figure out how how to deal with those. s523 00:41:01.080 --> 00:41:06.080 Um there's a whole bunch of middle parties that that are involved as well uh that are also inefficient. s524 00:41:06.080 --> 00:41:09.040 So, depending on the customer, everybody has a different goal. s525 00:41:09.040 --> 00:41:14.880 Um but in our conversations, it's it's always been like we have this very obvious problem to solve with technology. s526 00:41:14.880 --> 00:41:16.360 Let's solve that first. s527 00:41:16.360 --> 00:41:22.040 Um and again, the insights start surfacing afterwards of being like, "Oh, I didn't know that about my business. s528 00:41:22.040 --> 00:41:23.240 Let's solve that afterwards." s529 00:41:23.240 --> 00:41:23.840 So. s530 00:41:23.840 --> 00:41:26.840 Yeah, so the question is about decision APIs in Jv. s531 00:41:26.840 --> 00:41:34.320 Um as part of the I don't know if anybody's seen the paper um copies of the AI news that are that the conference is handing out. s532 00:41:34.320 --> 00:41:36.920 There's actually an article that I'm writing for that. s533 00:41:36.920 --> 00:41:40.760 Um so, we we've been using classification models uh for a long time. s534 00:41:40.760 --> 00:41:43.480 I've generally had uh strong opinion on that. s535 00:41:43.480 --> 00:41:44.160 Hasn't come out today. s536 00:41:44.160 --> 00:41:45.880 It's coming out in a later edition. s537 00:41:45.880 --> 00:41:50.680 Um I think it's very useful, uh but LLMs are not the right solution for everything, right? s538 00:41:50.680 --> 00:41:54.080 So, when we talked about this, right? s539 00:41:54.080 --> 00:41:56.280 This segment of AI can be split up as well, right? s540 00:41:56.280 --> 00:42:01.480 Classification, extractive uh versus sort of the more complex decision-based ones. s541 00:42:01.480 --> 00:42:03.800 Uh so, for us cost is really important. s542 00:42:03.800 --> 00:42:05.560 So, that's why we trained our own models as well. s543 00:42:05.560 --> 00:42:08.280 Accuracy is really important and hallucinations as well. s544 00:42:08.280 --> 00:42:11.160 So, I think Jv was not as surprised that it came out. s545 00:42:11.160 --> 00:42:22.920 It was more so a surprise of how long it took because I think the LLM like token max reasoning for every problem phenomenon lasted for a very long time until we started thinking about costs a lot more this year. s546 00:42:22.920 --> 00:42:28.120 So, uh once Javis sort of hit already, we could you can test it out. s547 00:42:28.120 --> 00:42:31.400 Um but we've sort of built our own models with the same paradigm in mind. s548 00:42:31.400 --> 00:42:32.600 So. s549 00:42:32.600 --> 00:42:34.600 Yeah, so I'll answer part two. s550 00:42:34.600 --> 00:42:41.480 Um so as an organization we have not moved the broader industry um to the extent we want, right? s551 00:42:41.480 --> 00:42:49.480 So, to say that we can say we've reduced the the claim uh time by 5 days on average for somebody across every patient. s552 00:42:49.480 --> 00:42:50.280 We'd love to say that. s553 00:42:50.280 --> 00:42:51.960 We we're not in that position yet. s554 00:42:51.960 --> 00:42:55.000 For specific organizations like we had some case studies on the website. s555 00:42:55.000 --> 00:42:57.120 I quickly put put some up as well. s556 00:42:57.120 --> 00:43:01.960 There's some significant ones within lines of business uh for cost reduction and speed. s557 00:43:01.960 --> 00:43:09.600 Um so, you can check those out and we can chat afterwards as well if you're looking for more specific sort of context on that, too. s558 00:43:09.600 --> 00:43:18.480 Yeah, so so the the first notice uh of injury or loss um or and depend depending on the on the sub domain. s559 00:43:18.480 --> 00:43:24.240 Yes, um so depending on the jurisdiction there's also turnaround time requirements when handling these. s560 00:43:24.240 --> 00:43:28.920 So, for example in Texas you only have a few hours of processing a claim to get an answer. s561 00:43:28.920 --> 00:43:33.160 In other states it might be 7 days, 21 days that you need to get a response. s562 00:43:33.160 --> 00:43:42.480 Now, it gets more complicated because if you're a worker who's been working in a factory, let's say for 30 years, you're 60 years old, you're going to have large medical record file, right? s563 00:43:42.480 --> 00:43:50.280 So, there's a lot of information here and in the initial report, but to do a full claim analysis you have to go back and you have to try to pull those. s564 00:43:50.280 --> 00:43:52.120 Maybe your name changed because you got married. s565 00:43:52.120 --> 00:43:55.960 Maybe you moved uh cities, providers, right? s566 00:43:55.960 --> 00:44:01.280 So, you have to reconstruct sort of that that full uh history as well. s567 00:44:01.280 --> 00:44:10.600 Uh and I think unfortunately in a lot of litigated cases there's information that's left out uh as well on purpose to due to the fact that a lot of organizations s568 00:44:10.600 --> 00:44:13.680 have not digitally transformed and can't process cases fast enough. s569 00:44:13.680 --> 00:44:25.360 So, uh if this was all that's needed, this would be uh much faster, but unfortunately or more realistically, there's a much broader backstory as well.