Can Oncology Workflows Run Without Human Touch? - Anant Shankhdhar, Risa Labs

AI Engineer · 16 min · 162 sentences · from YouTube's caption track

Each timecode opens YouTube at the start of that sentence. Line anchors (#s42) are the cue ids in the WebVTT, and every line carries its start and end seconds. All transcripts has every talk, and the whole corpus as one file.

1. 00:00Hi everyone.
2. 00:02My name is Anant Shankar.
3. 00:03I am an AI engineer at Trisca and I'll be talking about oncology automating oncology workflows from end to end.
4. 00:12So at Trisca, we are automating various workflows in oncology.
5. 00:16One of such workflows is prior authorizations where we file for authorizations for drugs for cancer patients.
6. 00:25So I'll give a brief overview about the workflow before we move further in the call.
7. 00:31So the first step is that we intake the orders that we get on a daily basis.
8. 00:36The second step is we verify whether the patient is actually eligible for getting the drugs based on the amount that they have in the insurance available with them.
9. 00:46This is called eligibility and benefits verification.
10. 00:50Next, we determine what all drugs in the for the patient require authorization.
11. 00:57So drug can basically fall into one of the following pathways.
12. 01:01So one is NAR which is no auth required.
13. 01:04Basically the drug does not require authorization.
14. 01:07Another can be that the authorization has already happened for the drug for a time period which is called auth on file.
15. 01:14And the third is auth required which means that the authorization needs to be performed for this drug.
16. 01:20So yeah.
17. 01:22This is the workflow.
18. 01:23I was So although our bots run and perform these steps, finally a human review is required in before submitting all these orders.
19. 01:34I was tasked with the with the problem to run some of these orders without any sort of human touch directly towards submission.
20. 01:47Which means I need to confidently identify which all orders can be proceeded without any human verification and then build the entire flow for them as well.
21. 01:58So, confidence is a key metric that we were working towards.
22. 02:04So, yeah.
23. 02:06How we did this was using four agents, namely EV agent, auth agent, necessity agent, and submission agent.
24. 02:13EV agent is for eligibility and business verification.
25. 02:17So, basically fetching the patient data and determining whether it is uh fine for moving forward.
26. 02:24Auth agent determines the key type of the status of the drug, whether authorization is required or not.
27. 02:31Uh and the necessity agent is the clinical brain of system of the system.
28. 02:35So, for the cases where authorization is required, we determine whether it has to be done on a whether it is right for the patient based on his vitals or not.
29. 02:45And finally, we move towards submission.
30. 02:50So, uh starting with the first step, how do we do this without any human in the loop?
31. 02:56So, one of the problems that we have here is that insurance details, insurance documents, everything is a scattered across dozens of portals, APIs, and documents.
32. 03:08And it is difficult to find that information at one place.
33. 03:13So, in order to in order to start with our pipeline, we need to get information from various portals as well as APIs.
34. 03:24In order to do so, we built a unified service that connects to different payer sources and gives the output in a normalized uniform format, which we can use for
35. 03:36processing further.
36. 03:37And we also added a deterministic decision engine to flag the cases which would not move forward, thus deterministically uh fixing some of the orders without any human in the loop.
37. 03:51So, here's how it works.
38. 03:53So, we have a coverage orchestrator which determines whether the patient will go for the API or the RPA path.
39. 04:00RPA is basically the automation.
40. 04:02Uh we perform the actions for the RPA or call the API and get the output in a fixed coverage result format which we then pass to our deterministic engine to determine whether the coverage is active or not.
41. 04:19If it is, we move it further in the PHN, else we stop that right there.
42. 04:24Now, one of the problems we saw here was that uh if we have to build this, then we will need to make custom integrations for different sorts of portals,
43. 04:34which is not not a very scalable uh process.
44. 04:40But, how we tackled it was that we used LLMs in the loop loop.
45. 04:45So, first of all, we made a huge repository of custom actions as well as popular actions that are required in to build RPAs.
46. 04:54Then, we built a LLM-based config generation which performs these actions and builds the config for for a portal to run on.
47. 05:06Uh which uh fix which reduces our development time significantly.
48. 05:12And finally, uh these automations are fragile, so it may happen that they break during the run time.
49. 05:20For this, we have a self-healing loop which identifies these cases during production hours and then mitigates them.
50. 05:28So, that thus preventing any sort of failures.
51. 05:31So, yeah, that is the first step.
52. 05:34Uh once we have done this, we move further in our chain.
53. 05:39So, as we can see our diagram the graph, uh the patient data is fetched.
54. 05:44We perform the eligibility verification.
55. 05:47We see if it is fine or not.
56. 05:49If it failed, we stop there, else we move forward.
57. 05:53So, yeah.
58. 05:54Moving forward, uh the next step was that data uh was to see uh what all drugs require authorization.
59. 06:04So, if I have to uh remove humans from the loop, one simple case that I saw was that uh if a drug has already been authorized, that is the authorization
60. 06:19is on file, or if the authorization is not required, then we can partly solve the order.
61. 06:25That is that we do not need human oversight on that part of the order.
62. 06:30Uh so, yeah.
63. 06:32Uh so, as the first step, we built a simple LLM extraction pipeline, which takes in patient's notes, performs LLM's extraction, and categorizes the drugs into these types.
64. 06:44The default being that authorization is required.
65. 06:47However, uh yeah.
66. 06:49So, we thought that they should would work, but we faced some issues here as well.
67. 06:55So, one popular issue was that uh the notes that we were using did not have enough data, which means that it was performing some some errors.
68. 07:07Uh Also, LLM uh extraction is sort of an indis- uh indeter- deterministic process.
69. 07:16That means that whatever outputs we get from here cannot be blindly tested.
70. 07:21So, we still need a human to review all these things.
71. 07:24It might improve the efficiency, but it will not eliminate the human.
72. 07:29So, in order to do so, uh we thought that what if I add more evidence to it.
73. 07:36So, if the LLM is saying that a drug is not uh no auth required, then I have some information backing that.
74. 07:43I can say that confidently or for the auth and file cases.
75. 07:47You already uh yeah.
76. 07:49So, in order to do so, we we leverage two other data sources.
77. 07:54So, one was the authorization letters.
78. 07:56So, this was basically the previous information.
79. 08:00So, across the previous runs that a given patient has, they have authorization letters available, which shows that these drugs were actually authorized.
80. 08:08So, now, instead of just having one source, I have two sources that give me the same information and wherever they conquer, I can say with confidence that this drug is already been authorized.
81. 08:21Second is the NAR case, the no auth required case.
82. 08:25So, here uh we found some other resources wherein from the portals or from some documents, we could find out that on a monthly basis, which are the drugs that the
83. 08:36payer does not work for.
84. 08:38The insurance does not work for, basically.
85. 08:40So, uh we use that information to build a payer rule knowledge base, which is basically a SQL database uh which was made from portal checks as well as LLM extractions.
86. 08:54So, we use all this information, we reconcile the evidence, and we only extract the auth auth statuses with a higher confidence.
87. 09:02So, if from this pipeline, we get whether a drug is not authorized not does not require authorization or is already authorized, we have a solid proof behind it.
88. 09:13Uh and a higher confidence uh to say that this is the case.
89. 09:19And of course, all of this is configurable.
90. 09:22So, uh in order if some mishap happens, we always have a plug over the cases.
91. 09:29So, yeah.
92. 09:30Uh another a good thing that happened from here is that we noticed that certain orders were completely eliminate We could eliminate certain orders completely from these two type of drugs.
93. 09:42Because it may happen that an order does not actually require require authorization at all.
94. 09:47The drugs are either are not requiring authorization or are actually authorized from before.
95. 09:53So, this enabled us no touch on a certain set of orders that we had.
96. 09:59Yeah.
97. 10:00Uh this slide gives a big uh small overview on how we build the payroll knowledge base for the NAR cases.
98. 10:06So, uh we had the documents which had this information on a time-wise scale.
99. 10:14We built an LLM extraction which uses LLMs uh which uh is configurable.
100. 10:23So, we can configure it on different types of documents and we can perform the extraction and we can get the drug-wise constraint that for this pair this is the this is how it will be treated for a period of time.
101. 10:36Another thing is that we leverage some historical information.
102. 10:40That is that we know that a certain organization uh treats this drug in a certain sort of way.
103. 10:47So, we stored that information and use that as well.
104. 10:50Apart from this, we also use some regular portal checks to get this information.
105. 10:56So, yeah.
106. 10:57Uh performing this movement moved our needle forward a bit.
107. 11:02So, now we had these deterministic checks along with our initial ineligibility verification.
108. 11:09So, whatever were not verifiable were flagged out before and whatever moved forward, we could determine these statuses for the drugs and we were able to eliminate some of the orders from here.
109. 11:25So, now that we solved for two types of drugs, let's move forward.
110. 11:34So, the next problem that we faced here was that some decisions do actually need clinical reasoning.
111. 11:40So, so far the drugs that we were we shifted to no touch were the ones where the information was directly available or indirectly available via another source.
112. 11:52But, we did not need to perform any sort of reasoning over here or any sort of question answer.
113. 11:58However, for the drugs that are actually auth required, we need to uh check whether the patient is actually eligible for them and also give sort of supportive evidence from where we are answering this
114. 12:14we are giving this information.
115. 12:17So, in order to do this, we built our third agent, which is the medical necessity agent.
116. 12:23So, the medical necessity agent answers simple and complex clinical questions per patient uh and attaches confidence score to any answer.
117. 12:32So, we escalate only the ones that actually need a clinician.
118. 12:37Uh so, we did a lot of work on this space.
119. 12:40We actually have another have a publication here as well, which I've mentioned.
120. 12:44So, how this agent works is that we have a new order and for every order, we read three things.
121. 12:51So, we read the patient notes.
122. 12:54We read the policy criteria.
123. 12:56So, the policy criteria is basically the criteria that needs to be met in order for this drug to be processed further.
124. 13:04Uh we from this criteria, we query the patient medical graph, which is a graph of biomarkers that are extracted for a patient.
125. 13:14And we determine what all biomarkers exist and what is the condition of this patient from here.
126. 13:20And then we use this thing these this information and pass it to an LLM to get an answered questioner with all the supportive and contradictory facts.
127. 13:32So, yeah.
128. 13:34And if we have the relevant information or we are able to determine this with a higher confidence, we move this forward.
129. 13:42And for the cases we do not have this enough information, we move keep that for human escalation.
130. 13:51So, yeah.
131. 13:52This is the next step.
132. 13:54So, after this medical necessity validation, we move from the first step where we fetch the data, we did the eligibility verification, we flagged the orders that were not eligible,
133. 14:09moved the rest forward, extracted the not needed and already authorized letters.
134. 14:14Uh got the got the cases where we can move that without any human intervention.
135. 14:22And then we did the medical necessity validation.
136. 14:26Whatever was not met was also moved to review.
137. 14:30So, yeah.
138. 14:31Finally, we assemble all this information and then we submit it back to the payers.
139. 14:38So, this is where our submission agent comes into place.
140. 14:40The submission agent is pretty much similar as the EVBV agent.
141. 14:44We have customized integrations for every sort of payer that we have.
142. 14:48Which is built using an LLM driven config as well as our repository of tools.
143. 14:57So, yeah.
144. 14:57This is the entire graph that we got through the entire journey.
145. 15:03Uh So, the green part shows the cases that we are going without human touch.
146. 15:08So, whenever eligibility verification passes, then enough information is found to not to prevent the drug from going for further medical necessity evaluation.
147. 15:22Then this medical necessity evaluation and the submission.
148. 15:27So, yeah.
149. 15:29Finally, I'd like to mention that these drugs are also all although built initially for this case are being used across multiple workflows.
150. 15:40We have extended the functionality to a more general one where the medical necessity agent can answer questions specific to any sort of workflow for the patient.
151. 15:52Similarly for the other agents as well.
152. 15:56So, yeah.
153. 15:57As the conclusion, I would say the no touch is growing on the share of every order.
154. 16:03So, we started with deterministic checks.
155. 16:07Agents only for the rules that where what rules can't decide.
156. 16:11We used multi-source of evidence to beat the single source of evidence to add confidence to our cases.
157. 16:18We added itself healing in order to scale our RPS.
158. 16:24And finally, we added the reasoning layer in order to deal with the cases where actually authentic authorization is required.
159. 16:32So, yeah.
160. 16:33Thanks for the yeah, thanks for your time.
161. 16:37Yeah.
162. 16:39Bye-bye.