WEBVTT NOTE Sentence-level transcript of https://www.youtube.com/watch?v=CDqzWpwkSls NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/CDqzWpwkSls.html. A cue ends where the next begins, or 2 s after its last word. s1 00:00:00.760 --> 00:00:05.480 Hello, my name is Angel Ermanlee, and I'm a software engineer at Duolingo. s2 00:00:05.480 --> 00:00:08.680 I work on security for the Duolingo English Test. s3 00:00:08.680 --> 00:00:11.440 Today's talk is the human in your loop isn't thinking. s4 00:00:11.440 --> 00:00:15.480 Build AI systems for discernment, not approval. s5 00:00:15.840 --> 00:00:18.000 So, let's get started with some background. s6 00:00:18.000 --> 00:00:20.080 What is human-in-the-loop AI? s7 00:00:20.080 --> 00:00:30.880 Human-in-the-loop AI is a framework where a system or process is actively involving a human who's participating in the operation, supervision, or decision-making for an automated system. s8 00:00:30.880 --> 00:00:37.880 Humans are involved because they're able to ensure the accuracy, safety, or ethical decision-making for that system. s9 00:00:37.880 --> 00:00:43.600 They are a piece of that decision-making where AI normally falls short. s10 00:00:43.600 --> 00:00:52.160 You can kind of think of this as a linear process where a model provides some sort of output, and a human sees that and makes a decision. s11 00:00:52.160 --> 00:00:55.920 Now, in the age of AI, trust is a big part of the conversation. s12 00:00:55.920 --> 00:01:03.120 Recently, as technology has developed, a lot of things that used to be manual cognitive tasks are now part of technology. s13 00:01:03.120 --> 00:01:05.880 So, no one really memorizes phone numbers anymore. s14 00:01:05.880 --> 00:01:08.880 They're just sitting in our contact list on our smartphones. s15 00:01:08.880 --> 00:01:15.440 When you're driving over somewhere, you just put the directions into your GPS, and you don't really think about the details of that route. s16 00:01:15.440 --> 00:01:19.400 Instead, you just trust the GPS to come up with the optimal way for you. s17 00:01:19.400 --> 00:01:32.160 When you're searching something in the search engine, you used to look at the top results, and now, as AI is becoming more increasingly integrated into our day-to-day life, you might be seeing some AI summaries at the top instead. s18 00:01:32.160 --> 00:01:38.400 Here's an example of Googling COVID-19 symptoms, and you see that Gemini summary at the top. s19 00:01:38.400 --> 00:01:48.920 And instead of looking at the CDC website or the WHO website, you might actually look to that as your final answer to the question that you were asking to the search engine. s20 00:01:48.920 --> 00:02:00.520 And as these pieces of AI are becoming more integrated into your day-to-day life in these atomic little things like searching for an answer or getting a result from an app, s21 00:02:00.520 --> 00:02:03.880 your trust of AI systems will increase and caution will decrease. s22 00:02:03.880 --> 00:02:07.360 And this is something that's happening across all of society. s23 00:02:07.360 --> 00:02:10.560 A study at Wharton was looking exactly that. s24 00:02:10.560 --> 00:02:17.320 How is AI reshaping human reasoning and what does it mean for us as we continue to use it day-to-day? s25 00:02:17.320 --> 00:02:20.720 They've observed an interesting phenomenon called cognitive surrender. s26 00:02:20.720 --> 00:02:27.000 When a human foregoes deliberation and adopts AI output as their own with minimal scrutiny. s27 00:02:27.000 --> 00:02:35.560 They saw this in a study where humans were asked to look at reasoning exams and they were given AI resources to answer those questions. s28 00:02:35.560 --> 00:02:42.280 They realized that AI can either supplement or supplant a human's thinking and the human might not even know it's happening. s29 00:02:42.280 --> 00:02:50.160 For example, for questions where the AI was right, the human performance increased by 25 percentage points. s30 00:02:50.160 --> 00:02:53.480 Whereas when the AI was wrong, it decreased by 15. s31 00:02:53.480 --> 00:03:05.280 This suggests that the humans were taking in that AI cognition and not really thinking critically about what that answer is and instead using those answers as part of their own s32 00:03:05.280 --> 00:03:13.400 reasoning together with their human instinct and deliberate reasoning amplifying those results. s33 00:03:13.480 --> 00:03:20.519 Most interestingly, they saw that 80% of participants were accepting those AI answers even when they were wrong. s34 00:03:20.519 --> 00:03:31.760 So, they were lowering that barrier to entry and just trusting the AI even if they weren't really critically examining the correctness of that result. s35 00:03:31.760 --> 00:03:39.239 So, at the Duolingo English Test, we wanted to look at the same thing and we published some research titled When Machines Mislead. s36 00:03:39.239 --> 00:03:45.160 It was a case study on the English exam, specifically in AI human interaction. s37 00:03:45.160 --> 00:03:47.920 So, for context, what is the DET? s38 00:03:47.920 --> 00:03:55.840 The DET is a high-stakes exam for that measures your English proficiency and it's fully online and remotely proctored. s39 00:03:55.840 --> 00:04:06.680 You can take it on your own machine in the comfort of your home and it still provides high-quality results that 6,000 programs worldwide are trusting day-to-day. s40 00:04:06.680 --> 00:04:16.720 As you can imagine with a fully online assessment, there are some interesting things that from a technological standpoint that have to happen to ensure good quality security. s41 00:04:16.720 --> 00:04:27.560 This includes identity verification, a lockdown testing environment, as well as a variety of ways in which AI assisted monitoring is happening to predict different types of cheating. s42 00:04:27.560 --> 00:04:40.160 Lastly, we have a final round of scrutiny where we have human proctors review all of the video footage of the exam as well as all of those AI results. s43 00:04:40.200 --> 00:04:48.040 Specifically for the study, we wanted to target one of our AI cheating detection detection systems, which is copy typing. s44 00:04:48.040 --> 00:04:57.440 Copy typing is a form of cheating where you're writing down uh information that you're reading currently and supposed to something that you're thinking right now at this moment. s45 00:04:57.440 --> 00:05:03.240 So, as you can imagine, your typing patterns differ when you're transcribing versus composing. s46 00:05:03.240 --> 00:05:06.680 And this is something that our custom model is measuring. s47 00:05:06.680 --> 00:05:13.560 It's looking at anomalies in keystroke patterns and it's flagging them for uh sessions that are unusual. s48 00:05:13.560 --> 00:05:20.120 So, our model is highly conservative and so we're prioritizing fairness for our test takers. s49 00:05:20.120 --> 00:05:23.000 Uh so, this is not a very common flag. s50 00:05:23.000 --> 00:05:33.919 However, when humans are human proctors are taking a look at these, they're well trained to um properly examine those segments and see if this is a flag. s51 00:05:33.919 --> 00:05:41.200 So, for experiments, we wanted to answer the question, would a skilled reviewer catch a false alarm or would they just rubber stamp it? s52 00:05:41.200 --> 00:05:46.600 So, we know that our proctors are highly accurate at detecting various forms of cheating. s53 00:05:46.600 --> 00:05:54.880 We wanted to see specifically if we give them a fake signal, how well would they be able to discern that this is totally legit session. s54 00:05:54.880 --> 00:05:59.480 The way we did that is by selecting sessions that had no cheating whatsoever. s55 00:05:59.480 --> 00:06:09.400 They were totally okay, and we made it look like they have this AI signal that says, "Hey, check if there's copy typing behaviors present at a specific moment." s56 00:06:09.400 --> 00:06:14.160 We presented them to the proctors as part of their normal workflow. s57 00:06:14.160 --> 00:06:17.480 They thought it was just another session that they were taking a look at. s58 00:06:17.480 --> 00:06:21.600 Um and this did not impact our test takers in any way. s59 00:06:21.600 --> 00:06:23.560 Their results were quite interesting. s60 00:06:23.560 --> 00:06:40.880 So, we found that despite the fact that our human reviewers are consistently scoring above 90% on their accuracy calibration metrics, they actually accepted 50% of these fake signals, meaning they were falsely accusing people of cheating half half the time. s61 00:06:40.880 --> 00:06:47.040 And this coin flip rate is something that is a strong suggestion of automation bias. s62 00:06:47.040 --> 00:06:58.200 This means that they're deferring their judgment to AI uh without giving a second thought to whether or not that there's evidence to corroborate this and if this is a true s63 00:06:58.200 --> 00:07:00.080 uh cheating session. s64 00:07:00.080 --> 00:07:04.520 Uh again to reiterate, these were not sessions that were actually going to our test takers. s65 00:07:04.520 --> 00:07:08.800 These were historical sessions, so there was no impact to our customers. s66 00:07:08.800 --> 00:07:24.040 However, this is something that was definitely very alarming to the team because in a high-stakes exam where these results are going to influence college admissions and visa decisions, this is not something that we can take uh we can just accept. s67 00:07:24.040 --> 00:07:26.520 We knew that the problem was not the model. s68 00:07:26.520 --> 00:07:32.320 Our model has a 1% false positive rate, and also these were sessions that were negatively predicted. s69 00:07:32.320 --> 00:07:34.320 We knew that our problem was not the people. s70 00:07:34.320 --> 00:07:38.480 Our people are highly skilled and very experienced with detecting cheating. s71 00:07:38.480 --> 00:07:44.400 So, we wanted to take a look at the interface, and that's where we targeted our solution. s72 00:07:44.400 --> 00:07:49.680 So, we decided to take a look a look at the human-AI interaction loop. s73 00:07:49.680 --> 00:07:55.760 Specifically, our proctors are given a series of guidelines about how to interact with the system that they have. s74 00:07:55.760 --> 00:08:00.080 And we wanted to just update that proctoring guideline to emphasize two things. s75 00:08:00.080 --> 00:08:03.680 First, the AI signal is just a preliminary alert. s76 00:08:03.680 --> 00:08:05.760 They're the final decision-maker. s77 00:08:05.760 --> 00:08:12.760 And second, when they're making that decision, they must find independent evidence in the video footage before upholding a flag. s78 00:08:12.760 --> 00:08:29.680 This simple copy change led a 21% increase in rejection rates, meaning that from 50% only half of sessions that were rejected uh the majority 71 were now determined to be totally okay. s79 00:08:29.680 --> 00:08:36.360 And this is really good because it is more closely matching sessions in production. s80 00:08:36.719 --> 00:08:43.320 And for an engineer, this means that we didn't have to tweak the model and we didn't have to tweak uh the UI in some way. s81 00:08:43.320 --> 00:08:51.440 We simply had to take a look at the copy change and this really changed the way that people were interfacing with that AI result. s82 00:08:51.440 --> 00:08:55.200 So, what are the implications of this? s83 00:08:55.240 --> 00:09:03.880 As an AI engineer, your interaction loop determines how effective your AI system is and also what you can learn from it. s84 00:09:03.880 --> 00:09:13.440 The way that you measure efficacy and the way that you can uh and the data that you collect from that to then drive your next iteration are really important. s85 00:09:13.440 --> 00:09:23.000 Earlier we talked about the uh human in the human in the loop AI as a linear interaction where a model goes to a human goes to a decision. s86 00:09:23.000 --> 00:09:28.160 However, in reality that process isn't linear, it's cyclical. s87 00:09:28.160 --> 00:09:38.320 Models provide some sort of output that goes into an interaction that goes into a human behavior which yields data for your evals and then later on your model. s88 00:09:38.320 --> 00:09:42.840 You can't really change the way a human behaves unless you are that human. s89 00:09:42.840 --> 00:09:48.680 But, what you can do is tweak that interaction such that it can elicit different results from that human behavior. s90 00:09:48.680 --> 00:09:50.200 And that data is golden. s91 00:09:50.200 --> 00:10:00.040 It leads to better analytics, model improvements, and future iterations, including the next generation of your model, or maybe some different product. s92 00:10:00.440 --> 00:10:06.280 Now, zeroing in on that interaction portion, we can look at how to evaluate your system. s93 00:10:06.280 --> 00:10:11.760 Ask yourself a question, "How can I measure and improve the efficacy of my AI system?" s94 00:10:11.760 --> 00:10:24.160 Looking at structured interactions that were designed intentionally, you can see that those interactions yield good data that are labeled signals that can then become training data and evals for a better model. s95 00:10:24.160 --> 00:10:26.640 And that is a flywheel that compounds. s96 00:10:26.640 --> 00:10:34.760 As you have the better model, you can have better data that then has better interactions with your human, which then continues to cycle. s97 00:10:34.760 --> 00:10:41.280 Think of your structured interactions as a system property that specifically yields high-quality data. s98 00:10:41.280 --> 00:10:46.080 That high-quality data is unlocking meaningful insights and quicker development iterations. s99 00:10:46.080 --> 00:10:51.360 You're not going to be spending days cleaning your data or trying to find useful insights. s100 00:10:51.360 --> 00:10:57.240 Instead, you're going to have structured data that is directly impacting your next iteration. s101 00:10:58.080 --> 00:11:00.320 And this is a compounding effect. s102 00:11:00.320 --> 00:11:02.080 Uh and it's cyclical. s103 00:11:02.080 --> 00:11:12.280 If you are not being intentional with that design, you can get stuck in a vicious cycle where your model is making confident calls, and because your interface is not actually s104 00:11:12.280 --> 00:11:16.960 uh eliciting that deliberation from a human, they just end up rubber stamping it. s105 00:11:16.960 --> 00:11:19.880 And that positive signal is logged as truth. s106 00:11:19.880 --> 00:11:25.760 So, over time, your model becomes more confident, and the human is not encouraged to think further. s107 00:11:25.760 --> 00:11:31.120 So, they continue to defer to the AI, and the AI becomes the person in the driving seat. s108 00:11:31.120 --> 00:11:34.080 Uh instead, what you want is a virtuous cycle. s109 00:11:34.080 --> 00:11:44.360 Uh, if you have an interface that forces independent judgment, uh, this allows the human to really think critically about the decisions they're making and have real disagreements surface. s110 00:11:44.360 --> 00:11:56.600 That means that you have true positive and negative labels that are honest and get logged to then continue to have model improvements that are targeting exactly where the model goes wrong. s111 00:11:56.600 --> 00:11:59.600 So, let's take a look at some examples. s112 00:11:59.800 --> 00:12:04.120 For example, we have other cheating flags that are driven by AI. s113 00:12:04.120 --> 00:12:06.520 As an example, we have headphone detection. s114 00:12:06.520 --> 00:12:14.360 On the left, you can see, uh, a bad pattern that we had where we were saying, "Hey, the model detected headphones at this moment. s115 00:12:14.360 --> 00:12:16.120 Can you flag it here?" s116 00:12:16.120 --> 00:12:19.160 And it was just asking a simple yes or no question. s117 00:12:19.160 --> 00:12:25.640 We realized that there's actually two questions hidden in this single, uh, single CTA. s118 00:12:25.640 --> 00:12:28.280 Uh, one, we're asking headphones were detected. s119 00:12:28.280 --> 00:12:30.360 Is that true or false? s120 00:12:30.360 --> 00:12:36.960 Uh, did the model correctly predict that at this video segment there are some set of pixels that look like headphones? s121 00:12:36.960 --> 00:12:46.160 And secondly, are we applying a flag that this person had a violation for it not correctly using headphones at this moment? s122 00:12:46.160 --> 00:12:47.200 Why is this important? s123 00:12:47.200 --> 00:12:55.839 For example, if there's somebody who has a hearing aid, that means that the model correctly predicted that there are headphones or earbuds detected. s124 00:12:55.839 --> 00:12:58.760 Uh, that is a true signal. s125 00:12:58.760 --> 00:13:03.320 However, we don't actually want to penalize this person for wearing, uh, hearing aids. s126 00:13:03.320 --> 00:13:06.280 So, we don't want to flag them for a violation. s127 00:13:06.280 --> 00:13:14.000 Previously, as to not falsely accuse somebody of cheating, we would select no, but that would be a bad signal to give to our model. s128 00:13:14.000 --> 00:13:23.280 So, breaking this up into two pieces is really important because we actually get more data and it's better quality and we're not harming the model in the long run. s129 00:13:23.280 --> 00:13:25.680 Another example is a quality tutor. s130 00:13:25.680 --> 00:13:32.240 On the left here, you see a pattern where the interaction is overwhelming and on the right delightful. s131 00:13:32.240 --> 00:13:36.080 So, this is an example of me talking to an LLM. s132 00:13:36.080 --> 00:13:41.240 I asked it to act as a writing tutor and provide feedback for a passage that I wrote. s133 00:13:41.240 --> 00:13:48.480 The passage was specifically written as if it was a English language learner, so it had a few mistakes or awkward sentences. s134 00:13:48.480 --> 00:13:59.880 It was a relatively brief paragraph and instead this LLM decided to give me 400 lines of text and it was very overwhelming and hard to parse. s135 00:13:59.880 --> 00:14:07.560 First, it decided to praise me, then give some sort of feedback and then completely rewrite my passage even though I never asked it to. s136 00:14:07.560 --> 00:14:09.400 The feedback was not direct. s137 00:14:09.400 --> 00:14:20.440 It was hard to tie feedback to specific parts of the input and it was so large that it would be difficult to actually improve your writing through this sort of interaction. s138 00:14:20.440 --> 00:14:22.640 And most importantly, this did not feel natural. s139 00:14:22.640 --> 00:14:28.080 If you were to send an email to your friend saying, "Hey, can you take a look at this thing that I wrote?" s140 00:14:28.080 --> 00:14:35.640 you would not expect them to send you an essay double the length telling you all the things that you did right and wrong and I'll also give you a whole new version. s141 00:14:35.640 --> 00:14:38.720 Instead, you would probably want it to look something like this. s142 00:14:38.720 --> 00:14:46.160 Here you have a screenshot of a Duolingo style writing tutor where the text is directly marked up. s143 00:14:46.160 --> 00:14:53.040 The green is signifying that this is something that was good and also providing some sort of feedback when you hover over it. s144 00:14:53.040 --> 00:14:58.760 Yellow signifying something is awkward or a little bit off and red being direct mistakes. s145 00:14:58.760 --> 00:15:07.000 When you hover over all of these markups, you can get direct actionable feedback that's concise and you can accept the suggestions in line. s146 00:15:07.000 --> 00:15:12.280 Most importantly, all the feedback is directly tied to a portion of text. s147 00:15:12.280 --> 00:15:15.440 This is more closely mimicking human behavior. s148 00:15:15.440 --> 00:15:26.560 Specifically, like in an academic setting, if you're handing your essay to a classmate, you would expect feedback in line as they mark up your text rather than an essay back. s149 00:15:26.560 --> 00:15:35.760 This is useful because over time, if you get these sorts of feedback [clears throat] points, you can improve in incremental steps in your writing. s150 00:15:35.760 --> 00:15:41.400 And this is really important for surfacing the exact same AI output. s151 00:15:41.400 --> 00:15:46.680 Lastly, and probably most applicably to an engineering audience, is a coding agent. s152 00:15:46.680 --> 00:15:59.440 Now, there are a lot of coding coding agents out there, and there's a lot of different UIs, people have their personal preferences, but I've seen that there's two patterns that a lot of out of the box coding agents are falling into. s153 00:15:59.440 --> 00:16:09.360 So, one pattern is that if you ask it to do something, it's going to touch a lot of files and give you a giant diff, basically doing the thing that you requested all in one go. s154 00:16:09.360 --> 00:16:18.320 This leads you probably to approve all the changes and then take a look at them maybe on GitHub, so that you can see all the things that it did. s155 00:16:18.320 --> 00:16:26.960 Another approach is that it will ping you a notification every single time it's changing some sort of file or function, and you have to keep clicking yes, yes, yes, s156 00:16:26.960 --> 00:16:29.880 just so that you can get to the result. s157 00:16:29.880 --> 00:16:34.160 Either way, in both of these cases, you're just becoming a rubber stamp. s158 00:16:34.160 --> 00:16:40.200 You're accepting all of the changes, and then you're taking a look at them all in one go and debugging later. s159 00:16:40.200 --> 00:16:46.800 Anything that went wrong is your responsibility to then identify, fix, re-prompt, or do manually. s160 00:16:46.800 --> 00:16:50.480 And it's not a very delightful experience. s161 00:16:50.480 --> 00:16:54.800 Instead, you probably want a coding agent that acts like a junior developer. s162 00:16:54.800 --> 00:16:59.880 You wouldn't want a junior developer to come onto your team and just be like, "Oh, sure, I can do that." s163 00:16:59.880 --> 00:17:03.320 and disappear for a day and then give you a thousand line PR. s164 00:17:03.320 --> 00:17:08.199 You also probably don't want a junior dev who's asking you a question every five minutes at your desk. s165 00:17:08.199 --> 00:17:20.480 Instead, you want someone who's able to plan, ask good questions, and document those design decisions, giving you a good spec and breaking up their PRs in meaningful and reviewable ways. s166 00:17:20.480 --> 00:17:32.960 This means that it's delightful for you as a mentor or in this case developer who's partnering with a coding agent to experience things in a way that is easily reviewable. s167 00:17:32.960 --> 00:17:44.520 You can highlight assumptions and maybe answer questions about them and you can easily see a plan and the things that the decisions that it's making before things go wrong. s168 00:17:44.520 --> 00:17:49.960 Also, the steps are manageable and you're not being pinged too many times. s169 00:17:50.160 --> 00:18:03.800 And then going a little bit into the data of side of this, if you have agentic coder and it's leading an interaction where a developer is encouraged to just quickly skim everything and not really discern. s170 00:18:03.800 --> 00:18:16.160 This means that you're just collecting these binary signals of accepted and rejected, usually skewing towards accepted, and that doesn't really give you that much information because it's just on some coding block. s171 00:18:16.160 --> 00:18:31.520 Instead, if an agent is acting as a partner with a developer steering, you can get rich data because it's structured specifically around different parts of the development cycle and it's capturing nuanced structured decisions about the different parts of s172 00:18:31.520 --> 00:18:32.760 the tasks that it's doing. s173 00:18:32.760 --> 00:18:41.280 For example, it can look at assumptions that were bad, the tradeoffs that it made, stylistic preferences and approaches that the developer's taking. s174 00:18:41.280 --> 00:18:50.960 And all of those pieces of data can then continue to make your experience as a like as your AI system coding system better. s175 00:18:50.960 --> 00:18:53.800 So, let's look at some design principles. s176 00:18:53.800 --> 00:18:56.840 First, you can engineer the reasoning. s177 00:18:56.840 --> 00:19:03.360 Think about what reasoning pattern do you want to elicit from a human and how does your interface challenge that? s178 00:19:03.360 --> 00:19:14.360 So, if you have a system that is specifically optimizing for independent judgment, reframe your system to think of the human as investigator, not just a validator. s179 00:19:14.360 --> 00:19:21.040 Don't just give a hey, is this looking good, but instead have it be a more thoughtful engaged effort. s180 00:19:21.040 --> 00:19:28.640 Surfacing assumptions is something that's very valuable when you care about a lot about the quality of a long-running task. s181 00:19:28.640 --> 00:19:37.120 If the model is stating them earlier on and asking for a sign-off, you can prevent any future miscommunication or having to go back and fix your assumptions. s182 00:19:37.120 --> 00:19:48.320 Weighing trade-offs is also very important because very often an LLM might give you some sort of option and it thinks it's best without having to actually weigh the different s183 00:19:48.320 --> 00:19:52.680 different trade-offs and design decisions that it's making in the background. s184 00:19:52.680 --> 00:20:05.280 Presenting those options and the reasoning for why allows the user to continue to be in control over the output and they can opine earlier on so that they can get the thing that they actually want. s185 00:20:05.280 --> 00:20:10.240 Lastly, for sustained attention, you want to build in friction exactly where the stakes are high. s186 00:20:10.240 --> 00:20:18.960 If you want the person to slow down and think deliberately, friction is your friend, which leads us to the next principle to match the friction to the stakes. s187 00:20:18.960 --> 00:20:33.520 In a high-stakes example like ours where the Duolingo English Test really matters for people and their livelihood, we want our human reviewers to be deliberately slow and very thoughtful with the decisions that they make. s188 00:20:33.520 --> 00:20:43.200 This means that we want to be adding review gates that are well-structured and friction so they can slow down and think about these things so that they can never become a rubber stamp. s189 00:20:43.200 --> 00:20:43.220 You s190 00:20:43.220 --> 00:20:43.360 [snorts] s191 00:20:43.360 --> 00:20:49.160 want these people the people to slow down and you want the output to remain high quality. s192 00:20:49.160 --> 00:20:52.000 You don't want noise and you want clarity. s193 00:20:52.000 --> 00:21:02.240 So, this means that you have to be actively thinking about what friction like what speed bumps you want to put along the road and how and what checkpoints. s194 00:21:02.240 --> 00:21:12.520 For systems where you have low oversight, for example, something delightful where you're just chatting with an AI, you want to make sure that it's relatively frictionless. s195 00:21:12.520 --> 00:21:26.840 Your design has to optimize for a happy someone who's happy with that output and doesn't really feel like there's a lot of stopping points or scenarios where it's kind of difficult to interact with the system. s196 00:21:26.840 --> 00:21:30.400 Instead, you want it to be absolutely seamless. s197 00:21:31.080 --> 00:21:35.480 Next principle is every interaction is already a label. s198 00:21:35.480 --> 00:21:48.720 Instead of having to select a portion of your data and have human annotators get you a better cleaner data set, you can kind of already start thinking about that specific interaction piece s199 00:21:48.720 --> 00:21:53.440 as already providing you labels and signals for your next iteration. s200 00:21:53.440 --> 00:22:01.640 So, if the agent plan was approved or a suggestion was accepted, this means that you did something right and your output matched the intent of the user. s201 00:22:01.640 --> 00:22:07.960 However, if the output was modified or in a recommendation was overridden, that probably signifies something bad. s202 00:22:07.960 --> 00:22:23.360 However, a lot of systems don't really capture that diff and the model falls short because what you get on that decision is a yes or no. And the human clicked yes, then manually went in and changed something or erased it completely. s203 00:22:23.360 --> 00:22:35.640 If you don't capture that diff, which could be something for the the AI falling short or maybe completely being wrong, you actually capture a false signal that then can pollute your data sets. s204 00:22:35.640 --> 00:22:39.560 Instead, think about how can you measure that diff? s205 00:22:39.560 --> 00:22:42.680 Lastly, there are also questions that a human might be asking. s206 00:22:42.680 --> 00:22:45.240 For example, explanations or follow-ups. s207 00:22:45.240 --> 00:22:51.040 And this could mean that the human has a low trust in the system or there was something wrong with the AI. s208 00:22:51.040 --> 00:22:58.000 Uh so, take a look at the kind of questions that are being asked and what sentiment they have. s209 00:22:58.200 --> 00:23:02.720 Another principle is to stop asking how to evaluate the model. s210 00:23:02.720 --> 00:23:09.640 Instead of building your system and then being like, "Hey, what kind of data can I capture to see if this is good?" s211 00:23:09.640 --> 00:23:14.080 Proactively think about what defines success for your AI system. s212 00:23:14.080 --> 00:23:21.320 Think about how you can be measuring that with concrete metrics and what data will you need to improve the system from the start. s213 00:23:21.320 --> 00:23:30.960 Those decisions will be informing the way you design that interaction such that you can ensure that you have the hard evidence for that next step. s214 00:23:30.960 --> 00:23:39.000 Every interaction should be helping the user to teach them how to improve the system and be a partner with the AI. s215 00:23:40.040 --> 00:23:46.800 This leads me to my final point, which is all of those things are leading you to engineer that interaction. s216 00:23:46.800 --> 00:23:53.120 There are different ways to engineer that interaction, which I've covered briefly, but some of the principles are this. s217 00:23:53.120 --> 00:23:56.200 First, you can have structured inputs and outputs. s218 00:23:56.200 --> 00:23:59.200 That way you don't just have vibes or giant walls of text. s219 00:23:59.200 --> 00:24:00.920 Instead, you have specifics. s220 00:24:00.920 --> 00:24:15.680 You can think of this in terms of forms that the user is filling out, structured output in forms of tables, markup UIs, for example, that writing tutor or maybe it's a design thing where you can highlight specific elements. s221 00:24:15.680 --> 00:24:22.680 All of those ways are having targeted interactions between the human and the AI ensuring a better result. s222 00:24:22.680 --> 00:24:25.520 Next, highlighting that those assumptions. s223 00:24:25.520 --> 00:24:38.240 If you are having assumptions being made by the model, if you surface them proactively and ask if it's legit, that's going to save you some tokens down the line so that the human is not having to fix or argue or correct with the correct the model s224 00:24:38.240 --> 00:24:38.800 output. s225 00:24:38.800 --> 00:24:48.360 Instead, it's being proactive and the having those things that are core pieces of information that are going to inform later decisions. s226 00:24:48.360 --> 00:24:51.480 Next is building in friction and review gates. s227 00:24:51.480 --> 00:25:05.440 That deliberate slowing down so that you can have good quality thought and just required for good quality decision-making and good quality data is all important because if you're bringing in that friction and review gates, s228 00:25:05.440 --> 00:25:14.280 people are going to start thinking clearly as if it's really their own thing rather than just something that AI is supplanting. s229 00:25:14.400 --> 00:25:22.760 Lastly, collecting explicit feedback is really important because as we've covered, every one of those interactions is a piece of data. s230 00:25:22.760 --> 00:25:34.400 So, all of those signals are going into your next generation of the model, and collecting explicit feedback, not just a thumbs up, thumbs down, but instead feedback at the correct touch points, s231 00:25:34.400 --> 00:25:41.040 uh with the correct amount of nuance is something that can really drive improvements directly. s232 00:25:41.040 --> 00:25:44.880 So, yes, uh you should be designing for discernment. s233 00:25:44.880 --> 00:25:51.400 Sometimes the fix is not a better model or more oversight, it's just engineering the interaction itself. s234 00:25:51.400 --> 00:25:53.760 Thank you.