WEBVTT NOTE Sentence-level transcript of https://www.youtube.com/watch?v=4uFVSLgD2Q4 NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/4uFVSLgD2Q4.html. A cue ends where the next begins, or 2 s after its last word. s1 00:00:01.000 --> 00:00:01.680 Hi everyone. s2 00:00:01.680 --> 00:00:03.200 My name is Gabe de Mesa. s3 00:00:03.200 --> 00:00:13.600 I'm an engineer here at OpenGov and today we're going to be talking about agents in production, specifically how OpenGov built and scaled OG Assist. s4 00:00:13.600 --> 00:00:21.120 Uh so um this presentation is going to be jam-packed with just so much good stuff. s5 00:00:21.120 --> 00:00:23.440 Uh we're going to talk about uh AI agents. s6 00:00:23.440 --> 00:00:25.040 We're going to talk about our harness. s7 00:00:25.040 --> 00:00:30.560 We're going to talk about um evals, observability, traces. s8 00:00:30.560 --> 00:00:34.560 We're going to talk about um tools and skills. s9 00:00:34.560 --> 00:00:38.000 Um It's There's going to be a lot of good stuff in here. s10 00:00:38.000 --> 00:00:50.600 We're going to talk to you guys about uh what we do at OpenGov and how we operate at the scale that uh we operate at um in production, so you'll be able to see a real use case and workload s11 00:00:50.600 --> 00:00:51.960 uh with AI agents. s12 00:00:51.960 --> 00:00:55.680 Um so without further ado, let's get started. s13 00:00:56.120 --> 00:00:57.160 Okay, agenda. s14 00:00:57.160 --> 00:01:03.040 So just really quickly going to go through uh high-level what we're going to talk about today. s15 00:01:03.040 --> 00:01:07.560 Uh I'm going to tell you guys a little bit about OG Assist and what uh OpenGov is. s16 00:01:07.560 --> 00:01:12.280 I'm going to tell you guys the origin story of how this all kind of came to be. s17 00:01:12.280 --> 00:01:17.760 Uh we're going to talk about OG Assist's uh big bet on effect. s18 00:01:17.760 --> 00:01:20.160 Uh a little bit into our core agent loop. s19 00:01:20.160 --> 00:01:24.320 Uh we're going to talk about the A2A protocol, evals, and sandboxing. s20 00:01:24.320 --> 00:01:27.800 We're going to talk about how we manage long context. s21 00:01:27.800 --> 00:01:35.080 We're going to talk about um monitoring, observability, how we collect feedback uh and how we iterate on that feedback. s22 00:01:35.080 --> 00:01:51.040 We're going to lastly uh also talk about tools and skills and how at OpenGov uh we use um AI not only externally uh that we uh serve to customers, but also internally to improve our development workflows. s23 00:01:51.600 --> 00:01:54.000 Just a little bit about me before we go any further. s24 00:01:54.000 --> 00:01:55.280 My name is Gabe. s25 00:01:55.280 --> 00:01:57.680 I'm a software engineer here at OpenGov. s26 00:01:57.680 --> 00:02:07.920 I work on the AI agents team and I'm one of the folks that helped build OG Assist and some of the systems that you guys will be seeing today. s27 00:02:07.920 --> 00:02:09.200 So, a little bit about OpenGov. s28 00:02:09.200 --> 00:02:16.040 OpenGov is a software company on a mission to power more effective and accountable government. s29 00:02:16.040 --> 00:02:18.000 So, OpenGov sells ERP software. s30 00:02:18.000 --> 00:02:21.880 That's things like budgeting, procurement, asset management, and permitting. s31 00:02:21.880 --> 00:02:38.040 And we were founded about 14 years ago and what's cool is um we have this thing called OG Assist and OG Assist is this little button on the top of all of our products in the in the navigation bar. s32 00:02:38.040 --> 00:02:48.920 And what's cool is all of our product suites and product teams have built tools and skills in order to power this button. s33 00:02:48.920 --> 00:03:03.160 So, for example, if I open up this this if I click this button and I open up OG Assist, it says, "Hey, I'm going to ask about rate codes, which is very specific to utility billing, the current product that I'm in." s34 00:03:03.160 --> 00:03:12.280 And you can see that inside of this kind of chat interface, I'm able to speak to an agent and the agent is able to make tool calls in order to s35 00:03:12.280 --> 00:03:15.720 look up information against data inside of that suite. s36 00:03:15.720 --> 00:03:26.400 So, it's really cool to be able to kind of first-party create these experiences through the capability that we've built called OG Assist. s37 00:03:27.320 --> 00:03:31.440 Okay, so just a quick story about how this all came to be. s38 00:03:31.440 --> 00:03:42.200 So, a little while back we we we saw that AI was really starting to take off and a principal uh spun up this new team called the AI agents team and asked me to join. s39 00:03:42.200 --> 00:03:51.440 And um instantly I said yes and OG Assist started to to grow and we started to integrate OG Assist into all our products. s40 00:03:51.440 --> 00:03:56.360 And uh not only our back-end capabilities, but also our front-end capabilities as well. s41 00:03:56.360 --> 00:04:06.600 So, you'll see that one of the capabilities that we give the agent is it's able to um see what's on the screen and and see and and and take action s42 00:04:06.600 --> 00:04:08.080 on what's on the page. s43 00:04:08.080 --> 00:04:12.200 So, you could see that uh I'm asking the agent here, "Hey, hey, what's on the screen? s44 00:04:12.200 --> 00:04:16.400 Can you maybe highlight uh some of the next steps that I could take?" s45 00:04:16.400 --> 00:04:19.120 So, you can see that the agent here is thinking. s46 00:04:19.120 --> 00:04:21.640 It's saying, "Okay, what tools do I have available to use?" s47 00:04:21.640 --> 00:04:26.840 And "Hey, let me go and highlight something that you could actually click on and and tell you more about it." s48 00:04:26.840 --> 00:04:29.480 So, just another capability of OG Assistant. s49 00:04:29.480 --> 00:04:34.600 Just a little short story about how this all came to be. s50 00:04:35.120 --> 00:04:37.400 So, the big bet on Effect. s51 00:04:37.400 --> 00:04:47.880 Um so, I really wanted to include this slide because um here on the Agents team, we made a huge bet to um to to bet on Effect. s52 00:04:47.880 --> 00:04:52.040 And suffice to say it's paid off in dividends. s53 00:04:52.040 --> 00:04:54.440 Um we write Effect. s54 00:04:54.440 --> 00:04:56.960 So, Effect is this library for TypeScript. s55 00:04:56.960 --> 00:05:01.960 It's open source and it helps you write better um TypeScript code. s56 00:05:01.960 --> 00:05:08.880 Uh you know, it's got a lot of uh stuff baked in it like a a schema similar to like Zod, if you've ever used that. s57 00:05:08.880 --> 00:05:17.840 It's also got um things for error handling, uh for logging, for traces, for uh it's just got so much in there. s58 00:05:17.840 --> 00:05:28.080 It really helps write better code and structure your code better and uh helps with architecture, spinning up new services for uh and and for us on the Agents team, really s59 00:05:28.080 --> 00:05:31.520 helping uh design and build the the core agent loop. s60 00:05:31.520 --> 00:05:40.680 So, you'll see throughout this presentation sprinkled in um how Effect on our team uh has paid off in dividends. s61 00:05:40.680 --> 00:05:50.080 So, we we really love Effect here at OpenGov and we encourage other folks to try it out and um yeah, let's keep going. s62 00:05:50.800 --> 00:05:52.600 The Effect Native Loop. s63 00:05:52.600 --> 00:06:02.520 So, originally we were on LangGraph and that was fine until the team really started to scale uh and our use cases started to evolve. s64 00:06:02.520 --> 00:06:17.520 So, we decided to move over to our own kind of Effect Native Agent Loop to have full regency over this uh Agent Loop such that if we have complex use cases or features that we need to build, s65 00:06:17.520 --> 00:06:22.880 we could kind of get in we we had full control of the of the Agent Loop. s66 00:06:22.880 --> 00:06:25.400 And not only that, but now we're fully on Effect. s67 00:06:25.400 --> 00:06:41.720 So, all the cool things you get with Effect is now propagated throughout the entire Agent Loop, like the tracing, structured concurrency, the logging, everything is more fine-grained control, and it it really allows us to really unlock the full potential s68 00:06:41.720 --> 00:06:45.320 uh having our own Agent Loop from the ground up. s69 00:06:45.320 --> 00:06:51.320 Um so, another thing I wanted to mention is on the left side, you'll see a code example. s70 00:06:51.320 --> 00:06:56.160 This is really the basics of the Effect Loop that we're using. s71 00:06:56.160 --> 00:07:05.240 Uh we're using this thing called the Effect AI package, and in that package, there's this thing called um there's a chat and a language model. s72 00:07:05.240 --> 00:07:16.840 So, with the chat, you can instantiate like an a chat, for example, and then you could stream text using um that that kind of stream text function, you could pass in a prompt. s73 00:07:16.840 --> 00:07:28.919 And what's cool is uh with a language model under the hood uh of since we're kind of doing dependency injection, we could pass in a different language model if we were to s74 00:07:28.919 --> 00:07:31.040 uh hot swap to another one, for example. s75 00:07:31.040 --> 00:07:43.320 So, really just having full control of our own Agent Loop just kind of gives us all the levers, and it really just unlocks the full capabilities of the model, and uh for the team as well, to have full s76 00:07:43.320 --> 00:07:46.360 agency over this this loop. s77 00:07:46.360 --> 00:07:50.480 Another thing I wanted to mention is the agent-to-agent protocol. s78 00:07:50.480 --> 00:07:54.600 So, here on the agents team we've had a lot of success with this protocol. s79 00:07:54.600 --> 00:07:58.600 So, this protocol being the protocol that Google created. s80 00:07:58.600 --> 00:08:08.360 Um kind of an open protocol for agents to intercommunicate, but um we found this very useful for uh defining our agent routes. s81 00:08:08.360 --> 00:08:15.600 So, for example, in the back end and our model and our schema to follow this kind of agent protocol. s82 00:08:15.600 --> 00:08:24.720 So, we modeled So, for example, there's this thing called an agent card which you see here and it's got the name of the agent, a description, etc., right? s83 00:08:24.720 --> 00:08:40.120 And having this kind of rigorous protocol, this rigorous spec really helped drive our development and drive alignment because, you know, all we had to do was um align with this spec and follow this spec and s84 00:08:40.120 --> 00:08:48.240 we knew that this was kind of the contract that our front end and back end would both consume and and produce. s85 00:08:48.240 --> 00:08:54.480 So, um this uh I would say also has been uh very helpful for us. s86 00:08:54.480 --> 00:08:57.880 And And what's really cool is A2A has a lot of extensions, right? s87 00:08:57.880 --> 00:09:01.960 So, you could extend the protocol uh add in like metadata. s88 00:09:01.960 --> 00:09:04.160 Uh there's also A2UI. s89 00:09:04.160 --> 00:09:12.080 Um so, lots of fun stuff uh with A2A protocol, but uh this is kind of what's worked for us. s90 00:09:12.080 --> 00:09:15.760 So, just sharing that with with you folks. s91 00:09:16.000 --> 00:09:18.400 Feedback and evals. s92 00:09:18.400 --> 00:09:22.280 So, here the quote is, "Shipping is the start, not the finish." s93 00:09:22.280 --> 00:09:29.720 So, what we do here uh on the agents team is we have kind of multiple ways we do evals and collect feedback. s94 00:09:29.720 --> 00:09:41.920 Um Obviously, you know, we'll have folks uh call in or or email us or just let us know and tell us, but the main way is we have this thumbs up and thumbs down mechanism and here s95 00:09:41.920 --> 00:09:44.360 someone is able to tell us, "Hey, this this worked really well. s96 00:09:44.360 --> 00:09:45.360 This was a great response." s97 00:09:45.360 --> 00:09:46.720 Or, "That wasn't a great response." s98 00:09:46.720 --> 00:09:50.080 And that signal we take and we're able to iterate on. s99 00:09:50.080 --> 00:09:55.760 Uh, and we can take it back and help improve, uh, you know, the response in the future. s100 00:09:55.760 --> 00:10:07.840 Um, we also have automated evals, so in in the in our CI we we have evals that run against real completions, so we could test the prompt against, "Hey, did it hit some tools? s101 00:10:07.840 --> 00:10:09.760 Did it do what it's supposed to do?" s102 00:10:09.760 --> 00:10:11.880 And that also helps with our accuracy. s103 00:10:11.880 --> 00:10:27.400 So, uh, those automated evals in conjunction with collecting feedback really help us, um, improve our uh, our our tools, our skills, um, our harness and and that's really how s104 00:10:27.400 --> 00:10:31.280 how we're able to iterate so fast and so quickly. s105 00:10:31.440 --> 00:10:32.600 Humans in the loop. s106 00:10:32.600 --> 00:10:41.400 So, this is a really cool feature we built where we deterministically interrupt the agent loop if there is a tool call approval required. s107 00:10:41.400 --> 00:10:50.160 So, if an agent tries to make a tool call that it needs human approval for, it'll show this UI and the human uh, can click accept or reject. s108 00:10:50.160 --> 00:10:55.760 So, explicitly rejecting or explicitly accepting, uh, the action that the agent is trying to make. s109 00:10:55.760 --> 00:11:06.360 And this ensures that, uh, you know, we're building trust and also ensuring that, uh, you know, we're being safe, especially when the agent's trying to do a mutating operation s110 00:11:06.360 --> 00:11:13.080 and always always always making sure that, um, humans are in the driver's seat. s111 00:11:13.560 --> 00:11:14.280 Sandboxing. s112 00:11:14.280 --> 00:11:27.640 So, another thing that we, uh, worked on, um, kind of similar to the safety slide we just saw was, um, whenever an agent tries to execute code or tries to create files, s113 00:11:27.640 --> 00:11:29.480 it does so in a sandbox. s114 00:11:29.480 --> 00:11:44.000 So, we gave our agents sandboxes such that it could spin up these sandboxes on demand and it could use those sandboxes to honestly write code, execute code, create files and it's kind of this safe ephemeral s115 00:11:44.000 --> 00:11:55.760 isolated space such that the agent can can can take action in there and not and we don't have to worry about uh any risk to you know our our production systems. s116 00:11:55.760 --> 00:12:09.080 Um and it's it's really cool cuz they also get uh tied uh tear down at the end so um in this example I said, "Hey, create a PDF uh for the folks of the AI Engineer Conference 2026." s117 00:12:09.080 --> 00:12:12.680 And um allow me to download it so I could share it with them. s118 00:12:12.680 --> 00:12:17.680 And you could see that the agent created this really cool PDF inside of that sandbox. s119 00:12:17.680 --> 00:12:25.680 So just really wanted to cover the sandbox feature and just give you a brief kind of overview of of sandboxing. s120 00:12:27.040 --> 00:12:28.160 Long context. s121 00:12:28.160 --> 00:12:43.839 So um inside of Ogie Assist, we have hit many hurdles like with especially with legacy models now like with uh token limits or just way too much just completely s122 00:12:43.839 --> 00:12:49.160 overloaded with context especially as conversations get longer. s123 00:12:49.160 --> 00:13:06.360 So we found that having some sort of um rolling summarization was more effective than you know always stuffing in the latest and most recent uh messages uh rather just s124 00:13:06.360 --> 00:13:18.320 you know give like a running summary after n number of messages and uh maybe you only want the like n minus five most recent messages or n minus 10 most recent messages, right? s125 00:13:18.320 --> 00:13:29.160 Um and it may be that you're only talking about a specific topic now, but you may want to refer to uh context earlier like a hundred messages above, then s126 00:13:29.160 --> 00:13:41.720 um it uh that's where kind of the the memory component comes in cuz when you have this rolling summary of a a really long conversation, then you could do recall over that uh summarization. s127 00:13:41.720 --> 00:13:46.120 And um you know, if you ask the agent, "Hey, remember that thing that we talked about?" s128 00:13:46.120 --> 00:13:49.640 then the agent within the thread will be like, "Yeah, I do know what you were talking about. s129 00:13:49.640 --> 00:13:51.760 I have kind of this short little tidbit." s130 00:13:51.760 --> 00:13:58.160 And it can, you know, follow up and and and do more kind of with that uh summary rolling summary in mind. s131 00:13:58.160 --> 00:14:02.280 So, that's kind of how we handled long context and memory, and it's worked pretty well for us. s132 00:14:02.280 --> 00:14:10.520 So, uh just wanted to share a little bit about that and how we've uh solved the long context problem. s133 00:14:11.320 --> 00:14:12.360 UI on the fly. s134 00:14:12.360 --> 00:14:20.280 So, um in this example, I said to the agent, "Hey, generate me a long essay, but give me some examples about what the essay could be about." s135 00:14:20.280 --> 00:14:31.920 So, what's really cool is the agent had this primitive registered uh uh this form, and it was able to build out this form for me at runtime and give me some options of what I could choose from. s136 00:14:31.920 --> 00:14:38.720 So, it feels very personal and it feels very kind of in the moment that it's able to to give me these options just at runtime. s137 00:14:38.720 --> 00:14:53.680 So, um this is kind of just a short little thing I wanted to include here about generative UI and how we uh are able to render UIs on the fly. s138 00:14:53.839 --> 00:14:56.240 You can't scale what you can't see. s139 00:14:56.240 --> 00:15:04.680 So, um this uh this kind of section is about tracing uh and observability. s140 00:15:04.680 --> 00:15:10.120 Really, what's cool about Effect is you kind of get tracing out of the box. s141 00:15:10.120 --> 00:15:23.840 Um you know, when you use these effect functions, they all get kind of tagged automatically with like these spans, and kind of the span gets picked up and feeds into these traces so that you can kind of get these s142 00:15:23.840 --> 00:15:26.400 kind of drill downs of these function calls. s143 00:15:26.400 --> 00:15:31.800 So, here's an example of a trace from the effect team. s144 00:15:31.800 --> 00:15:33.200 I have it linked. s145 00:15:33.200 --> 00:15:38.640 You could see that like, "Hey, when you hit this API, go to this endpoint, to this handler." s146 00:15:38.640 --> 00:15:44.040 And you know, etc. And it takes And what what's really cool is you could profile all your traces, right? s147 00:15:44.040 --> 00:15:49.600 So, this takes a total of this many seconds and you can see where the bottleneck is here. s148 00:15:49.600 --> 00:15:53.160 If there's a failure, you can cross-reference data across services. s149 00:15:53.160 --> 00:16:06.040 So, really, really important, especially working in agentic systems where we're we're integrating with other teams and other APIs and other um other platform capabilities. s150 00:16:06.040 --> 00:16:19.520 So, what's cool with effect is you get all this tracing out of the box and it really makes building this agentic experience, debugging it, and maintaining it just a breeze. s151 00:16:19.720 --> 00:16:20.840 Tools and skills. s152 00:16:20.840 --> 00:16:27.440 So, uh not only did we make a big bet on effect, but we also made a big bet on tools and skills. s153 00:16:27.440 --> 00:16:31.280 So, we believe that tools and skills are really all you need. s154 00:16:31.280 --> 00:16:37.839 And in this case, you could see on the left, we have this tool called get dad joke. s155 00:16:37.839 --> 00:16:50.160 And this is kind of the effect way and kind of the building blocks of how we do things here at OpenGov, but you know, this is pulled from the effect website, but you could see, "Hey, this is how you make a tool." s156 00:16:50.160 --> 00:16:58.480 And then you add it to a tool kit, which is a collection of tools, and then you can register this this tool kit with the with the language model. s157 00:16:58.480 --> 00:17:04.600 So, for example, if you had a prompt that said, "Hey, generate some dad jokes about pirates." s158 00:17:04.600 --> 00:17:05.040 Well, guess what? s159 00:17:05.040 --> 00:17:11.319 The agent has a tool uh that can help get a dad joke. s160 00:17:11.319 --> 00:17:25.520 So, um really, this is the the building blocks of how we did tools and eventually skills and really just is has paid off wonderfully for for our organization. s161 00:17:25.520 --> 00:17:37.680 So, um really recommend trying out this effect AI package from uh effect and um just trying out building out your own tools and skills. s162 00:17:38.160 --> 00:17:39.400 Developer velocity. s163 00:17:39.400 --> 00:17:48.600 So, not only do we build agents for our customers, but we also use agents um here internally uh in OpenGov. s164 00:17:48.600 --> 00:17:55.200 So, we uh use a lot of Claude and Cursor uh and it's just really been a game-changer for our team. s165 00:17:55.200 --> 00:18:05.480 Um it's funny because we're building tools and skills for customer-facing uh agents and and that has been great, but we're also building them internally as well to help accelerate our development workflows. s166 00:18:05.480 --> 00:18:14.200 So, things like Claude, Cursor, Claude agents, they really help accelerate how we read, write, review code and and ship. s167 00:18:14.200 --> 00:18:16.760 Um so, it's it's just been such an accelerate. s168 00:18:16.760 --> 00:18:22.080 So, definitely just wanted to mention that uh before we wrap up. s169 00:18:22.960 --> 00:18:23.800 That's it. s170 00:18:23.800 --> 00:18:24.880 Thanks so much for watching. s171 00:18:24.880 --> 00:18:25.840 You've made it to the end. s172 00:18:25.840 --> 00:18:29.280 Let's build agents that ship to production.