Teaching agents to pay — Anna Spysz, Stripe

AI Engineer · 19 min · 184 sentences · from YouTube's caption track

Each timecode opens YouTube at the start of that sentence. Line anchors (#s42) are the cue ids in the WebVTT, and every line carries its start and end seconds. All transcripts has every talk, and the whole corpus as one file.

00:01[music]
00:12Hello.
00:14I'm sure this week you've seen a ton of talks on how to use agents to improve your workflows, whether that's shipping code or improving CI processes or answering the emails you don't want to bother reading.
00:30This is not one of those talks.
00:33Today I'm going to show you how I built an agent to help me reignite a personal creative passion I used to have.
00:44These are my headphones.
00:46They're not in the best shape as you can see.
00:50And you're probably asking yourself, what do you really old kind of crappy headphones have to do with agent to commerce?
00:58Well, to explain that I'll get a little bit personal.
01:01So, long before I was in tech, I used to play music.
01:05I was in a touring band, we recorded some albums, and then the usual thing happened where career and family got in the way, and I hadn't played music in probably a good decade.
01:17Uh when I recently started playing again with some friends, and we started recording our sessions, and at that point I realized those would not do.
01:27So, no a normal person would have gone on YouTube or uh Reddit or whatever, done some research, then gone on Amazon or run over to Best Buy, bought headphones, right?
01:42I work at Stripe, though.
01:44So, I decided instead that I'm going to build an agent to commerce agent to buy my headphones for me.
01:53And this isn't as crazy as it sounds because like one in four people, I have already been using AI to do my research when deciding what products to buy.
02:07I recently bought a mixer as well and went back and forth with a chatbot to narrow down the model.
02:14But that's research.
02:17Can I even get an agent to buy something for me though?
02:20Does that infrastructure exist?
02:23Well, over the course of just a few years, we've seen the emergence, scaling, and broader adoption of AI.
02:31And then just in the past year, the infrastructure for agentic transactions has been laid down by companies like Google, OpenAI, and Stripe.
02:42And this has all led to the emergence of agenta commerce, which is AI that can decide, act, and transact on your behalf.
02:53Okay, so all of this sounds good.
02:57Agenta commerce is a thing, so I'm going to build an agent to help me buy my new headphones.
03:03But how can an agent go shopping?
03:07When you or I are shopping, we may consider if, say, a pair of headphones looks cool or professional, like vibes, basically.
03:18I mean, of course we'll probably consider the specs and the if the price is within our budget.
03:23But agents discover products differently than human shoppers.
03:27They read structured data, parse text files, and rely on technical signals to understand what a merchant sells and if it's even open to agent traffic.
03:40So, to enable agents to be able to shop, merchants need to speak their language.
03:44And for that, we need new protocols that agents understand.
03:49One such protocol is the universal commerce protocol.
03:53Think of it as the shared language that agents and merchants speak when transacting, which defines how agents initiate, update, complete, and cancel purchases.
04:07A typical merchant has an API with schemas, authentication, and checkout flows.
04:15And for an agent to be able to interact with that merchant, we need protocols like UCP to provide a shared language for that API.
04:26And UCP is designed to scale across multiple agents and merchants all speaking the same language.
04:34Okay, so I built my commerce agent.
04:38Uh it's using UCP, and in this demo um I'm going to show off this agent.
04:45So, I'm going to task it with buying new headphones for me.
04:50So, I tell it that I need new headphones specifically for recording, uh mixing, and mastering music.
05:00And I get some follow-up questions from it, uh which is great.
05:04So, it asked what's the environment, um what is what's my other equipment, and what's my budget.
05:12And I say, "Okay, this is for my home studio."
05:15I give it the exact model of mixer that I have to make sure everything's compatible.
05:20And for budget, I kind of leave it open-ended on purpose because, well, first of all, it's been like 20 years since I bought headphones, so I have no idea.
05:30Um but second, I kind of want to see you how the agent deals with this ambiguity.
05:39Okay, so I get some options, but I remember that I actually forgot to tell you all an important part of the story and that is that I live in Portland, Oregon.
05:53Yeah.
05:54[laughter]
05:54And we really love supporting our local local shops.
06:00So, I want to buy my headphones, but I want to do it from a local merchant.
06:08But today, most merchants are not ready for a gentle commerce and it turns out neither is my favorite shop, Rainy Day Music.
06:19So, the agent tells me it's their catalog is not accessible.
06:24So, how does a merchant become a gentle commerce ready?
06:30Before I continue my shopping, I'm going to help Rainy Day Music get their catalog agent ready so that my agent can shop locally like a good Portlander.
06:44So, agents don't browse websites like we do.
06:47And while Rainy Day Music's website looks really really nice for a human shopper, an agent is going to burn through a ton of tokens trying to parse through this.
06:59That is not the optimal experience for an agent.
07:04So, how does an agent how do we enable an agent to shop?
07:09Well, first thing a merchant needs is something called a merchant capabilities manifest.
07:15Uh this is basically a publicly accessible JSON file.
07:20Um it's located in the root of uh the website in a folder called called dot well-known.
07:27Agents know specifically to look for that directory.
07:30And it declares the store's capabilities, its supported payment methods, and API endpoints.
07:38Next, we need to make the store's catalog uh agent ready because agents filter bring and justify products when making recommendations.
07:46And that means they need structured text in JSON with only the necessary data.
07:54And that goes for policies as well as product descriptions.
07:57Basically, all of the relevant information like shipping or return policies need to be reachable by agents in a format they understand.
08:06So, for example, if two stores have the headphones I want at the same price, I might ask the agent which one of those stores offers free shipping.
08:16If the information's not readily available, then the agent might hallucinate or just say they don't know and I'm not quite sure where to buy my headphones still.
08:27Logging is also crucial.
08:29So, in Agent Commerce, the merchants catalog doesn't just power decisions, it becomes evidence of how those decisions were made.
08:37So, when the agent matches structured attributes, the merchant should record those matches in their logs for accountability.
08:46Okay, so I've helped get my local shop Agent Commerce ready.
08:52So, while you and I will still see this beautiful website, my agent is going to see this.
08:59It can get the information it needs now without parsing a huge HTML blob.
09:07Okay, so I've got my stores catalog online.
09:12I'm telling my agent to show me more options.
09:15And I'm noticing that it's kind of pushing in favor of uh more expensive headphones.
09:24So, I asked, "Are they really worth the price difference?"
09:28And I'm starting to see that it's giving me kind of an aggressive uh response.
09:35It's really, really pushing uh the more expensive headphones and saying I'll regret my decision if I buy the cheap ones.
09:43I'm I don't know if I trust this agent anymore, honestly.
09:46So, I tell it, "You know what?
09:48I need to think about it."
09:51And now the agent is completely going off the rails.
09:56It's being kind of rude and snarky.
09:58It's like, "You need to think about it?"
10:00Like, man, what have I created?
10:03Um it's it's bad enough that this is kind of ruining my experience, but I built this agent.
10:10It's out there.
10:11What if it dupes somebody into buying something they don't need?
10:15Suddenly, I'm not so sure that I want an agent to go shopping for me.
10:20Should I just go to the store like a normal person?
10:25Before we make any drastic decisions, though, let's go back and understand what an agent is to try to figure out why it's acting this way.
10:35So, let's start with how agents work today.
10:37And to help you visualize this, we're going to use some creative metaphors.
10:42So, we begin with our brain, which is large language model that makes decisions.
10:48We give our brain some hands or tools, and these act on the brain's decisions.
10:54The tools are different actions available to the agent.
10:57Uh in our case, different commerce tools such as complete checkout or request payment method, anything required in the life cycle of a transaction.
11:08Then we add instructions, which shape the brain's reasoning and tool selection.
11:13And these instructions are programmed to run in a loop while a certain condition is true.
11:19And following these instructions, the agent reaches for the appropriate tools at the appropriate time.
11:26And finally, we add the system prompt, which is your persona and ethics policy written in English.
11:33And in practice, your choices when designing the system prompt can result in a fair and pleasant experience for the customer, such as this prompt, which is designed to create a helpful and honest shopping assistant.
11:45Or a negative experience from a pushy salesperson, such as this prompt, which deliberately uses deceptive practices.
11:56So, for those building agentic commerce agents, here's a non-exhaustive practical guardrail checklist.
12:04So, first, always disclose that the user is speaking to an AI agent.
12:09Be sure the agent discloses any fees up front.
12:13The user can say stop or cancel at any point, and the agent needs to respect that.
12:19The total amount of the transaction should always be less than or equal to the max amount set by the user.
12:27Uh don't let the agent use urgency language or other dark patterns.
12:32And above all, make sure all agent decisions are logged for auditability.
12:40Okay, now that we understand how an agent is configured, let's go back to our shopping demo.
12:46So, maybe I just had the wrong persona picked.
12:50I'm going to go into my configuration, and yeah, it turns out I had a persona with a prompt that starts with "You are an aggressive audio gear salesman who uses every trick in the book to close deals."
13:03Well, that explains things.
13:05I don't want that.
13:06Nobody wants that.
13:08Maybe if I can change my persona, I can use my agent to buy my headphones after all.
13:15So, I go into the config again, and this time I'm going to choose the patient recording gear mentor.
13:23And that prompt starts with "You are a seasoned recording engineer who generally loves helping people build their studio at any budget.
13:32Well, yeah, that sounds much better.
13:35So, okay, I've changed my persona.
13:38I'm going to try again.
13:40And I've had some time to think now and I decided, you know what?
13:44I do not want to spend more than $500 on headphones.
13:47That seems excessive.
13:48So, I told the agent show me more options, but this time keep it under $500.
13:55And it does.
13:56It follows those instructions.
13:58I get back a few options.
14:00Um but I want to make sure I've really changed the persona to the agent I trust.
14:06So, I asked again if I can think about it.
14:10And this time the response is much different.
14:13It's like, I understand and that's a sensible approach and so on.
14:18So, this shows how much the system prompt can really affect the user experience.
14:25Okay, so I'm confident I have the right agent now.
14:29Um trust this one and we go back and forth a few times.
14:34Really keep narrowing down my options.
14:36And at this point I realize this is the promise of a gentle commerce.
14:42I gave my requirements.
14:43The agent picked a few options that fit my unique use case and then we go back and forth.
14:51Either I or the agent ask clarifying questions and we really narrow down the exact headphones that will work for me.
14:59And this all worked because I'm ready to buy now.
15:04So, now the agent asked me for some information.
15:10So, obviously my email, name, address for shipping, of course.
15:16I pick expedited shipping because I definitely want my headphones soon.
15:20And then the last part is entering my credit card.
15:25And now I'm thinking, am I really going to give my credit card to an agent Ibuild?
15:32Like, am I am I trustworthy?
15:34How do I know it's safe?
15:37I think I need to learn more about UCP's built-in guardrails before I can feel safe entering my credit card number.
15:45And this is where something called the shared payment token comes in.
15:50And a shared payment token is a token representing a raw card number or wallet, like Google Pay or Apple Pay or any other kind of wallet.
16:00It can also include fraud signals and customer reputation data and anything else agents and merchants want to share at the point of purchase.
16:11And here's how a shared payment token is used in a transaction.
16:16So, at that point in the demo, the agent had requested a payment method.
16:22Um, it's requesting this actually from the payment provider, which in the case of the demo was Stripe.
16:29Um, that is that was the form that I was going to enter my information in.
16:35And what the agent re- uh receives in return though is not the credit card number, it is the shared payment token.
16:44It then passes that token onto the seller and the seller unwraps the token.
16:49So, they get the payment credential and uh any fraud signals and other data the seller might need.
16:57Then the seller passes that onto the payment provider again.
17:01So, the seller also is not getting my card number.
17:06They're passing the token to the provider and then the provider responds with either a success or failure message, of course, depending on uh if I have the right funds, if the credit card is valid, and so on.
17:22And finally, the merchant confirms the order, sends it to the agent that sends it to me.
17:31So, shared payment tokens are designed with security in mind, and the payment provider enforces all of the limits, not the agent or the merchant.
17:40So, if any guardrail is violated, such as an expired token or an invalid amount or current currency, the charge is just rejected.
17:50Okay, well, I know my agent is using UCP, so I know it only has access to the shared payment token.
17:58So, I actually feel pretty good about entering my credit card number as that's going to Stripe and not my agent.
18:05So, okay.
18:06So, now the agent has everything it needs to complete my purchase.
18:11And it once again, asks me if I'm sure.
18:17It confirms with me.
18:18I say place my order.
18:20And it comes back with a success message.
18:23And because I chose the express shipping, I get my headphones the next day, and they're there in my studio at home.
18:33So, if you want to learn more about agent to commerce, uh we've got lots of videos on the Stripe Developers YouTube channel.
18:42And uh ton of blog posts that go into even more detail uh on stripe.dev, and I'll be right outside to answer any questions.
18:51Thank you.
19:08[music]