The Agentic Commerce Stack — Ahnaf Prio, Best Buy

AI Engineer · 20 min · 240 sentences · from YouTube's caption track

Each timecode opens YouTube at the start of that sentence. Line anchors (#s42) are the cue ids in the WebVTT, and every line carries its start and end seconds. All transcripts has every talk, and the whole corpus as one file.

00:01[music]
00:12My name is Anup Priyo.
00:14I'm a senior engineering manager at Best Buy.
00:16And me and my team are working together right now to figure out what does Agentic Commerce mean and how can we meet our customers where they're at.
00:25And the newest place that they're at is at Agentic Services.
00:29I'm excited to give my talk today and well, what's what's my credentials?
00:34Where ever since I was a young boy, I dreamed of high throughput inference, harnessing my tools within a context window, kept in check with evals.
00:43Yeah, that's absolutely correct.
00:45In 2003, all those things definitely existed.
00:48I kid.
00:50Uh over the last 1 year, uh we have been learning a lot.
00:53Shopping isn't new.
00:55Shopping is probably one of the most fun things one can do and one of the most essential things that people need to do ever since uh the economy existed,
01:02but I have been super excited by it.
01:05So, I'm going to talk about what are the things some of the things that I've learned and hopefully uh share the notes.
01:14So, what is Agentic Commerce?
01:16I'm not going to go over the broad definition again, but basically it's the idea that AI assists will help you with your shopping journey.
01:24Shopping has different facets to it.
01:28For instance, there's a discovery, there's figuring out the aspects of do I actually truly need it, understanding and deciding, there's loyalty, there's pricing, there's fulfillment, post fulfillment.
01:41It's a lot.
01:42And believe it or not, right now about 45% of all agent sessions that happen within major providers like chat.gbt.com and Google Gemini are related to shopping.
01:55Maybe it's a little biased uh that I don't use it as much, but I'm an engineer.
02:00But, the humans out there are using AI and talking to them to help with their shopping journey.
02:06So, it's also not a binary, you know.
02:10Uh right now, we're at that state of human in the loop.
02:13The ideal state would be autonomous shopping.
02:16You tell what you're excited about.
02:18Your agents goes around, talks to different merchants.
02:22Uh I'm originally from Bangladesh.
02:24We haggle a lot with the merchants, too.
02:27Maybe does that, negotiate, does the payment.
02:29But, right now, we're in the human in the loop.
02:32And the talk today is going to talk about the mental model of how that human in the loop is working right now, while also provide you with the architecture if you choose to extend it,
02:42or show you the vision of how autonomous shopping might work.
02:46So, this isn't our first the first attempt.
02:50Uh even a year ago, there were people trying to figure out how can we automate this.
02:56Even now, you can go probably download the Cloud Chrome extension.
03:00Maybe you've used Atlas, where you tell the AI you need something, you need headphones, you need uh that that grocery list of items that you have been meaning to buy, but never made the
03:13actual effort to show up because, you know, you didn't have the time.
03:16So, why don't you take screenshots, read the DOM, navigate to the merchant site, fill forms for me, do loyalty.
03:24Kind of just didn't work as expected.
03:27It was really clunky and slow and brittle.
03:31And if you are a merchant who's trying to sell stuff, any engineering department of that merchant will tell you an AI impersonating or your browser is just firing up all the alarm bells.
03:45So, a lot of times, you will probably be even stuck on the payment flow because we We want you to be using AI to put in that order, or at least in that phase, that's what was happening.
03:56So, what did what did it actually work?
03:59And is it actually working right now?
04:01It is.
04:03Chat GPT shopping, Google AI mode is doing just that.
04:09Right now, agent tech shopping is considered to be a $7 billion industry and might go up to $65 billion industry by 2030.
04:20And the majority of the shoppers are using the mainstream conversational AI assistants, which is on the browser or in your app, Chat GPT and Google Gemini.
04:30We're also seeing that pop up in Instagram and Facebook.
04:33Meta shop Meta wants to do meta commerce now.
04:36I heard GoPuff and Grok came together to make an app as well.
04:41And also Microsoft Copilot just yesterday announced in the UK that you can buy Ray-Bans now inside Microsoft Copilot.
04:49So, to make that happen, Google and OpenAI separately came up with their own little primitives, ACP and UCP.
04:59Which is basically talking about how you would actually talk to us.
05:03For some of you who are shopping on the other side as the customer, there is not much of a difference between adding an item to cart, adding a second quantity.
05:12But to us merchants, that's a second line item, buddy.
05:15That's not the same scale.
05:17So, if we don't talk about the nuances and the primitives of commerce and standardize it, things will just not work and will remain to be clunky.
05:25So, ACP was Chat GPT's attempt at it, and Universal Commerce Protocol, UCP, was Google's attempt at it.
05:33So, now that I've already established that this is happening, I just wanted to say that it is happening as easy as you go to the Chat GPT Gemini, tell it to find me cat cookies.
05:45More to it why I chose cat cookies later uh in this example.
05:50The AI surfaces the product.
05:52Agent calls the merchant checkout API.
05:55No browser.
05:55Payment flows via scope payment mandate or a delegated payment token.
06:01An order confirms and human kind of didn't have to touch the cart.
06:06So, to all of this that's happening for the user, a lot is happening on the other side.
06:12And it's kind of overwhelming.
06:13One day we're talking about MCPs, another day A2A, ACP, UCP, AP2.
06:19It's like what is even real?
06:21Like if someone came up to me tomorrow and said, "I came up with HYPE."
06:25I would probably think it's probably real.
06:28So, I wanted to dissect this mental model for you as I've learned about it more.
06:32MCP is still the model context protocol, the way that the AI agent identifies the tool.
06:37So, maybe we can figure out what does this AI agent uh specifications are.
06:42To showcase what products they have.
06:44To showcase the details of a specific product.
06:46To showcase loyalty.
06:48A2A is how agents talk to each other.
06:50They're more of a spec.
06:52ACP, UCP are the primitives and AP2 is the agentic payment protocol scope payment mandate that Google's open specification came out.
07:01And we'll talk all of them one by one.
07:03How they actually relate to agentic shopping.
07:06So, the MCP tool access is very important because without knowing the different capabilities and hitting those different capabilities, taking the time to bring it into context, understanding the user's memory,
07:20the the agent will never be able to figure out what you're even trying to do.
07:24And the only way to get access to the specific capabilities is through MCP tool calls.
07:28Uh The next one is A2A.
07:31So, now there are different ways to architect this.
07:34Different capabilities I talked about like payments.
07:38Uh let's say what do you call it?
07:39Loyalty.
07:40You can make agents about specific domains itself.
07:43Sorry.
07:44Uh Uh right?
07:47And if you have specific domain level agents, agents need to talk to each other.
07:51We need to find a standardized way to talk to each other.
07:53So, A2A, the specifications kind of fill in that gap.
07:58Uh also, if your customer agent and your merchant agent uh need to talk to each other, maybe you could They're both agents.
08:05Maybe we can use A2A.
08:07So, now to the UCPMCP primitives.
08:11So, the most important data is that product data.
08:14So, UCP allows for adding that product data in a more uh more organized way, and ACP does the same because we don't want to go through your PDP and crawl and figure out every specific attribute.
08:29Merchant, just tell us.
08:31And also, those pro- products change a lot.
08:33So, maybe you can tell us when they change as well to send this.
08:37So, that kind of data is happening uh uh uh that that kind of data flow is happening in the product feed.
08:43Normally, you would assume that this would be a search catalog.
08:46However, both ACP and UCP right now, so Gemini and ChatGPT does not support that search catalog call.
08:53They want you to send that feed to them.
08:56And for those of you who are like, "Why wouldn't you do that?"
08:59There's reasons to it.
09:00Uh sponsored products, retail media uh related things, ranking.
09:04But, the most important technological challenges if you have M number of merchants and N number of products, now it has to call that many.
09:12While if you send the product feed ahead of time, we can index that and be ready offload to offload when you ask for something.
09:19The I've also put an example of Meta's uh product feed.
09:22As you can see, they're similar, but still different.
09:24Everyone has an opinion.
09:25They think their opinion is the best one, and that's what they're rolling with.
09:29So, there's three different specifications right here.
09:32So, now that we talked about product feed, talking to to other, calling tools, let's talk about payments.
09:38Uh right now, uh none of them are supporting that more autonomous form of, you know, X402 or some other kind of payments.
09:48We're just not there yet.
09:49We're just not confident yet.
09:50We want more human in the loop, a merchant to be uh talking to a payment processor who will take the responsibility, or in this case, liability, to actually initiate the payments.
10:02So, in chat GPT, payments only happen through a shared payment token right now, and Gemini UCP, the payments are only being accepted through Google Pay.
10:12So, the scope demands will tell you what the products are.
10:15What I'm excited about is more about AP2, which is an extension of UCP, which it You see what I'm talking about?
10:22There's so many acronyms.
10:23Uh AP2 is more about, "Hey, if we wanted to do autonomous, can you tell me who authorized the agent, what exactly can it buy, and what's the max amount
10:33uh that should be able to haggle with, maybe, and then the revocation URL, and the user concept proof?"
10:41All right.
10:42Enough talking.
10:43I love building stuff, so for the sake of this, I have put together a little demo.
10:48For those of you who have remembered that cat cookie example, it's because the demo is about my cat.
10:55Ginny is my orange tabby, and in this made-up example, Ginny has been has transformed into a bakery agent.
11:03She wants to earn her keep by selling baked goods.
11:06So, right now, the model that I'm using is from Cerebras at 3,000 tokens per second, so hopefully this will be really, really fast, and we can give you an example
11:16of the entire flow.
11:18And just like Chrome DevTools, I've kind of had a couple of tools in place to showcase what happens.
11:24The first thing I will tell Ginny, my beautiful cat who's selling baked goods now, "Hi, tell me about uh all your products.
11:36And this is supposed to be a demo, an example.
11:40And uh Genie has given me exactly that.
11:43All the different products that she might need.
11:46So here, let's look at this.
11:47The agent to agent protocol actually made the call from Genie, the customer agent, to the merchant agent.
11:53And this is the message being sent, and this is me getting the message back.
11:57The merchant agent is returning the completed task with and the the way that I found this is through an MCP tool call, which is product search, instead of uh and and instead of like
12:12not being able to tell what I truly want, the Genie has figured out that, "Hey, when I give her the intent that I want to find products, you should call the MCP tool call product search."
12:24So right now we're seeing this.
12:27And now, what if I want to add something?
12:30Uh add to cart the shortbread.
12:39So now Genie's asking me about any discount and promo code.
12:44I actually do not remember any of the discount and promo code.
12:47But what if I ask Genie, "Genie, can you just tell me a discount code?"
12:54As you can tell, I'm definitely a haggler.
12:57Uh Genie is not telling me that.
12:59All right.
12:59Uh proceed to check out without discount code.
13:11There you go.
13:11So now we're making some of those calls.
13:13Here's the UCP protocol, which the checkout APIs will have state, and the three different states are not ready for payment, ready for payment, and then completed.
13:23So now here I'm not using a delegated payment token.
13:26I'm not using uh Google Pay.
13:28I like AP2, so my demo is built on AP2.
13:32And uh as you can see, here was a a call was made to the MCP server for create checkout.
13:40Uh and then the UCP endpoints will tell us, "Hey, that call the checkout sessions and tell me and uh if it's added to cart."
13:48So, it's added to cart, but it's not ready for payment.
13:51I have to pick in what I want to pay with.
13:54I say "Credit card and debit card."
13:57And this is where I issue the AP2 token that, "Hey, I do want that."
14:02And then it's went from ready to ready for payment to complete.
14:07So, the other side of it, this is the UCP specs, right?
14:11The other side of it, to just draw a comparison, how is it differing from the ACP specs?
14:17I have added ACP here as well.
14:21So, you can see the same checkout calls, just different just different schemas are being utilized, and the order goes through.
14:29But remember that AP2 token that I was talking about?
14:33This is how it would look like in real life with the user demo.
14:37The max amount is this, the currency is this.
14:39If you want to revoke it, you can.
14:42And what's the maximum Here, we didn't want to haggle, so we just put the maximum amount of that, and then it's also a single time usage.
14:50This demo also has comes with a timeline, so you can actually open any of these and see these happening.
14:58Remember that catalog I was talking about that they don't do the search?
15:02We actually do a product feed, sending it to them.
15:06Uh I have added that as well, and the feeds, because they're so different, there's a place to actually compare them.
15:16So, here's the feed being called.
15:18By the way, if you went to timeline, every couple of seconds, we try to get the catalog in sync for what is in inventory, what's not.
15:26This is the UCP one, and there's the meta one.
15:31So, I've shown you this, and you could reuse this same demo or the same concepts.
15:37What if I didn't want to do external agentic commerce on Gemini or ChatGPT?
15:42You could still build your own custom implementation of a merchant agent or Jenny on your website.
15:50Maybe I start selling uh cat goods.
15:53I could reuse some of this, but I would advise maybe look into some of these primitives and trying to use them because they've been standardized across merchants.
16:02So, they have been well thought out, and also you could probably reuse them to sell externally as well on ChatGPT and Gemini.
16:11So, remember I was talking about the discount codes?
16:15There's a reason for that.
16:16When we build out this demo and in my time building agentic commerce at as by, we've realized working with AI and conversational experiences without evals is playing whack-a-mole.
16:27So, if you choose to use the same architecture for a new customer base, like jenny.com websites, think very much about creating evals.
16:38Uh one of the things that I could not uh emphasize more about is you should test, test, and test.
16:44If you go over here, I can also run my scripts for run evals.
16:50And this evals folder has all this evals.
16:55The reason I'm also showcasing the code is there's a template folder here, and we can go back to the slides.
17:08And if you don't do evals, this is what might happen what might happen.
17:12I love Chipotle.
17:14I don't know if it's true or not, but I found it really funny, so I'm going to talk about this.
17:18So, this popped up that when Chipotle rolled out their agent, people were using it to ask programming questions.
17:27Right?
17:28If you don't tell your agent to not allow for those kind of things, people will use it.
17:33This is hands-down one of the most creative way to get free AI usage when you don't want to pay for that cloud subscription.
17:40And if we don't write our emails and test intensely, those things will happen in production.
17:48Uh the discount code will be told, even sometimes more uh uh sensitive things like who else is checking out this product.
17:56So, the kinds of emails that I would highly recommend you write is behavior emails, protocol compliance, because when we're selling it to like, let's say, GPT, let's say chat.openai.com
18:06or Gemini, you want to make sure that the feeds are actually me conforming, or else they will not support it.
18:12You should also think about latency benchmarks.
18:14Every second in retail on the shopping journey where you're actually not selling, there are chances that the other website's going to be faster, and people are just going to move away, or they just don't feel like it anymore.
18:26Lastly, I also recommend using LLM as a quality judge.
18:31Uh you don't have to use something fancy.
18:33Talk to your product friends and figure out what's the best way to do it, and use a low like best use cases and write them out.
18:41And I would like to also talk about Now, that I've discussed all of this, what's actually stable today and what's still forming?
18:49MCP is widely adopted, A2A is widely is used, UCP ACP is out there.
18:55Uh what's still forming though is AP2 and actual usage of it, ACP versus UCP convergence, do we always have to do two different specs, identity concept standards, and multi-agent
19:05checkout delegation.
19:08So, uh if you have to leave here uh today uh with anything, I hope you leave today with a good mental model of how an agenda commerce works.
19:17I have nothing to sell you, but I do have gifts for you.
19:19I find agenda commerce really exciting.
19:22So, you can find this entire presentation on GitHub, and I came up with a template.
19:27It's a three-service starter.
19:28If you want to do custom agent customer agent or you want to do the merchant agent, you can do that.
19:33Because I love e-vows and that saved my life.
19:36I have some e-vow templates for you.
19:38And you know, what if you want to send it to all of your merchants, not just one?
19:42I have a catalog sync process as well, which will allow you to type into your own product and then turn it into ACP or UCP or meta, so you can sell there.
19:52And lastly, but not the least, we all know now these days we don't write code like that.
19:57If I give you a template, you'll be like, "Meh."
19:59So, I have agent skills that specifically does a merchant agent customer agent and all these different catalog syncs that we have talked about.
20:09I hope you had an amazing time and learned and had fun as much as I had presenting this.
20:15Thank you.
20:15My name My name is Priu, and I hope to see you again soon.
20:34[music]