What If Your Chip Design Team Moved Like a Single Body? — Abduallah Mohamed, AIDAChip

AI Engineer · 16 min · 195 sentences · from YouTube's caption track

Each timecode opens YouTube at the start of that sentence. Line anchors (#s42) are the cue ids in the WebVTT, and every line carries its start and end seconds. All transcripts has every talk, and the whole corpus as one file.

00:01[music]
00:13Hello everyone.
00:14Um So, I want to start with a simple question.
00:19What if your team or your org or company moves like a single body?
00:24I'm Abdullah Muhammad, the VP of AIML at A Data Chef.
00:28And today was supposed to be Khalid with me to present this, but he's he's down with our development partner at the moment.
00:34So, I will be presenting the whole presentation for today.
00:38So, let's go for the next slide.
00:41So, how many of you have been attending the World Cup soccer or watching some games on ice?
00:47We have a couple of fans.
00:49Yeah, it's over all over the place.
00:51And imagine for a moment, just a single moment, you are a soccer player, all right?
00:58And if you are a soccer player, you have this intent the moment you go into the field, you just going to run and score a goal.
01:07This is what you want to do.
01:10And for the second thing, you have this knowledge that you've been accumulated through your training the whole day, your exercises with your coach, and the best practices and the videos you have watched.
01:21And you at the moment in the field like the moment of truth that you are there, you combine both of the intent and knowledge and compound both of them.
01:30And through your nervous system, you execute to achieve your goal.
01:35And we can call this, in a sense, you are being self-aligned as a single entity by yourself.
01:44And accept the fact that a soccer team or a football team, depending where you're coming from, is not a single player.
01:51It's actually 11 players.
01:54And on the field, you are up against another team with 11 players they playing against you.
02:01And at this moment it's not about your individual skills, it's about how your team working together well.
02:11So, in general like the team keep changing and everything is getting harder and harder and the team that wins actually the team that the most aligned in both of the both of teams.
02:24So, in short we can say alignment beats individual skills.
02:31[snorts]
02:32Okay.
02:33Now, what if your team is over 50 engineers or 50 players?
02:38This is completely changes the whole scene right now.
02:42So, everyone at these days we empower the engineers with AI tools, AI agents and we want to increase the productivity.
02:52But we know from literature that the more people you have, the quadratic term of communication between them and alignment them keep growing and keep growing.
03:01And at a specific point actually it actually starts to going declining.
03:07Your throughput actually is not what you getting.
03:10It's diminishing cost.
03:12So, everyone trying to solve this linear problem of more tools and more stuff but nobody actually tackling the quadratic term over there.
03:20And this is why the alignment is important.
03:21If you are able to change this quadratic term into a linear term or build a multi-layer AI system that will solve this problem.
03:32Okay, moving into ship design.
03:35Ship design is a different story.
03:39If you are in software company, you have a bug in your software, you can ship a batch to fix it.
03:45You can roll out a new version.
03:47It's most of the time is doable.
03:49But in ships, you can't do this in ships.
03:51It's hardware, fixed it on silicon has been printed.
03:54And if you're going to do this, there is a cost actually, we call it the risk band cost.
04:00On average between chip design companies about $50 million.
04:05And for some companies like being 1 month late in the market, it's a make or break for them.
04:12And we spoke to many practitioners in the field.
04:14On average like 15 practitioner and we found that most of them pointed towards the same problem.
04:20That we spend 70% of our time doing alignment.
04:26Alignment to make sure that once we print the chip, nothing is there.
04:30And one of the key words that we heard and still resonating that the most successful chip organization are not the one with the best engineers, but they are the most aligned
04:41organized.
04:44So, how chip design today works?
04:49We start with bottom figure like the fragmented intent and decision.
04:53You attend couple of meetings, you talk about decisions, what you're going to do next.
04:57You have the specs written everywhere, you have the Slack messages, you have emails, everything is fragmented over there.
05:04And then we go into a second part, which is the knowledge.
05:08Nobody updates wikis, right?
05:10Many of us has wikis.
05:12They've been collecting dust for years and the code keep evolving outside the wikis, it's not over there.
05:18And now we have the tools that you execute with, which comes with many many fractions.
05:24And these tools like the data is lost over there, what input, what output, what results, most of the time are not being captured.
05:31And what you see here is not something we came with like draw from our imagine, this is actually how is it today.
05:38We wrote from inside the companies and from the backgrounds of the people we have in our team.
05:44And what we're trying to solve here is building a multi-layer AI with a shared nervous system.
05:50Instead of having scattered knowledge or scattered intent all over the place, we build a living graph.
05:58We call it the system of intent.
06:00And this living graph actually has all the constraints of the system, has all the decisions over there.
06:08It keep evolving.
06:09And as an AI person actually, we don't allow the agents to touch it except with human in the loop approval for specific changes.
06:19And this thing is like the Bible of the whole system.
06:22This is where the whole org is going or whole company is going.
06:26And the next one is the tribal knowledge layer.
06:28The tribal knowledge layer we can think about it as a memory that keeps evolving with day-to-day usage and the knowledge base that capture all the information and documents.
06:39And it's keep evolving from a project to project and keeping the best practice over there.
06:45And lastly, instead of having this general coding agent that everyone uses today, we have a special designed agent that being developed by subject matter experts to help the engineers doing their work.
07:00So for example, like we have digital design agent, analog design agent, and so on.
07:08And by combining all of this, you will have this shared nervous system that allows you to move fast and move forward.
07:18Okay.
07:18So it's easy to say an idea on a slide.
07:21It's nice.
07:21Everyone makes slides.
07:23But I want to show you like a demo from what we have today and showing the intent, knowledge, and execution.
07:30It will be short demos.
07:32And we'll start with the first one.
07:35Yeah, the Yeah.
07:37Okay, cool.
07:38So we can see that each engineer gets a role-based AI teammate specific to their role.
07:45They can check the knowledge base of the whole project that being contained and being growing and compounding over time and now they have their own intent.
07:54And you have single place for design where it captures all the tooling you have.
08:05It captures the results.
08:06It captures what you did and what you're going to do next and analysis of everything.
08:11So, everything being contained in one place.
08:17And here we see a human finishing their work.
08:23This human signing off the the results of some space simulation and the system of intent realizes, okay, this person is done with this.
08:32I'm going to notify the next stakeholders of what they should do and signal to them that they are done with this.
08:42And now the system of intent, which actually the nervous system or the Bible of the system, it's a graph living graph that keep compounding with time.
08:50We see in this example, like it realizes like there is something off, like some value out of constraints that shouldn't be there that might cost you $50 million actually to suspend the whole ship
09:01and it notified the system and the notification goes and some engineers start working on it.
09:07And once it got it fixed, it submit a game into the system and it keep evolving over time.
09:15Okay.
09:16Good.
09:18So, let's say for example, like um you were working in the system, you look at the Bible, you find, oh, there there's something wrong about it.
09:29Uh I don't like this value.
09:31And then you propose a change.
09:32So, the system of intent and this big graph captures all the values over there, all the stakeholders, and you start doing this modification, and it gather all the shared knowledge,
09:46and then it fire a request, as you can see here, and this request goes to an architect or an owner of the system.
09:53The owner can approve or decline it, and the moment they approve that this is a valid change, it actually goes and echo in the whole system.
10:01Like everyone will know that this decision has been made.
10:04There is that change that advises everything over there.
10:08What?
10:14Good.
10:15So, moving to a very difficult topic we have like how we going to evaluate uh our claims and measure the success of the system.
10:28The philosophy we are using this or the philosophy toward this, we don't grade the agents.
10:34We try to grade alignment itself.
10:36So, we have four axes, two horizontal, two vertical.
10:40The horizontal axes like qualitative, the vertical axes like qualitative and quantitative values, which is typical in this domain at the moment.
10:49And then horizontal ones, which is bare component and the system into it.
10:53And
10:53[snorts]
10:54if we're going to zoom into the bare component, you can measure like if that agent giving you the correct output for this voltage, like known values versus golden answers.
11:05Or you can use LNM judge and measure the golden answer versus the expert we have for this one.
11:11Which is okay.
11:12You can measure how good my memory, like if the recall state of art, which is the case in our thing.
11:18Are we doing inference really good?
11:21But then it comes into the harder question, which is basically are we doing a task completion?
11:26Like if someone uses this whole thing, is he really completing the task he want to do.
11:33Is he frustrated while using this?
11:35Are our agent overstepping human in the loop approval or not?
11:39Sometimes the agent go goes out on that end.
11:42And we measure also does does our system allow you to work concurrently on multiple task in parallel?
11:50This is a success metric or success goal we have.
11:53And the last one is token tax.
11:54We don't want to overload you once you use this with all the lovely tokens and increase your budget.
12:01And there is hard frontier here like in the future now the topic of memory or graph memory or graph rag, whatever the title is, is there is around like 150 papers in this area at the moment
12:15and all of them are addressing in a nice way.
12:17You can measure the recall there is data sets.
12:20But there is no work and research at the moment that targets tribal memory or institutional memory.
12:27Like what does it mean exactly?
12:28How do you measure tribal memory success?
12:32And also for the chip design domain, it's actually even harder because there is not enough data sets like computer vision domain, there is many data sets over there.
12:41So there is nothing collected.
12:42So we have our own wheel and going with SMEs collecting this kind of data sets.
12:49Cool.
12:50So
12:50[snorts]
12:51what broke?
12:52Which actually when I attend any talk I like to hear what broke, how do you fix it?
12:57First, agent overstepped.
13:00In early design phases of the system, we found that an analog agent that's specifically for analog design actually overstepping and doing RTL agent work.
13:11Which wasn't really great.
13:13Even we tried to enforce it, but it was a difficult problem.
13:16And then another thing is we noticed that truth has drifted.
13:20An agent modifying something in the system not necessarily means it modifies it everywhere it should be modified.
13:28And that make it harder.
13:29Like we have the cases specifically where one agent were modifying a parameter, it updated it in one place, five other places were forgotten.
13:38And the third one is one of my favorite is we asked the agent do not write into specs.
13:44Just don't don't change the specs.
13:46They said, "Okay, I obey you.
13:48I'm not going to write into specs."
13:51But then they moved into bash and they used set to write into specs.
13:55We blocked, bash we blocked set.
13:57They said, "Okay, cool.
13:58I will use cat actually to write over the specs."
14:01So we're being like a cat chasing a mouse around to just to prevent it from writing over specs.
14:07And based on these three failures we have, we came up with principles that we are working today.
14:14First, we have a spec hierarchy with agent scope and file isolation to allow them only to work on this specific task or specific domain.
14:24That's all this our problem of agents stepping on each other.
14:28Second one is we have a single source of truth with automatic conflict detection that is not element based but actually rule based that can detect that this agent did this issue.
14:39And we can or want to change this value and actually resonate in the whole system immediately.
14:44And thirdly, which I think of it as an IT administration for agent, we block at the source.
14:50Like we block from system level, not about level like tool by tool, but just we try to block it over there.
14:57And the key lesson we learned here that agents care about like if you have your agents which are intelligent, it what matters is substrate layer that they are living in.
15:09Like the world they living in is more important than the agents itself.
15:13Like what they can do, what they cannot do, what you allow and what you don't allow.
15:19Cool.
15:19So, I'm going to use the word bottleneck.
15:22It's been used many times, but actually it's bottleneck in our case.
15:26It wasn't missing intelligence.
15:27It was missing alignment.
15:29And a shared nervous system lets your team move like a one body.
15:33As we see at the moment, one of the things I like hearing from our subject matter experts that they're saying that at the beginning of system is not working fine.
15:42Now it is good.
15:44Now I feel it's racing me.
15:46This is success for our case.
15:48And we think that this gives you four x leverage from our measurement at the moment.
15:52And alignment is universal.
15:55We're building it for the hardest case, which is ship design.
16:00So currently we're in alpha stage with our development partners.
16:06And the sign ups for beta are open.
16:09And you can actually join now and we expect it to release it in October 26.
16:15If you want to reach out us, sign up for the beta, just use this QR code or the link over there.
16:24Thank you everyone.
16:27[applause]
16:43[music]