Morgan Stanley's ALPHALAB: Multi-Agent Research Across Optimization Domains — Brendan Rappazzo

AI Engineer · 20 min · 162 sentences · from YouTube's caption track

Each timecode opens YouTube at the start of that sentence. Line anchors (#s42) are the cue ids in the WebVTT, and every line carries its start and end seconds. All transcripts has every talk, and the whole corpus as one file.

00:01[music]
00:12Well, thank you everyone for coming.
00:14Um today I'll be presenting what we've been building at Morgan Stanley, uh sort of auto research agent to try and automate quant research.
00:24Um and I wanted to just take the the first couple minutes to sort of explain, you know, give some context on our team and also um you know, why we're even pursuing trying to build this this auto research agent.
00:37And so, our group are relatively small.
00:39We're about 30 uh PhD AI researchers, and we kind of operate both like kind of half academic, so we're encouraged to um you know, publish papers, open-source code, share our research.
00:52And then the other half is sort of more applied internal work.
00:56And you know, I think a lot of our problems or or yeah, at least some of them are are can be fairly well-posed.
01:02They kind of have a similar shape to Kaggle where we have you know, an input time series data set, and our task is to just, you know, predict future values and maybe with some other constraints
01:13of wanting the model to be well-calibrated.
01:16Um and so, I think, you know, being on the sell side, we kind of have maybe less sort of adversarial selection, and so it it kind of lends itself naturally to this auto research framework.
01:27Um and you know, also I think there's we have a lot of sort of algorithms in production where we have a feeling if we could just put in more cycles, there might be some improvements to kind of squeeze out, whether it's
01:39just better hyperparameter tuning or exploring a lot of different kind of ensembling these different methods together.
01:47Also, you know, we have We work with a lot of different desks, and there's a feeling that like something we have built for say credit bonds could really transfer well to like muni bonds, and that kind of translation process should be something that can be
02:01automatable with with agents.
02:03And even like going to new trading desks and trying to build them new algorithms, there's sort of a lot of low-hanging fruit where uh they're not really using a lot of machine learning or or AI automation.
02:14And so, you know, even like a reasonably trained model should be able to have a big impact.
02:19And so, sort of in the last year and a half where these these agents became, you know, able to do long-horizon tasks, we've been interested in trying to do this automation.
02:30Um but it wasn't really until December of 2025, and I think this is a pretty common sentiment now that um it really felt possible for the first time with like with Opus 4.5
02:41and with, you know, these harnesses like Claude Code and Codex, where it really felt like the models were at a point where they could do these long-horizon tasks and also kind of the idea of
02:51putting them in these harnesses that that allow them to do so.
02:55And so, you know, really starting this year, we kind of made it a big effort to build this this auto research agent.
03:01And of course, sort of the the top-line goal is just maximizing P&L and maximizing the the number of new algorithms we can put in production.
03:10Um but, you know, we also had these design concerns.
03:13Like, of course, we wanted to play nicely and integrate with all of our data and sort of the pre-built scaffolding we have around backtesting and evals.
03:23We also wanted it to be able to run the full spectrum.
03:25So, like, you know, on one side, maybe it's a new data set, and we just want to say like, here's the path to the data, a natural language description of what we hope to predict,
03:37and the thing should go off and do research and build its own eval and start doing experimentation.
03:43And then sort of on the other side of the spectrum, maybe it's like, no, we have our data scripts, we have our eval, we actually have have good models we've already produced, and we just wanted to kind of churn and do more cycles and see if we can find an improvement.
03:57Um also, we wanted to build this to be really model agnostic, so we could use any of the frontier providers or increasingly, uh you know, any open-source model.
04:07And also, we wanted to build it in a way that you know, as these models get better and better, it's not sort of consuming what we built.
04:15Like, we kind of rise with the tide of the models.
04:18And lastly, you know, really carefully think about um how do we encode sort of like our enterprise knowledge as Morgan Stanley and then our human expertise as as quant researchers.
04:30And so, for for the rest of the talk, I want to start with sort of what I'm calling Alpha Lab 1.0, which is our first version we released.
04:39I think it was early April, and we put out a full like 40-page tech report going through all the details and results.
04:46We also open-sourced all the code on GitHub.
04:49So, I I kind of want to cover that more at a high level because all the details are so public, but I'm happy to talk afterwards in depth about any part.
04:58Um but then I really want to cover, you know, sort of what's happened since then.
05:03So, what were the initial results?
05:05What were the kind of failure cases since then because we we have encountered a lot of failures, and then talk about how we're really addressing those by building uh kind of our own rich set of evals and environments,
05:18and how
05:18[snorts]
05:18that's sort of allowing us to improve the harness and climb towards this sort of self-recursive improvement and sort of our grand vision now for Alpha Lab 2.0. And so, to start, you know, Alpha Lab
05:32is an agentic harness, and kind of going towards that that first side of the spectrum, the the goal is you have some data set, you know, let's say it's like a
05:43uh exchange rate data set or something, and you can just provide the path to the file or maybe it lives on an API and you can just say here's the API access
05:53and the API spec.
05:54And then just in natural language say what you want to predict.
05:57So maybe it's as simple as, you know, the simple exchange rate.
06:01I would just I'm curious in predicting one day out what the the rate will be.
06:06And the harness then works in these these three phases.
06:08So the first phase is research and I'll cover these all more in depth.
06:12The second phase is then actually building its own evaluation or backtesting and the third phase is sort of the mass experimentation, which is really the heart of the the harness.
06:24And then as output you get a suite of trained machine learning models that are trying to, you know, do the the prediction you care about.
06:34And one design choice we made, so the harness is actually all our own code.
06:40So we decided not to use any off-the-shelf agent framework.
06:44Um we wrote it all and really Claude wrote it all and I think like in the era of of Claude code, I I like this approach of kind of building your own from scratch because you get sort of
06:56max freedom and max like, you know, you're you're free to kind of tweak anything you care about.
07:03Um and so, you know, all the tool calls are done with these like functional tool calling and that also allows us to nicely really be provider agnostic.
07:12So OpenAI Anthropic or open source providers, it's very easy to to adapt the the harness to any any of those.
07:21As far as the actual tools, there's there's several but the sort of three main ones are one full shell access, so it can write any bash command.
07:31So this is how it's writing code, setting up its Python environment, editing code, running code.
07:37Um another important one is web search and this allows it to go read, you know, archive or technical blogs or anything like that and get sort of up to speed at
07:47um you know, at least in the public domain like sort of what's the state of the art methods.
07:52And then the third one is um we use Slurm to manage our GPU cluster, but the the higher level idea is just a a nice abstraction where the model can say like I'm training a
08:04a fairly big model.
08:06I need 4 H100s and this many CPUs and just kind of write the config config and submit the job and not have to worry about like doing hardware orchestration.
08:17Uh yeah, and so going to the the actual phases.
08:20So again, the first phase is this this research phase.
08:23And the idea here is kind of like almost like a super cloud MD file or a super init where we just want the system to go off and kind of build enough context
08:34such that it can start meaningfully uh you know, forming hypotheses and testing them.
08:39Um and we built, you know, Alpha Lab is kind of this server-side running thing, but we built this lightweight UI on top.
08:46And so sort of how we've done this is build this this scaffolding of the to-do list.
08:52So it's first, you know, prompted to to build a to-do list such that if it completed every item, it'd be able to start experimenting.
08:59And this allows us to kind of keep re-prompting should it try to exit early.
09:04And so you can see, you know, it's talking about setting up its Python environment, um doing different data loading, doing different kinds of statistical testing.
09:14And each item of the list it's instructed to build a like take notes in a markdown file.
09:19And this also allows feature agents to smartly query that and kind of manage their context dynamically.
09:26And and this is really where web search is used most because it will go off and read archive and and get kind of good context from the public domain as well.
09:36And so this can vary a lot, but um it takes, you know, roughly around 3 to 4 hours.
09:43The second phase and what um is most different in 2.0 is the eval building, but just to say sort of our first implementation, you know, of course the evaluation
09:53is the most important piece and LLMs aren't malicious, but they can make, you know, very silly mistakes and if you're optimizing against a bad a bad eval, the whole thing kind of falls apart.
10:05So our attempt to be more robust is have this sort of multi-agent framework.
10:10So one is tasked with first building the eval, like actually writing all of the code, and then that goes off to two critic agents.
10:18One that's told to be more high-level, like are there conceptual errors in our evaluation or any kind of forward leakage of information, and then one that's more programmatic.
10:28So it's like writing unit tests and integration tests.
10:31Um and they write up any issues they find of our issues, it goes back to the builder to fix, and this loop doesn't end until sort of all of them are happy that the eval is good.
10:43And then the third phase and really the heart of everything is this mass experimentation.
10:48And so we've we've chosen, you know, both in the code and in our UI, this is kind of formulated as a Jira board or Kanban board.
10:57And so there's this sort of strategist agent that gets to look and query all the context from the previous steps, and it's just supposed to keep coming up with experiments
11:07it wants to try, and it submits them to this like to implement column.
11:11And then as new cards come in, those get pawned off to worker agents that are tasked with actually writing the code to implement the strategy, writing the Slurm config, you know, what hardware does it need, submitting it to our cluster,
11:25waiting for the job to finish, and then look at, you know, the the machine learn learning training curves, like did it underfit, overfit, and then also looking at the eval results.
11:35And then it sort of writes this postmortem analysis, which goes back to the strategist as each job finishes.
11:41So, the strategist hopefully can do this kind of self-evolution.
11:45So, it can see like I suggested, you know, three variants of transformers that actually didn't work too well, but uh XGBoost is is working really well.
11:54So, I want to explore more tree methods or something like that.
11:58Um And also in this step is where we as users can steer.
12:02So, you can, you know, another reason we pick sort of this Jira formation is uh you can cancel cards, you can add your own cards.
12:11There's also a chat feature that's sort of cut off here, but you can chat with the strategist and kind of steer it towards more creative or kind of give it intuition of different
12:21methods that should try.
12:23Um and then we keep this, you know, ever-growing leaderboard where you can see like given your eval and given a held-out private validation set, like what are the best-performing
12:33models?
12:34And you can click in and see like, you know, from inception of the idea to the code, and you can pull it out and play with it.
12:41And then this is just sort of our UI 2.0, which is really just optimized to look cool, but you can see like, you know, it's suggesting experiments, and this is really sped up, but how each worker kind of pulls it through,
12:54um you know, building, deploying, and then analyzing the results.
12:59And um you know, in our paper, we did more sort of academic data sets.
13:05So, we looked at CUDA kernels, we looked at uh an academic traffic time series data set, and we did the kind of classic like Karpathy-style LM speed running.
13:16Um and it's sort of hard to benchmark AlphaLab, but we compared to more of a Karpathy-style like single agent in a loop going, and it did find a better
13:26uh training config for training an LLM.
13:30We also put this on a Kaggle competition, which was to It was, uh, hosted by Nvidia to fine-tune their NeMo Tron model to be a reasoning model.
13:40Uh, and it got in the top 12% of submissions.
13:44Which I think is decent, and also it only had sort of 10 iterations to work with because we joined late, and I think, you know, AlphaLab works best the more iterations it can explore.
13:54So, presumably or hopefully it would have done better had it, uh, had more time.
13:59And then I can say at a high level internally, there's been a handful of of models, all sort of of the flavor where we had a decent model already, but we just turn it over to AlphaLab to keep kind of churning on it.
14:13Uh, where it's found meaningful improvements that are now working their way through risk and going into production.
14:20And so with the my last couple minutes, I I want to talk about, um, you know, we we ran into some real issues and hard questions.
14:29Uh, I think the first was, you know, okay, so we had some some good results, but we've also had cases where it's really failed.
14:36And so it's kind of always in our head, like, how real is is any of this, you know, if it if it's failing on these hardest problems, how do we measure,
14:45how, you know, how real this is.
14:47And the second was, and I'm sure a lot of you might be thinking, like, okay, your research phase sounds reasonable, having a strategist and workers sounds reasonable, but isn't it sort of arbitrary,
14:58you know, like, how do you motivate these design choices?
15:01And how we feel now is like, and I think a big theme of this conference is, you know, should we be making these decisions at all?
15:08Like, this itself is a verifiable loop.
15:11Like, an LLM should be doing this this meta-optimization itself.
15:15And then lastly, you know, in this this 1.0 formulation, it's sort of like we give the data, we give the goal, and the LLM just goes off and does it.
15:25So, like how does that exactly encode our, you know, Morgan Stanley's enterprise knowledge, but just our expertise as as quants?
15:33Like it's sort of missing from the picture.
15:35And so, the answer to all three we feel is really in building our own evals and environments.
15:42And so, to just sort of go one by one through the questions, you know, to the first point, and this is a lesson I learned over and over, like monthly, working with these models,
15:52you have to start with good eval.
15:54Like it's such an obvious thing I think we were kind of overeager and wanted to treat it more like a a human researcher, but you of course need like a very clear way to measure.
16:04And so, the the biggest kind of change is now we're we're very opinionated about the eval.
16:11It's very much like Kaggle.
16:13And so, you know, we treat it as data and description in, the harness lives in the middle, and its only job is to submit containerized models.
16:22And it gets sort of the the feedback of like a public leaderboard score, but you as a user get to see a private leaderboard, like held-out validation, how's the model performing.
16:31And so, you know, in this case, you can measure like just for any given task, how well does your model do?
16:39But of course, once you have this strict kind of format, you can think, okay, you know, to me evals and environments are the same thing.
16:46It's just you train in environments.
16:49So, now what we've done is built on the order of like 10 to 20 really careful environments, and that becomes a reinforcement learning signal.
16:57And so, we're you know, quote-unquote AlphaLab meaning AlphaLab.
17:00So, once you have that way to measure, you know, you can do human tuning of the the harness.
17:06So, like maybe there should be two strategists, and maybe they should debate, or maybe there, you know, whatever kind of ideas you have, you at least have a way to
17:14measure and kind of manually hill climb.
17:16But what we're doing now is really this meta harness optimization, where the LLM is looking at the trace, is looking at the about, you know, the results, and improving the harness itself.
17:27And also sort of a a tangential axis is we're now collecting good traces from open source model and really touching weights and doing like GRPL or or other, you know, on policy distillation methods.
17:40And so, we see like the the best performing thing might be an orchestration of open source and close source models, but we're kind of optimizing the whole thing together.
17:50And to the last point, you know, how do we as as experts encode our expertise, we again think it's through environments.
17:58Like building environments, at least good environments, is really really hard work.
18:03You know, it's you have to be there's of course like the data that goes in that's proprietary, but you know, designing the the verifiable metrics is maybe the easy part, but then we're also doing this qualitative
18:14rubrics, where we kind of look at the traces and say, you know, what makes a good researcher, what's the thought process, and we can grade each roll out on, you know, how well it's following our research process.
18:27And so, having these good rubrics that become the signal for the model to learn, we feel is really how we're we're building our own expertise into the system.
18:37And so, just to kind of conclude the the 2.0 version is really having this strict environment and eval setup, and you know, whatever lives in the middle, we almost
18:47in the limit kind of don't care about.
18:49We can initialize it to Alpha Lab 1.0, but this it should really be this self-improving system.
18:55Um and just to leave kind of the the bigger picture headline result, and my feeling is you know, this ability to do general auto research, I think will kind of become a commodity.
19:07Like I think we're already seeing it with GLM 5.2, and so I really think all of your value as like an enterprise or human expert comes from building environments.
19:17And so like, you know, temporarily for us that we're still doing like manual tuning against the environment, but I think in the limit the the auto research can research itself and just be the self-improving
19:30um process.
19:32And so, um my site's on there and then also the the project page again the 1.0 version we released everything and our plan is to keep just releasing because again,
19:43we think the environments encode all of the the value.
19:47Uh so, thank you.
20:04[music]