How Forward Deployed Engineering is done at Ramp — Leo Mehr

AI Engineer · 14 min · 154 sentences · from YouTube's caption track

Each timecode opens YouTube at the start of that sentence. Line anchors (#s42) are the cue ids in the WebVTT, and every line carries its start and end seconds. All transcripts has every talk, and the whole corpus as one file.

00:01[music]
00:13Awesome.
00:13Thank you, guys.
00:14Awesome.
00:14It's great to to meet everyone.
00:16I mean, I hope that after the talk, you know, if you want to come out and we can chat, we'd love to.
00:20Um Cool.
00:22So, yeah, today my goal is to share with you guys the two most important principles from what we learned doing FDE at Ramp.
00:32So, just yeah, briefly a little bit about myself.
00:34Yeah, I'm a director of engineering at Ramp.
00:36Uh I joined the company 2 and 1/2 years ago when it was just you know, FDE was just two engineers at the time.
00:44And today, my org is about 30 engineers across four deployed developer API and our new AI services um business.
00:55So, I know this is kind of a running theme, but like no one knows what FDE is.
01:00So, I'm just going to spend a moment on that.
01:06So, yeah, I I I actually kind of like this meme.
01:08It's To me, it's kind of funny.
01:10Um I um but I I I actually think it's like totally wrong.
01:14I don't see this as the actual like true form of what FDE is.
01:19Um I don't see it as like the final evolution or like boss mode of technical go-to-market roles.
01:27Um now, this might be true at some companies, but at least at Ramp, it's a little bit different.
01:35So, FDE at Ramp, uh we live within the engineering organization.
01:41And our goal is to help Ramp win upmarket.
01:45So, with that in mind, what we do is we basically work on the core product and our new agentic features and make them work really well for our largest
01:56enterprise customers.
01:59So, that's just a little bit of intro context.
02:02I want to dig in and and today, like I said, there's just two things I'm going to share with you.
02:06Literally two things.
02:07Very easy talk.
02:10And these are the principles that I would say have really guided us and I would say probably the two most important things that we have.
02:16Always be scoping and scale with tokens.
02:21So, let's get with start with the first one.
02:23On scoping.
02:25So, I would say there's this thing where like people many many people think that as an FDE, your job is to just say yes to the customer.
02:33But that's wrong.
02:36If you were just to say yes, you know, instead of like beautiful Waymos that we have driving us around in San Francisco, you'd have something like this, you know.
02:45Yeah, horses with like rockets strapped to their legs.
02:48And the point is you you want to help the customer be successful.
02:55You want to try to figure out a way to say yes.
02:57But you actually want to deliver good software.
02:59You need to build the right thing.
03:01So, you don't just endlessly say yes to people.
03:04And I I do want to share an example of something that I would say happens somewhat regularly in one form or another at Ramp.
03:15So, it's Friday night and an enterprise sales rep comes to us with an urgent request that this super important strategic logo is only going to close if we build out an SAP S/4HANA integration.
03:32And I think that the default engineering reflex is like, "Shit.
03:37Like, where are the SAP API docs?
03:39Like, where do I find them and how do I build this integration?"
03:42But what an what a well-trained FDE would do is like pause for a second and say, "Okay.
03:49First of all, like, what's driving the urgency here?"
03:52Like, one thing I've seen is I've seen sales reps who like go kind of crazy because it's like the end of the quarter and they're trying to hit their quota and close the deal and not because the customer is the one driving the urgency.
04:03So, that's like one example.
04:05[clears throat]
04:06But, you know, as an FDE, you're asking tons of questions to gather context about what's important um and what actually is the right thing to build.
04:16And so, you might ask like, "Who's using this integration?
04:21Have we exhausted all the different workarounds?
04:23Is there something manual that we can do in the meantime?
04:25Does the customer have like technical resources?
04:27Can they hit our API such that we don't have to build this thing?"
04:32But, I'd say the most important thing that an FDE does is also looks beyond this one request and looks at the other prospects that are coming down the pipeline and other customers to see if anyone else would benefit
04:47from this as well.
04:48And the point is that by gaining all this context, you can do a better job of building the right thing.
04:57So, I want to share another story that was really really painful for us in the early days.
05:02We had this large enterprise customer and they needed this reimbursement feature on mobile.
05:10Unfortunately, our mobile team was totally swamped.
05:13So, we basically just had to roll up our sleeves as FDEs and just get find out a way to get things done.
05:20And we had two of the engineers on the team just like learn how to do iOS and Android development.
05:25And it was awesome.
05:27We were super excited.
05:28We're like, "Okay, we're going to ship this feature.
05:29It's going to be so good.
05:30Like, hell yeah."
05:32So, we grinded for a couple weeks, got the feature done on both platforms.
05:36And we go to the customer and we're like, "Awesome.
05:39Like, can you send us your list of, you know, beta users for Android?"
05:44And that's when they told us they only they they they require they mandate all of their employees to use iOS devices.
05:54So, you're like, what the Like Not not to the customer, you know, just internally.
06:00But like obviously it was super disappointing for us because we'd put all this effort in.
06:04And so, it was a a big lesson for us to remember the importance of scoping.
06:08Even some of the most basic assumptions like which you know mobile platform you build on it's it's super important um to validate them and and thus kind of emphasizes the importance of scoping up front.
06:20Now, Okay, so let's say that you and your team have become masters of scoping.
06:26You know, you're you're amazing.
06:28In today's world, this is not enough.
06:33So, unless you are scaling with model capabilities, you are going to fall behind.
06:40Now, I'm not going to belabor this point too much.
06:41I think like every talk in this uh in in this conference is probably some flavor of this, but like the point is that we basically have to reinvent our jobs constantly now.
06:52So, whatever work we are doing today, you know, for the most part it's knowledge work, we have to figure out how to have models and agents do it for us.
07:01And so, that brings me to the second half of this talk and the the other point that I want to convey today, which is all of us have to figure out how to scale with tokens.
07:13And the way that I interpret scaling with tokens for FTE is take a look at the whole life cycle of what an FTE does.
07:21From gathering context to scoping out a request to writing out a spec and then implementing the feature, each stage of that pipeline can be replaced with agents.
07:33And at first it seems kind of daunting.
07:35You're like, like how are you going to go and approach and like solve that?
07:39But if you break the problem down and then make progress on it, it's it's actually pretty tractable.
07:44And so, I'll share share with you guys one example of something Oops.
07:47Something that we um that we've done at Ramp.
07:49So, we have this internal Slack channel called FDE requests, and this is where account managers, solutions, uh sales reps will post whenever there is a blocker for a prospect or customer that's large enough, basically.
08:09And so, we get these requests.
08:11In this case, actually uh one of the CSMs on our team, Greg, posted here.
08:16And um if you were to It's actually a Notion workflow.
08:20If any of you work at Notion, by the way, thank you.
08:22We like use Notion so much.
08:24Um if you were to click open in Notion, you'd see like a pretty long request that has all the details of what what exactly it is.
08:31And the problem is there's a super high variance.
08:33Like, some people will submit like really detailed, good requests from the customer.
08:39And others are just going to submit like one line, like, "We need uh you know, we need this SAP integration."
08:46And before what would happen is we would have FDEs manually kind of go through this request.
08:53We'd read the whole thing, understand it, figure out what exists in the product, do a bunch of back and forth with the customer.
08:59And this is like exactly what the first half of the talk was about, always be scoping.
09:04You know, we would spend a lot of time really digging in and validating what exactly was uh you know, absolutely necessary.
09:11And so, you can see here what we what we did then was we basically um used Notion uh Notion agents to build a V1, which literally just took the request and asked a couple of questions.
09:24That was it.
09:25And um after It was kind of astonishing.
09:30Literally, after a couple of weeks, we found that it was like saving us a lot of time because, first of all, immediate like the latency of replies went from like hours or days to like, you know, seconds.
09:39And immediately, like the account reps that the account managers, the reps would start kind of engaging with this agent.
09:45And one of the things that we did was because it went so well, this this is actually um the more recent iteration of it.
09:51It's very cute, you know, the little penguin actually helps make it seem a little more friendly and approachable.
09:55Um and what it does is it actually goes and does several rounds of back and forth questioning with the submitter until it deems that it's ready to create a lot a spec, basically.
10:09And it's actually been incredible how helpful this has been for us.
10:12I I would say it's probably saved us like a large percentage, I don't know, 20% of the time that we'd spend on scoping out these requests.
10:22So, you know, this is this is a great example.
10:26For us, this has been really helpful.
10:27It's I'm super excited about this.
10:29It's going to help automate a lot of the work that we've been doing manually.
10:31But, um it's really just the first stage of this pipeline that I was alluding to.
10:37So, if you look at the first part here, like we've been able to make some progress on it.
10:44The last step as well, going from a a well-shaped spec to like a working product, obviously like Frontier models can like one-shot medium-size features.
10:53And so, the last part is also is is a lot easier for us.
10:56It's this middle part that I would say is super like gnarly and like unformed and difficult.
11:03And I'm I'm really excited about our team kind of investing a lot more and spending a lot more of our time just like building out this factory, building out agents to replace each one of these steps.
11:14And the thing is, if you look at if I were to say 6 to 12 months from now, like what does FDE at Ramp look like?
11:24Like these are the sorts of applied AI problems that we're going to be spending all of our time on, I think.
11:30Like, you know, making sure that the agent harness that's running each of those steps is running super smoothly.
11:36Um making sure that the the output quality of each of the outputs of the the pipeline is is actually good.
11:43That you know, with with evals, with rubrics, with human feedback.
11:48Um and there's of course like one of the biggest challenges, which is getting your agent the right context, you know, when you're making the alarm call, ensuring that it has the right context.
11:57So, there's like a lot of historical data, data about the the product.
12:01Imagine like all the knowledge that a product manager has in their head about their product.
12:05Like, how do you get that into an agent?
12:08Like, Notion docs and all your existing knowledge base and help articles only give you so much of that.
12:13Um yeah, skills, memories, tools.
12:16I could go on for a bit, but ultimately the most important thing here is that as an FD, we still have the responsibility of taste and judgment over the final output.
12:28So, that's going to be like the underlying kind of throughline.
12:33Okay.
12:35So, let's say that you've done an amazing job building out this factory.
12:40But the problem isn't to tie this to the first half of the talk.
12:44If you don't do a good job of scoping out requests or or building upon the principles of scoping things well, you're going to get a token maxing slop cannon.
12:56And so, the whole point is that you have to do these both because the other way around is actually quite bad as well.
13:04If you are, you know, amazing at scoping, but don't invest in building out this, you know, agent factory, you know, it's going to be over for you.
13:14Like, uh your your agent native competitors are just going to overtake you and outcompete.
13:21And so, that's why um in the end here, I want to close with the the the most important thing is that if you have both of these, it can set you up for success in the future.
13:36Always be scoping and scaling with tokens.
13:41The future of FD needs both.
13:43That's all.
13:44Thank you, guys.
13:45[applause]
14:02[music]