WEBVTT

NOTE Sentence-level transcript of https://www.youtube.com/watch?v=YnNF55QV0zs

NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/YnNF55QV0zs.html. A cue ends where the next begins, or 2 s after its last word.

s1
00:00:01.309 --> 00:00:03.309
[music]

s2
00:00:13.040 --> 00:00:20.040
Hello, I'm Ishan and welcome to can AI predict people like we predict the weather.

s3
00:00:20.040 --> 00:00:24.840
A field guide to the nascent field of synthetic personas.

s4
00:00:24.840 --> 00:00:32.720
Now, I'm sure most of you in this room at some point or another have prompted a large language model with a role prompt.

s5
00:00:32.720 --> 00:00:37.000
You are a fill in the blank and then the task.

s6
00:00:37.000 --> 00:00:48.560
Believe it or not, that core principle of steering a model's outputs as if it were a particular person or persona has turned into an entire category that companies are using

s7
00:00:48.560 --> 00:00:53.680
to test product concepts and messaging against synthetic respondents.

s8
00:00:53.680 --> 00:01:03.840
And it has moved from a novelty to market momentum, as you can see from both these headlines as well as the increase in funding for the last few years.

s9
00:01:03.840 --> 00:01:10.680
And the running analogy I want to leave you with is that synthetic personas are like weather forecasting.

s10
00:01:10.680 --> 00:01:16.360
Like weather forecasting, they were unlocked thanks to an increase in compute and data.

s11
00:01:16.360 --> 00:01:25.520
And like weather forecasting, they operate within a particular regime and going past that sometimes can go outside of where they're accurate.

s12
00:01:25.520 --> 00:01:31.240
So, for example, you can only predict the weather a certain number of days in advance.

s13
00:01:31.240 --> 00:01:36.280
Similarly with synthetic personas, there's only so far you can go before you'll run into issues.

s14
00:01:36.280 --> 00:01:42.760
And understanding those issues are as important as understanding their promise and their potential.

s15
00:01:43.520 --> 00:01:45.640
So, I'm Ishan Nand.

s16
00:01:45.640 --> 00:01:49.080
I'm the chief AI officer at Insight Sciences.

s17
00:01:49.080 --> 00:01:55.120
We construct LLM synthetic personas for market research and market insights teams.

s18
00:01:55.120 --> 00:02:02.360
And the reason for this talk is that most of the coverage in the space is very shallow, doesn't go into the technical details.

s19
00:02:02.360 --> 00:02:06.160
It's either outright hype or outright dismissal.

s20
00:02:06.160 --> 00:02:11.480
And it's really hard to separate the noise from what's real.

s21
00:02:11.480 --> 00:02:14.959
So what I want to cover is that messy middle of the technical details.

s22
00:02:14.959 --> 00:02:24.280
And you don't have to take my word for it, even though I'm a vendor in the space, because everything I'm going to talk about today is going to be based on published research.

s23
00:02:24.280 --> 00:02:35.600
So we're going to cover why now for synthetic personas, how they fail, some techniques to inspire you, and then some metrics to judge whether your synthetic persona is accurate or not.

s24
00:02:35.600 --> 00:02:48.280
Speaking of weather forecasting, another parallel is just like in the 1950s and '60s, we got computers that promised us, correctly, a future of accurate weather forecasts.

s25
00:02:48.280 --> 00:02:52.200
We were also promised, believe it or not, people forecasts.

s26
00:02:52.200 --> 00:03:04.920
This company, Simulmatics, were extensively covered by Jill Lepore, promised that they could simulate and predict the electorate using raw statistics and the computational power at the time.

s27
00:03:04.920 --> 00:03:07.400
Fortunately, that turned out not to be the case.

s28
00:03:07.400 --> 00:03:10.480
So you should approach claims like this with some humility.

s29
00:03:10.480 --> 00:03:13.120
But we have something they did not have then.

s30
00:03:13.120 --> 00:03:20.080
And that unlock is, again, more computational power, but also better modeling thanks to LLMs.

s31
00:03:20.080 --> 00:03:23.720
And LLMs unlock a new kind of simulation.

s32
00:03:23.720 --> 00:03:30.200
For the longest time, to simulate something meant to mathematize it in formulas or equations.

s33
00:03:30.200 --> 00:03:38.160
But certain things, how we feel, how we act, what choices we make, aren't always succumbing to the equations.

s34
00:03:38.160 --> 00:03:45.040
What LLMs offer us is a new medium, a new atomic unit of language itself that we can model against.

s35
00:03:45.040 --> 00:03:54.280
Now granted, they are based on math under the hood, but it gives us this intermediary layer that we can construct and simulate against that we couldn't before.

s36
00:03:54.360 --> 00:03:57.520
And the process can work.

s37
00:03:57.520 --> 00:04:03.000
I want to share with you one of the most well-known demonstrations of this in the field.

s38
00:04:03.000 --> 00:04:06.800
What they did is they took about a thousand humans.

s39
00:04:06.800 --> 00:04:14.360
They put them through about two and a half hours of extensive interviews about their background and their views and their attitudes.

s40
00:04:14.360 --> 00:04:19.720
And they put those people through a battery of personality tests and surveys.

s41
00:04:19.720 --> 00:04:30.000
Then they took those transcripts and they passed it to an AI agent and they had the AI agent take the same set of surveys and personality tests.

s42
00:04:30.000 --> 00:04:39.560
What they found was as the agents were basically about 83% aligned and predictive to the corresponding humans they were modeled against.

s43
00:04:39.560 --> 00:04:45.960
Now one caveat is that number is normalized against the uncertainty and noise of the humans themselves.

s44
00:04:45.960 --> 00:04:49.840
It's a theme we're going to come back to at the end of this talk.

s45
00:04:49.880 --> 00:04:59.000
But don't get too excited because synthetic personas are different from regular experiments and they're liable to confuse and fool you if you don't know how they fail.

s46
00:04:59.000 --> 00:05:05.080
So I'm going to cover three important failure modes that you need to know about when dealing with synthetic personas.

s47
00:05:05.080 --> 00:05:10.240
To understand the first one, I want to consider this prompt these researchers gave.

s48
00:05:10.240 --> 00:05:15.080
It's a very un It's a very ambiguous and very unsophisticated prompt.

s49
00:05:15.080 --> 00:05:21.760
It basically says you're a customer, I'm going to show you a product, I'm going to tell you the category, I'm going to give you its price.

s50
00:05:21.760 --> 00:05:27.240
Those are going to be the variables in the template and then I'm going to ask you to say whether you're going to purchase or not purchase.

s51
00:05:27.240 --> 00:05:32.200
Willingness to pay, willingness to purchase is basically the test.

s52
00:05:32.200 --> 00:05:39.320
What the researchers did is they recruited a panel of humans and put them through the same test and then they put the synthetic personas through the same test.

s53
00:05:39.320 --> 00:05:41.280
What they found is very interesting.

s54
00:05:41.280 --> 00:05:43.160
So the humans are here in red.

s55
00:05:43.160 --> 00:05:46.360
They do exactly what you would expect from basic economic theory.

s56
00:05:46.360 --> 00:05:53.960
As the purchase price increases, we see that the purchase probability goes down, slopes downward.

s57
00:05:53.960 --> 00:05:56.160
But the LLMs did something different.

s58
00:05:56.160 --> 00:05:58.800
They had this inverted U-shaped curve.

s59
00:05:58.800 --> 00:06:06.920
And particularly problematic is this area right here, where as the price is increasing, the purchase probability is going up.

s60
00:06:06.920 --> 00:06:09.160
That seems really bizarre.

s61
00:06:09.160 --> 00:06:20.360
Through a series of additional experiments, what they discovered was that the LLM was using the price as a proxy for other properties about the product that the humans were considering were fixed.

s62
00:06:20.360 --> 00:06:28.000
Things like the expiration date based on the price, what the price of competing products were also as the price changed.

s63
00:06:28.000 --> 00:06:36.280
And those correlations, those latent confounders that weren't clear and immediate, were actually confusing the result.

s64
00:06:36.320 --> 00:06:44.640
And the way to think about this is when an LLM is missing context, it has to potentially infer or invent confounders.

s65
00:06:44.640 --> 00:06:45.000
Right?

s66
00:06:45.000 --> 00:06:54.160
When we do a human experiment, if I put it like a gold watch on a table, I ask a human to walk in and estimate the price of it, everything about the environment is fairly fixed.

s67
00:06:54.160 --> 00:06:57.080
The human and their decisions are the random variable.

s68
00:06:57.080 --> 00:07:04.919
In a synthetic experiment, if you don't set it up properly, other parts of it actually become part of the random variable itself.

s69
00:07:04.919 --> 00:07:10.400
I like to say if it's a poorly grounded persona, it's a little like the LLM is playing improv with you.

s70
00:07:10.400 --> 00:07:12.440
It's like gold watch on a table?

s71
00:07:12.440 --> 00:07:14.640
Oh, well, we must be in a jewelry store, right?

s72
00:07:14.640 --> 00:07:16.440
It has to infer what's likely.

s73
00:07:16.440 --> 00:07:19.760
And maybe this is a rich person, so they're more likely to purchase.

s74
00:07:19.760 --> 00:07:29.320
And so the lesson is, we need to richly ground our personas in the personality, the context, and bizarrely, even the study's own construction.

s75
00:07:29.320 --> 00:07:34.280
In a human subject experiment, you want to hide the study construction from the participant.

s76
00:07:34.280 --> 00:07:40.919
But in the case of an LLM, they have no universe other than what's in the prompt, and you have to use the prompt to paint the world to prevent

s77
00:07:40.919 --> 00:07:43.520
any type of confounders.

s78
00:07:43.600 --> 00:07:46.080
Another failure mode is prompt sensitivity.

s79
00:07:46.080 --> 00:07:53.760
So, here's a researcher that took a question, they give the same question, same choices, they just swapped the order of the choices.

s80
00:07:53.760 --> 00:07:58.680
Yes was the first one in the first question, yes was the second option in the the second question.

s81
00:07:58.680 --> 00:08:03.320
What they found was that the model had an extremely strong order bias.

s82
00:08:03.320 --> 00:08:09.720
Basically, when they took the two results and they averaged them together, it washed out into noise, into 50/50.

s83
00:08:09.720 --> 00:08:13.560
Now, humans do have a first order bias, but not to this extent.

s84
00:08:13.560 --> 00:08:27.040
And so, the lesson here is that we need to durability test our personas to understand how they will change under reorderings, under rewordings, and even adversarial challenges to their opinions.

s85
00:08:27.240 --> 00:08:35.719
The third and final area that I want to highlight is that LLMs are trained on what people say, and they're not trained on what people do.

s86
00:08:35.719 --> 00:08:42.640
So, as a consequence, predicting stated attitudes tend to be easier than predicting actions or behaviors.

s87
00:08:42.640 --> 00:08:49.360
Both because they're clear and likely to be in the text, but also because they are natively text themselves.

s88
00:08:49.360 --> 00:08:56.480
So, this chart is from a bunch of researchers that used an LLM to try and predict known social science experiments.

s89
00:08:56.480 --> 00:09:00.839
The original point of this chart is to show that the LLMs are about as good as the experts.

s90
00:09:00.839 --> 00:09:07.600
LLM is in black in a circle, the experts are in blue, and you can see they're both doing about equally well in making the prediction.

s91
00:09:07.600 --> 00:09:11.920
But, the point I want to draw you to is that there are two categories of experiments here.

s92
00:09:11.920 --> 00:09:17.920
The top are surveys, those are natively language and text-based, and those reflect attitudes.

s93
00:09:17.920 --> 00:09:21.280
And on the whole, the models tend to do better there.

s94
00:09:21.280 --> 00:09:23.280
The bottom half is field experiments.

s95
00:09:23.280 --> 00:09:26.360
Those are behaviors, and those are things that need to be transcribed into actions.

s96
00:09:26.360 --> 00:09:31.400
They're less likely to be in the training data, and correspondingly, the LLM doesn't do as well.

s97
00:09:31.400 --> 00:09:38.640
So, the lesson we often tell our clients is consider questions that triangulate to behavior from attitudes.

s98
00:09:38.640 --> 00:09:51.760
As a hypothetical example, if you want to know about gym attendance, you might be better off, well, you can ask about both, but asking about attitudes towards working out rather than asking about attendance and see if that's a suitable proxy.

s99
00:09:51.760 --> 00:09:52.760
Okay.

s100
00:09:52.760 --> 00:09:59.480
Now, let's talk about three example techniques to kind of inspire your own synthetic personas.

s101
00:09:59.480 --> 00:10:02.920
So, the first one is just prompting the model.

s102
00:10:02.920 --> 00:10:09.200
Uh this right here is from the Argyle paper, which is really one of the seminal papers in this field.

s103
00:10:09.200 --> 00:10:14.000
In fact, it's so early that the model they used was a text completion model.

s104
00:10:14.000 --> 00:10:16.680
That's why this prompt isn't in the form of a chat.

s105
00:10:16.680 --> 00:10:19.120
It's a statement of I am.

s106
00:10:19.120 --> 00:10:24.280
So, they gave it a prompt that said, and for example, the middle column is basically where the context is.

s107
00:10:24.280 --> 00:10:25.480
I am a strong liberal.

s108
00:10:25.480 --> 00:10:30.360
I support progressive values, etc., etc. And at the end it says, "In 2016, I voted for."

s109
00:10:30.360 --> 00:10:38.160
And they basically let the model sample its completions and it says, uh Hillary Clinton, Bernie Sanders, Hillary Clinton, and so forth.

s110
00:10:38.160 --> 00:10:41.920
And you can see what happens with the conservative case on the top.

s111
00:10:41.920 --> 00:10:47.560
Since this time, obviously, there've been a lot more prompting techniques.

s112
00:10:47.560 --> 00:10:49.320
And a lot more models.

s113
00:10:49.320 --> 00:10:55.200
And I can't tell you which prompting technique and which model is going to work best for your use case.

s114
00:10:55.200 --> 00:11:02.040
What you are going to have to do is figure it out empirically by validating against some known human ground truth data.

s115
00:11:02.040 --> 00:11:03.839
You'll have to do what these guys did.

s116
00:11:03.839 --> 00:11:10.760
So, for example, here in this research, they're trying to figure out how well they can construct personas to represent voting patterns.

s117
00:11:10.760 --> 00:11:18.520
What they found was they compared here on the left is reality and on the right is their four different types of persona constructions.

s118
00:11:18.520 --> 00:11:26.000
And they didn't realize it at the time, but their persona construction was actually amplifying bias within the model as they got more and more detailed.

s119
00:11:26.000 --> 00:11:29.800
And they found it was actually throwing it further and further astray from reality.

s120
00:11:29.800 --> 00:11:31.839
So, you probably have a bunch of different ideas.

s121
00:11:31.839 --> 00:11:37.080
The answer is you're going to have to test it and validate it against ground truth.

s122
00:11:37.120 --> 00:11:46.960
The other natural thing you might expect is well, hey, we can fine-tune it, especially if it's missing data that isn't there, especially for example, if it's behaviors or something that wouldn't be in the training text.

s123
00:11:46.960 --> 00:11:49.960
And this is a great paper to be inspired by for this.

s124
00:11:49.960 --> 00:11:51.600
This is the Subpop paper.

s125
00:11:51.600 --> 00:12:03.880
Basically, they construct a prompt template, which is the demographic information, then the survey question they want to ask, and then they compare the known human data distribution to the distribution that comes out of the model,

s126
00:12:03.880 --> 00:12:09.080
and they do fine-tuning until the model and the human data align.

s127
00:12:09.320 --> 00:12:11.360
Now, here's the interesting thing.

s128
00:12:11.360 --> 00:12:20.440
When they did this, as you'd expect, the results that were from the populations they gave to the model, that's the ones in blue, improved.

s129
00:12:20.440 --> 00:12:24.920
But very interestingly, the ones in white also improved by almost the same degree.

s130
00:12:24.920 --> 00:12:28.040
Alignment improved even for the unseen groups.

s131
00:12:28.040 --> 00:12:30.720
That seems almost magical.

s132
00:12:30.720 --> 00:12:40.680
And some subsequent research has hinted that what might be really happening here is that the model itself has a latent understanding of these groups.

s133
00:12:40.680 --> 00:12:43.760
It just didn't know how to express it in the format of surveys.

s134
00:12:43.760 --> 00:12:48.600
And if you think about it, LLMs aren't used to doing surveys as a task.

s135
00:12:48.600 --> 00:12:54.640
And so they aren't going to be as good as fitting it, especially to a prompt format they may not have seen on the first go-around.

s136
00:12:54.640 --> 00:12:59.120
But fine-tuning actually is helping it learn the task or how to express itself.

s137
00:12:59.120 --> 00:13:03.960
So, a lesson you can kind of take away is that your persona that you're looking for is in there.

s138
00:13:03.960 --> 00:13:08.040
We just need to figure out the way to summon it or elicit it.

s139
00:13:08.040 --> 00:13:18.080
And that lesson actually takes us to the third technique, which I want to highlight to show how sophisticated your techniques can get if you're just using so-called prompting alone,

s140
00:13:18.080 --> 00:13:21.080
but using careful calibration and thinking.

s141
00:13:21.080 --> 00:13:24.720
So, in this one, this team did something very clever.

s142
00:13:24.720 --> 00:13:27.600
They set up a system prompt that was demographics.

s143
00:13:27.600 --> 00:13:29.600
They showed a product concept.

s144
00:13:29.600 --> 00:13:33.280
And then they asked how likely would you be to purchase this product?

s145
00:13:33.280 --> 00:13:42.400
And they gave it the same scale from one to five, five being most likely, one being the least likely to purchase, like you'd expect, kind of your basic naive prompting pattern.

s146
00:13:42.400 --> 00:13:51.000
And then they said, "Well, you know, hearkening back to that paper, although I don't know if they were inspired by it, they said, 'Well, large language models aren't used to doing surface,

s147
00:13:51.000 --> 00:13:54.760
but they are more used to expressing themselves in text.'

s148
00:13:54.760 --> 00:13:59.600
So, they said as instead of giving us a one to five rating, give us a set of text.

s149
00:13:59.600 --> 00:14:01.960
So, the example here is, 'I'm somewhat interested.

s150
00:14:01.960 --> 00:14:05.480
If it works well and isn't too expensive, I might give it a try.'

s151
00:14:05.480 --> 00:14:14.920
And then to map that text to the one through five willingness to pay, they had humans write out corresponding text for what they would expect.

s152
00:14:14.920 --> 00:14:17.560
So, if it's a one, "Hell no, I'll never buy that."

s153
00:14:17.560 --> 00:14:19.720
Five, "Absolutely, I'll buy 20."

s154
00:14:19.720 --> 00:14:20.320
Right?

s155
00:14:20.320 --> 00:14:30.000
They had them write out examples of each one of the different options, and then they measured the semantic similarity between the text that came out of the model and those human examples.

s156
00:14:30.000 --> 00:14:38.320
And that gave them a vector over which they can basically measure a probability distribution of where this text that came out of the model lands.

s157
00:14:38.320 --> 00:14:40.920
So, what I like about this is it's actually a distribution.

s158
00:14:40.920 --> 00:14:42.120
Kind of feels like, you know, human.

s159
00:14:42.120 --> 00:14:48.920
Some days I might say four, some might Some days I might say five in this graph, but rarely would I say one, two, or three in this example.

s160
00:14:48.920 --> 00:14:55.720
And what they were able to show is that they were not able to only reconstruct accurate values for willingness to pay.

s161
00:14:55.720 --> 00:14:57.800
They were able to capture the distribution.

s162
00:14:57.800 --> 00:15:06.480
Because one of the important failure modes we haven't talked about is that LLMs, even when they get the persona averages right, they very often lose the details.

s163
00:15:06.480 --> 00:15:09.920
The variations get muddled together in the middle.

s164
00:15:09.920 --> 00:15:16.839
This chart at the bottom, basically that horizontal axis is a measure of the entire shape similarity.

s165
00:15:16.839 --> 00:15:19.920
And one means perfectly identical, and zero means not.

s166
00:15:19.920 --> 00:15:28.160
And what you can see is the naive way, in the purple or I guess pink uh doesn't do as well as the yellow which is up near the top of the range.

s167
00:15:28.160 --> 00:15:35.840
So that means it really did a good job not only understanding what the ultimate choice was but how well that choice varied.

s168
00:15:35.840 --> 00:15:36.720
Okay.

s169
00:15:36.720 --> 00:15:41.320
Let's talk about how to measure alignment from a synthetic persona.

s170
00:15:41.320 --> 00:15:53.120
One of the things that our traditional market research clients are sometimes surprised by and disappointed is that you cannot use statistical synthetic personas to boost statistical significance.

s171
00:15:53.120 --> 00:15:59.720
You can take an underrepresented population and get more values out of it but you can't say it's statistically significant.

s172
00:15:59.720 --> 00:16:02.760
And to understand this it helps to go back to that weather analogy.

s173
00:16:02.760 --> 00:16:09.960
If I want to know how much it rains today in San Francisco and I used to live here so I know it rains a lot, I'd stick a weather gauge.

s174
00:16:09.960 --> 00:16:16.920
If I want to know with more certainty, I'd stick a thousand weather gauges and those would increase the accuracy of my estimate.

s175
00:16:16.920 --> 00:16:26.680
But if I want to know if it's going to rain tomorrow, if I take a forecast and I rerun it a thousand times without changing the input, that doesn't change my certainty of that forecast.

s176
00:16:26.680 --> 00:16:32.440
It improves my estimate of what the model is telling me but it doesn't make the forecast itself more accurate.

s177
00:16:32.440 --> 00:16:36.960
And that's what happens when you basically are rerunning a synthetic persona with no changes to input.

s178
00:16:36.960 --> 00:16:43.480
So the lesson is more synthetic samples aren't actually going to improve your statistical significance for the most part.

s179
00:16:43.480 --> 00:16:46.800
So what you need to do is you need to do what you do with weather forecast.

s180
00:16:46.800 --> 00:16:52.240
You'd basically check against what actually happened or in our case what humans actually said.

s181
00:16:52.240 --> 00:16:55.839
And that's where we're going to basically be measuring distributions of data.

s182
00:16:55.839 --> 00:17:04.400
Unlike classic e-vals where there's clearly a right and wrong and you can score how many were right and how many wrong, now we need to measure the data as a comparison of distributions.

s183
00:17:04.400 --> 00:17:07.000
And there are many ways for distributions to get wrong.

s184
00:17:07.000 --> 00:17:08.600
They could be completely wildly off.

s185
00:17:08.600 --> 00:17:12.760
They can as we mentioned get the average right but the shape of the distribution wrong.

s186
00:17:12.760 --> 00:17:19.079
And so you're going to need multiple metrics to capture how well your model is reflecting different personas.

s187
00:17:19.079 --> 00:17:28.120
Um I recommend using a correlation type metric along with one of these shape type metrics which capture what the underlying shape of the distribution is.

s188
00:17:28.120 --> 00:17:33.160
The other thing you need to do is estimate the fundamental noise in your ground truth data.

s189
00:17:33.160 --> 00:17:43.480
That experiment I talked about in the beginning where they got 83% accuracy, the key smart thing they did is they took those humans and they brought them back 2 weeks later

s190
00:17:43.480 --> 00:17:51.840
and they redid the battery of surveys and personality tests and they found that the humans on average were only 80% consistent to themselves.

s191
00:17:51.840 --> 00:17:58.880
So that sets a noise floor as how accurate our models could ever get because the humans themselves are fundamentally noisy.

s192
00:17:58.880 --> 00:18:03.760
And so the 83% is actually normalized against that.

s193
00:18:03.880 --> 00:18:06.680
If you can do this and bring your humans back, that's great.

s194
00:18:06.680 --> 00:18:08.000
Very often you can't.

s195
00:18:08.000 --> 00:18:16.880
So the way you can kind of artificially do this is take your ground truth human data, break it into two chunks, and then pretend one is synthetic and one is human,

s196
00:18:16.880 --> 00:18:29.680
and then measure the correlation and repeat that hundreds and thousands of times and average it, and that'll set kind of a noise floor that your ground truth data where half of it's synthetic, half of it real, could be the level of accuracy you could hope to get.

s197
00:18:29.680 --> 00:18:38.360
So hopefully by now you have an appreciation for why I think weather forecasts are the best lens to understand synthetic personas.

s198
00:18:38.360 --> 00:18:41.960
They are not people, they are forecasts, and we should treat them accordingly.

s199
00:18:41.960 --> 00:18:51.560
Both systems are bounded, both systems will be improving over time, and they're most trustworthy when they're validated against reality.

s200
00:18:51.800 --> 00:18:59.000
Um Now synthetic personas are very often cast in the market against human research.

s201
00:18:59.000 --> 00:19:02.440
And I think that's unfortunate because they're actually complementary to each other.

s202
00:19:02.440 --> 00:19:03.960
Let me give you two reasons why.

s203
00:19:03.960 --> 00:19:10.040
One is that we're entering an era where humans are no longer the sole economic actor.

s204
00:19:10.040 --> 00:19:20.240
Every action your human customer is taking in terms of awareness, consideration, or a purchase decision to buy is being increasingly mediated by AI agents.

s205
00:19:20.240 --> 00:19:24.560
So, a human-only study is actually not the gold truth.

s206
00:19:24.560 --> 00:19:30.760
What we really need to understand is what does the human plus agent ecosystem look like?

s207
00:19:31.440 --> 00:19:36.040
And then finally, the alternative to a synthetic persona is not human research.

s208
00:19:36.040 --> 00:19:39.440
In most cases, it's no research or it's somebody's opinion.

s209
00:19:39.440 --> 00:19:45.320
What really happens is you've done a survey of humans and you get a question and if it's in the survey, you can just answer it.

s210
00:19:45.320 --> 00:19:46.920
That's very simple to do.

s211
00:19:46.920 --> 00:19:52.160
But what typically happens is it's 2 months later and you're like, we need to answer this question which we didn't ask.

s212
00:19:52.160 --> 00:19:57.120
Well, then somebody needs to be like, "Uh I think it would be this by extrapolation."

s213
00:19:57.120 --> 00:20:01.880
Uh expert plus a synthetic persona is going to give you a better result to that.

s214
00:20:01.880 --> 00:20:08.160
So, what we like to tell customers is synthetic extends your human data to more phases of your development process.

s215
00:20:08.160 --> 00:20:11.400
It can go more places your existing research can't.

s216
00:20:11.400 --> 00:20:15.000
One of the most exciting directions is to actually run simulations.

s217
00:20:15.000 --> 00:20:25.240
We didn't get time for this, but it's called generative agent-based modeling where we can take each of these personas and simulate with the dynamics and how they'll interface interface and interact with each other.

s218
00:20:25.240 --> 00:20:33.360
And ultimately, what this will let you do is turn your human data into a living queryable asset.

s219
00:20:33.360 --> 00:20:36.680
If you're interested in doing that with your data, feel free to reach out to us.

s220
00:20:36.680 --> 00:20:42.440
We help market research and insights teams generate and use synthetic personas in AI.

s221
00:20:42.440 --> 00:20:48.360
You can find us on the web at insights sciences.ai and my contact information is on the slide.

s222
00:20:48.360 --> 00:20:49.680
I hope you have a good conference.

s223
00:20:49.680 --> 00:20:51.064
Thank you.

s224
00:20:51.064 --> 00:20:53.064
[applause]

s225
00:21:06.784 --> 00:21:08.784
[music]