WEBVTT

NOTE Sentence-level transcript of https://www.youtube.com/watch?v=IK8cC4FScps

NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/IK8cC4FScps.html. A cue ends where the next begins, or 2 s after its last word.

s1
00:00:13.040 --> 00:00:14.320
All right, let's begin.

s2
00:00:14.320 --> 00:00:15.839
So, nice to meet everyone.

s3
00:00:15.839 --> 00:00:17.119
My name is Johan Raju.

s4
00:00:17.119 --> 00:00:19.600
I'm a solutions architect here at Bright Data.

s5
00:00:19.600 --> 00:00:26.640
I've been with the company for about six years, and I've been working in this space of data collection and web scraping.

s6
00:00:26.640 --> 00:00:37.600
We initially started off as a proxy infrastructure company uh with the largest residential IP network in the world and then we moved into the data collection platform.

s7
00:00:37.600 --> 00:00:44.960
Now we're the world's largest web data collection platform collecting pabytes of data per month only second to Google.

s8
00:00:44.960 --> 00:00:51.120
So just a little information for you guys on you know where we started and and how how we moved over to today.

s9
00:00:51.120 --> 00:00:53.360
So I'm going to be covering two topics.

s10
00:00:53.360 --> 00:00:58.239
First is I'm going to be speaking about building AI agents with real-time web data.

s11
00:00:58.239 --> 00:01:11.439
Some tools that people are generally using when it comes to you know AI agents versus depending on your rag model with pre-index data and what are the advantages what we see in the industry and you know what changes

s12
00:01:11.439 --> 00:01:14.799
uh are to come and what it takes to uh run these agents.

s13
00:01:14.799 --> 00:01:19.600
And then finally, I'm going to have my hands-on exercise that you guys can follow along with me.

s14
00:01:19.600 --> 00:01:30.880
Uh, where I have a product intelligence agent that will essentially use a product from Amazon, get sentiment on Reddit, and then provide you analysis with, you know, should I buy that product?

s15
00:01:30.880 --> 00:01:47.439
Yes or no. So working in the scraping space for a long period of time, one thing I've noticed is, you know, when you think about building scraping or collecting data,

s16
00:01:47.439 --> 00:01:48.560
what does it really take?

s17
00:01:48.560 --> 00:01:58.640
Right now, the early customers in the scraping space that I've worked with, you know, when when you first start, it's always about, hey, I want these are my target sites and this is what I want to scrape, right?

s18
00:01:58.640 --> 00:02:01.280
You know, people usually start off with low volume.

s19
00:02:01.280 --> 00:02:07.200
you know, that's what comprises a general traditional scraping archite architecture.

s20
00:02:07.200 --> 00:02:11.440
Now, let's say I'm an e-commerce uh user and I want to collect e-commerce data.

s21
00:02:11.440 --> 00:02:24.879
You know, my biggest targets might be Amazon and Walmart because, you know, I can find a lot of products on there and I can get price intelligence and data on reviews and, you know, all the general information for a vast majority of products.

s22
00:02:24.879 --> 00:02:28.239
Now, when does scraping become challenging?

s23
00:02:28.239 --> 00:02:38.319
And this is a change that I've actually seen with customers that I work with today um occurring in the industry when it's not just about Amazon and Walmart anymore.

s24
00:02:38.319 --> 00:02:39.519
Now you want Shopppee.

s25
00:02:39.519 --> 00:02:41.280
Now you want Shopify stores.

s26
00:02:41.280 --> 00:02:45.200
Now you want um Etsy and you know a bunch of other sites.

s27
00:02:45.200 --> 00:03:02.959
Now if we take a step back and I mentioned you know you know your target sites in order to build scrapers around your target sites it takes generally about you know 3 to 5 days to develop a scraper if you're a developer and that's not only developing it if it's QAing it running it in development testing it

s28
00:03:02.959 --> 00:03:16.319
out and you know the whole endto-end pipeline now if we think about you know 10 sites okay might take a bit of time now you think okay I want to add a 100 sites a thousand sites this is where things start getting complicated.

s29
00:03:16.319 --> 00:03:25.519
Now before AI even came into the picture, we would developers would have to manually understand the site, you know, understand where the selectors are located.

s30
00:03:25.519 --> 00:03:35.519
Can I target these data points in the API and actually build crawlers around that knowledge and I've worked with many people, you know, that's that have been scraping for

s31
00:03:35.519 --> 00:03:39.040
10 plus years that have been managing and maintaining these crawlers.

s32
00:03:39.040 --> 00:03:49.280
But when it comes to scaling up, then you're thinking, okay, like now I'm adding more sites to what I already am scraping, but I also need to manage the scrapers that I already have.

s33
00:03:49.280 --> 00:03:56.640
So let's say Amazon makes a change tomorrow and two other sites make a change tomorrow, but you're managing a thousand sites.

s34
00:03:56.640 --> 00:04:02.640
This requires a lot more engineering resources and a lot more time for you to actually figure out, okay, what change was made?

s35
00:04:02.640 --> 00:04:04.000
How can I count counteract it?

s36
00:04:04.000 --> 00:04:05.439
How can I fix it?

s37
00:04:05.439 --> 00:04:08.560
And is this actually impacting my customers, my SLA?

s38
00:04:08.560 --> 00:04:10.799
and my data quality.

s39
00:04:10.799 --> 00:04:25.120
Now before we had this bottleneck of having to scale up your resources and when at those times when you would have to manage the thousand sites you'd need you know two times more engineers, four times more engineers etc. But now in this AI space,

s40
00:04:25.120 --> 00:04:39.199
none of that is needed because we have things like agents and agents can actually go out and discover and build sites, build scrapers for these sites a lot easier than, you know, team of developers and engineers.

s41
00:04:39.199 --> 00:04:45.600
Now, this is one classic example of an e-commerce site of targeting selectors on the page.

s42
00:04:45.600 --> 00:04:50.240
So, you can see here we're just in this example, it's just targeting price and title.

s43
00:04:50.240 --> 00:04:52.639
Now, what happens if the price has changed?

s44
00:04:52.639 --> 00:04:55.759
the selector for the prices changed, you know, tomorrow.

s45
00:04:55.759 --> 00:05:08.080
That will require some downtime on your end because, you know, if you're the one scraping the site because, you know, now you're not getting price and that can affect the rest of your pipeline that you have to maybe analyze products, you know, etc.

s46
00:05:08.080 --> 00:05:15.440
Now, usually that require like a team to go in, fix it, then maybe back fill on the data that you know you've lost.

s47
00:05:15.440 --> 00:05:20.560
But now with AI agents that can be automated and done automatically.

s48
00:05:20.560 --> 00:05:28.400
So when you also build scrapers, one thing to always know is every website is different.

s49
00:05:28.400 --> 00:05:37.680
Now sites in the example I use like Amazon and Walmart, they have two different layouts even though they both might have things like price, title, description.

s50
00:05:37.680 --> 00:05:49.759
So each site requires a unique scraper to build build for and the even more important than that is each site wants to protect their data.

s51
00:05:49.759 --> 00:06:02.880
Now sites usually like to ensure that no one else is uh you know has enough like the same product intelligence they have or you know depending on what type of data that they're you know trying to keep secure

s52
00:06:02.880 --> 00:06:11.840
they will always implement different types of anti-bot and different types of capture to throw you know scrapers and bots off from collecting that data.

s53
00:06:11.840 --> 00:06:14.639
Now I've seen all different types of blocks.

s54
00:06:14.639 --> 00:06:18.160
I've seen, you know, the most basic sites that you can get through with data center.

s55
00:06:18.160 --> 00:06:26.000
And then I've seen sites that, you know, are protected by Cloudflare, Okami, you know, you name it, that you actually need to have a residential IP address.

s56
00:06:26.000 --> 00:06:27.520
You need to have a real browser.

s57
00:06:27.520 --> 00:06:29.120
And you need to seem like a real user.

s58
00:06:29.120 --> 00:06:36.479
You know, if you click too fast on the page, if you scroll too fast, that could all trigger anti-bot if you're blocking images, things like that.

s59
00:06:36.479 --> 00:06:46.800
Now, like I mentioned before, engineering has been a bottleneck until the AI space came here when we we had the AI agents ready like you know able to build crawlers

s60
00:06:46.800 --> 00:06:56.080
and you'd have to actually have a deep understanding of the site you know um a lot of sites like I mentioned because the layout is very different it means that

s61
00:06:56.080 --> 00:07:05.599
they'll try to make sure the data is as hard as possible to collect for you and you would have to have a deep level of domain knowledge expertise to get to that data.

s62
00:07:05.599 --> 00:07:07.280
data.

s63
00:07:07.280 --> 00:07:12.160
Now this is a diagram of how the traditional scraping architecture look like.

s64
00:07:12.160 --> 00:07:22.160
So just to briefly go over this we have on the left hand side the you know the a the actor essentially what's triggering this crawl.

s65
00:07:22.160 --> 00:07:34.400
So some people like using like schedulers you know um cron jobs you name it where they want the data fed to them daily weekly etc. And that kicks off the job directly over to the next layer.

s66
00:07:34.400 --> 00:07:38.639
Some people like doing, you know, specific config, specific URLs.

s67
00:07:38.639 --> 00:07:40.560
Same uh same idea right there.

s68
00:07:40.560 --> 00:07:43.919
It goes and that goes right over to the proxy layer.

s69
00:07:43.919 --> 00:07:48.560
Now, the proxy is essentially how you mask yourself from being detected.

s70
00:07:48.560 --> 00:07:51.199
There's different types of proxies that you can use.

s71
00:07:51.199 --> 00:08:06.080
you know, some static IP addresses like data center ISP, some rotating residential IP addresses that come from like laptops, tablets, TVs that are more authentic to sites because they can say, "Okay, this is registered to an actual device, we'll let them in."

s72
00:08:06.080 --> 00:08:16.960
And then some tools that have anti-bot capabilities built in like what bright data offers web unlocker or browser API which is in the browser automation side of of of it

s73
00:08:16.960 --> 00:08:29.759
where the the actual product itself manages okay is my request getting blocked am I going to retry am I using the right IP that's all in a part of the layer when you think about the anti-bot side of things

s74
00:08:29.759 --> 00:08:37.919
now from there then the next layer is you know parsing the data and actually validating if the data is, you know, authentic.

s75
00:08:37.919 --> 00:08:48.399
That's a big part of, you know, after you actually get to the site and collect the data, am I getting price back 90% of the time and do I want to validate that I'm getting it back 90% of the time?

s76
00:08:48.399 --> 00:08:50.160
Do I need review data?

s77
00:08:50.160 --> 00:08:51.040
Do I need ratings?

s78
00:08:51.040 --> 00:08:52.640
You know, things like that.

s79
00:08:52.640 --> 00:09:04.480
You have to think about, okay, like how do I ensure that I'm not going to be setting up when the data goes through my pipeline that it's going to be checked accurately by validation rules.

s80
00:09:04.480 --> 00:09:07.920
Then we go into the post-processing side of like the ingestion layer.

s81
00:09:07.920 --> 00:09:24.480
So you know dduping records enriching your data pro processing your batches that is once you you know some people like delivering it to cloud stoages some people like delivering it to their internal systems once it gets there you know what are you doing with the data how do you ensure that you're not providing the same record twice

s82
00:09:24.480 --> 00:09:26.720
or you know unclean data.

s83
00:09:26.720 --> 00:09:39.760
So you know moving it from in the medallion architecture bronze silver and then finally moving it over to your database and if anything fails in that process then you need to start the whole

s84
00:09:39.760 --> 00:09:51.680
uh process over and this is where I was mentioning you know if something breaks in the antibbots layer or if it breaks and you know selector has changed you need to start the whole process over sometimes backfilling

s85
00:09:51.680 --> 00:10:00.480
you know maybe you might miss SLAs's and things like that And so here I'm going to be talking about now specifically AI.

s86
00:10:00.480 --> 00:10:09.440
Now how many of you guys today are using things like you know claw GPT any of the uh actual like AI brains?

s87
00:10:09.440 --> 00:10:10.320
Okay, cool.

s88
00:10:10.320 --> 00:10:19.839
How many of you guys have entered a query where it's it's you see it says thinking or looking for an answer or searching the web?

s89
00:10:20.000 --> 00:10:20.560
Okay, cool.

s90
00:10:20.560 --> 00:10:21.200
Nice.

s91
00:10:21.200 --> 00:10:31.200
So this is where you see and I've split it up into two where the left hand side is where you see okay when the AI is actually thinking or searching the web

s92
00:10:31.200 --> 00:10:42.640
that's where it decides okay the query that you gave me I actually need to go out and find the answer to this I can't use my pre-trained data so some prompts like I'll give a basic example

s93
00:10:42.640 --> 00:10:54.720
when's the best time to travel to San Francisco so if I say that sometimes the the uh AI brain will usually go over to the rag side, the async where it'll

s94
00:10:54.720 --> 00:11:01.760
be used to pre-train data that you know, hey, maybe the best time to travel is during the summer, and it'll actually provide that back to you.

s95
00:11:01.760 --> 00:11:15.040
Now, if I change my query and say, when's the best time to travel this year to San Francisco, then the AI the AI has to actually go out into the web and see, okay, like do I need to look at what's going on?

s96
00:11:15.040 --> 00:11:18.240
You know, is my user asking me for events that are going on?

s97
00:11:18.240 --> 00:11:24.720
you know, how do I like I need to make a decision this year real time to find out when I can recommend for them to travel.

s98
00:11:24.720 --> 00:11:33.920
So, these are two different paths that um each a the each AI can actually make and provide you an output with.

s99
00:11:33.920 --> 00:11:38.399
Some some things I noticed is like responses sometimes can be cached.

s100
00:11:38.399 --> 00:11:46.079
Like you know if someone's asking what's the weather today in in New York City like a few times a day that might be a good response to cash because

s101
00:11:46.079 --> 00:11:56.640
you only need one person to really ask that and you can you know what the weather is going to be for the day and you can just provide it, you know, so you don't have to burn through tokens or

s102
00:11:56.640 --> 00:12:04.399
you know um uh like money to actually go out in the web and and actually find that data.

s103
00:12:05.120 --> 00:12:10.800
Now like I mentioned there's like two paths to uh gather this intelligence.

s104
00:12:10.800 --> 00:12:13.680
The sync path and then the async path.

s105
00:12:13.680 --> 00:12:16.800
Both are advantageous for different cases.

s106
00:12:16.800 --> 00:12:26.959
The sync path is you know if you really actually want to go out on the web and actually find what you're looking for and async is if you know it's okay to rely on pre-trained data.

s107
00:12:26.959 --> 00:12:30.800
And both require different sort of setups.

s108
00:12:30.800 --> 00:12:40.079
And we also do offer a tool which I'm going to cover in the next slide um where you can actually use leverage Bright Data's tools to actually go out into the web and collect data.

s109
00:12:40.079 --> 00:12:51.680
Now one thing I actually noticed is if you go into any website and if any of you guys are familiar with like robots.txt, a lot of sites usually block things like Claudebot

s110
00:12:51.680 --> 00:12:52.880
and GPTbot.

s111
00:12:52.880 --> 00:13:06.240
So if you ask these engines these questions that are actually going out to the web and trying to find the data, they might not be able to get those data from those specific sites because they're actually blocking those you know GPT or claude from actually accessing the data.

s112
00:13:06.240 --> 00:13:15.360
Now that's where we sort of come in and because what our advantageous is advantage is we're able to actually unlock the web.

s113
00:13:15.360 --> 00:13:30.000
You know whether it's if you have a scraper or if you have an agent or you're using um you know something like cloud or GPT we have tools that will allow it to actually go out into the web without actually getting blocked and scraping data.

s114
00:13:30.000 --> 00:13:34.399
And we've been masters at this for for a long time now.

s115
00:13:34.399 --> 00:13:38.800
I'm going to be talking about the the web model context protocol.

s116
00:13:38.800 --> 00:13:40.480
That's what MCP means.

s117
00:13:40.480 --> 00:13:50.399
So it usually integrates with something like claude code, lang chain, you know, name it like you could think of it as like a brain that it needs to speak to

s118
00:13:50.399 --> 00:14:00.480
and that brain essentially connects to our MCP endpoint which is essentially 60 plus APIs that we control in our library.

s119
00:14:00.480 --> 00:14:03.519
And some examples you can see here are you know search engine.

s120
00:14:03.519 --> 00:14:09.839
So, you know, Google results, scraping pages as markdown files, and scraping pages as HTML.

s121
00:14:09.839 --> 00:14:18.320
Now, if I want to target specific sites like, you know, Amazon, Reddit, LinkedIn, Instagram, that's all included as well.

s122
00:14:18.320 --> 00:14:30.399
I actually, as an example, I I worked with someone two weeks ago that, you know, was trying to build a social media listener with the MCP, and one use case they said was, "Hey,

s123
00:14:30.399 --> 00:14:32.079
um, this is my name.

s124
00:14:32.079 --> 00:14:38.800
I want to go out into the web and collect me all of my social media profiles and return me how many followers I have.

s125
00:14:38.800 --> 00:14:41.519
So what this did was he was using it on cloud code.

s126
00:14:41.519 --> 00:14:54.399
It made a call to our MCP using the search engine first found the results his pages on Instagram, Tik Tok, LinkedIn, brought that back, made the API request to our

s127
00:14:54.399 --> 00:15:00.639
data pipeline and then brought back the results on how many followers you have and gave them the context back.

s128
00:15:00.639 --> 00:15:06.399
So that's the power of you know what this can do and how much of this the web that the MCP can unlock.

s129
00:15:06.399 --> 00:15:08.800
And also this works for browser automations.

s130
00:15:08.800 --> 00:15:23.760
Um so you know creating things that you want to use a browser for like clicking scrolling you know maybe door dash like automating an order or um automating you know on a travel site trying to find you know what's the best times to

s131
00:15:23.760 --> 00:15:26.160
buy a fight a flight book a flight.

s132
00:15:26.160 --> 00:15:30.160
Um, so things like that it's very advantageous for.

s133
00:15:30.160 --> 00:15:35.680
And I'm going to show you a little video on how the MCP works.

s134
00:15:35.680 --> 00:15:38.160
Let me bring.

s135
00:15:38.160 --> 00:15:48.399
So on the left hand side you can see this is clawed with bright data's MCP setup and on the right side this is without bright data's MCP setup on claude.

s136
00:15:48.399 --> 00:15:59.279
So you can see it's the same prompt being asked and it's use your browser to get the contents of this page which is a glass door URL and on the left side

s137
00:15:59.279 --> 00:16:10.800
you can see with bright data as MCP it actually went over to glass door and got the results but on the right hand side it actually got blocked from going into glass door.

s138
00:16:10.800 --> 00:16:27.279
So this is one perfect example of you know sites like Glaster I've worked with on a on a data collection level that's like a very protected site and you know this is one great example on why the MCP is advantageous you know pairing it up with something like claw or you know GPT or any sort of brains

s139
00:16:27.279 --> 00:16:30.639
that you know you'd want to run it on.

s140
00:16:35.040 --> 00:16:35.360
Okay.

s141
00:16:35.360 --> 00:16:40.480
So now I'm going to be speaking about an example use case.

s142
00:16:40.959 --> 00:16:51.680
So working with AI companies, these are examples of you know customers that I speak with that use bright data in any form of way.

s143
00:16:51.680 --> 00:17:04.640
So you know training models um fine-tune enriching their uh enriching their models or fine-tuning their models um providing AI agents real-time access like like we mentioned or enriching rags.

s144
00:17:04.640 --> 00:17:09.039
So in this example, I'm going to be speaking about specifically jobs, right?

s145
00:17:09.039 --> 00:17:20.240
So let's say someone came to me and said, "Hey, listen, I I need job data and I want to train a model with jobs data like maybe resume matching, you know, as an example."

s146
00:17:20.240 --> 00:17:32.720
So one thing we need to know is when you're thinking about jobs data, it's not like other, you know, collecting other forms of data, you know, where it's I can pull in the data and then I can just forget about it or train a model.

s147
00:17:32.720 --> 00:17:35.120
You need continuous updates about jobs.

s148
00:17:35.120 --> 00:17:47.600
You know, which companies are posting what, you know, how many jobs are available, what's the salary, you know, location, etc. So, this is why it's important that the data is actually structured and has a consistent output.

s149
00:17:47.600 --> 00:17:59.520
The actual performance is very key as well because you know if you're building a model to analyze like job patterns or you know anything in the job market you need that data flowing to you consistently

s150
00:17:59.520 --> 00:18:10.880
and you need also need this to be very reliable because if something breaks or you know if there's downtime then that can actually affect the actual in the entire pipeline that you're you want to run

s151
00:18:10.880 --> 00:18:18.080
and the timeline for this you know it's not only about getting data starting from today about jobs throughout.

s152
00:18:18.080 --> 00:18:24.640
Some people need jobs data previously, you know, like the past few months or past few years.

s153
00:18:24.640 --> 00:18:32.960
So that's why it's very important that you know, you're covering all of these because if you want to build a big picture on, you know, in this example for jobs

s154
00:18:32.960 --> 00:18:39.280
on, you know, how do I find train my model on jobs data so I can find the best matches for this position.

s155
00:18:39.280 --> 00:18:41.760
You need to understand how the trends changed.

s156
00:18:41.760 --> 00:18:53.520
you know maybe salary has increased for software engineers over time and you need to understand why what the demand is what's the current you know offering and you know you can you build your model around that so

s157
00:18:53.520 --> 00:19:04.640
this is like the example use case like an AI agent for resume matching so it's exactly what we discussed here um so immediately you need real-time access to actually

s158
00:19:04.640 --> 00:19:12.880
go through and um you know scrape the data and be able to train this model along with it.

s159
00:19:14.799 --> 00:19:15.440
Okay.

s160
00:19:15.440 --> 00:19:19.840
So, I'm going to be covering a hands-on exercise here.

s161
00:19:19.840 --> 00:19:24.400
So, I have a Google notebook here.

s162
00:19:24.400 --> 00:19:31.039
And if any of you guys are following along, there's two things that you require to actually run this script.

s163
00:19:31.039 --> 00:19:37.280
One is a bright data API token and one is a Gemini API key.

s164
00:19:37.280 --> 00:19:41.840
And I have the list of instructions in the Google Drive on how to set up both.

s165
00:19:41.840 --> 00:19:45.200
Um, and essentially I have everything set up here for you to run.

s166
00:19:45.200 --> 00:19:47.679
So I'll speak a bit about what we're doing here.

s167
00:19:47.679 --> 00:19:53.360
So first we're calling Bright Data's API endpoint to collect Amazon data.

s168
00:19:53.360 --> 00:19:56.160
We're collecting the response from Amazon.

s169
00:19:56.160 --> 00:20:06.160
Then calling the API for Reddit itself, capturing the raw Reddit data response and then performing AI analysis over that.

s170
00:20:06.160 --> 00:20:13.440
Now before I run this, I want to show you the exact data that we're getting from each of these APIs.

s171
00:20:13.440 --> 00:20:17.120
So Amazon, we're getting product level data.

s172
00:20:17.120 --> 00:20:27.280
So I picked a Tumblr cup as an example and essentially we have every single data that about the product.

s173
00:20:27.280 --> 00:20:34.960
So you know price, images, URLs, etc. And I I'll show you this once it runs as well.

s174
00:20:34.960 --> 00:20:48.799
My screen's a little small, so it might not show the entire thing, but and we also have a Reddit API where what I'm going to do in this uh script that I built is I'll actually run a keyword

s175
00:20:48.799 --> 00:20:53.600
search for that specific Tumblr cup.

s176
00:20:53.600 --> 00:21:02.240
And I'm going to be outputting posts that, you know, talk about what what what's exactly, you know, is should I buy this or not?

s177
00:21:02.240 --> 00:21:03.120
Is this worth it?

s178
00:21:03.120 --> 00:21:04.240
Is this not worth it?

s179
00:21:04.240 --> 00:21:05.919
What are people saying?

s180
00:21:05.919 --> 00:21:09.440
So, let's run this live.

s181
00:21:32.960 --> 00:21:33.280
All right.

s182
00:21:33.280 --> 00:21:33.520
Right.

s183
00:21:33.520 --> 00:21:36.640
So, we got our keys loaded here.

s184
00:21:41.200 --> 00:21:41.520
All right.

s185
00:21:41.520 --> 00:21:43.760
So, we got our Amazon data that came back.

s186
00:21:43.760 --> 00:21:59.039
You can see it has price, rating, you know, other sellers that are selling it, all of the actual data from the product itself, features, you know, you name it.

s187
00:22:00.000 --> 00:22:08.559
Now, we'll go ahead and trigger a Reddit search for the Stanley Quenter um Tumblr.

s188
00:22:08.559 --> 00:22:12.640
Just having, you know, reviews about the actual product itself.

s189
00:22:34.960 --> 00:22:41.200
And while this is running, I will show you a bit about what I spoke about the model context protocol, the MCP server.

s190
00:22:41.200 --> 00:22:44.240
So we have different layers.

s191
00:22:44.240 --> 00:22:58.799
I know I talked about things like you know browser automation and search engine but we also do cover things like you know e-commerce, social media, financial intelligence, research, business intelligence.

s192
00:22:58.799 --> 00:23:07.360
So these are can all be plugged into an MCP endpoint you know app store travels and we are still adding to it.

s193
00:23:07.360 --> 00:23:12.000
Currently we have about like 60 plus tools available to use for AI agents.

s194
00:23:12.000 --> 00:23:19.919
Um, some people prefer actually using scrapers themselves, but we do this will provide allow your AI agent to choose which tools to use.

s195
00:23:19.919 --> 00:23:50.880
So, it'll be a lot more streamlined and easier While this is running, I can take any questions if anyone has.

s196
00:24:05.520 --> 00:24:05.840
Yeah.

s197
00:24:05.840 --> 00:24:12.880
So the question was how come you guys can scrape resume level data but you know bots can't do it right.

s198
00:24:12.880 --> 00:24:24.960
So what what gives bright data the advantage and this is why we're able to do it whereas like you know things like bots aren't able to is because we have the world's largest infrastructure like proxy infrastructure

s199
00:24:24.960 --> 00:24:33.919
and where you're talking about you know a few requests maybe like you know maybe a few days worth of scraping pages like you know glass door as I use as an example

s200
00:24:33.919 --> 00:24:47.360
might work right but let's say after a while glass door catches on like hey you know he's using a data center IP he's accessing the page you nothing's actually being done on the page and then they start adding all these layers of anti-bot

s201
00:24:47.360 --> 00:24:48.799
into the picture.

s202
00:24:48.799 --> 00:25:01.679
That's when companies like us bright data would come into the play because you need to actually not only own like a scraper or a bot, but you need to think about how do I bypass you know what the site is protected with the capture

s203
00:25:01.679 --> 00:25:06.240
and how do I do it consistently without triggering the block.

s204
00:25:06.240 --> 00:25:15.919
So, we've been doing it for years and we have tools that bake into this anti-bot logic where we're changing our approach to a site every time at scale, but

s205
00:25:15.919 --> 00:25:21.039
sometimes um comp like people that are starting off scraping might be doing it in-house and managing it.

s206
00:25:21.039 --> 00:25:32.960
You have to also think that's also engineering time and resource time because trying to figure out what change did that site make also takes time, even probably more time than managing the scraper.

s207
00:25:35.840 --> 00:25:36.400
Okay, cool.

s208
00:25:36.400 --> 00:25:39.039
Looks like it came back with data here.

s209
00:25:39.039 --> 00:25:40.720
So, I can show you.

s210
00:25:40.720 --> 00:25:46.159
So, it h it c captured a few Reddit posts about the actual Tumblr itself.

s211
00:25:46.159 --> 00:25:54.400
So, some comments about, you know, what people were saying about the actual Tumblr.

s212
00:25:54.400 --> 00:26:02.559
It looks like people were saying it was embarrassing and people were obsessed, but and then it gives the actual response here.

s213
00:26:02.559 --> 00:26:06.559
So you could see actually buy it.

s214
00:26:06.559 --> 00:26:08.240
It's on a decent discount.

s215
00:26:08.240 --> 00:26:13.120
You know, there's overwhelmingly positive reviews, but a few minor mixed opinions.

s216
00:26:13.120 --> 00:26:18.480
There's high demand, but low stock on Amazon because it returned that from the data.

s217
00:26:18.480 --> 00:26:23.760
And you know, a few other analysis points about the actual Tumblr and the amount of reviews it has.

s218
00:26:23.760 --> 00:26:30.320
It actually dissected all the data points that we returned from our API and gave you a decision on yes, should I buy it?

s219
00:26:30.320 --> 00:26:31.279
Should I not buy it?

s220
00:26:31.279 --> 00:26:42.400
Now this is just one example of you know using data from starting from Amazon running it on Reddit and then making a decision but you can think of it it think of this on like a bigger picture you know you can

s221
00:26:42.400 --> 00:26:50.640
it doesn't have to be e-commerce it can be any sort of use case you know as long as we have like we have the tools that can allow you to go to the page and access the data.

s222
00:26:50.640 --> 00:27:01.919
Now it's up to you what logic and how you want to build that pipeline and build it at scale as well because what one thing we do best is actually provide the data at scale.

s223
00:27:02.640 --> 00:27:02.960
Cool.

s224
00:27:02.960 --> 00:27:10.400
I think we're uh able to wrap up and if you guys um want to c if you have questions after I'll be available or you could stop by at the Bright Data booth.

s225
00:27:10.400 --> 00:27:13.200
But thank you for your time guys.

s226
00:27:26.688 --> 00:27:28.688
[music]