WEBVTT

NOTE Sentence-level transcript of https://www.youtube.com/watch?v=T72nqdC92PM

NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/T72nqdC92PM.html. A cue ends where the next begins, or 2 s after its last word.

s1
00:00:16.640 --> 00:00:17.199
All right.

s2
00:00:17.199 --> 00:00:18.320
Um, hi everyone.

s3
00:00:18.320 --> 00:00:18.960
Good afternoon.

s4
00:00:18.960 --> 00:00:19.920
I'm Krishna.

s5
00:00:19.920 --> 00:00:23.600
Uh, and I'm a general manager at Servin.

s6
00:00:23.600 --> 00:00:36.719
Um today I'm going to tell you how a three billion parameter model uh small enough to run on a single GPU gets state-of-the-art uh in document AI and beats

s7
00:00:36.719 --> 00:00:41.040
models 100 times larger in size.

s8
00:00:41.040 --> 00:00:44.960
It is also a slightly unusual model in two ways.

s9
00:00:44.960 --> 00:00:49.039
Firstly u the language model used is not a standard transformer.

s10
00:00:49.039 --> 00:01:05.040
Um and the entire model building from data to training to compute uh was done end to end in India and for English and for 22 official Indian languages uh which I think uh are personally

s11
00:01:05.040 --> 00:01:09.439
uh uh one of the hardest document intelligence problems uh in the world right now.

s12
00:01:09.439 --> 00:01:14.400
Um this is how we went from scratch to SOTA.

s13
00:01:14.400 --> 00:01:15.119
Who are we?

s14
00:01:15.119 --> 00:01:20.799
Uh we are serv uh a sovereign foundation model company uh in India.

s15
00:01:20.799 --> 00:01:24.640
We work across different modalities voice, text and vision.

s16
00:01:24.640 --> 00:01:29.280
Uh in voice we have speech to text and texttospech models.

s17
00:01:29.280 --> 00:01:41.360
In text we have uh a 30 billion and 100 billion parameter uh uh models and in vision we have a document intelligence model which is what we are going to talk about today.

s18
00:01:41.439 --> 00:01:46.560
Um, India is largely missing from the machine readable world.

s19
00:01:46.560 --> 00:02:01.600
Um, according to a published language distribution study, well under 1% of the common crawl uh corpus on which frontier models are trained uh have uh um Indian language representation.

s20
00:02:01.600 --> 00:02:14.080
Now in several forums you may have heard Frontier Labs say that um India is their um India is one of their larger uh uh and fastly growing markets and to that end

s21
00:02:14.080 --> 00:02:22.400
um India is still missing uh in terms of data from frontier models that uh um are being trained every day here.

s22
00:02:22.400 --> 00:02:23.680
Now why is that?

s23
00:02:23.680 --> 00:02:29.680
Uh primarily not because there is uh a lack of data or the knowledge does not exist.

s24
00:02:29.680 --> 00:02:37.200
It is because the data has never been digitized and we are solving that problem at server.

s25
00:02:37.680 --> 00:02:41.920
Um why is Indic document intelligence hard?

s26
00:02:41.920 --> 00:02:47.920
Uh firstly the goal is that of knowledge extraction not just purely text extraction.

s27
00:02:47.920 --> 00:02:52.720
Uh extracting pure text without logical coherence is meaningless.

s28
00:02:52.720 --> 00:03:03.280
And secondly in index scripts the shape of uh the words and the language you see are very different from what a machine sees.

s29
00:03:03.280 --> 00:03:14.640
that is Indian languages have a complex uh set of uni codes fused together to get right the language model needs to be strong at all 22 languages.

s30
00:03:14.640 --> 00:03:26.080
Third, majority of Indian languages can be considered low resource due to which there is a lot of uh um lack in data today to train these models.

s31
00:03:26.319 --> 00:03:33.120
Our answer to this is server vision India's first sovereign vision language model built from scratch.

s32
00:03:33.120 --> 00:03:42.720
um a three billion parameter model with state space architecture data compute and training all of which done in India.

s33
00:03:42.720 --> 00:03:51.120
When we started the initial work in late 2025 most VLMs uh in the OCR space were monolith VLMs.

s34
00:03:51.120 --> 00:04:03.040
uh they performed page level OCR and we at the time took a contrarian bet to focus on block level OCR instead adding a document hardness around the model.

s35
00:04:03.040 --> 00:04:18.400
Um a lot of models since then uh released uh uh recently in 2026 have converged onto the same uh hardness plus small model paradigm for OCR which shows a lot of value in the direction we uh set off.

s36
00:04:18.400 --> 00:04:30.720
Serverom vision in particular uh has two hardness modules one for layout and one for reading hardness and a state space model uh VLM for block level OCR.

s37
00:04:31.120 --> 00:04:37.120
Again um today uh why do we use state space and not a transformer?

s38
00:04:37.120 --> 00:04:46.240
Um most OCR models today uh like general VLMs, open source VLMs like Quen, Gemma etc. are all transformers based.

s39
00:04:46.240 --> 00:04:50.000
We took an alternate approach uh using an SSM.

s40
00:04:50.000 --> 00:04:51.280
Why?

s41
00:04:51.280 --> 00:04:58.479
Both are fundamentally sequenced models transformers and SSMs but with very different underlying mechanics.

s42
00:04:58.479 --> 00:05:08.160
A transformer has every token look at every other token where the compute grows with square of the sequence length.

s43
00:05:08.160 --> 00:05:14.479
Uh that is the L cross L interactions and the memory goes as the sequence grows as well.

s44
00:05:14.479 --> 00:05:18.240
Now on the other hand, SSMs have a single state.

s45
00:05:18.240 --> 00:05:23.360
They maintain a single state through the sequence updating it token by token.

s46
00:05:23.360 --> 00:05:29.440
Now as compute grows only linearly, the memory stays constant for SSMS.

s47
00:05:29.440 --> 00:05:34.080
Now why is this the right uh architecture for OCR?

s48
00:05:34.080 --> 00:05:44.960
It really depends on the uh uh trade-offs here uh for long documents in particular which can lead up to uh uh 5 to 10,000 uh visual tokens per page

s49
00:05:44.960 --> 00:05:52.400
and quadratic complexity of you know uh compute and memory becomes very expensive for inference.

s50
00:05:52.400 --> 00:06:08.560
On the other hand, um doing block level OCR with somewhat of a lossy recall using SSMs can be justifi justified to uh um avoid high compute costs imposed by transformers.

s51
00:06:08.960 --> 00:06:11.680
So how do you actually train this?

s52
00:06:11.680 --> 00:06:26.720
We built a staged curriculum four stages each uh building on the last um stage one is texton pre-training uh 13 trillion tokens across English, Indian language, text, math and code.

s53
00:06:26.720 --> 00:06:39.520
This builds the three billion parameters language backbone and a strong language prior is what lets the model resolve a smudged or an ambiguous text from the image.

s54
00:06:39.520 --> 00:06:47.120
the same way you can read a half blurred word because you know which word is supposed to uh u be at the right place.

s55
00:06:47.120 --> 00:06:55.520
So we focused on building the competence of the language model first uh before the model ever even sees a pixel.

s56
00:06:55.520 --> 00:07:02.400
Stage two is where we performed continual pre-training uh on 300 million image text pairs.

s57
00:07:02.400 --> 00:07:10.000
This teaches the language model general vision capabilities and teaches how to see how to interpret pixels and so on.

s58
00:07:10.000 --> 00:07:18.319
Then comes stage three where we performed uh supervised fine-tuning on 100 uh uh million uh OCR uh samples.

s59
00:07:18.319 --> 00:07:27.919
Now the um stage three is primarily focused on letting a general VM model become strong at OCR.

s60
00:07:27.919 --> 00:07:45.440
What that includes is having diverse data across all 22 languages and English and also incorporating all kinds of uh uh document components such as tables and uh um equations and handwritten documents and so forth.

s61
00:07:45.440 --> 00:07:55.199
Now and stage four is re reinforcement learning which helps push past the ceiling of whatever uh supervised fine-tuning is able to achieve.

s62
00:07:55.199 --> 00:07:58.560
Here you will see this is a standard recipe.

s63
00:07:58.560 --> 00:08:06.400
However, the moat here is the two things underneath the data layer and the eval layer.

s64
00:08:06.400 --> 00:08:09.039
Um the first is the data engine.

s65
00:08:09.039 --> 00:08:13.919
For most of the 22 languages, there is no label data available readily.

s66
00:08:13.919 --> 00:08:19.520
When label data does not exist, building the data engine becomes hard.

s67
00:08:19.520 --> 00:08:22.240
And that is something we have done extensively.

s68
00:08:22.240 --> 00:08:39.039
We've built pipelines to create synthetic data and doc data from real documents and also helped build the uh pipeline for continuously improving the data that goes into training based on the eval performance and so on.

s69
00:08:39.039 --> 00:08:49.839
So we are currently actively looking at the paradigm of RLMs um as uh uh we explore the agentic vision capabilities for our models in the upcoming releases.

s70
00:08:49.839 --> 00:08:52.240
The second mode is the eval.

s71
00:08:52.240 --> 00:08:59.200
You cannot reach state-of-the-art if uh um you cannot measure how well your model performs.

s72
00:08:59.200 --> 00:09:10.480
We have curated extensive evals to ensure what we are measuring is truly sota and also truly meaningful in terms of usefulness for end uh uh users.

s73
00:09:10.480 --> 00:09:20.240
So I want to take just a minute to dwell on our RL pipeline uh stage four because uh a lot of gain comes from RL.

s74
00:09:20.240 --> 00:09:33.360
Essentially in OCR correctness is a machine readable problem right a lot of tests can be set up to uh um reward and score the models against uh uh these

s75
00:09:33.360 --> 00:09:35.519
uh samples that you create.

s76
00:09:35.519 --> 00:09:41.279
And in the world of deterministic OCR, these are all machine uh uh verifiable.

s77
00:09:41.279 --> 00:09:45.440
Hence, RL gives us a major boost.

s78
00:09:45.440 --> 00:09:54.160
Sample a group, score with unit tests, reinforce what the average uh uh value to beat is, and then repeat the process.

s79
00:09:54.160 --> 00:09:58.880
And this makes RLVR for OCR very very scalable.

s80
00:09:58.880 --> 00:10:10.880
Now after all the from scratch training we've done and the data effort we've made we were able to establish sot in two of uh the global English benchmarks one is

s81
00:10:10.880 --> 00:10:19.839
CRB bench and the other is omnidoc bench at launch we were at 84.3 on CR and then 93.2 into on omnidok bench.

s82
00:10:19.839 --> 00:10:31.279
Um the models uh that have released since have moved the needle quite a bit and we'll soon have a a stronger model in the global leaderboard as well.

s83
00:10:31.279 --> 00:10:44.640
Secondly and more importantly um on 22 Indian languages we have an unbeaten lead even compared to all the frontier models such as Gemini and uh chat GPT and Opus and so on.

s84
00:10:44.640 --> 00:10:53.680
And this is where we've extended the lead significantly and we remain uh uh strong compared to all of these newer models that have come in.

s85
00:10:53.680 --> 00:11:10.959
Now server vision pars our agentic document intelligence workbench called aure where we enable human in the loop agentic digitizations uh and uh extractions and then injections for various uh downstream

s86
00:11:10.959 --> 00:11:12.880
uh document intelligence problems.

s87
00:11:12.880 --> 00:11:28.959
We provide confidence scores um and we uh have block level grounding and then also the ability to do uh agentic proof reading etc. Now benchmarks and soda are uh uh one thing they have their place.

s88
00:11:28.959 --> 00:11:43.120
Today some of the largest enterprises in the world from insurance to banking to governments to historical preservation organizations are using server vision to digitize more than 35 million pages across English

s89
00:11:43.120 --> 00:11:46.240
and uh uh 22 Indian languages.

s90
00:11:46.240 --> 00:11:54.320
The model is available as API and on prem and then the agentic platform.

s91
00:11:54.480 --> 00:12:01.440
So in conclusion until 4 months ago uh there was no sovereign model uh from India.

s92
00:12:01.440 --> 00:12:15.200
Today we have server vision which was trained from scratch and has established sot um at a price point that is extremely competitive with uh all the other solutions including open and close source.

s93
00:12:15.200 --> 00:12:22.320
Uh first we've started by uh uh solving some of the hardest problems for Indian language document intelligence.

s94
00:12:22.320 --> 00:12:34.320
Soon uh we'll be releasing general purpose uh VLMs which are capable of much more uh uh vision capabilities and uh look forward to uh uh all of you trying out our models.

s95
00:12:34.320 --> 00:12:36.533
Thank you.

s96
00:12:36.533 --> 00:12:38.533
[applause]

s97
00:12:39.600 --> 00:12:40.959
Happy to answer any questions.

s98
00:12:40.959 --> 00:12:42.959
Yeah.

s99
00:12:48.320 --> 00:12:49.600
Yes.

s100
00:12:49.600 --> 00:12:49.920
Yes.

s101
00:12:49.920 --> 00:12:50.399
Absolutely.

s102
00:12:50.399 --> 00:12:57.680
Uh in general, the language capability of the 22 languages enhances English uh by quite a margin.

s103
00:12:57.680 --> 00:13:04.639
And this is applicable to any low resource language, not just Indian low resource languages.

s104
00:13:05.600 --> 00:13:07.600
Yeah.

s105
00:13:27.760 --> 00:13:28.240
Right.

s106
00:13:28.240 --> 00:13:38.800
Um we don't quite go in that direction with this model because this is a vision focused uh uh model wherein we are focused on extracting information or knowledge from documents.

s107
00:13:38.800 --> 00:13:47.360
But yes, there could be uh some parallels to um helping models use general languages to speed up coding as well.

s108
00:13:47.360 --> 00:13:53.120
But yeah, that's not the uh periphery of this uh uh work.

s109
00:13:53.440 --> 00:13:55.440
Yeah.

s110
00:14:10.160 --> 00:14:10.639
Right.

s111
00:14:10.639 --> 00:14:22.240
Um so there are two things uh we uh create artificial documents synthetic documents as they are called for general post-raining uh that includes SFT and RL.

s112
00:14:22.240 --> 00:14:29.920
Um however coming to your specific question on RL uh we don't uh you can generate uh synthetic documents there as well.

s113
00:14:29.920 --> 00:14:40.880
However, um the best thing to do is to take real world documents complex enough that uh help you set up unit tests or different kinds of uh uh rewards, right?

s114
00:14:40.880 --> 00:14:51.519
like say a a reward based on character error rate or a a reward based on table structure or a math equation or something uh that is pertinent to a language

s115
00:14:51.519 --> 00:15:02.480
or like grammar rewards or so on and then help the model iteratively improve based on uh um rules that the model is able to produce at different uh u settings.

s116
00:15:02.480 --> 00:15:03.519
Yeah.

s117
00:15:03.519 --> 00:15:05.519
Yes.

s118
00:15:21.440 --> 00:15:24.240
Sorry, did you say big fan of Chundra?

s119
00:15:24.240 --> 00:15:24.560
All right.

s120
00:15:24.560 --> 00:15:26.320
Uh, that's from a different lab.

s121
00:15:26.320 --> 00:15:29.839
Uh, I'm also a big fan of the lab that produced Chundra.

s122
00:15:29.839 --> 00:15:32.880
But yes, took your question uh on Indic benchmark.

s123
00:15:32.880 --> 00:15:50.079
Yes, we will release the server indic benchmark that we have created for 22 languages and it uh spans um a a huge time period starting from 1800s until modern day and also different kinds of layouts, different kinds of documents in Indian languages.

s124
00:15:50.079 --> 00:15:56.000
You can imagine documents with pros, poetry, literature, uh tables, finance and all of those things.

s125
00:15:56.000 --> 00:16:00.880
We'll soon release that benchmark uh in public as well.

s126
00:16:25.440 --> 00:16:41.519
So translation is again uh transliteration is again not directly involved here because in OCR what we care about is high fidelity extraction right uh you want even if there is a mistake in an image you want that mistake to be extracted correctly and not the model

s127
00:16:41.519 --> 00:16:53.920
uh uh making changes at whim uh hence that is not directly applicable but yes uh what we see is a lot of data that is coming through transliteration for even OCR training now and the

s128
00:16:53.920 --> 00:17:02.800
um jury is out on what is the quality of that data and how useful it is etc Yeah.

s129
00:17:27.760 --> 00:17:28.319
Right.

s130
00:17:28.319 --> 00:17:30.480
Um there are few things to consider.

s131
00:17:30.480 --> 00:17:44.799
Um first is no other model frontier uh closed source or open model does well with complex Indian language documents right so for a size of uh for a country

s132
00:17:44.799 --> 00:18:01.679
with 1.4 4 billion people you want to be able to address their day-to-day lives in uh uh terms of document intelligence problems because there is a lot of paperwork in India right it is a up and cominging country and it's still being digitized as we speak and it's

s133
00:18:01.679 --> 00:18:06.960
important that representation or ability to digitize the country exists to begin with.

s134
00:18:06.960 --> 00:18:19.280
Secondly on uh the training side what we have uh uh done in particular is to create data that can be added to a lot of training downstream.

s135
00:18:19.280 --> 00:18:32.080
So uh we are at a very early stage in terms of AI becoming you know a regular parts of our lives and we need the data to start to be able to eventually get to a place where we can

s136
00:18:32.080 --> 00:18:48.559
uh have personalized agents in the language you prefer in language I prefer and however I want in uh uh in some so to do all of that we need to start somewhere and the data needs to be created and if the data is of good quality then that helps the model become sota

s137
00:18:48.559 --> 00:19:06.080
and If a model is sot then uh from uh insurance agencies to government organizations to others who care about sovereignity in AI right uh as a government body I cannot have a model deployed elsewhere and I don't know where the data is going for

s138
00:19:06.080 --> 00:19:18.559
uh uh the transcriptions so I need to be able to control where the data is sent and when it is used how much it is used etc. So sovereignity becomes very crucial and hence this model really is now

s139
00:19:18.559 --> 00:19:24.799
uh uh like it's been what four months since we launched and already we are we are digitizing 35 million pages.

s140
00:19:24.799 --> 00:19:33.600
So that is that shows that the market was waiting for something sovereign in this space that can really kickstart uh the AI digitization wave in India.

s141
00:19:33.600 --> 00:19:45.200
So yeah primarily like three aspects one on uh sovereignity the other is the model uh capability itself and the third is data uh that is required to train these models.

s142
00:19:54.320 --> 00:20:04.960
Yeah, it is it is uh um 40% uh indic and then uh the rest is English and comprises of math and code etc.

s143
00:20:08.080 --> 00:20:09.679
All right.

s144
00:20:09.679 --> 00:20:12.480
Just a curious question.

s145
00:20:24.960 --> 00:20:37.600
Um currently the uh the deployments we've made uh with this model are in different states uh uh trying to digitize regional languages and along with all the English and

s146
00:20:37.600 --> 00:20:49.039
mixed language uh uh documents and hence we'll soon uh be able to train with the um uh feedback we get from uh the current deployment.

s147
00:20:49.039 --> 00:20:53.440
So yeah, we have started that uh pipeline as well.

s148
00:21:07.439 --> 00:21:09.439
[music]