Guardians of the State: An Air-Gapped AI Fortress for Consumer Data — Rachna Srivastava, DFPI

AI Engineer · 21 min · 166 sentences · from YouTube's caption track

Each timecode opens YouTube at the start of that sentence. Line anchors (#s42) are the cue ids in the WebVTT, and every line carries its start and end seconds. All transcripts has every talk, and the whole corpus as one file.

00:01[music]
00:12By the time this presentation ends, someone, somewhere will make life-altering decisions based on something generated entirely by AI.
00:30Last 30 years, digital infrastructure is based on these unwritten rules.
00:39If you see a face or you hear a voice, you trust someone behind it.
00:47If you see a signature, you trust someone has approved it.
00:53Every business transaction, every government workflow is based on this foundation of trust.
01:03So, trust is a invisible layer on top of it all this digital infrastructure based on.
01:12But, generative AI trashed it completely.
01:17AI agent can clone a voice.
01:20Synthetic face can bypass identity check.
01:25AI can impersonate people at a speed no criminal organization have been able to do that before.
01:36So, in the era of generative AI, seeing is no longer believing, neither is hearing.
01:44The cost of deception is collapsed and the speed of deception is exploded.
01:54Hi, my My is Rachna Srivastava.
01:56I work for California Department of Financial Protection and Innovation.
02:02Our mission is to protect 39 million Californian and their financial identities.
02:11Let me tell you what we do.
02:13When a massive fraud attack in in the state of California, we identify the fraud, we examine the fraud, and we run enforcement.
02:26Let me take you behind the scene what we actually do.
02:29Fraud comes, we investigate the fraud.
02:33When the investigation happen, we collect all the evidences for the fraud.
02:38When we have all the evidence, we take that evidence to the court.
02:43And in the court, the defense attorney has only one job, to attack the system that we have built.
02:54So, the question arises, how should we build a system which is credible?
03:03What are the attributes we need to have in a system which can appear in the court?
03:11And appear in the court means system should be defendable.
03:16Defendable means it should be we should be able to explain the system at every step of the process.
03:23We should be able to reproduce the issue at every step of the process.
03:27We should be able to audit the issue at every step of the process because everything, every data that we produce, it's going to appear in the court.
03:40So, how do you build that kind of system?
03:43You Since we deal with financial data, we have to ensure that the data is secure for sure.
03:51We have to ensure that the data is evolving and learning.
03:57Why?
03:57Because the fraud space is changing every day.
04:01We have to ensure that the data is available to be appear in the court and reproducible.
04:10So, what I'm going to show you now, the system that we built, and the lessons we have learned the hard way.
04:21So, the first and foremost decision that we have taken was to build our solution offline.
04:31So, you might be wondering like it's it's extreme.
04:35Like, why are you Okay, why do you want to create a solution offline when we have a secure cloud?
04:44So, before we show you what we build, let's talk about what do industry do in the normal way.
04:54So, the gold standard of security is encryption.
04:59Encryption means you encrypt the data at rest or encrypt the data on the transit.
05:06That means the thing is but for machine learning model to work, the data has to be decrypted and available in the plain text in the model memory.
05:21And if the data is already decrypted and present in the plain text, it is prone to prompt injection attack.
05:30Secondly, let's talk about private endpoint.
05:33Private endpoints are actually a isolated environment given to you from your cloud provider.
05:43But cloud providers actually own the on the on the disk.
05:51So, the question the thing is that according to the Cloud Act federal government can access your data which is in the cloud and they don't even have to tell you that your data have been accessed.
06:11Let's talk about certification.
06:14FedRAMP compliant, I also certified SOC 2 compliant.
06:20All these compliance are just paper.
06:24And we have seen these highly compliant organization fail over and over again.
06:35So, only way you can make your data secure to be available in the court to present as a as a evidence is to build the solution offline.
06:53So, like everyone else we thought like what a big deal, let's just download a open-source model, create an isolated environment, spin up some GPU, add some system prompt, and add some guardrails,
07:07and done we are running.
07:09Push the live data into it.
07:12We did the same like everyone else.
07:15And the model the system collapsed in 2 hours.
07:22And our first instinct was you know, we took the model free model from online.
07:29Model is not good for our use case.
07:35But the issue was we were we were treating machine learning model as a magic box instead of as a data pipeline.
07:49So, anything that garbage comes into the system, we were expecting the model to clean up and do all the processing.
07:58So, to solve this problem, we introduce Kafka for data ingestion, a Spark for data processing, and large language model for reasoning.
08:10Three different tools to solve three different problems.
08:18We use Kafka for these three things.
08:22First, the data that enter into our system does not come at a consistent speed.
08:30When a fraud attack happen, it attack all over the state.
08:35So, we get a high spike of traffic into our system.
08:40So, we needed a tool that can buffer that traffic so that the downstream component can access the data at a consistent speed.
08:51And Kafka did that.
08:53Secondly, we wanted all the events that occur in the system to come and store in a sequential order.
09:05When did you open the account?
09:06When did the transaction happen?
09:08The order of events in the fraud is extremely important.
09:13Kafka helped us solve that problem.
09:16But the main problem that Kafka helped us solve is reproducibility.
09:24In Kafka, it let you move the checkpoint to the point at which the decision was made, and it helped you replay the event.
09:34And this replayability of the event is the proof that we present in the courtroom.
09:44So, now Kafka did solve our data storage problem.
09:49But, the data that enter into the system is really messy.
09:54We get 10 different format of bank statement.
09:57We get audio file.
09:59We get screenshot, facts statement.
10:05And, if you dump all this data to model memory, then no wonder model hallucinate.
10:13So, we introduced a Spark because a Spark let us process this data behind the scene in the clusters of CPU instead of working on GPU.
10:27And, Spark let it let us clean the data.
10:32And, then when we process send the clean data through the model memory, to our utter surprise, the same model started giving us so good results.
10:45It found the connection between the points which we never thought expected.
10:52So, we learn our first lesson.
10:55The lesson is most of the data problem in AI are data engineering problem wearing AI mask.
11:06So, I recommend you to consider using data engineering tool to solve those problems.
11:14Now, our model memory is filled with a very condensed clean data.
11:22But, then we found our next issue.
11:24The issue was the memory was filled with lot of sensitive information.
11:30Your credit card number, your bank account number, your social security number.
11:37So, we introduce something called cryptographic vault.
11:42Which is basically a SHA-256 cryptographic algorithm with hardware security module.
11:51So, in the simple sense, it means that as we get the data into the system, cryptographic vault which convert the data into cryptographic hash.
12:03But, the key that is used to do the conversion is physically attached to our server rack.
12:13So, if tomorrow somebody is able to get our data, they have to walk into our office physically, break the server rack, get the key to actually make sense of the data.
12:28Then, we learn our second lesson.
12:30If the stakes are really high, trust hardware over software.
12:41Now, we have a model which is working really well.
12:45All the PII is redacted.
12:48We thought, let's just run a quick round of load testing and release this product.
12:54The problem is in the cloud, the scaling is unlimited.
13:01You get a high spike of traffic, it is spins off more server, it's it's distribute the load to those server, and you are done.
13:10But, when you are creating an isolated environment, you have limited GPU, limited compute, limited VRAM.
13:19So, we have to step back and figure out what is the actual problem here.
13:25The problem was we were using one state of the art machine learning model to do every single processing.
13:35The same model was doing the summarization, entity extraction, fraud ring detection.
13:46So, we were actually making a neurosurgeon take the blood pressure of every single patient.
13:55To solve this, we introduced triage nurse or semantic router.
14:02And the goal of semantic router is as we get the data, it analyzes the data and forward the request to the smallest possible model which is capable of processing
14:16that request.
14:19And by making this simple architecture change, we found that more than 80% of of our task could be easily done by the smallest, fastest, cheapest model.
14:36By not adding any new GPU, we could process three times more traffic and cost of processing each request reduced to nearly 70%.
14:55Now, we have done the unit testing, we have done the load testing, and then we found the hardest problem of all.
15:08The problem was we built this highly secure system, but the system was not learning.
15:19System did not know what is happening in the threat space.
15:24So, the question arises, how do we make our system learn without creating a security hole?
15:36Usually in industry, people solve this problem by configuring software firewall.
15:44But any configuration can be misconfigured.
15:48And once you have misconfigured, your highly secure system will be highly exploited one.
15:57We decided to not trust the configuration, and we took help from physics.
16:04We introduce one-way data diode, which is basically a fiber optics cable physically cut into half.
16:16The first half of the cable is connected to the internet to receive the data.
16:22Second half of the cable is connected to our solution.
16:26First of the half of the cable has laser transmitter, which receives the data from the internet and transmit the data to the second half.
16:36[snorts]
16:36Second half has a laser receiver to receive the data from the first half.
16:41But there is no laser transmitter from our end to the outside world.
16:48So, it is physically im- possible for data to leak from the system.
16:55And this is how we ensure 100% guarantee of the security of the system.
17:09Data diode solves the problem of directionality of the data.
17:15But anything enter into the data is is considered to be unsafe till proven.
17:23So anything from outside first first lands into quarantine zone.
17:30Where we run a spark job that runs a validation on each input data.
17:36And when all the validation is successful, then the data goes to the production layer.
17:45In the production layer, we also save the data into Apache Iceberg.
17:52Apache Iceberg is a time traveled queryable immutable data store.
18:02We enter the data in this data store because 2 years from now one of our results from our system goes to the court.
18:15We cannot present We cannot go to the court and tell them, "You know what?
18:18This system is result is produced by AI and we don't know anything about it."
18:26At that moment, we travel the Apache Iceberg, go to the point at which the decision was made and get the state of the system at that moment from the database.
18:41And that is state of the system appear as a proof in the court.
18:49And that is how we build a solution which is secure, which is evolving, and which is defensible in the court.
19:07This is the whole architecture end to end.
19:11If you see this only at the very end of the system, we are when a user logs into the system, they we authorize the user into multi-factor factor authorize
19:25authentication and only then the data is reverted.
19:30Data is not opened till the very end at the browser where the user is actually evaluating the threat case.
19:45So, don't look at the solution as a fraud detection solution.
19:49This is the architecture of the future.
19:53Soon the same architecture will be used to predict the health care, banking e-statements, legal and other domains.
20:04In the end, I just want to say one thing.
20:08Build solution that can be trusted.
20:13And remember, trust is not a policy.
20:17Trust is a physical property of the system.
20:22You have to build the trust from day one into your hardware, into your physics, into your architecture or it's not there.
20:33It's that simple.
20:35So, years from now nobody will remember the models you trained.
20:38Nobody will remember the benchmark you received.
20:42People will only remember the solution that you have built can be trusted when it needed the most.
20:53Thank you so much.
21:10[music]
21:12Mhm.